Our study focuses on examining physical activity, as one of the most influential health determinants by domains and dimensions among Roma, the largest vulnerable ethnic minority in Europe. The study was carried out on a sample representative of the Hungarian Roma (HR) population (n = 350) living in segregated colonies in Northeast Hungary in comparison with the Hungarian general (HG) population sample (n = 343) from the same region. Data were collected using the International Physical Activity Questionnaire (IPAQ) long-form and physical activity was quantified as MET-min/week. Scores were calculated for walking, moderate and vigorous-intensity activities for each domain (work, transport, domestic and gardening, leisure) and as an overall total. The HR population—similarly to the HG—is characterized by moderate or high physical activity; however, this level is achieved by work and housework/gardening instead of leisure time activities, which is worryingly low among HR females, but its prevalence is significantly (p < 0.001) lower in both sexes than among the HG population in the vigorous activity category. HR men move (walk and cycle) significantly more during transport than HG men. Our results may direct the attention of decision-makers to improve the health of Roma by increasing leisure-time physical activity.