We present a direct test of the long-standing hypothesis that food competition limits primate group size. Group size is a critical social variable because it constrains most other aspects of social organization. We develop a simple population-specific index of indirect feeding competition based on daily foraging costs. This index explains nearly two-thirds of between-population variation in mean group sizes of mostly fruit-eating (but not of mostly leaf-eating) primates. Group size is also significantly related to body size and terrestriality (or use of open country), which are suspected correlates of predation risk, although feeding competition remains an important predictor of group size even when these correlates are controlled. Phylogeny also appears to be important: the differences between observed mean population group sizes and those predicted using ecological factors are most positive for the Old World monkeys and most negative for the lemuroids in our sample. The weak relationship between group size and feeding competition found for folivorous species may be explained either by the energetic constraints of a leafy diet or by limits to group size imposed by infanticide as a habitual male reproductive strategy.