We examined the association of total comorbid score and specific chronic conditions including cardiovascular diseases, musculoskeletal conditions, diabetes mellitus, stroke, hypertension, and cancer with several cognitive domains across four different age groups: young adults (ages 18–34), young middle-aged adults (ages 35–50), middle-aged adults (ages 51–64), and older adults (ages >64). Cognitive tests measuring global ability, executive function, memory function, and perceptual speed ability were administered to 384 African Americans. Total comorbid score was computed by summing up the number of chronic conditions. Results showed an inverse association between total comorbid scores and executive and memory functions in the total sample. With the exception of the youngest group, stroke was the only prominent predictor of poor performance for all age groups, but the impact was greater in the younger age groups compared with older adults. These results suggest that the impact of medical conditions on domain specific tasks may be modified by age.