Appropriate consideration of contextual factors is essential for ensuring internal and external validity of randomized and non-randomized evaluations. Contextual factors may confound the association between delivery of the intervention and its potential health impact. They may also modify the effect of the intervention or programme, thus affecting the generalizability of results. This is particularly true for large-scale health programmes, for which impact may vary substantially from one context to another. Understanding the nature and role of contextual factors may improve the validity of study results, as well as help predict programme impact across sites.

This paper describes the experience acquired in measuring and accounting for contextual factors in the Multi-Country Evaluation of the IMCI (Integrated Management of Childhood Illness) strategy in five countries: Bangladesh, Brazil, Peru, Uganda and Tanzania. Two main types of contextual factors were identified. Implementation-related factors include the characteristics of the health systems where IMCI was implemented, such as utilization rates, basic skills of health workers, and availability of drugs, supervision and referral. Impact-related factors include baseline levels and patterns of child mortality and nutritional status, which affect the scope for programme impact.

We describe the strategies used in the IMCI evaluation in order to obtain data on relevant contextual factors and to incorporate them in the analyses. Two case studies – from Tanzania and Peru – show how appropriate consideration of contextual factors may help explain apparently conflicting evaluation results.

Author notes

1Programa de Pós-Graduação em Epidemiologia, Universidade Federal de Pelotas, Brazil, 2Gates Malaria Partnership, London School of Hygiene and Tropical Medicine, London, UK, 3Ifakara Health Research and Development Centre, Ifakara, Tanzania, 4Instituto de Salud del Niño and San Marcos University, Lima, Peru, 5Universidade Federal do Ceará, Fortaleza, Brazil, 6International Centre for Diarrhoeal Disease Research, Bangladesh, Dhaka, Bangladesh, 7School of Public Health, Makerere University, Kampala, Uganda, 8Department of Child and Adolescent Health and Development, Division of Family and Child Health, World Health Organization, Geneva, Switzerland, 9WHO Consultant, 2081 Danby Road, Ithaca, NY, USA and 10Division of Nutritional Sciences Cornell University, Ithaca, NY, USA