Design effects and intraclass correlation coefficients from a health facility cluster survey in Benin

Extract

To the Editor: For at least two decades, disease control programs in developing countries have used clinical guidelines for health workers to improve the management of ill children and prevent childhood deaths. To monitor health worker performance and evaluate the implementation of such guidelines, a common method has been to conduct health facility cluster surveys in which health workers are observed during consultations [1,2]; a cluster is usually defined as all, or a sample of, consultations occurring at a given health facility on a given day. Data from these surveys tend to be correlated because of the similarity in case management quality for children seen at the same health facility (often all children are seen by only one or two health workers).

When planning such a survey, the sample size of consultations must account for potential correlation in the data. A common approach is to estimate an ‘effective’ sample size, n, with a formula that does not account for correlation, and then to calculate a final sample size by multiplying n by an estimate of the design effect. The design effect is ‘the ratio of the actual variance of a sample to the variance of a simple random sample of the same number of elements’ ([3], p. 258); it also equals ρ(m − 1) + 1, where ρ is the intraclass correlation coefficient and m is the average number of consultations per cluster ([3], p. 162). The intraclass correlation coefficient is a measure of the homogeneity of elements within clusters and has a maximum value of +1 when there is complete homogeneity within clusters, and a minimum value of −1/(m − 1) when there is extreme heterogeneity within clusters ([3], p. 171). For survey planners, however, a practical problem with this approach is deciding what number to use for the design effect: firstly, health facility surveys can measure dozens of quality-of-care variables and each variable may have a different design effect; and secondly, to the best of our knowledge, there are no published design effects, or estimates of ρ from which design effects could be calculated, from surveys in developing countries to provide guidance (results have been published from surveys in industrialized countries [4]).