Abstract
Objective Studies identifying risks and evaluating interventions for human immunodeficiency virus (HIV) and other sexually transmitted infections often rely on self-reported measures of sensitive behaviours. Such self-reports can be subject to social desirability bias. Concerns over the accuracy of these measures have prompted efforts to improve the level of privacy and anonymity of the interview setting. This study aims to determine whether such novel tools minimize misreporting of sensitive information.
Methods Systematic review and meta-analysis of studies in low- and middle-income countries comparing traditional face-to-face interview (FTFI) with innovative tools for reporting HIV risk behaviour. Crude odds ratios (ORs) and 95% confidence intervals (CIs) were calculated. Cochran’s chi-squared test of heterogeneity was performed to explore differences between estimates. Pooled estimates were determined by gender, region, education, setting and question time frame using a random effects model.
Results We found and included 15 data sets in the meta-analysis. Most studies compared audio computer-assisted self interview (ACASI) with FTFI. There was significant heterogeneity across studies for three outcomes of interest: ‘ever had sex’ (I2 = 93.4%, P < 0.001), non-condom use (I2 = 89.3%, P < 0.001), and number of partners (I2 = 75.3%, P < 0.001). For the fourth outcome, ‘forced sex’, there was homogenous increased reporting by non-FTFI methods (OR 1.47; 95% CI 1.11–1.94). Overall, non-FTFI methods were not consistently associated with a significant increase in the reporting of all outcomes. However, there was increased reporting associated with non-FTFI with region (Asia), setting (urban), education (>60% had secondary education) and a shorter question time frame.
Conclusion Contrary to expectation, differences between FTFI and non-interviewer-administered interview methods for the reported sensitive behaviour investigated were not uniform. However, we observed trends and variations in the level of reporting according to the outcome, study and population characteristics. FTFI may not always be inferior to innovative interview tools depending on the sensitivity of the question as well as the population assessed.
Introduction
Intervention programmes aimed at reducing HIV and sexually transmitted infection (STI) incidence often rely on self-reported measures of behaviours because of the difficulties in directly measuring the infrequent occurrences of infection. Such self-reports are subject to recall bias and social desirability bias.1 Concerns over the accuracy of these measures have prompted efforts to improve data collection through the modification of interview modes.2
Different systematic and methodological reviews of research into sexual behaviour have been published in the past decade.1–5 Most have been sceptical that the dominant mode of data collection—face-to-face interview (FTFI)—is the most effective means of gathering high-quality data, and have urged the adoption of alternative methods.6 There are a number of reasons that may lead to inconsistencies between actual and self-reported behaviour, such as poor recall and the desire to conceal socially prohibited behaviour. In general, intentional misreporting of sensitive behaviours is motivated either by a desire to underreport socially proscribed behaviour or a tendency to exaggerate socially desirable behaviour.1 Examples are culturally dependent and may depend on intervention efforts.
Several studies suggest that improving the level of privacy and anonymity that an interview method affords can greatly enhance the accuracy of self-reporting, particularly stigmatizing or illegal behaviour.1,4 As a result, researchers have developed alternatives to the FTFI such as self-administered questionnaires (SAQs). Although SAQs are thought to reduce biases associated with FTFI, variable literacy levels amongst respondents and difficulties in following complicated skip patterns mean that responses on SAQs may be less internally consistent.1 Computerization attempts to deal with these challenges, including computer-assisted personal interviewing (CAPI), which evolved into computer-assisted self-interviewing (CASI). Computer-based responses were then amalgamated with tape-recorded questioning to create audio-CASI (ACASI).
A growing body of research in the United States indicates that computer-based interviews can dramatically increase reports of sensitive behaviour, such as multiple partners or injecting drug use (IDU), compared with interviewer-administered surveys.7–12 However, relatively few studies have been conducted in low- and middle-income countries comparing ACASI with FTFI,13–20 and the results of these studies do not seem to be consistent across behaviours.21–31 More recently, handheld personal digital assistants (PDAs) have emerged as a tool for collecting risk behaviour data, due to the advantages of portability.32,33 Such computerized methods may still pose some challenges for participants with low literacy and less experience with technology.
Non-computerized interview tools have also been developed for enhancing privacy among low-literacy populations; these include tape-recorded interviews trialled in China,34 informal confidential voting interviews (ICVIs) developed in Zimbabwe,35,36 polling booth surveys (PBSs) in India37,38 and assisted self-administered questionnaires (ASCQs) in Tanzania.39–40 Interviewer bias aside, even highly motivated and uninhibited respondents may have trouble being able to recollect past sexual events.1 In another strategy, the introduction of coital diaries in several low- and middle-income countries has achieved some success in minimizing recall bias.41–45
There are a number of published papers on alternative methods to collect sensitive information. Although these studies provide insight, few quantitatively compare new tools with the traditional FTFI within one population. This article aims to review the empirical data collected in low- and middle-income countries (since >90% of HIV infections take place in these populations5) that compare FTFI with innovative interview tools for reporting HIV risk behaviour, including sexual behaviour and injecting drug use. The purpose of this review is to determine if new non-FTFI techniques reduce misreporting by reducing interviewer bias or otherwise.
Methods
Search strategy
A systematic review of existing literature on alternative interview tools was conducted in three stages. First electronic searches of PubMed, LILACS, Web of Knowledge and Embase were carried out from June 2008 to November 2008 (details of search terms are provided in Figure S1, supplementary data are available at IJE online.). The PubMed search was updated in April 2009. Titles were evaluated and irrelevant articles discarded. Potentially relevant publications’ abstracts were evaluated and the manuscripts were retrieved as PDFs or paper copies for evaluation. These searches were complemented with searches in general engines (e.g. Google Scholar). Finally, the bibliographies of relevant papers identified for inclusion were also searched to identify additional relevant publications. Six authors were contacted, all of whom replied, to provide additional information that was not available from the published article.16,22,26,31,35,39
Two independent investigators (A.E.P. and G.B.G.) screened all the records to determine relevant records. A shortlist was then prepared by both and checked for compatibility.
Inclusion criteria
Types of study
We included any study [randomized controlled trial (RCT) or observational studies] investigating the reporting of HIV risk behaviour, including sexual information or IDU, which compared interviewing tools with the paper-based FTFI.
Types of population
We accepted all studies that took place in low- and middle-income countries (according to World Bank classification) in order to examine the feasibility of alternative interview methods in resource-poor settings. There was no restriction on date or language of publication. Only one paper was returned in Chinese and this was translated into English for the analysis.34
Types of intervention
There was no restriction by study design. The influence of study design was explored in the analysis. To simplify the comparison between methods, recall bias will not be explored in this review.
Types of outcome
Four potentially sensitive outcomes available in all papers were identified and included for review:
‘Have you ever had sex’: since many studies focused on adolescents, it was assumed that social desirability bias may be particularly great among school pupils concerned about peer stigma and adult punishment for pre-marital sex. For example, in Tanzania school pupils are severely punished if they are found to have had sex.46 Therefore, according to our hypothesis a higher proportion should report ever having sex in the alternative interview method.
Non-condom use included a range of questions such as lifetime, last 6 months, last 2 weeks, and last sex act. Although there was no uniform question asked across all papers, it was assumed that it is socially undesirable to report non-condom use regardless of time frame; therefore, respondents would be more likely to admit unprotected sex if the interviewer bias had been removed. In religious communities or where condom use is associated with risky behaviour, condom promotion might be frowned upon and it would be ‘desirable’ to deny condom use. Such a circumstance was not addressed in any of the papers in the review, so we assume condom use is socially desirable.
‘Ever been forced to have sex’: since this outcome was expected to have strong negative connotations, it was expected that in a more private interview setting an increased number of respondents would report forced sex in the alternative interview tool.
Number of partners: time over which partner numbers are elicited and the cut-off points between the categories for the number of partners categorical variable varied across studies. It was assumed that it would be socially undesirable to report a higher number of partners, independently of the categorization used. In order to assess the impact of these variations, time frame and partner cut-off points were included in the subgroup analyses.
Analysis
We hypothesized that innovative methods that bypass the interviewer will be more successful than FTFI in achieving self-reports of risky behaviour, as the impact of interviewer bias has been removed. For this reason, a comparison was made between alternative interviewing tools (mostly ACASI) and FTFI.
Sample size was defined as the number of people who took part in the questionnaire-based surveys because the denominator for each of the four outcomes varied within and between studies. Crude odds ratios (ORs) and the 95% confidence intervals (CIs) were calculated for each study and outcome using FTFI as the comparative group. We assume that if one interview method was really superior, the direction of the association of most included studies would be the same, and thus relatively homogeneous, even though the magnitude of the effect could vary by study characteristics (e.g. gender, region). The Cochran Q test and I-squared were used to assess heterogeneity across studies.47 If there was significant heterogeneity and the studies were not considered combinable, no overall pooled estimate was provided, and sub-group analyses based on study or population characteristics such as gender, education, locale (urban/rural), region (Africa/South America/Europe/Asia) and study design (RCT/cross-sectional) was undertaken to explore potential causes of heterogeneity.
Surveys that interviewed both male and female respondents were considered independent and treated as two separate studies in the meta-analysis, in order to examine the impact of gender. For instance it was anticipated that girls, for whom secrecy about sexual activity is the norm, would be particularly susceptible to reporting more premarital sex in a more private interview setting.22 However, boys may exaggerate their level of sexual activity in the FTFI. Regarding education level of the participants in the studies, we created an aggregate study-level variable: more or less than 60% of the sample having received secondary school education. As a range of education levels had been reported across studies, a cut-off had to be determined that maximized the number of studies in two comparable groups for analysis.
Summary estimates in the subgroup analyses were calculated using random effects models,48 which take into account the within- and between-study variance across studies. Forest plots were used to explore graphically the heterogeneity found between the studies and between the pooled estimates by subgroups.49
The systematic review was undertaken following MOOSE guidelines for reviews of observational studies since not all studies were randomized trials.50
Results
Nearly 1700 articles were identified. Of these, 105 were selected to be reviewed in full. Twenty-one references satisfied the primary selection criteria, examining interviewing techniques in resource-poor settings. Eight additional papers were identified from the references. Of these 29 references, five were excluded because they did not contain quantitative results and four studies compared interviewing tools with SAQ only (the search results are summarized in Table S1; supplementary data are available at IJE online). Of the remaining 20 references, 5 reported results from the same data set and were combined together. As a result, there were a final total of 15 studies, with a total of 20 references as some articles described the same study. The final 15 studies were divided by gender, interview mode and district to create 25 independent data sets to be included in the meta-analysis, the characteristics of which are described in Table 1. The flow chart summarizing the results of the search on self-reported HIV- and STI-associated behaviour in low- and middle-income countries, by interview mode, is shown in Supplementary Data (supplementary data are available at IJE online).
Summary of selected studies comparing alternative quantitative interview tools with FTFI in low- and middle-income countries
| Study population (grey) and reporting estimates (white) | Design | Gender | Tool | ΣN | Age | Ever had sex, n (N – n)a | Non- condom use, n (N – n)a | Timeframe of non-condom use question | Forced sex, n (N − n)a | Number of partners, n (N − n)a | Number of partners cut-off | Timeframe number of partners |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Urban Brazil, injecting drug users | ||||||||||||
| Simoes (2006)14,15 | RCT | C | ACASI | 367 | >18 | 260 (35) | 279 (88) | Inconsistentb | 67 (300) | >2 | 6 months | |
| FTFI | 368 | 338 (30) | 260 (108) | 40 (328) | >2 | |||||||
| Urban Vietnam, adolescents | ||||||||||||
| LeLC (2006)16 | RCT | M | ACASI | 378 | 15–24 | 69 (309) | 50 (22) | Last sex act | 1 (375) | 3.46c | [Mean] | Lifetime |
| M | FTFI | 356 | 54 (302) | 47 (14) | 0 (352) | 1.46c | ||||||
| F | ACASI | 393 | 29 (364) | 50 (22) | 2 (385) | 1.92c | ||||||
| F | FTFI | 465 | 24 (441) | 47 (14) | 0 (461) | 1.06c | ||||||
| Urban Brazil, adolescents | ||||||||||||
| Hewett18 and Mensch19 (2008) | RCT | F | ACASI | 409 | 15–21 | 243 (68) | Last sex act (vaginal sex) | 1.6c | [Mean] | Past 6 months | ||
| FTFI | 409 | 254 (94) | 1.3c | |||||||||
| Urban Thailand, adolescents | ||||||||||||
| Van Griensven (2006)20 | RCT | C | PASI | 328 | 15–21 | 192 (136) | 238 (90) | Last sex act | ||||
| FTFI | 317 | 180 (137) | 184 (133) | |||||||||
| ACASI | 325 | 183 (142) | 259 (66) | |||||||||
| FTFI | 317 | 180 (137) | 184 (133) | |||||||||
| Rural Kenya, adolescents | ||||||||||||
| Mensch (2003)22 and Hewett (2004)23,24 | RCT | M (K) | ACASI | 405 | 15–21 | 193 (212) | 250 (109) | Lifetime | 17 (83) | 45 (55)d | >1 | Lifetime |
| M (K) | FTFI | 361 | 217 (142) | 262 (99) | 9 (91) | 41 (59)d | >1 | |||||
| F (K) | ACASI | 343 | 146 (197) | 256 (69) | 31 (69) | 35 (65)d | >1 | |||||
| F (K) | FTFI | 349 | 168 (180) | 275 (74) | 13 (87) | 21 (79)d | >1 | |||||
| >1 | ||||||||||||
| M (N) | ACASI | 688 | 271 (417) | 111 (160) | ||||||||
| M (N) | FTFI | 821 | 548 (273) | 182 (365) | ||||||||
| F (N) | ACASI | 603 | 138 (465) | 79 (48) | ||||||||
| F (N) | FTFI | 721 | 335 (386) | 178 (155) | ||||||||
| Urban India, college students and slum-dwelling adolescents | ||||||||||||
| Potdar (2005)25 | RCT | M (C) | ACASI | 300 | 18–22 | 18 (282)d | 205 (95)d | Lifetime | 19 (281)d | 25 (275)d | >2 | Lifetime |
| M (C) | FTFI | 300 | 11 (289)d | 243 (57)d | 2 (298)d | 13 (287)d | >2 | |||||
| M (S) | ACASI | 300 | 33 (267)d | 134 (166)d | 24 (276)d | 143 (157)d | >2 | |||||
| M (S) | FTFI | 300 | 105 (195)d | 174 (126)d | 13 (287)d | 143 (157)d | >2 | |||||
| Urban Russia, drug addiction patients | ||||||||||||
| Edwards (2008)26 | XSect | C | ACASI | 180 | >18 | 79 (101)d | Past 6 months | 47 (133)d | >4 | Past 6 months | ||
| FTFI | 180 | 72 (108)d | 50 (130)d | >4 | ||||||||
| Urban Zimbabwe, general (adult) population | ||||||||||||
| Minnis (2007)27 | RCT | F | ACASI | 338 | 18–35 | 122 (216) | Lifetime | 76 (262) | >1 | Lifetime | ||
| FTFI | 315 | 106 (209) | 77 (238) | >1 | ||||||||
| Urban India, adolescents | ||||||||||||
| Jaya (2008)28 | XSect | M | ACASI | 290 | 15–19 | 78 (212) | 33 (257) | |||||
| M | FTFI | 293 | 62 (228) | 23 (267) | ||||||||
| F | ACASI | 233 | 8 (225) | 8 (225) | ||||||||
| F | FTFI | 242 | 3 (230) | 3 (230) | ||||||||
| Rural Malawi, adolescents | ||||||||||||
| Mensch (2008)30 | RCT | F | ACASI | 226 | 15–21 | 79 (147) | 27 (73)d | >1 | Lifetime | |||
| FTFI | 275 | 132 (143) | 17 (84)d | >1 | ||||||||
| Urban Kenya, sex workers | ||||||||||||
| Van der Elst (2009)31 | XSect | M | ACASI | 259 | >18 | 75 (36) | Inconsistent with regular partnerb | 23 (236) | 2 | [Median] | Past month | |
| M | FTFI | 259 | 72 (67) | 10 (249) | 1 | |||||||
| F | ACASI | 139 | 213 (46) | 9 (129) | 2 | |||||||
| F | FTFI | 139 | 230 (29) | 3 (132) | 1 | |||||||
| Rural China, general population | ||||||||||||
| Xia (2004)34 | RCT | F | Tape | 287 | 18–49 | 41 (246) | ||||||
| FTFI | 305 | 46 (259) | ||||||||||
| Rural Zimbabwe, general population | ||||||||||||
| Gregson (2002, 2004)35,36,e | RCT | M | ICVI | 1351 | 15–54 | 999 (353) | Past 2 weeks | 112 (1239) | >1 | Past month | ||
| M | FTFI | 418 | 311 (107) | 21 (398) | >1 | |||||||
| F | ICVI | 1511 | 1304 (207) | 35 (1476) | >1 | |||||||
| F | FTFI | 691 | 624 (67) | 8 (683) | >1 | |||||||
| Rural India, female sex workers | ||||||||||||
| Hanck (2008) 38 | XSect | F | PBS | 269 | >18 | 55 (164) | Last sex act | |||||
| FTFI | 812 | 81 (322) | ||||||||||
| Rural Tanzania, adolescents | ||||||||||||
| Plummer (2004)39,40,f | RCT | M | ASCQ | 1430 | 15–21 | 1430 (1142) | 1986 (68) | Lifetime | 149 (2423) | 240 (1055) | >2 | Past month |
| 53 (1242) | >4 | |||||||||||
| M | FTFI | 1343 | 1327 (1245) | 2024 (30) | 3 (2569) | 105 (1238) | >2 | |||||
| 7 (1336) | >4 | |||||||||||
| F | ASCQ | 449 | 481 (1684) | 2225 (172) | 266 (1899) | 48 (401) | >2 | |||||
| 4 (445) | >4 | |||||||||||
| F | FTFI | 502 | 489 (1676) | 2348 (49) | 4 (2161) | 32 (470) | >2 | |||||
| 2 (500) | >4 | |||||||||||
| Study population (grey) and reporting estimates (white) | Design | Gender | Tool | ΣN | Age | Ever had sex, n (N – n)a | Non- condom use, n (N – n)a | Timeframe of non-condom use question | Forced sex, n (N − n)a | Number of partners, n (N − n)a | Number of partners cut-off | Timeframe number of partners |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Urban Brazil, injecting drug users | ||||||||||||
| Simoes (2006)14,15 | RCT | C | ACASI | 367 | >18 | 260 (35) | 279 (88) | Inconsistentb | 67 (300) | >2 | 6 months | |
| FTFI | 368 | 338 (30) | 260 (108) | 40 (328) | >2 | |||||||
| Urban Vietnam, adolescents | ||||||||||||
| LeLC (2006)16 | RCT | M | ACASI | 378 | 15–24 | 69 (309) | 50 (22) | Last sex act | 1 (375) | 3.46c | [Mean] | Lifetime |
| M | FTFI | 356 | 54 (302) | 47 (14) | 0 (352) | 1.46c | ||||||
| F | ACASI | 393 | 29 (364) | 50 (22) | 2 (385) | 1.92c | ||||||
| F | FTFI | 465 | 24 (441) | 47 (14) | 0 (461) | 1.06c | ||||||
| Urban Brazil, adolescents | ||||||||||||
| Hewett18 and Mensch19 (2008) | RCT | F | ACASI | 409 | 15–21 | 243 (68) | Last sex act (vaginal sex) | 1.6c | [Mean] | Past 6 months | ||
| FTFI | 409 | 254 (94) | 1.3c | |||||||||
| Urban Thailand, adolescents | ||||||||||||
| Van Griensven (2006)20 | RCT | C | PASI | 328 | 15–21 | 192 (136) | 238 (90) | Last sex act | ||||
| FTFI | 317 | 180 (137) | 184 (133) | |||||||||
| ACASI | 325 | 183 (142) | 259 (66) | |||||||||
| FTFI | 317 | 180 (137) | 184 (133) | |||||||||
| Rural Kenya, adolescents | ||||||||||||
| Mensch (2003)22 and Hewett (2004)23,24 | RCT | M (K) | ACASI | 405 | 15–21 | 193 (212) | 250 (109) | Lifetime | 17 (83) | 45 (55)d | >1 | Lifetime |
| M (K) | FTFI | 361 | 217 (142) | 262 (99) | 9 (91) | 41 (59)d | >1 | |||||
| F (K) | ACASI | 343 | 146 (197) | 256 (69) | 31 (69) | 35 (65)d | >1 | |||||
| F (K) | FTFI | 349 | 168 (180) | 275 (74) | 13 (87) | 21 (79)d | >1 | |||||
| >1 | ||||||||||||
| M (N) | ACASI | 688 | 271 (417) | 111 (160) | ||||||||
| M (N) | FTFI | 821 | 548 (273) | 182 (365) | ||||||||
| F (N) | ACASI | 603 | 138 (465) | 79 (48) | ||||||||
| F (N) | FTFI | 721 | 335 (386) | 178 (155) | ||||||||
| Urban India, college students and slum-dwelling adolescents | ||||||||||||
| Potdar (2005)25 | RCT | M (C) | ACASI | 300 | 18–22 | 18 (282)d | 205 (95)d | Lifetime | 19 (281)d | 25 (275)d | >2 | Lifetime |
| M (C) | FTFI | 300 | 11 (289)d | 243 (57)d | 2 (298)d | 13 (287)d | >2 | |||||
| M (S) | ACASI | 300 | 33 (267)d | 134 (166)d | 24 (276)d | 143 (157)d | >2 | |||||
| M (S) | FTFI | 300 | 105 (195)d | 174 (126)d | 13 (287)d | 143 (157)d | >2 | |||||
| Urban Russia, drug addiction patients | ||||||||||||
| Edwards (2008)26 | XSect | C | ACASI | 180 | >18 | 79 (101)d | Past 6 months | 47 (133)d | >4 | Past 6 months | ||
| FTFI | 180 | 72 (108)d | 50 (130)d | >4 | ||||||||
| Urban Zimbabwe, general (adult) population | ||||||||||||
| Minnis (2007)27 | RCT | F | ACASI | 338 | 18–35 | 122 (216) | Lifetime | 76 (262) | >1 | Lifetime | ||
| FTFI | 315 | 106 (209) | 77 (238) | >1 | ||||||||
| Urban India, adolescents | ||||||||||||
| Jaya (2008)28 | XSect | M | ACASI | 290 | 15–19 | 78 (212) | 33 (257) | |||||
| M | FTFI | 293 | 62 (228) | 23 (267) | ||||||||
| F | ACASI | 233 | 8 (225) | 8 (225) | ||||||||
| F | FTFI | 242 | 3 (230) | 3 (230) | ||||||||
| Rural Malawi, adolescents | ||||||||||||
| Mensch (2008)30 | RCT | F | ACASI | 226 | 15–21 | 79 (147) | 27 (73)d | >1 | Lifetime | |||
| FTFI | 275 | 132 (143) | 17 (84)d | >1 | ||||||||
| Urban Kenya, sex workers | ||||||||||||
| Van der Elst (2009)31 | XSect | M | ACASI | 259 | >18 | 75 (36) | Inconsistent with regular partnerb | 23 (236) | 2 | [Median] | Past month | |
| M | FTFI | 259 | 72 (67) | 10 (249) | 1 | |||||||
| F | ACASI | 139 | 213 (46) | 9 (129) | 2 | |||||||
| F | FTFI | 139 | 230 (29) | 3 (132) | 1 | |||||||
| Rural China, general population | ||||||||||||
| Xia (2004)34 | RCT | F | Tape | 287 | 18–49 | 41 (246) | ||||||
| FTFI | 305 | 46 (259) | ||||||||||
| Rural Zimbabwe, general population | ||||||||||||
| Gregson (2002, 2004)35,36,e | RCT | M | ICVI | 1351 | 15–54 | 999 (353) | Past 2 weeks | 112 (1239) | >1 | Past month | ||
| M | FTFI | 418 | 311 (107) | 21 (398) | >1 | |||||||
| F | ICVI | 1511 | 1304 (207) | 35 (1476) | >1 | |||||||
| F | FTFI | 691 | 624 (67) | 8 (683) | >1 | |||||||
| Rural India, female sex workers | ||||||||||||
| Hanck (2008) 38 | XSect | F | PBS | 269 | >18 | 55 (164) | Last sex act | |||||
| FTFI | 812 | 81 (322) | ||||||||||
| Rural Tanzania, adolescents | ||||||||||||
| Plummer (2004)39,40,f | RCT | M | ASCQ | 1430 | 15–21 | 1430 (1142) | 1986 (68) | Lifetime | 149 (2423) | 240 (1055) | >2 | Past month |
| 53 (1242) | >4 | |||||||||||
| M | FTFI | 1343 | 1327 (1245) | 2024 (30) | 3 (2569) | 105 (1238) | >2 | |||||
| 7 (1336) | >4 | |||||||||||
| F | ASCQ | 449 | 481 (1684) | 2225 (172) | 266 (1899) | 48 (401) | >2 | |||||
| 4 (445) | >4 | |||||||||||
| F | FTFI | 502 | 489 (1676) | 2348 (49) | 4 (2161) | 32 (470) | >2 | |||||
| 2 (500) | >4 | |||||||||||
Design of sampling method—RCT: randomized controlled trial (i.e. respondents were randomly allocated to either innovative interview tool or FTFI), XSect: cross-sectional survey where respondents answered both interview tools. Gender—C; combined male and female studies; M: male data were reported and analysed independently; F: female data were reported and analysed independently; Mensch23 (2003) and Hewett24,25 (2004) have been divided by gender and location of study—Kisumu and Nyeri districts (K) and (N), respectively; Potdar (2005)26 has been divided by population type—College students (C) and slum-swelling adolescents (S); Outcome—n: the number of respondents who answered positively to the question in each interview tool and N − n: the number of respondents who answered negatively to the question in each interview tool. Number of partners—the threshold of partner number varied across the studies. ΣN is the number of subjects who participated in the study by interview tool. Please note that the number of respondents answering each question may vary; for example, those that had never had sex would not answer a question on non-condom use. The total number of respondents answering each question can be calculated by adding together the number answering by alternative interview type plus FTFI.
an and N–n may not always add up to the total sample population as some respondents may not have answered the outcome question. ORs were then derived by (odds of reporting behaviour in non-FTFI)/(odds of reporting behaviour in FTFI).
bThese papers did not report a time frame of condom use but whether the respondent reported ‘inconsistent’ condom use. We assume, therefore, that it is not desirable to deny condom use.
cThe mean number of partners were given by interview type in this study; as a result they were not included in the meta-analysis.
dThe results in these papers were given as percentages; these were then re-calculated using the total sample size to calculate absolute numbers.
eGregson et al.: Round 1 data has been analysed only.
fPlummer et al. reported number of partners for several thresholds, those reporting more than two and four partners have been included in the meta-analysis for ASCQ data in 1998 only. Note: Some outcomes are blank as not every question was asked in each paper.
Summary of selected studies comparing alternative quantitative interview tools with FTFI in low- and middle-income countries
| Study population (grey) and reporting estimates (white) | Design | Gender | Tool | ΣN | Age | Ever had sex, n (N – n)a | Non- condom use, n (N – n)a | Timeframe of non-condom use question | Forced sex, n (N − n)a | Number of partners, n (N − n)a | Number of partners cut-off | Timeframe number of partners |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Urban Brazil, injecting drug users | ||||||||||||
| Simoes (2006)14,15 | RCT | C | ACASI | 367 | >18 | 260 (35) | 279 (88) | Inconsistentb | 67 (300) | >2 | 6 months | |
| FTFI | 368 | 338 (30) | 260 (108) | 40 (328) | >2 | |||||||
| Urban Vietnam, adolescents | ||||||||||||
| LeLC (2006)16 | RCT | M | ACASI | 378 | 15–24 | 69 (309) | 50 (22) | Last sex act | 1 (375) | 3.46c | [Mean] | Lifetime |
| M | FTFI | 356 | 54 (302) | 47 (14) | 0 (352) | 1.46c | ||||||
| F | ACASI | 393 | 29 (364) | 50 (22) | 2 (385) | 1.92c | ||||||
| F | FTFI | 465 | 24 (441) | 47 (14) | 0 (461) | 1.06c | ||||||
| Urban Brazil, adolescents | ||||||||||||
| Hewett18 and Mensch19 (2008) | RCT | F | ACASI | 409 | 15–21 | 243 (68) | Last sex act (vaginal sex) | 1.6c | [Mean] | Past 6 months | ||
| FTFI | 409 | 254 (94) | 1.3c | |||||||||
| Urban Thailand, adolescents | ||||||||||||
| Van Griensven (2006)20 | RCT | C | PASI | 328 | 15–21 | 192 (136) | 238 (90) | Last sex act | ||||
| FTFI | 317 | 180 (137) | 184 (133) | |||||||||
| ACASI | 325 | 183 (142) | 259 (66) | |||||||||
| FTFI | 317 | 180 (137) | 184 (133) | |||||||||
| Rural Kenya, adolescents | ||||||||||||
| Mensch (2003)22 and Hewett (2004)23,24 | RCT | M (K) | ACASI | 405 | 15–21 | 193 (212) | 250 (109) | Lifetime | 17 (83) | 45 (55)d | >1 | Lifetime |
| M (K) | FTFI | 361 | 217 (142) | 262 (99) | 9 (91) | 41 (59)d | >1 | |||||
| F (K) | ACASI | 343 | 146 (197) | 256 (69) | 31 (69) | 35 (65)d | >1 | |||||
| F (K) | FTFI | 349 | 168 (180) | 275 (74) | 13 (87) | 21 (79)d | >1 | |||||
| >1 | ||||||||||||
| M (N) | ACASI | 688 | 271 (417) | 111 (160) | ||||||||
| M (N) | FTFI | 821 | 548 (273) | 182 (365) | ||||||||
| F (N) | ACASI | 603 | 138 (465) | 79 (48) | ||||||||
| F (N) | FTFI | 721 | 335 (386) | 178 (155) | ||||||||
| Urban India, college students and slum-dwelling adolescents | ||||||||||||
| Potdar (2005)25 | RCT | M (C) | ACASI | 300 | 18–22 | 18 (282)d | 205 (95)d | Lifetime | 19 (281)d | 25 (275)d | >2 | Lifetime |
| M (C) | FTFI | 300 | 11 (289)d | 243 (57)d | 2 (298)d | 13 (287)d | >2 | |||||
| M (S) | ACASI | 300 | 33 (267)d | 134 (166)d | 24 (276)d | 143 (157)d | >2 | |||||
| M (S) | FTFI | 300 | 105 (195)d | 174 (126)d | 13 (287)d | 143 (157)d | >2 | |||||
| Urban Russia, drug addiction patients | ||||||||||||
| Edwards (2008)26 | XSect | C | ACASI | 180 | >18 | 79 (101)d | Past 6 months | 47 (133)d | >4 | Past 6 months | ||
| FTFI | 180 | 72 (108)d | 50 (130)d | >4 | ||||||||
| Urban Zimbabwe, general (adult) population | ||||||||||||
| Minnis (2007)27 | RCT | F | ACASI | 338 | 18–35 | 122 (216) | Lifetime | 76 (262) | >1 | Lifetime | ||
| FTFI | 315 | 106 (209) | 77 (238) | >1 | ||||||||
| Urban India, adolescents | ||||||||||||
| Jaya (2008)28 | XSect | M | ACASI | 290 | 15–19 | 78 (212) | 33 (257) | |||||
| M | FTFI | 293 | 62 (228) | 23 (267) | ||||||||
| F | ACASI | 233 | 8 (225) | 8 (225) | ||||||||
| F | FTFI | 242 | 3 (230) | 3 (230) | ||||||||
| Rural Malawi, adolescents | ||||||||||||
| Mensch (2008)30 | RCT | F | ACASI | 226 | 15–21 | 79 (147) | 27 (73)d | >1 | Lifetime | |||
| FTFI | 275 | 132 (143) | 17 (84)d | >1 | ||||||||
| Urban Kenya, sex workers | ||||||||||||
| Van der Elst (2009)31 | XSect | M | ACASI | 259 | >18 | 75 (36) | Inconsistent with regular partnerb | 23 (236) | 2 | [Median] | Past month | |
| M | FTFI | 259 | 72 (67) | 10 (249) | 1 | |||||||
| F | ACASI | 139 | 213 (46) | 9 (129) | 2 | |||||||
| F | FTFI | 139 | 230 (29) | 3 (132) | 1 | |||||||
| Rural China, general population | ||||||||||||
| Xia (2004)34 | RCT | F | Tape | 287 | 18–49 | 41 (246) | ||||||
| FTFI | 305 | 46 (259) | ||||||||||
| Rural Zimbabwe, general population | ||||||||||||
| Gregson (2002, 2004)35,36,e | RCT | M | ICVI | 1351 | 15–54 | 999 (353) | Past 2 weeks | 112 (1239) | >1 | Past month | ||
| M | FTFI | 418 | 311 (107) | 21 (398) | >1 | |||||||
| F | ICVI | 1511 | 1304 (207) | 35 (1476) | >1 | |||||||
| F | FTFI | 691 | 624 (67) | 8 (683) | >1 | |||||||
| Rural India, female sex workers | ||||||||||||
| Hanck (2008) 38 | XSect | F | PBS | 269 | >18 | 55 (164) | Last sex act | |||||
| FTFI | 812 | 81 (322) | ||||||||||
| Rural Tanzania, adolescents | ||||||||||||
| Plummer (2004)39,40,f | RCT | M | ASCQ | 1430 | 15–21 | 1430 (1142) | 1986 (68) | Lifetime | 149 (2423) | 240 (1055) | >2 | Past month |
| 53 (1242) | >4 | |||||||||||
| M | FTFI | 1343 | 1327 (1245) | 2024 (30) | 3 (2569) | 105 (1238) | >2 | |||||
| 7 (1336) | >4 | |||||||||||
| F | ASCQ | 449 | 481 (1684) | 2225 (172) | 266 (1899) | 48 (401) | >2 | |||||
| 4 (445) | >4 | |||||||||||
| F | FTFI | 502 | 489 (1676) | 2348 (49) | 4 (2161) | 32 (470) | >2 | |||||
| 2 (500) | >4 | |||||||||||
| Study population (grey) and reporting estimates (white) | Design | Gender | Tool | ΣN | Age | Ever had sex, n (N – n)a | Non- condom use, n (N – n)a | Timeframe of non-condom use question | Forced sex, n (N − n)a | Number of partners, n (N − n)a | Number of partners cut-off | Timeframe number of partners |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Urban Brazil, injecting drug users | ||||||||||||
| Simoes (2006)14,15 | RCT | C | ACASI | 367 | >18 | 260 (35) | 279 (88) | Inconsistentb | 67 (300) | >2 | 6 months | |
| FTFI | 368 | 338 (30) | 260 (108) | 40 (328) | >2 | |||||||
| Urban Vietnam, adolescents | ||||||||||||
| LeLC (2006)16 | RCT | M | ACASI | 378 | 15–24 | 69 (309) | 50 (22) | Last sex act | 1 (375) | 3.46c | [Mean] | Lifetime |
| M | FTFI | 356 | 54 (302) | 47 (14) | 0 (352) | 1.46c | ||||||
| F | ACASI | 393 | 29 (364) | 50 (22) | 2 (385) | 1.92c | ||||||
| F | FTFI | 465 | 24 (441) | 47 (14) | 0 (461) | 1.06c | ||||||
| Urban Brazil, adolescents | ||||||||||||
| Hewett18 and Mensch19 (2008) | RCT | F | ACASI | 409 | 15–21 | 243 (68) | Last sex act (vaginal sex) | 1.6c | [Mean] | Past 6 months | ||
| FTFI | 409 | 254 (94) | 1.3c | |||||||||
| Urban Thailand, adolescents | ||||||||||||
| Van Griensven (2006)20 | RCT | C | PASI | 328 | 15–21 | 192 (136) | 238 (90) | Last sex act | ||||
| FTFI | 317 | 180 (137) | 184 (133) | |||||||||
| ACASI | 325 | 183 (142) | 259 (66) | |||||||||
| FTFI | 317 | 180 (137) | 184 (133) | |||||||||
| Rural Kenya, adolescents | ||||||||||||
| Mensch (2003)22 and Hewett (2004)23,24 | RCT | M (K) | ACASI | 405 | 15–21 | 193 (212) | 250 (109) | Lifetime | 17 (83) | 45 (55)d | >1 | Lifetime |
| M (K) | FTFI | 361 | 217 (142) | 262 (99) | 9 (91) | 41 (59)d | >1 | |||||
| F (K) | ACASI | 343 | 146 (197) | 256 (69) | 31 (69) | 35 (65)d | >1 | |||||
| F (K) | FTFI | 349 | 168 (180) | 275 (74) | 13 (87) | 21 (79)d | >1 | |||||
| >1 | ||||||||||||
| M (N) | ACASI | 688 | 271 (417) | 111 (160) | ||||||||
| M (N) | FTFI | 821 | 548 (273) | 182 (365) | ||||||||
| F (N) | ACASI | 603 | 138 (465) | 79 (48) | ||||||||
| F (N) | FTFI | 721 | 335 (386) | 178 (155) | ||||||||
| Urban India, college students and slum-dwelling adolescents | ||||||||||||
| Potdar (2005)25 | RCT | M (C) | ACASI | 300 | 18–22 | 18 (282)d | 205 (95)d | Lifetime | 19 (281)d | 25 (275)d | >2 | Lifetime |
| M (C) | FTFI | 300 | 11 (289)d | 243 (57)d | 2 (298)d | 13 (287)d | >2 | |||||
| M (S) | ACASI | 300 | 33 (267)d | 134 (166)d | 24 (276)d | 143 (157)d | >2 | |||||
| M (S) | FTFI | 300 | 105 (195)d | 174 (126)d | 13 (287)d | 143 (157)d | >2 | |||||
| Urban Russia, drug addiction patients | ||||||||||||
| Edwards (2008)26 | XSect | C | ACASI | 180 | >18 | 79 (101)d | Past 6 months | 47 (133)d | >4 | Past 6 months | ||
| FTFI | 180 | 72 (108)d | 50 (130)d | >4 | ||||||||
| Urban Zimbabwe, general (adult) population | ||||||||||||
| Minnis (2007)27 | RCT | F | ACASI | 338 | 18–35 | 122 (216) | Lifetime | 76 (262) | >1 | Lifetime | ||
| FTFI | 315 | 106 (209) | 77 (238) | >1 | ||||||||
| Urban India, adolescents | ||||||||||||
| Jaya (2008)28 | XSect | M | ACASI | 290 | 15–19 | 78 (212) | 33 (257) | |||||
| M | FTFI | 293 | 62 (228) | 23 (267) | ||||||||
| F | ACASI | 233 | 8 (225) | 8 (225) | ||||||||
| F | FTFI | 242 | 3 (230) | 3 (230) | ||||||||
| Rural Malawi, adolescents | ||||||||||||
| Mensch (2008)30 | RCT | F | ACASI | 226 | 15–21 | 79 (147) | 27 (73)d | >1 | Lifetime | |||
| FTFI | 275 | 132 (143) | 17 (84)d | >1 | ||||||||
| Urban Kenya, sex workers | ||||||||||||
| Van der Elst (2009)31 | XSect | M | ACASI | 259 | >18 | 75 (36) | Inconsistent with regular partnerb | 23 (236) | 2 | [Median] | Past month | |
| M | FTFI | 259 | 72 (67) | 10 (249) | 1 | |||||||
| F | ACASI | 139 | 213 (46) | 9 (129) | 2 | |||||||
| F | FTFI | 139 | 230 (29) | 3 (132) | 1 | |||||||
| Rural China, general population | ||||||||||||
| Xia (2004)34 | RCT | F | Tape | 287 | 18–49 | 41 (246) | ||||||
| FTFI | 305 | 46 (259) | ||||||||||
| Rural Zimbabwe, general population | ||||||||||||
| Gregson (2002, 2004)35,36,e | RCT | M | ICVI | 1351 | 15–54 | 999 (353) | Past 2 weeks | 112 (1239) | >1 | Past month | ||
| M | FTFI | 418 | 311 (107) | 21 (398) | >1 | |||||||
| F | ICVI | 1511 | 1304 (207) | 35 (1476) | >1 | |||||||
| F | FTFI | 691 | 624 (67) | 8 (683) | >1 | |||||||
| Rural India, female sex workers | ||||||||||||
| Hanck (2008) 38 | XSect | F | PBS | 269 | >18 | 55 (164) | Last sex act | |||||
| FTFI | 812 | 81 (322) | ||||||||||
| Rural Tanzania, adolescents | ||||||||||||
| Plummer (2004)39,40,f | RCT | M | ASCQ | 1430 | 15–21 | 1430 (1142) | 1986 (68) | Lifetime | 149 (2423) | 240 (1055) | >2 | Past month |
| 53 (1242) | >4 | |||||||||||
| M | FTFI | 1343 | 1327 (1245) | 2024 (30) | 3 (2569) | 105 (1238) | >2 | |||||
| 7 (1336) | >4 | |||||||||||
| F | ASCQ | 449 | 481 (1684) | 2225 (172) | 266 (1899) | 48 (401) | >2 | |||||
| 4 (445) | >4 | |||||||||||
| F | FTFI | 502 | 489 (1676) | 2348 (49) | 4 (2161) | 32 (470) | >2 | |||||
| 2 (500) | >4 | |||||||||||
Design of sampling method—RCT: randomized controlled trial (i.e. respondents were randomly allocated to either innovative interview tool or FTFI), XSect: cross-sectional survey where respondents answered both interview tools. Gender—C; combined male and female studies; M: male data were reported and analysed independently; F: female data were reported and analysed independently; Mensch23 (2003) and Hewett24,25 (2004) have been divided by gender and location of study—Kisumu and Nyeri districts (K) and (N), respectively; Potdar (2005)26 has been divided by population type—College students (C) and slum-swelling adolescents (S); Outcome—n: the number of respondents who answered positively to the question in each interview tool and N − n: the number of respondents who answered negatively to the question in each interview tool. Number of partners—the threshold of partner number varied across the studies. ΣN is the number of subjects who participated in the study by interview tool. Please note that the number of respondents answering each question may vary; for example, those that had never had sex would not answer a question on non-condom use. The total number of respondents answering each question can be calculated by adding together the number answering by alternative interview type plus FTFI.
an and N–n may not always add up to the total sample population as some respondents may not have answered the outcome question. ORs were then derived by (odds of reporting behaviour in non-FTFI)/(odds of reporting behaviour in FTFI).
bThese papers did not report a time frame of condom use but whether the respondent reported ‘inconsistent’ condom use. We assume, therefore, that it is not desirable to deny condom use.
cThe mean number of partners were given by interview type in this study; as a result they were not included in the meta-analysis.
dThe results in these papers were given as percentages; these were then re-calculated using the total sample size to calculate absolute numbers.
eGregson et al.: Round 1 data has been analysed only.
fPlummer et al. reported number of partners for several thresholds, those reporting more than two and four partners have been included in the meta-analysis for ASCQ data in 1998 only. Note: Some outcomes are blank as not every question was asked in each paper.
Figures 1–4 show ORs and 95% CIs for reporting HIV risk behaviour by interview mode. An OR >1 meant respondents were more likely to report a risk behaviour using an alternative interview mode than by FTFI. The funnel plots in Supplementary Data (supplementary data are available at IJE online) are reasonably symmetrical; therefore we can be reasonably confident in assuming there is no publication bias for these outcomes.
![OR for ‘ever had sex’ and ‘non-condom use’ (ref. group: FTFI). This figure summarizes the 17 studies that report ‘ever had sex’, where the summary effect was more favourable for the FTFI than the alternative interview mode but with significant heterogeneity (I2 = 93.4% P < 0.001). It also summarizes the 21 studies that reported on ‘non-condom use’, some of which were included in the ‘ever had sex’ analysis. Respondents were more likely to report non-condom use in the FTFI, although the difference was not statistically significant and there was significant heterogeneity across studies (I2 = 89.3% P < 0.001). Four papers were divided into separate studies by gender [(M) = Male and (F) = Female] as the data were reported by individual sex. Potdar and Koenig analysed slum-dwelling (S) and college (C) youth individually and so have been divided into two separate studies. Van Grievensen reported individual comparisons between ACASI (A) and PASI (P) with FTFI. Mensch sampled two regions in Kenya (Kisumu and Nyeri) where the author further divided the samples by gender; therefore, the data have been split into four categories: Kisumu female (FK) and male (MK) and Nyeri female (FN) and male (MN)](https://oup.silverchair-cdn.com/oup/backfile/Content_public/Journal/ije/39/6/10.1093_ije_dyq114/1/m_dyq114f1.gif?Expires=1709884775&Signature=zXSFg2niC4BakXBxzWdQXwurIXqSPeMn2Zx-4DGgQoIzHMV4vFFylGPKlTT1C4MNzuB4lBjaSthSKCrGZrDzEn5RUwX-UzBDsT-TYCxXDP7NUWnyN~Jsec9rHQb2NtykxZEYS4AYo6vab5ZCaqf-ozfmPVb0PMbxTncIwu22ySxoV2G2GFRzb6il7gkhp3CgfOxVGudibCevuLJcysOQ7cKCZniR4XulGgzAB5BTbFGG4oOvSEuJA4F9JsMBU4ArKDcsYQsExm4Tx~TNOYCwjrHVhWtndI8bF85ERi9yEQ-QCU3vyuKatmTsKhdCJEuTQotuoaePibFvlivhVofZIg__&Key-Pair-Id=APKAIE5G5CRDK6RD3PGA)
OR for ‘ever had sex’ and ‘non-condom use’ (ref. group: FTFI). This figure summarizes the 17 studies that report ‘ever had sex’, where the summary effect was more favourable for the FTFI than the alternative interview mode but with significant heterogeneity (I2 = 93.4% P < 0.001). It also summarizes the 21 studies that reported on ‘non-condom use’, some of which were included in the ‘ever had sex’ analysis. Respondents were more likely to report non-condom use in the FTFI, although the difference was not statistically significant and there was significant heterogeneity across studies (I2 = 89.3% P < 0.001). Four papers were divided into separate studies by gender [(M) = Male and (F) = Female] as the data were reported by individual sex. Potdar and Koenig analysed slum-dwelling (S) and college (C) youth individually and so have been divided into two separate studies. Van Grievensen reported individual comparisons between ACASI (A) and PASI (P) with FTFI. Mensch sampled two regions in Kenya (Kisumu and Nyeri) where the author further divided the samples by gender; therefore, the data have been split into four categories: Kisumu female (FK) and male (MK) and Nyeri female (FN) and male (MN)
(a) Pooled estimates for subgroup analysis for ‘ever had sex’ (ref. group: FTFI). Since there is significant heterogeneity across all studies for ever had sex (I2 = 93.4% P < 0.001), the source of variation was explored by grouping the studies according to study-level characteristics. More educated respondents had a significantly higher pooled estimate of reporting sex in non-FTFI methods as well as studies from Asia and those sampled in urban areas presenting higher pooled OR. (b) Pooled estimates for subgroup analysis for ‘non-condom use’ (ref. group: FTFI). Since there is significant heterogeneity across all studies for non-condom use (I2 = 89.3% P < 0.001), pooled estimates were calculated. There were no statistically significant differences by study characteristics for reporting non-condom use by interview mode
OR estimates for ‘forced sex’ and ‘number of partners’ (ref. group: FTFI). This figure summarizes the 12 studies reporting ‘ever been forced to have sex’ and 12 studies (not identical) reporting ‘number of partners’. There was statistically significant and consistent increased reporting of coerced sex (I2 = 0% P = 0.55) and higher number of partners (I2 = 75.3% P < 0.001) among non-FTFI participants
(a) Pooled estimates for subgroup analysis for ‘ever forced to have sex’ (ref. group: FTFI). Although there is homogeneity across all studies for forced sex (I2 = 0% P = 0.972), pooled estimates were calculated to examine trends in reported forced sex by study characteristics. Those sampled in African rural areas and studies where <60% of the sample had secondary education presented higher pooled OR. The difference between subgroups, however, was not significant. (b) Pooled estimates for subgroup analysis for ‘number of partners’ (ref. group: FTFI). Since there is significant heterogeneity across all studies for number of partners (I2 = 78.8% P < 0.001), pooled estimates were calculated. This figure shows that those sampled in rural areas presented higher pooled OR. There was also a significant difference in the time frame of the question, with a higher number of partners reported in the past month compared with the past six months or over a lifetime. Plummer et al. have reported number of partners over the >2 and >4 threshold; therefore, the sample was divided into two in order to explore the sensitivity of the two cut-off points. It is important to note that the same individuals were not included in both the >2 and >4 groups. Given the small sample size in the Plummer analysis, the OR was calculated without the Plummer et al. results included and were as follows: Gender: female 1.63 (1.22–2.18) and male 1.33 (0.96–1.85); region: Africa 1.49 (1.13–1.94); education: 60% more than secondary 1.34 (0.95–1.91); locale: Rural 1.69 (1.34–2.13); time frame of question: past month 1.78 (1.31–2.42)
Description of the characteristics of the studies included
Out of the total of 15 studies, 6 studies were carried out in Asia and Africa, 2 in South America and 1 in Europe. Among these, 8 interviewed adolescents, 3 the general population, and 2 among IDUs and sex workers. Most of the studies compared FTFI with ACASI (11 studies) and there was 1 study comparing with Palm-top assisted self-interviewing (PASI), tape-recorded interviews, ICVI, PBS and ASCQ.
Since potential biases are likely to occur for non-randomized trials compared with randomized trials, the heterogeneity of the included studies by study design was investigated (see Figures S6–S9; supplementary data are available at IJE online). Since the meta-analysis demonstrated no difference by study design, the results from the different study types were grouped together in the analysis. Study population, study design, gender, interviewing tool, age, study outcomes and total sample size are summarized in Table 1.
‘Ever had sex’
There were a total of 17 studies reporting ‘ever had sex’. As shown in Figure 1, there is significant heterogeneity between studies (I2 = 93.4%, P < 0.001) preventing a summary estimate to be interpreted. However, in the subgroup analysis presented in Figure 2A, studies with a higher proportion of participants having finished secondary education had a significantly higher pooled estimate of reporting sex in non-FTFI methods. There were trends observed between region, locale and study design, with studies from Asia, those sampled in urban areas and cross-sectional studies presenting significantly higher pooled OR. However, although the CIs overlap for these subgroups and the difference in reporting was therefore not statistically significant, there is indication that the new tool elicited a greater risk behaviour reporting in non-FTFI for the outcome ‘ever had sex’.
As a result of the distinctions emerging in Figure 2A, ‘ever had sex’ by study design, region, education and locale were explored to examine heterogeneity between studies, as shown in Supplementary Data and Supplementary Data (supplementary data are available at IJE online), respectively. Reporting ever had sex was higher in all studies carried out in Asia and urban areas, with the exception of boys recruited in slums of India, which is likely to be confounded by lack of education.25
‘Non-condom use’
There were 21 studies that reported on ‘non-condom use’. As shown in the bottom part of Figure 1, there is also significant heterogeneity present across all studies for non-condom use (I2 = 89.3%, P < 0.001) preventing us to quote a summary statistic. Subgroup analysis to explore the source of variation is included in Figure 2B. In contrast to ‘ever had sex’, there were fewer clear distinctions emerging. There was a trend to an increased reporting of non-condom use among non-FTFI participants in all regions except Africa. This effect was particularly pronounced regarding South America. We also observed a trend in urban areas towards a higher effect estimate, which supported the findings of ‘ever had sex’. However, the CIs are large and overlap for these subgroups. ‘Last sex act’ seemed to show a greater increase in reporting of non-condom use with respect to questions asked with a longer time frame. In contrast to ‘ever had sex’, those sampled through RCTs presented with higher pooled OR and as a result this outcome was explored to examine heterogeneity between studies, as shown in Supplementary Data (supplementary data are available at IJE online).
‘Ever forced to have sex’
There were a total of 10 studies reporting ‘ever been forced to have sex’. There was consistent increased reporting of coerced sex among non-FTFI participants in all studies with no significant heterogeneity between them (Figure 3: I2 = 0% P = 0.55). Contrary to expectation, Figure 4A shows a trend towards less educated respondents as well as rural, African studies presenting a slightly higher pooled estimate of reporting forced sex in non-FTFI method, although the CIs overlap for these subgroups.
‘Number of sexual partners’
There were a total of 12 studies reporting ‘number of partners’. Nearly all studies demonstrated increased reporting of a higher number of sexual partners among non-FTFI participants (Figure 3), with significant heterogeneity observed between them (P < 0.01). Figure 4B shows a higher reporting in non-FTFI among more educated respondents but contrary to the trends observed for ‘ever had sex’ and non-condom use, studies from rural areas presented the highest pooled OR in non-FTFI. We also observed a similar trend regarding studies in the African region and RCTs, but in these subgroup analyses CIs overlap. Figure 4B also shows that studies with a shorter reporting time frame (past month) report higher numbers of partners in the non-FTFI. Furthermore, as the number of partner threshold increases (i.e. more stigmatized behaviour) the difference between the FTFI and alternative tool also tends to increase. As a result of the distinctions emerging in Figure 4B, the number of partners by region and time frame were explored to examine heterogeneity between studies, as shown in Supplementary Data (supplementary data are available at IJE online), respectively.
A comparison was also made between innovative interview tools and paper-based SAQs. Although the results have not been analysed in detail, the summary Supplementary Data (supplementary data are available at IJE online) outlines the similar estimates and lack of difference between SAQ and alternative interview methods for the outcome ‘ever had sex’ (OR 1.07; 95% CI 0.91, 1.25) and non-condom use (OR 0.87; 95% CI 0.67, 1.12). Although forced sex also demonstrated increased reporting in the non-FTFI/SAQ method, the difference was not as significant.
Discussion
This review evaluates the impact of innovative modes of data collection in low- and middle-income countries by collating studies that compare two or more interview tools on one population in reporting HIV risk behaviour. The purpose of this meta-analysis was to identify patterns and sources of disagreement among the results. With no gold standard in sexual behaviour research, it is difficult to determine whether modification of interview mode (e.g. self-administered vs interviewer-administered) increases the accuracy of reporting sensitive behaviour. However, the results showed that alternatives to FTFIs do not always yield higher estimates of risky behaviour.
Our hypothesis was based upon previous research indicating that a method providing greater anonymity and privacy than conventional interviewer-administered methods was more likely to yield higher affirmative response rates to sensitive questions.35 This theory was based upon the assumption that respondents’ perception of social norms, and their notions of acceptable behaviour, influence their willingness to respond truthfully to interview questions. It was further believed that differences in reporting would be greater among women and those with higher education.
This hypothesis was not confirmed for the reporting of ‘ever had sex’, where there was no significant difference between FTFI and non-interviewer administered tools, and little difference between gender. However, more educated respondents did have a higher pooled estimate for reporting sex in the non-FTFI methods. Moreover, a trend was observed in Asian, urban and cross-sectional studies suggesting an increase for reporting ‘ever had sex’ in the non-FTFI method, although the summary effect did not reach statistical significance. Overall, the results imply that FTFI may not be inferior to innovative interview tools in eliciting this behaviour. One possibility is that having started sexual intercourse was not perceived as particularly stigmatized. A second reason is that respondents had become more comfortable in the presence of an interviewer, or in the case of computerized methods inhibited by the technology.
There was no significant difference by interview tool in the reporting of non-condom use. However, higher levels of non-condom use were reported in non-FTFI studies in South America, and a similar trend was observed for studies in Asia, urban areas, those sampled through RCTs, and, unexpectedly, among respondents with <60% education. As mentioned previously, the variability of the results may be attributed to the impact of stigma in (religious) communities where condom use is associated with risky behaviour and therefore it would be ‘desirable’ to deny condom use. The time frame of the question seems to have been important, with questions using a shorter recall period such as ‘condom use at last sex’ resulting in increased reporting of non-condom use.
There was a significant increase in the reporting of forced sex in the non-interviewer administered tools. Rural and African studies as well as those less educated reported more forced sex by non-FTFI, although the effect was not significant. Subgroup analysis for number of partners also showed that studies in Africa and rural areas were higher in non-FTFI. However, participants with a secondary education and asking the question within a recent time frame showed an increase in the reporting of higher number of sexual partners in the non-interviewer administered tools.
There are limitations to the results of the meta-analysis. First, not all of the studies identified asked about every behaviour; for example, those sampling adult sex workers did not ask if the respondents had ever had sex. Second, question phrasing did vary across studies, for example, the time frame for non-condom use and the number of sexual partners. A third reason points to bias that could not be attenuated even with increased privacy and anonymity. For example, Hewett et al. suggested that STI-positive participants were more likely than STI-negative participants to misreport their behaviour in the FTFI.18 Finally, the response rate was not available for all studies, so it was difficult to know if it was a factor influencing the performance of the different methods.36
Any interpretation of these results is based on two sets of assumptions. First, as outlined above, no self-report tool can be proven to be more accurate than another given that no gold standard exists with which to compare alternative methods. Second, many assumptions are made about people’s social desires when answering questions that may not always go in the direction expected.
In further developing methods for collecting HIV and STI risk behaviour data, multiple methods assessing self-reported behavioural risks should be complemented with the appropriate biological markers of sexual activity, as appropriate depending on the type of study. Although these tests may provide evidence of the validity of self-reported behaviours, their application is limited because the exposure period captured by a biomarker may not be the relevant exposure period for the study. Moreover, infection probabilities are moderated by other factors irrelevant to the particular risk behaviour, including prevalence in the general population, partner status, biological susceptibility of the respondents, and availability of STI testing and treatment.18 Only a few studies have attempted to validate the reporting of behaviour with STI biological markers within an interview mode experiment in a resource-poor setting.18,20,23
In conclusion, the findings of the review have important implications for research design and data interpretation. The results of this review show that the relationship and success of novel interviewing methods has proved complex in a low- and middle-income country context. The results suggest that strongly stigmatized behaviour such as forced sex was significantly more likely to be reported in a non-FTFI setting, whereas other outcomes such as ‘ever had sex’ and non-condom use did not vary by interview mode. Further research efforts to understand factors affecting the degree of measurement error obtained with different interview methods are required, as different tools may generate very different conclusions about what should constitute essential elements of HIV prevention programmes.
Funding
Medical Research Council, UK, to A.E.P. and G.B.G.
Acknowledgements
The authors would like to thank the following people for their contribution in providing raw data for the analysis: Barbara Mensch, Paul Hewett, Joanna Rankin and Barbara Miller at the Population Council; Simon Gregson and Ide Cremin from Imperial College; Mary Plummer; Erica Edwards, Le Cu Linh; and Eduard Sanders. The authors would also like to acknowledge Isaac Fung for helping to access the Chinese-language paper34 and also for his help in translating the results into English.
Conflict of interest: None declared.
