Abstract
Background. Crowdsourcing, the process of shifting individual tasks to a large group, may enhance human immunodeficiency virus (HIV) testing interventions. We conducted a noninferiority, randomized controlled trial to compare first-time HIV testing rates among men who have sex with men (MSM) and transgender individuals who received a crowdsourced or a health marketing HIV test promotion video.
Methods. Seven hundred twenty-one MSM and transgender participants (≥16 years old, never before tested for HIV) were recruited through 3 Chinese MSM Web portals and randomly assigned to 1 of 2 videos. The crowdsourced video was developed using an open contest and formal transparent judging while the evidence-based health marketing video was designed by experts. Study objectives were to measure HIV test uptake within 3 weeks of watching either HIV test promotion video and cost per new HIV test and diagnosis.
Results. Overall, 624 of 721 (87%) participants from 31 provinces in 217 Chinese cities completed the study. HIV test uptake was similar between the crowdsourced arm (37% [114/307]) and the health marketing arm (35% [111/317]). The estimated difference between the interventions was 2.1% (95% confidence interval, −5.4% to 9.7%). Among those tested, 31% (69/225) reported a new HIV diagnosis. The crowdsourced intervention cost substantially less than the health marketing intervention per first-time HIV test (US$131 vs US$238 per person) and per new HIV diagnosis (US$415 vs US$799 per person).
Conclusions. Our nationwide study demonstrates that crowdsourcing may be an effective tool for improving HIV testing messaging campaigns and could increase community engagement in health campaigns.
Clinical Trials Registration. NCT02248558.
,
At a 1906 county fair in England, a group of individuals was asked to guess the weight of an ox. The median estimate of the crowd was accurate to within 1% of the actual weight and better than any estimate from agricultural experts. This shows the wisdom of crowds or communities in specific contexts [1]. Crowdsourcing is the process of shifting individual tasks to a large group. It often involves open contests and is enabled through multisectoral partnerships [2, 3]. Crowdsourcing has been used extensively in the private sector [2] and has been championed by the National Institutes of Health in the United States as an effective tool to solicit new ideas in health research [4].
Crowdsourcing may overcome 3 common problems encountered in designing and implementing new human immunodeficiency virus (HIV) testing interventions. First, when designing interventions, researchers tend to gravitate toward ideas that resemble previous work, resulting in less creative ideas [5, 6]. Second, when community-based HIV testing interventions are implemented, the input of key affected populations is often relatively limited [7, 8]. Third, aside from HIV testing programs arising from community-based participatory research [9], many HIV testing campaigns result from a top-down, expert-driven process [10]. In contrast, crowdsourcing draws on the collective knowledge of the community instead of experts, empowering communities to develop novel and creative solutions.
The World Health Organization has recognized community engagement as a key social enabler for scaling up HIV testing services and eliminating new HIV infections [8], and HIV community engagement has been associated with increased HIV test uptake [11], effective task-shifting [12], and expansion of HIV treatment services [12–14]. Community engagement is increasingly important as key populations bear a greater burden of the HIV epidemic, now accounting for more than half of all new HIV infections in many countries [8]. Crowdsourcing could be a powerful tool to enhance HIV testing campaigns by harnessing the power of community engagement to generate creative, new ideas to promote HIV testing.
Our research group organized a creative contributory contest [15] to solicit videos promoting HIV testing. The contest included an open call to the public, transparent judging, and a showcase of top videos. To compare the effectiveness of a crowdsourced intervention vs a health marketing intervention to promote first-time HIV testing among men who have sex with men (MSM) and transgender individuals in China, we conducted a noninferiority pragmatic randomized controlled trial (RCT).
METHODS
Intervention Development
The crowdsourced intervention has been described in detail elsewhere [15]. Development of the intervention for this trial included the following steps. First, we posted a public call for videos promoting HIV testing and hosted a call to increase awareness of the contest. Second, a group of multisectoral judges, including researchers, community health leaders, public health and marketing experts, and business leaders, evaluated each of the video entries with a score of 1 (worst) to 10 (best). The judges identified a single contest winner based on the capacity to reach untested individuals, generating excitement, and community responsiveness. Finally, the winning video was included in the intervention arm of this trial (Supplementary Data A). The 1-minute video showed 2 Chinese men falling in love and getting tested for HIV together. The health marketing video intervention was developed independently from the contest by a small marketing company with the guidance of a municipal public health bureau (Supplementary Data B). The 1-minute health marketing video included a cartoon providing HIV education and promoting HIV testing.
Study Design and Participants
We chose a noninferiority design without a control group for 2 reasons. First, there is already substantial evidence demonstrating the effect of brief marketing interventions on HIV testing described in 2 Cochrane systematic reviews [10, 16]. Second, given the need to expand HIV testing among MSM in China [17], there would be ethical concerns associated with withholding an evidence-based intervention from untested MSM [18].
We recruited participants from Chinese MSM Web portals [19]. An MSM Web portal is an online entry point for social networking, finding sex partners, exchanging news, and banner advertising. We selected 1 MSM Web portal each in the northern, southern, and eastern regions of China. These 3 portals collectively have an estimated 90 000 unique users each day. We piloted an online survey to evaluate the effectiveness of the crowdsourced vs the health marketing video among 150 MSM and transgender individuals. Extensive formative work, including interviews with MSM and input from survey design experts and anthropologists, informed the development of the online survey tool (Qualtrics) [20]. We followed standard guidelines for reporting online surveys and RCTs (Supplementary Data C) [21–23].
Participants were recruited through a banner link on the MSM Web portal home pages and an announcement about the study was sent to registered users of the portals. Participants who clicked on the link were directed to eligibility screening and consent [19]. Inclusion criteria were being born biologically male, having had anal sex with a man at least once, at least 16 years of age, never tested for HIV, and able to provide a cell phone number for follow-up. Biologically born males who currently identified as female or transgender were included. Participants who entered the same telephone number more than once to join the survey were excluded.
Randomization and Follow-up
After individuals were screened for eligibility, enrolled, and completed the survey, each study participant was randomly assigned to either watch the crowdsourced video or the health marketing video [24]. Participants, Web portal administrators, and researchers were all masked to group assignment. A text message was sent to participants 3 weeks after survey completion asking about HIV test uptake and test result. An identical second follow-up text message was sent to nonresponders in the fourth week.
Measures
The primary outcome of this study was self-reported first-time HIV testing. We prespecified a self-reported HIV testing outcome because HIV testing in China is largely anonymous [25], HIV self-testing is common [26], and using facility-based capture of testing outcomes would fail to reach the substantial portion of MSM in China who do not seek facility-based services. Secondary outcomes were cost per first-time HIV test and per new HIV diagnosis and change in likelihood of HIV testing (Likert scale rating).
Sociodemographic characteristics collected in the baseline questionnaire included age, education, income, marital status, and sexual orientation disclosure. Participants were asked whether they had engaged in anal intercourse in the last 6 months, and if so, whether they had used a condom during their most recent instance of anal sex. Participants were also asked if they had ever used a gay mobile application to find a sex partner. We collected data on likelihood of HIV testing before and after the participants watched the respective videos (very unlikely, unlikely, likely, and very likely).
We collected cost data for all expenditures from organizations that submitted entries to the crowdsourcing contest—the organization that organized the crowdsourcing contest, Social Entrepreneurship for Sexual Health (SESH Global), the local public health bureau that guided the development of the health marketing video (Chinese Center for Disease Control and Prevention), and the 3 Web portals that provided participant recruitment platforms for our study. Costs included all resources in addition to those resources donated by community-based organizations for all submitted videos, equipment and facilities for both arms (crowdsourced video and health marketing video), and volunteer personnel costs, including judging. Data collected also included costs for the above organizations to coordinate, generate, evaluate, and disseminate videos.
Statistical Analysis
We examined the hypothesis that the crowdsourced video was not inferior to the health marketing video to promote HIV testing. The primary endpoint was difference in proportions having self-reported HIV testing, with a noninferiority margin of −3%. The lower limit of a Wald 2-sided 95% confidence interval (CI) was used to evaluate noninferiority. Effect modification was assessed using a linear probability model [27] based on 3 prespecified subgroups: video watching (first-time vs multitime), Web portal viewed (northern Web portal compared to the other 2 Web portals), and risk behavior (recent condomless anal sex, defined by use of condom during last anal sex), all measured at baseline.
Demographics and sexual behaviors were compared between participants who replied to the postsurvey text message and those who did not. According to the prespecified statistical analysis plan, the primary analysis included individuals who replied to a text message (ie, a complete case analysis). As a sensitivity analysis, multiple imputation was used to impute the missing responses at follow-up. Predictors in the imputation model were age, highest education, province from which the individual accessed the study, study arm, prior exposure to the intervention video (crowdsourced or health marketing), and the Web portal through which an individual accessed the study.
In addition, to test the effectiveness of crowdsourcing vs health marketing video on the change in the likelihood of HIV testing by participants, a Cochran-Armitage trend test was conducted. All individuals were asked how likely they were to test for HIV immediately before and after watching the video. Likelihood of HIV testing was measured on a 4-point numerical Likert scale (very unlikely, somewhat unlikely, somewhat likely, and very likely). For all comparisons, a 2-sided P value <.05 was considered a statistically significant difference. Statistical analyses were conducted using SAS version 10.3 software (SAS Institute, Cary, North Carolina).
For cost data collected, we estimated the total and incremental unit costs of introducing a crowdsourced and a health marketing video to promote HIV testing per first-time HIV tester and per new HIV diagnosis [28]. We first calculated the total cost for each arm (crowdsourced video and health marketing video), then divided these costs by the number of participants newly tested and the number of newly identified HIV cases in each arm to obtain the incremental unit costs. In addition, the ratios and cost savings between the 2 arms were calculated using the health marketing group as the reference group.
Ethical Statement
The study protocol was approved by the institutional review boards of the Guangdong Provincial Centre for Skin Diseases and Sexually Transmitted Infection Control, the University of North Carolina at Chapel Hill, and the University of California, San Francisco (Supplementary Data D). The study was registered with ClinicalTrials.gov [19] (identifier NCT02248558), prior to trial enrollment. A data monitoring and safety board was not established because there were minimal risks associated with the intervention.
RESULTS
Study Participants
Overall, the study link was clicked 5339 times. Of these, 1536 withdrew from the survey prior to reading the consent form and 656 were excluded for not signing the consent form. Among the remaining 3147 clicks, 1328 did not meet eligibility requirements, and 395 duplicates were excluded (by checking recorded phone numbers). A total of 1424 persons completed the online survey, including 721 (51%) participants who had never been tested for HIV. Of these 721 individuals, 352 were randomly assigned to the crowdsourced intervention and 369 to the health marketing intervention (Figure 1).
Study cohort. Duplicates were assessed by mobile phone number. Abbreviation: HIV, human immunodeficiency virus.
Participants accessed the video evaluation survey from 31 provinces in 217 cities (Supplementary Data E). A majority of participants were between 16 and 25 years old (69%), had disclosed their sexual orientation (57%), had a college degree or higher (70%), were never married (85%), and used gay mobile applications to find sex partners (76%). Almost all participants (95%) identified as men and the rest identified as women or transgender. Nearly a third of participants (31%) who had sex in the previous 6 months reported condomless anal sex with their most recent partner. Demographics and behaviors were similar between the 2 randomly assigned study arms (Table 1).
Baseline Characteristics of Study Participants Recruited Into the Randomized Controlled Trial in China, 2014 (N = 721)a
| No. (%) | ||
|---|---|---|
| Characteristic | Crowdsourced (n = 352) | Health Marketing (n = 369) |
| Sex | ||
| Male | 334 (95) | 351 (95) |
| Transgenderb | 18 (5) | 18 (5) |
| Age, y | ||
| 16–20 | 120 (34) | 114 (31) |
| 21–25 | 136 (39) | 130 (35) |
| 26–30 | 50 (14) | 62 (17) |
| 31–35 | 28 (8) | 36 (10) |
| ≥36 | 18 (5) | 27 (7) |
| Highest education | ||
| High school or below | 108 (31) | 105 (29) |
| College | 231 (66) | 239 (65) |
| Graduate education | 13 (4) | 25 (7) |
| Annual income, US$ | ||
| <3000 | 113 (32) | 128 (35) |
| 3000–6000 | 104 (30) | 102 (28) |
| 6001–9500 | 84 (24) | 85 (23) |
| 9501–15 000 | 37 (11) | 38 (10) |
| ≥15 001 | 14 (4) | 16 (4) |
| Marital status | ||
| Never married | 311 (88) | 304 (82) |
| Married/engaged | 31 (9) | 50 (14) |
| Separated/divorced/widowed | 10 (3) | 15 (4) |
| Disclosure of sexual orientationc | ||
| Disclosed to others | 199 (56) | 210 (57) |
| Not disclosed to others | 153 (44) | 159 (43) |
| Ever used gay mobile applicationd | ||
| Yes | 275 (79) | 267 (73) |
| No | 71 (21) | 97 (27) |
| Recent condomless anal sexe | ||
| No recent sex | 47 (15) | 58 (18) |
| Condomless anal sex | 86 (28) | 74 (23) |
| Anal sex with a condom | 174 (57) | 185 (58) |
| No. (%) | ||
|---|---|---|
| Characteristic | Crowdsourced (n = 352) | Health Marketing (n = 369) |
| Sex | ||
| Male | 334 (95) | 351 (95) |
| Transgenderb | 18 (5) | 18 (5) |
| Age, y | ||
| 16–20 | 120 (34) | 114 (31) |
| 21–25 | 136 (39) | 130 (35) |
| 26–30 | 50 (14) | 62 (17) |
| 31–35 | 28 (8) | 36 (10) |
| ≥36 | 18 (5) | 27 (7) |
| Highest education | ||
| High school or below | 108 (31) | 105 (29) |
| College | 231 (66) | 239 (65) |
| Graduate education | 13 (4) | 25 (7) |
| Annual income, US$ | ||
| <3000 | 113 (32) | 128 (35) |
| 3000–6000 | 104 (30) | 102 (28) |
| 6001–9500 | 84 (24) | 85 (23) |
| 9501–15 000 | 37 (11) | 38 (10) |
| ≥15 001 | 14 (4) | 16 (4) |
| Marital status | ||
| Never married | 311 (88) | 304 (82) |
| Married/engaged | 31 (9) | 50 (14) |
| Separated/divorced/widowed | 10 (3) | 15 (4) |
| Disclosure of sexual orientationc | ||
| Disclosed to others | 199 (56) | 210 (57) |
| Not disclosed to others | 153 (44) | 159 (43) |
| Ever used gay mobile applicationd | ||
| Yes | 275 (79) | 267 (73) |
| No | 71 (21) | 97 (27) |
| Recent condomless anal sexe | ||
| No recent sex | 47 (15) | 58 (18) |
| Condomless anal sex | 86 (28) | 74 (23) |
| Anal sex with a condom | 174 (57) | 185 (58) |
a The baseline characteristics are shown for individuals who had never tested for human immunodeficiency virus and were thus eligible for the randomized video intervention.
b Born biologically male and now identify as female or transgender.
c Has told anyone besides sexual partner about their sexual orientation or sexual behaviors.
d Eleven participants were missing data for ever used gay mobile application.
e Only for those who reported condomless anal sex with most recent partner in previous 3 months.
Baseline Characteristics of Study Participants Recruited Into the Randomized Controlled Trial in China, 2014 (N = 721)a
| No. (%) | ||
|---|---|---|
| Characteristic | Crowdsourced (n = 352) | Health Marketing (n = 369) |
| Sex | ||
| Male | 334 (95) | 351 (95) |
| Transgenderb | 18 (5) | 18 (5) |
| Age, y | ||
| 16–20 | 120 (34) | 114 (31) |
| 21–25 | 136 (39) | 130 (35) |
| 26–30 | 50 (14) | 62 (17) |
| 31–35 | 28 (8) | 36 (10) |
| ≥36 | 18 (5) | 27 (7) |
| Highest education | ||
| High school or below | 108 (31) | 105 (29) |
| College | 231 (66) | 239 (65) |
| Graduate education | 13 (4) | 25 (7) |
| Annual income, US$ | ||
| <3000 | 113 (32) | 128 (35) |
| 3000–6000 | 104 (30) | 102 (28) |
| 6001–9500 | 84 (24) | 85 (23) |
| 9501–15 000 | 37 (11) | 38 (10) |
| ≥15 001 | 14 (4) | 16 (4) |
| Marital status | ||
| Never married | 311 (88) | 304 (82) |
| Married/engaged | 31 (9) | 50 (14) |
| Separated/divorced/widowed | 10 (3) | 15 (4) |
| Disclosure of sexual orientationc | ||
| Disclosed to others | 199 (56) | 210 (57) |
| Not disclosed to others | 153 (44) | 159 (43) |
| Ever used gay mobile applicationd | ||
| Yes | 275 (79) | 267 (73) |
| No | 71 (21) | 97 (27) |
| Recent condomless anal sexe | ||
| No recent sex | 47 (15) | 58 (18) |
| Condomless anal sex | 86 (28) | 74 (23) |
| Anal sex with a condom | 174 (57) | 185 (58) |
| No. (%) | ||
|---|---|---|
| Characteristic | Crowdsourced (n = 352) | Health Marketing (n = 369) |
| Sex | ||
| Male | 334 (95) | 351 (95) |
| Transgenderb | 18 (5) | 18 (5) |
| Age, y | ||
| 16–20 | 120 (34) | 114 (31) |
| 21–25 | 136 (39) | 130 (35) |
| 26–30 | 50 (14) | 62 (17) |
| 31–35 | 28 (8) | 36 (10) |
| ≥36 | 18 (5) | 27 (7) |
| Highest education | ||
| High school or below | 108 (31) | 105 (29) |
| College | 231 (66) | 239 (65) |
| Graduate education | 13 (4) | 25 (7) |
| Annual income, US$ | ||
| <3000 | 113 (32) | 128 (35) |
| 3000–6000 | 104 (30) | 102 (28) |
| 6001–9500 | 84 (24) | 85 (23) |
| 9501–15 000 | 37 (11) | 38 (10) |
| ≥15 001 | 14 (4) | 16 (4) |
| Marital status | ||
| Never married | 311 (88) | 304 (82) |
| Married/engaged | 31 (9) | 50 (14) |
| Separated/divorced/widowed | 10 (3) | 15 (4) |
| Disclosure of sexual orientationc | ||
| Disclosed to others | 199 (56) | 210 (57) |
| Not disclosed to others | 153 (44) | 159 (43) |
| Ever used gay mobile applicationd | ||
| Yes | 275 (79) | 267 (73) |
| No | 71 (21) | 97 (27) |
| Recent condomless anal sexe | ||
| No recent sex | 47 (15) | 58 (18) |
| Condomless anal sex | 86 (28) | 74 (23) |
| Anal sex with a condom | 174 (57) | 185 (58) |
a The baseline characteristics are shown for individuals who had never tested for human immunodeficiency virus and were thus eligible for the randomized video intervention.
b Born biologically male and now identify as female or transgender.
c Has told anyone besides sexual partner about their sexual orientation or sexual behaviors.
d Eleven participants were missing data for ever used gay mobile application.
e Only for those who reported condomless anal sex with most recent partner in previous 3 months.
Primary HIV Testing Outcome
Of the 721 participants, 624 (87%) replied to the text message. Response rates were similar between the 2 study arms (crowdsourcing, 87% [307/352]; health marketing, 86% [317/369]). Of the 624 total respondents, 225 (36%) reported having tested for HIV within 4 weeks after watching their assigned HIV test promotion video. In the crowdsourced intervention arm, 114 of 307 (37%) reported testing for HIV compared with 111 of 317 (35%) in the health marketing arm. For the complete case analysis, the estimated difference in proportions between arms was 2.1% (95% CI, −5.4% to 9.7%). Using multiple imputation, the estimated difference in proportions was 3.1% (95% CI, −4.5% to 10.7%) (Table 2). In both analyses, the CI included values below the prespecified noninferiority margin of −3%, so noninferiority was not demonstrated. Of those who tested for HIV, 30.6% reported a positive test.
Noninferiority Analysis of Randomized Controlled Trial in China, 2014 (N = 721)a
| Video | Tested, No. (%) | Difference in Proportions, % | 95% CI, % |
|---|---|---|---|
| Complete-record analysis (n = 624) | |||
| Crowdsourced | 114/307 (37.1) | 2.1 | (−5.4 to 9.7) |
| Health marketing | 111/317 (35.0) | ||
| Multiple imputation analysis (n = 721)b | |||
| Crowdsourced | 132/352 (37.5) | 3.1 | (−4.5 to 10.7) |
| Health marketing | 127/369 (34.4) | ||
| Video | Tested, No. (%) | Difference in Proportions, % | 95% CI, % |
|---|---|---|---|
| Complete-record analysis (n = 624) | |||
| Crowdsourced | 114/307 (37.1) | 2.1 | (−5.4 to 9.7) |
| Health marketing | 111/317 (35.0) | ||
| Multiple imputation analysis (n = 721)b | |||
| Crowdsourced | 132/352 (37.5) | 3.1 | (−4.5 to 10.7) |
| Health marketing | 127/369 (34.4) | ||
Abbreviation: CI, confidence interval.
a Noninferiority analysis assessed the difference in proportions of human immunodeficiency virus testing between crowdsourced and health marketing interventions.
b Ten imputations were conducted to attain the average tested number and percentage.
Noninferiority Analysis of Randomized Controlled Trial in China, 2014 (N = 721)a
| Video | Tested, No. (%) | Difference in Proportions, % | 95% CI, % |
|---|---|---|---|
| Complete-record analysis (n = 624) | |||
| Crowdsourced | 114/307 (37.1) | 2.1 | (−5.4 to 9.7) |
| Health marketing | 111/317 (35.0) | ||
| Multiple imputation analysis (n = 721)b | |||
| Crowdsourced | 132/352 (37.5) | 3.1 | (−4.5 to 10.7) |
| Health marketing | 127/369 (34.4) | ||
| Video | Tested, No. (%) | Difference in Proportions, % | 95% CI, % |
|---|---|---|---|
| Complete-record analysis (n = 624) | |||
| Crowdsourced | 114/307 (37.1) | 2.1 | (−5.4 to 9.7) |
| Health marketing | 111/317 (35.0) | ||
| Multiple imputation analysis (n = 721)b | |||
| Crowdsourced | 132/352 (37.5) | 3.1 | (−4.5 to 10.7) |
| Health marketing | 127/369 (34.4) | ||
Abbreviation: CI, confidence interval.
a Noninferiority analysis assessed the difference in proportions of human immunodeficiency virus testing between crowdsourced and health marketing interventions.
b Ten imputations were conducted to attain the average tested number and percentage.
We assessed the differences in proportions tested between the crowdsourced and health marketing arms using effect modification (Table 3). There was no significant effect modification between interventions for video watching frequency (P = .30), Web portal (P = .20), and recent condomless sex behavior (P = .77) (Table 3). Participants who saw their assigned video more than once were more likely to report HIV test uptake compared with participants who saw the video only once. For example, those who watched the crowdsourced video more than once were more likely to test for HIV compared with those who watched the crowdsourced video only once, with a risk difference of 25.8% (95% CI, 15.0%–36.7%) (the associations between baseline covariates and multitime video watching are presented in Supplementary Data F).
Subanalyses of Crowdsourced and Health Marketing Interventions in Randomized Controlled Trial in China, 2014
| Subgroup | Crowdsourced Tested/Total, No. (%) | Health Marketing Tested/Total, No. (%) | Difference in Proportions, % (95% CI) | P Value for Interactiona |
|---|---|---|---|---|
| Video watching | ||||
| Multitime | 66/126 (52) | 67/151 (44) | 8 (−4 to 20) | .30 |
| First time | 48/181 (27) | 44/166 (27) | 0 (−27 to 27) | … |
| Web portal | ||||
| Northern portal | 106/280 (38) | 90/266 (34) | 4 (−4 to 12) | .20 |
| Other portalsb | 8/27 (30) | 21/51 (41) | −11 (−32 to 11) | … |
| Condomless sexc | ||||
| No recent sex | 36/83 (43) | 33/94 (35) | 8 (−6 to 22) | .77 |
| Condomless anal sex | 28/71 (39) | 26/62 (42) | −3 (−19 to 14) | … |
| Anal sex with a condom | 50/153 (33) | 52/161 (32) | 1 (−10 to 11) | … |
| Subgroup | Crowdsourced Tested/Total, No. (%) | Health Marketing Tested/Total, No. (%) | Difference in Proportions, % (95% CI) | P Value for Interactiona |
|---|---|---|---|---|
| Video watching | ||||
| Multitime | 66/126 (52) | 67/151 (44) | 8 (−4 to 20) | .30 |
| First time | 48/181 (27) | 44/166 (27) | 0 (−27 to 27) | … |
| Web portal | ||||
| Northern portal | 106/280 (38) | 90/266 (34) | 4 (−4 to 12) | .20 |
| Other portalsb | 8/27 (30) | 21/51 (41) | −11 (−32 to 11) | … |
| Condomless sexc | ||||
| No recent sex | 36/83 (43) | 33/94 (35) | 8 (−6 to 22) | .77 |
| Condomless anal sex | 28/71 (39) | 26/62 (42) | −3 (−19 to 14) | … |
| Anal sex with a condom | 50/153 (33) | 52/161 (32) | 1 (−10 to 11) | … |
Abbreviation: CI, confidence interval.
a Wald test.
b Southern and eastern portals combined.
c Reported condomless anal sex with most recent partner in last 6 months, n = 624.
Subanalyses of Crowdsourced and Health Marketing Interventions in Randomized Controlled Trial in China, 2014
| Subgroup | Crowdsourced Tested/Total, No. (%) | Health Marketing Tested/Total, No. (%) | Difference in Proportions, % (95% CI) | P Value for Interactiona |
|---|---|---|---|---|
| Video watching | ||||
| Multitime | 66/126 (52) | 67/151 (44) | 8 (−4 to 20) | .30 |
| First time | 48/181 (27) | 44/166 (27) | 0 (−27 to 27) | … |
| Web portal | ||||
| Northern portal | 106/280 (38) | 90/266 (34) | 4 (−4 to 12) | .20 |
| Other portalsb | 8/27 (30) | 21/51 (41) | −11 (−32 to 11) | … |
| Condomless sexc | ||||
| No recent sex | 36/83 (43) | 33/94 (35) | 8 (−6 to 22) | .77 |
| Condomless anal sex | 28/71 (39) | 26/62 (42) | −3 (−19 to 14) | … |
| Anal sex with a condom | 50/153 (33) | 52/161 (32) | 1 (−10 to 11) | … |
| Subgroup | Crowdsourced Tested/Total, No. (%) | Health Marketing Tested/Total, No. (%) | Difference in Proportions, % (95% CI) | P Value for Interactiona |
|---|---|---|---|---|
| Video watching | ||||
| Multitime | 66/126 (52) | 67/151 (44) | 8 (−4 to 20) | .30 |
| First time | 48/181 (27) | 44/166 (27) | 0 (−27 to 27) | … |
| Web portal | ||||
| Northern portal | 106/280 (38) | 90/266 (34) | 4 (−4 to 12) | .20 |
| Other portalsb | 8/27 (30) | 21/51 (41) | −11 (−32 to 11) | … |
| Condomless sexc | ||||
| No recent sex | 36/83 (43) | 33/94 (35) | 8 (−6 to 22) | .77 |
| Condomless anal sex | 28/71 (39) | 26/62 (42) | −3 (−19 to 14) | … |
| Anal sex with a condom | 50/153 (33) | 52/161 (32) | 1 (−10 to 11) | … |
Abbreviation: CI, confidence interval.
a Wald test.
b Southern and eastern portals combined.
c Reported condomless anal sex with most recent partner in last 6 months, n = 624.
Secondary Outcomes
A Cochran-Armitage trend test for the change in likelihood of HIV testing before and following the respective videos did not detect a significant difference between the 2 interventions.
Table 4 shows the results of the costing analysis, including total and incremental unit cost per person tested and HIV-infected person identified in each group, the ratio between the 2 groups, and the cost saved by the crowdsourced method.
Costing Data Associated With the 2 Interventions in China, 2014a
| Cost | Crowdsourced (n = 352) | Health Marketing (n = 369) | Cost Savingsb | Ratioc |
|---|---|---|---|---|
| Total cost, US$ | 14 926 | 26 358 | 11 432 | 0.56 |
| Follow-up results | ||||
| No. of testers | 114 | 111 | NA | NA |
| No. of HIV-infected cases | 36 | 33 | NA | NA |
| Cost per person tested, US$ | 131 | 238 | 107 | 0.55 |
| Cost per new HIV-infected case identified, US$ | 415 | 799 | 384 | 0.52 |
| Cost | Crowdsourced (n = 352) | Health Marketing (n = 369) | Cost Savingsb | Ratioc |
|---|---|---|---|---|
| Total cost, US$ | 14 926 | 26 358 | 11 432 | 0.56 |
| Follow-up results | ||||
| No. of testers | 114 | 111 | NA | NA |
| No. of HIV-infected cases | 36 | 33 | NA | NA |
| Cost per person tested, US$ | 131 | 238 | 107 | 0.55 |
| Cost per new HIV-infected case identified, US$ | 415 | 799 | 384 | 0.52 |
Abbreviations: HIV, human immunodeficiency virus; NA, not applicant.
a Costing data per person tested and per HIV-positive individual identified by using crowdsourced and health marketing interventions among men who have sex with men and transgender individuals.
b Cost savings was calculated as cost in health marketing arm minus cost in crowdsourcing arm.
c Ratio is defined as the cost of the crowdsourced arm relative to the health marketing arm.
Costing Data Associated With the 2 Interventions in China, 2014a
| Cost | Crowdsourced (n = 352) | Health Marketing (n = 369) | Cost Savingsb | Ratioc |
|---|---|---|---|---|
| Total cost, US$ | 14 926 | 26 358 | 11 432 | 0.56 |
| Follow-up results | ||||
| No. of testers | 114 | 111 | NA | NA |
| No. of HIV-infected cases | 36 | 33 | NA | NA |
| Cost per person tested, US$ | 131 | 238 | 107 | 0.55 |
| Cost per new HIV-infected case identified, US$ | 415 | 799 | 384 | 0.52 |
| Cost | Crowdsourced (n = 352) | Health Marketing (n = 369) | Cost Savingsb | Ratioc |
|---|---|---|---|---|
| Total cost, US$ | 14 926 | 26 358 | 11 432 | 0.56 |
| Follow-up results | ||||
| No. of testers | 114 | 111 | NA | NA |
| No. of HIV-infected cases | 36 | 33 | NA | NA |
| Cost per person tested, US$ | 131 | 238 | 107 | 0.55 |
| Cost per new HIV-infected case identified, US$ | 415 | 799 | 384 | 0.52 |
Abbreviations: HIV, human immunodeficiency virus; NA, not applicant.
a Costing data per person tested and per HIV-positive individual identified by using crowdsourced and health marketing interventions among men who have sex with men and transgender individuals.
b Cost savings was calculated as cost in health marketing arm minus cost in crowdsourcing arm.
c Ratio is defined as the cost of the crowdsourced arm relative to the health marketing arm.
The total cost for the crowdsourced group was $14 926 (all values shown as US dollars), which was lower than the health marketing group ($26 358), with a cost savings of $11 432. The incremental unit cost for promoting HIV testing among MSM and transgender individuals in China was $131 in the crowdsourced group and $238 in the health marketing group (45% reduction) (Table 4). The incremental unit costs per newly diagnosed HIV infection in the crowdsourced group and health marketing group were $415 and $799, respectively (48% reduction).
DISCUSSION
Our nationwide study spanned 31 provinces and 217 cities in China, identifying MSM with sociodemographic and risk behaviors similar to that of a nationwide survey of >40 000 MSM [17]. We found that a crowdsourced HIV testing campaign, despite not meeting the noninferiority hypothesis, was largely successful in promoting first-time HIV testing among MSM and transgender individuals compared with a health marketing campaign. Promoting HIV testing among key populations is a major global health priority [29]. However, most campaigns focused on improving HIV testing among key populations have shown limited capacity to reach hidden populations [30] and have limited community engagement [10, 29]. Our data extend previous research promoting HIV testing among key populations by using a randomized study design, measuring first-time testing, calculating cost, and expediting recruitment using online MSM portals. One of the 3 Web portals recruited 1100 participants in only 72 hours. Our study expands the limited literature on crowdsourcing [31] and is the first randomized clinical trial evaluating crowdsourcing methods to improve a health outcome. Our study suggests that crowdsourcing generates innovative health messages and may increase community engagement [32].
Our study was able to effectively reach a large number of high-risk MSM who had never received an HIV test before and never told their doctor about their sexual orientation. While several previous HIV testing interventions have effectively reached men already engaged in health systems [33], few interventions have focused on or been able to reach the subset of MSM who do not disclose their sexuality. In our sample, almost half of the men (43%) had never disclosed their sexual orientation to anyone (except sex partners), such as a physician or health professional. This highlights the power of the Internet as a tool for reaching subsets of key populations who may not disclose their sexual orientation or seek formal facility-based services. The observation that the Internet may be a useful tool for recruiting high-risk, closeted MSM is consistent with research from Peru [34] and the United States [35].
Our results also suggest that crowdsourcing may be a cost-saving tool for increasing key population engagement in HIV services provision. In our study, the cost of a crowdsourced intervention was approximately half that of the cost of a health marketing intervention ($14 926 vs $26 358). The cost to identify new HIV cases in our study was higher than the cost described in South Africa [36], but lower than costing studies in the United States [37] and Spain [38]. However, our study did not quantify the organizational capacity building accrued to the community-based organizations that submitted videos, data that might be useful in structuring HIV testing programs in the future. As international HIV funding decreases around the world, community-based organizations are increasingly resource constrained and may benefit from the additional capacity building resulting from crowdsourcing to enhance service delivery.
Our study has several limitations. First, both video interventions were brief and relatively simple. Yet other research suggests that the effect of such videos on testing would be observable soon after viewing [16]. We anticipate that such a media intervention would be one component of a comprehensive HIV intervention package. Second, we did not collect testing or biological data to verify text message self-reports. Nevertheless, previous studies have demonstrated text messages to reliably correlate with health outcomes and sexual behaviors [39, 40]. Third, 13% of participants did not respond to our text message and this could introduce bias. However, the responders and nonresponders were similar in sociodemographics and risk behaviors, and the imputation results accounting for nonresponse closely matched the complete case data. Finally, the prespecified noninferiority criteria was not met because self-reported testing rates in both groups were much higher than anticipated, resulting in a wider than planned CI for the difference in proportions HIV tested.
Our results suggest the potential for crowdsourcing to spur creative, new ideas for improving health and engaging communities. Our initial findings and research methods should be further expanded with larger and more powerful studies in the future. In addition, future qualitative studies should further investigate factors influencing our results to aid in adaptation of these approaches to new situations. This new tool may be especially useful in low- and middle-income countries where civil society organizations are often constrained or less able to directly inform public health programs. Crowdsourcing contests may help create more engaging, effective, and creative campaigns.
Notes
Acknowledgments. We thank Dr Ying-Ru Lo, Dr Pengfei Zhao, Dr Peter Vickerman, Dr Kate Mitchell, Ben Cheng, Dr Kate Muessig, Thomas Cai, Dee Poon, Gang Meng, Lloyd Chao, Richard Youngblood, and Kit Hung for their support. We thank all the study participants and staff members at SESH Global, Danlan, Jiangsu Tongzhi, Yunnan Tongzhi and the Guangdong Provincial Center for Skin Diseases and STI Control who contributed. A complete list of acknowledgements is available in Supplementary Data D.
Disclaimer. The listed grant funders played no role in any step of this study.
Financial support. This work was supported by the National Institute of Allergy and Infectious Diseases (NIAID), US National Institutes of Health (1R01AI114310-01 to J. T. and C. W.); University of North Carolina (UNC)–South China STD Research Training Centre (Fogarty International Center grant number 1D43TW009532-01 to J. T.); UNC Center for AIDS Research (NIAID grant number 5P30AI050410-13 to J. T., K. M., and M. H.); University of California, San Francisco Center for AIDS Research (NIAID grant number P30 AI027763 to C. W. and P. V.); National Institute of Mental Health (grant number R00MH093201 to C. W.); and the UNC Chapel Hill, Johns Hopkins University, Morehead School of Medicine, and Tulane University Fogarty Fellowship (FIC grant number R25TW0093 to W. T.). This publication was supported by the National Center for Advancing Translational Sciences at the National Institutes of Health (grant number UL1TR001111).
Potential conflicts of interest. All authors: No reported conflicts. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
References
Author notes
W. T., L. H., and J. B. contributed equally to this work.
Comments