Community utilization of risk and protective factor data for prevention planning in Chile and Colombia

Summary

Many effective community health service delivery systems implemented in the USA assess risk and protective factors (RPFs) for youth problem behaviors in a community, and report these data back to local coalitions for prevention planning. This study examined whether community prevention coalitions in Chile and Colombia perceived these reports of RPFs—based on the results of the Communities That Care Youth Survey—to be understandable, valid, useful, and worth disseminating. Thematic content analysis was used to analyze qualitative data collected from 7 focus groups with 75 coalition members. Results indicated heterogeneity between and within coalitions in terms of participants’ understanding of RPFs. However, most participants found reports of RPFs to be easy to understand, thorough, ‘true’ to their communities, and useful for diagnosing and prioritizing needs, action planning, and mobilizing others in their communities. Findings suggest the viability of preventive systems that rely on community-level RPF data, for use in Latin America.

prevention, Communities That Care, risk and protective factors, Colombia, Chile

INTRODUCTION

Substantial evidence exists regarding the role of communities in efforts to effectively prevent youth health and behavior problems (Stith et al., 2006; Catalano et al., 2012; Feinberg, 2012). Community involvement is distinct from preventive health intervention in other domains such as peer groups, families, clinics, and schools, in that communities represent a broad constituency of local actors and, when functional, incorporate a plurality of perspectives. Community involvement in local health initiatives has been related to improved readiness for implementation of health services, more appropriate ‘fit’ of selected interventions for target populations, a higher degree of fidelity to intervention protocols, and greater development of resources that sustain interventions over time (Stith et al., 2006).

The community prevention coalition is one of the primary vehicles for mobilizing communities toward improving health and behavior outcomes. Coalitions provide organizational structure; foster a common culture, language and direction; and focus resources to be more efficient in effecting change (Arthur et al., 2010). Research examining elements of effective community prevention coalitions indicate that a science-based public health approach is one of the most successful mechanisms by which community activation leads to changes in health and behavior (e.g. Brown et al., 2014).

A key element of public health approaches to prevention and promotion is the use of epidemiological data to guide local programing and evaluation of initiatives. With its focus on risk and protective factors (RPFs) that predict and/or modify outcomes, the community diagnosis model (Feinberg, 2012) explicates the use of community-level aggregate data for identifying developmentally oriented and domain-specific points of vulnerability (e.g. family conflict, antisocial peer associations, community disorganization) and selecting appropriate interventions that address these vulnerabilities.

Currently, there are several community-based systems that use an RPF-focused approach to guide the selection, implementation, and/or evaluation of preventive interventions for youth (Chinman et al., 2008; Hunter et al., 2009; Orwin et al., 2012; Osgood et al., 2013; Greenberg et al., 2015). One of the most widely used systems to prevent health and behavior problems in youth is Communities That Care (CTC; see www.communitiesthatcare.net), a prevention service delivery system that incorporates the community diagnosis model. CTC is a manualized intervention that enables communities to address adolescent health and behavior problems, such as substance use and youth violence, by developing local community coalitions and implementing evidence-based preventive interventions that are aligned with local needs (Hawkins and Catalano, 2002; Hawkins et al., 2002). CTC is implemented in five phases. During Phase 1, a community leader who agrees to champion the CTC process invites other community stakeholders to participate, helps identify any readiness issues that need to be resolved, and obtains support from the school district for the administration of the CTC Youth Survey (CTCYS), an epidemiological survey of community youth (Arthur et al., 2002, 2007). In Phase 2, a prevention coalition is formed and its members are trained in science-based prevention. In Phase 3, the coalition uses RPF data collected with the CTCYS to assess youth risks and strengths, identify priority areas for intervention and gaps that need to be addressed, and assess existing community resources. During Phase 4, the prevention coalition creates an action plan to reduce youth risk and strengthen protection, set goals, and select evidence-based interventions that align with their needs. In Phase 5, coalitions implement and evaluate the selected interventions (Hawkins, 1999; Brooke-Weiss et al., 2008).

CTC was developed in the USA, and hundreds of communities in the USA have implemented the system (Feinberg et al., 2010; Oesterle et al., 2018; Chilenski et al., 2019; Fagan et al., 2019) and successfully used CTCYS data to prioritize and develop action plans for improving youth health (Hawkins et al., 2008, 2014; Arthur et al., 2010; Feinberg et al., 2010; Fagan et al., 2019). Other countries such as Australia, Croatia, Germany, and the Netherlands have implemented CTC adaptations (Jonkman et al., 2008; Oesterle et al., 2012; Steketee et al., 2013; Basic, 2015; European Monitoring Centre for Drugs and Drug Addiction, 2017; Groeger-Roth et al., 2018; Toumbourou et al., 2019). Until recently, however, CTC and its youth survey have not been used in Latin American countries. In order for this approach to be viable in the Latin American context, community members must consider epidemiologic RPF data to be useful, acceptable, and interpretable for local decision making, evaluation, and dissemination.

This study examined the use of community reports of RPF data in seven communities from two Latin American countries (Colombia and Chile) that are currently implementing adaptations of the CTC system to prevent drug use and other health and behavioral problems. Prevention of adolescent drug use is a pressing need for both countries, which have alarming rates of substance use compared to other countries in the Americas. For example, data from national surveys (2015) indicate that Chilean secondary students have, by far, the highest rates of tobacco and marijuana use in the Americas (including North, South, Central America, and the Caribbean), with nearly one fourth of secondary students using tobacco during the past month and a third using marijuana during the past year (Inter-American Drug Abuse Control Commission, 2019). Chilean trends in marijuana use have rapidly increased during the past decade, and perceptions of risks of marijuana use are low (Servicio Nacional para la Prevención y Rehabilitación del Consumo de Drogas y Alcohol, 2015). Chile also has the highest rates of cocaine and cocaine base paste use among secondary students in the Americas. In Colombia, adolescent alcohol use is the biggest problem: a 2016 survey indicated that over 40% of secondary school students reported using alcohol in the past month and less than half perceive drinking to intoxication to be of high risk. Although rates of alcohol use have been declining among Colombian school students over the past decade, they are still higher than all but two other countries in the Americas (Inter-American Drug Abuse Control Commission, 2019).

Gathering feedback from stakeholders and community participants is critical for enhancing the pragmatic utility of data (Glasgow and Riley, 2013) and facilitating the implementation and scale-up of evidence-based interventions (Spoth et al., 2013). In this study, we used qualitative data collected from prevention coalition members to assess how local communities interpret youth RPF data, if and how they use it, and what adaptations are needed to improve utilization. Our research questions were: (i) Are reports of RPF data understandable? (ii) Do they seem valid or ‘true’ to local community coalition members? (iii) Are the RPF reports useful? and (iv) How should they be disseminated? The CTCYS has been translated into several languages and is used in several countries, for example, Australia (Toumbourou, 2017), Colombia (Pérez-Gómez et al., 2016), Germany (Groeger-Roth, 2017), the Netherlands (Jonkman et al., 2005), Trinidad and Tobago (Maguire, 2013), Iran (Baheiraei et al., 2016), and Malaysia (Razali and Kliewer, 2015). However, this study fills a gap in the literature by qualitatively studying the perceptions of local community members regarding the interpretability and utility of CTCYS data for prevention planning in countries that do not have a tradition of local decision making.

METHODS

Context and setting

The study was part of an international collaboration between researchers in two Universities in the United States, a nongovernmental organization (NGO) in Colombia, a foundation in Chile, and local community coalitions in Colombia and Chile, working on adapting and piloting CTC for Latin America. Focus group data were collected in four Colombian and three Chilean communities implementing adapted versions of the CTC system, called Comunidades Que Se Cuidan (CQC) in Spanish. The implementation of CQC began in Colombia in 2011 with a pilot in two communities, and expanded to another six sites in the next 2 years (Mejía-Trujillo et al., 2015; Pérez-Gómez et al., 2016). In Chile, the first three communities began implementing CQC in 2014. In each country, a local research and implementation team (a Colombian NGO and a Chilean foundation) worked with researchers with extensive CTC experience, from two US universities, in the translation and adaptation process, receiving technical assistance to preserve fidelity to key elements of the original model.

In each country, trained survey administrators collected youth data using paper-and-pencil versions of the CTCYS (Arthur et al., 2002, 2007) in classroom settings during the regular school day. An abbreviated version of the CTCYS—measuring 10 risk factors, 3 protective factors, and substance use outcomes—was administered to 33 790 students in Colombia in Grades 6 through 11 (the highest level of high school in Colombia) in 68 public schools in the CQC communities. The Chilean version of the survey measured 22 risk factors; 12 protective factors; and substance use, delinquency, and violence outcomes, and was administered to 2191 students in Grades 6 through 12 in 14 public and subsidized schools serving youth from the target communities. The original CTCYS was translated and back-translated to Spanish, and then reviewed and adapted by the local country teams to ensure cultural adequacy. While the Chilean team used the full CTCYS, the Colombian team opted for a shorter version to accommodate the time constraints of their local schools and students, where a longer version was not feasible. Supplementary Appendix S1 includes a list of the RPFs and outcomes measured in each country. Survey findings were reported in profiles of RPFs, as well as substance use and problem behavior charts, by grade level, using the methodology and format described by Arthur et al. and illustrated in Supplementary Appendix S2 Figure S1 (Arthur et al., 2007). In the RPF profiles, bars represent the percentages of youth who were at high risk (or low protection) for each given factor (Arthur et al., 2007). All risk factors (and all protective factors) were presented in a single graph to facilitate the identification of factors that were most (or least) elevated when compared to the rest. The survey reports also included explanations of RPFs, survey measures, instructions on how to read the charts, and guidelines about what communities can do with the information. Trained facilitators shared the reports with community coalitions during a CQC workshop designed to help coalitions learn how to use the data to prioritize RPFs to be targeted with preventive interventions.

Data collection approach

We used focus group qualitative methodology because it allowed community prevention coalition members to express their opinions in their own voice, from their own perspectives and cultures, sharing their personal and local understanding and experience with the CQC system and the RPF data in particular. Research has shown that qualitative methods are useful for understanding cross-cultural transfer of prevention programs (Sundberg et al., 1995) and that focus groups have been successfully used to evaluate cultural appropriateness and to ensure vocabulary is appropriate to target audiences (Linhorst, 2002). The focus group approach was also well fitting because coalition members had experience working together and discussing as a group, and because it highlighted agreements and disagreements among group members, or opinions that were more or less widely spread.

Sample

Focus groups were conducted with CQC coalition members in four Colombian and three Chilean communities (see Supplementary Appendix S3). The Colombian implementation team selected four communities out of a total of eight coalitions operating at the time, based on their availability, allowing the US members of the research team to travel and conduct all focus groups within a limited period. Cost and logistic considerations prevented the researchers from interviewing all coalitions. The coalitions operated in communities that can be described as small, self-contained towns (average population 30 000). One community (LC, population approximately 28 000) was a popular weekend destination located approximately 18 km from the Colombian capital, Bogotá and the other three were located in the coffee-growing region of Quindío. Known for its rich agricultural lands and tourism, Quindío is one of the smallest and most scenic departments in Colombia, with a total population of 543 000 inhabitants, 87% of which live in urban centers and 13% in rural areas (Gobernación del Quindío, 2014). Approximately half the population of Quindío live in its capital city of Armenia. Quindío is made up of 12 different municipalities, 5 of which were designated as sites to implement the CQC system. All four Colombian sites used in this study were characterized as having higher than average rates of unemployment (>19%). Additionally, the four Colombian sites were designated as ‘early alert’ locations prompting high-priority attention from Colombian governmental authorities (i.e. police, child welfare, public health).

In Chile, the focus groups were conducted in all three existing CQC communities, located in the metropolitan region in or near Santiago. Two Chilean coalitions, PL and LE, were formed in small neighborhoods located within a larger, urban municipality. PL is a small residential area composed mostly of three-story public housing buildings in deteriorating condition, housing approximately 600 families. Two known drug trafficking families live in the community. Aside from an elementary school, it has limited access to municipal services and is physically isolated (cut off by a highway) from the rest of the municipality. The municipality that houses this community has recently transitioned mostly from rural to industrial, and has a multidimensional poverty rate of 19%. Multidimensional poverty rate estimates by Berner (2016) and Chile’s Ministry of Social Development (Ministerio de Desarrollo Social de Chile, 2015) include income as well as deficits in education, health, housing, work, and social security.

LE is also a small, very dense urban residential area within a larger municipality; it houses approximately 2200 families. The municipality has a multidimensional poverty rate of 28% (Ministerio de Desarrollo Social de Chile, 2015). LE has many services (schools, churches, healthcare provider, police, etc.) and a history of strong political activism (e.g. it is known for political incidents in the 1960s that ended in the death of several residents).

The third Chilean coalition, SJ, is located in a mountainous suburban municipality in the outskirts of the capital that incorporates both urban and rural areas (population approximately 14,000; 30% rural). The coalition represents the entire municipality, which has a poverty rate of 21% (Ministerio de Desarrollo Social de Chile, 2015). The area includes a hydroelectric plant and is an important tourist attraction, with hiking, river rafting, restaurants, and hotels.

All of the communities in both countries were characterized by high levels of poverty and adolescent drug use. A total of 75 participants (51 females and 24 males) participated in the seven focus groups (see Supplementary Appendix S3). Group sizes ranged from 7 to 17 participants, with an average of 11 and a median of 9 participants per group. Although including 17 participants exceeds the recommended size for a focus group, limited availability of meeting space and constraints regarding travel time in a foreign country prevented us from creating two smaller groups for this coalition. In order to allow all voices to be heard, we decided to not exclude any of the participants that showed up; we accommodated all participants rather than enforcing a position that would be politically uncomfortable for local organizers, and disrespectful to the excluded community members. Researchers were cognizant of the potential effects of such an irregularity and attentive to it while analyzing the data. The decision was made to maintain the data for this group (rather than discard it) because it appeared to corroborate findings from the other six, typically sized groups (which had between 7 and 13 participants). Groups consisted of members of the local CQC coalitions which had been formed as part of the CQC system implementation about a year before the focus groups were conducted; they included representatives from different sectors of the community (e.g. healthcare, education, municipal government, foundations, churches, businesses, police, local organizations, and residents). The local CQC community coordinator was also present. The coordinator is integral to the coalition and is the only paid position in the group. We wanted to keep coalitions as intact as possible, and interview them in their natural settings; therefore, we kept the coordinator in the group but were attentive to the potential influence of his or her role during facilitation and analysis. Age was not asked directly during focus groups, but coalition records indicate that there was a large range, from young adults in their 20s to older community members in their 70s, with the majority between 30 and 50 years of age.

Procedures

Participants were recruited by the country-specific CQC implementation teams and CQC coordinators. Participation in the focus groups was voluntary and participants were given a small thank-you gift worth less than $5. Focus groups were scheduled to occur after community prevention coalitions had received and examined the community reports with the results of their respective surveys, measuring youth health and behavior and RPFs. In Colombia, focus groups were conducted during a 1-year period spanning 2013 to 2014, when communities had completed Phase 4 of the CQC process and had developed their action plans. Due to logistical issues and local availability, in Chile, focus groups were conducted during 2015 while communities were transitioning from CQC Phase 3 to 4, but before they had completed their action plans. Focus groups were conducted in the local communities in one of their regular meeting spaces (e.g. school classroom), and in Spanish, facilitated and moderated by the first author, a native Spanish speaker originally from Chile, with extensive qualitative interviewing experience. The second author—also fluent in Spanish and from Colombian background—observed, took notes, tracked time, and asked follow-up questions when needed. The research team’s language proficiency minimized cross-language barriers between researchers and participants (Squires, 2009). A member of the local implementation team also was present during the Colombian focus groups.

A semi-structured interview protocol (see Supplementary Appendix S4) was used to assess coalition members’ perceptions of the CQC process, and particularly, how they understood the health risk and protective factor data and if/how they had used it. Participants were asked about their experiences with the CQC system (e.g. what they had learned, how they would describe the process) and about their understanding and use of the reports of community RPF data; for example, ease of interpretation of reports, format, compatibility of the report data with their actual knowledge of the community, what other RPFs might be included, what should change, if the reports were informative, how they were used, and their potential for further dissemination. This constituted the basis for the analytical questions. Focus group sessions lasted approximately 1.5 h and were audio-recorded; hand-written notes also were taken. Recordings were transcribed by a native Spanish speaker and reviewed by the facilitator. Transcripts and recordings were stored in password-protected electronic files, and names or personal identifiers were removed from the transcripts to ensure participant privacy. The study protocol was given an exempt status by the University of Washington Institutional Review Board for studies with human subjects.

Data analysis

Data were analyzed with NVivo software version 10 [QSR International (Americas) Inc., 2014] using a conventional thematic content analysis approach (Krueger and Casey, 2000; Stirling, 2001; Hsieh and Shannon, 2005; Braun and Clarke, 2006) where statements of different length in the transcribed text were coded and organized into themes. An iterative process was used during the analysis: all transcripts were read through before the coding process began. After a few transcripts were re-read, initial codes were created; after subsequent transcripts were re-read, codes were revised and earlier interview transcripts were reanalyzed. The interview protocol, which contained questions that addressed the main research aims, also was used to help create the codes and themes as focus group transcripts were analyzed with the analytical questions in mind (Roditis and Halpern-Felsher, 2015). The initial and revised coding performed by the first author was reviewed by the second author (who attended all focus groups and who read and reviewed all transcripts and coding summary documents), and discrepancies were discussed until agreement was found. Quotes from the Spanish-language transcripts were translated into English for publication, with assistance from bilingual colleagues from Chile, Colombia, and Peru to avoid as much as possible loss of meaning during translation (van Nes et al., 2010).

RESULTS

Focus group findings were grouped into four thematic groups, each addressing one of our research questions: (i) Are reports of RPF data understandable? (interpretability); (ii) Do they seem valid or ‘true’ to local community coalition members? (validity); (iii) Are the reports of RPFs useful? (utility); and (iv) How should they be disseminated? (dissemination). Though each community was unique and had slightly different experiences, many of the overall results were very similar across the two countries. Where important cross-country differences were observed, they were noted in the text. Table 1 provides example quotes from focus group participants for each theme.

Table 1:

Sample quotes by thematic area

Theme	Sample quote
Interpretability
Clarity of reports	‘Easy to understand’, ‘clear language’, ‘friendly’.
Demonstrating understanding of survey and RPFs	‘One of the main ingredients has to do with the [youth] survey because that’s […] what you use for planning the work coming ahead…that will give us an account of the risk factors that exist in that particular community, and what will be their potential to work on the protective factors.’
Report features valued	‘We already knew there was [substance] use. But the report let us see the levels, the quantities, that we are exceeding the national percentages; we couldn’t believe that we were so high!’
How reports can be improved	‘We have struggled to understand what ‘community disorganization’ is because [we thought] it was lack of stable organizations, like neighborhood associations, and it includes other factors like trash, lack of adequate street lighting, graffiti, lack of safety, perception of crime.’
Validity
Reports were thorough	‘All the factors measured are important because they cover everything in a person’s life; factors at the individual and parent level, the community, family, and school … So no aspect is left out […] it’s very complete.’
Reports felt true	‘It was not surprising, especially low commitment to school, we see that daily.’ ‘People may not like [what the survey shows] but by all means, that’s the reality here.’
Reports expanded their knowledge	‘I knew of the risk factors, that we’re seeing on a daily basis, but did not know about the protective factors,’ ‘things are clearer now. […] It opened our eyes to do prevention […].’
Questioning report validity	‘With the concept of rebelliousness, you can also consider it something positive, because if you’re in a context where the status quo is horrible, and you want to generate change, then you have to adopt a rebellious attitude.’
What is missing from the reports	‘Unemployment [as a risk factor], because it’s the root of so many things, because when there’s no employment there’s no health, no education, no recreation, the kids are alone.’
Utility
Utility as a diagnostic and prioritization tool	‘It’s good to have a diagnosis. It is the first time we can really see how many are consuming and at what ages. Before we had nothing […] now it’s all consolidated.’ “It’s quantified! We had detected the problems—we live with them daily—but which one is more important, less important, higher incidence […] it allows you to use the [data].’
Utility for action	‘[The report] has been a great source of support. If we decide to start a project or program, if we decide to strengthen what we have or to create something new, things that can help us, the report helps us to decide what we strengthen and what new things we should do.’
Utility for community mobilization and to create awareness	‘Reports […] led us to a higher level of commitment as a coalition; when we observe and understand the risk factors, the graphics, [we think] ‘what can I do to collaborate?’ […] This generates a connection. I think that in the future it will allow others to join with a common interest in serving the community.’
Utility as a ‘door opener’	“Unfortunately, levels [of drug use and risk] are high, but to have it documented has provided us the opportunity to negotiate resources. …Having a formal report helps us to be prioritized at the municipal level. It can open doors for us, to activities and resources.’
Dissemination
Utility of disseminating survey data	‘All of the citizens of [our community] should get to know this diagnosis because when you know how things really are, you realize that it’s everyone’s problem and not an individual’s problem.’ ‘The more it is known, the more the community will feel empowered with what the community board is doing.’
What to disseminate	‘Share the data, the reports […] The language is fine if you want to share with businesses but if you want to share with the community you have to simplify it and make it more user friendly.’
With whom and how to disseminate	‘[Share with] authorities but also residents in the community,’ ‘with schools, parents,’ ‘… with a communications campaign, brochures or such, with a language that’s closer to the people, friendly and easy to read and absorb and that invites people to participate.’ ‘We did some dissemination through the local community radio.’
Potential negative consequences of dissemination	‘I don’t think it’s negative [to share the reports with others] … it’s a positive impact, because we are showing the rest of the community that things can change.’ ‘It can be misused, because for example, when you see in the news ‘X neighborhood has high levels of drug use and trafficking’ then people don’t want to hire anyone from that neighborhood because they’ve seen it on TV.’ ‘There are many delinquents and drug traffickers, and if they’re taking away some of their business […] there will be some fear on their part, they might feel threatened with the actions we’re taking and [us] being empowered.’

Interpretability

When participants were asked directly about the interpretability of the reports, the majority expressed that they were clear, easy to understand, and user friendly. Additionally, we included indirect checks for understanding as a safeguard against a tendency of some participants to provide socially desirable answers. For example, participants were asked how they would explain CQC to other communities and what elements they considered to be its ‘main ingredients’, while probing more specifically about the role of RPFs.

Responses varied considerably. Many participants refrained from providing their own descriptions of CQC, and among those who did, surveys and RPFs were not spontaneously mentioned by the majority. However, each group had people who mentioned the role of the surveys and the importance of addressing RPFs for prevention. In some groups, several participants appeared to be quite knowledgeable and articulate regarding CQC (e.g. naming, describing, or illustrating the RPFs that were elevated in their communities). However, one community appeared to have a less clear vision of the role of RPF assessment in prevention. In this community, very few participants spontaneously brought up the concepts of measuring RPFs, or using data to facilitate decision making.

Among the report features that participants particularly valued were the charts and statistics—being able to see actual levels of drug use and RPFs. They appreciated that the reports were ‘serious’ and ‘high quality’. When exploring what could be improved to make the reports more interpretable, no general themes appeared across all groups, but a few suggestions emerged from individual communities; for example, a community in Chile suggested adapting the language for the risk factor community disorganization to improve the construct’s meaning.

Validity

Most participants felt the reports were thorough, covered all important aspects of young people’s lives, and they would not change what was measured in the survey. In the majority of the groups, participants expressed that the reports felt ‘true’: survey results confirmed what they already saw in their communities. Furthermore, several people felt the survey reports expanded their knowledge by clarifying concepts or systematizing them more clearly.

Despite the overall feeling that the reports were valid, there were a few instances in which participants questioned the validity of the survey results; those themes appeared to be country and community specific and did not generalize across all groups. One Chilean community questioned the accuracy of the survey results on use of cocaine paste (pasta base), which appeared lower than expected based on their local observations. The group suggested that this might be due to the fact that the survey missed school dropouts (who may consume the drug more than school attenders) or people who come to buy the drug in the community but do not live there. In this same community, some participants questioned the validity of youth rebelliousness as a risk factor, given the particular social/political context that surrounds their community and country. From a cultural adaptation standpoint, especially for the generation that grew up during the military dictatorship (including many coalition members), being rebellious was considered by some to be positive, heroic, and necessary to generate needed social change. A similar situation occurred with the term graffiti. In some Chilean communities—with a longstanding tradition of political murals and urban mural art—graffiti had a different, more positive connotation than it might have in the USA.

In yet another Chilean community, participants questioned the validity of survey responses considering that children may not understand certain words in the survey (e.g. names of prescription drugs) or lose interest in responding to a long survey and mark random answers. Further, they suggested that the results might be more ‘believable’ if the survey population included students who were out of the regular school system (e.g. dropouts, gang members), and alternative strategies, such as focus groups, were used with such youth.

Participants were asked what might be missing from the surveys and reports. No general themes emerged across groups, but a few suggestions that were context dependent were made by communities in Colombia: two communities mentioned RPFs they felt were missing, such as unemployment (a big problem in their particular community) and the fact that their community was considered a party town where outsiders came to drink (a particular characteristic of their town), and the need to distinguish between rural and urban context because urban context might be different, more permissive, and dangerous (mentioned by coalition members in a rural area that perceived big difference between their environment and city). They also suggested the benefits of extending survey efforts to private schools (since only public schools were surveyed in Colombia) and parents (not just children).

Utility

Reports were considered to be highly useful, and the most prevalent theme was using reports to inform, diagnose, and prioritize. Coalition members in every community talked about how they obtained important information from the reports that could help them more accurately represent their needs.

Respondents also said data could be used for action. However, this was operationalized differently in the two countries. In two of the Colombian communities, coalition members talked about how the data from the surveys had been useful for developing their community prevention action plans. These communities had used the information to prioritize and implement strategies that could address the risk factors they considered most relevant in their communities. A couple of Colombian communities also felt their data had contributed to shaping the municipality’s policies. In Chile, where communities had not yet developed detailed CQC action plans (because they were in a slightly earlier phase of CQC implementation), the data were useful to help them focus on the next steps.

Survey data also had a mobilizing effect for participants and their communities, generating commitment to the CQC work. A Colombian participant explained that the CQC data allowed them to use ‘a common language’ and get people from different sectors talking about common issues, while a Chilean participant said the data helped people feel responsibility for their community’s problems, and not expect that all solutions come from public agencies. Additionally, in the Colombian—but not in Chilean—communities, several people mentioned that having a report with their community’s data had ‘opened doors’ and helped them mobilize resources.

Dissemination

Most coalition members saw the utility of disseminating reports beyond their coalition because sharing the data could increase knowledge among others in the community. Sharing data could also help create visibility, activate others, and create a sense of shared responsibility. Most Colombian coalitions already had begun disseminating their work and sharing survey results with municipal organizations, which informed municipality’s policies for adolescents. Chilean coalitions, at an earlier CQC stage, had had fewer opportunities for dissemination. Participants were asked what specifically they would share with others. The majority would share the data (e.g. RPF profiles and substance use graphs), though some people saw the value of adapting reports for various audiences (e.g. using a shorter version or simplifying language for different audiences, such as parents, school staff, municipal employees, etc.).

Coalition members explained that they had shared or planned to share survey and RPF data with community residents, families, school staff, institutions (e.g. municipality, police, healthcare), community groups (e.g. neighborhood associations), and businesses. Participants suggested using different media (e.g. radio, internet, social media) to communicate survey findings—as well as other information about coalition activities. One group also discussed strategies they could use to communicate findings to different audiences, such as using games if working with youth.

Although not a widespread concern or theme, one community from each country mentioned that they would like support from the implementing institution specifically for dissemination. For example, one Chilean community with active local drug trade felt they needed help to avoid backlash from drug traffickers in the area, whose business could be affected by a coalition working to prevent drug use.

When probed as to whether they feared that there could be potential negative consequences from sharing the data beyond their coalition, many people said no, but some acknowledged possible adverse effects. Participants in one Chilean community mentioned unnecessary ‘panic’, ‘gossip’, or ‘blaming’; fears of stigmatizing their community; and the possibility that the dissemination of ‘negative data’ on their community might be threatening to politicians, who would ‘look bad’ if this information came to light. Coalition members in two communities (one in each country) with high levels of drug traffic expressed worry that disseminating the survey reports might intimidate the local drug dealers and delinquents, who would feel that the prevention coalition was threatening their interests. Some participants offered suggestions for dealing with these potential negative consequences, such as educating the audience first or rephrasing language to make it appropriate for a broader audience.

DISCUSSION

Results of this study show that the coalition members in Colombia and Chile valued the RPF data reports that were part of the prevention system implemented in their communities. For the most part, they considered the reports to be clear, informative, and comprehensive. The data felt true to their communities and reflected their experiences, while also expanding their understanding. They valued its features and regarded them as highly useful to diagnose the needs of their youth, prioritize and create action plans, while generating awareness, mobilizing people, securing support, and disseminating findings beyond the coalition. Despite the generally positive feedback from coalition members, some suggestions were offered to improve the data’s fit to local needs and perspectives. Examples include refining the terminology used to describe some RPFs to improve cultural relevance, expanding surveys to other types of schools, or adding alternative data collection methods.

There were both commonalities and differences between groups. Every community visited was open to sharing their perspectives, and the overall perception across groups was that the reports were clear and understandable, valid, useful, and worth disseminating. However, there was variation across groups in the particular issues raised or the discussion time given to certain topics. Community coalition members also varied in terms of articulation of their understanding of the CQC process and the role of the RPF data.

We found more similarities than differences between Colombian and Chilean coalitions in cross-country comparisons. One theme came up only in the Colombian communities: utility of reports as a “door opener” to facilitate resources. We hypothesize this might be because the Colombian communities were at a later stage of CQC when the focus groups were conducted. Chilean communities, in contrast, had not finalized their action plans, and coalitions had not yet started to focus on program implementation. Chilean coalitions had not yet had the opportunity to share their reports and assess the impact of their data with others agencies. Another difference is that Chilean communities offered more detailed suggestions for improvement regarding the survey and reports. For example, one of the coalitions had very specific feedback about certain terms used (e.g. ‘community disorganization’, ‘graffiti’); other coalitions had suggestions about survey length and methods. These issues did not come up in any of the Colombian coalitions who used a shorter survey that did not incorporate any of the RPFs that use the specific terms described above.

This study addresses a gap in the research literature by examining a small but crucial part of the CQC system: how local community coalition members perceive, interpret, value, and use reports of RPF data. This is important to examine because CQC, like other community diagnosis models, relies on coalition members’ use of data for decision making, and will work only if coalitions find the data to be interpretable, valid, useful, and are willing to share it more broadly. The study incorporated the voices of actual coalition members and addresses important principles in applied community research and health promotion with multicultural populations, such as participation and relevance, by actively engaging the community in providing feedback, and by using it to ensure that the intervention is relevant to the needs of the community (Frankish et al., 2007; Bermudez Parsai et al., 2011). Using this type of participant feedback to shape future reports of RPF data contributes to the translation of research into real-world settings by helping us deliver ‘scientific knowledge in a manner more explicitly based on the needs, interest, and desires of the people who are ostensibly the end users’ (Page-Reeves et al., 2015, p. 13)].

Limitations

There are several issues to consider as we interpret these findings. First, participants’ understanding of the RPF data likely depends on multiple factors, including the reports themselves and the quality of the training received. In both the CTC and adapted CQC systems, the reports are presented to the coalitions through a training on their interpretation and use. Thus, when we ask about interpretability and utility of the RPF data, we cannot divorce these issues from the format by which the data are conveyed nor from the context of the CQC trainings—which are aspects that we did not observe.

Another potential issue impacting trustworthiness of our data involves social desirability. We were aware before conducting the focus groups that participants might not feel comfortable voicing negative comments or recognizing limitations. Although we did encounter many positive comments, several participants also offered criticism and questioned methods or findings. We also realized that social desirability might influence some to say that they understood the reports more than they actually did. We addressed this by asking directly about participants’ understanding and by probing more indirectly for perceptions. When asked directly, most people said the reports were ‘easy to understand’, yet when probed for comprehension, not all participants could clearly articulate what RPFs are or which were the most prevalent in their community.

Limitations to this study also relate to the restricted generalizability of findings to communities other than the ones represented in this study. The seven sites visited, although geographically and socially diverse, were a convenience sample of communities interested in improving the health and well-being of their youth through prevention. Study communities had agreed to implement the CQC system, consistently attend training workshops, and meet coalition responsibilities. Thus, focus group participants may not represent members of other coalitions who may not have reached the same level of readiness to use the RPF data for local programing of health initiatives. Moreover, communities with insufficient local resources, high levels of economic and political instability, and severe public health and social service needs may not be at a place where the use of epidemiologic data for long-term planning is practicable.

Furthermore, our study may hold a methodological limitation in that more vocal participants may have been “heard” more than quiet ones, despite efforts to elicit opinions from all participants. However, none of the community coordinators had a monopolizing role in the conversation and all gave appropriate space to their fellow coalition members. Further research using individual interviews and coalition member surveys could mitigate this potential in the future.

Despite these limitations, the study findings suggest that the community diagnosis model using RPF data and the work of local prevention coalitions may be a viable approach for preventing community health and behavior problems in Colombia and Chile, and perhaps in similar Latin American communities. This bodes well for prevention systems such as CTC that rely on community-level RPF data for prevention planning and evaluation, and is a positive indicator of success for current efforts to implement the CQC adaptation of CTC. The need for preventive interventions in Latin America is pressing: Colombia has the highest rate of adolescent alcohol use in the Americas, while Chile has the highest level of adolescent use of tobacco and drugs such as marijuana, cocaine, ecstasy, and prescription tranquilizers (Inter-American Drug Abuse Control Commission, 2015). As more prevention programs are adapted or developed indigenously for use in Latin America, there is more need for service delivery systems and approaches that can ensure appropriate selection of interventions, implement them with fidelity, and create local ownership of initiatives for sustained engagement of public health initiatives. Community-based prevention coalitions have been shown to be an effective mechanism to accomplish these ends in upper income North American countries; this study adds evidence that these coalitions could be effective mechanisms in prevention efforts aimed at improving the health and well-being of youth in Latin American countries.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Health Promotion International online.

ACKNOWLEDGMENTS

The authors thank the members of the prevention coalitions in Colombia and Chile who participated in the focus groups.

FUNDING

This work was supported by the US National Institute on Drug Abuse [DA031175]; the Fundación San Carlos de Maipo (Chile); the Corporación Nuevos Rumbos (Colombia); the Colombian Ministry of Health and Social Protection; and the Colombian Institute on Family Welfare. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding agencies.

ETHICS APPROVAL

The study protocol was given an exempt status by the University of Washington Institutional Review Board.