Identifying Promising Themes for Adolescent Vaping Warnings: A National Experiment

Abstract

Introduction

Adolescent vaping remains a problem in the United States, yet little is known about what health warning themes most discourage vaping among adolescents. We sought to identify the most compelling themes for vaping warnings for US adolescents.

Methods

Participants were a national probability sample of 623 US adolescents aged 13–17 years, recruited in the summer of 2020. Adolescents were randomized to one of the five warning message themes about the potential health effects of vaping: 1. chemical harms, 2. lung harms, 3. Coronavirus Disease 2019 (COVID-19) harms, 4. nicotine addiction, or 5. control (messages about vape litter). The primary outcome was perceived message effectiveness (PME; 3-item scale). Secondary outcomes were negative affect (fear), attention, anticipated social interactions, and message novelty.

Results

Adolescents rated the chemical, lung, and COVID-19 harms warning messages higher on PME than nicotine addiction and control (all p < .05), while nicotine addiction was rated higher than control (p < .05). The chemical, lung, and COVID-19 harms warning themes also elicited greater negative affect than nicotine addiction and control (all p < .05). For all other secondary outcomes, the COVID-19 harms warning message theme was rated higher than nicotine addiction and control (all p < .05).

Conclusion

Adolescents perceived warning message themes about lung, chemical and COVID-19 health effects of vaping as more effective than nicotine addiction. To discourage vaping, the FDA and others should communicate to youth about the health effects of vaping beyond nicotine addiction.

Implications

Adolescents rated warning message themes about the lung, chemical, and COVID-19 health effects of vaping as more effective than nicotine addiction, while nicotine addiction was rated as more effective than control themes about vaping litter. To discourage vaping among adolescents, health messaging should expand message themes to communicate about a broader set of health effects of vaping beyond nicotine addiction.

Introduction

Although vaping among youth in the United States has declined in recent years, national estimates suggest one in ten high schoolers currently vape,¹ and vaping frequency among current youth vapers remains high.² Vaping can pose a threat to adolescent health because the e-liquids from many vaping devices contain nicotine,³ which increases the risk of nicotine addiction.⁴ Another concern is that vaping devices may expose users to toxic chemicals (eg, formaldehyde, acrolein) that have the potential for respiratory harm.^5,6 Lastly, some studies have suggested that vaping may lead to other harmful substance use behaviors, including combustible cigarette smoking and alcohol use;^7–9 though, the evidence from these investigations is limited by potential confounders.

Communicating about the harms of vaping—through product warnings and communication campaigns—should be a key part of a comprehensive strategy to prevent vaping among adolescents, particularly those who may be susceptible to or current e-cigarette users.^10–12 Recent data from the US National Youth Tobacco Survey found that adolescents who had ever used e-cigarettes and who self-reported high exposure to vaping warnings were more likely to perceive even occasional use of e-cigarettes as harmful.¹³ In addition, among those who had never vaped, exposure to vaping warning messages reduced adolescents’ likelihood of choosing e-cigarette products in a discrete choice experiment.¹⁴ Currently, the Food and Drug Administration (FDA) requires only a single warning on commercial vaping products sold in the United States, which focuses on how e-cigarette products contain nicotine, an addictive chemical.¹⁵ While these warnings may benefit adult smokers seeking alternatives to traditional cigarettes, warnings about addiction may have a limited impact on youth, as vaping-related harms and hazards (eg, chemical exposure, respiratory damage) may be more effective themes to discourage vaping compared to warning themes about nicotine.^16–18

In addition, studies have shown that exposure to the FDA’s The Real Cost vaping prevention advertisements increases negative attitudes and risk beliefs about vaping among youth.¹⁹ While The Real Cost vaping prevention campaign has focused on both addiction and the health harms of vaping, little is known about the relative efficacy of these and other themes in the context of adolescent vaping prevention campaigns.

Finally, the ongoing COVID-19 pandemic may present an opportunity to develop impactful and timely adolescent vaping warning messages. For example, it is possible that some of the respiratory effects associated with vaping^20,21 could result in more severe cases of COVID-19 among adolescents who vape. For instance, a recent study found that ever use of e-cigarettes and dual use of cigarettes and e-cigarettes may increase adolescents’ likelihood of contracting COVID-19 and developing symptoms,²² respectively, although the findings from this study have since been questioned.^23–26 While the relationship between vaping and COVID-19 currently remains unclear, it is useful to test messages about vaping and COVID-19 while the science on this issue develops.²⁷

To date, most of the literature about the effects of vaping warning messages and themes has come from adult samples; however, evidence suggests that adolescents may process tobacco-related warning messages differently than adults.²⁸ In addition, adolescence is characterized by significant cognitive and social developments, which can increase youth susceptibility to risky behaviors such as tobacco product use.²⁹ The current study aimed to experimentally test warning themes about vaping among a national probability sample of US adolescents (aged 13–17 years). We sought to examine receptivity to different warning themes to inform the development of effective vaping prevention messages.

Methods

Participants and Procedure

Participants were a national probability sample of US adolescents (ages 13–17 years) recruited in September and October of 2020 from the AmeriSpeak panel, a probability-based panel maintained by the National Opinion Research Center (NORC) at the University of Chicago in the United States. NORC randomly selected US households using area probability and address-based sampling, with a known, non-zero probability of selection from the NORC National Sample Frame. For the current study, adolescents were drawn from AmeriSpeak panel households. To address panel attrition due to the COVID-19 pandemic, NORC also invited adolescents ages 13–17 years living in AmeriSpeak panel households who had not yet joined the teen panel to take part in the study. In total, 1 351 households had age-eligible children and received information about the study. Parents from 1 002 households (74% of those eligible) provided informed consent, and 624 adolescents assented and completed the survey (62% of households whose parents consented; 46% of all eligible households). One participant had extensive missing data and was excluded from analyses, resulting in N = 623.

The survey assessed vaping, combustible cigarettes, and other tobacco product use (eg, little cigars, hookah). For vaping and other tobacco product use questions, adolescents were shown images of products for reference. For this experiment, we randomly assigned adolescents to one of the five warning message themes: 1. control, 2. nicotine addiction, 3. chemical harms, 4. lung harms, or 5. COVID-19 harms. Each theme condition contained three text-based warning messages about vaping in white text on a black background (see Table 1 for the verbatim text used in the warning message stimuli and the supplemental document for images of the stimuli that were shown to participants).

Table 1.

Vaping Warning Messages Used in the Experiment.

Control	Nicotine addiction	Chemical harms	Lung harms	COVID-19 harms
Vape litter requires cleanup. Discard vape refills properly.	Vapes contain nicotine.	Vaping can expose you to formaldehyde.	Vaping causes coughing and wheezing.	Vaping increases your chances of COVID-19 infection.
Please refrain from littering vape refills.	Vapes contain nicotine. Nicotine is an addictive substance.	Vape liquids contain harmful chemicals.	Vaping damages your lungs.	Vaping makes you more likely to develop a severe case of COVID-19.
Vapes don’t biodegrade. Please do not litter.	Nicotine is an addictive substance	Vaping can expose you to acrolein.	Vaping causes asthma problems.	Vaping increases your risk of being hospitalized for COVID-19.

Control messages were about the environmental effects of disposable vaping devices and vaping-related litter,^30,31 and were informed by previous studies.^17,32 The warning messages about nicotine addiction, chemical harms, and lung harms were informed by the landmark 2018 National Academies of Sciences, Engineering, and Medicine (NASEM) report on e-cigarettes,⁵ as well as more recent studies.^33–37 The COVID-19 warning messages were based on emerging research about the association between the use of vaping products and respiratory illness, including COVID-19^20–22; though, evidence about a potential link between vaping and COVID-19 is still developing as acknowledged above.

We opted to use causal language in the framing of the text of the warning messages (e.g., “vaping causes coughing and wheezing”) to increase comparability across message themes and to enhance understanding of the warning text.^38–40 While some of the statements go beyond the evidence from the 2018 NASEM report, we justify this decision based on the fact that this experiment is a test of the relative efficacy of warning themes rather than specific product warnings being proposed for implementation on packaging and advertising. After exposure to all three warnings, participants filled out assessments of perceived message effectiveness (PME) (primary outcome), negative affect, attention, social interactions, and message novelty (secondary outcomes). At the end of the survey, adolescents were presented with a debrief message directing them to The Real Cost FDA website with up-to-date information about the harms of vaping and smoking.

Participants in the current experiment were part of a larger adolescent vaping prevention study.⁴¹ Briefly, prior to the current experiment, the larger study had adolescent participants view two tobacco prevention video ads and answer items about the ads. Participants received an incentive equivalent to $12 for completing the study. The University of North Carolina Institutional Review Board approved this study.

Measures

Vaping Status

The survey assessed whether youth had vaped in the past 30 days, and those who had were classified as a current user. If they had vaped before, but not in the past 30 days, we assessed whether they thought they would vape in the future, on a 4-point scale ranging from definitely not (a score of 1) to definitely yes (a score of 4).⁴² If they answered anything other than “definitely not,” we classified them as at-risk of vaping. For youth who had never vaped, the survey assessed whether they had ever been curious about vaping,⁴³ and also if they thought they would vape in the future.⁴² If they answered anything other than “definitely not” to both questions, we classified them as at-risk of vaping. We classified all other adolescents as not-at-risk of vaping.

Current Other Tobacco Product Use

The survey assessed current cigarette smoking by asking adolescents if they had smoked a cigarette in the past 30 days.⁴⁴ The survey also assessed the current use of other tobacco products (OTPs) by having adolescents select other tobacco products that they had used in the past 30 days. OTPs in this list were 1. traditional cigars, 2. cigarillos, filtered cigars, or little cigars, 3. pipe filled with tobacco, 4. hookah and 5. smokeless tobacco.⁴⁴

Perceived Message Effectiveness

We selected PME as our primary outcome because it is sensitive to differences among messages and predicts behavior change.^45,46 The survey assessed PME using a 3-item validated scale.^46,47 After exposure to all three warning messages, the survey asked participants how much the messages: 1. made vaping seem unpleasant to them, 2. made them concerned about the health effects of vaping, and 3. discouraged them from wanting to vape. The 5-point response scale ranged from “not at all” (coded as 1) to “a great deal” (5) (see Supplementary Table 2 for more information about how we assessed PME). PME items were averaged together. The reliability of the scale was high, α = .95.

Secondary Outcomes

The survey assessed three constructs (negative affect, attention, and anticipated social interactions) from the Tobacco Warning Model,⁴⁸ an empirically-driven model demonstrating how tobacco warnings influence tobacco use behavior. Negative affect was assessed by asking participants how much the warning messages made them feel scared¹²; attention was assessed by asking participants how much the warning messages grabbed their attention¹²; and anticipated social interactions were assessed by asking participants how likely they would be to talk about the messages with others in the next week.⁴⁹ In addition, studies have shown low-to-moderate awareness of some vaping-related harms.¹⁶ As such, we assessed message novelty by asking adolescents whether they learned something new from the messages they were shown. The survey utilized 5-point scales (eg, “not at all” [1] to “a great deal” [5]) for responses to all secondary outcomes (see Supplementary Table 2 for more information about how we assessed each secondary outcome).

Demographics

The survey assessed participant age, gender, sexual attraction, race, Hispanic ethnicity, and education of the parent with the most years of schooling. Participants also answered (all that apply) if anyone who they live with now: 1. smokes cigarettes, 2. smokes cigars, cigarillos, or little cigars, 3. uses chewing tobacco, snuff, or dip, 4. uses e-cigarettes, and 5. uses another form of tobacco (eg, hookah or waterpipe, snus).⁵⁰

Data Analysis

We ran one-way ANOVAs to evaluate differences among the warning conditions, using PME and our secondary outcomes (eg, attention, negative affect) as dependent variables. We computed pairwise comparisons for all one-way ANOVAs using Tukey HSD, along with Cohen’s d effect sizes for PME. Exploratory analyses examined whether participant characteristics moderated the effect of warning conditions on PME. Moderating characteristics were vaping status (current vaper/at-risk of vaping vs. not at-risk of vaping), age (12–15 years vs. 16–17years), gender (male vs. female), race (white vs. all others), Hispanic ethnicity, household income (<$50 000 vs. $50 000 or higher), parent education (highest educated parent did not have vs. had a bachelor’s degree), and sexual attraction (only attracted to the opposite gender vs. all others). We computed moderation analyses using separate two-way ANOVAs with interaction terms included in each model (eg, age*PME). Statistical analyses were conducted in R (version 3.6.2)⁵¹ and used two-tailed tests with a critical alpha of .05.

Results

Participant Demographics

The mean age of adolescents was 15 years (Table 2). Most participants reported being female (53%) and white (65%), and about one-fifth (19%) identified as Hispanic. Participants were predominately attracted to the opposite sex only (68%). About half (49%) of adolescents had a parent with a bachelor’s degree or higher. Finally, household income levels varied, with 39% of participants living in households with an income of less than $50 000, 35% between $50 000 and $74 999, and 26% at $100 000 or greater.

Table 2.

Participant Characteristics; N = 623.

	n (%)
Age, M years (SD, range)	15 (1.34, 13–17)
Race
White	404 (65%)
Black or African American	102 (16%)
Asian	23 (4%)
American Indian or Alaskan Native	4 (1%)
Native Hawaiian or other Pacific Islander	1 (<1%)
More than one race	66 (11%)
Other/did not answer	23 (4%)
Hispanic	119 (19%)
Gender
Female	329 (53%)
Male	269 (43%)
Other/nonbinary	17 (3%)
No response	8 (1%)
Sexual attraction
Attracted to the opposite sex only	426 (68%)
All others	197 (32%)
Parent education1
Less than high school	9 (1%)
High school (or equivalent)	73 (12%)
Some college or associate’s	236 (38%)
Bachelor’s degree	155 (25%)
Graduate degree	150 (24%)
Household income
US $0-$24 999	99 (16%)
US $25 000-$49 999	146 (23%)
US $50 000-$74 999	116 (19%)
US $75 000-$99 999	98 (16%)
US $100 000 or greater	164 (26%)
Vaping status
Not at-risk of vaping	241 (39%)
At-risk of vaping	293 (47%)
Current vaper	89 (14%)
Current other tobacco product use
Combustible cigarettes	52 (8%)
Little cigars and cigarillos	40 (6%)
Traditional cigars	22 (4%)
Hookah	21 (3%)
Smokeless tobacco	12 (2%)
Pipe filled with tobacco	4 (1%)
Tobacco product use in the home	222 (36%)

Note. M = Mean; SD = standard deviation;

variable represents the education of the parent in the household with the most years of schooling.

Fourteen percent of adolescents were current vapers, while 47% were at-risk of vaping and 39% were not at-risk of vaping. Current vapers used an e-cigarette, on average, 5.43 days (standard deviation [SD] = 6.71) out of the past 30 days. Current other tobacco product use among the sample included combustible cigarettes (8%), little cigars and cigarillos (6%), traditional cigars (4%), hookah (3%), smokeless tobacco (2%), and pipe filled with tobacco (1%). Finally, more than one-third (36%) of adolescents lived in households with someone else who used tobacco products.

Effects of Vaping Warning Messages on Study Outcomes

Warning message themes varied in their impact on PME, our primary outcome [F(4, 617) = 35.93, p < .001; Table 3]. Post-hoc tests showed that adolescents rated the chemical, lung, and COVID-19 warning message themes higher on PME than both nicotine addiction (ds = .60, .66, and .53, respectively) and control (ds = 1.14, 1.20, and 1.07, respectively; all p < .05), while nicotine addiction was rated higher on PME than control (d = .58; p < .05; Figure 1) (see Supplementary Table 3 for separate ratings of the warning message themes for each of the three PME items).

Table 3.

Impact of Warning Message Themes on Primary and Secondary Outcomes.

Warning message theme	PME	Attention	Negative affect (fear)	Anticipated social interactions	Message novelty
	M (SD)	M (SD)	M (SD)	M (SD)	M (SD)
Control (vape litter)	2.59 (1.44)a	2.42 (1.37)a	2.07 (1.34)a	1.99 (1.22)a	2.82 (1.26)a
Nicotine addiction	3.34 (1.17)b	2.95 (1.32)b	2.46 (1.37)a	2.24 (1.30)a,b	2.28 (1.42)b
Chemical harms	4.00 (1.03)c	3.33 (1.25)b,c	3.09 (1.33)b	2.45 (1.25)b,c	3.29 (1.15)c
Lung harms	4.05 (1.00)c	3.30 (1.25)b,c	3.01 (1.38)b	2.64 (1.28)b,c	2.95 (1.39)a,c
COVID-19 harms	3.93 (1.06)c	3.62 (1.32)c	3.26 (1.42)b	2.78 (1.39)c	3.38 (1.30)c

Note. M = mean; SD =standard deviation; PME = perceived message effectiveness. In each column, cells with different subscripts differ at p < .05 based on Tukey HSD pairwise comparisons; control, n = 116; nicotine addiction, n=127; chemical harms, n = 127; lung harms, n = 132; COVID-19 harms, n = 120.

Figure 1.

PME Differences Across Warning Message Theme.

Warning message themes also varied in their impact on negative affect [F(4, 617) = 15.72, p < .001]. The chemical, lung, and COVID-19 harms warning message themes elicited greater negative affect than nicotine addiction and control (all p < .05), but the nicotine addiction and control themes did not differ.

Further, warning message themes varied in their impact on attention [F(4, 617) = 14.70, p < .001], anticipated social interactions [F(4, 617) = 7.00, p < .001], and message novelty [F(4, 617) = 13.92, p < .001]. For attention, the COVID-19 harms warning message theme was rated higher than both nicotine addiction and control, while the chemical harms, lung harms, and nicotine addiction themes were rated higher than control (all ps < .05). Findings were similar for anticipated social interactions; however, the nicotine addiction warning message theme was not rated higher than the control. For the message novelty outcome, the COVID-19 and chemical harms warning message themes were rated higher than both nicotine addiction and control; and the lung harms and control themes were rated higher than nicotine addiction (all ps < .05).

Moderation Analyses

Parent education moderated the effect of warning message theme on PME [F(4, 612) = 4.47, p = .001]. Adolescents whose highest educated parent had at least a bachelor’s degree rated the COVID-19 warning message theme higher on PME (M = 4.16, SD = .78) than those whose parents did not have a bachelor’s degree (M = 3.74, SD = 1.20; p = .027), whereas adolescents whose highest educated parent did not have a bachelor’s degree rated the control warning message theme higher on PME (M = 2.96, SD = 1.42) than those whose had at least a bachelor’s degree (M = 2.17, SD = 1.34; p = .003). For PME, the warning message theme had no interaction with age, gender, race, Hispanic ethnicity, household income, sexual attraction, or vaping status (all ps > .05).

Discussion

This experiment sought to examine responses to and identify compelling warning message themes about vaping among a national sample of US adolescents (aged 13–17 years). Messages about the chemical, lung and COVID-19 health effects of vaping were rated higher on PME than those about nicotine addiction. In addition, messages about nicotine addiction were rated higher on PME than control messages about vaping litter, suggesting that addiction warnings hold some value in communicating vaping harms to youth. While prior studies have examined responses to warnings and warning themes among adults,^16,17,27 our experiment extends this work to adolescents, a priority population for vaping prevention.

A key finding from this experiment was the superiority of health harm warnings over addiction warnings for adolescents, a finding previously shown only among adults.¹⁷ Currently, nicotine addiction is the most common theme used in vaping prevention messages for youth,⁵² yet our data suggest it is not the most effective. Our findings are in line with work showing that adolescents sometimes discount the seriousness of addiction and often fail to understand the consequences of nicotine addiction.^53,54 While the health harms evidence continues to develop, further efforts are needed to develop effective health harms messages based on the latest scientific evidence. Such messages should then be placed in channels that target youth, or in cases where adult smokers will be exposed (eg, warnings on packaging or advertising), should be pre-tested to avoid any unintended consequences.¹⁷

Among adolescent participants, COVID-19 warning messages were rated higher than nicotine addiction and control messages on PME as well as on attention, negative affect, and social interactions—all key constructs of the Tobacco Warnings Model.⁴⁸ Because a relationship between vaping and COVID-19 is not yet settled science, and studies have shown conflicting findings,^22,55,56 additional work is needed before consideration of disseminating warnings on this topic. Nonetheless, our findings emphasize the importance of communicating with youth about novel health harms of vaping as soon as the research evidence is conclusive.

Notably, PME ratings for the five warning message themes did not substantially differ across several participant characteristics such as sexual attraction, household income, and vaping status—which is important given that a goal of vaping warning messages for adolescents should be to both prevent initiation and encourage cessation among current users. The subgroups analyzed in this experiment largely mirrored results from the overall study sample, with PME ratings from the chemical, lung, and COVID-19 harms warning message themes scoring the highest, followed by warnings message themes about nicotine addiction, and lastly control. The only variable that exhibited an interaction on PME was parent education; though, the significance of this interaction was modest and could reflect multiple testing errors. Overall, these findings suggest that warning messages about the various health effects of vaping may have an impact on adolescents regardless of vaping status and other socioeconomic and demographic factors. This finding builds upon prior studies showing that warning messages tend to work well across diverse populations.^27,32,57,58

Findings from our research also have important policy implications. As previously discussed, the FDA currently requires a single warning about nicotine addiction on e-cigarette packaging and advertisements in the United States.¹⁵ This warning strategy could be strengthened by messaging about additional health harms, such as the chemical exposures and respiratory effects of using e-cigarettes.⁵ It is important, however, that studies with adult smokers evaluate such warnings to ensure that messages do not create inaccurate risk perceptions, and also that they do not discourage adult smokers from completely switching to vaping. Such unintended consequences should be examined given the fact that adult smokers would likely experience high exposure to vaping warnings on e-cigarette products and advertisements. While current evidence does not support the notion that exposure to vaping warnings encourages adult tobacco users to smoke cigarettes,¹⁷ further work on any new warnings being considered for packaging and advertising is warranted.

The warning message themes examined in the current study could also inform vaping prevention campaigns targeted to adolescents. For example, FDA’s The Real Cost vaping prevention campaign has used both addiction and health harm messages, and our findings suggest that a greater emphasis on the health harms of vaping in the campaign would be beneficial. Analyses of federal, state, and local vaping prevention campaigns have shown a variety of themes being used in such campaigns,⁵² and our results bolster the focus of such campaigns on the health effects of vaping.

Strengths of this experiment include a national probability sample of adolescents in the United States, an experimental design that compared several vaping warning themes, the use of a validated multi-item scale to measure the primary outcome, and the inclusion of adolescent current vapers and those at-risk of vaping. Limitations of this experiment include that participants were exposed to the warning messages only a single time and our assessments were only of self-reported message reactions. Future work should test the impact of multiple exposures to messages overtime on outcomes such as susceptibility to vaping and e-cigarette use. In addition, research in this area may wish to use other methods of evaluating adolescent reactions to vaping warning message themes, such as experiments using eye-tracking software. Another limitation is that we assessed outcomes only after exposure to all three warning messages, and thus we can only speak to the impact of the theme but not the individual warnings in each condition. A final limitation was that all adolescents saw two vaping prevention videos ads as part of a separate study prior to being exposed to the warning message stimuli, which may have sensitized participants ahead of this warnings experiment.

Conclusion

Adolescents in the current experiment rated warning messages about the chemical, lung, and COVID-19 health effects of vaping highest on PME, followed by addiction and then control messages. These results can inform both further research and policy—for instance, by systematically developing, implementing, and evaluating messages for e-cigarette prevention beyond nicotine addiction to maximize the ability of such prevention messages to discourage youth vaping. As new warnings are developed, it is important that they reflect the current science on the harms of vaping, which continues to evolve as new evidence emerges.

Supplementary Material

A Contributorship Form detailing each author’s specific involvement with this content, as well as any supplementary data, are available online at https://academic.oup.com/ntr.

Funding

This project was supported by grant number R01CA246600 from the National Cancer Institute and FDA Center for Tobacco Products (CTP). MGH’s time on this project was supported by grant number K01HL147713 from the National Heart, Lung, and Blood Institute of the National Institutes of Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or the Food and Drug Administration.

Declaration of Interest

JAR and JMS have served as paid consultants in government litigation against tobacco companies. SMN has served as a paid expert witness in government litigation against tobacco companies.

Data Availability

The data underlying this article will be shared upon reasonable request to the corresponding author.