Cancer Misinformation and Harmful Information on Facebook and Other Social Media: A Brief Report

Abstract

There are few data on the quality of cancer treatment information available on social media. Here, we quantify the accuracy of cancer treatment information on social media and its potential for harm. Two cancer experts reviewed 50 of the most popular social media articles on each of the 4 most common cancers. The proportion of misinformation and potential for harm were reported for all 200 articles and their association with the number of social media engagements using a 2-sample Wilcoxon rank-sum test. All statistical tests were 2-sided. Of 200 total articles, 32.5% (n = 65) contained misinformation and 30.5% (n = 61) contained harmful information. Among articles containing misinformation, 76.9% (50 of 65) contained harmful information. The median number of engagements for articles with misinformation was greater than factual articles (median [interquartile range] = 2300 [1200-4700] vs 1600 [819-4700], P = .05). The median number of engagements for articles with harmful information was statistically significantly greater than safe articles (median [interquartile range] = 2300 [1400-4700] vs 1500 [810-4700], P = .007).

The internet is a leading source of health misinformation (1). This is particularly true for social media, where false information spreads faster and more broadly than fact-checked information (2). Health misinformation threatens public health and must be addressed urgently (3-6) because it hinders delivery of evidence-based medicine, negatively affects patient–physician relationships, and can result in increased risk of death (7‐10). Because patients use social media for health information, addressing misinformation has become a critical public health goal (11). This is especially true in cancer care, where use of unproven therapies is associated with decreased survival (10). Here, we quantify the accuracy of cancer treatment information on social media, its potential for harm, and how engagement differs by factualness and harm.

Using web-scraping software (Buzzsumo.com) (12), we searched for the most popular English language articles containing relevant keywords for the 4 most common cancers (breast, prostate, colorectal, and lung). These articles include any news article or blog posted on Facebook, Reddit, Twitter, or Pinterest between January 2018 and December 2019. Each article had a unique link (URL) that allowed for tabulation of engagements, defined as upvotes (Twitter and Pinterest), comments (Reddit and Facebook), and reactions and shares (Facebook). Thus “total engagement” represents aggregate engagement from multiple platforms. The top 50 articles from each cancer type were collected, representing 200 unique articles. The vast majority of engagements were Facebook engagements and therefore analyzed separately.

Two National Comprehensive Cancer Network panel members were selected as content experts from each site (breast: M.M. and J.W.; prostate: T.D. and D.S.; colorectal: S.C. and J.H.; and lung: J.B. and W.A.). Content experts reviewed articles’ primary medical claims and completed 4-question assessments adapted from assessments of factuality and social media credibility (13,14) through an iterative process with L.S., J.T., and A.F. to account for cancer-specific claims (Supplementary Figure 1, available online). Expert reviewers were not compensated.

The experts used two 5-point Likert scales per article (Supplementary Figure 1, available online), and the 2 reviewers’ ratings were summed per article within each domain. Misinformation was defined as summary scores greater than or equal to 6, representing “mixture both true and false,” “mostly false,” and “false” information, and harmful information was defined as any rating by at least 1 reviewer of “probably harmful” and “certainly harmful” information. Inter-rater agreement was evaluated by Cohen kappa (ĸ) coefficient. The proportion of articles classified as misinformation and harmful information was reported, followed by descriptions of why articles were so rated, with multiple selections allowed.

The association of total and Facebook engagements between misinformation and harm was assessed using a 2-sample Wilcoxon rank-sum (Mann-Whitney) test. Statistical analyses were performed using Stata, version 16.1 (StataCorp). All statistical tests were 2-sided, and P less than .05 was considered statistically significant.

Of the 200 articles, 37.5% (n = 75), 41.5% (n = 83), 1.0% (n = 2), 3.0% (n = 6), and 17.0% (n = 34) were from traditional news (online versions of print and/or broadcast media), nontraditional news (digital only), personal blog, crowd-funding site, and medical journals, respectively. Following expert review, 32.5% (n = 65; ĸ = 0.63, 95% confidence interval [CI] = 0.50 to 0.77) contained misinformation, most commonly described as misleading (title not supported by text, statistics/data that did not support the conclusion [28.8%, 111 of 386]), strength of the evidence mischaracterized (weak evidence portrayed as strong or vice versa [27.7%, 107 of 386]), and unproven therapies (not studied, insufficient evidence [26.7%, 103 of 386]).

In total, 30.5% (61 of 200; ĸ = 0.66, 95% CI = 0.52 to 0.80) of articles contained harmful information, described as harmful inaction (could lead to delay or not seeking medical attention for treatable or curable condition [31.0%, 111 of 358]), economic harm (out-of-pocket financial costs associated with treatment or travel [27.7%, 99 of 358]), harmful action (potentially toxic effects of the suggested test or treatment [17.0%, 61 of 358]), or harmful interactions (known or unknown medical interactions with curative therapies [16.2%, 58 of 358]). Select qualitative examples of primary medical claims made for each assessment question are reported in Supplementary Table 1 (available online). Among articles containing misinformation, 76.9% (50 of 65) contained harmful information.

The median number of engagements was 1900 (interquartile range [IQR] = 941-4700), and 96.7% were Facebook engagements. The median engagement for articles with misinformation was greater than factual articles (median [IQR] = 2300 [1200-4700] vs 1600 [819-4700], P = .05) (Figure 1, A;  Table 1). The median engagement for articles with harmful information was statistically significantly greater than for safe articles (median [IQR] = 2300 [1400-4700] vs 1500 [810-4700], P = .007) (Figure 1, B;  Table 1). These findings were consistent for Facebook engagements (Table 1). Reddit and Twitter engagements were statistically significantly associated with misinformation and harm (all P < .05). However, Pinterest engagements were associated with neither misinformation nor harm (all P > .63).

Figure 1.

Box plots showing the association of total online engagements with cancer articles defined as A) factual and misinformation, and B) safe and harmful. P values were calculated using a 2-sided 2-sample Wilcoxon rank sum (Mann-Whitney test). Medians are shown within boxes that represent the interquartile ranges and the error bars that represent the ranges with outside values excluded.

Table 1.

The association between cancer article misinformation, harm, and total engagements and Facebook engagements

Analysis and categories	Articles	Total engagements	Facebook engagements
	No. (%)	Median No. (IQR)	Median No. (IQR)
Total	200 (100)	1900 (941-4700)	1800 (903-4650)
Misinformation analysis
Factual	135 (67.5)	1600 (819-4700)	1500 (769-4700)
Misinformation	65 (32.5)	2300 (1200-4700)	2300 (1200-4600)
Pa		.05	.03
Harm analysis
Safe	139 (69.5)	1500 (810-4700)	1500 (746-4700)
Harmful	61 (30.5)	2300 (1400-4700)	2300 (1400-4600)
Pa		.007	.005

a

Two-sided 2-sample Wilcoxon rank sum (Mann-Whitney) test. Expert reviewers demonstrated substantial agreement on presence of misinformation (ĸ = 0.63; 95% confidence interval = 0.50 to 0.77) and harmful information (ĸ = 0.66; 95% confidence interval = 0.52 to 0.80). IQR = interquartile range.

Between 2018 and 2019, nearly one-third of popular social media cancer articles contained misinformation and 76.9% of these contained harmful information. These data show that cancer information on social media is often inconsistent with expert opinion. This leaves patients in the confusing and uncomfortable position of determining the veracity of online information themselves or by talking to their physician. Most concerning, among the most popular articles on Facebook, articles containing misinformation and harmful information received statistically significantly more online engagement. This could result in a perpetuation of harmful misinformation, particularly within information silos curated for individuals susceptible to this influence.

Limitations of this study are that we included only the most popular English language cancer articles. Furthermore, although BuzzSumo data are obtained directly from social media platforms’ Application Programming Interface (12), there is a small possibility that this dataset is incomplete or that engagements do not match those recorded internally by the platforms. The data lack important qualitative information, but this was determined to be beyond the scope of this report. Lastly, reviewer bias towards conventional cancer treatments is possible; however, questions were structured to avoid stigmatization of nontraditional cancer treatments.

Collectively, these data show one-third of popular cancer articles on social media from 2018 and 2019 contained misinformation, and the majority of these contained the potential for harm. Research on patient discriminatory ability, identification of populations at risk of misinformation adoption, and predictors of belief in online cancer misinformation are underway. Further research is needed to address who is engaging with cancer misinformation; its impact on scientific belief, trust, and decision making; and the role of physician–patient communication in correcting misinformation. These findings could help lay the groundwork for future patient-specific tools and behavioral interventions to counter online cancer misinformation.

Funding

This study was funded, in part, by the Huntsman Cancer Institute. Briony Swire-Thompson’s, PhD effort was funded by an NIH Pathway to Independence Award.

Notes

Role of the funder: The funders did not participate in the design of the study, the collection, analysis, and interpretation of the data; the writing of the manuscript; nor the decision to submit the manuscript for publication.

Disclosures: Dr Johnson and Dr Parsons had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Skyler B. Johnson, MD, has no financial conflicts of interest. Jessica Baumann, MD is on the advisory boards from Pfizer and Kura. Stacey Cohen, MD is on the advisory boards of Kallyope, Acrotech, Natera, receives grant money from NIH P30-CA015704-44 and through institutional research funds including Boston Biomedical/Sumitomo Dainippon Pharma Oncology, Isofol, and Polaris. Tanya Dorff, MD is on the advisory boards of Abbvie, Advanced Accelerator Applications, Bayer, BMS, Exelixis, Janssen, and Seattle Genetics. Joleen Hubbard, MD is on the advisory boards of Bayer and Taiho and receives research funding to her institution from Merck, Boston biomedical, Treos Bio, Taiho, Senhwa pharmaceuticals, Bayer, Incyte, TriOncology, Seattle Genetics, and Hutchinson MediPharma. Daniel Spratt, MD receives personal fees from Janssen, Bayer, AstraZeneca, and Blue Earth, and funding from Janssen. All other researchers reported no financial conflicts of interests.

Author contributions: SBJ, MP, BST, LDS, JT, and AF conceptualized the study and methodology. JT and AF provided expertise, feedback and supervision. SBJ and MP verified research outputs, conducted formal analysis and maintained the research data. SBJ, TD, MSM, JHW, SAC, WA, JB, JH and DES conducted research and investigations. Writing of the original draft was conducted by SBJ, MP, CLB, BST, TO, LDS, JT, AF and review and editing was conducted by SBJ, MP, TD, MSM, JHW, SAC, WA, JB, JH, DES, CLB, BST, TO, LDS, JT and AF.

Data Availability

The data underlying this article are available online and the datasets were derived from sources in the public domain: buzzumo.com.