Engagement with fact-checked posts on Reddit

Abstract Contested factual claims shared online are of increasing interest to scholars and the public. Characterizing temporal patterns of sharing and engagement with such information, as well as the effect of sharing associated fact-checks, can help us understand the online political news environment more fully. Here, we investigate differential engagement with fact-checked posts shared online via Reddit from 2016 to 2018. The data comprise ∼29,000 conversations, ∼849,000 users, and ∼9.8 million comments. We classified the veracity of the posts being discussed as true, mixed, or false using three fact-checking organizations. Regardless of veracity, fact-checked posts had larger and longer lasting conversations than claims that were not fact-checked. Among those that were fact-checked, posts rated as false were discussed less and for shorter periods of time than claims that were rated as true. We also observe that fact-checks of posts rated as false tend to happen more quickly than fact-checks of posts rated as true. Finally, we observe that thread deletion and removal are systematically related to the presence of a fact-check and the veracity of the fact-check, but when deletion and removal are combined the differences are minimal. Theoretical and practical implications of the findings are discussed.

contained in the top-level post or to information contained in a subsequent comment. We also removed 45 instances in which the top-level post itself contained a link to a fact checking organization. Although 46 we believe that research into how fact checking organizations themselves are discussed on the site is needed, that question is beyond the scope of this research. We next removed any posts in which the 48 top-level post that contained a link to a fact checking website contained a link to a non-fact check or to 49 a fact check with a rating that did not contain a veracity rating (e.g., a rating of "inconclusive" or "not 50 enough information"). 51 We next endeavored to ensure that the information in the top-level post was not in dispute via 52 conflicting fact-checks. We collected the URL that was contained in the post itself and searched across 53 all posts with a fact check rating for any that were to the same URL. In any case in which a URL was 54 repeated, we checked for disagreements in the fact check ratings and removed all instances in which a 55 disagreement occurred (1.3% of URLs contained disagreeing comments). In total, we identified 10,786 56 top-level posts that had been fact checked directly and were not in dispute. 57 We next took all URLs from the posts with fact check ratings and searched all other posts on Reddit 58 for top-level posts that contained one of the fact checked URLs. We identified 18,472 such posts. We 59 then collected all comments associated with these posts.

60
In total, we identified 29,258 posts that were associated with 10,309 unique URLs. There were 61 14,361 posts that were false, 4,600 that were mixed, and 10,297 that were true that corresponded to 62 5,154 false, 1,635 mixed, and 3,520 true URLs.    Table S1: Top 25 subreddits in number of news stories that were included in the study.
We compare across conversations using the following measures: and when the last comment to that post is made in hours. We note that it is possible that threads 75 will continue to receive comments after our data collection is complete, and so the lifetime of a 76 post may extend beyond the time period of observation. 77 We calculate each of the above measures for each of the conversations in our dataset. Below, we 78 show the summary statistics for each of these measures across veracity for the size of the conversation 79 in Table S2, for the activity of the conversation in Table S3, for the maximum depth of a thread in 80   Table S4, and for the lifetime of the conversation in Table S5. For each measure, we conducted a two-       The above tables show the summary statistics across our core measures of attention to conversations 96 across all conversations. However, it is possible that some of the differences that we observe are due 97 to the presence of the fact check in the conversation. In the below tables, we separately report the 98 summary statistics for conversations that contain a fact check and those that do not. Table S6 shows 99 summary statistics for the size of conversations, Table S7 shows summary statistics for the activity in 100 conversations, Table S8 shows summary statistics for the maximum depth of conversations, and Table S9 101 shows summary statistics for the size of conversations that contain a fact checking comment.
102 Table S10 shows summary statistics for the size of conversations, Table S11 shows summary statis-103 tics for the activity in conversations, Table S12 shows summary statistics for the maximum depth of 104 conversations, and Table S13 shows summary statistics for the size of conversations that do not contain 105 a fact checking comment.
106 Table S14 shows summary statistics for the scores that posts that received a fact-checking comment 107 received. Table S15 shows the summary statistics that comments that linked to a fact check received.