Participant retention in a continental-scale citizen science project increases with the diversity of species detected

Abstract

Sustaining the efforts of volunteers is a challenge facing citizen science programs. Research on volunteer management shows that a diversity of factors may be correlated with sustained volunteerism. In the present article, we explore retention of participants in a large-scale citizen science project. We focus on Project FeederWatch, a bird-monitoring program. Using data from 17,991 participants, we found that the probability of retention increased with the diversity of species (species richness) reported by a participant, but retention was unrelated to the overall abundance of birds reported. Participants who successfully submitted an observation were more likely to remain in the project the following year (82.0% interannual retention) than people who registered but never submitted an observation (39.7%). Two measures of effort were positively correlated with retention. This work provides a case study for examining how demographic information and scientific data collected by participants can be mined to understand volunteer retention in environmental monitoring projects.

Efforts to engage the public in scientific research and biodiversity projects continue to grow in popularity as programs proliferate and public interest increases (Dickinson et al. 2012, Theobald et al. 2015). Involving the public in research can potentially generate scientific knowledge at temporal and spatial scales previously unattainable by scientists alone while providing the participants with learning opportunities (Bonney et al. 2014, McKinley et al. 2017). Citizen science has added to our collective knowledge about disparate topics ranging from changes in species range distributions (Prince and Zuckerberg 2014) to new discoveries about the universe (Lintott et al. 2009). Despite the scientific productivity of engaging the public in research, many project managers find contributions from their participants can be fleeting or sporadic (Raddick et al. 2013, Sauermann and Franzoni 2015). Studies show that for large-scale projects (particularly in the online sphere), engagement is often short term, consisting of limited per capita contributions, with the bulk of the work being performed by a relatively small group of dedicated individuals (Raddick et al. 2013, Sauermann and Franzoni 2015, Crall et al. 2017, Rosenblatt et al. 2022). However, as the participants’ experience within a project grows, so too does data quality (Thiel et al. 2014, Lewandowski and Specht 2015), making participant retention critical to successful citizen science programs.

Sustaining the efforts of volunteers is one of the greatest challenges for citizen science practitioners (Rotman et al. 2014). Understanding factors correlated with participant retention are important for many reasons, including data quality, project sustainability, maximizing the return on investment in the citizen science initiative, participant management, and improving learning by the participants (Beirne and Lambin 2013, Robinson et al. 2021). From the perspective of generating quality scientific results, citizen science initiatives often benefit from retaining participants (Lewandowski and Specht 2015, Parrish et al. 2019). High turnover rates among participants, for instance, can lead to biased trend estimates in programs designed to monitor wildlife populations (Dambly et al. 2020). Retention is also important if a goal is to move participants from simple to more complex tasks (Sauermann and Franzoni 2015). Furthermore, returning users tend to contribute a disproportionately large portion of the data set. An analysis of data submissions to various projects in the online citizen science platform Zooniverse, for instance, recorded that the top 10% of contributors submitted between 71% and 88% of the data (mean [M] = 79%; Sauermann and Franzoni 2015). This skewed distribution highlights the importance of understanding the factors contributing to participant retention, and potential differences between online and hands-on projects.

Understanding factors related to recruitment and retention of participants is important so project managers can follow best practices for sustaining participation (Hart et al. 2022). Retention rates in citizen science projects in the natural sciences realm are poorly understood, but limited data indicate significant variability in retention past the first year (approximately 10%–86%; Crimmins et al. 2014, West and Pateman 2016, Parrish et al. 2019). Strategies that potentially assist in facilitating volunteer retention include providing regular feedback to the participants, understanding and addressing participant motivations, providing opportunities for the participants to interact with one another, and providing room within the project for the participants to grow (West and Pateman 2016, Martin et al. 2016b). Although they are logical, these approaches are rarely tested, and the relative influence of such factors on participant retention remain largely unexplored.

Although participant retention is relatively unexplored in the field of citizen science, the literature on volunteerism may provide insight for practitioners in citizen science (Clary et al. 1998, Finkelstien 2009). This literature suggests that factors influencing sustained, long-term volunteerism fall into three broad groups: organizational, situational, or dispositional (figure 1; Craig-Lees et al. 2008). Organizational factors include the management and treatment of volunteers by the organization, and the commitment of volunteers to that organization (Craig-Lees et al. 2008). Situational factors are those that are likely to affect volunteering for an individual (e.g., the free time that participants have available for volunteerism, the distance they need to travel to volunteer, or access to transportation; Craig-Lees et al. 2008). Dispositional factors are characteristics particular to the individual participant, such as their motivations, beliefs, prosocial or proenvironmental attitudes, and perceived competence (Penner 2002, Craig-Lees et al. 2008). To these three groups, we propose adding a fourth factor, project-level factors, specific to citizen science. Project-level factors include participant effort (e.g., the number of observations previously submitted), data success (i.e., the participant's ability to locate and identify the focus of the project), and the ability to navigate the data entry system, which may contribute to a participant's sense of success.

Figure 1.

The factors that are hypothesized to affect long-term participant engagement in citizen science programs. The factors in bold text are included in this study.

Many data streams already exist within projects with information about participant behavior and the factors correlated with a participants’ continued engagement. In the present article, we use data embedded in the data collection processes of a large-scale citizen science project to explore factors that may be correlated with the sustained contributions of the project participants. Specifically, we focus on interannual retention in Project FeederWatch, a program operating in the United States and Canada that engages the public to observe and count wild birds visiting supplemental feeding stations (bird feeders, Bonter and Greig 2021). We test for relationships between interannual retention and measures of perceived success in achieving the goals of the project (e.g., detecting an abundance of birds or a variety of species). Furthermore, we test for relationships between retention and measures of effort by the participants, as well as demographic covariates. Because people tend to respond positively to increasing bird diversity (Lerman and Warren 2011), we predict that retention rates would increase as the participants detect an increasing abundance and diversity of birds. Because past behavior may be correlated with future behavior (Ouellette and Wood 1998), we predict that retention rates would be greater for longer-term participants and for those who engaged more deeply during the previous observation period. Finally, we discuss the implications of our findings for practitioners who manage citizen science programs.

Project FeederWatch

To examine the influence of participant success and effort on participant retention in citizen science, we examined data from Project FeederWatch (hereafter, FeederWatch; www.feederwatch.org). Since 1987, the Cornell Lab of Ornithology and Birds Canada have operated FeederWatch to engage participants in protocol-driven counts of birds that visit supplemental feeding stations (Bonter 2012, Bonter and Greig 2021). The advantages of using FeederWatch for this research include the project's large geographic scale, the large number of participants (more than 35,000), and the longevity of the project (more than 36 years). Data from FeederWatch are scientifically robust and have been used to examine topics including changes in the distribution and abundance of birds (Bonter and Harvey 2008), climate and range shifts (Zuckerberg et al. 2011, Prince and Zuckerberg 2014, Strong et al. 2015, Greig et al. 2017), invasive species (Bonter et al. 2010, Davis et al. 2013, Koenig et al. 2013), behavior (Miller et al. 2017), disease dynamics (Hochachka and Dhondt 2000, Hartup et al. 2001), and best practices in citizen science (Bonter 2012, Bonter and Cooper 2012).

The FeederWatch season runs from November through April annually, with participants contributing standardized counts as often as once per week during each season. Participants register for the project annually and pay a small registration fee. Individuals retain the same unique observer identification number through time, allowing longitudinal tracking of individual participation, effort, and bird observations. Most of the participants complete an optional site description form on which they report the number and types of bird feeders maintained during the FeederWatch season, as well as the number of months each year in which they actively provide food to birds. This information allows us to quantify effort expended on the hobby of feeding birds.

Statistical methods

Because of differences in project administration between the two countries that could influence interannual retention rates (the organizational factors in figure 1), we limited the analysis to participants based in the United States only. We examined retention from the November 2012–April 2013 (hereafter 2013) season to the November 2013–April 2014 (hereafter 2014) season, during which a participant was retained if the individual registered for the 2013 season and renewed their participation for the 2014 season. We take an expansive view of participation that may include engagement other than submitting bird observations. For instance, the participants may have engaged in sharing photos, downloading educational materials, reviewing the data exploration tools on the FeederWatch web site, or financially supporting the project by renewing their membership. We chose the 2013–2014 time period because of the availability of demographic information about the participants in those years. We developed a series of logistic regression models to test whether the participant returned to the project (binary response variable) as a function of project-level factors including measures of success in the project, previous commitment to FeederWatch, and effort expended toward the hobby of feeding birds, and personal situational factors including observer demographics and location (figure 1).

Our initial logistic regression model was designed to examine the relationship between successful previous data submission (a project-level factor) and retention. This model included the entire set of 17,991 people who registered for the project in 2013 and examined the probability of a participant returning for the 2014 season as a function of whether the participant submitted any data during the 2013 season (binary logistic regression in R; RStudio version 4.1.1). The data set was split into a model creation file (70%, n = 12,593) and a model testing file (30%, n = 5398) to quantify the success rate of the model in predicting the outcomes in the test data file. Tests of model performance indicated that the model fit the data (Hosmer–Lemeshow goodness of fit test, p = 1.00; area under the curve [AUC] = 0.723).

Next, we further examined the relationship between interannual retention and project-level factors, while also considering a suite of situational (demographic and geographic) factors. We ran a second logistic regression where we limited the data set to 8935 participants who submitted at least one checklist in 2013 and for whom we had demographic data. Demographic variables included participant age (range = 20–90, M = 62.4) and gender. Effort and commitment variables included the number of years that the participant had engaged in FeederWatch prior to 2014 (tenure: range = 1–16 years, M = 6.2), and the number of unique FeederWatch counts submitted in the 2013 season (checklists: range = 1–21, M = 11.5). Measures of success in the project included the mean number of species reported per checklist submitted during the 2013 season (species: range = 1.0–39.4, M = 11.8), the average number of individual birds reported per checklist during the 2013 season (total birds: range = 1.0–568.3, M = 52.8), and the proportion of nonnative birds reported per count (nonnative: range = 0–1, M = 0.1). Nonnative species are often considered undesirable and their presence may be discouraged (Bailey et al. 2020, Phillips et al. 2021, Andrade et al. 2022); therefore, hosting a low proportion of nonnative species may be equated with success. The species included in the nonnative category included the four most widespread and commonly reported species that were introduced to North America, the Eurasian collared-dove (Streptopelia decaocto), rock pigeon (Columba livia), European starling (Sturnus vulgaris), and house sparrow (Passer domesticus). We included the latitude and longitude of each count site in the full model to account for potential geographic trends in the probability of retention and the distribution of species across the United States. This data set was split into a model creation file (70%, n = 6221) and a model testing file (30%, n = 2714). Model performance analysis indicated that the model successfully predicted outcomes in the subset of testing data (AUC = 0.821).

Data related to the effort expended on feeding birds were available for a smaller subset of participants (n = 4602). These project-level factors (figure 1) were included in a third logistic regression. These variables included the total number of bird feeders maintained (range = 1–53, M = 7.2), the number of different types of feeders maintained (range = 1–8, M = 3.8, including combinations of hopper, tube, thistle, fruit, nectar, platform, hanging, ground, suet, water feature, or other feeders), and the total number of months per year that the participant fed birds. Because most participants fed in each month of the year, we converted total months fed to a binary variable (fed 12 months versus fed less than 12 months). This third regression model included the variables that were significant in the previous model (species per checklist, total checklists submitted, tenure, and age) as well as the three variables related to bird feeding effort. We again split the data set into a model creation file (70%, n = 3206) and a model testing file (30%, n = 1396). Tests of model performance also indicated that this model successfully predicted outcomes in the testing data set (AUC = 0.801).

Finally, to examine variables that may potentially influence the diversity and abundance of birds reported on a FeederWatch count, we created general linear models with either mean species diversity (species richness) or mean total birds reported as response variables and the total number of feeders maintained, total feeders, the diversity of feeders maintained, whether or not the participant fed birds year-round (binary), tenure in the project, latitude, and longitude as explanatory variables (GLM in R, Gaussian distribution, n = 4602 sites). We confirmed that model residuals were normally distributed.

Findings

Of the 17,991 FeederWatch participants located in the United States during the 2013 season, 9635 (53.6%) submitted at least one checklist. Participant retention between the 2012–2013 and 2013–2014 FeederWatch seasons was 62.4%. The probability of returning to the project was greater for participants who submitted data in the 2013 season (M = .822, 95% confidence interval = .813–.831) than for those who did not submit data (M = .399, 95% confidence interval = .387–0.412, z = 46.69, p < .001).

Several project-level and personal situational factors were correlated with interannual retention. The likelihood of a participant returning to participate in FeederWatch from 2013 to 2014 increased with increased species diversity (the number of species) per checklist (z = 3.61, p < .001). The likelihood of interannual retention increased with increasing effort in the project, including the number of checklists submitted in 2013 (z = 15.91, p < .001) and tenure in the project (z = 15.53, p < .001). Older participants were more likely to return than younger participants (z = 6.55, p < .001), and men and women were equally likely to remain in the project (z = −1.19, p = .233). The abundance of birds reported (average number of birds reported per checklist) was not correlated with retention (z = −0.24, p = .810), nor was the percentage of nonnative birds reported correlated with retention (z = −0.08, p = .936). The retention probabilities did not differ geographically (latitude: z = 0.22, p = .830; longitude: z = −0.31, p = .755). See figure 2 for standardized beta estimates from this model.

Figure 2.

Standardized beta estimates from the logistic regression model testing for the relationship between a suite of variables recorded in 2013 and interannual retention rates for Project FeederWatch from the 2013 to 2014 season. The model included measures of success in the project (the average number of species reported per checklist, the average number of birds reported per checklist, the proportion of birds reported per checklist that were nonnative species), measures of effort in the project (the total number of checklists submitted in 2013, the number of years participating in the project [tenure]), demographic variables (the participant’s age and gender), and location (latitude, longitude). The error bars represent 95% confidence intervals; significant predictors are indicated by the confidence intervals that do not cross zero.

A total of 4602 participants reported the amount of effort expended on feeding wild birds. This effort was not correlated with interannual retention in FeederWatch. Neither the total number of feeders maintained (z = 0.20, p = .842), nor the number of different types of feeders maintained (z = 0.94, p = .347) was correlated with retention. Likewise, whether the participant fed birds year-round (z = −1.91, p = .056) was not correlated with retention. Examining factors correlated with perceived success, however, revealed the importance of effort expended on the hobby. The diversity of species reported on a FeederWatch count increased with the total number of feeders maintained (figure 3a; t = 9.23, p < .001), increased with the diversity of feeder types maintained (figure 3b; t = 6.63, p < .001), increased from west to east (figure 3c; longitude variable, t = 9.16, p < .001), increased from north to south (figure 3d; latitude variable, t = −8.63, p < .001), increased with tenure in the project (figure 3e; t = 7.43, p < .001), and was greater for people who fed during all months of the year (figure 3f; t = 3.12, p = .002).

Figure 3.

The relationship between the mean total number of bird species reported on a checklist and (a) the number of feeders maintained by an observer, (b) the number of different types of feeders maintained, (c) longitude, (d) latitude, (e) the number of years a participant has engaged in the Project FeederWatch program, and (f) whether the participant feeds birds in all 12 months of the year (Y = fed 12 months). The data are predicted values, and the error bars represent 95% confidence intervals from the general linear model.

Factors associated with retention

Our study shows that interannual retention rates of participants in a wildlife-focused citizen science project are positively correlated with participants seeing a diversity of species. Humans tend to respond positively to biological diversity, placing greater value on more diverse plant (Blanchette et al. 2021) and animal (Andrade et al. 2022) systems. Witnessing greater biodiversity is also correlated with increases in human wellbeing (Fuller et al. 2007, Dallimer et al. 2012). In the context of gathering data for a citizen science project, detecting greater diversity may also be correlated with a greater sense of success in the project (project-level factors; figure 1). Although contributing to science may be a strong motivation for initial participation (Dickinson et al. 2012, Robinson et al. 2021), our research suggests that perceived success—however that is defined by the participants within a given project—may play a key role in maintaining that motivation over the long term. For FeederWatch participants, success may be perceived as detecting a diversity of bird species rather than an abundance of individual birds. This pattern, where detecting diversity is more important than detecting large numbers of individuals, is also present in the North American Amphibian Monitoring Program (Marsh and Cosentino 2019). Although we detected no relationship between the types of species detected (native versus nonnative) and participant retention, species composition may matter in other contexts. For instance, surveys reveal strong biases against nonnative species (house sparrows, Passer domesticus) in the context of recording data on nesting birds because the sparrows may kill or displace native species (Phillips et al. 2021). In contrast, nonnative species may be perceived positively in other situations, such as with observations of colorful rosy-faced lovebirds (Agapornis roseicollis) in Arizona (Andrade et al. 2022).

Overall, the 62% interannual retention rate for participants in Project FeederWatch is high in comparison to other large-scale citizen science initiatives. For instance, Nature's Notebook, a program for tracking the phenology of annual events, retains 45%–55% of the participants from year to year (Crimmins et al. 2014). COASST (Coastal Observation and Seabird Survey Team), a project designed to monitor bird mortality on beaches, retains approximately 40% of its participants beyond the first year and reports interannual retention rates for other citizen science projects ranging from approximately 10% to 86% (Parrish et al. 2019). The participant retention rates for online projects are generally lower than field-based projects. In Zooniverse, an online crowd sourcing platform for digitizing data from a wide diversity of subject areas, approximately one quarter of the participants returned for a second session (M = 27%, range = 17%–40%, Sauermann and Franzoni 2015), with a lower percentage of users engaging over the long term. Furthermore, participant retention within a project can be highly variable from year to year, depending on the project (Andow et al. 2016).

Low rates of participant retention can be particularly problematic when drop-out patterns are not random (e.g., related to personal situational factors such as a suboptimal sampling location). For instance, the North American Amphibian Monitoring Program recorded higher rates of attrition in participants who monitored locations with high levels of traffic noise, high traffic volume, and low forest cover (Marsh and Cosentino 2019). These site variables are likely important covariates in models of species abundance or diversity, so biased attrition of participants could affect data interpretation. Because nonrandom patterns of participant attrition can lead to biases in sampling, understanding factors correlated with participant retention is important for using data sets generated by the public.

Our results demonstrate that other project-level factors, including the effort expended by participants (measured through the number of checklists submitted and tenure in the project), were positively correlated with retention. Likewise, other studies have shown that the frequency and duration of volunteer participation can influence ongoing commitment to projects (Ryan et al. 2001). Other personal situational factors, including some demographic variables, were also correlated with the probability of interannual retention in FeederWatch, with older individuals remaining engaged at a higher rate than younger individuals. Few comparable studies exist, but participants over 40 years old are more likely than younger people to continue monitoring beaches for the COASST project (Parrish et al. 2019). Men were marginally more likely to stay engaged in COASST than women (Parrish et al. 2019), whereas the gender variable was not significant in our study.

There was a positive correlation between the types and number of feeders that participants maintained and the diversity of species detected. Although not within the scope of this article, future work examining longitudinal changes in the abilities of participants to detect species would aid in data analyses. Geography and local habitat features are also likely important because of their direct connection to local species diversity. It is likely that participants who live in areas with habitat that supports greater species diversity are also more likely to continue participating in a wildlife-focused citizen science project.

In addition to factors that may affect participant retention discussed in the present article, a suite of other factors including situational, organizational, and dispositional factors likely contribute to participant behavior (figure 1). Situational barriers to retention may include the costs of participation, travel and the logistics of site access, liability concerns, and health or safety concerns (Phillips 2017). Organizational factors, including how project teams communicate with participants may be important (Bonney et al. 2009, van der Wal et al. 2016), as can the use of incentives, recognition, or rewards (Newman et al. 2012). Supporting pathways for social interaction among the participants or between the participants and the project team may encourage retention (Beeden et al. 2014, Cappa et al. 2016). Project managers need to consider the impact of the protocol design on participant recruitment and retention (Wald et al. 2016, Burgess et al. 2017). Furthermore, the increasing reliance on new technologies for data collection means that technological barriers can affect contributions (Newman et al. 2012, Martin et al. 2016a).

Future research on factors associated with long-term engagement in citizen science will help elucidate which project level, situational, organizational, or dispositional factors are important for improving retention rates, and will enable projects to sustain engagement via strategically targeted recruitment and retention efforts. Such research will also be important for interpreting the strengths and weaknesses of different data sets because it may allow researchers to identify and possibly account for biases in sampling that stem from potential biases in participant retention. The ability for projects to respond to and account for these factors has significant implications for the long-term success and impact of the projects (Beirne and Lambin 2013, Thiel et al. 2014). From the perspective of scientific output, the quality of the data submitted by participants tends to increase with their longevity in the project (Parrish et al. 2019). Encouraging the continued engagement of people participating in such projects, therefore, should be a main goal of practitioners managing citizen science initiatives.

Acknowledgments

Special thanks to the thousands of people who participate in Project FeederWatch, making this work possible. Thanks to our colleagues at Birds Canada who manage the program in Canada, to the Project FeederWatch participant support team, and the tech team at the Cornell Lab of Ornithology who maintain world-class data entry platforms and databases. We thank Rachael P. Mady for assistance and review of R scripts. Data and analysis code are freely available in Mendeley Data: https://data.mendeley.com/datasets/87h4hxp33r/1).

Author Biographical

David N. Bonter ([email protected]) is codirector, Emma I. Greig ([email protected]) is the project leader, and Tina B. Phillips ([email protected]) is the assistant director of the Center for Engagement in Science and Nature at the Cornell Lab of Ornithology, at Cornell University, in Ithaca, New York, in the United States. Victoria Y. Martin ([email protected]) is a former postdoctoral research associate at the Cornell Lab of Ornithology and is a current postdoctoral research fellow in the School of Earth and Environmental Sciences at the University of Queensland, in Brisbane, Queensland, Australia.