https://orcid.org/0000-0003-1760-679X
Abstract
In countries with limited civil registration and vital statistics systems, assessing the impact of health crises requires precise retrospective mortality data. We tested whether calendar methods improve survey or census data on dates of recent household deaths registered in a Health and Demographic Surveillance System (HDSS).
Between April and June 2023, we randomized 578 households with HDSS-registered deaths in Guinea-Bissau to interviews by using (i) a standard questionnaire with close-ended questions about dates of deaths, or similar questionnaires supplemented with (ii) a physical calendar of local events printed on paper or (iii) a digital calendar implemented on tablets. We evaluated the accuracy of reported dates through record linkages to HDSS data.
No deaths were reported in 11.8% of the 508 participating households. In other households (n = 448), informants reported 574 deaths since January 2020. Relative to the standard questionnaire, neither the physical calendar nor the digital calendar improved the proportion of deaths reported in the same month and year as recorded by using surveillance data. The physical and digital calendars reduced the share of missing data on dates of deaths (6.1% and 3.2%, respectively, versus 13.1% with the standard questionnaire). Reported dates of deaths obtained by using the digital calendar were more weakly correlated with surveillance data than those collected in other arms. Using the digital calendar also added 1.15 minutes to the data collection.
Digital calendars do not improve the reporting of dates of deaths in surveys or censuses. Further trials of the use of a physical calendar in retrospective interviews about recent household deaths are warranted.
We tested whether using calendars of local events during household surveys and censuses can help ascertain more precisely the reported dates of recent deaths in settings in which civil registration and vital statistics systems are limited.
Neither paper-based nor digital calendars improved the accuracy of reported dates of recent deaths, but paper-based calendars increased data completeness without increasing interview duration.
Further improvements in the collection of data on recent household deaths are needed to better document the impact of crises such as epidemics, conflicts, and natural disasters on population health.
Introduction
Periodic counts of deaths are essential for monitoring population health [1, 2]. In high-income countries, this information is extracted from civil registration and vital statistics (CRVS) systems that operate continuously [3]. It allows the detection of periods of excess mortality, such as those that occurred during the COVID-19 pandemic, in near real time [3–5]. In many low- and middle-income countries (LMICs), CRVS systems are incomplete [6]. Representative mortality data are only generated every few years from retrospective household surveys and/or censuses [7]. Some of these inquiries ask respondents to list all of the deaths that have recently occurred in their household [8] and to indicate the demographic characteristics of the deceased [9]. By comparing the list of reported deaths to the list of current household members, they allow estimations of the sex- and age-specific mortality rates for a reference period that covers one or several years before data collection.
The retrospective mortality data collected during surveys and censuses might also help to better ascertain the magnitude of recent mortality fluctuations in LMICs caused on a month-to-month basis by epidemics, conflicts, or natural disasters [10]. However, not all surveys and censuses that collect data on recent household deaths also ascertain the dates of reported deaths. Furthermore, when questions about dates of events are included in a survey or census, they might generate imprecise data [11].
To improve the reporting of dates of demographic events, surveys and censuses that are conducted in LMICs often use calendars of local events [12]. These tools allow respondents to anchor their answers to events with known dates that have happened in close temporal proximity to the event of interest. Such calendars have referenced local elections, festivities, or weather-related disasters (e.g. droughts, floods). These time markers might help to reduce the proportion of reported events with unknown dates. They might also ensure that the reported date is more consistent with the true date of an event.
Calendars of local events used in surveys and censuses predominantly employ annual scales, e.g. to better assess the age of respondents [13, 14]. Except for a few studies that were conducted in the wake of conflicts or natural disasters [15–17], they have seldom been used to improve the reporting of recent household deaths on a monthly scale. We conducted a multi-arm randomized trial of the use of calendars of local events in interviews about recent household deaths. We hypothesized that, compared with standard close-ended questions about the month and year of death, the use of calendars would yield more accurate data on the dates of recent household deaths.
Methods
Study setting
We worked in Guinea-Bissau—a low-income country in west Africa (Fig. 1). The country has a population of ∼2 million and an estimated life expectancy at birth of 60 years [18]. Leading causes of death include neonatal conditions, lower respiratory infections, HIV/AIDS, and tuberculosis [19]. The COVID-19 pandemic has also spread among various population groups and disrupted daily activities [20, 21].
Map of Guinea-Bissau and the study area. Source: DIVA-GIS (2023) for shape files.
The trial was conducted in five neighborhoods of Bissau, the capital city [22, 23]. These areas are covered by a Health and Demographic Surveillance System (HDSS), which follows >100 000 people [24]. HDSS fieldworkers conduct household visits every 2 months to detect and monitor pregnancies, births, and deaths. Young children are followed up every 3–4 months, from birth until their third birthday, to gather data on vaccinations and other health behaviors [24, 25]. Among the rest of the population, deaths are recorded periodically during censuses, with the most recent taking place between September 2019 and October 2020. Evaluations of health interventions conducted in HDSS neighborhoods also routinely include the collection of mortality data. In 2020–21, fieldworkers recorded deaths among participants in assessments of the role of face masks and Oral Polio Vaccines in limiting the impact of COVID-19 on population health [26, 27].
We used records generated by these HDSS activities as a sampling frame to recruit participants and as a common benchmark against which we compared the data obtained from different versions of questionnaires about recent household deaths. Despite frequent follow-up, some deaths might be missed by HDSS fieldworkers and characteristics of recorded deaths (e.g. dates, ages) might be misreported. We thus acknowledge that HDSS data are not a perfect source of information on recent mortality [28–30] but also believe that, thanks to its features, including shorter recall, it provides a good standard against which to evaluate the trial arms.
Participants
We selected households in which at least one death had occurred among regular household members between January 2020 and May 2022, according to HDSS records. Households were located by using the numbering system established by the HDSS and were visited in person. In each household, a regular member aged 18+ years and who spoke Crioulo (the vernacular language) was invited to serve as the informant. Households in which no eligible informant was available or an informant asked to reschedule the interview were revisited later. Each household was visited up to three times.
Data collection instruments
We asked informants to describe their own sociodemographic background and the characteristics of their households. They were then asked whether there had been any death among regular household members since January 2020. Following recommendations [31], we anchored this question to the occurrence of an event of national relevance, i.e. the second round of the last presidential elections held on 29 December 2019. If the informant reported deaths during this period, then s/he was asked to report the number of deaths, list the deceased members by name, and provide details about each of them.
We implemented three approaches to asking about dates of reported deaths. The control arm of the trial followed the standard questionnaire (‘SQ’) used in surveys and census. For each listed death, data collectors asked informants: ‘In what month and year did [NAME] die?’ The experimental arms also included this question, but interviewers and respondents were aided by a calendar of local events. To devise this calendar, we drew a preliminary list of events that included COVID-related events (e.g. the national mandate for mask use in May 2020), political happenings (e.g. an attempted coup in February 2022), festivities (e.g. a popular music festival in May 2022), and annual holidays (e.g. Christmas Day). We then asked some residents from one of the HDSS neighborhoods whether they knew of these events. Events that were not recalled were excluded. Other events that were suggested by local residents and corroborated by HDSS data collectors were added. The final calendar only included public events (Supplementary Fig. S1) [See online supplementary material for a color version of this figure.]
In the first experimental arm (‘physical calendar’, PC), the calendar was displayed on an A4 sheet of paper. It was bracketed by months in columns and years in rows, thus defining cells that covered the period from January 2020 to the month of the interview. Events were listed in relevant cells, with some cells containing multiple events. Interviewers used the PC to confirm or correct the dates of deaths that were spontaneously reported by respondents or to determine the date for which respondents stated not knowing the month and/or year of death.
In the second experimental arm, a ‘digital calendar’ (DC) was designed to address concerns that interviewers might not systematically use the PC in ascertaining dates of deaths. Following the initial question about month and year of death, the DC progressed to one of three screens. If respondents had reported a month and year of death, then the application displayed a portion of the calendar covering ±6 months around the initial report. Interviewers used events displayed on this screen to confirm or correct the reported date. If respondents had only reported the year of death, then the application displayed the monthly calendar for the reported year and interviewers used that information to locate the month. If respondents only recalled the month of death, then the application showed a list of events and interviewers asked whether the death had happened around the time of one of those events. If the respondent identified an event close to the date of death, then the application showed a portion of the calendar covering ±6 months from the selected event to determine the month/year.
In both experimental arms, if respondents remained unable to recall both the month and the year of death, then they were asked to indicate another event (not included in the calendar) that had occurred around its time. Study investigators looked up the date of the reported event and imputed it as the date of death.
Randomization
Selected households were randomized 1:2:2 to an interview with SQ (control arm), PC, or DC (experimental arms). The randomization was stratified by neighborhood of residence and time elapsed since the most recent household death recorded by the HDSS. All randomization procedures were conducted on the list of selected households, using random numbers generated in Stata, v.18 [32].
Primary outcome
The main study outcome was the proportion of deaths reported to have occurred in the same month and year as indicated by the HDSS. To assess this outcome, we attempted to link each reported death to its HDSS record by using a manual approach [33]. We did not use information about dates of deaths to establish links. We planned to compare each experimental arm to the control arm of the trial. We used the multiarm R package to perform sample size calculations [34]. Assuming that 65% of deaths in the control arm were reported to have occurred on the same date as indicated by the HDSS, we sought to detect whether each type of (experimental) calendar increased concordance with the HDSS-recorded date by 15%age points. We accounted for multiple comparisons with the control arm by using Dunnett’s correction, setting the significance level at α = 0.03. Under the 1:2:2 allocation ratio, and with the power set to 0.8 for each comparison, we required linked reports of 101 deaths in the control arm and 202 deaths in each of the experimental arms.
Data collection
Three interviewers and one team supervisor conducted the trial from April to June 2023. All interviewers had prior data collection experience, but not in the study areas. They were trained to conduct interviews by using each of the trial’s instruments. All data were collected on tablets using SurveyCTO [35]—a common data collection platform in LMICs. To prevent crossovers, we created separate data collection forms with preloaded lists of assignments for each arm. Data collectors did not have access to any mortality-related data from the HDSS. This study was approved by the institutional review board of New York University-Abu Dhabi (Protocol HRPP-2023–39) and by the Comité Nacional de Ética na Saúde, in Guinea-Bissau (ref: 026/CNES/INASA/2023).
Analytical methods
In each arm of the trial, we described participation rates and the characteristics of participating households. We then tested null hypotheses in which we assumed that there was no difference in outcomes between the control arm and each experimental arm. We used a χ2 test to compare the distribution of households by number of reported deaths between study arms. To evaluate our primary outcome, as well as the proportion of reported deaths with missing date components, we used logistic regressions, controlling for stratification variables (i.e. time since death and neighborhood of residence), as recommended for the analysis of stratified randomized trials [36]. Among linked deaths with complete dates, we measured absolute differences (in months) between the reported date of death and the HDSS-recorded date. To test for differences between arms in such error patterns, we used nonparametric Kruskal–Wallis tests. In each arm, we also assessed the correlation between monthly series of deaths obtained from retrospective reports and from HDSS records. Finally, we estimated the amount of time required to complete each questionnaire (in minutes). If the PC or DC required significantly more time to complete than the SQ, then it may be impractical to adopt them in surveys and censuses. All standard errors were adjusted for clustering of deaths within households. Supplementary Table S1 presents the CONSORT checklist for the trial [37, 38]. [See online supplementary material for a color version of this table.]
Results
We identified 578 households with at least one death between January 2020 and May 2022 according to HDSS records. Among those, 508 were interviewed (87.9%). Participation was slightly lower among households assigned to PC. Table 1 presents the sociodemographic characteristics of the participating households.
Study participation and characteristics of participating households by study arm.
| Study arm | |||
|---|---|---|---|
| Variables | Standard questionnaire [n (%)] | Physical calendar [n (%)] | Digital calendar [n (%)] |
| Interview final result | |||
| Interview with wrong household | 1 (0.8%) | 8 (3.5%) | 1 (0.4%) |
| Vacant/empty housing unit | 7 (5.9%) | 26 (11.4%) | 20 (8.7%) |
| Absent | 0 (0.0%) | 1 (0.4%) | 4 (1.7%) |
| Postponed/refused | 1 (0.8%) | 0 (0.0%) | 1 (0.4%) |
| Consented and completed | 110 (92.4%) | 193 (84.6%) | 205 (88.7%) |
| Household informant characteristics | |||
| Age (years) | |||
| 18–29 | 44 (40.0%) | 77 (39.9%) | 89 (43.4%) |
| 30–49 | 47 (42.7%) | 80 (41.5%) | 82 (40.0%) |
| 50–64 | 11 (10.0%) | 23 (11.9%) | 26 (12.7%) |
| 65+ | 7 (6.4%) | 13 (6.7%) | 7 (3.4%) |
| Unknown, but 18+ | 1 (0.9%) | 0 (0.0%) | 1 (0.5%) |
| Sex | |||
| Male | 36 (32.7%) | 57 (29.5%) | 60 (29.3%) |
| Female | 74 (67.3%) | 136 (70.5%) | 145 (70.7%) |
| Education level | |||
| None | 17 (15.5%) | 18 (9.3%) | 15 (7.3%) |
| Primary (≤6th grade) | 20 (18.2%) | 32 (16.6%) | 32 (15.6%) |
| Secondary (7th–12th grade) | 60 (54.5%) | 116 (60.1%) | 125 (61.0%) |
| High school or higher | 13 (11.8%) | 27 (14.0%) | 33 (16.1%) |
| Civil status | |||
| Single | 53 (48.2%) | 99 (51.3%) | 118 (57.6%) |
| Officially/traditionally married/cohabiting | 34 (30.9%) | 59 (30.6%) | 57 (27.8%) |
| Widowed | 19 (17.3%) | 30 (15.5%) | 27 (13.2%) |
| Divorced/separated | 4 (3.6%) | 5 (2.6%) | 3 (1.5%) |
| Currently employed | |||
| Yes | 30 (27.3%) | 41 (21.2%) | 56 (27.3%) |
| No | 79 (71.8%) | 152 (78.8%) | 149 (72.7%) |
| Unsure | 1 (0.9%) | 0 (0.0%) | 0 (0.0%) |
| Household characteristics | |||
| Household size | |||
| <4 members | 9 (8.2%) | 21 (10.9%) | 12 (5.9%) |
| 4–9 | 68 (61.8%) | 115 (59.6%) | 130 (63.4%) |
| 10+ | 33 (30.0%) | 57 (29.5%) | 63 (30.7%) |
| Sleeping rooms per person | 0.4 (0.3) | 0.5 (0.2) | 0.4 (0.2) |
| Location of primary water source | |||
| In own dwelling | 42 (38.2%) | 95 (49.2%) | 101 (49.3%) |
| In own yard/plot | 64 (58.2%) | 88 (45.6%) | 89 (43.4%) |
| Elsewhere | 4 (3.6%) | 10 (5.2%) | 15 (7.3%) |
| No. of observations | 110 | 193 | 205 |
| Study arm | |||
|---|---|---|---|
| Variables | Standard questionnaire [n (%)] | Physical calendar [n (%)] | Digital calendar [n (%)] |
| Interview final result | |||
| Interview with wrong household | 1 (0.8%) | 8 (3.5%) | 1 (0.4%) |
| Vacant/empty housing unit | 7 (5.9%) | 26 (11.4%) | 20 (8.7%) |
| Absent | 0 (0.0%) | 1 (0.4%) | 4 (1.7%) |
| Postponed/refused | 1 (0.8%) | 0 (0.0%) | 1 (0.4%) |
| Consented and completed | 110 (92.4%) | 193 (84.6%) | 205 (88.7%) |
| Household informant characteristics | |||
| Age (years) | |||
| 18–29 | 44 (40.0%) | 77 (39.9%) | 89 (43.4%) |
| 30–49 | 47 (42.7%) | 80 (41.5%) | 82 (40.0%) |
| 50–64 | 11 (10.0%) | 23 (11.9%) | 26 (12.7%) |
| 65+ | 7 (6.4%) | 13 (6.7%) | 7 (3.4%) |
| Unknown, but 18+ | 1 (0.9%) | 0 (0.0%) | 1 (0.5%) |
| Sex | |||
| Male | 36 (32.7%) | 57 (29.5%) | 60 (29.3%) |
| Female | 74 (67.3%) | 136 (70.5%) | 145 (70.7%) |
| Education level | |||
| None | 17 (15.5%) | 18 (9.3%) | 15 (7.3%) |
| Primary (≤6th grade) | 20 (18.2%) | 32 (16.6%) | 32 (15.6%) |
| Secondary (7th–12th grade) | 60 (54.5%) | 116 (60.1%) | 125 (61.0%) |
| High school or higher | 13 (11.8%) | 27 (14.0%) | 33 (16.1%) |
| Civil status | |||
| Single | 53 (48.2%) | 99 (51.3%) | 118 (57.6%) |
| Officially/traditionally married/cohabiting | 34 (30.9%) | 59 (30.6%) | 57 (27.8%) |
| Widowed | 19 (17.3%) | 30 (15.5%) | 27 (13.2%) |
| Divorced/separated | 4 (3.6%) | 5 (2.6%) | 3 (1.5%) |
| Currently employed | |||
| Yes | 30 (27.3%) | 41 (21.2%) | 56 (27.3%) |
| No | 79 (71.8%) | 152 (78.8%) | 149 (72.7%) |
| Unsure | 1 (0.9%) | 0 (0.0%) | 0 (0.0%) |
| Household characteristics | |||
| Household size | |||
| <4 members | 9 (8.2%) | 21 (10.9%) | 12 (5.9%) |
| 4–9 | 68 (61.8%) | 115 (59.6%) | 130 (63.4%) |
| 10+ | 33 (30.0%) | 57 (29.5%) | 63 (30.7%) |
| Sleeping rooms per person | 0.4 (0.3) | 0.5 (0.2) | 0.4 (0.2) |
| Location of primary water source | |||
| In own dwelling | 42 (38.2%) | 95 (49.2%) | 101 (49.3%) |
| In own yard/plot | 64 (58.2%) | 88 (45.6%) | 89 (43.4%) |
| Elsewhere | 4 (3.6%) | 10 (5.2%) | 15 (7.3%) |
| No. of observations | 110 | 193 | 205 |
Study participation and characteristics of participating households by study arm.
| Study arm | |||
|---|---|---|---|
| Variables | Standard questionnaire [n (%)] | Physical calendar [n (%)] | Digital calendar [n (%)] |
| Interview final result | |||
| Interview with wrong household | 1 (0.8%) | 8 (3.5%) | 1 (0.4%) |
| Vacant/empty housing unit | 7 (5.9%) | 26 (11.4%) | 20 (8.7%) |
| Absent | 0 (0.0%) | 1 (0.4%) | 4 (1.7%) |
| Postponed/refused | 1 (0.8%) | 0 (0.0%) | 1 (0.4%) |
| Consented and completed | 110 (92.4%) | 193 (84.6%) | 205 (88.7%) |
| Household informant characteristics | |||
| Age (years) | |||
| 18–29 | 44 (40.0%) | 77 (39.9%) | 89 (43.4%) |
| 30–49 | 47 (42.7%) | 80 (41.5%) | 82 (40.0%) |
| 50–64 | 11 (10.0%) | 23 (11.9%) | 26 (12.7%) |
| 65+ | 7 (6.4%) | 13 (6.7%) | 7 (3.4%) |
| Unknown, but 18+ | 1 (0.9%) | 0 (0.0%) | 1 (0.5%) |
| Sex | |||
| Male | 36 (32.7%) | 57 (29.5%) | 60 (29.3%) |
| Female | 74 (67.3%) | 136 (70.5%) | 145 (70.7%) |
| Education level | |||
| None | 17 (15.5%) | 18 (9.3%) | 15 (7.3%) |
| Primary (≤6th grade) | 20 (18.2%) | 32 (16.6%) | 32 (15.6%) |
| Secondary (7th–12th grade) | 60 (54.5%) | 116 (60.1%) | 125 (61.0%) |
| High school or higher | 13 (11.8%) | 27 (14.0%) | 33 (16.1%) |
| Civil status | |||
| Single | 53 (48.2%) | 99 (51.3%) | 118 (57.6%) |
| Officially/traditionally married/cohabiting | 34 (30.9%) | 59 (30.6%) | 57 (27.8%) |
| Widowed | 19 (17.3%) | 30 (15.5%) | 27 (13.2%) |
| Divorced/separated | 4 (3.6%) | 5 (2.6%) | 3 (1.5%) |
| Currently employed | |||
| Yes | 30 (27.3%) | 41 (21.2%) | 56 (27.3%) |
| No | 79 (71.8%) | 152 (78.8%) | 149 (72.7%) |
| Unsure | 1 (0.9%) | 0 (0.0%) | 0 (0.0%) |
| Household characteristics | |||
| Household size | |||
| <4 members | 9 (8.2%) | 21 (10.9%) | 12 (5.9%) |
| 4–9 | 68 (61.8%) | 115 (59.6%) | 130 (63.4%) |
| 10+ | 33 (30.0%) | 57 (29.5%) | 63 (30.7%) |
| Sleeping rooms per person | 0.4 (0.3) | 0.5 (0.2) | 0.4 (0.2) |
| Location of primary water source | |||
| In own dwelling | 42 (38.2%) | 95 (49.2%) | 101 (49.3%) |
| In own yard/plot | 64 (58.2%) | 88 (45.6%) | 89 (43.4%) |
| Elsewhere | 4 (3.6%) | 10 (5.2%) | 15 (7.3%) |
| No. of observations | 110 | 193 | 205 |
| Study arm | |||
|---|---|---|---|
| Variables | Standard questionnaire [n (%)] | Physical calendar [n (%)] | Digital calendar [n (%)] |
| Interview final result | |||
| Interview with wrong household | 1 (0.8%) | 8 (3.5%) | 1 (0.4%) |
| Vacant/empty housing unit | 7 (5.9%) | 26 (11.4%) | 20 (8.7%) |
| Absent | 0 (0.0%) | 1 (0.4%) | 4 (1.7%) |
| Postponed/refused | 1 (0.8%) | 0 (0.0%) | 1 (0.4%) |
| Consented and completed | 110 (92.4%) | 193 (84.6%) | 205 (88.7%) |
| Household informant characteristics | |||
| Age (years) | |||
| 18–29 | 44 (40.0%) | 77 (39.9%) | 89 (43.4%) |
| 30–49 | 47 (42.7%) | 80 (41.5%) | 82 (40.0%) |
| 50–64 | 11 (10.0%) | 23 (11.9%) | 26 (12.7%) |
| 65+ | 7 (6.4%) | 13 (6.7%) | 7 (3.4%) |
| Unknown, but 18+ | 1 (0.9%) | 0 (0.0%) | 1 (0.5%) |
| Sex | |||
| Male | 36 (32.7%) | 57 (29.5%) | 60 (29.3%) |
| Female | 74 (67.3%) | 136 (70.5%) | 145 (70.7%) |
| Education level | |||
| None | 17 (15.5%) | 18 (9.3%) | 15 (7.3%) |
| Primary (≤6th grade) | 20 (18.2%) | 32 (16.6%) | 32 (15.6%) |
| Secondary (7th–12th grade) | 60 (54.5%) | 116 (60.1%) | 125 (61.0%) |
| High school or higher | 13 (11.8%) | 27 (14.0%) | 33 (16.1%) |
| Civil status | |||
| Single | 53 (48.2%) | 99 (51.3%) | 118 (57.6%) |
| Officially/traditionally married/cohabiting | 34 (30.9%) | 59 (30.6%) | 57 (27.8%) |
| Widowed | 19 (17.3%) | 30 (15.5%) | 27 (13.2%) |
| Divorced/separated | 4 (3.6%) | 5 (2.6%) | 3 (1.5%) |
| Currently employed | |||
| Yes | 30 (27.3%) | 41 (21.2%) | 56 (27.3%) |
| No | 79 (71.8%) | 152 (78.8%) | 149 (72.7%) |
| Unsure | 1 (0.9%) | 0 (0.0%) | 0 (0.0%) |
| Household characteristics | |||
| Household size | |||
| <4 members | 9 (8.2%) | 21 (10.9%) | 12 (5.9%) |
| 4–9 | 68 (61.8%) | 115 (59.6%) | 130 (63.4%) |
| 10+ | 33 (30.0%) | 57 (29.5%) | 63 (30.7%) |
| Sleeping rooms per person | 0.4 (0.3) | 0.5 (0.2) | 0.4 (0.2) |
| Location of primary water source | |||
| In own dwelling | 42 (38.2%) | 95 (49.2%) | 101 (49.3%) |
| In own yard/plot | 64 (58.2%) | 88 (45.6%) | 89 (43.4%) |
| Elsewhere | 4 (3.6%) | 10 (5.2%) | 15 (7.3%) |
| No. of observations | 110 | 193 | 205 |
In 448 of the 508 interviewed households (88.2%), informants reported that at least one death had occurred among regular members since January 2020. Characteristics of the deceased are shown in Supplementary Table S2. [See online supplementary material for a color version of this table.] There were no differences between each experimental arm and the control arm in whether respondents reported at least one death (SQ versus PC P = 0.48; SQ versus DC P = 0.62) or in the number of deaths reported to have occurred during the reference period (SQ versus PC P = 0.16; SQ versus DC P = 0.49, Fig. 2).
Distribution of household deaths reported in the trial, by study arm. Note: χ2 test P-values for pairwise comparison between the control arm (Standard questionnaire) and each experimental arm (Physical calendar; Digital calendar).
Informants reported that 574 deaths had occurred since January 2020. We linked 82% (n = 472/574) of those deaths to HDSS records, without differences in linkage rates between control and experimental arms (SQ versus PC P = 0.28; SQ versus DC P = 0.32). Deaths among individuals who had recently joined the household and deaths due to accidents were overrepresented among unlinked deaths (see Supplementary Table S3). [See online supplementary material for a color version of this table]. Fourteen linkages pertain to individuals who had migrated out of the study areas prior to death and for whom the HDSS did not include death records.
In total, we evaluated the reported dates of 458 deaths. Among those, 50.0% of deaths elicited by the SQ had the same month and year of death as recorded by the HDSS (Fig. 3). In the experimental arms, the estimated proportions were 52.6% in the PC arm (P = 0.74) and 48.0% in the DC arm (P = 0.73).
Share of linked deaths reported to have occurred in the same month and year as in the Bandim Health Project Health and Demographic Surveillance System (HDSS). Note: Vertical black lines represent 95% confidence intervals estimated from logistic models controlling for stratification variables. P-values are from the same models comparing the control arm (Standard questionnaire) and each experimental arm (Physical calendar; Digital calendar).
Household informants did not report the month or year for 39 out of 574 reported deaths (6.8%). In the control arm, an estimated 13.1% of reported deaths had missing data on one of these components (Fig. 4). In the experimental arms, the estimated proportion was 6.1% with PC (SQ versus PC P = 0.04) and 3.2% with DC (SQ versus DC P = 0.002).
Share of missing data on dates of death components, by study arm. Notes: The survey sample includes only deaths that were ascertained as having occurred from January 2020 onwards (n =574). Vertical black lines represent 95% confidence intervals estimated from logistic models controlling for stratification variables. P-values are from the same models comparing the control arm (Standard questionnaire) and each experimental arm (Physical calendar; Digital calendar).
When deaths were not reported in the same month and year as recorded by the HDSS, the median absolute difference from the HDSS records (Fig. 5) was 9 months in the control arm [inter quartile range (IQR): 1–12], 11 months in the PC arm (IQR: 3–12, SQ versus PC P = 0.16), and 12 months in the DC arm (IQR: 2–12, SQ versus DC P = 0.13). Among the observed discrepancies, 25.7% of deaths were displaced by 12 months in the control arm versus 32.9% in the PC arm (SQ versus PC P = 0.45) and 33.0% in the DC arm (SQ versus DC P = 0.43).
Absolute difference in months between Bandim Health Project Health and Demographic Surveillance System (HDSS) and trial-reported dates of death, by study arm. The graph excludes three observations (two in the Digital calendar and one in the Physical calendar) with month differences exceeding 40 months for visualization purposes.
We observed similar levels of correlation between the monthly series of deaths obtained from the HDSS data and the retrospective data generated by SQ or PC (Fig. 6, r = 0.57 and r = 0.60, respectively). The monthly series generated by DC yielded a lower level of correlation with the HDSS series (r = 0.40).
Number of deaths with complete dates reported in each study arm and the Bandim Health Project Health and Demographic Surveillance System (HDSS).
Interview durations (Fig. 7) were similar in the SQ (median: 13’20’’, IQR: 9’20’’ to 16’49’’) and PC arms (median: 13’01’’, IQR: 10’08’’ to 19’01’’; SQ versus PC P = 0.39). Relative to the control SQ arm, interviews aided by DC were longer (median: 14’29’’, IQR: 10’34’’ to 19’48’’; SQ versus DC P = 0.01).
Interview duration by study arm and number of reported deaths since January 2020.
Discussion
In this trial, the use of calendars of local events did not improve the reporting of dates of recent deaths collected during surveys or censuses typically conducted in LMICs. Although calendar tools reduced the extent of missing data on dates of death, the proportion of deaths reported to have occurred in the same month and year as recorded by the HDSS did not vary between control and experimental arms.
Our study has several limitations. First, our comparisons between control and experimental arms might have lacked sufficient power to reject various null hypotheses, in large part because (i) we assumed higher levels of concordance with HDSS records than attained during the study and (ii) more households than expected did not report any recent household death. Limited statistical power might have prevented us from detecting differences between study arms, e.g. in the magnitude of errors in reported dates. Second, participation was slightly lower in households assigned to interviews with the physical calendar. This might have confounded our comparison between trial arms. However, the observed characteristics of participating households were similar across all arms. Third, we did not investigate whether the effects of calendars might vary with the characteristics of households and recent deaths. Knowledge of dates of deaths might depend on factors such as the age of the deceased or the underlying cause of death. The dates of accidental deaths, for instance, might be better known because they are more frequently reported to local authorities (e.g. police). Detailed subgroup analyses of the use of calendars would, however, require larger sample sizes [39]. Fourth, our questionnaires asked informants to report household deaths that occurred over a long reference period (>3 years). While this is not uncommon [40], in many LMICs, surveys and censuses often only elicit deaths of the past 12 months, as recommended by the United Nations [41]. Our sample of households included too few deaths in the 12 months before the trial to investigate the effectiveness of calendars for such a recent period. Fifth, the reference HDSS dataset remains affected by errors and omissions. Some of the discrepancies observed in our trial might thus be attributable to limitations of HDSS records rather than to errors in retrospectively collected data. Finally, it is unclear whether our results might extend to other populations in Guinea-Bissau or other LMICs. We conducted our study in urban areas, where residents often have higher numeracy levels than residents of more rural areas. They might also be more accustomed to reporting dates of events because of frequent interactions with HDSS data collectors or with formal institutions such as governmental offices. The effectiveness of calendars of local events in improving dates of death reporting might thus differ in other LMIC settings.
Nevertheless, this study has multiple implications. First, our results do not support the use of a digital calendar to improve the reporting of dates of deaths. Not only did this approach fail to improve reported dates, but it also increased the interviewing time relative to standard close-ended questions about dates of death. Second, studies are needed to determine the optimal design and use of a physical calendar of local events in collecting data on recent household deaths. Indeed, while the physical calendar at times led to larger errors in reported dates than the standard approach, it also fostered higher levels of data completeness without increasing interviewing times. Future studies could thus explore whether excluding annual events from those listed in the calendar might reduce the proportion of deaths displaced by exactly 12 months in interviews aided by a physical calendar. These studies might also investigate protocols in which interviewers do not systematically use the physical calendar to verify the date of every reported death, but instead only use it when respondents do not know the month and/or year of death. Finally, the high levels of omissions and errors documented in retrospective data on recent household deaths further stress the need for large investments in strengthening CRVS systems in LMICs [42].
Ethics approval
The trial was approved by the institutional review board of New York University-Abu Dhabi (Protocol HRPP-2023–39) and by the Comité Nacional de Ética na Saúde in Guinea-Bissau (Guinean-Bissau National Ethics Committee) (ref: 026/CNES/INASA/2023).
Use of artificial intelligence (AI) tools
None.
Author contributions
Conceptualization: S.H., A.B.F., O.T. Data collection: A.B.F., D.A.A.F., O.T. Formal analysis: O.T., S.H. Methodology: S.H., A.B.F., O.T. Project administration: A.B.F., S.H. Supervision: A.B.F., S.H., O.T. Writing—original draft: O.T. Writing—review and editing: O.T., A.B.F., D.A.A.F.
Supplementary data
Supplementary data is available at IJE online.
Conflict of interest
None declared.
Funding
The study was supported by grant R01HD088516 from the Eunice Kennedy Shriver National Institute of Child Health and Human Development, and grant R03AG070660 from the National Institute on Aging.
Data availability
Data from the BHP’s HDSS and the trial can be made on a collaborative basis provided approval from the Guinean Ethics Committee (www.bandim.org).
