A Systematic Review and Meta-Analysis on the Efficacy of Virtual Reality Pedestrian Interventions to Teach Children How to Cross Streets Safely

Abstract

Objective

Over 7,000 American children die from pedestrian injuries annually, and pedestrian injury ranks among the top 5 causes of unintentional child death. Prevention efforts are multifaceted. One strategy, use of virtual reality (VR) to teach children to cross streets, is of growing interest to public health practitioners. The present study is a systematic review and meta-analysis that examined the efficacy of using VR to teach children pedestrian safety.

Methods

Following PRISMA guidelines, searches among 7 databases were completed, followed by abstract/full-text screening and data extraction. Hedge’s g was computed for the effect sizes of 3 outcomes: pedestrian knowledge, pedestrian-relevant cognition (attention to traffic, time to contact, start delay), and pedestrian behaviors (safe crossings, unsafe crossings). Risk of bias was assessed using Cochrane guidelines. Meta-regression analyses and subgroup analyses were conducted including 3 moderators: age, sex, and immersion level.

Results

A total of 20 studies, reported in 24 articles, were included in the qualitative analysis. Meta-analysis of the 12 studies with sufficient quantitative data available showed a statistically significant medium effect of VR safety interventions on child pedestrian knowledge and behavior. Mixed results emerged for the effect of VR safety interventions on children’s pedestrian-relevant cognition. Age and sex moderated the effect of VR training on pedestrian knowledge.

Conclusions

This synthesis of the literature on pediatric VR pedestrian safety interventions suggests an overall beneficial impact of VR interventions to teach children how to cross streets safely. Efforts should continue to develop and disseminate effective VR interventions.

Registration ID

CRD42022309352

child pedestrian safety, meta-analysis, systematic review, unintentional injury, virtual reality

Unintentional injury is a leading cause of pediatric hospitalization and death. Global estimates from 2019 indicate over 250,000 fatal unintentional injuries annually among children ages 0–19 (Institute for Health Metrics and Evaluation [IHME], 2023), with over 7,000 occurring in the United States (West et al., 2021). Associated medical costs for fatal unintentional pediatric injuries in the United States were over $102 million in 2020, and medical costs related to non-fatal unintentional pediatric injury exceeded $35 billion (Centers for Disease Control and Prevention [CDC], 2023).

Pedestrian injuries account for a large portion of unintentional pediatric deaths both in the United States and globally (IHME, 2023); in the United States, pedestrian injury ranks among the top five causes of unintentional pediatric fatalities (CDC, 2023). Medical costs associated with fatal and non-fatal unintentional pedestrian injury in the United States exceeded $7 million and $471 million, respectively, in 2020 (CDC, 2023). Consequently, pediatric pedestrian injuries constitute a major public health concern. Prevention efforts are urgently needed.

The primary means to reduce pediatric pedestrian injuries tend to be multifaceted efforts encompassing increased adult supervision, pedestrian-friendly road and traffic engineering, and traffic calming where children frequently walk (Cloutier et al., 2021). Also critical are attempts to teach pedestrian safety skills to children (Schwebel, Barton, et al., 2014). A 2014 meta-analysis by Schwebel et al. found a small to medium effect size (standardized mean difference [SMD] = .31) of behavioral interventions to teach children pedestrian safety in improving safety outcomes, with strong evidence (SMD = .48) supporting interventions conducted in individual or small group settings (Schwebel, Barton, et al., 2014). The analysis also indicated that these interventions may have differential impacts on particular aspects of pedestrian safety (i.e., safe route selection vs. safe gap selection) and highlighted interventions targeting dash-out prevention, crossing at parked cars, and safe route selection as achieving the most successful training outcomes.

Since 2014, several new studies have investigated the efficacy of pedestrian safety interventions for children (Feng et al., 2020; Gu & Sosnovsky, 2017; Khan et al., 2021; Schwebel, Combs, et al., 2016). A recent systematic review on the prevention of unintentional pediatric injury briefly summarized the findings of this updated research and concluded that intervention efforts aiming to teach children pedestrian safety were largely effective in improving safety behaviors and knowledge (Bou-Karroum et al., 2022).

One strategy to train children in pedestrian safety has emerged especially prominently over recent years, use of virtual reality (VR). Typically focused on the pedestrian task of identifying safe traffic gaps to cross within traffic, VR offers children repeated practice at the complex cognitive-perceptual task of judging traffic and determining when to cross streets, without exposing children to real-world risks of actual traffic. It is increasingly low-cost (some interventions use low-end smartphones and plastic holders; Schwebel et al., 2017) and therefore can be used to train large groups of children simultaneously in school, faith-based, or community center settings (Feng et al., 2020; Gu & Sosnovsky, 2017; Schwebel, Combs, et al., 2016). The present study implemented meta-analytic techniques to determine the efficacy of using VR to teach children pedestrian safety skills.

Beyond our primary goal of determining the efficacy of using VR to teach children pedestrian safety skills, we also examined three factors as possible covariates influencing the efficacy of VR interventions. First, we considered the type of VR intervention that was used. Driven largely by the evolution of VR technology over time as it pertains to capacity for immersion of children into the environment, we considered whether children may learn better when fully immersed into a highly realistic virtual environment compared to a simpler environment in which they merely view traffic (and practice relevant cognitive-perceptual skills) on two-dimensional screens. Early research suggests the latter approach may offer an effective pedestrian safety training tool (Glang et al., 2005; McComas et al., 2002; Schwebel, McClure, et al., 2014), but no published research compares efficacy of training across different levels of VR immersion.

Second, the role of sex on child pedestrian safety and training is unresolved. Males are more likely to experience unintentional pedestrian injury (West et al., 2021) and to engage in riskier street crossing behaviors (Barton & Schwebel, 2007), but conflicting findings are reported concerning sex disparities in other aspects of pedestrian behaviors like route selection (Barton et al., 2012; Barton & Schwebel, 2006). No systematic study has examined the moderating role of sex on pedestrian safety training.

Finally, given the critical aspects of cognitive development involved in pedestrian behavior, age is widely recognized as playing an important role in predicting child pedestrian safety outcomes (Bart et al., 2008; Barton et al., 2012; Barton & Schwebel, 2006, 2007). Efforts to identify the optimal age for VR-based learning are absent from the published literature, suggesting it is not yet clear at what age children are best able to learn to cross streets safely with sufficient practice. Attempts to train children when they are too young could yield unethical training that iatrogenically increases risk among children who believe they have learned safety. Attempts to train children when they are too old could be irrelevant, as the children may already have relevant skills. Thus, exploration of how child age might moderate efficacy of VR-based pedestrian safety training is necessary, and the power of a meta-analytic review can help address the issue.

In summary, the present systematic review addressed three aims: (1) qualitatively synthesize existing evidence concerning VR interventions to teach children to cross the street safely; (2) quantitatively synthesize the aggregated effects of VR interventions on children’s pedestrian-related knowledge, cognition, and behavior using meta-analyses; and (3) identify whether the effect of intervention strategies might differ by level of VR immersion, child sex, and across different ages of children.

Method

Ethics Review and Data Availability

The present study was exempt from Institutional Review Board (IRB) consideration because the research activities are not human subjects research, as defined by 45 CFR 46. Aggregated data collected for the systematic review and meta-analysis are available in the article and in its supplementary material. Individual data collected for the preparation of this manuscript are stored through Comprehensive Meta Analysis (CMA) Version 3 (Borenstein et al., 2013) and Covidence (Veritas Health Innovation, 2020) and will be shared on reasonable request to the corresponding author. This review was registered on PROSPERO on April 13, 2022, with the following ID: CRD42022309352. The last author on this paper (D.C.S.) authored several of the papers included in the present review. Due to potential conflict of interest, this author was not involved in the actual determination of efficacy of any of the individual papers.

Search Strategy

PRISMA Guidelines were followed to conduct a systematic review (Page et al., 2021). Seven databases were searched: PsychInfo, PubMed, Engineering Village, Web of Science, Cochrane Library, Transportation Research Information Services (TRIS) Database, and National Highway Traffic Safety Administration (NHTSA) Database. The following search terms were used for all database searches: child* AND (pedestrian* OR street* OR crossing*) AND (“virtual reality” OR simulation) AND (training OR intervention OR education). Additional literature search methods supplemented results from these searches, including contact with global experts in the field and follow-up of references in relevant articles. All database searches were limited to peer-reviewed articles and conference presentations published and indexed before January 1, 2023.

Inclusion and Exclusion Criteria

Inclusion criteria comprised: (1) The study used virtual reality (VR) as the method of intervention, with VR broadly defined as “software representations of real (or imagined) agents, objects and processes; and a human–computer interface for displaying and interacting with these models” (Barfield et al., 1995, p. 476). (2) The purpose of the VR intervention was to teach children about knowledge, cognition, or behavior related to pedestrian safety. (3) At least a portion of the study sample comprised typically developing (e.g., no reported developmental disorders) children or adolescents <18 years old. (4) Studies implemented any type of interventional research design. (5) Interventions included a training or teaching component for participants. (6) Studies originated from any region of the world and were published in any language. Studies were excluded if they did not meet all of the criteria.

Title and Abstract Screening

As detailed in Figure 1, a total of 969 entries were identified following the initial search, which resulted in 751 entries for the title and abstract screening stage after 188 duplicates were removed (first through Endnote, then through Covidence). During this stage, two researchers (C.H.M., L.M.S.) reviewed the title and abstract of each entry and independently determined whether the entry should be included or excluded for the next stage according to the inclusion and exclusion criteria of this project. Strong interrater agreement (average proportionate agreement across reviewers = 0.96) was achieved and most conflicts resolved through discussion between these two authors. Any unresolved conflicts were finalized by one or both of the other authors.

Figure 1.

PRISMA flow diagram. Note. From Page et al. (2021).

Full-Text Screening

The title/abstract screening yielded a total of 77 entries to proceed to the full-text screening stage. During this stage, we attempted to retrieve the full text of all included entries from our institutional library systems first. If the full text was not available at the co-authors’ institutions, interlibrary loan requests were made and internet searches conducted. For entries that we were unable to obtain through those methods, email requests were sent to the corresponding authors.

Next, two researchers (C.H.M., L.M.S.) independently reviewed and determined whether each full text article should be included or excluded based on our pre-determined criteria. Strong inter-rater agreement was achieved (average proportionate agreement across reviewers = 0.97) and conflicts resolved through discussion between these 2 authors. Unresolved conflicts were addressed by one or both other authors. A total of 53 entries were excluded at this stage, five for missing full texts and 48 based on exclusion criteria, as listed in Figure 1. Both the title/abstract and full-text screening stages were completed using the Covidence Systematic Review Software (Veritas Health Innovation, 2020).

Data Extraction for Synthesis

A standardized data extraction form was created by three members of the study team (C.H.M., L.M.S., J.S.) to extract relevant data for qualitative synthesis from the entries retained from the full-text screening stage. The qualitative sections of the data extraction form included article title, authors, journal, year of publication, country where the study was conducted, sample size, sex ratio, descriptive statistics for child age, VR immersion level, outcome domain and measurement, and study design.

To conduct a meta-analysis addressing the efficacy of VR pedestrian safety interventions for children, quantitative sections of the data extraction form gathered numerical data regarding the intervention’s efficacy as reported within each included study. Quantitative extraction was based on calculated effect sizes reported in the article, means and standard deviations for outcomes of interest, and/or other reported statistics that could be used to calculate effect sizes. For studies that did not report sufficient data to compute effect sizes, efforts were made to contact the corresponding authors to request either computed statistics or raw data. Studies that were otherwise eligible for inclusion but lacked sufficient quantitative data for effect size calculations presented via one of the 100 data formats provided by software CMA3.0 were excluded from the quantitative synthesis. Similar to previous steps, all data extraction was performed by two independent co-authors (C.H.M., L.M.S.), with disagreement resolved by discussion among the coders and other co-authors in project meetings. The number and characteristics of studies eligible for data extraction are detailed in the Results section.

Types of Outcomes

Primary outcomes in the included studies were proposed by two authors during the screening process (C.H.M., L.M.S.) and approved by the other authors. Learning how to cross the street is a complex task that can be divided into several components. The following three outcome domains were identified upon review and agreed upon by all co-authors: one variable for pedestrian safety knowledge (safety knowledge), three variables for pedestrian-related cognition (attention to traffic, time to contact, start delay), and two variables for pedestrian behavior (safe crossings, unsafe crossings).

Pedestrian Safety Knowledge

The pedestrian safety knowledge construct considers general knowledge that a child possesses about pedestrian rules and safe practices. Pedestrian safety knowledge can be assessed in various ways, including through written or oral tests of general pedestrian safety knowledge, hazard identification tasks that require children to recognize hazards in photos or videos that may lead to an unsafe pedestrian outcome, and route selection tasks involving identification of safe and unsafe routes to cross the street on diagrams, maps, or models.

Pedestrian Cognition

The pedestrian cognition construct refers to the decision-making component of child pedestrian behavior. Prior to and during the physical act of crossing the street, pedestrians engage in sophisticated cognitive processes that involve perceiving stimuli, judging the dynamic nature of risk from oncoming traffic as well as the static nature of the physical road-crossing, and then initiating motor processes to cross the street. To ensure safety, these complex cognitive processes must continue until the pedestrian safely crosses the street. A primary goal of pedestrian safety training with children in VR is to improve the efficiency and accuracy of relevant cognitive processes. Both theoretical conceptualizations and available data indicate three commonly assessed outcomes that tap into children’s cognitive processing of pedestrian situations: attention to traffic, time to contact, and start delay.

Attention to traffic refers to the frequency of looking left and right toward traffic prior to and/or while crossing the street. Usually measured over time (e.g., looks/minute or looks/second), higher values indicate greater attention to oncoming traffic. Time to contact generally refers to the smallest temporal gap, in seconds or milliseconds, between the pedestrian and any oncoming vehicle during a street-crossing. Larger gaps are generally considered to indicate safer decision-making. Start delay refers to the time in seconds after the last car passes and before the pedestrian initiates crossing the street. Quicker and more efficient decision-making leads to shorter start delays and is generally considered to indicate more sophisticated pedestrian-related cognition. As children train in pedestrian safety, experts expect start delays will decrease.

Pedestrian Behavior

The pedestrian behavior construct refers to actual behavior while crossing the street: was the child safe or unsafe in their crossing? Researchers assess this construct in various ways. Some count the number or percent of safe crossings (e.g., Arbogast et al., 2014; Luo et al., 2021; McComas et al., 2002), and others count the number or percent of unsafe crossings (e.g., Morrongiello et al., 2018; Schwebel, Combs, et al., 2016; Schwebel, McClure, et al., 2014; Schwebel, Shen, et al., 2016; Schwebel et al., 2018; Schwebel & McClure 2014b). Further, unsafe crossings are defined differently across studies. Many studies tally hits or collisions (child is directly struck by a virtual vehicle while crossing the street; e.g., Bart et al., 2008; Josman et al., 2008; Schwebel et al., 2018) while others also include close calls (usually conceptualized as the child was within 1 s of being struck by a vehicle while crossing; e.g., Schwebel, Combs, et al., 2016; Schwebel, McClure, et al., 2014) as well as hits to determine a total number of unsafe crossings. Based on the availability of published data as well as the authors’ conceptualization of pedestrian behavior, our meta-analysis organized pedestrian behavior into 2 general outcomes: safe crossings and unsafe crossings.

Safe crossings were considered as any behavior that resulted in a safe outcome after crossing the street. This included children crossing a virtual street without being struck by a vehicle, children correctly choosing to cross the street when it was safe, and children correctly choosing not to cross the street when it was unsafe. Successful pedestrian safety interventions result in increased safe crossing behaviors.

Unsafe crossings were considered any behavior that resulted in an unsafe outcome. This included hits or collisions, close calls, and a combined variable of hits and close calls. In some studies, it also included incorrectly choosing not to cross the street when it was safe. Successful pedestrian safety interventions result in decreased unsafe crossing behaviors.

Detailed descriptions of each of these outcomes appear in Supplementary Table 1. When data were collected both post-intervention and after a follow-up retention period, we used post-intervention results. When data were available from multiple measures for a single outcome domain, a “primary measure” and its effect size were selected for inclusion in the meta-analysis. Primary measures were selected based on the comprehensiveness and commonality of the selected outcome measures across the included studies and in the general literature. We favored more comprehensive measures to assess target outcomes (e.g., hits and close calls) over narrowly scoped measures (e.g., hits only) for the same target outcome. Overarching safety scores were prioritized over individual safety concepts. Further, we prioritized VR outcomes over simulated real-world measures when both were available. Other pedestrian outcomes not included in our primary main analyses are listed in Supplementary Table 2.

Moderators

The following variables were considered to identify whether the effect of some intervention strategies might differ using meta-regression and sub-group analyses: age of participants (continuous: mean age), percentage of female participants (continuous), and level of VR immersion (categorical: non-immersive, semi-immersive, or fully immersive). Both age and gender were extracted from the demographics section of each included study. Studies that did not report age or sex were excluded from the moderation analyses.

The level of VR immersion was extracted from the methodology section of each included study and was separated into three categories based on criteria developed by Slater & Wilbur (1997) and updated by Kardong-Edgren et al. (2019). Three categories were used, VR-Low, VR-Moderate, and VR-High, based on four characteristics: (a) inclusiveness of the devices, (b) extensiveness of the immersion, (c) vividness of the visual stimuli, and (d) motion capture and proprioceptive feedback.

Inclusiveness incorporated presence of stimuli from the outside world. Low immersion devices include numerous external signals, such as a joystick or mouse to control the virtual environment. Moderate inclusiveness has some external signals, such as noise from a computer fan, and high immersion incorporates limited signals indicating the presence of devices in the physical world, such as the weight of an eye-tracking device. Extensiveness refers to sensory modalities and spatial orientation involved in the simulation. Low immersion generally accommodates one sensory modality and stimuli are not spatially oriented; moderate immersion accommodates one to two sensory modalities and stimuli may or may not be spatially oriented; and high immersion accommodates greater than two sensory modalities and spatial orientation of stimuli. Thus, low immersion often involves a computer monitor presentation with limited field of view, moderate immersion a large-screen projection with extended field of view, and high immersion a head-mounted device or surround “cave” projection.

Vividness of the visual stimuli was conceptualized as follows: low immersion consisted of low fidelity and visual/color resolution to replicate features of the simulated environment but not in a detailed manner; moderate immersion applied moderate fidelity and visual/color resolution to replicate features of the simulated environment; and high immersion maintained high fidelity and visual/color resolution to closely replicate multiple features of the simulated environment. Last, when considering motion capture and proprioceptive feedback (the ability to capture movement), low immersion involves no motion capture and no matching proprioceptive feedback, moderate immersion involves body segment motion capture and the visual experience is somewhat altered to match proprioceptive feedback, and high immersion involves full-body motion capture and a visual experience that closely matches proprioceptive feedback. For example, high immersion might depict an avatar in the virtual environment that closely aligns with the actual movement of the participant in the real-world environment.

As part of the data extraction process, level of immersion was coded based on the aforementioned criteria (Kardong-Edgren et al. 2019). Similar to previous steps, coding of these values was performed by 2 independent co-authors (C.H.M., L.M.S.); no discrepancies emerged. We intended to use level of immersion as a moderator and include it in moderation analyses, however, due to unexpected low sample size, we instead conducted a subgroup analysis to examine differences between level of immersion.

Risk of Bias Assessment

Two levels of bias were evaluated. Bias at the meta-analysis level was reduced by including all possible studies using a systematic, comprehensive literature search. Biases at the level of individual study were assessed according to Cochrane guidelines (Higgins & Green, 2011). For randomized studies, the following domains were evaluated using the Cochrane Risk-of-Bias tool for randomized trials (RoB 2; Higgins et al., 2022): random sequence generation (selection bias), allocation concealment (selection bias), blinding of outcome assessment (detection bias), incomplete outcome data (attrition bias), and selective reporting (reporting bias). Within each domain, a series of questions (“signaling questions”) aimed to elicit information about features relevant to risk of bias. Based on answers to the signaling questions, judgements of “low,” “some concerns,” or “high” risk of bias were assigned.

For non-randomized studies, the Cochrane Risk of Bias In Non-randomized Studies of Interventions (ROBINS-I) tool (Sterne et al., 2022) was used to evaluate bias at the pre-intervention, at-intervention, and post-intervention stages. Pre-intervention domains included bias due to confounding (confounding bias) and bias in selection of participants into the study (selection bias). The at-intervention domain included bias in classification of interventions (information bias), and the post-intervention domains included bias due to deviations from intended interventions (confounding bias), bias due to missing data (selection bias), bias in measurement of the outcome (information bias), and bias in selection of reported results (reporting bias). Similar to the RoB 2 tool, within each domain, signaling questions facilitate gathering information relevant to risk of bias. Based on answers to the signaling questions, judgements for each bias domain and for overall risk of bias were scored as “low,” “moderate,” “serious,” or “critical” risk of bias. Two co-authors (C.H.M., L.M.S.) rated all biases and discrepancies were resolved through discussion.

Data Analysis Plan

First, descriptive characteristics from each eligible study were extracted and summarized (Table I). Distribution frequencies and percentages were calculated. Second, we evaluated each included study for risk of bias according to Cochrane guidelines (Cochrane Handbook for Systematic Reviews of Interventions, 2011). Third, meta-analytic assessments of the effect of VR pedestrian safety interventions were performed on the three outcomes of pedestrian knowledge, cognition, and behavior. For all studies, Hedge’s g, its 95% confidence interval (95% CI as assessment of certainty/precision for the Hedge’s g), and the associated z and p values were computed. Hedge’s g was chosen as an unbiased version of Cohen’s d (Lin & Aloe, 2021) and can be similarly interpreted using the following criteria regarding the size of interventional effects: small (0.20–0.49), medium (0.50–0.79), and large (>0.80). Overall effect sizes were estimated using random-effect models to account for potential between-study heterogeneity. These effect sizes can be interpreted as the overall efficacy of VR pedestrian safety interventions in teaching children how to cross streets safely across the three main outcome domains. Study-level effect sizes and meta-analyses were calculated using CMA3.0 (Borenstein et al., 2013). In accordance with best practices and due to similarities between the results noted across fixed and random effects models for each outcome, random effects models were used for all analyses (Bell, Fairbrother, & Jones, 2019). Heterogeneity was estimated using I² statistics, which are defined as the percentage of variance due to heterogeneity—rather than sampling error—in the total variance (Higgins & Thompson, 2002). An I² of 25% is considered low, 50% moderate, and 75% high (Higgins et al., 2003).

Table I.

Characteristics of Studies Included in the Qualitative Synthesis (Total N Across Studies = 1,896)

Author(s), year	Overall sample	Comparison	Control	VR immersion	Comparison groups	Methodology and procedure	Relevant outcome domains	Country
	N, M (SD) age in years or % grade, % female
Arbogast et al. (2014)* (Study 1)	348, 49% second grade, 51% third grade, 54% 64.5% Spanish-speaking	180, 52% second grade, 48% third grade, 53% 64% Spanish-speaking	168, 46% second grade, 54% third grade, 55% 65% Spanish-speaking	Low	Didactic group and videogame intervention	RCT (1) Pre-test (2) Training (3) Post-test	Pedestrian knowledge Pedestrian cognition Pedestrian behavior	United States
Bart et al. (2006)*,a (Study 2)	86, 9.30 (1.48), 64% Central and Northern Israel	11, No age or sex data Central and Northern Israel	10, No age or sex data Central and Northern Israel	Moderate	Control group and training group	RCT (1) Pre-test (2) Training (3) Post-test	Pedestrian behavior	Israel
Bart et al. (2008)*,a (Study 2)	21, 8.94 (1.30), 62% Central and Northern Israel	10, No age or sex data Central and Northern Israel	10, No age or sex data Central and Northern Israel	Moderate	Control group and training group	RCT (1) Pre-test (2) Training (3) Post-test	Pedestrian behavior	Israel
Congiu et al. (2007) (Study 3)	71, 8.04 (No SD), 51% No race or ethnicity data	36, No age or sex data No race or ethnicity data	35, No age or sex data No race or ethnicity data	Moderate	Control group and intervention group	RCT (1) Pre-test (2) Training (3) Post-test (4) Follow-up	Pedestrian cognition Pedestrian behavior	Australia
Feng et al. (2020) (Study 4)	79, No age or sex data Hubei province	N/A	N/A	High	N/A	Pre/post (1) Pre-session (2) Training (3) Post-session (4) Wrap up	Pedestrian cognition Pedestrian behavior	People’s Republic of China
Glang et al. (2005)* (Study 5)	36, 53% kindergarten/first grade, 47% second/third grade, 42% No race or ethnicity data	N/A	N/A	Low	N/A	Pre/post (1) Pre-test (2) Post-test (3) Field test (real-life)	Pedestrian knowledge	United States
Gu et al. (2015) (Study 6)	10, No age or sex data German-speaking	N/A	N/A	Low	N/A	Pre/post (1) Pre-test (2) Training (3) Post-test	Pedestrian knowledge Pedestrian cognition	Germany
Gu and Sosnovsky (2017)* (Study 7)	58, 50% second graders and 50% third graders, 47% No race or ethnicity data	No data	No data	Low	Immediate and delayed feedback	RCT (1) Pre-test (2) Training (3) Post-test	Pedestrian knowledge Pedestrian cognition	Germany
Khan et al. (2021)* (Study 8)	10, No age or sex data No race or ethnicity data	N/A	N/A	Moderate	N/A	Pre/post (1) Pre-test (2) Traditional training (3) VR training (4) Post-test	Pedestrian knowledge	Switzerland
Luo et al. (2020)* (Study 9)	79, 33% first grade, 33% second grade, 33% third grade, 50% Hubei province	N/A	N/A	High	N/A	Pre/post (1) Pre-test (2) Debrief (3) Post-test (4) Debrief	Pedestrian knowledge Pedestrian cognition Pedestrian behavior	People’s Republic of China
Luo et al. (2021)* (Study 10)	150, 7.9 (No SD), 49% second grade, 51% third grade, 43% Hubei province	Condition 1 = 39, 7.9 (No SD), 49% second grade, 51% third grade, 46% Condition 2 = 38, 50% second grade, 50% third grade, 7.8 (SD), 37% Condition 3 = 42, 7.9 (No SD), 48% second grade, 52% third grade, 45% Condition 4 = 31, 7.8 (No SD), 45% second grade, 55% second grade, 45%		High	Condition 1—debriefing, individual Condition 2—no debriefing, individual Condition 3—debriefing, group Condition 4—no debriefing, group	RCT – 2 × 2 factorial design: (1) Comparing performance individually versus observing in a group (2) Debriefing versus no debriefing Pre/post (1) Pre-test (2) Training + debrief (3) Post-test	Pedestrian knowledge Pedestrian cognition Pedestrian behavior	People’s Republic of China
McComas et al. (2002)* (Study 11)	95, Grades 4–6, 54% 49% From an urban school—62% 51% From a suburban school—46% Included French-speaking participants	47: 25 Urban, 22 Suburban No age or sex data	48: 22 Urban, 26 Suburban No age or sex data	Moderate	Control group and intervention group	RCT (1) Pre-test (2) VR pre-test (3) VR training (4) VR post-test (5) Post-test	Pedestrian knowledge Pedestrian cognition	Canada
Morrongiello et al. (2018)* (Study 12)	130, 8.77 (1.44), 50% 94% White	Intervention/Where = 44, 8.90 (No SD), 52%, 95% White Intervention/How = 43, 8.59 (No SD), 47%, 98% White Control/Where = 33, 8.92 (No SD), 55%, 96% White Control/How = 21, 8.70 (No SD), 48%, 100% White		High	Condition 1—intervention, where to cross Condition 2—intervention, how to cross Condition 3—control, where to cross Condition 4—control, how to cross	RCT—2 × 2 Factorial Design: (1) intervention and control (2) Where and how to cross Pre/post (1) Pre-test (2) Training + debrief (3) Post-test	Pedestrian knowledge Pedestrian cognition Pedestrian behavior	Canada
Oxley et al. (2008) (Study 13)	71, 8.04 (No SD), 51% Melbourne	36 No age or sex data	35 No age or sex data	Moderate	Control group and intervention group	RCT (1) Pre-test (2) Training (3) Post-test (4) Follow-up	Pedestrian cognition Pedestrian behavior	Australia
Remenyi et al. (2003) (Study 14)	100, No Age or Sex Data 25 Primary, Rural 25 Primary, Metro 25 Secondary, Rural 25 Secondary, Metro Melbourne	Computer simulation: 2 total classes (1 from each school) Computer simulation + real-world: 2 total classes (1 from each school)		Low	Computer simulation only and computer simulation + real-traffic practice	Pre/post (1) Pre-test (2) Training (3) Post-test (4) Follow-up test	Pedestrian knowledge Pedestrian behavior	Australia
*Schwebel, Combs, et al. (2016) (Study 15)	44, 8.01 (.56), 51% 52% Black, 48% White	N/A	N/A	Moderate	N/A	Pre/post (1) Pre-test (2) Training (3) Post-test	Pedestrian cognition Pedestrian behavior	United States
Schwebel & McClure (2014a)b (Title: Children’s Pedestrian Route Selection: Efficacy of a Video and Internet Training Protocol) (Study 16)	231, 8.0 (.70), 58% 40% Black 53% White 7% Other or Biracial	VR training: 59, 7.9 (.67), 54% 39% Black 59% White 2% Other Video/computer training: 57, 8.1 (.63), 61% 40% Black 53% White 7% Other Streetside training: 57, 7.9 (.68), 61% 39% Black 53% White 9% Other No contact control: 58, 8.1 (.63), 52% 43% Black 47% White 10% Other	N/A	Moderate	Three training conditions and one control	RCT (1) Pre-test (2) Field test (3) Training (4) Post-test (5) Post-field (6) Follow-up test (7) Follow-up field	Pedestrian knowledge Pedestrian behavior	United States
Schwebel & McClure (2014b)*,b (Title: Training Children in Pedestrian Safety: Distinguishing Gains in Knowledge from Gains in Safe Behavior) (Study 16)	231, 8.0 (.70), 57% 40% Black, 53% White 7% Other or Biracial	VR training: 52, 7.9 (.67), 54% 39% Black 59% White 2% Other Video/computer training: 52, 8.1 (.63), 61% 40% Black 53% White 7% Other Streetside training: 52, 7.9 (.68), 61% 39% Black 53% White 9% Other No contact control: 55, 8.1 (.63), 52% 43% Black 47% White 10% Other		Moderate	Three training conditions and one control	RCT (1) Pre-test (2) Field test (3) Training (4) Post-test (5) Post-field (6) Follow-up test (7) Follow-up field	Pedestrian knowledge Pedestrian cognition Pedestrian behavior	United States
Schwebel, McClure, et al. (2014)*,b (Study 16)	231, 8.0 (.70), 58% 40% Black 53% White 7% Other or Biracial	VR training: 59, 7.9 (.67), 54% 39% Black 59% White 2% Other Video/computer training: 57, 8.1 (.63), 61% 40% Black 53% White 7% Other Streetside training: 57, 7.9 (.68), 61% 39% Black 53% White 9% Other No contact control: 58, 8.1 (0.63), 52% 43% Black 47% White 10% Other		Moderate	Three training conditions and one control	RCT (1) Pre-test (2) Field test (3) Training (4) Post-test (5) Post-field (6) Follow-up test (7) Follow-up field	Pedestrian knowledge Pedestrian cognition Pedestrian behavior	United States
Schwebel, Shen, et al. (2016)*,b (Study 16)	Child sample: 59, 7.9 (.67), 54% 39% Black 59% White 2% Other or Biracial	Adult Sample Comparisons: N = 33, 21.8 (5.44), 58% 33% Black 52% White 15% Other N = 96, 21.7 (5.15), 61% 30% Black 50% White 20% Other N = 108, 20.7 (4.20), 58% 38% Black 44% White 18% Other		Moderate	N/A	Pre/post (1) Pre-test (2) Training (3) Post-test	Pedestrian cognition Pedestrian behavior	United States
*Schwebel et al. (2018) (Study 17)	56, 9.3 (0.30), 45% 89.3% Han ethnicity	N/A	N/A	High	N/A	Pre/post (1) Pre-test (2) Training (3) Post-test	Pedestrian cognition Pedestrian behavior	People’s Republic of China
Thomson et al. (2005) (Study 18)	129, 7.17 (No SD), 9.14 (No SD), 11.14 (No SD) No sex data No race or ethnicity data	Control—pre/post: 49, 7.2, 9.1, and 11.1 (No SDs) Control—pre/post/follow-up: 46, 7.2, 9.1, and 11.1 (No SDs) Intervention: 34, 7.1, 9.2, and 11.2 (No SDs) No sex data		Low	Control (pre/post only), control (pre/post/follow-up), and intervention	RCT (1) Pre-test (2) Training (3) Post-test (4) Follow-up test	Pedestrian knowledge Pedestrian cognition Pedestrian behavior	United Kingdom
Tolmie et al. (2002) (Study 19)	50, 6.67 (No SD), 36% No race or ethnicity data	Adult guidance: 18 Peer discussion: 18 Control: 14 No age or sex data		Low	Control, adult guidance—1:1 with an adult, peer discussion—peers worked in groups of 3	RCT (1) Pre-test (2) Training (3) Post-test	Pedestrian knowledge Pedestrian cognition Pedestrian behavior	United Kingdom
Tolmie et al. (2005) (Study 20)c	63, 6.5 (No SD), 38% No race or ethnicity data	Adult–child: 24 Adult–group: 24 Control: 15 No age or sex data		Low	Control, Adult-Child – 1:1 With an Adult, and Adult-Group – Peers Worked in Groups of 3 With an Adult	RCT (1) Pre-test (2) Training (3) Post-test	Pedestrian knowledge Pedestrian cognition Pedestrian behavior	United Kingdom

Note. Superscripts indicate merging because the publications report data from the same study.

a

Bart et al. (2006, 2008).

b

Schwebel & McClure (2014a, 2014b); Schwebel, McClure, et al. (2014), and Schwebel, Shen, et al. (2016).

c

Tolmie et al. (2005) report internally on two studies for the singular paper, but the samples are the same, therefore, they are recorded as one study.

*

Indicates studies included in the meta-analysis.

Last, moderator and subgroup analyses were conducted. Using the Meta Regression 2 feature in CMA3.0, we completed single predictor meta-regression models to analyze the influence of each moderator (mean age and percentage of female participants) on each outcome. Then, we conducted subgroup analyses as an alternative to moderation analyses among the three “immersion” levels for each outcome to descriptively examine differences in outcome variables based on immersion level. Applying subgroup analyses for level of immersion allowed us to evaluate the dispersion or variability of effect sizes across studies represented for each outcome at each level of immersion. A significant amount of dispersion suggests that the studies included at the designated immersion level for an outcome (low immersion for unsafe crossings) presented a wide variety of effect sizes. Subgroup analyses also permitted us to compare differences in effect size based on immersion level per outcome. This was assessed using a Q-test based on analysis of variance. A significant Q-test suggests that effect size differed significantly between levels of immersion for an outcome.

Results

Qualitative Synthesis

Basic Characteristics of the Included Studies

A total of 20 studies (reported in 24 articles) met eligibility criteria for the qualitative synthesis. Table I summarizes basic characteristics of the included studies. Additional intervention details (including study design, comparison group choice for interpreting individual effect sizes, and safety intervention topics) and VR features (including computer software, monitor set up, level of immersion) are listed in Supplementary Table 2. Interventions took place in a variety of settings, including schools, churches, community centers, and at home. Training structure varied also by study. The number of training sessions ranged from 1 to 25, the length of training sessions from 15 to 45 min, and the number of crossings per training sessions from 3 to 24. Traffic patterns presented during training also differed by study, with some including multiple stages of difficulty and others only including one level. Difficulty level change involved variation in both speed and density of the traffic (e.g., Light—25 mph, 8 vehicles/min, Moderate—30 mph, 12 vehicles/min, Heavier—35 mph, 16 vehicles/min). Feedback on how the child crossed the street presented as both virtual feedback via a verbal or visual message on the VR monitor or in-person commentary by a research assistant, parent, or peer. See Supplementary Table 2 for additional details.

Sample Characteristics

A total of N = 1,896 unique participants were represented across all studies. Data were collected in eight nations, with equal numbers of studies conducted in the United States and China (k = 4, 20.00% of studies each), followed by Australia and Canada each at k = 3 (15.00%). Additional information on all eight countries appears in Table I. The age of child participants in the studies ranged from 4 to 15 years. Of the 20 included studies, 15 (75.00%) provided sex statistics, with sex distributions ranging from 36% to 64% female in the overall samples.

Study Design and Intervention Characteristics

Of the included studies, 13 (65.00%) were randomized interventions and seven (35.00%) used non-randomized pre/post-designs. Using immersion definitions from Kardong-Edgren et al. (2019), eight (40.00%) studies used VR interventions with a VR-Low immersion level, seven (35.00%) a VR-Moderate immersion level, and five (25.00%) a VR-High immersion level. The structure of the VR intervention, including factors like the number of training sessions, length of the training sessions, and feedback delivered during training, varied widely across the studies, as detailed in Supplementary Table 2.

Type of Outcomes

Thirteen studies (68.42%) measured pedestrian safety knowledge (Studies 1, 5–12, 14, 16, 18–20). A general written or oral pedestrian safety test was the most common method to evaluate safety knowledge (k = 6, 46.15% of studies), closely followed by hazard identification (k = 5, 38.46%) and route selection (k = 5, 38.46%) tasks. Fifteen studies (78.95%) evaluated pedestrian-relevant cognition (Studies 1, 3–4, 6–7, 9–13, 15–19); of them, 11 (73.33%) reported data related to attention to traffic, three (20.00%) reported on time to contact, and seven (46.67%) reported start delay times. Finally, 14 studies (73.68%) included assessment of pedestrian behaviors (Studies 1–4, 9–10, 12–19), with seven (50.00%) documenting safe crossings and 11 (78.57%) unsafe crossings. Supplementary Table 2 provides details about these outcomes and the studies including them.

Risk of Bias Assessment

For the 17 papers (13 studies) using randomized interventions (Studies 1–3, 7, 10–14, 16, 18–20), risk of bias ratings were made in five categories: random sequence generation, allocation concealment, blinding of outcome assessment, incomplete outcome data, and selective reporting. Ratings for the randomized and non-randomized studies are reported in Tables II and III, respectively. A large portion of the papers demonstrated low risk of bias across the domains. Specifically, of the 17 papers, the following had low ratings in each category: 12 for random sequence generation, 14 for allocation concealment, 11 for blinding of outcome assessment, 16 for incomplete outcome data, and 13 for reporting bias. Some concerns about bias were observed for three papers on random sequence generation and allocation concealment, six on blinding of outcome assessment, one on incomplete outcome data, and four on selective reporting. Only two papers had high risk based on random sequence generation. Given these ratings, most included randomized studies demonstrate minimal risk of bias.

Table II.

Risk of bias ratings for randomized studies

Study (study no.)	Selection bias (randomization)	Selection bias (allocation)	Detection bias	Attrition bias	Reporting bias
Arbogast et al. (2014)(1)	Some concerns	Some concerns	Low	Low	Low
Bart et al. (2006)(2)	Low	Low	Low	Low	Low
Bart et al. (2008)(2)	Some concerns	Low	Low	Low	Low
Congiu et al. (2007)(3)	Low	Some concerns	Low	Low	Some concerns
Gu and Sosnovsky (2017)(7)	Some concerns	Low	Some concerns	Low	Some concerns
Luo et al. (2021)(10)	Low	Low	Some concerns	Low	Low
McComas et al. (2002)(11)	Low	Some concerns	Some concerns	Low	Some concerns
Morrongiello et al. (2018)(12)	Low	Low	Low	Low	Low
Oxley et al. (2008)(13)	Low	Low	Low	Low	Low
Remenyi et al. (2003)(14)	High	Low	Some concerns	Low	Some concerns
Schwebel & McClure (2014b) (16)	Low	Low	Low	Low	Low
Schwebel & McClure (2014a) (16)	Low	Low	Low	Low	Low
Schwebel, McClure, et al. (2014)(16)	Low	Low	Low	Low	Low
Thomson et al. (2005)(18)	Low	Low	Some concerns	Low	Low
Tolmie et al. (2002)(19)	High	Low	Some concerns	Some concerns	Low
Tolmie et al. (2005), Study 1 (20)	Low	Low	Low	Low	Low
Tolmie et al. (2005), Study 2 (20)	Low	Low	Low	Low	Low

Note. Cochrane risk-of-bias tool for randomized trials (RoB 2) is recommended for use in Cochrane reviews. Judgements can be “Low” or “High” risk of bias, or can express “Some concerns.”

Table III.

Risk of bias ratings for studies with non-randomized designs

Study (Study #)	Confounding bias	Selection bias	Information bias	Confounding bias	Selection bias	Information bias	Reporting bias
	(pre)	(pre)	(at)	(post)	(post)	(post)	(post)
Feng et al. (2020)(4)	Moderate	Low	Low	Low	Moderate	Moderate	Low
Glang et al. (2005)(5)	Moderate	Low	Low	Moderate	Moderate	Moderate	Low
Gu et al. (2015)(6)	Moderate	Low	Low	Moderate	Moderate	Low	Low
Khan et al. (2021)(8)	Moderate	Moderate	Low	Moderate	Low	Low	Low
Luo et al. (2020)(9)	Moderate	Low	Low	Low	Low	Moderate	Low
Schwebel, Combs, et al. (2016)(15)	Low	Low	Low	Low	Low	Low	Low
Schwebel, Shen, et al. (2016)(16)	Low	Low	Low	Low	Low	Low	Low
Schwebel et al. (2018)(17)	Moderate	Low	Low	Moderate	Moderate	Moderate	Low

Note. Orange = pre-intervention bias; Green = at-intervention bias; Blue = post-intervention bias. The Cochrane Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) tool is recommended for assessing the risk of bias in non-randomized studies of interventions included in Cochrane Reviews. Based on answers to the signaling questions, judgements for each bias domain, and for overall risk of bias, can be “Low,” “Moderate,” “Serious,” or “Critical” risk of bias.

For the eight papers (seven studies) using nonrandomized interventions (Studies 4–6, 8–9, 15–17), risk of bias ratings were made in seven categories: confounding, selection of participants into the study, classification of interventions, missing data, deviations from intended interventions, measurement of the outcome, and selection of reported results. As detailed in Table III, evidence for risk of bias was mixed across domains. Moderate risk was noted for six on pre-intervention confounding, one for selection of participants into the study, four for missing data, four for deviations from intended interventions, and four for measurement of the outcome. Low risk was rated for every paper regarding classification of interventions and selection of reported results. Additionally, low risk was rated for two for pre-intervention confounding, seven on selection of participants into the study, four for missing data, four for deviations from intended interventions, and four studies for measurement of the outcome. Given these ratings, most included non-randomized studies demonstrate variable risk of bias ranging from low to moderate concern.

Meta-Analysis: Interventional Effect on Outcomes

A total of 12 studies qualified for inclusion in the meta-analysis (Studies 1–2, 5, 7–12, 15–17). Together, those 12 studies included 1,323 participants. A meta-analysis of the overall effect of VR interventions to teach children to cross streets safely was conducted examining outcomes of pedestrian knowledge, pedestrian-related cognition, and pedestrian behavior. Refer to Tables IV and V for detailed results of the meta-analysis, as well as the forest plots included in Supplementary Figures 1–6. Heterogeneity indicators I² were 67.91% for pedestrian knowledge, 70.57% for attention to traffic, 86.50% for time to contact, 41.77% for start delay, 59.00% for safe crossings, and 35.89% for unsafe crossings, suggesting most outcomes were highly heterogeneous. Two outcomes, start delay and unsafe crossings, had somewhat lower levels of heterogeneity but for consistency we maintained the random effects model application across all outcomes.

Table IV.

Study-Level Effect Sizes and Moderators for Studies Included in the Meta-Analysis.

		Effect size and 95% CI			Moderators
Study	Outcome	g	LL	UL	Age (M)	Sex (% F)	Immersion Type
Arbogast et al. (2014)(1)	Safety Knowledge	0.51	0.29	0.72	8.01	54	Low
	Attention To Traffic	−0.13	−0.38	0.12
	Safe Crossings	0.50	0.06	0.94
Bart et al. (2004, 2006)(2)	Safe Crossings	1.01	0.21	1.99	9.30	64	Moderate
	Unsafe Crossings	1.70	0.73	2.67
Glang et al. (2005)(5)	Safety Knowledge	1.07	0.67	1.49	6.94	42	Low
Gu and Sosnovsky (2017)(7)	Attention To Traffic	0.57	0.10	1.04	8.00	47	Low
Khan et al. (2021)(8)	Safety Knowledge	1.23	0.26	2.20	N/A	N/A	Moderate
Luo et al. (2020)(9)	Safety Knowledge	0.38	0.16	0.61	N/A	51	High
	Attention To Traffic	0.14	−0.08	0.36
Luo et al. (2021)(10)	Safety Knowledge	0.88	0.42	1.35	7.85	42	High
	Attention To Traffic	0.57	0.12	1.03
	Safe Crossings	0.44	−0.01	0.89
McComas et al. (2002)(11)	Safe Crossings	0.15	0.00	0.29	10.50	62	Moderate
Morrongiello et al. (2018)(12)	Unsafe Crossings	0.71	0.18	1.24	8.77	50	High
Schwebel, Combs, et al. (2016)(15)	Attention To Traffic	0.26	−0.16	0.68	8.01	51	Moderate
	Start Delay	0.55	0.11	0.99
	Time To Contact	−0.01	−0.51	0.31
	Unsafe Crossings	0.27	−0.15	0.69
Schwebel & McClure (2014b) (16)	Attention To Traffic	0.30	−0.07	0.66	7.90	54	Moderate
Schwebel, McClure, et al. (2014)(16)	Start Delay	0.00	−0.36	0.36
	Unsafe Crossings	0.46	0.10	0.83
Schwebel, Shen, et al. (2016] (16)a	Attention To Traffic	−0.40	−0.77	−0.03	7.90	54	Moderate
	Start Delay	0.41	0.04	0.78
	Time To Contact	−0.41	−0.78	−0.04
	Unsafe Crossings	0.44	0.07	0.82
Schwebel et al. (2018)(17)	Start Delay	0.46	0.19	0.74	9.00	45	High
	Time To Contact	0.46	0.19	0.74
	Unsafe Crossings	0.46	0.19	0.74

Note. g = Hedge’s g; CI = Confidence Interval; LL = lower limit of the confidence interval; UL = upper limit of the confidence interval; % F = percent female.

a

Dashed line is to indicate different outcomes used for Schwebel, Shen et al. (2016) paper, but all three papers are represented as Study 16.

Table V.

Summary of Meta-Analytic Results Separated by Outcome

	Number of studies	Effect size and 95% CI			Heterogeneity
Type of outcome	k	g	LL	UL	I2
Pedestrian knowledge	5	0.70	0.42	0.98	67.91
Pedestrian cognition
Attention to traffic	7	0.16	−0.08	0.40	70.57
Time to contact	3	0.002	−0.55	0.55	86.50
Start delay	4	0.35	0.12	0.59	41.77
Pedestrian behavior
Safe crossings	4	0.40	0.08	0.71	59.00
Unsafe crossings	6	0.51	0.30	0.73	35.89

Note. Bolded confidence intervals indicates significance. I² = Heterogeneity was estimated using I² statistics, which are defined as the percentage of variance due to heterogeneity in the total variance. An I² of 25% is considered low, 50% moderate, and 75% high. g = Hedge’s g; CI = confidence interval; LL = lower limit of the confidence interval; UL = upper limit of the confidence interval.

Pedestrian knowledge results (five studies, five effect sizes) indicated a statistically significant and positive effect of small to medium magnitude (g = 0.70, 95% CI = [0.42, 0.98]) for VR pedestrian safety interventions on children’s pedestrian safety knowledge following intervention.

For pedestrian cognition (10 studies, 10 effect sizes), results were separated by the three individual outcomes: attention to traffic (seven studies, seven effect sizes), start delay (four studies, four effect sizes), and time to contact (three studies, three effect sizes). Start delay analysis indicated a significant and positive effect of small to medium magnitude (g = 0.35, 95% CI = [0.12, 0.59]), but the VR interventions exerted no significant impact on attention to traffic (g = 0.16, 95% CI = [−0.08, 0.40]) or time to contact (g = 0.002, 95% CI = [−0.55, 0.55]).

Regarding pedestrian behavior (10 studies, 10 effect sizes), results for safe crossings (four studies, four effect sizes) indicated a significant and positive effect of small to medium magnitude (g = 0.40, 95% CI = [0.08, 0.71]) following VR training. Unsafe crossings (seven studies, seven effect sizes) results indicated a significant and positive effect of small to large magnitude (g = 0.51, 95% CI = [0.30, 0.73]).

Meta-Analysis: Moderator Analyses

The rightmost columns of Table IV describe moderator data for each included study and Table VI includes results of the single predictor meta-regression models examining moderators on intervention effects. Two moderators were considered, age and percentage of female participants. Both age and percentage of female participants significantly moderated the effect of VR interventions on pedestrian knowledge, such that samples with younger children and higher percentages of females demonstrated greater improvements in safety knowledge after VR interventions. Percentage of female participants also trended toward significantly moderating the effect of VR intervention on attention to traffic (p = .05), such that samples with higher percentages of females demonstrated greater improvements in attention to traffic.

Table VI.

Meta-Regression Analysis of Moderating Effects of Age and Sex on Pedestrian Safety Outcomes

Meta-regression analyses
Moderator	Pedestrian knowledge	Attention to traffic	Start delay	Safe crossings	Unsafe crossings
Age	−0.41 [−0.73, −0.08]	−0.12 [−4.90, 4.67]	0.22 [−0.32, 0.76]	−0.12 [−0.36, 0.11]	0.31 [−0.13, 0.74]
Sex (% female)	−0.42 [−0.07, −0.01]	−0.05 [−0.09, 0.00]†	−0.03 [−0.07, 0.01]	−0.00 [−0.05, 0.04]	0.03 [−0.02, 0.07]

Note. Bolded coefficients and confidence intervals indicate statistically significant moderating effects.

†

p = .05.

Table VII includes the results of the subgroup analyses examining the differential effects of the level of VR immersion (low, medium, or high) on pedestrian training outcomes. For each level of immersion, effect sizes within the level varied from small to large based on the pedestrian safety outcome of interest. Overall, only time to contact revealed a significant difference in effect size based on immersion level (Q_between = 12.26, p < .001), with high immersion demonstrating a stronger, positive effect on time to contact performance compared to moderate immersion. In general, moderate levels of immersion demonstrated the greatest effect sizes (ranging from 0.46 to 1.28) for all outcomes except attention to traffic and time to contact. There was also evidence of significant dispersion (or heterogeneity) of effect sizes across studies within the immersion levels. Safe crossings had significant dispersion at the low level of immersion, start delay and time to contact had dispersion at the high level of immersion, unsafe crossings demonstrated dispersion within both moderate and high levels of immersion, and pedestrian knowledge demonstrated dispersion all three levels of immersion.

Table VII.

Subgroup Analysis of Level of Immersion on Pedestrian Safety Outcomes

Subgroup analyses
Moderator	Pedestrian knowledge	Attention to traffic	Start delay	Time to contact	Safe crossings	Unsafe crossings
Immersion	(g, CI)	(g, CI)	(g, CI)	(g, CI)	(g, CI)	(g, CI)
Low	0.76 [0.20, 1.32]	0.19 [−0.50, 0.88]	n/a	n/a	0.50 [0.06, 0.94]	n/a
Moderate	1.28 [0.26, 2.20]	0.05 [−0.41, 0.88]	0.46 [−0.02, 0.63]	−0.27 [−0.57, 0.04]	0.52 [−0.40, 1.43]	0.53 [0.18, 0.89]
High	0.59 [0.11, 1.08]	0.31 [−0.10, 0.73]	0.30 [0.19, 0.74]	0.46 [0.19, 0.74]	0.44 [−0.01, 0.89]	0.52 [0.27, 0.76]
Q-value between	1.34	0.70	0.54	12.26	0.04	0.01

Note. Bolded g values and confidence intervals indicate statistically significant dispersion within studies at that immersion level. Bolded Q-values indicate statistically significant differences in effect sizes between immersion levels.

Discussion

Our systematic review examined the efficacy of VR pedestrian safety interventions to teach children to cross streets more safely. Across the 20 included studies (24 articles), VR-based pedestrian safety interventions were generally found to be effective. We found statistically significant and positive effects of VR interventions on children’s pedestrian safety knowledge, pedestrian-related cognition, and pedestrian behavior, mostly at the small to medium effect size level. These findings extend previous meta-analytic results on behavioral pediatric pedestrian safety interventions (Schwebel, Barton, et al., 2014) and recent conclusions from a systematic review of pediatric unintentional injury preventions, including pedestrian safety programs (Bou-Karroum et al., 2022).

The efficacy of VR based pedestrian safety interventions varied somewhat across outcome measures, confirming suggestions that street-crossing is a complex task requiring multiple cognitive-perceptual processes that may respond differently to interventions (Schwebel, Barton, et al., 2014). Ultimately, a successful intervention must train children in all relevant skills needed to cross streets safely and thus the behavior outcomes—presence of safe crossings and absence of unsafe crossings—are arguably most important for translation to improved public health. We found small to medium magnitude effect sizes for these outcomes (g = .40 for safe crossings and .51 for unsafe crossings), indicating training in VR helps children become safer pedestrians.

Two analyses yielded null results, those assessing the cognition-based outcomes of attention to traffic and time to contact. One interpretation of these results is that VR interventions are more effective in shaping certain aspects of pedestrian safety than others (Schwebel, Barton, et al., 2014), but we propose alternative explanations. Attention to traffic is consistently present in children from a young age (Schwebel, McClure, et al., 2014) and children as young as 3 years are taught to “look left, right, left” at street crossings (National Highway Traffic Safety Administration, n.d.). Thus, it may be that VR training is unnecessary to teach attention to traffic because it is already honed by the developmental stage when children are capable of learning to cross streets safely. Instead of rehearsing the act of attending to traffic in VR settings by looking both ways, children must rehearse the cognitive process of perceiving and processing the stimuli they are attending to in both directions. Simply looking at traffic does not yield safe crossing; a safe pedestrian must look at but also process the traffic they are seeing and then act upon that cognitive processing. This complex process develops later in childhood and may become more efficient and accurate through training in VR settings. The simpler cognitive process of attending to traffic in both directions may develop through practice and training earlier in childhood.

The null results for the time to contact outcome may stem from the process of training in safety through VR programs. Novice pedestrians are likely to be overly cautious, waiting for very large gaps in traffic before crossing and thus having large time to contact measures at baseline. As children become more competent pedestrians through training, they learn to select tighter traffic gaps to cross within, and thus may have shorter—but still safe—time to contact outcomes. Although time to contact provides a useful measure of pedestrian-related cognitive skill, it may not follow a linear change trajectory as children become safer pedestrians. Dangerously short time to contact outcomes (including time to contact outcomes of 0 that are typically scored in a crash) diminish, but moderately short outcomes may become increasingly common as children develop street-crossing competence and confidence.

Our moderator analyses were designed to examine the impact of child age and sex on VR-based training efficacy. We found that age had a significant impact on safety knowledge outcomes, with younger children showing larger gains. This result should be interpreted cautiously given the restricted age range included (most studies focused only on children ages 7–8). However, since younger children generally demonstrate riskier street crossing behaviors (Bart et al., 2008; Barton et al., 2012; Barton & Schwebel, 2006, 2007), the results are encouraging and may indicate the possibility of using VR to reduce pedestrian risk among more vulnerable younger children. It also is possible that the result indicates greater amount of learning required by young children to reach levels of street crossing ability comparable to older children; some studies may have experienced a ceiling effect on knowledge assessments among older children.

We found a significant moderating effect of VR training efficacy by sex regarding pedestrian knowledge. Further, although not statistically significant based on a p value <.05, we also saw a trend toward significance for sex moderating the effect of VR training on attention to traffic (p = .052). Both patterns suggest that samples with higher percentages of females demonstrated greater improvements. Given that boys experience more pedestrian injuries (West et al., 2021) and take more risks when crossing streets (Barton & Schwebel, 2007), it is possible that they require a longer and more in depth training protocol to obtain training gains equivalent to girls.

Finally, subgroup analyses were conducted to descriptively examine patterns between VR immersion and training efficacy. Children seemed to learn pedestrian safety equally well with cruder, less immersive, and often older two-dimensional programs displayed on computer monitors compared to recent, modern, and fully immersive goggle-based VR training programs. This finding offers promise for dissemination of interventions, as more basic and lower-cost VR simulations may be sufficient and equally effective to teach children about pedestrian safety compared to more expensive high-end, fully immersive, and highly realistic simulations. One possible explanation for this finding considers the skills being learned. VR is hypothesized to teach children street-crossing skills through repeated practice at the complex cognitive-perceptual task of judging moving traffic and determining when it is safe to cross within traffic gaps. If true, it may not matter whether children are looking at oncoming traffic that is highly realistic and presented in a fully immersive 3-dimensional Oculus environment versus oncoming traffic represented by simple rectangles on a 2-dimensional computer screen. The level of VR immersion and simulation quality may impact children’s engagement and enjoyment, but from a cognitive training perspective the same skills are practiced and learned.

Implications

Overall, our results offer promise for injury prevention and public health planning. VR is no longer a futuristic and expensive technology available only to the privileged (Bailey & Bailenson, 2017). It is widely available, relatively inexpensive, and can be delivered on smartphones that are now ubiquitous in global society (Somaiya, 2015; Wohlsen, 2015). We now have the capacity to disseminate VR-based training programs to children widely and easily, and the evidence to suggest that such programs will help children learn to cross streets safely. These programs might be delivered through schools, faith-based organizations, community organizations, pediatrician offices, or any number of other avenues. Such efforts might prioritize communities where children frequently walk and will help us address one of the leading causes of pediatric mortality in the United States and across the world, pediatric pedestrian injuries.

Challenges remain, however. First, it is unclear at what age pedestrian safety training should be delivered. Our review included children ranging in age from 4 to 15. We are still learning when children are actually able to learn to cross streets safely, given sufficient practice in VR or other settings. Evidence collected in a bicycling VR setting suggests child bicyclists may not behave like adult cyclists until age 14 (O'Neal et al., 2018). The American Academy of Pediatrics suggests children are generally safe pedestrians by age 10 (Committee on Injury, Violence, and Poison Prevention, & American Academy of Pediatrics, 2009). Continued research to uncover the age when children can learn to be safe pedestrians—meaning they engage with traffic in a manner comparable to adults—is needed.

Second, it is unclear how much training children require. We do not yet understand when children are safe pedestrians, nor how to know what amount of training in VR settings is sufficient for children to become safe when crossing streets independently and unsupervised. Efforts to create assessment tools, perhaps in VR, would be logical. We need ways to measure children’s pedestrian competence before allowing them to cross streets alone, as training children partially in VR and then allowing them to cross streets independently could create iatrogenic effects that unintentionally increase risk of injury rather than decreasing it.

Last, we must extend existing VR programs to represent the full complexity of real-world street environments. Most research to date has been conducted in relatively simple simulated street environments presenting one or two lanes of traffic, sometimes with traffic moving only in one direction. Simulations of inclines and declines, curves, and obstacles such as parked cars or shrubbery that impede vision of oncoming traffic are largely absent from the published literature. Also absent are more complex crossings (e.g., three to six lanes of traffic, common especially in global settings), more complex situations (e.g., addition of motorcycles, bicyclists, scooters, emergency vehicles, and all the other complexities that appear on global roadways), and more complex decisions (not just selecting safe traffic gaps but also selecting safer routes to destinations) and also adding time pressure and other emotional components of safe street crossing (Juzdani et al., 2020; Morrongiello et al., 2015; Wang et al., 2021). Continued research is needed, as is continued development of VR simulations. All efforts should consider both locally relevant contextual environments and potential generalizability of findings to other locales. Process evaluation should be incorporated into research designs.

Limitations

The present study had limitations. First, like most meta-analytic studies we selectively included a single measure to represent larger pedestrian constructs, even when multiple measures were reported for an outcome. This leads to a loss of available information but best captures the larger construct by merging representative data across the included studies. Second, there was substantial heterogeneity in the outcomes included in the larger constructs. For example, safety knowledge was represented by a variety of assessments that ultimately were classified into a single construct of pedestrian safety knowledge. This merging may oversimplify outcomes but is commonly conducted in meta-analyses to capture larger constructs with available data. Future research might work to standardize assessment using the most valid strategies to evaluate pedestrian safety outcomes.

A third limitation relates to study design and available data across the included studies, which led to small sample sizes across analyses. This may be especially problematic for the meta-regression analyses. Although CMA does not impose a minimum restriction for number of studies included in a meta-regression analysis, our sample sizes were small and should be interpreted with caution. Conceptually, descriptive data match the results we obtain and together the insights provide a solid baseline for consideration in future intervention development. Fourth, few included studies included real-life assessments or “field” tests to evaluate translation of learning into real-life pedestrian settings. Ethical standards complicate such assessments, as children cannot be placed on actual live roadways, but real-life simulations and observations are possible (Bart et al., 2008; Schwebel, McClure, et al., 2014; Schwebel et al., 2017, 2018) and should be considered in future research. Last, we would have liked to include an evaluation of the most effective training structure for VR pedestrian safety interventions—number of training sessions needed to achieve learning, length of each session, and type of feedback during training. Many published studies did not include specific details on these factors, however, and when it was reported the practices varied quite widely and were difficult to quantify. We therefore omitted this analysis from our review. Future research might consider ways to evaluate these questions using meta-analytic strategies.

Conclusions

VR offers an encouraging platform to teach children pedestrian safety. Training in VR appears to positively impact pedestrian safety knowledge, start delay, and safe and unsafe crossings. Additional research is needed, but our results suggest widely accessible formats such as non-immersive computer simulations may be sufficient for learning; expensive and highly immersive VR environments may not be necessary to achieve desired child pedestrian safety training goals. Continued research, refinement of VR training programs, and translation into practice are recommended.

Acknowledgments

The authors thank the UAB Lister Hill librarians for their extensive assistance in training the first author on conducting the literature search for the systematic review. Thanks also to all the researchers and laboratories completing this important work, the participating community partners supplying resources and space for pedestrian trainings, and the families willing to engage in the studies.

Author Contributions

Casie Morgan (Conceptualization [equal], Data curation [lead], Formal analysis [lead], Methodology [equal], Project administration [lead], Visualization [lead], Writing – original draft [lead], Writing – review & editing [lead]), Lindsay Stager (Data curation [supporting], Methodology [supporting], Writing – original draft [equal], Writing – review & editing [equal]), David C. Schwebel (Conceptualization [equal], Funding acquisition [lead], Supervision [equal], Writing – review & editing [equal]), and Jiabin Shen (Formal analysis [equal], Methodology [supporting], Supervision [equal], Validation [supporting], Visualization [supporting], Writing – review & editing [equal]).

Funding

This work was supported by the Eunice Kennedy Shriver National Institute of Child Health & Human Development of the National Institutes of Health under Award Number R01HD088415. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Supplementary Data

Supplementary data can be found at: https://academic.oup.com/jpepsy.

Conflicts of interest

None declared.