A Measure of Caregivers’ Distress-Promoting Behaviors During Toddler Vaccination: Validation of the OUCHIE-RV

Abstract

Objective

The current study discusses the development and preliminary validation of an observational measure of caregivers’ distress-promoting behavior (OUCHIE-RV) in the context of toddler routine vaccination.

Methods

Measure validation was based on a sample of caregiver–toddler dyads (N = 223) from a longitudinal cohort-sequential study who were observed during toddlers’ 12-, 18-, or 24-month routine vaccinations. Validity was assessed using correlations and cross-lagged path analysis and associations were tested between OUCHIE-RV composite scores and toddlers’ pain-related distress as well as caregivers’ state anxiety, soothing behaviors, heart rate, and heart rate variability. Interrater reliability was examined using intraclass correlations (ICC) and kappa coefficients.

Results

Results of a cross-lagged path analysis indicated positive concurrent (β = .27 to β = .37) and cross-lagged predictive associations (β = .01 to β = .34) between OUCHIE-RV composite scores (reflecting the frequency and intensity of caregivers’ distress-promoting behavior) and toddlers’ pain-related distress. OUCHIE-RV composite scores were negatively associated with caregivers’ concurrent use of physical comfort (r = −.34 to −.24) and rocking (r = −.36 to −.19) and showed minimal associations with caregivers’ use of verbal reassurance (r = .06 to .12), state anxiety (r = −.02 to r = .09), heart rate (r = −.15 to r = .05), and heart rate variability (r = −.04 to r = .13). Interrater reliability was strong (Cohen’s k = .86 to .97, ICC = .77 to .85).

Conclusions

Findings provide support for the validity and reliability of the OUCHIE-RV as a research tool for measuring caregiver behaviors that promote toddlers’ pain-related distress during routine vaccinations and contribute to a better understanding of the dynamics of caregiver–toddler interaction in acute pain contexts.

acute pain, infancy and early childhood, measure validation, parenting

Introduction

Parents and other primary caregivers play a central role in supporting infants’ and young children’s ability to cope with pain-related distress (Sobol-Kwapińska et al., 2020). Efforts to better understand caregivers’ influence on acute pain outcomes in early childhood have led to the development of behavioral coding systems which characterize and quantify various aspects of caregiving behavior during painful medical procedures, such as the Child-Adult Medical Procedure Interaction Scale (Blount et al., 1989) and the Measure of Adult and Infant Soothing and Distress (MAISD; Cohen et al., 2005). Most coding systems focus on measuring frequently observed aspects of caregiver behavior to examine how individual behaviors or clusters of behaviors relate to child pain outcomes. However, these coding systems have been suggested to be limited by a lack of a theoretical foundation and an underrepresentation of caregiver nonverbal behaviors (Bai et al., 2018). Therefore, the current study aimed to provide evidence to support the validity of a new researcher-oriented version of a measure of caregiver behavior during toddler vaccination, the Opportunities to Understand Childhood Hurt’s Inoculation Evaluation—Research Version (OUCHIE-RV) that addresses limitations of existing coding systems, incorporates current theory and research concerning the impact of insensitive or distress-promoting caregiver behaviors on pain-related distress (Pillai Riddell et al., 2022; Sobol-Kwapińska et al., 2020), and conforms to best practice guidelines for the development of behavioral coding systems in pediatric psychology (Chorney et al., 2015).

Historical and Theoretical Foundations of the OUCHIE Tools

The predecessor to the current OUCHIE-RV tool was the OUCHIE—Clinical Version (OUCHIE-CV; Pillai Riddell et al., 2018a). The fundamental difference between the current tool and its predecessor is that the earlier version was a clinical tool intended for in vivo (i.e., scored as the appointment was taking place) use by the vaccinating health professional, whereas the current measure is intended for more fine-grained post hoc coding based on video footage. The previous version focused on the presence or absence of eight core caregiver behaviors that exacerbated infant distress and were not captured by existing observational tools. These behaviors were identified based on consultation with experts in observational coding of caregiver sensitivity or infant pain-related distress, observation of thousands of infant vaccination appointments, and a series of discussion groups with clinicians who provide vaccinations (see Pillai Riddell et al., 2018a for more detail). The current measure (OUCHIE-RV) evolved from this foundation to provide more fine-grained data on the frequency and intensity of these distress-promoting behaviors for intended use in research contexts. Moreover, it evolved to focus on three behavioral domains (i.e., affect incongruence, attention diversion, physical distance) that were derived from the eight discrete behaviors.

The OUCHIE tools draw on core theoretical frameworks within the developmental psychology literature (i.e., Ainsworth’s [1974] concept of parental sensitivity and the Emotional Availability framework [Biringen, 2000]) and the pediatric pain literature (i.e., the Development of Infant Acute Pain Responding- Revised 2022 model [DIAPR-R 2022; Pillai Riddell et al., 2022]). The Emotional Availability framework and Ainsworth’s concept of parental sensitivity, which are both grounded in Attachment Theory (Bowlby, 1969/1982), define dimensions of caregiving responses that influence the quality of caregiver–child interactions in distress and non-distress contexts (Ainsworth et al., 1974; Biringen, 2000). A fundamental principle of both frameworks is that caregiver behavior must be understood in the caregiver–child dyadic context. According to this view, most discrete behaviors are not inherently sensitive or insensitive; rather, the appropriateness of behavior is defined by the degree of attunement or non-attunement between the child’s signaled need and the caregiver’s contingent response. This principle underlies a core feature of the OUCHIE tools, which is that caregiver behaviors are only coded when the child is in moderate to high pain-related distress. According to Attachment Theory, moderate to high distress behaviors (e.g., crying) are clear signals from the child of an acute attachment need for caregiver proximity and comfort. Focusing on moments of moderate to high distress allows for theoretically informed assumptions regarding which specific caregiver behaviors support or undermine adaptive regulation of pain-related distress.

The OUCHIE-RV behavior domains reflect caregiver responses that, based on theory and decades of pediatric pain research, have been demonstrated or hypothesized to maintain or exacerbate child distress in acute pain contexts (e.g., routine vaccinations). Fundamentally, the behaviors were selected because they communicate a lack of recognition or contingent responding on the part of the caregiver to the child’s distress signaling, presumably leading to a maintenance or escalation of distress. In addition, OUCHIE-RV behaviors may also provide insight into caregivers’ internal experiences of the vaccination context. For example, the DIAPR-R 2022 model (Pillai Riddell et al., 2022) delineates the biological, psychological, and social factors underlying caregivers’ behavioral responses to infants’ acute pain and suggests that caregivers’ physiological and emotional responses, triggered by witnessing their child’s distress, may directly impact caregivers’ behavioral responses to their child in pain.

The Current Study

The current study reports on the development and psychometric evaluation of the OUCHIE-RV. The study had two primary aims:

1. To examine the reliability of the OUCHIE-RV based on interrater reliability (Aim 1a) and internal consistency (Aim 1b).

2. To examine the concurrent validity of the OUCHIE-RV based on associations with toddlers’ pain-related distress.

Exploratory aims of the study included examining associations between the OUCHIE-RV and other aspects of caregivers’ response to their child’s painful procedure, including their use of soothing behaviors (i.e., physical proximity, rocking, and verbal reassurance), self-reported state anxiety, and measures of physiological arousal (i.e., heart rate [HR]) and regulation (i.e., high-frequency heart rate variability [HRV]).

We hypothesized that findings would provide support for the concurrent validity of the OUCHIE-RV as a measure of caregivers’ distress-promoting behavior, based on positive associations between OUCHIE-RV composite scores (higher values of which reflect greater frequency and/or intensity of distress-promoting behavior) and toddlers’ pain-related distress. In exploratory analyses, we expected that OUCHIE-RV composite scores would have positive associations with caregiver behaviors known to promote infant distress (i.e., verbal reassurance; Sobol-Kwapińska et al., 2020), negative associations with caregiver behaviors known to reduce distress (i.e., physical comfort, rocking), positive associations with caregivers’ state anxiety and physiological arousal, and negative associations with caregivers’ physiological regulation.

Methods

Participants

Participants were drawn from a larger longitudinal cohort-sequential study (N = 234) that followed healthy, typically developing toddlers and their caregivers during routine vaccination appointments across the second year of life (12, 18, and/or 24 months). Exclusion criteria included prematurity (<37 weeks’ gestation), neonatal intensive care unit stay during infancy, suspected or confirmed developmental delay, chronic illness, and caregiver self-reporting a lack of fluency in English. The current study used a subsample of 223 dyads with video data available for at least one appointment. For dyads who participated at multiple time points, only data from the final participation point were included in the current analyses to support a more even distribution of toddlers across the second year of life (i.e., 12, 18, 24 months). Dyads’ participation at a single time point (vs. multiple time points) was not significantly related to demographic variables (i.e., caregiver age, education, relation to child, ethnicity, or toddler sex).

In the analyzed sample (N = 223), toddlers (53% male) were on average 18 months old (M = 18.2, SD = 4.6). Caregivers ranged in age from 20 to 62 years (M = 36.2, SD = 5.3) and were primarily mothers (80%) but also included fathers (19%) and grandparents (1%). Additional demographic details are in Table I.

Table I.

Demographic Characteristics (N = 223)

	M (SD) or %
Child sex (%)
Male	53
Female	47
Relationship to child (%)
Mother	80
Father	19
Grandparent	1
Caregiver education (%)
Graduate school or professional training	48
University graduate (4 years)	34
Partial university (at least 1 year)	2
Trade school/community college	14
High school graduate	1
Self-reported heritage culture (%)
North American	22
Asian	24
European	25
Latin, Central, or South American	4
African	1
Caribbean	1
Oceanian	1
Other ethnic/cultural origins	8
Mixed	14
Self-reported acculturation to North American culturea	7.9 (2.0)
Self-reported acculturation to self-reported heritage culturea	6.3 (2.8)

a

Ranges from 0 to 10, with higher scores reflecting greater identification with mainstream North American or self-reported heritage culture.

Procedure

Ethical approval was obtained from the Research Ethics Board at the participating university. Recruitment and data collection occurred in two large community pediatric clinics in urban areas of a large, multicultural city (Toronto, Canada). Caregivers of toddlers scheduled for a vaccination appointment were made aware of the study upon entering the pediatric clinic. If the caregiver agreed to be approached by researchers, a research assistant explained the study and screened for inclusion and exclusion criteria. Informed consent was obtained from all participating caregivers. All data collection occurred between December 2015 and December 2019 by research assistants who were blinded to study hypotheses.

During the appointment, caregiver–toddler dyads were video recorded and both members of the dyad were connected to MindWare wireless ambulatory monitors (MW1000A), which recorded electrocardiography (ECG) data at a sampling rate of 500 Hz. Three ECG electrodes were placed on each caregiver and toddler over the right clavicle, under the bottom left rib, and under the bottom right rib (ground electrode). MindWare Biolab software (version 3.3) was used for continuous ECG acquisition and The Observer XT software was used to synchronize video and cardiac data capture. Video and cardiac data were collected beginning 2 min before the vaccination needle was given and ending up to 5 min after the vaccination needle. Caregivers provided state anxiety ratings at the beginning of the recording period (pre-needle) and at the end (post-needle). Once ECG monitoring equipment was in place, dyads were observed with minimal interference from the research team. As per standard practice in the participating clinics, all caregivers held their toddlers during the vaccination. Caregivers were able to move around unimpeded by the ECG monitoring equipment; however, given the small size of the clinic rooms, most caregivers’ locomotion was minimal. Vaccinations administered during the appointments included pneumococcal conjugate, meningococcal conjugate, measles/mumps/rubella, varicella, and/or diptheria/tetanus/pertussis/polio/haemophilus influenzae type B [DTaP-IPV-Hib]. Following their participation, caregivers received a pamphlet on evidence-based needle pain management strategies. A schematic of data collection timelines is provided in Figure 1. All data processing and coding were completed in lab using The Observer XT and MindWare HRV software (version 3.1.5) by separate coding teams who were blinded to participant data on other measures and study hypotheses. Data are available upon request.

Figure 1.

Schematic overview of data collection during the vaccination appointment. Note. FLACC = Face, Legs, Activity, Cry, Consolability scale; HR = heart rate; HRV = heart rate variability; OUCHIE-RV = Opportunities to Understand Childhood Hurt’s Inoculation Evaluation—Research Version; MAISD = Measure of Adult and Infant Soothing and Distress.

The current analysis involves caregiver soothing behavior and toddlers’ pain-related distress, caregiver cardiac responses, and caregiver state anxiety. Previous studies of this cohort have examined toddlers’ self-regulatory behaviors (Gennis et al., 2023) and behavioral and physiological reactivity (Waxman et al., 2020; Waxman et al., 2021) and caregiver–toddler physiological attunement (Di Lorenzo-Klas et al., 2023a, 2023b).

Measures

Demographic Questionnaire

A demographic questionnaire asked about caregiver age, self-reported heritage culture, education, relation to child, and toddler age and sex. Caregivers’ self-reported heritage culture was coded into the ethnic and cultural categories defined by Statistics Canada (2021), including North American, European, Caribbean, Latin/Central/South American, African, Oceanian, and “other” ethnic and cultural origins (e.g., Jewish, Christian, Muslim). Caregivers were also asked to rate the extent to which their way of life reflected mainstream North American culture and their heritage culture (respectively) on a 0–10 scale. This measure of acculturation allows researchers to better understand the extent to which mainstream North American culture and caregivers’ own heritage culture are reflected in caregivers’ daily life. It was adapted (see Pillai Riddell et al., 2018b) from the Vancouver Index of Acculturation (Ryder et al., 2000) and Berry’s (1992) bidimensional model of acculturation. Higher scores reflect greater identification with mainstream or heritage culture.

Caregivers’ Distress-Promoting Behavior

Measure Development and Coder Training. Caregivers’ use of distress-promoting behavior was measured using the OUCHIE-RV. Development of the OUCHIE-RV followed best practice guidelines for the development and modification of behavioral coding systems in pediatric psychology (Chorney et al., 2015), the relevant steps of which are described below.

To ensure the content validity of the OUCHIE-RV, item generation was informed by theory, prior research, and the clinical and research experience of the research team (see Introduction). The OUCHIE-CV behaviors (Pillai Riddell et al., 2018a) were classified into one of the three OUCHIE-RV behavior domains (i.e., affect incongruence, attention diversion, physical distance),¹ which were coded during discrete epochs across the post-needle period. This approach was chosen to balance capturing themes in caregiver behaviors with achieving temporal specificity, given our goal of developing a tool for use in research contexts.

Affect incongruence codes the frequency and intensity of caregiver affective responding that is inappropriate or incongruent in response to a distressed child. Affect incongruence may be communicated verbally (through the content or tone of vocalizations) or nonverbally (through facial expression) and may suggest fearful, angry, overly positive, or completely restricted (i.e., flat) caregiver affect. Attention diversion codes the frequency and intensity of caregivers diverting their attention away from their distressed child (e.g., by ignoring their child’s bids for attention or shifting their attention to someone else in the room), determined based on how caregivers direct their gaze, vocalizations, and other actions. Physical distance codes the frequency and intensity of lapses in caregivers’ close physical contact with their distressed child (e.g., caregiver separating from distressed toddler or holding toddler away from caregiver’s body in a way that prevents chest-to-chest contact).

A preliminary coding manual was developed and piloted with a team of undergraduate psychology student coders, a clinical psychology graduate student (SB), and a clinical health psychologist and behavioral scientist (RPR) by coding an initial subset of the sample. Operational definitions for the behavior domains were revised during the pilot coding phase and the coding manual was finalized. Coding was completed by one graduate student primary coder with extensive behavioral coding experience and two undergraduate psychology student coders with no prior behavioral coding experience prior to the pilot coding described above. Training occurred over an approximately 3-month period during which the lead coder (SB) provided weekly coding assignments requiring approximately 3 hr of weekly coding and coding discrepancies were reviewed during a 1-hr weekly consensus meeting. A description of the coding scheme and calculation of OUCHIE-RV composite scores follows.

Coding of Distress-Promoting Behavior During Toddler Moderate to High Distress. As aforementioned, only caregiver behavior that occurred during the toddler’s moderate to high distress was coded. To ensure that coders could reliably distinguish no or low distress from moderate to high distress, toddlers’ distress state (i.e., 0 = no or low distress, 1 = moderate or high distress) was coded during each 30-s epoch, based on the toddler’s vocalizations (i.e., the presence of full-lunged crying) using a definition from an established behavioral pain scale (Taddio et al., 1995). To measure the frequency and intensity of each of the three behavior domains (i.e., affect incongruence, attention diversion, and physical distance) over time, each of the three domains was rated on a 0–2 scale (0 = no evidence for the domain, 1 = some evidence for the domain, 2 = marked [i.e., high intensity] or pervasive [i.e., high frequency]) evidence for the domain) every 30 s during the 3 min after the vaccination needle. The full coding scheme is presented in Supplementary File 1. Coding time typically ranged from 10 to 20 min per video for trained coders. Twenty percent of the dataset was double-coded and reliability with the primary coder was examined (see Results section).

Calculation of Composite Scores.An overall OUCHIE-RV composite score was calculated for each of the first 3 min post-needle (i.e., Post 1, Post 2, and Post 3). Composite scores were calculated by averaging over all three behavior domains for the two epochs within each minute. Epochs during which toddlers were not in a state of moderate to high distress were not factored into the calculated score. Composite scores range from 0 to 2 and higher scores indicate more frequent or intense use of distress-promoting behavior in response to toddler distress during that particular minute.

Caregiver Physiological Arousal and Regulation

ECG data were processed using MindWare HRV 3.1.5 software, which derived caregiver HR based on the identification of R waves (Berntson et al., 1997) and high-frequency HRV based on spectral analysis of RR intervals (i.e., the intervals between successive heart beats or R waves) using frequency bands within the range of spontaneous respiration for adults (0.12–0.40; Cacioppo et al., 2000). Manual data inspection and artifact editing (i.e., in the event of software misidentification or equipment malfunction) were conducted by coders who were trained by an experienced primary coder and impacted <5% of data for all participants. Interrater reliability with the primary coder, calculated on a randomly selected 20% of the sample, was strong (intraclass correlations [ICCs] ranging from 0.96 to 0.99). Caregiver HR (reported as beats per minute [BPM]) and HRV (reported as the natural logarithm of milliseconds squared) were calculated for consecutive 30-s epochs during the minute before the needle and 3 min post-needle, which were then averaged within each minute to yield average caregiver HR and HRV at Baseline, Post 1, Post 2, and Post 3. Caregiver HR was used to index caregivers’ physiological arousal and HRV was used to index caregivers’ physiological regulation (Appelhans & Luecken, 2006).

Caregiver Posture

Because posture can influence physiological responses (Berntson et al., 2016), caregiver posture was coded from video footage throughout the same time frame as cardiac data collection for use as a covariate in subsequent analyses. Postural position (i.e., seated or standing) was continuously coded and used to calculate the percentage of time spent standing at Baseline, Post 1, Post 2, and Post 3.

Caregiver Pre- and Post-Needle State Anxiety

Caregivers rated their state anxiety on two occasions—once pre-needle and a second time approximately 3 min post-needle—by verbally responding to the question: “How worried are you right now on a scale of 1 to 10, where 1 is ‘no worry at all’ and 10 is ‘the most worry possible’?”, asked by a research assistant. Higher scores reflect more state anxiety. Previous research has supported the validity of subjective ratings of distress as a measure of acute emotional discomfort (Tanner, 2012).

Caregiver Soothing Behaviors

Caregiver soothing behaviors were coded using the MAISD (Cohen et al., 2005), a behavioral coding system used to code the frequency of discrete caregiver behaviors. Three soothing behaviors were analyzed in the current study, including physical comfort (i.e., rubbing, patting, or massaging child), rocking (i.e., swaying or bouncing child), and verbal reassurance (e.g., making reassuring comments to child such as “it’s ok”). Each behavior was coded as present or absent during 5-s epochs beginning immediately after the needle and continuing for 3 min. Proportion scores for each behavior, reflecting the proportion of epochs during which the behavior was observed out of all codable epochs, were calculated for Post 1, Post 2, and Post 3. Scores range from 0 to 1 and higher scores reflect more frequent use of the soothing behavior. Coding was conducted by three undergraduate students who were trained by an experienced post-graduate primary coder. Interrater reliability, calculated as single-measure ICCs based on a randomly selected 20% of videos coded by the primary coder, ranged from 0.8 to 0.93 for physical comfort, 0.96 to 0.99 for rocking, and 0.81 to 0.95 for verbal reassurance across Post 1, Post 2, and Post 3.

Toddlers’ Pain-Related Distress

Toddler distress was measured using the Face, Legs, Activity, Cry, Consolability (FLACC) scale (Merkel et al., 1997). The FLACC is a behavioral coding system used to code the intensity of behavioral distress as indicated by five behaviors, including toddler facial expression, leg movement, activity level, crying, and consolability (reverse-scored). Each behavior was rated on a 0–2 intensity scale during 15-s epochs beginning 1 min before the needle and continuing for 3 min after the needle. Composite scores for Baseline, Post 1, Post 2, and Post 3 were calculated as the sum of all five behaviors across the epochs within each minute. Scores range from 0 to 40 and higher scores reflect more behavioral distress. Coding was conducted by a team of undergraduates who were trained by an experienced graduate student primary coder. Single-measure ICCs, based on a randomly selected 20% of the videos coded by the primary coder, ranged from 0.90 to 0.94 across Baseline, Post 1, Post 2, and Post 3.

Analysis Plan

In preliminary analyses, caregiver gender, education, and age, and toddler age and sex were examined as potential covariates of OUCHIE-RV scores using correlations.

Aim 1: Reliability of the OUCHIE-RV

To address Aim 1a, we examined interrater reliability between the primary coder and each secondary coder for the OUCHIE-RV composite scores and toddler distress state variables based on the 20% of the dataset that was double-coded. Two-way random effects single-measure ICCs examined interrater reliability for OUCHIE-RV composite scores and kappa coefficients examined interrater reliability for the dichotomous toddler distress state variables for each of the six 30-s epochs. Thresholds of 0.7 and 0.6 have been proposed to reflect adequate reliability for ICCs (Mitchell, 1979) and kappa coefficients (Horner, 2005), respectively. To address Aim 1b (i.e., internal consistency), we calculated correlations between each composite score (e.g., OUCHIE-RV Post 1) and the behavior domain scores used to derive the composite score (e.g., Affect Incongruence Post 1, Attention Diversion Post 1, Physical Distance Post 1).

Aim 2: Validity of the OUCHIE-RV

To address Aim 2, we first calculated correlations between OUCHIE-RV composite scores and toddlers’ pain-related distress. To further evaluate Aim 2 and determine the variance accounted for in toddlers’ pain-related distress by the OUCHIE-RV over and above toddlers’ previous distress behavior, we estimated an autoregressive cross-lagged path analysis model. This type of analysis is ideal for examining the interactive influences of two variables over time because it allows for the examination of three types of relationships within the same model, including (1) predictive within-measure relationships (e.g., prediction of OUCHIE-RV Post 2 composite score from OUCHIE-RV Post 1 composite score), (2) predictive between-measure relationships (e.g., prediction of FLACC Post 2 from OUCHIE-RV Post 1 composite score), and (3) concurrent (residual) between-measure relationships (e.g., association between OUCHIE-RV Post 2 composite score and FLACC Post 2 after accounting for prediction from OUCHIE-RV Post 1 and FLACC Post 1). Toddlers’ pre-needle pain-related distress (FLACC Baseline) was included in the model as a predictor of Post 1 OUCHIE-RV and FLACC since it can impact subsequent toddler and caregiver behavior (Waxman et al., 2020). Multiple indices assessed model fit, including the comparative fit index (CFI), Tucker-Lewis index (TLI), and root mean square error of approximation (RMSEA). Conventional benchmarks of RMSEA < 0.06, TLI > 0.90, and CFI > 0.90 were used to evaluate model fit (Bentler & Bonett, 1980; Hu & Bentler, 1999).

Exploratory Aims: Associations with Other Aspects of Caregiver Responses

To address our exploratory aims, we calculated correlations between OUCHIE-RV composite scores and toddlers’ pain-related distress, caregiver state anxiety, caregiver physiological arousal and regulation, and soothing behaviors (i.e., physical comfort, rocking, and verbal reassurance).

The autoregressive cross-lagged path analysis model (Aim 2) was estimated in R (R Core Team, 2022) using the lavaan package (Rosseel, 2012). Maximum likelihood estimation was used, which allows for the inclusion of cases with missing data on outcomes. All other statistical analyses were conducted in SPSS (version 28) using pairwise deletion to address missing data. Alpha levels of .05 were used for all statistical tests.

Results

Data checking was performed by screening all outcome variables for evidence of outliers (> 3 standard deviations from the mean). A small number of outliers were identified, the removal of which did not substantially impact statistical outcomes. Thus, outliers were retained. Rates of missing data ranged from 5% to 7% for self-report and behavioral variables (most often due to blocked video footage) and from 32% to 33% for physiological variables (most often due to equipment failure).² Missingness was not significantly associated with caregiver age, education, relation to child, ethnicity, or toddler sex.

Sample characteristics are in Table I and descriptive statistics for all primary variables are in Table II. Participating caregivers reflected a high level of educational attainment, a wide range of self-reported heritage cultures, moderate to high acculturation to North American culture, and moderate acculturation to self-reported heritage cultures (see Table I). OUCHIE-RV scores were not associated with caregiver gender, education level, or age, or toddler age or sex, based on correlations (ps > .05).

Table II.

Descriptive Statistics for Study Variables

	n	Mean (SD)	Range
OUCHIE-RV composite scores
OUCHIE-RV Post 1	201	0.88 (0.45)	0–2
OUCHIE-RV Post 2	134	0.74 (0.45)	0–1.83
OUCHIE-RV Post 3	97	0.63 (0.47)	0–2
Toddlers’ pain-related distress
FLACC Baseline	217	8.26 (9.33)	0–39
FLACC Post 1	222	23.61 (9.2)	0–40
FLACC Post 2	221	14.92 (11.14)	0–36
FLACC Post 3	219	11.37 (11.1)	0–39
Caregiver state anxiety
Pre-needle	220	2.43 (2.5)	0–10
Post-needle	221	1.83 (2.34)	0–9
Caregiver physiological arousal and regulation
HR Baseline	157	93.76 (14.39)	37.06–131.28
HR Post 1 (bpm)	158	92.77 (14.81)	62.38–153.29
HR Post 2 (bpm)	159	91.49 (14.67)	62.02–151.48
HR Post 3 (bpm)	158	90.32 (14.03)	57.57–139.42
HRV Baseline (ln[ms2])	157	4.76 (1.2)	1.1–9.16
HRV Post 1 (ln[ms2])	158	4.94 (1.18)	2.06–7.46
HRV Post 2 (ln[ms2])	158	4.84 (1.21)	2.06–8.06
HRV Post 3 (ln[ms2])	158	4.73 (1.16)	1.74–7.74
Caregiver soothing behavior
Physical Comfort Post 1	223	0.57 (0.29)	0–1
Physical Comfort Post 2	223	0.42 (0.31)	0–1
Physical Comfort Post 3	222	0.36 (0.3)	0–1
Rocking Post 1	223	0.31 (0.32)	0–1
Rocking Post 2	223	0.31 (0.35)	0–1
Rocking Post 3	222	0.27 (0.35)	0–1
Verbal Reassurance Post 1	218	0.24 (0.21)	0–0.92
Verbal Reassurance Post 2	219	0.12 (0.16)	0–0.75
Verbal Reassurance Post 3	217	0.09 (0.13)	0–0.75

Note. FLACC = Face, Legs, Activity, Cry, Consolability scale; HR = heart rate; HRV = heart rate variability; OUCHIE-RV = Opportunities to Understand Childhood Hurt’s Inoculation Evaluation—Research Version; bpm = beats per minute; ln(ms²)= natural logarithm of milliseconds squared.

Aim 1: Reliability of the OUCHIE-RV

Interrater reliability for the dichotomous OUCHIE-RV toddler distress codes, calculated as kappa coefficients, ranged from .86 to .97. Interrater reliability for the continuous OUCHIE-RV composite scores, calculated as ICCs, ranged from .77 to .85 across coding pairs. This suggested strong interrater reliability.

Regarding internal consistency, behavior domain-total correlations ranged from r = .60 to .74 at Post 1, from r = .57 to .78 at Post 2, and from r = .70 to 0.78 at Post 3 (all ps < .001), suggesting moderate to strong positive associations between each OUCHIE-RV behavior domain and the overall OUCHIE-RV composite score within each minute.

Aim 2: Validity of the OUCHIE-RV

Correlations between the three OUCHIE-RV composite scores and toddlers’ pain-related distress are in Table III. OUCHIE-RV composite scores generally had significant, small positive associations with toddlers’ pain-related distress within concurrent (r = .28 to .36) and subsequent minutes (r = .24).

Table III.

Correlations Between OUCHIE-RV Composite Scores and Toddler and Caregiver Variables

	OUCHIE-RV Post 1			OUCHIE-RV Post 2			OUCHIE-RV Post 3
	n	r	p	n	r	p	n	r	p
Toddlers’ pain-related distress
FLACC Baseline	195	0.21	0.004	128	0.15	0.1	94	0.07	0.49
FLACC Post 1	200	0.28	<.001	134	0.18	0.04	97	−0.04	0.69
FLACC Post 2	199	0.24	<.001	134	0.29	<.001	97	−0.03	0.79
FLACC Post 3	197	0.22	0.002	133	0.24	0.01	97	0.36	<.001
Caregiver state anxiety
Pre-needle	198	0.09	0.23	133	0.03	0.7	96	−0.02	0.89
Post-needle	199	0.05	0.46	132	0.04	0.69	95	−0.01	0.93
Caregiver physiological arousal and regulation
HR Baseline	138	0.02	0.78	96	−0.04	0.72	69	−0.15	0.23
HR Post 1	138	0.03	0.7	95	−0.04	0.73	68	−0.14	0.25
HR Post 2	139	0.05	0.59	96	−0.06	0.6	68	−0.13	0.31
HR Post 3	138	0.02	0.83	95	−0.09	0.4	67	−0.12	0.34
HRV Baseline	138	0.1	0.24	96	0.13	0.22	69	0.04	0.75
HRV Post 1	138	0.02	0.78	95	−0.01	0.93	68	0.1	0.41
HRV Post 2	138	0.08	0.38	95	0.03	0.79	67	0.05	0.71
HRV Post 3	138	0.03	0.7	95	−0.04	0.73	67	0.08	0.53
Caregiver soothing behaviors
Physical Comfort Post 1	201	−0.28	<.001	134	−0.23	0.01	97	−0.09	0.36
Physical Comfort Post 2	201	−0.24	<.001	134	−0.34	<.001	97	−0.1	0.35
Physical Comfort Post 3	200	−0.21	0.002	134	−0.15	0.08	97	−0.24	0.02
Rocking Post 1	201	−0.19	0.01	134	−0.21	0.01	97	−0.31	0.002
Rocking Post 2	201	−0.16	0.03	134	−0.31	<.001	97	−0.21	0.04
Rocking Post 3	200	−0.14	0.06	134	−0.24	0.004	97	−0.36	<.001
Verbal Reassurance Post 1	196	0.09	0.2	132	0.14	0.11	95	−0.08	0.45
Verbal Reassurance Post 2	197	0.13	0.08	132	0.12	0.18	95	−0.15	0.14
Verbal Reassurance Post 3	195	−0.01	0.95	132	0.07	0.42	95	0.06	0.58

Note. Significant correlations are bolded.

FLACC = Face, Legs, Activity, Cry, Consolability scale; HR = heart rate; HRV = heart rate variability; OUCHIE-RV = Opportunities to Understand Childhood Hurt’s Inoculation Evaluation—Research Version.

To further examine associations between the three OUCHIE-RV composite scores and toddlers’ pain-related distress, we estimated an autoregressive cross-lagged path analysis model. The model fits the data adequately (CFI = 0.97, TLI = 0.92, RMSEA = 0.08). Results are depicted in Figure 2, and complete estimates are in Supplementary File 2. Predictive within-measure path results indicated that the OUCHIE-RV Post 1 composite score significantly predicted the OUCHIE-RV Post 2 composite score. In addition, toddler pain-related distress at each minute significantly predicted toddler pain-related distress in the subsequent minute. Predictive between-measure path results indicated that caregivers’ use of distress-promoting behavior did not significantly predict subsequent toddler pain-related distress. However, higher toddler pain-related distress at Baseline and Post 1 significantly predicted higher caregiver distress-promoting behavior at Post 1 and Post 2, respectively. Concurrent between-measure path results indicated significant residual associations between caregiver distress-promoting behavior and toddler pain-related distress within each of the three post-needle minutes.

Figure 2.

Autoregressive cross-lagged path analysis model of relations between OUCHIE-RV composite scores and toddlers’ pain-related distress. Note. FLACC = Face, Legs, Activity, Cry, Consolability scale; OUCHIE-RV = Opportunities to Understand Childhood Hurt’s Inoculation Evaluation—Research Version. Standardized regression coefficients reported. Double-headed arrows reflect concurrent associations and single-headed arrows reflect predictive associations. Bolded paths are significant at p < .05. Comparative fit index (CFI) = 0.97, Tucker-Lewis index (TLI) = 0.92, root mean square error of approximation (RMSEA) = 0.08, n = 217.

Exploratory Aims: Associations With Other Aspects of Caregiver Responses

Correlations were calculated between OUCHIE-RV composite scores and caregiver state anxiety, caregiver physiological arousal, and soothing behaviors (i.e., physical comfort, rocking, and verbal reassurance), the results of which are reported in Table III.³ OUCHIE-RV composite scores had small to moderate negative correlations with caregivers’ concurrent use of physical comfort (r =−.34 to −.24) and rocking (r = −.36 to −.19) and negligible to small associations with caregivers’ use of verbal reassurance (r = .06 to .12). Correlations between OUCHIE-RV composite scores and caregivers’ state anxiety (r = −.02 to .09), physiological arousal (r = −.15 to .05), and physiological regulation (r = −.04 to .13) were generally negligible.

Discussion

The current study introduced the OUCHIE-RV, a novel observational measure of caregivers’ distress-promoting behavior during toddler vaccination that is based on coding caregivers’ verbal and nonverbal behaviors in reaction to their toddlers’ distress. Results provide evidence for the validity and reliability of the OUCHIE-RV as a tool for quantifying the frequency and intensity of caregivers’ distress-promoting behavior during toddler vaccination.

Aim 1: Reliability of the OUCHIE-RV

To explore measure reliability, Aim 1 examined interrater agreement and internal consistency. Results suggested that the OUCHIE-RV can be reliably coded by a team of undergraduate students with training by an experienced coder, supporting the feasibility of the measure’s use in other research and clinical contexts. Each behavioral domain of the OUCHIE-RV had moderate to strong positive associations with OUCHIE-RV composite scores, which were similar in magnitude across each minute post-needle. This suggests that associations between each composite score and the behavioral domains used to derive the score are internally consistent across time. These findings provide strong evidence for the reliability of the OUCHIE-RV.

Aim 2: Validity of the OUCHIE-RV

To examine the concurrent validity of the OUCHIE-RV as a measure of caregivers’ distress-promoting behaviors, Aim 2 tested the hypothesis that the OUCHIE-RV would be positively associated with toddlers’ pain-related distress. Results provided support for this hypothesis. As predicted, OUCHIE-RV composite scores for each minute of the post-needle period were significantly associated with greater concurrent and subsequent pain-related distress, with small- to medium-sized correlations.

To obtain a more nuanced understanding of these relationships across the post-needle period, a cross-lagged model was used to estimate how much variance the OUCHIE-RV explained in toddlers’ pain-related distress after accounting for toddlers’ distress and caregivers’ distress-promoting behavior in the previous minute. Results showed that within each minute, more frequent or intense caregiver distress-promoting behavior was still significantly associated with greater concurrent toddler distress, providing further support for OUCHIE-RV behaviors as distress-promoting. Despite the small to moderate bivariate correlations between OUCHIE-RV in a given minute and toddler distress in the subsequent minute that were identified in preliminary analyses, caregiver behavior in a given minute did not predict toddler distress in a subsequent minute beyond what was already predicted by concurrent caregiver behaviors when investigated in the cross-lagged model. This finding is consistent with the DIAPR-R 2022 model, which posits that the caregiver and infant are each reacting to and influencing the other’s responses on a moment-to-moment basis through dyadic feedback loops (Pillai Riddell et al., 2022). Furthermore, greater toddler distress at baseline and during the first minute post-needle significantly predicted more caregivers’ distress-promoting behavior within the subsequent minute. Since infants with higher levels of anticipatory (i.e., pre-needle) distress typically take longer to regulate following the needle (Waxman et al., 2020), one interpretation is that caregivers themselves become dysregulated by the initial signs of their child’s dysregulation, resulting in non-attuned soothing behaviors.

Exploratory Aims: Associations With Other Aspects of Caregiver Responses

Exploratory aims of the current study were to examine associations between the OUCHIE-RV and other aspects of caregivers’ psychological, physiological, and emotional responses within the vaccination context. Results indicated partial support for hypotheses. As predicted, the OUCHIE-RV had significant negative associations with measures of caregivers’ proximal soothing behaviors (i.e., physical comfort, rocking), which were small to moderate in magnitude. Proximal soothing responses are considered to reflect a prototypical sensitive caregiving response (e.g., Biringen, 2000), and so the finding of small- to moderate-sized negative associations with the OUCHIE-RV could suggest that distress-promoting caregiver behavior reflects a unique dimension of caregiver behavior, rather than simply reflecting the absence of sensitive behavior. The hypothesized positive association between the OUCHIE-RV and caregivers’ use of verbal reassurance (e.g., saying “You’re okay”) was largely unsupported, with correlations in the negligible to a small range. Verbal reassurance has been shown to be soothing when used sparingly but distress-promoting when used at high frequency (Racine et al., 2012), even when reassurance occurs in the context of other sensitive verbalizations (Campbell et al., 2018). Given that frequent verbal reassurance is only one form of affect incongruence captured within the OUCHIE-RV composite score, this finding may suggest that this particular behavior is less often accompanied by other aspects of distress-promoting behaviors. Moreover, the fact that verbal reassurance as coded by the MAISD captures all use of reassuring statements by caregivers during the post-needle period, regardless of whether the timing or intensity of these statements would likely be attuned or non-attuned to the child’s needs, may also underlie the relatively small associations with the OUCHIE-RV composite scores.

Caregivers’ distress-promoting behavior was not significantly associated with caregivers’ self-reported state anxiety or their physiological arousal and regulation. Previous research in a pain context has linked parents’ use of distress-promoting behaviors with lower baseline HRV and higher state catastrophizing during child venipuncture (Constantin et al., 2022). Similarly, findings from non-pain-related laboratory-based stressors suggest that higher levels of parents’ physiological arousal and lower levels of regulation relate to greater use of distress-promoting parenting behavior (Lorber & O’Leary, 2005; Miller et al., 2015). The absence of these associations in our sample of caregivers with relatively low socioeconomic risk may suggest that factors other than high arousal and low regulatory capacity (e.g., lack of knowledge about the impact of behaviors) are primarily driving caregivers’ use of distress-promoting behaviors.

Clinical Implications

Parent-focused interventions are an important tool used to improve pain outcomes in infants and toddlers (Pillai Riddell et al., 2018b), and several findings from the current study can inform future interventions. For example, caregivers’ distress-promoting behavior was not associated with their state anxiety or physiological responses but was predicted by toddlers’ distress during previous minutes. This suggests that distress-promoting behaviors may reflect caregivers’ well-intentioned but non-attuned soothing response. Initiatives to educate caregivers on the impact of caregiving behaviors (Kain et al., 2007; Pillai Riddell et al., 2018b) may be expected to be fruitful and broadly relevant as first-line interventions in vaccination contexts alongside more targeted interventions focused on supporting caregivers’ emotional regulation. Importantly, education initiatives should focus on providing caregivers with a repertoire of adaptive soothing responses that can be trialed and adapted in response to toddlers’ reactions. We found that caregivers’ distress-promoting behavior in a given minute did not predict future toddler distress, suggesting that caregivers can adapt initially unsuccessful behavioral responses without carry-over effects to the remainder of the interaction.

Limitations and Future Directions

Strengths of the current study include the use of a universally relevant and standardized pain context and the recruitment of a large culturally diverse sample of healthy toddlers and caregivers. However, several limitations should be noted. First, the high educational attainment of participating caregivers and our focus on healthy, typically developing toddlers may impact the generalizability of the OUCHIE-RV. Although the domains of distress-promoting parent behaviors coded by the OUCHIE-RV are grounded in theory and empirical evidence drawn from socioeconomically diverse samples, supporting their generalizability, future studies should examine the validity of the OUCHIE-RV in samples characterized by more socioeconomic diversity as well as with caregivers of children with developmental disabilities, medical comorbidities or traumatic medical experiences, and other factors that may impact on how toddlers communicate pain-related distress and how caregivers interpret and respond to their child’s pain-related distress. Second, due to our grounding in Attachment Theory, we only coded caregiver behavior that occurred following the painful stimulus (i.e., post-needle) as this is when the toddler’s needs are not ambiguous. However, we acknowledge that caregivers’ behavior prior to the needle may impact toddlers’ subsequent distress behavior. Including toddlers’ baseline distress behavior as a covariate within the cross-lagged path analysis, as we have done, would be expected to partly, but not completely, account for this influence. Another potential limitation of the study is that results may not generalize to dyads in which the toddler expressed no or low distress post-vaccination (i.e., behavioral responses characteristic of an avoidant attachment style). However, this was very rare in our study (∼5%). Finally, because data for the current study were collected prior to the COVID-19 pandemic, future research should examine whether pandemic-related changes in healthcare practices (e.g., masking requirements) may impact the usability or validity of the OUCHIE-RV. Emerging evidence suggests that face masks hamper the accurate detection of certain emotions (e.g., happiness) more than others (e.g., fear; Carbon, 2020; McCrackin et al., 2021), which may have implications for both coder reliability and the extent to which toddlers discern and react to caregivers’ facial expressions.

Conclusions

We have presented a novel behavioral coding scheme, grounded in Attachment Theory, that captures verbal and non-verbal aspects of caregiver behaviors that promote toddlers’ pain-related distress. Our results provide support for the reliability and validity of the OUCHIE-RV as a measure of caregivers’ distress-promoting behavior. We found that when caregivers respond to toddlers’ moderate to high pain-related distress in a way that communicates a lack of recognition or misinterpretation of the child’s distress signals, such as through non-contingent effect, maintaining physical space between themselves and their child, or diverting their attention away from their distressed child, toddlers respond by maintaining or escalating their pain-related distress. The OUCHIE-RV supports a better understanding of the aspects of caregivers’ behavioral responses that impact young children’s coping during painful procedures and offers an empirically and theoretically grounded research tool for the objective assessment of caregiver behavioral responses in future research.

Footnotes

1

Initially, we planned to code four domains of parent distress-promoting behavior. The fourth domain (intrusive action, including the items “forceful” and “face cover” from the OUCHIE-CV) was dropped from the coding scheme due to a low base rate in the current sample.

2

Missing physiological data rates exceeding 20%–30% are common among studies that measure cardiac data during conditions other than a resting state (Lan et al., 2022; Mills-Koonce et al., 2009; Sturge-Apple et al., 2011).

3

Caregiver posture (i.e., percentage of time in a standing position) was examined as a covariate in analyses involving HR and HRV. Because its inclusion did not alter results, results are reported without the covariation of caregiver posture.

Supplementary Data

Supplementary data can be found at: https://academic.oup.com/jpepsy.

Funding

This work was supported by the Natural Sciences and Engineering Research Council of Canada (grant number RGPIN-2020-07140, to RPR), the Ontario Ministry of Research and Innovation (grant number 532653, to RPR), the Canadian Foundation for Innovation (grant number 29908, to RPR), the Canadian Institutes of Health Research (grant number GSD-164166, to SB), the Ontario Graduate Scholarship (to SB), and the Meighen Wright Graduate Scholarship in Maternal-Child Health (to SB). SB is a trainee member of the Pain in Child Health (PICH) program, a research training initiative at The Hospital for Sick Children. Funders were not involved in the design or conduct of the study.

Conflicts of interest

None declared.

Author Contributions

Shaylea D. Badovinac (Conceptualization [equal], Data curation [lead], Formal analysis [lead], Investigation [equal], Methodology [lead], Writing—original draft [lead], Writing—review & editing [lead]), David B. Flora (Conceptualization [supporting], Formal analysis [supporting], Methodology [supporting], Supervision [supporting], Writing—review & editing [supporting]), Heather Edgell (Formal analysis [supporting], Methodology [supporting], Supervision [supporting], Writing—review & editing [supporting]), Dan Flanders (Resources [supporting]), Hartley Garfield (Resources [supporting]), Eitan Weinberg (Resources [supporting]), Deena Savlov (Resources [supporting]), and Rebecca R. Pillai Riddell (Conceptualization [equal], Data curation [supporting], Formal analysis [supporting], Funding acquisition [lead], Investigation [equal], Methodology [supporting], Resources [lead], Supervision [lead], Writing—review & editing [supporting])

Acknowledgments

The authors gratefully acknowledge the families who participated in this research as well as Dr. Hannah Gennis, whose contributions to the development of the OUCHIE—CV helped lay the foundation for the OUCHIE-RV.