Racial Disparities in Pulse Oximeter Device Inaccuracy and Estimated Clinical Impact on COVID-19 Treatment Course

Abstract

Arterial blood oxygen saturation as measured by pulse oximetry (peripheral oxygen saturation (SpO₂)) may be differentially less accurate for people with darker skin pigmentation, which could potentially affect the course of coronavirus disease 2019 (COVID-19) treatment. We analyzed pulse oximeter accuracy and its association with COVID-19 treatment outcomes using electronic health record data from Sutter Health, a large, mixed-payer, integrated health-care delivery system in Northern California. We analyzed 2 cohorts: 1) 43,753 non-Hispanic White (NHW) or non-Hispanic Black/African-American (NHB) adults with concurrent arterial blood gas oxygen saturation/SpO₂ measurements taken between January 2020 and February 2021; and 2) 8,735 adults who went to a hospital emergency department with COVID-19 between July 2020 and February 2021. Pulse oximetry systematically overestimated blood oxygenation by 1% more in NHB individuals than in NHW individuals. For people with COVID-19, this was associated with lower admission probability (−3.1 percentage points), dexamethasone treatment (−3.1 percentage points), and supplemental oxygen treatment (−4.5 percentage points), as well as increased time to treatment: 37.2 minutes before dexamethasone initiation and 278.5 minutes before initiation of supplemental oxygen. These results call for additional investigation of pulse oximeters and suggest that current guidelines for development, testing, and calibration of these devices should be revisited, investigated, and revised.

arterial blood gas, blood oxygenation, coronavirus, electronic health records, health equity, pulse oximetry, real-world data

This article is linked to "Invited Commentary: Undiagnosed and Undertreated—the Suffocating Consequences of the Use of Racially Biased Medical Devices During the COVID-19 Pandemic" (https://doi.org/10.1093/aje/kwad019).

Abbreviations

 
• ABG

  arterial blood gas

 
• CDC

  Centers for Disease Control and Prevention

 
• CI

  confidence interval

 
• COVID-19

  coronavirus disease 2019

 
• ED

  emergency department

 
• FDA

  Food and Drug Administration

 
• NHB

  non-Hispanic Black/African-American

 
• NHW

  non-Hispanic White

 
• SaO₂

  arterial oxygen saturation

 
• SpO₂

  peripheral oxygen saturation

Editor’s note: An invited commentary on this article appears on page 714.

In February 2021, both the Food and Drug Administration (FDA) and the Centers for Disease Control and Prevention (CDC) issued public statements regarding possible suboptimal accuracy of pulse oximeters in individuals with darker skin pigmentation (1, 2). The CDC noted the potential for underdetection of hypoxemia in darker-skinned people with coronavirus disease 2019 (COVID-19), citing data from several studies (3–9). Despite this caution, an oxygen saturation level of less than 94%, as measured by pulse oximetry, remains a key determinant of severe disease (10, 11). Pulse oximetry therefore influences treatment decisions for people with COVID-19, such as whether to initiate hospital-based treatments (e.g., supplemental oxygen or intravenous dexamethasone) (11, 12).

Pulse oximeters are external monitoring devices that clip onto a part of the body, typically a fingertip or earlobe, and emit a light that passes through the tissue and blood that is then detected by a sensor (13). The amount of light not absorbed is used to estimate the blood’s oxygen saturation (13). Pulse oximeter accuracy can be affected by multiple factors such as nail polish, skin features (thickness, pigmentation, temperature), and tobacco use (1). Several studies have also shown pulse oximeters to be less accurate at lower oxygen saturations among severely ill patients (14, 15).

Studies typically measure pulse oximeter accuracy by comparing pulse oximetry with concurrent oxygen saturation measured by arterial blood gas (ABG), which is considered the gold standard for measuring arterial blood oxygen saturation (16). An ABG measurement requires a sample of arterial blood, usually taken from the radial artery. This is more invasive and uncomfortable than pulse oximetry and cannot be performed as frequently. ABG evaluates the partial pressures of gases (oxygen and carbon dioxide, among others) and blood pH (acid–base content) in the sample, more accurately assessing oxygenation (17). Historical studies have reported mixed results when comparing the accuracy of pulse oximetry with that of ABG. Some studies have found that pulse oximeters overestimate oxygenation, while other studies suggest the opposite (4, 7, 14, 18–20). Overestimation of blood oxygenation by pulse oximetry in darker-skinned (as compared with lighter-skinned) individuals has been documented in the medical literature for more than 30 years; however, the clinical importance of this systematic measurement inaccuracy has not been clearly established (5, 6, 8, 21).

A 2020 observational study of 2 large hospital cohorts found that people with COVID-19 who identified as Black were nearly 3 times more likely than those who identified as White to have occult hypoxemia not detected by pulse oximetry (9). The well-documented differences in pulse oximeter accuracy between darker- and lighter-skinned people could be a driver of these results. Given the central role of oxygen saturation in management of COVID-19 patients, even small systematic inaccuracies have the potential to affect access to treatment and patient outcomes (13, 22). For example, in one study of COVID-19 patients in New York, New York, Lancet et al. (23) estimated that each 1% decrease in prehospital oxygen saturation (as measured by pulse oximetry) was associated with 7% higher odds of death. Systematic overestimation of true blood oxygenation by pulse oximetry within Black patients with COVID-19—even by a modest amount—can therefore lead to underestimation of disease severity, potentially causing delays in care and contributing to health disparities. This is supported by a 2022 study that showed a link between predicted pulse oximeter–induced underestimation of blood oxygenation and delayed identification of treatment eligibility for supplemental oxygen in COVID-19 patients (24). However, the estimated associated delay in treatment (in minutes or hours) that could be directly attributable to differential pulse oximeter measurement accuracy, or its impact on other aspects of the COVID-19 treatment course, is not yet known.

To address this gap, we first investigated pulse oximeter/ABG discrepancies within non-Hispanic Black/African-American (NHB) and non-Hispanic White (NHW) patients of a large, integrated health-care system in Northern California. After establishing the existence of differential pulse oximeter accuracy, we analyzed possible impacts of these differences on COVID-19–related treatment and outcomes. We hypothesized that differential overestimation of oxygen saturation by pulse oximetry could negatively affect timely access to treatment for NHB individuals with COVID-19.

METHODS

Study setting

We used electronic health record data from Sutter Health, a large, mixed-payer, integrated health-care delivery system in Northern California, described elsewhere (25). Sutter Health delivers comprehensive medical services in over 100 ambulatory-care clinics and 24 acute-care hospitals, caring for approximately 3.5 million people each year. The Sutter Health Institutional Review Board approved this study.

Analytical populations and data extraction

We extracted blood oxygen saturation measurements taken by ABG and by pulse oximetry—hereafter referred to as arterial oxygen saturation (SaO₂) and peripheral oxygen saturation (SpO₂), respectively—between January 2020 and February 2021 at any Sutter Health hospital for adults who self-identified as either NHW or NHB (cohort 1). Individuals who self-identified as multiracial or Hispanic were excluded. We used racial/ethnic categories consistent with the US Census categories and the Office of Management and Budget, and we use the designation “Hispanic” instead of the commonly used term “Latinx” for consistency with US Census categories (and with Sutter Health self-reported data collection) (26). We limited our sample to NHB and NHW patients for maximal potential contrast, and also for comparability with other recent studies (9, 27). We paired each SaO₂ measurement with the nearest recorded SpO₂ measurement for the same person, truncated at ±10 minutes from the earlier of ABG specimen “taken time” or “result time.” We defined hypoxemia as an SaO₂ level less than 90% (28).

Our second study population (cohort 2) comprised all adults who visited a hospital emergency department (ED) with COVID-19 between July 2020 and February 2021 and self-identified as NHW or NHB (as above). COVID-19 visits were defined by an International Classification of Diseases, Tenth Revision, hospital diagnosis (any position) of either U07.1 (COVID-19) or J12.82 (COVID-19 pneumonia), and only the first qualifying visit for each individual was included. July 2020 was chosen because the guidelines for dexamethasone treatment based on oxygen saturation were released in late June of 2020 (29). We excluded hospital visits for which we had no documented SpO₂ measurement.

We compiled patient-level sociodemographic and clinical characteristics, including SpO₂ at ED presentation, age, sex, homelessness status, tobacco smoking status (ever having smoked), and Charlson comorbidity index score (30, 31). We separately considered diagnosed asthma, cancer, congestive heart failure, chronic obstructive pulmonary disease, cardiovascular disease, type 2 diabetes, depression, hypertension, liver disease, obesity, and renal disease. We examined the following outcomes related to the COVID-19 treatment course: 1) measurement of oxygen saturation by SaO₂, 2) time spent in the ED, 3) hospital admission, 4) dexamethasone administration and timing, 5) oxygen supplementation and timing, and 6) return to the hospital after discharge home. Detailed information on data extraction and preparation can be found in Web Appendix 1 (available at https://doi.org/10.1093/aje/kwac164) and Web Table 1.

Statistical analysis

Analysis of pulse oximetry measurement error.

We compared paired SpO₂/SaO₂ measurements between the NHB and NHW subgroups (cohort 1) using pairwise statistical tests, specifically the Wilcoxon rank-sum test for quasicontinuous variables and the χ² test for binary variables. The Wilcoxon rank-sum test was chosen due to the nonnormality of the data. The unit of analysis was the individual oxygen saturation measurement pair. We also plotted median SpO₂ for each value of SaO₂ to graphically explore the agreement between SaO₂ and SpO₂ within subgroups. To evaluate the influence of measurement timing differences, we conducted a sensitivity analysis restricted to SpO₂/SaO₂ measurement pairs with a time difference of 0.

Analysis of impact on COVID-19 treatment outcomes.

Following the potential outcomes framework from causal inference, we used G-computation to build 2 counterfactuals to assess the possible impacts of differential SpO₂ measurement error on COVID-19–related outcomes for NHB patients (32).

We first modeled the relationship between baseline characteristics and each COVID-19 treatment outcome. We used the subset of NHW patients rather than the entire study population to avoid identification of the NHB subgroup via some combination of predictor variables (e.g., due to differences in utilization patterns). This assumes that the associations between baseline characteristics and outcomes are similar across the entire study population. Logistic regression was used for binary outcomes and negative binomial regression for time-related outcomes (minutes spent in the ED, etc.). Negative binomial models were chosen over Poisson models both to allow 0 values and due to mean/variance inequality.

To estimate the impact of SpO₂ measurement error, we first used the counterfactual models described above to predict the expected marginal outcomes for NHB patients under the assumption that the relationship between the predictors and the outcomes did not differ between the NHB and NHW subgroups. Mean predicted (counterfactual) outcomes were then subtracted from the observed mean NHB patient outcomes to yield marginal estimates of the mean difference. We then shifted the observed SpO₂ value by the NHW-NHB measurement difference observed in the pulse oximetry measurement error analysis described above and used this updated SpO₂ value to generate a second set of predictions from our counterfactual models. This second set of predictions estimated the expected (counterfactual C₁) outcomes again assuming that both the predictor-outcome relationship and the SpO₂ measurement error did not differ between the NHB and NHW subgroups. This second set of predicted (counterfactual C₂) mean outcomes was again compared with observed mean outcomes from NHB patients. To isolate the estimated effect of differential SpO₂ measurement error on outcomes for NHB patients, we computed the mean difference between the 2 predicted (counterfactual) outcomes for each individual (C₁ − C₂). If differential SpO₂ measurement error does not affect a given outcome, we would expect the difference between the 2 counterfactual outcomes to be 0. Additional methodological detail is provided in Web Appendix 2. Standard errors and 95% confidence intervals for the estimated mean differences in outcomes were computed using the nonparametric bootstrap with 2,000 iterations. Analyses were conducted in R, version 4.1.0 (R Foundation for Statistical Computing, Vienna, Austria). We also summarized descriptive data for cohort 2 and conducted unadjusted pairwise comparisons using χ² and Wilcoxon rank-sum tests.

RESULTS

Analysis of pulse oximetry measurement error

We identified a total of 43,753 SaO₂/SpO₂ measurement pairs (8,626 NHB, 35,127 NHW) (Table 1). SpO₂ values were higher for the NHB population on average (median, 99; mean = 96.94) than for the NHW population (median, 98; mean = 96.42) (P < 0.001). SaO₂ values were similar numerically between the two groups (NHB: median, 96%; mean = 94.49%; NHW: median, 96%; mean = 94.88%); however, the distributions were found to be statistically different (P = 0.001). SaO₂ measurements were lower on average than concurrent SpO₂ measurements (NHB: median difference, −2 percentage points; mean difference = −2.45 percentage points; NHW: median difference, −1 percentage point; mean difference = −1.53 percentage points), and the measurement difference was 1% larger for the NHB population (P < 0.001). Hypoxemia not detected by SpO₂ within the NHB patient group was 5.50% as compared with 3.01% within the NHW patient group (P < 0.001). NHB individuals were also more likely to have an SaO₂ measurement below 94% paired with an SpO₂ measurement of 94% or above (14.71% vs. 9.52%; P < 0.001). The sensitivity analysis agreed with the main analysis (see Web Table 2).

Table 1

Paired Arterial Oxygen Saturation and Peripheral Oxygen Saturation Measurements Taken Among Non-Hispanic Black/African-American and Non-Hispanic White Individuals in Northern California, January 2020–February 2021

				SpO2Measurement,%b		SaO2Measurement,%c		Measurement Difference, PP
Race/Ethnicity	No. of Paired Measurementsa	No. of Unique Patients	No. of Unique Encounters	Median	Mean (SD)	Median	Mean (SD)	Median	Mean (SD)	Proportion ofHypoxemia Not Detected bySpO2, %d	Proportion ofSaO2Below 94% Not Detectedby SpO2, %e
NH Black	8,626	2,616	2,126	99	96.94 (5.55)	96	94.49 (6.42)	−2	−2.45 (6.91)	5.50	14.71
NH White	35,127	10,514	9,113	98	96.42 (4.76)	96	94.88 (5.77)	−1	−1.53 (5.52)	3.01	9.52
P for differencef				<0.001		0.001		<0.001		<0.001	<0.001

Abbreviations: NH, non-Hispanic; PP, percentage points; SaO₂, arterial oxygen saturation; SD, standard deviation; SpO₂, peripheral oxygen saturation.

^a Each SaO₂ measurement was paired with the nearest recorded SpO₂ measurement for the same person, truncated at ±10 minutes from the SaO₂ specimen “taken time” or the SaO₂ “result time,” whichever was earlier.

^b Percentage of blood oxygen saturation measured via pulse oximetry.

^c Percentage of blood oxygen saturation measured via arterial blood gas.

^d An SaO₂ level below 90% paired with a concurrent SpO₂ measurement of 94% or above.

^e An SaO₂ level below 94% paired with a concurrent SpO₂ measurement of 94% or above.

^f Wilcoxon rank-sum test for quasicontinuous variables; χ² test for categorical variables.

Figure 1A shows a consistently larger discrepancy between measurements for NHB patients, although the median SpO₂ is higher than the median SaO₂ for both groups across the range of SaO₂ values. There is also an increase in SpO₂ overestimation as SaO₂ decreases. The distribution of SaO₂ − SpO₂ for NHB individuals is shifted to the left of the NHW distribution and is slightly more dispersed (see Figure 1B).

Figure 1

Paired arterial blood gas (arterial oxygen saturation (SaO₂)) and pulse oximetry (peripheral oxygen saturation (SpO₂)) blood oxygenation measurements taken in Northern California between January 2020 and February 2021, by race/ethnicity (non-Hispanic Black: n = 8,626; non-Hispanic white: n = 35,127). SaO₂ and SpO₂ are expressed in percentage units.

Analysis of impact on COVID-19 treatment outcomes

A total of 8,735 encounters (7,036 NHW, 1,699 NHB) met our study criteria. NHB individuals were younger on average (a median age of 51 years (NHB) vs. 60 years (NHW); P < 0.001) and had higher SpO₂ levels at presentation (a median of 98% (NHB) vs. 97% (NHW); P < 0.001). A higher proportion of NHB patients were experiencing homelessness (3.1% vs. 1.8%; P = 0.001), and patient insurance types differed between the two groups (see Table 2). The comorbidity profiles at ED presentation also differed, with higher proportions of cancer, chronic obstructive pulmonary disease, and depression in the NHW patient group; the NHB patient group had higher percentages of asthma, congestive heart failure, obesity, and renal disease (Table 2). Unadjusted outcome comparisons are shown in Web Table 3.

Table 2

Baseline Characteristics^a of COVID-19 Encounters Originating in a Hospital Emergency Department for Individuals Identifying as Non-Hispanic Black/African-American or Non-Hispanic White, Northern California, July 2020–February 2021

	Race/Ethnicity
Variable	NH Black (n = 1,699)	NH White (n = 7,036)	P for Differenceb
Age, years			<0.001
Mean	51.3	58.8
Median	51.0	60.0
Female sex	52.2	52.3	0.919
Initial SpO2c level, %			<0.001
Mean	96.7	95.9
Median	98.0	97.0
Homelessness	3.1	1.8	0.001
Type of health insuranced
Medicaid	41.8	22.3	<0.001
Medicare	30.6	45.0	<0.001
Uninsured or unknown	4.9	3.0	<0.001
Charity care	2.9	1.6	<0.001
Commercial or other insurance	24.7	31.0	<0.001
Mean CCI score	1.3	1.2	0.472
Comorbiditye
Asthma	8.7	5.6	<0.001
Cancer	3.3	5.3	<0.001
Congestive heart failure	14.4	11.4	0.001
COPD	5.4	6.9	0.020
Cardiovascular disease	8.8	7.9	0.188
Type 2 diabetes	8.7	8.7	0.908
Depression	1.6	3.6	<0.001
Hypertension	19.7	21.0	0.225
Liver disease	2.7	2.2	0.228
Obesity	7.9	6.3	0.017
Renal disease	19.2	14.9	<0.001
Ever smoking tobacco	37.6	39.2	0.211

Abbreviations: CCI, Charlson comorbidity index; COPD, chronic obstructive pulmonary disease; COVID-19, coronavirus disease 2019; NH, non-Hispanic; SpO₂, peripheral oxygen saturation.

^a Values are percentages of patients unless otherwise indicated.

^b Wilcoxon rank-sum test for quasicontinuous variables; χ² test for percentages.

^c Percentage of blood oxygen saturation measured via pulse oximetry.

^d Values may not total 100%, as a single encounter may have had more than 1 payer (e.g., Medicare/Medicaid).

^e Comorbid conditions assessed as of emergency department presentation; does not include conditions diagnosed during the hospital visit.

Differences between observed and counterfactual (C₁) COVID-19 outcomes for NHB patients represent the estimated racial bias in outcomes for the NHB patient group (Table 3). NHB patients had higher probabilities of being admitted to the hospital (2.4 percentage points, 95% confidence interval (CI): 0.2, 4.6), of returning to the hospital after discharge home from the ED (8.3 percentage points, 95% CI: 4.6, 11.9) or after inpatient admission (8.7 percentage points, 95% CI: 0.7, 16.7), and of having SaO₂ measured prior to the decision to admit or discharge (12.9 percentage points, 95% CI: 4.1, 21.7). Observed mean time to admission decision was lower than its counterfactual estimate (−15.9 minutes, 95% CI: –31.5, –0.3).

Table 3

Differences in Observed COVID-19 Treatment Outcomes for Non-Hispanic Black/African-American Patients (n = 1,699) Compared With Counterfactual Outcomes Derived Under the Assumption of No Racial Bias, Northern California, July 2020–February 2021

Variable	Mean Observed (O) Outcome	Mean Counterfactual (C1) Outcome	Difference (O − C1)	95% CI
Admitted to the hospital, %a	34.5	32.1	2.4	0.2, 4.6
Treated with dexamethasone, %a	22.5	21.2	1.3	−0.9, 3.5
Treated with supplemental oxygen, %a	29.1	27.6	1.5	−0.6, 3.6
Returned to the hospital after discharge home, %a
ED visitsb	37.0	28.7	8.3	4.6, 11.9
Admissionsc	43.2	34.5	8.7	0.7, 16.7
Measurement of SaO2d before admission/discharge decision, %a,e	62.7	49.8	12.9	4.1, 21.7
Time to admission decision, minutesf	176.6	192.5	−15.9	−31.5, −0.3
Time spent in the ED, minutes
Admittedg	541.2	502.0	39.1	−5.0, 83.2
Discharged home from EDb	250.6	227.3	23.4	−10.5, 57.2
Time to dexamethasone, minutesh	1,069.8	961.4	108.4	−328.0, 544.8
Time to supplemental oxygen, minutesi	718.5	1,030.1	−311.7	−715.2, 91.9

Abbreviations: CI, confidence interval; COVID-19, coronavirus disease 2019; ED, emergency department; SaO₂, arterial oxygen saturation.

^a Values are expressed as mean predicted probability, shown in percentage units.

^b Patients who were discharged home after the ED visit only (n = 997).

^c Patients who were discharged home after inpatient admission (n = 236).

^d Percentage of blood oxygen saturation measured via arterial blood gas.

^e Patients with SaO₂ level measured (n = 217).

^f Patients with a decision to admit (n = 641).

^g Patients who were admitted to the hospital (n = 586).

^h Patients who were given dexamethasone (n = 383).

ⁱ Patients who were given supplemental oxygen (n = 484).

The second set of counterfactual outcomes (C₂) represents the expectation under the assumption that the SpO₂ value for the NHB population is shifted upward by 1%, thereby equalizing the NHB and NHW measurement error (Table 4). In contrast to the above results, there was no difference (racial bias) in the probability of admission under this assumption, although the probability of treatment with supplemental oxygen was lower (−3.0 percentage points, 95% CI: –5.2, –0.8). Compared with C₁, the difference in the probability of returning to the hospital after discharge home was significant, but it was smaller for patients not admitted (7.0 percentage points, 95% CI: 3.3, 10.7) and larger for those who were admitted (9.7 percentage points, 95% CI: 1.7, 17.6).

Table 4

Differences in Observed COVID-19 Treatment Outcomes for Non-Hispanic Black/African-American Patients (n = 1,699) After Correction for Differential Peripheral Oxygen Saturation Measurement, Northern California, July 2020–February 2021

Variable	Mean Observed (O) Outcome	Mean Counterfactual (C1) Outcome	Mean Counterfactual (C2) Outcome (Corrected SpO2a)	Difference (O − C2) (Corrected SpO2)	95% CI	Difference BetweenCounterfactuals(C1− C2)	95% CI
Admitted to the hospital, %b	34.5	32.1	35.2	−0.7	−2.9, 1.6	−3.1	−3.4, −2.8
Treated with dexamethasone, %b	22.5	21.2	24.3	−1.8	−4.0, 0.5	−3.1	−3.4, −2.7
Treated with supplemental oxygen, %b	29.1	27.6	32.1	−3.0	−5.2, −0.8	−4.5	−4.9, −4.2
Returned to the hospital after discharge home, %b
ED visitsc	37.0	28.7	30.0	7.0	3.3, 10.7	−1.2	−1.9, −0.5
Admissionsd	43.2	34.5	33.5	9.7	1.7, 17.6	1.0	0.2, 1.7
Time to admission decision, minutese	176.6	192.5	186.5	−9.9	−24.2, 4.4	6.0	3.4, 8.6
Time spent in the ED, minutes
Admittedf	541.2	502.0	498.5	42.7	−1.2, 86.5	3.6	1.3, 5.8
Discharged home from EDc	250.6	227.3	236.4	14.3	−20.6, 49.1	−9.1	−12.2, −6.0
Time to dexamethasone, minutesg	1,069.8	961.4	924.2	145.6	−287.1, 578.3	37.2	20.1, 54.3
Time to supplemental oxygen, minutesh	718.5	1,030.1	751.7	−33.2	−370.3, 304.0	278.5	181.0, 376.0

Abbreviations: CI, confidence interval; COVID-19, coronavirus disease 2019; ED, emergency department; SpO₂, peripheral oxygen saturation.

^a Percentage of blood oxygen saturation measured via pulse oximetry.

^b Values are expressed as mean predicted probability, shown in percentage units.

^c Patients who were discharged home after the ED visit only (n = 997).

^d Patients who were discharged home after inpatient admission (n = 236).

^e Patients with a decision to admit (n = 641).

^f Patients who were admitted to the hospital (n = 586).

^g Patients who were given dexamethasone (n = 383).

^h Patients who were given supplemental oxygen (n = 484).

Comparing the 2 counterfactuals (C₁ − C₂) allows us to estimate any differences in outcomes for NHB individuals that may be attributable to the differential SpO₂ measurement error observed between NHB and NHW patients (Table 4). This differential measurement error was associated with decreased probabilities of admission (−3.1 percentage points, 95% CI: –3.4, −2.8), dexamethasone treatment (−3.1 percentage points, 95% CI: –3.4, –2.7), treatment with supplemental oxygen (−4.5 percentage points, 95% CI: –4.9, –4.2), and return to the hospital after discharge from the ED (−1.2 percentage points, 95% CI: –1.9, –0.5). It was also associated with an increase in the probability of returning to the hospital after admission (1.0 percentage point, 95% CI: 0.2, 1.7) and with an increase in time to treatment: 37.2 additional minutes before dexamethasone initiation (95% CI: 20.1, 54.3) and 278.5 additional minutes before initiation of supplemental oxygen (95% CI: 181.0, 376.0). Differences (<10 minutes) were also estimated in minutes to admission decision and minutes spent in the ED (see Table 4).

DISCUSSION

In this observational study, we investigated whether or not pulse oximetry systematically underestimated oxygen saturation in patients who identified as NHB as compared with NHW counterparts. We also assessed whether or not differences in oxygen saturation measurement affected hospital admission, care delivered, or return to the hospital postdischarge among patients with COVID-19. We found evidence of differential pulse oximeter measurement error in NHB individuals, resulting in nonrandom overestimation of blood oxygenation as compared with NHW individuals. NHB individuals were also more likely to have hypoxemia not detected by pulse oximetry. For NHB patients presenting in the ED with COVID-19, we found that overestimation of oxygen saturation was associated with underestimation of the need for admission and underestimation of the need for treatment with dexamethasone and supplemental oxygen. Additionally, we observed associated delays in dexamethasone initiation and initiation of oxygen supplementation.

Our findings are in line with and strengthen the results of several recent studies examining SpO₂ measurement error in the context of race and ethnicity in the intensive care unit (9, 27, 33, 34). Our results also extend those of a 2022 study that established a relationship between underestimation of blood oxygenation by pulse oximetry and delays in identification of eligibility for supplemental oxygen (24). Our results diverge from one 2022 study, in which investigators assessed pulse oximeter accuracy by race/ethnicity in patients with COVID-19 admitted to a single critical-care unit in the United Kingdom (27). Unlike our study, those investigators reported no statistically significant difference between Black patients and White patients in hypoxemia not detected by pulse oximetry. The study was limited by a small sample size and a single institution, and the reported study results also do not support the authors’ conclusions; the article states that the proportion of SaO₂ measurements less than 90% with paired SpO₂ measurements of 94% or above was 29.8% (71/238) for White patients and 71.1% (27/38) for Black patients (27). This is a more than 2-fold increase for Black patients, and a comparison by χ² test yields a P value less than 0.001.

We also observed differential timing of the initial ABG for patients with SaO₂ measurements taken, which was not examined in other studies. NHB patients had a much higher rate of SaO₂ measurement in the ED prior to the decision to admit or discharge home, a difference that persisted after accounting for baseline demographic and clinical characteristics. This may indicate that clinicians were reluctant to rely on the SpO₂ measures alone given the full clinical pictures and other indicators of need for intervention upon presentation to the ED. In a recent cohort study evaluating intensive care unit patients, Wong et al. (33) found a higher likelihood of assessment by ABG among White patients (5.6%) as compared with other racial/ethnic groups (Asian, 3.4%; Black, 2.8%; Hispanic, 1.9%); however, that study was not focused on patients with COVID-19 and the investigators did not consider timing of the ABG measurement in relation to the decision to admit or discharge from the ED.

Unlike many other studies of discrepancies in pulse oximeter accuracy between NHB and NHW patients, our study did not restrict the analysis to a specific patient subgroup (such as critical-care or intensive-care-unit patients) and instead included all available measurements. This makes our results more generalizable and suggests that the observed discrepancies hold true across a broader spectrum of hospitalized patients. Perhaps the most important contribution of the current study to the literature, however, is our novel analysis of how the discrepancy in oxygen saturation as measured by pulse oximeter was associated with differences in the delivery of evidence-based care for COVID-19. To our knowledge, only one other study to date has examined this relationship; that study examined delays in eligibility for oxygen supplementation only and did not quantify downstream effects on treatment course (24). Our study therefore makes an important contribution to this topic.

Missed hypoxemia by pulse oximeter in NHB patients at intake may have given a false impression to health-care providers that NHB patients’ presentations were less severe than the clinical reality. The much higher rate of ABG blood oxygen measurement for NHB patients prior to the decision to admit or discharge home could also indicate that clinicians were less inclined to rely upon pulse oximetry alone to determine the need for intervention. Pulse oximeter–mediated underestimation of the severity of COVID-19 illness at ED presentation could have driven the delays in care and differences in the perceived need for therapies described in this study. These differences and delays in care have the potential to exacerbate preexisting disparities in COVID-19 survival for NHB patients across the United States.

Diagnostic tools and devices that differ in accuracy based on skin tone can be associated with differences in care, as we have shown in our study. This presents a serious threat to ongoing efforts to achieve health equity. Current FDA guidelines for pulse oximeters require only that they be tested for accuracy within at least 10 healthy subjects (producing at least 200 paired SpO₂/SaO₂ measurements), which must include at least 2 darkly pigmented individuals (or 15% of the study group, whichever is larger) (35). Our results suggest that this may not be adequate to calibrate pulse oximetry machines to function adequately in NHB patients or other populations of patients with darkly pigmented skin. There is a growing recognition that one important driver of differential device accuracy may be lack of adequate representation by diverse racial/ethnic groups in clinical trials and validation studies (36–38). Recently, the FDA-established Office of Minority Health and Health Equity launched an initiative to enhance equity in clinical trials by identifying barriers to clinical trial enrollment for underrepresented populations (including but not limited to ethnicity, race, age, disability, and geography) (39). To achieve equity in clinical device development and validation studies, novel and targeted interventions are needed to increase engagement for people from underrepresented groups in device trials and validation studies and to develop metrics and standards to facilitate equitable inclusion and transparency.

This study had several limitations. First, we relied upon self-reported race/ethnicity as a proxy for skin pigmentation. Standardized and accurate assessment of skin tone (e.g., via the Fitzpatrick scale) is not consistently done in health systems for all patients (40). Our approach will likely produce conservative estimates given that there will be a spectrum of actual skin tones in the study population. Given this, we cannot establish definitively from this study that differential device accuracy based on skin pigmentation was the sole driver of the observed differences between NHB and NHW patient measurements. Second, our study was reliant upon the accuracy and completeness of the electronic health record data upon which it was based. There could have been inaccuracies in the documented SaO₂ and SpO₂ measurement times. It is also possible that comorbidity information could have been missing due to differential health-care utilization; this could have biased our estimates if the missingness was associated with race/ethnicity. To assess for nonrandom distribution of missing data, we compared the rates of available diagnosis data between the NHW and NHB COVID-19 patient groups and found no evidence of nonrandom missingness. Third, because this was not a population-based study, we could not extrapolate larger impacts of differential pulse oximeter accuracy, such as underestimation of COVID-19 disease severity among nonhospitalized patients or associated delays in hospitalization. We hope that our work will help inspire future studies that will delve further into these and other important questions, such as how differential device error could affect the treatment course for other conditions.

The novel coronavirus severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and the resulting COVID-19 pandemic created a uniquely urgent need for treatment guidelines concurrent with the discovery of new knowledge about the virus and the disease. Our study suggests that the CDC’s guidelines for hospital admission and subsequent inpatient COVID-19 care are based on measures obtained from a clinical tool which may be insufficiently sensitive to detect hypoxia in NHB patients. These results call for additional investigation of pulse oximeters by the FDA, and suggest that current guidelines for development, testing, and calibration of these devices should be revisited, investigated, and revised. The number of dark-skinned individuals needed to increase accuracy should be clearly defined and new guidelines disseminated. Until recommendations for systemic change occur, physicians and care providers at the frontlines of COVID-19 triage must remain vigilant. Our results suggest that NHB patients with COVID-19 may benefit from liberal use of confirmatory ABG, especially when other clinical factors are inconsistent with pulse oximetry measurements. There are also broader implications beyond COVID-19, as differential pulse oximeter accuracy has the potential to exacerbate disparities for any condition that relies upon blood oxygenation measurement to inform clinical decision-making.

ACKNOWLEDGMENTS

Author affiliations: Institute for Advancing Health Equity, Sutter Health, Sacramento, California, United States (Sylvia E. K. Sudat, Stephanie Brown, Alice R. Pressman, Kristen M. J. Azar); Center for Health Systems Research, Sutter Health, Walnut Creek, California, United States (Sylvia E. K. Sudat); Department of Epidemiology and Biostatistics, School of Medicine, University of California, San Francisco, San Francisco, California, United States (Paul Wesson, Alice R. Pressman, Kristen M. J. Azar); Department of Epidemiology and Biostatistics, School of Medicine, University of California, San Francisco, San Francisco, California, United States (Kim F. Rhoads); Philip R. Lee Institute for Health Policy Studies, University of California, San Francisco, San Francisco, California, United States (Kim F. Rhoads); Alta Bates Summit Medical Center, Oakland, California, United States (Stephanie Brown); Berkeley Emergency Medical Group, San Ramon, California, United States (Stephanie Brown); Roots Community Health Center, Oakland, California, United States (Noha Aboelata); Research Section of Primary Care and Population Health, Department of Medicine, School of Medicine, Stanford University, Stanford, California, United States (Noha Aboelata); California Pacific Medical Center, San Francisco, California, United States (Aravind Mani); and Pacific Inpatient Medical Group, San Francisco, California, United States (Aravind Mani).

Please reach out to the corresponding author with data availability questions.

We thank Zijun Shen for his assistance with data extraction.

The views expressed in this article are those of the authors and may not represent those of their affiliated institutions.

Conflict of interest: none declared.