Validation of a Screening Risk Index for Serious Prescription Opioid-Induced Respiratory Depression or Overdose in a US Commercial Health Plan Claims Database

Abstract

Objective

To validate a risk index that estimates the likelihood of overdose or serious opioid-induced respiratory depression (OIRD) among medical users of prescription opioids.

Subjects and Methods

A case-control analysis of 18,365,497 patients with an opioid prescription from 2009 to 2013 in the IMS PharMetrics Plus commercially insured health plan claims database (CIP). An OIRD event occurred in 7,234 cases. Four controls were selected per case. Validity of the Risk Index for Overdose or Serious Opioid-induced Respiratory Depression (RIOSORD), developed previously using Veterans Health Administration (VHA) patient data, was assessed. Multivariable logistic regression was used within the CIP study population to develop a slightly refined RIOSORD. The composition and performance of the CIP-based RIOSORD was evaluated and compared with VHA-based RIOSORD.

Results

VHA-RIOSORD performed well in discriminating OIRD events in CIP (C-statistic = 0.85). Additionally, re-estimation of logistic model coefficients in CIP yielded a 0.90 C-statistic. The resulting comorbidity and pharmacotherapy variables most highly associated with OIRD and retained in the CIP-RIOSORD were largely concordant with VHA-RIOSORD. These variables included neuropsychiatric and cardiopulmonary disorders, impaired drug excretion, opioid characteristics, and concurrent psychoactive medications. The average predicted probability of OIRD ranged from 2% to 83%, with excellent agreement between predicted and observed incidence across risk classes.

Conclusions

RIOSORD had excellent predictive accuracy in a large population of US medical users of prescription opioids, similar to its performance in VHA. This practical risk index is designed to support clinical decision-making for safer opioid prescribing, and its clinical utility should be evaluated prospectively.

Opioids, Risk, Overdose, Respiratory Depression, Index

Introduction

Prescription opioid sales and consumption have quadrupled in the United States since 1999, primarily due to a substantial expansion in the prescription of opioid analgesics to manage diverse types of pain conditions [1–5]. Deaths from prescription opioid poisoning or overdose have increased in near parallel proportions, from 4,030 in 1999 to more than 14,000 in 2014 and they exceeded 15,000 in 2015 (US death statistics involving prescription opioids exclude non-methadone synthetic opioids [e.g., fentanyl, tramadol] from 2014 onwards due to a sharp surge in deaths involving synthetic opioids that is driven largely by illicitly manufactured fentanyl, which is indistinguishable from prescription fentanyl as a cause of death on death certificates) [6,7,8]. Approximately 80% of fatal overdoses are considered unintentional [9]. Morbidity associated with increased opioid prescribing is reflected in increased emergency department (ED) visits for serious opioid-induced respiratory depression (OIRD) and subsequent hospitalization [2,5,10–12].

The life-threatening manifestation of opioid overdose is central respiratory depression, which can result in respiratory arrest and be presaged by profound central nervous system (CNS) depression [13,14]. No validated tools are currently available to estimate the risk of experiencing the most serious opioid-related adverse consequence, life-threatening OIRD [3]. Therefore, in prior work, we developed a pilot screening risk index to predict a patient's likelihood of experiencing overdose or serious opioid-induced respiratory depression (RIOSORD) and to characterize their associated risk factor profile [15,16]. This information, provided by a clinically practical, evidence-based risk index, can support personalized clinical decision-making, including preventive interventions, for safer opioid prescribing.

To develop the screening instrument, we conducted a retrospective case-control study using administrative health care data from almost 1.9 million US Veterans Health Administration (VHA) patients who were dispensed an opioid during 2010 to 2012 [15]. Risk factors during the previous six months were identified among 817 cases who experienced an OIRD event and 8,170 controls who did not. Predictors included demographic variables, health conditions, and prescription drug and health care utilization factors. The index was then created with 15 of the variables most strongly associated with experiencing a life-threatening opioid emergency [16]. Risk classes determined from the predicted probability distribution indicated excellent predictive validity for OIRD in the VHA population. The objective of the current study was to assess, validate, and refine RIOSORD in a larger, non-VHA population more representative of US medical users of prescription opioids.

Methods

Study Design and Setting

This nested case-control study utilized a limited PharMetrics Plus data set from the IMS Health Real-World Data Adjudicated Claims–US Database (IMS Health Incorporated, Plymouth Meeting, PA, USA). This health plan claims database comprises fully adjudicated medical and pharmacy claims and enrollment information on over 115 million unique, de-identified individuals since 2006. The database's health plans are largely commercial and from self-insured employer groups; a small set of commercial Medicare and Medicaid patients is also included (communication with data vendor, December 5, 2016). This study, which used a limited data set, was reviewed by the Western Institutional Review Board (IRB) and was determined to be exempt from full review.

Study Sample and Participants

The study population included 18,365,497 patients in the commercially insured health plan (CIP) database with at least one opioid pharmacy claim between January 1, 2009, and December 31, 2013, excluding opioid-containing cough/cold products. Among patients with nonmissing age and sex values, we identified a cohort of patients who experienced prescription OIRD as determined by an algorithm developed in our prior work (see Supplementary Table 1) [16]. Using codes in the International Classification of Disease, 9th Revision, Clinical Modification (ICD-9-CM) [17] and Current Procedural Terminology (CPT) for critical care procedures [18], an OIRD event was defined by a listed code for prescription opioid poisoning plus a code for either a life-threatening respiratory or CNS adverse effect or mechanical ventilation or critical care. OIRD events that involved poisoning codes exclusively related to heroin were excluded. In patients with more than one OIRD event during the study period, only the first (index) event was analyzed. Four controls were randomly selected per case and assigned the case's index date. Neither matching nor control replacement was used. Eligible patients had nonmissing age and sex values and continuous medical and pharmacy benefits for the six-month baseline period before the index date. One case that occurred near the beginning of the study period could not be assigned controls due to insufficient six-month baseline data. Among patients meeting eligibility criteria, 7,234 cases were identified and 28,932 controls were thus assigned, for a total study sample of 36,166.

Variables

The dependent variable of interest was the index OIRD event. Independent variables assessed during the six months before the index date included demographic factors; the component comorbidities of the Charlson Comorbidity Index [19–21]; other selected pain- and non-pain-related health conditions [22,23]; prescription medication information, including opioid active ingredient, formulation, and maximum prescribed daily morphine equivalent dose (MED) [24,25], as well as selected nonopioid medications that may impact serious adverse opioid effects (nonopioid analgesics, benzodiazepines, antidepressants, muscle relaxants, other sedatives, antipsychotics, and stimulants); and metrics indicative of health care utilization [15].

Variables in this CIP validation study were used exactly as or constructed as closely as possible to those used in the VHA development study [16]. Some modifications were required to accommodate inherent differences between the disparate data sources. For example, the CIP data lacked adequate information regarding race/ethnicity, marital status, body mass index, and motor vehicle accidents, but included additional details on the route of opioid administration. These additional routes (sublingual, buccal, nasal, and rectal) were characterized for descriptive purposes, but route was dichotomized for multivariable analysis (oral and nonoral). To facilitate clinical utility, MED was collapsed into a dichotomous variable for the CIP analysis (<100 mg/day and ≥100 mg/day). The “opioid dependence” and “substance abuse and nonopioid dependence” variables were combined into a single “substance use disorder” (SUD) variable in CIP, consistent with the updated definition in the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5) [26].

Statistical Methods

Characterization of the CIP Study Population

Univariate statistics were calculated to characterize the CIP study population. Bivariate statistics assessed differences between cases and controls. Continuous variables of interest that were not normally distributed were compared using Wilcoxon rank sum tests. Chi-square tests were used to compare proportions of categorical variables.

Validation of VHA-RIOSORD in the CIP Study Population

The performance of the pilot RIOSORD, initially developed in the VHA study population, was then assessed in the CIP study population. VHA-based RIOSORD scores were calculated for all cases and controls in the CIP sample, and a risk class model was fit [27] to estimate the predicted probability of experiencing an OIRD event based on the risk index scores. Performance of the VHA-based RIOSORD was assessed using the risk class model C-statistic and by comparing the predicted probabilities of OIRD events with observed occurrences in the CIP study population.

Refinement of VHA-RIOSORD Using CIP Data

Thereafter, given the inherent differences between the VHA and CIP populations (e.g., sex, age distribution, and prevalence of certain health conditions [see Supplementary Tables 2 and 3], as detailed elsewhere [28]) and relevant health care system practices (e.g., drug formularies and clinical management), the VHA-based risk index was refined and evaluated using the CIP data. Multiple logistic regression was used to examine predictors of OIRD using the CIP data, starting with the variables included in the original VHA-RIOSORD logistic regression model. While clinical judgment was used to guide initial variable inclusion, final variable selection was determined by eliminating variables with P values greater than 0.10 from the model sequentially unless they were identified as confounders (i.e., removal from the model resulted in a 20% or greater change in parameter estimates for one or more of the other variables when compared with the original model). This final model developed using the CIP data was then compared with the model initially developed using VHA data; it is presented in the Results section.

The refined RIOSORD logistic regression model was then used to select and assign point values for the items included in a CIP-based RIOSORD. Items in the final CIP-based risk index were selected from those initially included in VHA-RIOSORD and from the logistic model variables clinically and statistically significantly associated with an OIRD event in the CIP study population. Overarching considerations for item selection included 1) statistical strength of association; 2) published research reports of a consistent association between the variable and OIRD; and 3) feasibility of a health care professional in a busy clinical setting readily obtaining valid information for all index items from the patient's medical record or chart.

Le Gall’s methods were used to calculate point values for items in the risk index [27]. Each item's regression-generated β coefficient was multiplied by 10 and rounded to the nearest integer. The item point values were then summed for each patient, yielding the risk index score [27,29]. A multiple logistic regression model then used the patients’ RIOSORD scores, with OIRD as the outcome variable, to produce predicted probabilities of the outcome occurring during the following six months [27,29]. A transformation was used in the logistic regression model to reduce the skewness of the risk index scores and improve model calibration [25,26]. The logistic regression formula used was

$$\text{Logit}={\beta }_0+{\beta }_1{\mathrm{X}}_1+{\beta }_2{\mathrm{X}}_2$$

with X₁= the risk index score and X₂ = ln(risk score + 100), with ln indicating natural logarithm [27,29].

Model performance was assessed with the Hosmer-Lemeshow test for overall model goodness of fit and receiver operating curves and corresponding C-statistics for model discrimination between patients with and without the outcome. Finally, to test the predictive validity of the risk index, the distribution of predicted probabilities was compared by percentiles (risk classes) with the observed occurrence of OIRD in the CIP study population. Patient count, average predicted probability of the outcome, and observed incidence of events were computed for each risk class. All statistical analyses were conducted in SAS v. 9.4 (SAS, Cary, NC, USA).

Results

Characteristics of the CIP sample and comparisons between cases and controls are shown in Supplementary Table 2 and described in detail elsewhere [28]. Given the nature of the two data sources (commercial health insurance vs an integrated governmental health care system), it is not surprising that the CIP and VHA samples differed substantially in proportions of many variables of interest (see Supplementary Table 3). Importantly, CIP had greater representation of females and younger patients.

VHA-RIOSORD Performance in CIP Data

RIOSORD, as originally developed using VHA data, was assessed in the CIP sample without modification except as necessary to accommodate four variables that were largely unavailable in CIP. VHA-based RIOSORD scores were calculated for all cases and controls in the CIP sample, and a risk class model was fit [27] to estimate the predicted probability of an OIRD event based on the risk index scores. The VHA-based risk class model's C-statistic was 0.85 in CIP data compared with 0.88 in VHA data [16], indicating consistently good discrimination between patients with and without an OIRD event in both study populations using the patients' VHA-based RIOSORD scores. Furthermore, when divided into risk classes based on predicted probabilities, the observed incidence of OIRD events in each risk class agreed well with that expected based on predicted probabilities (Table 1).

Table 1

VHA-based RIOSORD in CIP data: Risk classes and predicted probability of serious opioid-induced respiratory depression*

		OIRD event during the next 6 mo (all patients, N = 36,166)
Risk class	VHA RIOSORD score, points	Average predicted probability (95% CI), %	Actual observed incidence, %
1	0–2	2.5 (2.5–2.5)	2.3
2	3–4	4.3 (4.3–4.3)	4.7
3	5–7	6.6 (6.5–6.6)	7.0
4	8–11	10.3 (10.3–10.4)	10.3
5	12–14	14.7 (14.7–14.8)	18.1
6	15–18	19.9 (19.8–20.0)	16.2
7	19–22	29.7 (29.6–29.9)	28.9
8	23–31	45.2 (45.2–45.6)	46.7
9	≥32	75.7 (75.3–76.0)	75.5

Model performance: C-statistic = 0.85.

CI = confidence interval; OIRD = opioid-induced respiratory depression; RIOSORD = Risk Index for Overdose or Serious Opioid-induced Respiratory Depression.

*

 Interpretation example: A patient with a RIOSORD score of 30 is predicted to have a 45% chance, on average, of experiencing a life-threatening opioid emergency such as an overdose or serious respiratory depression within the six months after the RIOSORD score is calculated.

Multivariable Logistic Regression in CIP Data

Due to the inherent differences between the VHA and CIP populations (e.g., demographics and prevalence of comorbidities) and differences in drug formularies and clinical management practices between the health care systems, the VHA-based risk index was then refined and evaluated using the CIP data. Starting with the variables included in the original VHA risk index model, modifications included collapsing four MED strata into a dichotomous variable (<100 mg/day and ≥100 mg/day) and combining “opioid dependence” and “substance abuse and nonopioid dependence” into a single “substance use disorder” (SUD) variable [26]. Final variables were selected from the updated set, and corresponding coefficient estimates were calculated via multivariable logistic regression to yield a model using CIP data only (Table 2). OIRD predictors were largely concordant between the VHA [16] and CIP study populations.

Table 2

CIP-based multivariable logistic regression: Factors associated with serious opioid-induced respiratory depression*^,†

Covariate	Odds ratio	95% confidence interval
Demographic
Age group, y
18–34 (reference)
35–54	1.05	0.95–1.15
55+	1.16	1.04–1.29
Male	1.03	0.95–1.11
US census region
Northeast (reference)
Midwest	1.20	1.08–1.33
South	1.09	0.99–1.23
West	1.39	1.23–1.58
Clinical
Individual CCI comorbidities
Heart failure	2.06	1.74–2.44
Peripheral vascular disease	0.91	0.72–1.14
Cerebrovascular disease	2.52	2.18–2.92
Chronic pulmonary disease	1.72	1.56–1.89
Serious autoimmune rheumatologic disease	1.47	1.23–1.77
Chronic hepatitis/cirrhosis	1.39	0.96–2.00
Warfarin treatment	0.79	0.66–0.95
Renal disease with renal impairment	2.17	1.83–2.57
Any malignancy, including leukemia and lymphoma	1.09	0.93–1.29
Skin (pressure) ulcers	1.50	1.18–1.90
Metastatic solid tumor	0.95	0.73–1.23
Other selected comorbidities
Non-pain-related
Substance use disorder	12.74	11.46–14.16
Bipolar disorder/schizophrenia‡	2.85	2.44–3.32
Sleep apnea	1.33	1.16–1.52
Cardiovascular disease	0.98	0.81–1.20
Nonmalignant pancreatic disease	2.07	1.56–2.75
Skin infections/abscesses	1.14	1.00–1.30
Pain-related
Recurrent headache	1.73	1.57–1.90
Active traumatic injury	1.53	1.41–1.65
Prescription drugs
Opioids
By active ingredient
Hydrocodone	1.30	1.20–1.41
Oxycodone	1.32	1.19–1.45
Hydromorphone	1.50	1.38–1.64
Morphine	2.93	2.49–3.43
Fentanyl	3.72	3.10–4.46
Methadone	2.80	2.22–3.51
Tramadol	1.19	1.08–1.31
By formulation§
Not ER/LA (reference)
ER/LA	1.73	1.51–1.99
By route§
Nonoral (reference)
Oral	1.90	1.54–2.34
Maximum prescribed daily morphine equivalent dose, mg/d
<100 (reference)
≥100	2.04	1.87–2.24
Selected nonopioid drugs
Benzodiazepines	2.35	2.18–2.54
Antidepressants	2.19	2.03–2.36
All-cause health care utilization
≥1 ED visit	1.52	1.41–1.65
≥1 d of hospitalization	1.12	1.02–1.23

Model performance: C-statistic = 0.91.

CCI = Charlson Comorbidity Index; ED = emergency department; ER/LA = extended-release/long-acting; MED = morphine equivalent dose.

*

 A serious prescription opioid–related respiratory or central nervous system (CNS) depression event was defined as a listed opioid poisoning or external cause code occurring within ±1 day of a listed 1) CNS or respiratory adverse effect code or 2) mechanical ventilation or critical care code. All primary and nonprimary codes were considered.

†

 The multivariable logistic regression model includes all variables retained at a P value of less than 0.10 as well as all variables considered to be confounders (i.e., removal from the model resulted in a 20% or greater change in parameter estimates for one or more of the other variables). All of these variables are presented in this table and summarize the output from the model in which they were simultaneously tested.

‡

 Bipolar disorder and schizophrenia were combined into one variable, “bipolar disorder/schizophrenia,” for multivariable modeling.

§

 Missing opioid formulation (ER/LA), route, and MED information were analyzed in the reference group in regression modeling. Sensitivity analyses were conducted to examine the impact of this and found no appreciable difference between such models relative to those in which the missing data were excluded.

The single factor most strongly associated with OIRD in the CIP study population was a SUD diagnosis at a health care encounter during the six months before the index event. Bipolar disorder or schizophrenia and cerebrovascular disease were strongly associated with increased odds of OIRD and recurrent headache was a moderate predictor. Other strongly associated comorbidities included renal disease, heart failure, and nonmalignant pancreatic disease. Chronic pulmonary disease and sleep apnea were moderate risk factors.

The prescription opioids that were most strongly associated with OIRD (Table 2) were fentanyl, morphine, and methadone. Hydromorphone, oxycodone, hydrocodone, and tramadol prescriptions, as well as maximum prescribed daily MED of 100 mg or greater and extended-release/long-acting (ER/LA) formulation, were moderate predictors of an OIRD event. Concurrent benzodiazepine or antidepressant prescription strongly increased the likelihood of experiencing OIRD.

The likelihood of experiencing OIRD was greater in patients age 55 years or older and in those with increased health care utilization in the six months before the index date, including one or more ED visits or days hospitalized.

CIP-RIOSORD Performance in CIP Data

After refining the original VHA-based model using CIP data (CIP-based model), the statistically significant correlates remaining in the final model that were retained as items in the CIP-RIOSORD are shown in Table 3. Table 4 shows the CIP-based risk classes with corresponding ranges of CIP-RIOSORD scores and compares the average predicted probability for each risk class with the corresponding observed incidence of OIRD. Based on risk factors active during the six months before the index date, the average predicted probability of an event ranged from 2% in the lowest risk class to 83% in the highest, and the actual occurrence of an event increased commensurately. The CIP-based risk class model's C-statistic in CIP data was 0.90, indicating an excellent discrimination of 90% between patients with and without an event, and was similar to, although somewhat better than, the VHA-based risk class model's performance in CIP data (C-statistic = 0.85) (Table 1).

Table 3

CIP-based risk index for serious opioid-induced respiratory depression (RIOSORD)

Question*	Points for “yes” response
In the past 6 months, has the patient had a health care visit (outpatient, inpatient, or ED) involving any of the following health conditions?†
Substance use disorder (abuse or dependence)?	25
(This includes alcohol, amphetamines, antidepressants, cannabis, cocaine, hallucinogens, opioids, and sedatives/anxiolytics)
Bipolar disorder or schizophrenia?	10
Stroke or other cerebrovascular disease?	9
Kidney disease with clinically significant renal impairment?	8
Heart failure?	7
Nonmalignant pancreatic disease (e.g., acute or chronic pancreatitis)?	7
Chronic pulmonary disease (e.g., emphysema, chronic bronchitis, asthma, pneumoconiosis, asbestosis)?	5
Recurrent headache (e.g., migraine)?	5
Does the patient consume:
Fentanyl?	13
Morphine?	11
Methadone?	10
Hydromorphone?	7
An extended-release or long-acting formulation of any prescription opioid?‡	5
A prescription benzodiazepine?	9
A prescription antidepressant?	8
Is the patient's current maximum prescribed opioid dose ≥100 mg morphine equivalents per day? (Include all prescription opioids consumed on a regular basis)	7
Total point score (maximum = 146)

*

 This questionnaire is intended to be completed and interpreted by a health care professional. It is not a replacement for clinical judgment and is intended to guide and inform clinical decision-making in patients who are treated with opioids.

†

 The condition does not have to be the primary reason for the visit, but it should be entered in the chart or electronic health record as one of the reasons or diagnoses for the visit.

‡

 Extended-release/long-acting (ER/LA) formulation and certain opioid active ingredients were significantly and independently associated with the likelihood of overdose in the model. As such, ER/LA and each active ingredient are included and scored as independent factors in the risk index. For example, a fentanyl ER formulation or methadone receives RIOSORD risk points for both the active ingredient and the ER/LA formulation. A short-acting fentanyl receives points for the active ingredient only. ER/LA risk points are counted only once, regardless of the number of ER/LA opioid products that the patient consumes.

Table 4

CIP-based RIOSORD: Risk classes and predicted probability of experiencing serious opioid-induced respiratory depression*

		OIRD event during the next 6 mo (all patients, N = 36,166)
Risk class	CIP RIOSORD score, points	Average predicted probability (95% CI), %	Actual observed incidence, %
1	0–4	1.9 (1.9–1.9)	2.1
2	5–7	4.8 (4.8–4.9)	5.4
3	8–9	6.8 (6.8–6.8)	6.3
4	10–17	15.1 (15.1–15.3)	14.2
5	18–25	29.8 (29.7–30.0)	32.2
6	26–41	55.1 (54.8–55.4)	58.8
7	≥42	83.4 (83.2–83.7)	82.4

Model performance: C-statistic = 0.90.

CI = confidence interval; OIRD = opioid-induced respiratory depression; RIOSORD = Risk Index for Overdose or Serious Opioid-induced Respiratory Depression.

*

 Interpretation example: A patient with a RIOSORD score of 30 is predicted to have a 55% chance, on average, of experiencing a life-threatening opioid emergency such as an overdose or serious respiratory depression within the six months after the RIOSORD score is calculated.

Discussion

Validation of RIOSORD

RIOSORD had excellent predictive accuracy in a retrospective validation in a large commercial health plan database, similar to its performance of nearly 90% accurate discrimination in the VHA database in which it was developed. RIOSORD is the first known screening tool for the risk of opioid overdose. It is designed to support clinical decision-making and safer opioid prescribing. It provides an evidence-based, real-time, quantitative estimate of a patient's likelihood of experiencing a life-threatening opioid emergency in the next six months and also characterizes the patient's specific risk factor profile.

This external validation study confirmed that several well-accepted risk factors for fatal opioid overdose were also strong predictors for largely nonfatal OIRD events. The strongest OIRD predictors in the CIP sample were selected health conditions and prescription drug characteristics. These factors were largely consistent between the VHA and CIP samples despite substantial differences between the source populations; both are discussed in detail elsewhere [28]. Several of the differences between the two populations and health care systems made it important to examine and refine the risk model based on the CIP data, rather than stopping with the VHA-derived model deployed in the CIP data. This refinement enabled the development of a risk class model based on the CIP population, which is more representative than the VHA of US medical users of prescription opioids, and allowed subsequent comparison with the original VHA-derived model. There was modest improvement in performance (C-statistic = 0.90 vs 0.85) between the CIP-based risk class model in the CIP data and the original VHA-derived risk class model in the CIP data. Thus, the CIP-RIOSORD is appropriate, and probably preferred, for general use, as it accounts for a population more generalizable for US patients treated with prescription opioids, more diverse drug formularies, and a broader range of clinical practice than the VHA. Nonetheless, additional evaluation will be required to determine which model performs best in any specific clinical setting.

Intended Use and Application of Results in the Clinical Setting

RIOSORD was designed and intended to fill a practical need for a relatively brief screening instrument with optimum simplicity and predictive validity in a typically busy clinical practice. Its excellent predictive accuracy is predicated, in part, on complete and accurate responses to queries about active clinical conditions and details of currently prescribed medications. RIOSORD is intended for completion and interpretation by a health care professional familiar with the patient's medical history or using the patient's medical record.

Risk calculators are commonly used in clinical practice to estimate the likelihood of an adverse outcome associated with an intervention [29–32]. An accurate, individualized risk estimate and risk factor profile can enhance and simplify a patient-centered informed consent process, which involves informing the patient about the risks and benefits of a proposed treatment [33–35]. Valid risk stratification may facilitate more rational prioritization of preventive interventions and limited clinical resources to patients identified as having elevated risk who are likely to benefit most [36,37]. Modifiable elements identified in the RIOSORD risk factor profile can be specifically targeted for mitigation. Improved control of even chronic conditions such as pulmonary and mental health disorders can reduce their impact on the likelihood of experiencing OIRD. A baseline risk score can be calculated before initiating opioid treatment and reassessed and tracked longitudinally during chronic opioid treatment to evaluate fluctuations in risk level in response to introducing interventions or changes in the patient's clinical condition or treatment regimen. Preventive and potentially life-saving interventions and recommended precautions in all opioid-treated patients include heightened caution when selecting opioids and escalating dosage, comanagement with addiction, mental health or pain specialists as indicated, patient and caregiver education and awareness, high vigilance for known risk factors and manifestations of OIRD, and, in opioid emergencies, timely resuscitation and administration of the opioid reversal agent naloxone, such as the take-home formulations that are approved for use in nonmedical settings [13,34,38–40]. Primary care professionals express concern about opioid analgesic misuse and iatrogenic addiction and find managing patients with chronic pain to be stressful [41]. More than half feel that they are inadequately trained in prescribing opioids and treating/managing opioid use disorder [42]. Use of RIOSORD creates a uniquely practical opportunity for recurring education and reinforcement of evidence-based best practices to frontline health care professionals regarding comprehensive pain management and opioid prescribing.

Strengths and Limitations

An important strength of this validation study is use of the largest known sample of prescription opioid-related overdose cases and controls. Further, RIOSORD's consistently high level of predictive accuracy in two of the largest national claims databases, which represent disparate and independent populations (privately vs publicly insured), reinforces its likely external generalizability to other US populations.

Nonetheless, some important limitations should be noted. First, administrative health care data do not capture all known predictors, such as behavioral and social characteristics, family history and genotype, medication adherence, medications obtained from other sources, illicit substance use, and specifics regarding the indication for prescribing an opioid (e.g., analgesia vs medication-assisted treatment of opioid use disorder) or nonopioid medication. As such, RIOSORD may be subject to residual confounding. Additionally, administrative data are subject to missing or incomplete data, coding errors, and misclassification. Finally, the OIRD outcome is defined by an administrative health care coding algorithm whose clinical validation using linked medical records is planned but not yet completed. Of note, an analysis of administrative database research studies found that only 12% of studies using diagnostic codes (e.g., ICD-9 or ICD-10) measured or referenced the statistical association of the coding definition with the clinical concept that it was intended to represent (i.e., clinical validation) [43].

Conclusion

RIOSORD is a novel, evidence-based, statistically robust decision support tool designed to provide an individualized, quantitative estimate of the risk of and risk factor profile for a life-threatening respiratory/CNS opioid emergency among patients prescribed opioids. The excellent predictive accuracy of the pilot (VHA) version of RIOSORD was validated retrospectively in a US population of approximately 18 million medical users of prescription opioids. RIOSORD's predictive accuracy and clinical utility should be evaluated prospectively under real-world conditions. Additional insights regarding the mechanism(s) by which each identified risk factor enhances risk and refinements to improve RIOSORD's performance may result from its examination in clinically important subgroups or by assessment of interactions among risk factors. Examples include age groups, sex, patients with certain concurrent health conditions (e.g., mental health disorders or SUD) or medications (e.g., benzodiazepines), overdose outcome (fatal vs nonfatal), opioid indication (e.g., pain vs medication-assisted treatment of opioid use disorder), and duration and regularity of opioid use. Finally, integration of RIOSORD within the electronic health record can facilitate automated risk calculation and longitudinal tracking and provide individualized information to support point-of-care decision-making and recurring education of health care professionals regarding OIRD risk mitigation.

Acknowledgments

The authors express their gratitude to Seddon Savage, MD, MS, for her thoughtful review and comments on this paper.

Supplementary Data

Supplementary Data may be found online at http://painmedicine.oxfordjournals.org.

Author notes

Funding sources: This research was funded by Kaleo, Inc., Richmond, Virginia.

Disclosure and conflicts of interest: The study was conceived, designed, executed, and reported by the authors, who had sole control over the data set, data analysis, and decision to publish. Kaleo, Inc., reviewed the final manuscript for proprietary information and did not change the manuscript. Drs. Zedler, Joyce, and Murrelle are principals of Venebio Group, LLC, which has research and consulting agreements with Kaleo, Inc., and Indivior, PLC, and report no additional potential conflicts of interest. All other authors report no potential conflicts of interest.