616. Predicting Misdiagnoses of Infectious Disease in Emergency Department Visits

Abstract Background Diagnostic error leads to delays of care and mistaken therapeutic decisions that can cascade in a downward spiral. Thus, it is important to make accurate diagnostic decisions early on in the clinical care process, such as in the emergency department (ED). Clinical data from the Electronic Health Record (EHR) could identify cases where an initial diagnosis appears unusual in context. This capability could be developed into a quality measure for feedback. To that end, we trained a multiclass machine learning classifier to predict infectious disease diagnoses following an ED visit. Methods To train and evaluate our classifier, we sampled ED visits between December 31, 2016, and December 31, 2019, from Veterans Affairs (VA) Corporate Data Warehouse (CDW). Data elements used for prediction included lab orders and results, medication orders, radiology procedures, and vital signs. A multiclass XGBoost classifier was trained to predict one of five infectious disease classes for each ED visit based on the clinical variables extracted from CDW. Our model was trained on an enriched sample of 916,562 ED visits and evaluated on a non-enriched blind testing set of 356,549 visits. We compared our model against an ensemble of univariate Logistic Regression models as a baseline. Our model was trained to predict for an ED visit one of five infectious disease classes or “No Infection”. Labels were assigned to each ED visit based on ICD-9/10-CM diagnosis codes used elsewhere and other structured EHR data associated with a patient between 24 hours prior to an ED visit and 48 hours after. Results Classifier performance varied across each of the five disease classes (Table 1). The classifier achieved the highest F1 and AUC for UTI, the lowest F1 for Sepsis, and the lowest AUC for URI. We compared the average precision, recall and F1 scores of the multiclass XGBoost with the ensemble of Logistic Regression models (Table 2). XGBoost achieved higher scores in all three metrics. Table 1. Classification performance XGBoost testing set performance in each disease class, visits with no labels, and macro average. The infectious disease classes with the highest score in each metric are shown in bold. Table 2. Baseline comparison Macro average scores for XGBoost and baseline classifiers. Conclusion We trained a model to predict infectious disease diagnoses in the Emergency Department setting. Future work will further explore this technique and combine our supervised classifier with additional signs of medical error such as increased mortality or anomalous treatment patterns in order to study medical misdiagnosis. Disclosures All Authors: No reported disclosures

Conclusion. The year of COVID19 occurred in 2 distinct waves. W1 was short and intense. The age and gender distributions were the same between the waves. Even though wave 2 was numerically greater, the cases in SNF were statistically less than the first wave likely from improved IP practice initiated in W1. The numbers of visits per patient, a surrogate for LOS, was statistically less in W2. The decline in test positivity paralleled deployment of vaccination. Despite an intensity of exposure of 158 patients/provider or 1198 visits/provider to SARSCoV2 infected persons only 8% of the clinician staff were infected. ID clinical practice can use electronic databases to help describe regional outbreaks of transmissible disease giving additional perspective across the care continuum. A more usable standard tool would enhance this capacity.

Predicting Misdiagnoses of Infectious Disease in Emergency Department Visits
Alec B. Chapman, MS 1 ; Kelly Peterson, M.S. Computational Linguistics 2 ; Wathsala Widanagamaachchi, PhD 1 ; Makoto M. Jones, MD 3 ; 1 University of Utah, Salt Lake City, Utah; 2 University of Washington, Salt Lake City, Utah; 3 IDEAS Center of Innovation, VA Salt Lake City Health Care System, Salt Lake City, Utah

Session: P-27. Clinical Practice Issues
Background. Diagnostic error leads to delays of care and mistaken therapeutic decisions that can cascade in a downward spiral. Thus, it is important to make accurate diagnostic decisions early on in the clinical care process, such as in the emergency department (ED). Clinical data from the Electronic Health Record (EHR) could identify cases where an initial diagnosis appears unusual in context. This capability could be developed into a quality measure for feedback. To that end, we trained a multiclass machine learning classifier to predict infectious disease diagnoses following an ED visit.
Methods. To train and evaluate our classifier, we sampled ED visits between December 31, 2016, and December 31, 2019, from Veterans Affairs (VA) Corporate Data Warehouse (CDW). Data elements used for prediction included lab orders and results, medication orders, radiology procedures, and vital signs. A multiclass XGBoost classifier was trained to predict one of five infectious disease classes for each ED visit based on the clinical variables extracted from CDW. Our model was trained on an enriched sample of 916,562 ED visits and evaluated on a non-enriched blind testing set of 356,549 visits. We compared our model against an ensemble of univariate Logistic Regression models as a baseline. Our model was trained to predict for an ED visit one of five infectious disease classes or "No Infection". Labels were assigned to each ED visit based on ICD-9/10-CM diagnosis codes used elsewhere and other structured EHR data associated with a patient between 24 hours prior to an ED visit and 48 hours after.
Results. Classifier performance varied across each of the five disease classes ( Table 1). The classifier achieved the highest F1 and AUC for UTI, the lowest F1 for Sepsis, and the lowest AUC for URI. We compared the average precision, recall and F1 scores of the multiclass XGBoost with the ensemble of Logistic Regression models (Table 2). XGBoost achieved higher scores in all three metrics. Background. Dalbavancin and Oritavancin are semisynthetic lipoglycopeptides (LGP) that are FDA-approved for treatment of skin and soft tissue infections, but emerging data supports LGP use for other serious gram positive (GP) infections. We describe our experience with LGP during the COVID-19 pandemic.
Methods. We initiated a quality improvement project to assess the use of LGP for label and off-label indications at the Atlanta Veterans Affairs Health Care System. We define serious GP infections as infective endocarditis, osteomyelitis, joint infections, or bacteremia. Patients with serious GP infections that receivedLGP were selected at the treating physician's discretion. We reviewed medical records of all patients receiving at least one dose of long-acting LGP from March 1, 2020 -May 31, 2021. We described patient demographics, clinical information,and outcomes (90-day readmission).
Results. Nineteen patients with GP infections received LGP (table). Overall, the most common infection was cellulitis 7 (35%); 14 patients received LGPs for serious GP infections. All patients received at least one other non-LGP antibiotic for at least 2 days, majority vancomycin (60%) and cefazolin (30%). Overall, the median hospital stay among patients who received LGP was 8.5 days (range: 2-45 days), for those with serious GP infections the median hospital stay was 15 days (range: 4-45). 90% of patientswho received LGP were discharged home. Number of LGP doses ranged from 1 to 6 doses total, based on type of infection. Sixteen veterans (80%) followed up in outpatient clinicfollowing discharge within 2 weeks, two patients were discharged to home hospice due to complications of underlying malignancies and two patients were lost to follow up. Noadverse drug events were reported, and none with serious GP infections required rehospitalization at 90 days.