Clinical Validation of a Novel Quality Management System for Blood Gas, Electrolytes, Metabolites, and CO-Oximetry.

BACKGROUND
Quality management of point-of-care (POC) blood gas testing focuses on verifying instrument accuracy and precision, in addition to performing daily quality control (QC) checks every 8 h and with each patient test (unless internal calibration is verified every 30 min). At the POC, a risk-based approach is suitable to address both systemic and transient sample-specific errors that may negatively impact patient care.


METHODS
We evaluated the performance of the GEM® Premier™ 5000 with next generation Intelligent Quality Management 2 (iQM®2) (Instrumentation Laboratory, Bedford, MA), from the analysis of approximately 84,000 patient samples across 4 sites. Continuous iQM2 was compared to intermittent liquid QC, either manual or automated, at 2 sites. Analysis of error flags for patient samples and statistical characteristics of QC processes, including method sigma and average detection time (ADT) for an error, were examined.


RESULTS
ADT was approximately 2 min with iQM2 and varied from hours to days with intermittent QC. iQM2 Process Control Solutions (PCS) precision was similar or better (>6 sigma for all analytes) than manual (sigma 3.0 for pO2) or automated internal QC (sigma 1.3 for tHb and sigma 3.3 for pO2). In addition, iQM2 detected errors in ∼1.4% of samples, providing an additional safeguard against reporting erroneous results.


CONCLUSIONS
The findings in this study demonstrate excellent performance of the GEM Premier 5000 with iQM2 including >6 sigma precision for all analytes and faster error detection times. These benefits address risk in different phases of testing that are not easily detected by intermittent performance of liquid QC (manual or automated).


INTRODUCTION
Point-of-care testing (POCT) has clinical benefits in acute care settings, including improved result turnaround time, expedited clinical decisions, and time-to-treatment that enhance patient outcomes (1)(2)(3)(4)(5). The economic benefits of POCT include reduced patient length of stay, decreased labor costs due to reduced steps for sample processing, and overall institutional monetary savings.
POCT operators have varying levels of education and expertise with diagnostic testing and face increased time restrictions, as they balance caring for patients while conducting POCT. This presents a risk of errors throughout the testing cycle (5)(6)(7)(8)(9). The majority of errors (70%) occur during sample collection and preparation, prior to sample insertion into the instrument (preanalytical test phases, which includes pre-preanalytical phase as defined by Plebani et al. (7)). Clinical staff operators have increased potential for operator variability and distractions at the point-of-care (POC), potentially adversely impacting patient results (4,5,6,8,9).
Quality control (QC) for blood gas testing has focused on error detection and risk reduction primarily in the analytical phase of the testing process (4,10). The analysis of liquid QC conducted intermittently every 6-8 h detects errors at the time of QC analysis but can miss several sources of error in between scheduled QC analysis. Therefore, additional checks and processes are required to continuously monitor/mitigate sources of error throughout the testing cycle (1,4,(7)(8)(9)(10), in particular, bubbles, interferences, clots, or other transient events that could affect the analytical performance of the analyzer (11).
To help mitigate preanalytical errors, blood gas manufacturers have implemented additional safeguards in their latest devices. The GEM Premier 5000 (Instrumentation Laboratory, Bedford, MA) employs continuous quality checks throughout the testing process-prior to sample measurement-ensuring hardware, software, and analytical functionality, and during and after sample measurement, using innovative patternrecognition algorithms. The first-generation of Intelligent Quality Management, or iQM, has been previously reviewed (12,13), and has been well recognized for its capability for detecting systemic or sample-specific errors in real-time. The secondgeneration, iQM2, includes new checks and IntraSpect TM technology, which detects sensor patterns related to errors during the sample measurement process, potentially caused by microclots, micro-bubbles, interferences, or any other transient errors (14)(15). These types of errors can go undetected with liquid-based QC processes, and are likely to cause erroneous results (10). The purpose of this study is to evaluate the performance of iQM2 on the GEM Premier 5000 for error detection and automatic correction of errors, including those originating from preanalytical and analytical phases of testing. In addition, the study compares error-detection capabilities of continuous iQM2 vs intermittent QC testing.

METHODS AND MATERIAL
iQM2 performance data from 4 hospitals, where the GEM Premier 5000 is in routine clinical use,

IMPACT STATEMENT
The GEM V R Premier TM 5000 was evaluated during clinical use to assess iQM2 performance by monitoring more than 84,000 POC patient samples. The study compared the performance of continuous Process Control Solutions (PCS) to intermittent scheduled QC by assessing method precision, sigma, probability of error detection (Ped), probability of false rejection (Pfr) and average detection time (ADT) for errors.
Additionally, the study quantified errors detected by iQM2 in clinical sites across 4 institutions. The GEM Premier 5000 detected errors common in POC testing due to preanalytical sources that are not otherwise detectable or correctable by intermittent QC. were included in this study. No patient information or analyte measurements were used and nonpatient-related information such as QC data, analyzer iQM2 reports, and cartridge reagent solution data were analyzed to generate the data presented in this article. The Institutional Review Board determined that the study does not qualify as "human subject" research per §46.102(f) (2).

ARTICLE
Three hospitals across the United States (US) and 1 hospital in the United Kingdom (UK), analyzing more than 84,000 samples on 106 GEM The data from all analyzers were collected and processed using a commercially avaliable software (Microsoft V R Excel).

GEM Premier 5000 with iQM2
The GEM Premier 5000 is designed to analyze whole blood samples rapidly for blood gas, electrolytes, glucose, lactate, hematocrit, total bilirubin (tBili), and CO-Oximetry (tHb, O 2 Hb, COHb, MetHb, HHb, and sO 2 ) at the POC or in a central laboratory (16). The system includes 2 components: the instrument and a disposable, multi-use cartridge (GEM PAK), which contains 5 process control solutions (PCS), sensors and optical cell for electrochemical, CO-Oximetry (CO-Ox), and tBili measurements. The GEM PAK 5 PCSs (A, B, C, D, and E) evaluate the performance of each analyte during cartridge use-life. Each PCS contains known quantities of analytes traceable to National Institutes of Standards and Technology (NIST), internal standards, and factory-assigned concentrations stored on the cartridge. PCSs are performed continuously throughout the day to confirm cartridge analytical performance and monitor accuracy at medical decision levels. PCS B is exposed to sensors and the CO-Ox module during the uselife except when other PCSs or blood samples are being measured. PCS A, C, D, and E are run at continuous frequencies or on-demand by iQM2 for error detection and corrective actions. PCS B signals are monitored every 30 s to evaluate analytical or fluidic system stability and a fluidic refresh is initiated when system or fluidic instability is detected. During each cycle, PCSs are measured and compared against expected values (or iQM2 limits) for analytical accuracy. If a process is outside the limits, a correction is automatically initiated or if unrecoverable, the sensor/cartridge is rejected. PCSs are exposed to the same fluidic pathway as a patient sample or ampoule-based or internal liquid QC and are affected by the same factors as a sample. Upon insertion of the GEM PAK into the instrument and warmup, AutoPAK Validation (APV) automatically validates the integrity of the PCSs and overall performance of the analytical system. After successful completion of APV, iQM2 takes over to manage the QC process, replacing the need for intermittent ampoule QC or internal liquid QC (16).
The analytical performance of the GEM Premier 5000 is constantly monitored by iQM2 throughout GEM PAK use-life. The iQM2 quality system is based on 5 modules, developed to identify errors, initate corrective actions tailored to the source of error, and document corrective actions taken: 1. System: monitors the function of vital hardware components (i.e., sensors, optics, pumps, electrical and mechnical controls) before each sample analysis. 2. Sensor: performs 5 levels of PCSs throughout the day for real-time error detection. 3. IntraSpect: detects transient sample-specific errors through pattern recognition using the patient sample as a control.

Pattern
Recognition: applied to every patient sample to identify common sources of error such as micro-clots or fibrin strands on the sensors (Fig. 1), and interference by compounds such as benzalkonium 5. Stability: verifies stability of PCSs and cartridge integrity during GEM PAK use-life.
A major innovation in the second generation iQM system is IntraSpect (IS) technology that monitors sample quality using the sensor measurement response curve of each patient sample. During sample measurement, IS technology collects a series of sensor output readings and applies a pattern recognition software to detect abnormal sensor behavior caused by a transient event that could affect sensor performance (e.g., micro-clots, micro-bubbles). iQM2 with IS technology is a novel form of patient data QC using the patient sample essentially as its own control.
An example of iQM2 functionality during a sample process, including system and sensor checks, IS technology, and pattern recognition checks (in this case, a clot pattern) is described in Figure 1. Prior to sample aspiration, PCS process and pattern recognition checks ensure the analyzer  readiness for a patient sample. During aspiration, sample volume and sample integrity are verified, and sample aspiration is halted if insufficient volume, bubbles, or flow blockage occurs. After sample aspiration, temperature of the sensor and CO-Ox modules are verified to be 37 C and IS technology analysis of sample mV pattern is performed to assess sample quality. If pattern coefficients are within limits, results are reported. If outside limits, the sample is flagged and the affected analyte results are not reported. Following sample measurement, PCS B is analyzed and if the sensor baseline is outside limits, using pattern recognition, a specific corrective action is applied. Figure 1 demonstrates an example of a clot pattern and corrective action (using clot bust cleaning corrective action), followed by baseline verification with PCS B and A. During each PCS A process, reagent stability is confirmed.

Statistical Analysis of iQM2 PCSs and Intermittent QC Processes
To characterize the performance of QC, the following statistical parameters were calculated: average, total standard deviation (SD), which includes all types of variation, and coefficient of variation (CV%) to support the following calculations: Total Allowable Error (TEa) is the total error allowable for each analyte. TEa values used in this evaluation were derived from Clinical Laboratory Improvement Amendments (CLIA) 88 and College of American Pathologist (CAP) Proficiency Test limits.
• Control Limit Calculation ¼ QC limit for each analyte/SD (or %CV).
• Probability of false rejection (Pfr) ¼ 1-Cumulative normal standard distribution of Control Limit.
• Overall Pfr and Ped were calculated by weighted average of the probabilities of the levels of each QC system (P L1 *N L1 þ where P is the Pfr or Ped per PCS or QC level and N the number of replicates. Overall Pfr and Ped is the probability of Pfr or Ped by any of the QC levels in any given run. • Overall average run length for rejectable quality (ARL) ¼ 1/ Overall Ped.
• Overall average detection time for the check (ADT) ¼ ARL x sampling time (or how often the QC material is run in the system).
Pfr, Ped, and ADT for each level of PCS or QC were calculated as described in Westgard et al. (12).

Additional Evaluation of the iQM2 Process
Analysis of any iQM2 user-reported sample errors detected during the routine clinical use of 105 GEM PAK cartridges across 4 hospitals is included in this evaluation. Percentage of sample flags, sources of errors detected, time for corrective action per event, and overall time per corrective action were calculated.

RESULTS
Pooled PCS data from the GEM Premier 5000 systems from all 4 hospitals, as well as the data from the intermittent manual QC from the GEM Premier 5000 and auto QC from the ABL800 FLEX analyzer were examined to determine the performance characteristics of each QC process. Deoxygenated hemoglobin (HHb) and tBili values were not included in the analysis on the ABL800 system because those parameters were not enabled.
Detailed statistical analysis for each PCS and QC process is summarized and presented in Tables  1-3 for PCSs, manual QC on the GEM Premier 5000 system, and auto QC on ABL800 FLEX, respectively. An example of this analysis is illustrated below for pH and PCS A, with a mean of 6.90 and   Method sigma average values for the iQM2 system were >6 for all reported analytes. For manual QC (GSE) analyzed on the GEM Premier 5000, the sigma values were >6 for most analytes, except pO 2 sigma ¼ 3.0, Na þ sigma ¼ 5.8, and tHb sigma ¼ 5.2. For automated QC on the ABL800 FLEX, sigma values were >6 for all analytes, except pO 2 sigma ¼ 3.3, pCO 2 sigma ¼ 5.8, and tHb sigma ¼ 1. 3.
ADT values for the iQM2 system were 2 min for most analytes, except Na þ (5 min) and glucose (21 min). ADT for traditional QC performed with an ampoule (GSE) were >100 h for pH, pO 2 , Na þ , tHb, and CO-Ox, 23 h for pCO 2 , 14 h for K þ , 9 h for Ca þþ , 33 h for Cl -, 90 h for glucose, 16 h for lactate, and 12 h for tBili. ADT for the automated intermittent QC (on the ABL800) were >100 h for Na þ , Cl -, and glucose, 6 h for pH, K þ , O 2 Hb, and MetHb, 12 h for pCO 2 , 64 h for pO 2 , 22 h for Ca þþ , 72 h for lactate, 89 h for tHb, and 8 h for COHb. iQM2 sample errors reported to operators were analyzed from the cartridge data files and are shown in Table 4. Out of 83,964 samples analyzed on the cartridges, 1,142 (1.367%) samples were flagged by iQM2. iQM2 pattern recognition detected sample interference in 522 samples (0.631%) and clots in 259 (0.31%) samples. IS technology detected errors on 361 (0.426%) samples and prevented reporting of these results. The corrective actions for these events varied from immediate resolution of IS errors and interfering substances to 11 min for clot removal and recovery.
Two IS examples of errors exceeding TEa that were observed in this study are reviewed in Figure 2. Table 4. Summary data for iQM2 flags and estimated range of impact of error, and average time to error correction (samples n ¼ 83,964). Pre-pre-analytical, pre-analytical

DISCUSSION
Performance of the GEM Premier 5000 was compared to manual QC (GSE) testing on the GEM Premier 5000 and auto QC testing on the ABL800 FLEX analyzer, shown in Tables 1-3. Sigma metrics were evaluated since it is difficult to compare precision performance across different analytes. The GEM Premier 5000 demonstrated equal or better performance across all 5 PCSs with >6 sigma for all analytes compared to manual or automated intermittent QC processes. Method sigma values of <6 were noted for pO 2 , Na þ , and tHb for GSE on the GEM Premier 5000 and for pCO 2 , pO 2 , and tHb analytes for auto QC on ABL800. The analyte pO 2 consistently showed lower method sigma ¼ 3.0 and 3.3 on GSE manual and auto QC, respectively, which is expected since pO 2 is the most challenging parameter with high variability caused by external factors (e.g., temperature or air contamination) and narrow total allowable error limits. With the exception of pO 2 and tHb, all other analytes demonstrated high QC process capabilities, with >6 sigma values on iQM2 or other QC processes.
Calculation of ADT demonstrated the capability of error detection and time for iQM2 detected errors compared to other QC processes (common to other blood gas analyzers). This ensures timely correction and availability of the analytical system for patient sample testing. There is a noticeable difference in the ADT results between iQM2 and other QC processes. iQM2 is capable of detecting errors in 2 min for all analytes with the exception of Na þ (5 min) and glucose (21 min). This is consistent with previous evaluations of the original iQM process where Na þ and glucose were also the sensors with higher ADT (12,13). In the case of glucose, an increase in detection time is observed when compared with previous publications due to tighter TEa limits adopted over the past decade. In the case of Na þ , the ADT decreased from previous publications due to the improvement in sensor precision. However, the ADT for intermittent QC processes required a minimum of 8 h for GEM Premier 5000 using manual QC (GSE) or a minimum of 6 h for automated QC on the ABL800. For many analytes, with traditional QC (manual or automated), the system was not able to detect the error, even after several days of QC testing.
The observed low ADT values for GEM Premier 5000 are the result of high frequency system monitoring, high Ped values without causing high Pfr (false rejections), and Pfr close to 0 for most analytes that significantly outperforms the capabilities of intermittent QC processes. The results obtained in this study suggest the need for more frequent monitoring of QC on some blood gas systems to increase the effectiveness of systemic and sample-specific error detection.
The most significant benefit of the GEM Premier 5000 was the detection of sample errors reported to operators during patient sample testing. iQM2 detected 1,142 (1.4%) errors in 83,964 patient samples, addressing risks in preanalytical or analytical phases of the testing process (Table 4). For these flagged sample errors, iQM2 automatically performed corrective actions, either error-specific corrections, analyte disablement, or result suppression in 100% of the cases in minutes. Corrective actions performed automatically by the system included clot removal routine, interference flagging, other corrections specific to the error patterns detected, and analyte suppression in the case of an IS error. The analyzer was unavailable for patient testing due to automated corrective actions 0.186%, or 43 min/GEM PAK over the average use-life of the tested GEM PAKs.
Out of the 1.4% of sample errors detected, iQM2 with IS technology was responsible for capturing (0.4%) of sample-specific errors for key blood gas, electrolyte, and metabolite results (Table 4). For every sample, IS collects the sensor response of all electrochemistry-based analytes (pH, pCO 2 , pO 2 , Na þ , K þ , Cl -, Ca þþ , Glu, and Lac) during the sample process, and evaluates the sample response against a pattern library developed for each sensor. If the sensor response does not agree with the expected sensor behavior, this indicates that an anomaly has affected that sensor for that sample and the result is suppressed. IS technology evaluated the performance of > 750,000 sensor responses (84,000 samples across 9 sensors) during this evaluation. On the 361 flagged samples, IS technology detected patterns outside the acceptable behavior on one or more analytes for each flagged sample. Previous studies demonstrated that IS  (14)(15). Two IS examples of errors exceeding TEa that were observed in this study are reviewed in Figure 2. iQM2 IS technology suppressed a glucose result from NYU Langone (reading 15 mg/dL instead of 100 mg/dL when the TEa is 10 mg/dL) and a Clresult from Vanderbilt University Hospital (reading 95 mmol/l instead of 110 mmol/L, when TEa is 5.5 mmol/L). In both cases, the sensor output behavior of the detected samples deviated significantly from samples analyzed on the same system with similar concentrations. The cause of the observed abnormal behavior cannot be confirmed, however, analysis of the sensor response suggest that it could have been caused by a micro bubble (<1 mm of diameter) over the sensor. Bubbles of this size are undetectable by the user or aqueousbased QC material and can have a major impact on the analytical performance of the sensor. iQM2 IS technology demonstrated the ability to detect potential preanalytical errors.

Study Limitations
The authors recognize that there are additional commercially available blood gas analyzers with comparable QC methodologies. Those systems were not included in the study due to management or availability limitations. This study was not intended to compare the various types of errors that could be detected by the GEM vs other blood gas analyzers on the market in the same sample. A study directly comparing error detection capability of marketed blood gas analyzers would be a different study to consider in the future.

CONCLUSION
This multi-site clinical evaluation of the GEM Premier 5000 demonstrated real-time error detection of both systemic and transient errors at the POC. The precision (SD or %CV) and method sigma of PCSs show similar performance to, or better than, intermittent QC, either manual or automated (Tables 1-3). Additionally, the high Ped and continuous monitoring checks performed by iQM2 demonstrated a dramatically lower ADT (minutes) of errors compared to intermittent QC performed on a 6-or 8-h schedule (ADT of hours to days). Around 1.4% of errors detected within minutes by iQM2 for preanalytical, analytical, or transient reasons, (before, during, or after sample run) may have gone undetected with intermittent QC. The new IS technology in iQM2 is able to detect and suppress the results of at least one sensor on 0.4% of the samples, using the new pattern recognition features for detecting abnormal sample responses from transient events. iQM2 increases error detection vs previous generations of iQM by 0.5%, as reported in the clinical setting by Toffaletti et al. (13).
Actual errors were detected and corrected in routine operation for this study in a number of cases. iQM2 detected specific errors that can occur due to preanalytical sources that cannot otherwise be controlled or corrected by intermittent QC. iQM2 identified an error in 1.4% patient samples that could have gone undetected with intermittent analysis of QC. iQM2 not only detected the error, but also immediately notified the operator and initiated a corrective action to solve the sample-specific error in minutes with only minor impact on system availability (0.186% or 43 min over the average use-life of the GEM PAK). Resolution of errors on other systems typically require user technical intervention with unpredictable downtime. Error detection capabilities performed on every sample during analysis, as demonstrated on the GEM Premier 5000, provides additional vigilance and safeguards in POC operations against reporting erroneous results that can impact clinical treatment.