Impact of technology-assisted versus manual sterile compounding on safety and efficiency in a Canadian community hospital

Abstract Purpose Interventions to improve the safety and efficiency of manual sterile compounding are needed. This study evaluated the impact of a technology-assisted workflow system (TAWS) on sterile compounding safety (checks, traceability, and error detection), and efficiency (task time). Methods Observations were conducted in an oncology pharmacy transitioning from a manual to a TAWS process for sterile compounding. Process maps were generated to compare manual and TAWS checks and traceability. The numbers and types of errors detected were collected, and task times were observed directly or via TAWS data logs. Results Analysis of safety outcomes showed that, depending on preparation type, 3 to 4 product checks occurred in the manual process, compared to 6 to 10 checks with TAWS use. TAWS checks (barcoding and gravimetric verification) produced better traceability (documentation). The rate of incorrect-drug errors decreased with technology-assisted compounding (from 0.4% [5 of 1,350 preparations] with the manual process to 0% [0 of 1,565 preparations] with TAWS use; P < 0.02). The TAWS increased detection of (1) errors in the amount of drug withdrawn from vials (manual vs TAWS, 0.4% [5/1,350] vs 1.2% [18/1565]; P < 0.02), and (2) errors in the amount of drug injected into the final container (manual vs TAWS, 0% [0/1,236] vs 0.9% [11/1,272]; P < 0.002). With regard to efficiency outcomes, TAWS use increased the mean mixing time (manual vs TAWS, 275 seconds vs 355 seconds; P < 0.001), had no significant impact on average visual checking time (manual vs TAWS, 21.4 seconds vs 21.6 seconds), and decreased average physical checking time (manual vs TAWS, 58.6 seconds vs 50.9 seconds; P < 0.001). Conclusion In comparison to manual sterile compounding, use of the TAWS improved safety through more frequent and rigorous checks, improved traceability (via superior documentation), and enhanced error detection. Results related to efficiency were mixed.

M anual sterile compounding of chemotherapy is prone to errors, which have caused patient harm and death. [1][2][3][4] This is because manual (ie, nonautomated) processes are dependent on human vigilance; technicians may perform compounding steps from memory rather than following step-by-step instructions, which can lead to variability in compounding practices (eg, preparation of more than 1 admixture at a time 4 ), and forgetting or mixing up steps. For example, analysis of manual workflows has shown 11 latent errors, 3 of which are catastrophic and depend solely on human vigilance to prevent. 5 Furthermore, manual compounding relies on human double checks, which are vulnerable to interruptions, miscommunication, and delays (eg, waiting for a team member to become available). [5][6][7] Human fallibility also remains a risk factor that can lead to errors (eg, misreading drug labels or syringe measurements or unnecessary costs (eg, failing to use a used partial vial before it expires 8 ). For example, research shows humans have misread syringes (by at least 1 syringe graduation) 4% of the time, 9 and adding a double check does not reliably detect accuracy. 10 Manual compounding has been associated with dose deviation errors of greater than 10% in up to 23% of preparations 11,12 ; overdosing may harm patients 13,14 and underdosing may impair the effectiveness of treatment. 15 Manual compounding also requires additional effort from staff to sign off on or document compounding steps, which can slow compounding throughput, result in incorrect or illegible documentation, and be difficult to search (eg, paper records cannot be easily searched). Interventions are urgently needed to improve chemotherapy compounding safety and efficiency.
Despite the claimed benefits of TAWS use, there is a lack of rigorous safety and efficiency data. Studies comparing manual versus TAWS processes have had methodological challenges in collection of both efficiency data (eg, use of unsynchronized punch clocks, which impacted measurement of task time) 29 and safety data (eg, comparing manual vs TAWS error rates for different medications and reliance on self-reporting, 30 which may lead to underestimation of errors 31 ). Further, multicenter evaluations have reported varying impacts across institutions but have not fully characterized the workflow processes to understand the differences. 32,33 Therefore, objectively comparing the safety and efficiency of TAWS versus manual compounding is needed to comprehensively understand their respective benefits and drawbacks, thereby informing evidence-based interventions to prevent compounding errors and patient harm from continuing.
To this end, we compared manual and TAWS sterile compounding in a Canadian community oncology ambulatory care pharmacy on (1) safety, as measured by the frequency and type of checks, traceability (documentation of preparation), and error frequency, and (2) efficiency, as measured by compounding task time.

Methods
setting. This study was conducted at the an oncology pharmacy in Toronto, Canada. The pharmacy operated on weekdays, from 8 am to 4:30 pm, providing treatments to approximately 30 patients per day and compounding approximately 55 preparations per day. The daily operational team consisted of 2 or 3 pharmacists and 2.5 pharmacy technician full-time equivalents (all dedicated to the oncology pharmacy and compounded sterile preparations). study design. This was an observational study in which we collected data on a common set of drugs before and after they were transitioned from a manual to a TAWS compounding process. Institutional ethics board approval was acquired (REB #20-0026) and pharmacy staff provided consent.
intervention: description of taWs. The TAWS evaluated was the Pyxis IV Prep (formerly CATO) system, version 2.46.08 (BD, San Diego, CA). The TAWS provides barcode verification of compounding materials, stepby-step compounding instructions, and a combination of a precise weight scale along with a database of densities for different drugs and fluids to calculate the precise dose or volume being manipulated (ie, gravimetric verification). To support documentation of the components and amounts of drug and diluent used, photographs are automatically taken throughout the TAWS workflow. Table 1 contrasts features employed by the TAWS against the manual workflow.
Data collection procedure. The observers recorded task time and errors detected by pharmacy staff or TAWS (details are provided below); observers did not identify errors themselves. Three human factors specialists familiarized themselves with the compounding workflow via in-person observations and orientation by pharmacy staff during the month prior to data collection. Observers also collected pilot data to develop the data collection form in Microsoft Excel for Mac (Microsoft Corporation, Redmond, WA)

KeY PoiNts
• The implementation of a technology-assisted workflow system (TAWS) in a sterile compounding outpatient pharmacy increased the frequency and rigor of safety checks for each compounded preparation.
• The TAWS safety features detected and resolved more errors than the previously used manual compounding process and outperformed the traceability of manual processes by recording photographs and generating time-stamped digital audit logs.
• The use of the TAWS increased preparation time for pharmacy technicians in the cleanroom but decreased the review time for pharmacists. The real-time video review is triggered by the cleanroom technician verbally calling out "Check please!", stating the patient's name, and then showing the drug label to the video camera, which is being viewed remotely by a pharmacist. The pharmacist then reads the label and identifies the corresponding patient's order in their own order records. This approach minimizes bias by leaving it to the pharmacist to manually reconcile the drug name, dosage, container, and quantities of drug or reconstitution diluent present in the syringe being presented to the video camera.
Continued from previous page Table 1. Description of Error Types and Impact and generated a process map, which included all checks and documentation. During formal data collection, 2 observers were present on each observation day and conducted a daily debriefing and merging of observations data sets in the late afternoon. Timeline. Formal data collection occurred over 4 months (August 17-December 11, 2020) in 3 phases: preimplementation (August 17-September 14), acclimation (September 15-October 30), and postimplementation (November 1-December 11). Data on manual preparations were collected in all 3 phases because medications were transitioned to TAWS workflow gradually. In the preimplementation phase, data collection occurred only for manual preparations. On September 15, the pharmacy team transitioned 11 medications to TAWS workflow. During the acclimation phase, no task time data was collected since staff were still adjusting, potentially impacting their efficiency. However, TAWS safety features (eg, barcode scanning, gravimetric verification) were immediately active, so error data from the acclimation phase was included. Starting on November 1 (the start of the postimplementation phase), both safety and task time data from TAWS preparations were included for analysis. See eAppendix Table A1 for additional detail.
Sample size. Pharmacy records from the prior year showed roughly 600 preparations were performed per month for the 11 medications being transitioned to TAWS compounding. As in a prior study by other researchers, 23 we planned to collect data on 600 manual and 600 TAWS preparations. However, in midstudy the pharmacy leadership changed plans and transitioned all medications to TAWS compounding. Therefore, we included data on all compounded medications, and the sample size exceeded 1,000 preparations in both the manual workflow and TAWS data. (Note: the pharmacy transitioned medications to TAWS compounding in a rolling fashion. Therefore, observers collected data on manual preparations in parallel with gathering of data on TAWS preparations until all medications had been transitioned to TAWS compounding [November 25, 2020]. Regardless of when medications were transitioned to TAWS preparation, task time was recorded as of November 1, 2020.) Inclusion and exclusion criteria. We included all medications for which a patient label was printed. However, we excluded oral medications, batchcompounded medications (ie, doses not specific to a patient), and medications prepared less than 7 times in either workflow. Additional exclusions specific to each analysis are described below.
safety metrics. The evaluated safety metrics consisted of checks and traceability, errors, and dose accuracy (for the TAWS preparations only, since there is no comparable method of accurately assessing dose deviation in the manual workflow).
Checks and traceability. Process maps of manual and TAWS workflows were generated based on the observations and analyzed to quantify and characterize the types of checks and traceability procedures (ie, documentation) produced by both workflows.
Errors. Observers recorded the following errors: (1) "within-date" unused partial vials (UPVs) not used, (2) incorrect container selected, (3) incorrect drug selected, (4) incorrect amount of reconstitution diluent withdrawn, (5) incorrect amount of reconstitution diluent injected, (6) incorrect container adjustment, (7) incorrect amount of dose withdrawn, and (8) incorrect amount of dose injected. Table 1 provides descriptions of each error type. In the manual compounding process, errors were detected by pharmacy staff. Specifically, observers recorded when pharmacy staff verbalized a concern to another team member and requested a change (eg, an incorrect drug was provided to the cleanroom technician, who asked for a corrected vial). The observers did not attempt to detect any errors besides those detected and verbalized by the pharmacy team. Similarly, errors in TAWS process were detected by verbalizations by pharmacy staff. However, errors were also detected by reviewing the data logs for barcode scanning failures or for gravimetric measurements that were flagged as "out of tolerance." Outof-tolerance events in TAWS logs were manually reviewed to exclude workflow artifacts (eg, a technician forgot to remove an item from the scale but quickly recognized and resolved the problem). Out-of-tolerance events for dose withdrawals were only counted if they occurred during the final withdrawal required for the ordered dose (the TAWS accounts for earlier errors by adjusting the amount requested in the final withdrawal so errors in early withdrawals do not impact the patient).
Dose accuracy. Dose accuracy cannot be determined in the manual workflow because it depends on a human visually inspecting the amount of drug in a syringe; this visual inspection can only provide a pass-or-fail assessment (ie, it cannot assess how far the measurement is from the ordered dose). In contrast, preparations in TAWS workflow benefit from gravimetric verification checks that can detect minute differences from the ordered dose. Therefore, we only analyzed dose accuracy for TAWS preparations by reviewing the data logs to assess the distribution of dose measure deviations. We calculated "percent dose deviation" by subtracting the ordered dose from the actual dose in the container and dividing that value by the ordered dose. Ordered doses with volumes less than 2 mL (assuming a drug density of 1 g/mL) were excluded from the analysis because the institution configured the TAWS to extend dose tolerances in these cases. efficiency metrics. Observers recorded task times in Excel using a preformatted worksheet with multiple columns to represent each task substep; observers used keyboard shortcuts to note the time for each substep (eg, the moment a preparation was placed into or taken out of the biologic safety cabinet [BSC]). Duration data collection was found to be reliable between observers when interrater reliability was calculated from pilot observations (intraclass correlation coefficient [ICC] of >0.75 34 ). The sample sizes reported for the 3 task times vary because we excluded preparations with incomplete data (eg, because the view of the cleanroom was momentarily blocked).
We calculated 3 task times: Data analysis. All statistical tests were conducted on IBM SPSS Statistics for Macintosh, Version 27.0 (IBM Corporation, Armonk, NY). Differences in error frequency between the manual and TAWS processes were analyzed using a chi-square test, and differences in task time were evaluated using a t test. Outliers in task duration were excluded. Descriptive statistics of dose accuracy in TAWS gravimetric preparations were calculated in Microsoft Excel.
safety metrics. Checks and traceability. Table 2 presents comparative data on the differences in verification checks and traceability for the aM J heaLth-sYst PharM | VOLUME 79 | NUMBER 19 | OCTOBER 1, 2022 (2) photographic record of final container and drug vial(s) used

Continued on next page
manual and TAWS workflows across 7 compounding steps. The exact numbers of checks varied by preparation, but the manual workflow included 3 or 4 verification checks, whereas the TAWS workflow included 6 to 10 checks. More checks were conducted for preparations requiring reconstitution (eg, checking that the correct amount of diluent was used) or a container (eg, checking that the correct type and size of intravenous bag or elastomeric infuser was used). For select medications, the TAWS in our institution was not configured to require gravimetric measurements (the specific density of the drug is not known, so gravimetric calculations cannot be performed). These volumetric preparations therefore require 2 to 4 fewer checks than gravimetric preparations. There are important differences in the traceability produced by the 2 workflows. In the manual workflow, pharmacy staff are required to manually sign off on and record information on paper forms to indicate their involvement in preparation, and record key details (eg, the lot numbers of the drug vials used). In the TAWS workflow, documentation of each verification check (eg, barcode scan result, gravimetric measurement, photographs of preparation, staff involved) is automatically and electronically recorded in data logs.
A visual depiction of the process changes encountered when converting a manual workflow to a TAWS workflow is captured in the eAppendix ( Figure A1).
Errors: UPV not used. There were 2 incidents where a within-date UPV was not used when available (one in the manual compounding process and one in the TAWS process). In the manual preparation process, a vial of albumin-bound paclitaxel was thrown out in error, but the error was detected by a cleanroom technician, who remembered that they had used the vial earlier in the day. During TAWS preparation a paclitaxel vial was thrown out in error, but the availability of the UPV was flagged by the TAWS and resolved. As a result, there were no significant Electronic record of approval of photographs; paper form is initialled by pharmacist after final product is inspected; remaining dose and barcode of used partial vial also recorded to aid in future preparations that may use the used partial vial

Summary
The manual workflow has a range of 3-4 checks, depending on the workflow. Documentation is paper based, with cleanroom technicians and pharmacists making extra effort to manually sign off on paper-based forms, which are later stored and require manual human review if preparation details are needed. There is no record of what the pharmacists approve through the video camera. Technicians rely on reference documents to know which diluent to use and the appropriate quantity to use (or memorized these details). The TAWS workflow has a range of 6-10 checks, depending on the workflow. Documentation is electronic, with cleanroom technicians' and pharmacists' actions automatically tracked by the electronic system. Photographs of key steps in the preparation, as well as gravimetric measurements, are captured as part of the preparation record, and the digital signatures of the accountable technician and pharmacist are captured automatically. The TAWS workflow standardizes the sequence of compounding steps across technicians given the automated workflow. The step-by-step procedure minimizes the need for memorization (eg, which diluent to use for which medication, when to perform each step), which simplifies training.
Abbreviation: TAWS, technology-assisted workflow system. a Barcode scanning of all components does not occur up front. Components are verified "just in time" prior to use to reduce risk of selection errors. b Container adjustments involve removal of fluid from an intravenous bag prior to injection of the prescribed drug, or the addition of a diluent to an elastomeric infuser prior to injection of prescribed drug.
c Microsoft Access is a registered trademark of Microsoft Corporation, Redmond, WA. Four of the drug selection errors were due to mix-ups of trastuzumab products, specifically Herceptin (Genentech) and Ogivri (Mylan Institutional Inc.). Table  3 summarizes these data, and further details on each selection error are described in the eAppendix (Table A2).

Continued from previous page
Errors: measurement.  14.1%, and dose injection errors from -14.7% to -5.2%. These data show that the TAWS detected and prevented several concentration or dosing errors. Of note, the sample sizes for each error type vary because some errors only occurred for certain preparation types (eg, preparations for which reconstitution is not always required and those administered as syringes could not encounter container adjustment errors). Further details on each detected error are provided in the eAppendix (Tables  A3-A6).
Dose accuracy of TAWS preparations. The average dose deviation of the final compounded product was -0.27% (mean, -0.21%; SD, 1.2%). All preparations made with the TAWS had a dose deviation less than ±5%, and 1,484 out of 2,211 preparations (67%) were within 1% of the ordered dose. Notably, the sample size for the dose accuracy analysis (n = 2,211) was different from that for other TAWS analyses (n = 1,565) because we included all recorded gravimetric preparations even if observers were not present at the time.
efficiency metric. Use of the TAWS increased average mixing time (manual vs TAWS, 275 seconds vs 355 seconds; P < 0.001), had no significant impact on visual checking time (manual vs TAWS, 21.4 seconds vs 21.6 seconds), and decreased physical checking time (manual vs TAWS, 58.6 seconds vs 50.9 seconds; P < 0.001). Table 5 provides a summary of these findings.

Discussion
Our findings demonstrated that the TAWS workflow, compared to a manual workflow, offered improved safety and mixed effects on efficiency (ie, reduced time to physical inspection of the final product but increased mixing time in the cleanroom).
Our process mapping revealed that the TAWS workflow involved a higher frequency of technology-supported checks that were more rigorous than human checks in the manual workflow. The increased checks in the TAWS workflow led to significantly increased detection of measurement errors but lower detection of incorrect drug errors. This second finding is misleading due to 2 factors. First, in the manual workflow, incorrect drug errors were detected by the cleanroom technician and verbalized to the staging technician and thus easily noticed and recorded by the observers. In the TAWS workflow, the staging technician was alerted to the error by a barcode scanning failure, and there was no verbalization of the error, so it was missed by observers. Second, the TAWS provides Table 3.  Table 2, the TAWS workflow includes 2 barcode scans. Selection errors detected and resolved via the first barcode scan are not recorded; only errors in the second scan are recorded. As a result, the true detection rate is likely underestimated.  22,35 Notably, the configuration of TAWS in our institution was not able to detect errors in drug brand (eg, different brands of bortezomib), so further refinements to the TAWS may increase the safety benefits further. Additionally, the use of gravimetric checks in the TAWS workflow detected significantly more measurement errors than the manual compounding process. This is because manual verification of syringe amounts is vulnerable to human fallibility, 9 unlike gravimetric measurement supported by a TAWS. 10 The gravimetric checks ensured a low average dose deviation (-0.27%, n = 2,214) for TAWS preparations in our study, which is in line with previous studies reporting average dose deviations of -0.62% (n = 11,874) 22 and -0.8% (n = 3,156). 23 Unfortunately, we were unable to independently assess dose accuracy in the manual workflow (this would have been possible only if we had duplicated key steps in the manual workflow with gravimetric measurements). Therefore, we cannot conclusively state that dose accuracy was improved with TAWS use versus the manual workflow, but TAWS implementation considerably improved the ability to detect measurement errors.

Comparison of Selection Errors in Manual and TAWS Workflows
Measurement errors were detected in 2.1% of our TAWS preparations, which was lower than the 7% to 8% detection rate reported in other studies evaluating TAWS compounding. 22,24,27 Variation in error rates, as well as changes over time, has been aM J heaLth-sYst PharM | VOLUME 79 | NUMBER 19 | OCTOBER 1,2022 documented in multicenter studies evaluating TAWS use. 24,27 Future studies exploring the impact of contextual factors (eg, staffing ratios, workflow configuration) on error rates may help inform efforts to optimize safety.
While the safety benefits of TAWS use are unequivocal, its impact on efficiency is more nuanced. In our study the TAWS process increased mixing time in the cleanroom but reduced the task time for physical checking activities; this finding had precedent in a prior study. 23 The increased mixing time with a TAWS workflow versus a manual workflow is due to the increased checks ( Table 2). These additional checks generate a richer audit trail automatically, whereas the manual workflow requires technicians to sign off on the preparation on a separate paper form and also write the drug vial lot number used. A further benefit of a TAWS workflow was its replacement of the video checks that were required in the manual workflow. Video reviews required cleanroom technicians to interrupt pharmacists to ask for them to visually inspect and approve the amounts measured in a syringe through the camera feed. This simultaneously delayed cleanroom technicians and raised safety issues for pharmacists, who had to interrupt their review of drug orders for other patients. Therefore, the increased mixing time required by the TAWS offset the streamlining of the review process, as pharmacists could review the photographs at a time of their choosing and also retain the photographs as part of a preparation's documentation.
The automatic electronic documentation that a TAWS produces may also generate other efficiencies. For example, TAWS tracking of drug inventory (including UPVs) may eventually replace inventory and restocking tracking activities currently handled by the staging technician in between preparations; this was outside the scope of our study. We also did not compare the time required to train staff on the manual process versus the TAWS process. Given that a TAWS process provides considerable decision support and reduces the need for memorization, it may reduce training time and decrease barriers to increasing the number of staff available to compound. Additional research on the cost implications of these time savings would be helpful to provide a broader analysis of the impact of a TAWS on pharmacy efficiency.
We also found that pharmacists required less time to physically inspect the final compounded product in the TAWS process versus the manual process despite the processes being identical (although TAWS preparations had an additional label to review). One possible reason for the time reduction is that photographic review in the TAWS process occurs at a time of the pharmacist's choice, unlike in a manual workflow wherein pharmacists are interrupted to perform a video review. As a result, pharmacists may be able to better coordinate the timing of the photographic review with the physical check, thereby increasing the efficiency of the latter. In our study the switch to a photographic review may also have improved the safety and efficiency of pharmacists' review of new clinical orders, as they were no longer interrupted to verify compounding accuracy via video checks; this aspect of the pharmacists' workflow was outside the scope of our study and not measured, but it may be a useful metric to record in future studies.
Limitations. There were several limitations to our study. First, there are benefits of TAWS use that were not explicitly captured in our findings. For example, during manual compounding, pharmacists were regularly interrupted to perform a video double-check of key steps, which may have impacted the efficiency and accuracy of those interrupted tasks (eg, drug order verification and transcription tasks) but which we did not collect data on. Second, our sample size may have been insufficient to evaluate infrequent errors (eg, incorrect-container errors). Third, the human visual verification of measured doses in the manual workflow was not accurate enough to assess dose deviation errors. As a result, we were unable to compare dose accuracy between the manual and TAWS workflows. Fourth, observers did not independently check preparations for errors that the pharmacy team missed; it is possible more errors occurred in the manual workflow than we have reported here. Fifth, we used a short timeframe for the pharmacy staff to acclimate to the new system; the task times we reported may have been different if we had delayed TAWS data collection. Sixth, we did not compare the difference in total preparation time between the manual and TAWS processes because the 2 processes ran in parallel during the study. For example, multiple manual and TAWS preparations were staged together or were deliberately shuffled depending on which medication a patient needed first; this could have artificially altered the overall processing time for some preparations for reasons that were unrelated to the workflow used. As a result, our efficiency analysis compared task times for compounding steps that were directly comparable (eg, steps performed by a pharmacy staff member working on a single preparation at a time).

conclusion
Our study found that a TAWS improves sterile compounding safety. Use of a TAWS increased the detection of measurement errors and resulted in fewer drug selection errors (eg, drug mix-ups). Further, the TAWS automatically documented all verification checks (including photographs) to provide a complete record of the preparation, whereas the manual workflow required manual, paper documentation. While these added safety benefits come with an increased mixing time in the cleanroom, many hospitals may find the trade-off worthwhile.

acknowledgments
The study team thanks Raymond Ha, BScPhm, RPh, for his assistance with interpreting the measurement errors.

Disclosures
Funding for the project was provided by Becton Dickinson Canada Inc. Mr. Fan and Dr. Trbovich have previously conducted research funded by BD Canada Inc. and received speaking honoraria for presenting at events sponsored by BD Canada Inc. Ms. Rolko served (uncompensated) on a BD advisory committee in 2018 and 2019. The other authors have declared no potential conflicts of interest.