Abstract
In medicine, the use of standard operating procedures (SOPs) is often evaluated using questionnaires (QUES). However, QUES can have limitations with regard to method, thus leading to errors. Simulation (SIM) offers another opportunity for evaluation. We hypothesized that medical errors in the evaluation of SOPs using QUES could be detected by SIM, and that SIM is better qualified to demonstrate applied medicine.
We investigated the use of SOPs in anaesthesia, rapid sequence induction (RSI), by means of a QUES (n=42) or SIM (n=42) among 84 anaesthesiologists. Seven measures for preventing aspiration during induction of anaesthesia were examined and evaluated according to a predetermined points system.
The average number of times that precautionary measures to prevent aspiration were mentioned in the QUES [4.8 (0.9)] or performed during SIM [5.0 (1.1)] did not differ between the two groups. Pre-oxygenation was the most frequently described or performed measure (95% vs 93%). However, other measures, such as avoidance of positive pressure ventilation (45% vs 85%), differed significantly between the two groups.
QUES and SIM are powerful instruments for evaluating the implementation of SOPs such as RSI. SIM demonstrates automated behaviours and thus more clearly represents behaviours used in clinical practice than is possible to demonstrate using QUES. Using a combination of these two instruments, method errors resulting from the individual instruments can be reduced.
Standard operating procedures (SOPs) are components of clinical pathways that were introduced to improve diagnostic and therapeutic management within medicine. They are based on current studies and recommendations from experts and professional organizations. The introduction of these procedures into everyday use in hospitals has led to a reduction in morbidity, mortality, and hospital costs.1
In general, SOPs are often evaluated using instruments such as questionnaires (QUES) or observer studies.2,3 However, the reliability of an investigation using QUES alone depends on the response rate and the representativeness of the survey.4 Additionally, method errors can also occur, due to missing data, polarity reversal (1 instead of 6), contradictory, or inconsistent answers or due to answers that tend towards social desirability, all of which can lead to a bias in the results.5
Simulation (SIM) enables successful training of technical and non-technical skills.6 SIM also provides the opportunity to observe and evaluate technical performance.7–9 Thus, SIM represents an excellent means of evaluating the use of SOPs, with a special focus on applied practice. The objective of this study was to examine the results of the evaluation of an SOP with the generally recognized instrument, QUES, and to compare those with the results of a possible new tool, SIM. Currently, we are not aware of any other simulator-supported studies comparing the performance of QUES and SIM. We chose rapid sequence induction (RSI) procedure, which is indicated by any risk of aspiration of the contents of the stomach during induction of anaesthesia. We hypothesized that method errors in the evaluation using QUES could be detected using SIM, SIM is better qualified to demonstrate applied medicine, and that SIM is better suited than QUES to the implementation of medical standards.
Methods
The study was approved by the medical faculty of the University of Heidelberg, Germany, and written informed consent was obtained from all of the participants. The investigation performed by QUES was blinded and the answers could not be traced back to the participants.
Our SIM centre is located in our Department of Anaesthesiology. It comprises an operating room that is separated from the control room by a two-way mirror. The operating theatre includes a standard anaesthetic apparatus (Sulla 808V, Draeger®, Luebeck, Germany), a physiological patient monitor (GE®, Eagle 5000, Sollingen, Germany), an anaesthesia cart, an operating table (Maquet®, Rastatt, Germany), a surgical instrument table, and a patient high-fidelity simulator mannequin (HPS, METI®, Sarasota, FL, USA) located on the operating table. This simulated world presents an environment for administering anaesthesia that is quite close to reality.10
An anaesthesia nurse from our department supported the participants from the SIM group during anaesthesia. The nurse was familiar with all of the medical devices. The participants were told ahead of time that the nurse could be trusted and would only provide correct information.
Eighty-four anaesthesiologists from external hospitals and from within our department participated in the study. They were randomized into two groups of equal size with 42 participants each. One group received a QUES; the other group participated in a session of SIM. Participants described anaesthesia or administered it to a 20-yr-old, 72 kg man in good general health who had not undergone any previous operations. For more than 3 h, the patient had been suffering with typical symptoms of acute appendicitis, with nausea and vomiting, and was at an increased risk of aspiration. Participants were told that a stomach tube had been inserted in the emergency room. The stomach had been suctioned, and the tube had been removed by the nurse shortly before anaesthesia induction. All participants had free choice of medical equipment and medications for anaesthesia.
The participants in the SIM group received a brochure about the options and limitations of SIM and the structure of the SIM centre. The SIM session was video-recorded using two cameras. Recording and camera set-up was performed by a member of the study group who had been briefed beforehand.
Participants in the QUES group received an open QUES based on a pilot study done by our department. This QUES contained only basic information as described above, and participants were asked to describe in detail their way of induction of anaesthesia with regard to precautions, choice of drugs, and the order of administration. The QUES were collected and analysed retrospectively.
Criteria for inclusion in the study were participation on a voluntary basis, professional experience of >0.5 yr and no previous experience with simulators. Only QUES that were filled out completely and video recordings that were technically acceptable were included in the analysis process.
To obtain the highest possible level of reliability, two independent anaesthesiologists who had not been involved in collecting the data evaluated the QUES and video recordings.
The participants indicated their degree of anaesthesiological training (anaesthesiologist with board certification vs anaesthesiologist without board certification), their gender, and their clinical experience. We used the video recordings and the QUES to evaluate the precautionary measures used to prevent aspiration during the RSI.2,3 For evaluation, we used investigations that had already been carried out regarding RSI.2,3 Seven precautions to prevent aspiration were examined.
Pre-oxygenation: Detailed description in the QUES group, or performance in the SIM group, according to the current recommendations11 (e.g. sealed face mask for >3 min, an Feo2 of 0.9 was achieved).
Change of the horizontal position of the patient: Detailed description in the QUES group, or performance in the SIM group according to the current recommendations12 (e.g. head down or semi-sitting position).
Performance of a RSI that was truly ‘rapid’: a truly ‘rapid’ RSI was considered to be done, when terms such as ‘RSI’, ‘rapid’, ‘fast’, or other similar expressions were described in the QUES, or mentioned in the SIM.
Prepared suction.
Arranged endotracheal tube with stylet.
Applying cricoid pressure.
Avoidance of positive pressure ventilation (APPV) before securing the airway with an endotracheal tube.
The last four precautions were either detailed in the QUES group or judged on performance and orders to the nurse in the SIM group.
For each of the respective preventative measures implemented, one point was given (Table 1).
Investigated precautions for preventing aspiration during a RSI
| Precautionary measure | Weight |
|---|---|
| Pre-oxygenation | 1 |
| Semi-sitting position | 1 |
| Performance of a fast RSI | 1 |
| Prepared suction | 1 |
| Arranged endotracheal tube with stylet | 1 |
| Application of cricoid pressure | 1 |
| APPV | 1 |
| Maximum number of possible points | 7 |
| Precautionary measure | Weight |
|---|---|
| Pre-oxygenation | 1 |
| Semi-sitting position | 1 |
| Performance of a fast RSI | 1 |
| Prepared suction | 1 |
| Arranged endotracheal tube with stylet | 1 |
| Application of cricoid pressure | 1 |
| APPV | 1 |
| Maximum number of possible points | 7 |
Investigated precautions for preventing aspiration during a RSI
| Precautionary measure | Weight |
|---|---|
| Pre-oxygenation | 1 |
| Semi-sitting position | 1 |
| Performance of a fast RSI | 1 |
| Prepared suction | 1 |
| Arranged endotracheal tube with stylet | 1 |
| Application of cricoid pressure | 1 |
| APPV | 1 |
| Maximum number of possible points | 7 |
| Precautionary measure | Weight |
|---|---|
| Pre-oxygenation | 1 |
| Semi-sitting position | 1 |
| Performance of a fast RSI | 1 |
| Prepared suction | 1 |
| Arranged endotracheal tube with stylet | 1 |
| Application of cricoid pressure | 1 |
| APPV | 1 |
| Maximum number of possible points | 7 |
Additionally, we also examined the way the anaesthesia was performed with regard to choice of drugs (dose and substance). Classification was performed as follows: propofol 1.5–2.5 mg kg−1, thiopental 3–5 mg kg−1, fentanyl 1–5 µg kg−1, sufentanil 0.5–2 µg kg−1, rapifen 10–40 µg kg−1, rocuronium 0.6–0.9 mg kg−1, and succinylcholin chloride 1–1.5 mg kg−1.
Both the patients’ characteristics obtained and the performances of RSI were analysed statistically. We used a paired t-test for independent pairs to compare the number of points received when taking precautions towards aspiration in the two groups. Each precaution was compared between the two groups using the χ2 test. P-values of <0.05 were considered significant, and those <0.01 were considered highly significant. All data are presented as mean (sd).
Results
All video cassettes from the SIM group (n=42) were of sufficient audiovisual quality for evaluation. The return rate for the QUES group was 95.2%, which means that 40 of the 42 QUES could be evaluated.
The two groups did not differ with regard to gender, clinical experience, and level of education in anaesthesia (Table 2). Twenty women and 22 men participated in the SIM study. In the QUES group, 22 women and 18 men answered the QUES. The average level of clinical experience in the QUES group was somewhat longer (5.8 yr) than that in the SIM group (4.0 yr), but this was not significant (P=0.09). In the SIM group, nine participants were anaesthesiologists with board certification and 33 were anaesthesiologists in training. There were 15 anaesthesiologists with board certification and 25 anaesthesiologists in training in the QUES group.
Patients’ characteristics of participants and mean score of aspiration prophylaxis. Data are presented as mean (sd)
| SIM group | QUES group | |
|---|---|---|
| Gender: male/female | 20/22 | 22/18 |
| Clinical experience (yr) | 4.0 (3.9) | 5.8 (5.1) |
| Number of board certified anaesthesiologists | 9 | 15 |
| Aspiration prophylaxis score | 5.0 (1.1) | 4.8 (0.9) |
| Number of participants from university | 14 | 9 |
| Number of participants from other hospital | 28 | 31 |
| SIM group | QUES group | |
|---|---|---|
| Gender: male/female | 20/22 | 22/18 |
| Clinical experience (yr) | 4.0 (3.9) | 5.8 (5.1) |
| Number of board certified anaesthesiologists | 9 | 15 |
| Aspiration prophylaxis score | 5.0 (1.1) | 4.8 (0.9) |
| Number of participants from university | 14 | 9 |
| Number of participants from other hospital | 28 | 31 |
Patients’ characteristics of participants and mean score of aspiration prophylaxis. Data are presented as mean (sd)
| SIM group | QUES group | |
|---|---|---|
| Gender: male/female | 20/22 | 22/18 |
| Clinical experience (yr) | 4.0 (3.9) | 5.8 (5.1) |
| Number of board certified anaesthesiologists | 9 | 15 |
| Aspiration prophylaxis score | 5.0 (1.1) | 4.8 (0.9) |
| Number of participants from university | 14 | 9 |
| Number of participants from other hospital | 28 | 31 |
| SIM group | QUES group | |
|---|---|---|
| Gender: male/female | 20/22 | 22/18 |
| Clinical experience (yr) | 4.0 (3.9) | 5.8 (5.1) |
| Number of board certified anaesthesiologists | 9 | 15 |
| Aspiration prophylaxis score | 5.0 (1.1) | 4.8 (0.9) |
| Number of participants from university | 14 | 9 |
| Number of participants from other hospital | 28 | 31 |
Clinical experience (Fig. 1), gender, and level of education (data not shown) in the two groups did not correlate with the number of points achieved for precautions against aspiration.
Clinical experience had no effect on the mean number of points for precautions against aspiration during the RSI. SIM, simulation group; QUES, questionnaire group.
The average number of points was 5.0 (1.1) in the SIM group. This did not significantly differ from those in the QUES group [4.8 (0.9)] (Table 2). The frequency of generally accepted precautions against aspiration (Fig. 2) such as pre-oxygenation (SIM: 95.2% and QUES: 95.0%) or the application of cricoid pressure (83.3% and 92.5%, respectively) did not differ between the SIM and the QUES groups. There were significant differences, however, among the semi-sitting position, performance of a ‘rapid’ RSI, prepared suction, use of an endotracheal tube with stylet and APPV. However, there was a significant difference (P<0.01) between the two groups for the APPV (45% vs 85%, respectively) and performance of a ‘rapid’ RSI (23% vs 88%, respectively).
Different frequency of aspiration prophylaxis in the QUES (questionnaire) and SIM (simulation) groups. All data presented in per cent. *SIM vs QUES (P<0.05); #SIM vs QUES (P<0.01). APPV, avoidance of positive pressure ventilation.
The medication used during the RSI is presented in Table 3. Only 17 of the 40 participants (42.5%) in the QUES group answered the question satisfactorily.
Percentage representation of dose and frequency of administered induction agents during the RSI in the SIM and the QUES group. Data are numbers (per cent). *Two anaesthesiologists chose alfentanil in the SIM group. #One chose atracurium in the SIM group
| Dose | SIM | QUES | |
|---|---|---|---|
| Propofol | <1.5–2.5 mg kg−1 | 0 | 0 |
| =1.5–2.5 mg kg−1 | 8 (19.0%) | 0 | |
| >1.5–2.5 mg kg−1 | 16 (38.3%) | 6 (35.3%) | |
| Thiopental | <3–5 mg kg−1 | 0 | 0 |
| =3–5 mg kg−1 | 4 (9.5%) | 3 (17.6%) | |
| >3–5 mg kg−1 | 14 (33.3%) | 8 (47.1%) | |
| Sufentanil* | <0.5–2 µg kg−1 | 6 (14.3%) | 1 (5.8%) |
| =0.5–2 µg kg−1 | 0 | 0 | |
| >0.5–2 µg kg−1 | 0 | 0 | |
| Fentanyl* | <1–5 µg kg−1 | 0 | 8 (47.1%) |
| =1–5 µg kg−1 | 34 (80.9%) | 8 (47.1%) | |
| >1–5 µg kg−1 | 0 | 0 | |
| Succinylcholin# | <1–1.5 mg kg−1 | 0 | 1 (5.9%) |
| =1–1.5 mg kg−1 | 32 (76.2%) | 8 (47.1%) | |
| >1–1.5 mg kg−1 | 2 (4.8%) | 2 (11.8%) | |
| Rocuronium# | <0.6–9 mg kg−1 | 0 | 0 |
| =0.6–9 mg kg−1 | 6 (14.3%) | 2 (11.8%) | |
| >0.6–9 mg kg−1 | 1 (2.4%) | 4 (23.5%) |
| Dose | SIM | QUES | |
|---|---|---|---|
| Propofol | <1.5–2.5 mg kg−1 | 0 | 0 |
| =1.5–2.5 mg kg−1 | 8 (19.0%) | 0 | |
| >1.5–2.5 mg kg−1 | 16 (38.3%) | 6 (35.3%) | |
| Thiopental | <3–5 mg kg−1 | 0 | 0 |
| =3–5 mg kg−1 | 4 (9.5%) | 3 (17.6%) | |
| >3–5 mg kg−1 | 14 (33.3%) | 8 (47.1%) | |
| Sufentanil* | <0.5–2 µg kg−1 | 6 (14.3%) | 1 (5.8%) |
| =0.5–2 µg kg−1 | 0 | 0 | |
| >0.5–2 µg kg−1 | 0 | 0 | |
| Fentanyl* | <1–5 µg kg−1 | 0 | 8 (47.1%) |
| =1–5 µg kg−1 | 34 (80.9%) | 8 (47.1%) | |
| >1–5 µg kg−1 | 0 | 0 | |
| Succinylcholin# | <1–1.5 mg kg−1 | 0 | 1 (5.9%) |
| =1–1.5 mg kg−1 | 32 (76.2%) | 8 (47.1%) | |
| >1–1.5 mg kg−1 | 2 (4.8%) | 2 (11.8%) | |
| Rocuronium# | <0.6–9 mg kg−1 | 0 | 0 |
| =0.6–9 mg kg−1 | 6 (14.3%) | 2 (11.8%) | |
| >0.6–9 mg kg−1 | 1 (2.4%) | 4 (23.5%) |
Percentage representation of dose and frequency of administered induction agents during the RSI in the SIM and the QUES group. Data are numbers (per cent). *Two anaesthesiologists chose alfentanil in the SIM group. #One chose atracurium in the SIM group
| Dose | SIM | QUES | |
|---|---|---|---|
| Propofol | <1.5–2.5 mg kg−1 | 0 | 0 |
| =1.5–2.5 mg kg−1 | 8 (19.0%) | 0 | |
| >1.5–2.5 mg kg−1 | 16 (38.3%) | 6 (35.3%) | |
| Thiopental | <3–5 mg kg−1 | 0 | 0 |
| =3–5 mg kg−1 | 4 (9.5%) | 3 (17.6%) | |
| >3–5 mg kg−1 | 14 (33.3%) | 8 (47.1%) | |
| Sufentanil* | <0.5–2 µg kg−1 | 6 (14.3%) | 1 (5.8%) |
| =0.5–2 µg kg−1 | 0 | 0 | |
| >0.5–2 µg kg−1 | 0 | 0 | |
| Fentanyl* | <1–5 µg kg−1 | 0 | 8 (47.1%) |
| =1–5 µg kg−1 | 34 (80.9%) | 8 (47.1%) | |
| >1–5 µg kg−1 | 0 | 0 | |
| Succinylcholin# | <1–1.5 mg kg−1 | 0 | 1 (5.9%) |
| =1–1.5 mg kg−1 | 32 (76.2%) | 8 (47.1%) | |
| >1–1.5 mg kg−1 | 2 (4.8%) | 2 (11.8%) | |
| Rocuronium# | <0.6–9 mg kg−1 | 0 | 0 |
| =0.6–9 mg kg−1 | 6 (14.3%) | 2 (11.8%) | |
| >0.6–9 mg kg−1 | 1 (2.4%) | 4 (23.5%) |
| Dose | SIM | QUES | |
|---|---|---|---|
| Propofol | <1.5–2.5 mg kg−1 | 0 | 0 |
| =1.5–2.5 mg kg−1 | 8 (19.0%) | 0 | |
| >1.5–2.5 mg kg−1 | 16 (38.3%) | 6 (35.3%) | |
| Thiopental | <3–5 mg kg−1 | 0 | 0 |
| =3–5 mg kg−1 | 4 (9.5%) | 3 (17.6%) | |
| >3–5 mg kg−1 | 14 (33.3%) | 8 (47.1%) | |
| Sufentanil* | <0.5–2 µg kg−1 | 6 (14.3%) | 1 (5.8%) |
| =0.5–2 µg kg−1 | 0 | 0 | |
| >0.5–2 µg kg−1 | 0 | 0 | |
| Fentanyl* | <1–5 µg kg−1 | 0 | 8 (47.1%) |
| =1–5 µg kg−1 | 34 (80.9%) | 8 (47.1%) | |
| >1–5 µg kg−1 | 0 | 0 | |
| Succinylcholin# | <1–1.5 mg kg−1 | 0 | 1 (5.9%) |
| =1–1.5 mg kg−1 | 32 (76.2%) | 8 (47.1%) | |
| >1–1.5 mg kg−1 | 2 (4.8%) | 2 (11.8%) | |
| Rocuronium# | <0.6–9 mg kg−1 | 0 | 0 |
| =0.6–9 mg kg−1 | 6 (14.3%) | 2 (11.8%) | |
| >0.6–9 mg kg−1 | 1 (2.4%) | 4 (23.5%) |
Fentanyl [mean 166.2 (53.8) μg] was the most preferred opioid in the SIM group (P<0.01) (Table 3). The second most frequently used opioid was sufentanil [mean dose: 15 (5) μg], followed by alfentanil [2.5 (0.5) mg] used in 4.8% of the cases.
Precurarization was performed in 31 cases in the SIM group. Rocuronium was used in 43.8% of the cases, atracurium in 15.7%, vecuronium in 15.7%, cisatracurium in 6.3%, and mivacurium in 3.1%. No prophylaxis was used in 15.7% of the cases when succinylcholine chloride was administered. With regard to frequency of administration, propofol [182.9 (32.8) mg] and thiopental [441.7 (60.7) mg] did not differ (P=0.19). Succinylcholine chloride [101.5 (9.7) mg] was significant as the preferred neuromuscular blocking drug in 80.9% (P<0.01) of the cases, followed by rocuronium [52.9 (6.9) mg] with 16.7%.
In the QUES group, fentanyl [181.3 (49.6) μg] was the most frequently described opioid (Table 3). In 41.2% of the cases, no opioid was specified. In 64.8% of the cases, succinylcholine chloride [110 (38.2) mg] was the most frequently used neuromuscular blocking drug, followed by rocuronium (35.3%). In 71.4% of the cases, a non-depolarizing neuromuscular blocking drug was administered prophylactically as part of precurarization. Two participants did not describe any precurarization. Thiopental [409.1 (51.4 mg)] and propofol [226.7 (37.3 mg)] were used for induction.
Discussion
Our study demonstrated a significant difference in the results of the two test instruments when investigating the performance of an SOP. Although the average number of precautionary measures named or performed to prevent aspiration did not differ between the two groups, there were significant qualitative differences. In the QUES group, the semi-sitting position and prepared suction were described much more frequently, whereas in the SIM group, APPV, ‘rapid’ performance of RSI, and arranged endotracheal tube with stylet were applied most frequently.
Qualitative and quantitative differences in the implementation of precautions against aspiration during the RSI could result from a varying group of patients. Thus, surveys of anaesthesia in obstetrics demonstrated considerable differences in the application of precautionary measures and selection of medications when compared with non-obstetric operations.2,3 In the current study, however, both groups were to perform a RSI on an acute appendicitis case. There are also regional and international differences in the performance of RSIs.2,3,13 In a national comparison of anaesthesia standards, Fassoulaki and colleagues13 demonstrated that the performance of pre-oxygenization and application of cricoid pressure is subject to variations of 14–100% and 20–100%, respectively. This could be caused by varying recommendations from the individual anaesthesiological associations or regionally different discussions regarding the purpose and intent of the individual precautionary measures.12–14 Also, clinical experience may correlate with the application of medical standards.15 In this study, we were unable to prove a connection between professional experience and the selection of and the number of precautionary measures taken during the RSI. Byrne and Greaves16 recently described how difficult it is to find a correlation between clinical performance and clinical experience.
The implementation of medical standards is often investigated using surveys. Various methods have been established for this purpose. These include face-to-face interviews, postal or telephone surveys, and more recently, the Internet. QUES frequently serve as the basis of these investigations.5 However, low return rate or restricted representation can limit the value of a study.5 Methodological errors during an investigation with a QUES due to missing data, reversal of polarity (1 instead of 6), and contradictory or inconsistent answers can also lead to a skewing of the results.17 A very frequent error, answering questions according to social desirability, can lead to a discrepancy between the theories described in the QUES (with regard to an evidence-based medicine) and behaviour that would actually occur in practice. These errors become obvious when comparing the groups’ responses during implementation of semi-sitting position and preparing suction. Although these measures are generally considered very important3,8 and the participants using the QUES indicated them as such, in the clinical reality provided for the SIM group, they were much less frequently applied.
Normally, the QUES determines the extent of existing knowledge on the part of the person surveyed. Recurrent, standardized activities that are performed unconsciously, or automatically, are generally not stored in the brain as processes, but only as products. This decreases the reliability of a QUES.18 This explains the discrepancy between the two groups when performing the APPV, ‘rapid’ RSI, and the use of an endotracheal tube with stylet. Owing to the degree of realism it provides and to its ability to represent and quantify practical skills,6,15 SIM provides an appropriate means of testing these automatic behaviours, and in this regard, it has a clear advantage over the QUES.7
But SIM is not free of methodological errors. The current SIM used in this study represents a realistic anaesthetic situation10 in which technical and non-technical skills can be learnt and practiced.19 However, being observed can cause unusual stress, much like in a test. On the other hand, the Hawthorne effect can occur. These two effects can change the natural behaviour of the test subject in simulated practice, thus endangering the external validity of the study. Technical problems (such as computer crashes) and limits (such as no palpable body temperature and no visible cyanosis) can countermand the suggestion of reality and thus lead to changes in practical procedures.20
In both groups, fentanyl (SIM: 90.8% and QUES: 94.2%) was most frequently used at a sufficient dose.2 According to another survey, alfentanil was the preferred drug in 88% of the cases, whereas in this study, only two participants in the SIM group and none in the QUES group chose to administer it.3 This could be due to differences in the group of patients or to differing teaching methods in the study from Thwaites and colleagues.2,3 Sufentanil was administered in both groups (SIM: 11.9% and QUES: 5.9%); however, the dose selected [15 (5) μg] was well below the expected dose of 0.5–2 µg kg−1.21
The majority of participants in the SIM group used propofol, whereas the medication described most often in the QUES group was thiopental. The latter record is in keeping with surveys done in the UK.2,3 The two induction agents used here enable similar intubation conditions.22 Surprisingly, despite the known high rate of side-effects, succinylcholine was predominantly used in both groups. This is in agreement with the results of other studies.2,3
This study does not provide a general representation of the current situation during performance of a RSI in Germany. Participants came mostly from one region, and the number of participants was too low to make a generally valid, representative statement.5 We purposely did not evaluate participants using both instruments, thus decreasing the risk of a bias regarding training effect or recall.6,23 The open QUES was selected because although RSI is a standardized process, there is not sufficient existing information to enable a stringent construction of the QUES. But, these data should help future researchers to deal with these limitations. The clear methodical differences between the two evaluation instruments demonstrate the difficulties involved in developing adequate training based on the data attained. To establish precautionary methods and adequate teaching based on the results of the QUES, one would emphasize on APPV, rapid performance of RSI, and correct preparation of the intubation, although data acquired by SIM would put the semi-sitting position and preparation of suction in first place. One must be aware of this discrepancy if one wishes to measure performance. However, combination of the two investigating instruments could enable the creation of better, clearer training objectives.
In conclusion, the results of this study demonstrate that SIM is a suitable method to attain data. Compared with the established method, which is investigation via QUES, SIM can easily detect method errors, such as social desirability. Beyond that, SIM is more suitable for verifying automatic behaviours and thus more closely represents the applied practice, and so SIM could be better suited to the implementation of medical standard. A combination of the two evaluation methods can result in a strategy to better reproduce SOPs, to generate new SOPs, and thus to increase patient safety.
Funding
Medical Faculty of University, University of Heidelberg, Germany (f. 203 453).
Acknowledgements
We would like to thank other members of our simulator team: C. Busch, T. Boeker, H. Gahderi, A. Schmitz, and E. Zink.
References
Author notes
The data were presented in part as a poster at the German Anaesthesia Congress (Deutscher Anästhesiecongress—DAC) in Munich from April 16–19, 2005, at the SESAM meeting in Bristol, UK from May 20–22, 2005 and also at the annual meeting of the European Society of Anaesthesia in Vienna, Austria from May 28–31, 2005.
Comments
Re: Are Simulators an effective tool for teaching and training?
Sir, Dr. D’Mello has underlined the important role of simulation in aviation and medicine. The diversity of this teaching tool in simulating a “realistic” world with a “low to high fidelity” has offered a new dimension of safe learning and training of technical and non-technical skills.[1 2] In general, trainees in anaesthesia evaluate the simulated world as very realistic.[3 4] Additionally, training of technical skills and especially of non technical skills are rated very educational and useful[2 5], and participants even show an increase in self-reported non technical skills, e.g. crisis resource management (CRM) behaviours.[3] Due to this favourable rating by the participants, simulator training has become very popular and successful in the last years. Unfortunately, as Dr. D’Mello mentioned “cost effectiveness of simulator-training is still pending”. Furthermore, most of these reported and measured improvements after simulator training are only based on a subjective impression/perception. In general, an absolute objective proof of the effectiveness of simulator training improving of technical or non technical performance in simulated and clinical critical incidents is still missing. Moreover, some studies even showed a failure of simulation training in improving performance.[6] Superiority of simulation for teaching or training compared to other established traditional (simple) teaching tools is also absent. But these facts should not disappoint the trainers and the participants and they should not question the application or the effect of simulation in general. Simulation has demonstrated to be a qualified, valide and reliable assessment tool to describe technical [1 4 7] and non-technical performance.[3] But, where simulation fails to show an improvement of participants’ behaviour, one could question if the applied training and of course its evaluation were adequate? Forrest et al. (2002) showed using a high-fidelity simulator increases novice anaesthetists technical performance of rapid sequence induction.[1] And Yee et al. (2005) showed after even a single exposure to CRM-session an improvement of anaesthesia residents in non-technical skills.[2] Both studies failed to show an additional improvement when training was continued. So should the content of simulator training be more adapted to the increased and modified requirements of already well trained participants? Yes of course, since novice and expert anaesthetists have different knowledge and experience, their needs of training is also unequal. And so it is not surprising, that non-technical and technical performance do correlate with clinical experience of participants. [1 4] Additionally, it has to be kept in mind that trained novice participants are certainly becoming more expert in the field of training. But if the training is adapted to the need of the participants, then of course statistical methods for evaluation of improved participants’ behaviour also have to be adjusted for an accurate presentation of the improved skills. For planning further investigations on effectiveness of simulation training it is necessary to accurately define the participants’ needs of training and the goals of the training in advance. Here, studies presenting their limitations and their proposals for accurate inclusion criteria for studying participants’ performance provide valuable assistance by designing a study.[1-3 6] Clearly proved evidence of the benefit of simulation in medical training will help to increase the funding of simulation. With more financial support, more anaesthesiologists will be frequently more trained to be prepared to quickly diagnose and treat critical life- threatening incidents, and so to increase safety, as it is common today in aviation.
References
1. Forrest FC, Taylor MA, Postlethwaite K, Aspinall R. Use of a high- fidelity simulator to develop testing of the technical performance of novice anaesthetists. Br J Anaesth 2002; 88 :338-44.
2. Yee B, Naik VN, Joo HS, Savoldelli GL, Chung DY, Houston PL, et al. Nontechnical skills in anesthesia crisis management with repeated exposure to simulation-based education. Anesthesiology 2005; 103:241-8.
3. Blum RH, Raemer DB, Carroll JS, Sunder N, Felstein DM, Cooper JB. Crisis resource management training for an anaesthesia faculty: a new approach to continuing education. Med Educ 2004; 38 :45-55.
4. Devitt JH, Kurrek MM, Cohen MM, Cleave Hogg D. The validity of performance assessments using simulation. Anesthesiology 2001; 95:36-42.
5. Berkenstadt H, Kantor GS, Yusim Y, Gafni N, Perel A, Ezri T, et al. The feasibility of sharing simulation-based evaluation scenarios in anesthesiology. Anesth Analg 2005; 101 :1068-74.
6. Olympio MA, Whelan R, Ford RP, Saunders IC. Failure of simulation training to change residents' management of oesophageal intubation. Br J Anaesth 2003; 91:312-8.
7. Zausig YA, Bayer Y, Hacke N, Sinner B, Zink W, Grube C, et al. Simulation as an additional tool for investigating the performance of standard operating procedures in anaesthesia. Br J Anaesth 2007; 99: 673- 678.
Conflict of Interest:
None declared
Re: Universal Patient-Protection Algorithm
Sir, I really support this very important issue raised by the author. It is in the responsibility of every physician to prevent any harm from the patients while treating them. The implementation of algorithms into medical practice has been very successful in the past, and standard operating procedures have shown to be reliable and helpful tools. Additionally, following these procedures, for example in basic or advance life support or in airway management, has led to a reduced morbidity and mortality.[1,2]
The precautions to prevent aspiration during the rapid sequence induction described and evaluated in our study present a proper basis to formulate a safe and successful standard operation procedure in anaesthesia. For sure, an expansion with other precautions like, the measurement and documentation of intra-cuff pressures of endotracheal tubes is desirable. This might reduce the incidence of tracheal trauma in tracheal intubated patients in the future.
1.Nolan JP, Deakin CD, Soar J, Böttiger BW, Smith G; European Resuscitation Council. European Resuscitation Council guidelines for resuscitation 2005. Adult advanced life support. Resuscitation. 2005 ; 67: S39-86
2.Heidegger T, Gerig HJ, Henderson JJ.Strategies and algorithms for management of the difficult airway. Best Pract Res Clin Anaesthesiol. 2005; 19: 661-74 Anaesthesiol. 2005; 19: 661-74
Conflict of Interest:
None declared
Sir, I read with interest the paper by Y.A Zausig and colleagues (1) and wish to comment on this as well as share my personal views. . Since antiquity, simulation have been used in healthcare, representations in clay and stone were used to demonstrate clinical features of disease states and their effects on humans. Models have been found from many cultures and continents. Currently, simulators are being used extensively throughout the world, in the field of aviation. Since I am a trainee pilot myself, I have found the usefulness of simulators in mimicking hazardous situations and performing dangerous maneuvers. Simulators tend to prevent accidents by their ability to train pilots to a higher level of competence than is possible in actual airplane training(2,3). That is because many maneuvers can be performed in simulators that are just too dangerous in airplanes and they can be repeated multiple times until the pilot is honed to a very sharp edge. Similar to the aviation environment, anesthesia practice in the operating room involves multiple tasks requiring a high degree of vigilance, procedural, monitoring and decision-making skills in a dynamic, complex environment which is affected by the simultaneous interactions of the different members of the operative team. As in aviation, critical life- threatening incidents are rare, but when they do occur, they have potentially disastrous consequences unless the anesthetist is able to quickly diagnose and correct the problem. Simulation technology provides a potential way of learning and practicing all the skills involved in anesthesia, including crisis management without harm to a real patient(4)
Types of medical simulators available- P (One P) Simulation (Low fidelity): A “One P” simulator includes anatomical models such as the ”Ressuci” dolls which are used for training in cardio pulmonary resuscitation. These mannequins can be passive, active, or interactive based on the level of training. Simple mouth to mouth breathing and chest compression is taught on a passive simulator (P1 p) . More advanced models may simulate wounds, or the result of wounds such as a pneumothorax. If the wound simulated bleeding, or air movement, then it would be a (P1a) simulator.
PP (Two P) Simulator (Intermediate fidelity): Advanced CPR with cardiac life support is taught on a mannequin combined with a computer program which simulates the electrocardiogram. In this case, both the patient and the diagnostic test are being simulated. This is a (P1 a P2a) simulator. If the electrocardiographic portion of the simulator has been programmed to respond to the administration of medications, or electrocardioversion, then this would be a (P1 a P2 i ) simulator.
PPP (Three P) Simulator (High fidelity): usually includes a mannequin and software which simulates the patient interactively. e.g. Leiden Anaesthesia stimulator
PPPP ( Four P) Expert systems: Although the need for Four P simulators is rare. The analysis shows that such a machine would be an expert system, with all elements of the clinical interaction represented. From a theoretical point of view, this machine could be used to study the economic and cost-benefit aspects of the health care process(5).
Currently simulators are being used for both teaching and training and their popularity is increasing. 1. They can simulate real life threatening scenarios such as anaphylaxis, failed intubation / failed ventilation as well as rare scenarios such as malignant hyperpyrexia 2. there is no risk to real patients 3. repeated assessments are possible 4. video taping for review and discussion 5. with reduction in working hours for trainee doctors, simulators can play a valuable role in both training and assessment of trainees
Cost effectiveness of simulators- Definitive studies evaluating the cost effectiveness are still pending. Despite the enthusiasm and high ratings given by the participants to such training programs and the intuitive usefulness of training with a full scale simulator, the efficacy of such training compared to traditional methods of teaching is difficult to prove. The simulated setting will never fully recreate real life and the unpredictability in which real patients may respond to various interventions. Whether training on a full scale simulator makes a significant difference in patient outcome is even more difficult to ascertain. Because of its substantial cost, full scale simulators also have to answer to the question of their cost effectiveness, as compared to traditional methods of teaching in anesthesia. However Chopra et al(6) found that with the use of simulator training,there was indeed an increase in the performance by anaesthetists dealing with emergency situations and helped in better adherence to guidelines. Simulators have even been introduced into the Objective structured clinical examination scenarios for the primary FRCA
In conclusion from my own personal experience with high fidelity simulators, I believe they do improve both competence and confidence with dealing with emengency situations and could indeed be used as an assessment tool of competency
references:
1. Simulation as an additional tool for investigating the performance of standard operating procedures in anaesthesia Y.A. Zausig, Y. Bayer, N. Hacke, B. Sinner, W. Zink, C. Grube, and B. M. Graf Br. J. Anaesth. 2007 99: 673-678
2.What impact does the use of flight simulators have on commercial aviation? Robert J. Boser Editor-in-Chief; Airline Safety.Com
3.Use of advanced simulator technology in aviation education. S. Thatcher; International Journal of modeling and simulation
4. Full scale computer simulators in anesthesia training and evaluation, A. Wong; Canadian Journal of Anesthesia 51:455-464 (2004)
5. A Typology of Simulators for Medical Education, G.Meller Journal of Digital Imaging, August 1997
6. Chopra V, Gesink BJ, de Jong J, Bovill JG, Spierdijk J, Brand R. Does training on an anaesthesia simulator lead to improvement in performance? Br J Anaesth 1994; 73: 293–300
Conflict of Interest:
None declared
Y. A. Zausig and colleagues' suggestion of the SIM and QUES instruments as adjuncts to advance safety during anaesthesia is commendable. One threat to safety during anaesthesia is post-intubation tracheal stenosis caused by regional ischaemic necrosis of the airway. This clinical condition is often progressive and can take up to 3 weeks to manifest - commonly misdiagnosed for asthma.
Tracheal stenosis remains the most common indication for tracheal resection and reconstruction. Prognosis following these surgical procedures which yield a failure rate of 3.9% and a mortality rate of 2.4% has been reported at the Harvard Medical School [Postintubation tracheal Stenosis.Wain JC.Harvard Medical School, 25 Shattuck Street, Boston, MA 02115, USA]
A universal algorithm against such deliterious trauma should be instituted in the wider interest of patient safety . Each anesthesia and ICU setting must document intra-cuff pressures of endotraceal tubes against the brands used as regularly as possible and intubated tracheas examined endoscopically following extubation . When lesions are spotted they should be recorded, notified and treated appropriately.
Ultimately, to offer patients their rightful widest level of protection,the provision of more information to the patient including the possible misadventures of tracheal stenosis is necessary as a benchmark for care prior to Anesthesia and ICU intubation
John George Fellow - Malaysian Institute of Medical Laboratory Sciences 4 Lrg 4/48 F, 46050 , PJ, Selangor , Malaysia
Conflict of Interest:
None declared