Changing work practices and the evolution of more complex interventions in cardiovascular medicine are forcing a paradigm shift in the way doctors are trained. Implantable cardioverter defibrillator (ICD), trasncatheter aortic valve implantation (TAVI), carotid artery stenting (CAS), and acute stroke intervention procedures are forcing these changes at a faster pace than in other disciplines. As a consequence, cardiovascular medicine has had to develop a sophisticated understanding of precisely what is meant by ‘training’ and ‘skill’. An evolving conclusion is that procedure training on a virtual reality (VR) simulator presents a viable current solution. These simulations should characterize the important performance characteristics of procedural skill that have metrics derived and defined from, and then benchmarked to experienced operators (i.e. level of proficiency). Simulation training is optimal with metric-based feedback, particularly formative trainee error assessments, proximate to their performance. In prospective, randomized studies, learners who trained to a benchmarked proficiency level on the simulator performed significantly better than learners who were traditionally trained. In addition, cardiovascular medicine now has available the most sophisticated virtual reality simulators in medicine and these have been used for the roll-out of interventions such as CAS in the USA and globally with cardiovascular society and industry partnered training programmes. The Food and Drug Administration has advocated the use of VR simulation as part of the approval of new devices and the American Board of Internal Medicine has adopted simulation as part of its maintenance of certification. Simulation is rapidly becoming a mainstay of cardiovascular education, training, certification, and the safe adoption of new technology. If cardiovascular medicine is to continue to lead in the adoption and integration of simulation, then, it must take a proactive position in the development of metric-based simulation curriculum, adoption of proficiency benchmarking definitions, and then resolve to commit resources so as to continue to lead this revolution in physician training.
Complex interventional cardiovascular procedures such as carotid artery stenting (CAS) and transcatheter aortic valve implantation (TAVI) have demonstrated good clinical results in high-risk patients but are associated with high morbidity and mortality.1,2 Cardiovascular Societies have long taken the lead in attempting to define the necessary training requirements for competency in new devices and procedures.3 These consensus training competency recommendations have been largely designated by length of time in formal residency/fellowship training programmes or proctored and un-proctored procedural volume. Procedural volume criteria have historically represented a surrogate measure of operator procedural proficiency. One recent multi-societal Clinical Competency Guidelines has, for the first time, suggested that metric-based simulation training may be a possible pathway for operator training in a new procedure, CAS.3 Additionally in the USA, the Food and Drug Administration (FDA) has also recently adopted simulation training as a part of a new procedure/device training4 and the American Board of Internal Medicine (ABIM) has adopted simulation training as modular element in the required recertification process for Interventional Cardiology Board certification. Despite these recent changes, the question still remains—how should interventionists optimally be trained in new complex procedures such as CAS, TAVI, as well as other new high-risk procedures in the future?
Globally, against a backdrop of reduced work hours5,6 and reduced patient training opportunities there is greater public scrutiny and accountability about the skill acquisition process, particularly in the highly technical specialities such as surgery and interventional medicine. This has forced the medical community to develop a more sophisticated understanding of what is meant by ‘training’ and ‘skill’7 and the skill acquisition process in a clinical and regulatory environment that is radically different from residency training of the early 20th century.8Figure 1 shows and details the different levels of skill acquisition process for any level of medical trainee plotted against a traditional learning curve.7 A fundamental function of any medical educational and training curriculum should be to facilitate the attainment of proficiency by the transition of the trainee from one skill level to the next.
One of the most powerful approaches to training that has been used in medical training for decades has been ‘on-the-job training’ on patients.6 This apprenticeship model has been largely based on graded responsibility in the operating room or cardiovascular lab while performing real procedures on actual patients. However, it has become increasingly evident that, the operating room and cath lab should no longer be used as the primary training environment for the acquisition of basic procedural skills. Medical education and procedural training are now evolving using the new tools of better-defined educational curriculum, online training, and using virtual reality (VR) simulation in the procedural skills laboratory to train physicians. Ultimately this new medical education pathway may supplant much of the early learning curve of ‘physicians in training’ which would normally take place on real patients.9
Simulation-based procedural training
Virtual reality-based simulation training has been proposed as a solution for the reduced procedural skills exposure as well as a new way to measure the procedural skill acquisition in operators who undertake new procedural training, frequently many years after completing their formal residency/fellowship.9,10 How to effectively train practicing physicians in new procedures is becoming increasingly important since new complex procedures and devices are constantly being developed in interventional cardiovascular medicine. The idea of using VR simulation to train the certain key elements of a procedure in a risk-free environment, while measuring operator performance has been the driving force in the development of VR training curriculum.11 Ideally, VR simulation should allow a trainee to practice and rehearse skills necessary during the procedure while giving them the performance feedback. Furthermore, when mistakes made during the procedure, so-called proximate feedback is immediately given to them.9 It is this coupling of all these feedback features (i.e. procedure practice, procedural performance measures, and proximate feedback) that makes VR simulation training such a powerful, effective, and efficient tool in the process of learning in procedural medicine.10 As has been demonstrated in other high-skill professions such as aviation, VR simulation is a powerful, effective, and measurable approach to skills learning in complex tasks.12
What makes VR procedural medicine training so different from traditional repeated practice training is that it involves training using a systematic mentor-based approach that gives continual feedback to the trainee, proximate to their performance.13 Virtual reality procedural practice training with proximate feedback9,14 affords the trainee the opportunity to see what they did right and more importantly what they did wrong, thereby speeding the learning process.
Metric-based performance feedback for procedural medicine
In all levels of medical procedural training, performance feedback to the trainee is more effective when it is objective, consistent, and precise. Non-specific instructor feedback such as ‘uses unnecessary force with catheters’ or ‘repeatedly makes tentative or awkward moves with instruments’15 certainly gives an indication that there are skills deficits, but this type of feedback is vague and open to miss-interpretation. More precise feedback might include ‘traumatic engagement of the ostium without gaining vessel access’ or ‘advancing a catheter without a guide-wire to support it’.9 These specific feedback performance measures or ‘metrics’ are clearly identified performance characteristics that are unambiguously operationally defined so that they can be reliably measured.7,9 Metrics are not necessarily catastrophic events, but they do indicate performance characteristics that deviate from optimal performance. Good metrics are easily and reliably recognizable, scoreable and thereby can be used to measure the progressing skills acquisition.9 Metrics should ideally capture the subtle essence of what represents optimal procedural performance and taken together constitute proficient or expert performance. In procedural training it is these subtle aspects of operator performance that instructors aspire to teach trainees in addition clinical judgement.
Interventional Cardiology has been fortunate that the most sophisticated VR simulators currently available in medicine are dedicated for learning endovascular procedures.10 Also, VR simulation-based training curriculum have been developed, used widely, and endorsed by major cardiovascular societies.11 However, it is important to note that prior to any VR simulation system being used for high-stakes assessment of operator performance, such as certification or credentialing, rigorous validation studies must be done. This ensures that the skills trained on the simulator reflect the appropriate skills required in the actual procedure.
It has been demonstrated in prospective, randomized, double-blinded studies that surgical residents trained to a metric-based level of proficiency on a simulator make significantly fewer objectively assessed intra-operative errors.16,17,18 Likewise, validation studies on VR simulation training for endovascular skills have demonstrated similar results.9,19–25
The science from these studies has not only shown that VR simulation is a very effective and efficient approach to training but it has also forced instructors to begin understanding precisely what it is about the traditional approach in procedural training that has made it effective. It has made instructors carefully identify and define which key elements of the procedure are critical and ultimately define operator proficiency. These performance elements must then be incorporated into the VR simulation and objectively measured by the simulation system during the training. Finally the trainee's performance on the simulator must be benchmarked (usually representing expert performance of the same VR-simulated task). Ultimately, prospective, randomized validation studies must then be done to assess whether the VR simulation training transfers to cath lab performance.
To date this training approach has been shown to be as effective for the acquisition of very advanced technical skills as traditional apprenticeship training models in the relatively few procedures that have been thoroughly validated.14–17,19,20,25 Two randomized trials, in surgery (VR to OR)16 and in Interventional Cardiology (STRIVE)25 have demonstrated significantly reduced procedural errors (>50% reduction in VR trained groups) in residents learning laproscopic cholecystectomy16 and attending physicians newly learning carotid angiographic procedures.25
Although multiple medical disciplines, particularly surgery, have greatly contributed to the field of medical simulation cardiovascular medicine has accelerated the acceptance of VR simulation as an important education tool by developing and implementing worldwide training programmes, by initiating clinical guidelines that incorporate VR simulation as a potential training pathway and by making VR simulation an integral part of national scientific session meetings and national/international fellow education.3,11
In 2005, the Society of Cardiovascular Angiography and Interventions (SCAI) and the American College of Cardiology were the first to incorporate into clinical guidelines the suggestion that metric-based VR simulation training may be an acceptable training pathway for a new procedure, i.e. CAS.3 Cates11 reported on the establishment of the first medical society endorsed-tiered CAS curriculum incorporating the use of a validated metric-based simulation training programme leading to operator certification upon completion. This three-tiered programme consisted of (i) structured didactic educational programme, (ii) online interactive modular education and testing curriculum, and (iii) an intense mentor-guided instruction using metric-based simulation training until the trainee demonstrated benchmarked proficiency. The same type of CAS-tiered simulation educational system was modified and adapted for individual world markets and then applied with industry partners in Asia, Australia, and South America. In one collaboration, SCAI and the Brazilian Interventional Cardiology Society implemented this formal, tiered, CAS training programme with VR simulation (which certified operators in CAS) from all over Latin America. A similar simulation methodology was recently used in the USA to train and certify cardiologists in acute stroke intervention. In that programme two simulation platforms were used: a clot retrieval wet lab simulation used under standard cath lab fluoroscopy along with a full-physics VR simulator for intracranial stent placement.26
Cardiovascular medicine has also led by introducing VR simulation into the standardized formal Board Certification process in the USA. The American Board of Internal Medicine (ABIM) has for the first time included simulation as a module for the completion of the ‘maintenance of certification’ (MOC) requirement for recertification in Interventional Cardiology. As a result of all these factors, VR simulation has become a mainstay of accepted education at most national and international cardiovascular meetings and scientific sessions.
On the regulatory side, since 2004 when the initial FDA approval for CAS and subsequent training of CAS operators included VR simulator training,4 most new high-risk cardiovascular technologies have included VR simulation training as a required element of operator training and certification in the roll-out of those devices and is currently part of FDA guidelines for device approval.27
Cardiovascular simulation systems
Teamwork, operator confidence, and the interpersonal skills necessary for the smooth function of a cardiovascular procedure team are without doubt essential for safe, effective, and efficient procedure suite functioning. Dysfunctional relationships have been repeatedly highlighted as a recurrent source of factors that compromise operative performance impacting on safety, e.g. the Bristol Case28 and the To Err is Human Report.29 The importance of ‘human factors’ is only surpassed by the operator's ability to perform the procedure effectively, efficiently, and safely, all within the known distractions of a busy lab environment and dynamic patient care setting. One of the major advantages of high-fidelity VR simulation is that it affords the trainee the opportunity to perform the entire operative procedure, with the same devices, used in the same order as they would during a real patient procedure. These simulations also require the trainee to perform the procedure while looking at a realistic cardiovascular clinical interface (e.g. fluoroscopic images, digital roadmaps, vital signs, etc.) and allow for the trainee to be challenged with real-life change of events as they might occur during the course of an actual procedure. In this way, the operator teams can be exposed to real-life clinical situations and learn and react together.30
There are a number of VR simulation vendors who manufacture stand-alone full-procedural simulators and also allow for team training with support staff. One such system is Simantha (Medical Simulation Corporation, USA) Figure 2A, which also offers simulations for endovascular abdominal aortic aneurism repair (EVAR) (Figure 2B), and transcatheter aortic valve implantation (TAVI) (Figure 2C). This simulation system also has coronary intervention modules that are currently being used by the ABIM for MOC in Interventional Cardiology Board recertification. The ability of this system to monitor and give automated immediate operator technical feedback to device movement is limited due to the lack of full-physics capability within the VR system. Validation studies using this system are limited; however this system is well suited for team training and cognitive operator training capability.
Simbionix (USA) offers a full-procedural simulator on their AngioMentor platform (Figure 3A) for a wide range of cardiovascular procedures including, TAVI (Figure 3B) and Intracranial Aneurysm Coil Embolization (Figure 3C). The same platform offers peripheral vascular and coronary intervention modules. These types of VR simulations have been widely used and are important because of their completeness and their similarity to the in vivo procedure. Although this system can represent important constituent elements of an effective VR simulation, it is lacking in the capability of the simulator to give proximate, metric-based feedback on changeable aspects of procedural performance, e.g. catheter and wire movements on a second-by-second basis. The AngioMentor system can measure the length of time taken to perform the procedure, the amount of fluoroscopy or contrast agent used, which devices were used in what order, percentage of a lesion covered by the stent, etc., which are important summative performance metric units. They are, however, of limited value without contextual metric-based operator/device-related performance assessments. For example, a procedure could have been performed quickly with minimal fluoroscopy or contrast agent, but the operator could have missed important procedural steps or indeed performed the procedure in a way that repeatedly exposed the patient to unacceptable risk. Furthermore, information on whether a trainee chooses the correct catheter, wire, stent, or valve also does not provide information on what was done with these devices intra-operatively. This simulation system currently only provides summative information but not metric-based assessments of fluid aspects of operator performance that can be used for proximate feedback and has, to date, limited validation studies.
One example of an insightful use of multiple simulations for training cardiovascular procedural skills has been developed by Hartwig Retzlalff (Medtronic in Tolochenaz, Switzerland). At this facility (Figure 4A) physicians are trained in the cognitive and procedural skills necessary for the implantation of implantable cardioverter defibrillators and cardiac resynchronization therapy. Navigation with catheters, wires and leads, and appropriate placement of the leads within the chambers of the heart are fundamental skills that are trained using this system.
However, uniquely this simulation programme combines these catheter-based skills training and surgical simulations in one training system. Trainees not only learn important catheter wire and lead handling skills but also gain experience in careful attention to overall procedural detail, including aseptic technique and atraumatic tissue handling during the creation of the pocket for holding the pacing device (Figure 4B). No validation or metrics currently exist for this system but its comprehensiveness is intuitively attractive.
A Swedish company Orzone AB (Gothenburg) has taken the idea of a dedicated, full-procedural, high-fidelity cardiovascular simulation workstation a step further and created a dedicated integrated operating/catheterization laboratory. This state-of-the-art training space (Orcamp, Figure 5) simulates the operating room (including table), instrument controls, and video monitors that the operator must interact with during the procedure. The room can be configured to duplicate most modern catheterization laboratory set-ups and is capable of coupling to most high-fidelity cardiovascular simulators. It was developed to increase the similarity between the learning/training environment and the in vivo operating environment particularly for new procedures that blur the lines between catheter-based disciplines and surgery such as TAVI and EVAR. Thus, this simulation system can be used to train procedural skills on a cardiovascular simulator as well as instrument control, ergonomic challenges in operating on an operating room (OR) patient, all while operators are fully gowned and gloved (with lead aprons, thyroid collars, and leaded glasses). It can also be used as a team training environment particularly for newly constituted teams or for the development of team-working protocols for these new high-risk devices. Currently, no validation or metrics exist for this system.
Without question the most validated full-physics VR simulator in cardiovascular medicine is the vascular intervention simulation trainer (VIST) system (Figure 6D) by Mentice AB (Gothenburg, Sweden), e.g. 9–11, 18–20, 27–30. Learning is more effective and efficient if performance feedback is given proximate to actions and that is what VIST is able to accomplish and deliver. This type of simulation model is rendered from real-patient anatomy (see Figure 6A–C) and the simulation can deliver assessments on trainee performance on a second-by-second basis through algebraic approximations of the vascular tree and using the full-physics characteristics of the catheter–vessel wall interactions (as shown in Figure 6B). This simplified representation of the vessel surface allows for much faster collision detection between devices and simulated vessel wall. VIST also uses programmed properties of the devices (stiffness, friction, etc.) to calculate the forces of movement, which then will determine the shapes and feedback that the user will see and feel. Feedback is calculated in real time and depends on the interaction between the virtual devices and the virtual anatomy. These detailed renderings of patient anatomy are extracted from computer tomography or magnetic resonance angiography (MRA) information acquired from real patients and the results are VR simulations that look good, but more importantly, behave in a way that concord with real patient experiences. This approach has recently been corroborated with the use of the patient-specific anatomic data from a particular patient undergoing carotid artery stenting with embolic protection.23 In that case, for the first time in man, the patient-specific MRA carotid anatomy of an individual potential CAS patient was rendered for the VIST simulator and the CAS procedure was rehearsed on the VIST simulator just before it being performed live on that specific patient (i.e. ‘mission rehearsal’).23 The in vivo case demonstrated direct concordance to the VR-simulated case. This allowed for appropriate device selection, device sizing and included the identification of difficulties with vessel access and the embolic protection location, which since it was identified in the VR-simulated case, was avoided during the live in vivo case. Although currently this level of VR simulation fidelity is very time consuming to create (i.e. ∼2 days) the future of interventional procedural practice will likely incorporate this type of ‘mission rehearsal’ particularly, for challenging patients, for difficult anatomy, or for high-risk or high-cost procedures.23
Additionally on the VIST system, the metric-based, full-physics, high-fidelity simulation allows trainees to receive immediate proximate feedback on their performance (as shown in Figure 7) thus allowing trainees the opportunity to identify, in real time, what they have done wrong and correct it. This type of performance assessment and immediate feedback can be applied to something as simple as ‘ advancing a catheter without a wire supporting it’ to more complex, risk-prone, and technically challenging complete procedures such as aortic valve implantation or carotid stenting. However, detailed, reliable operator, and device assessments of accurate peri-procedure catheter/device techniques (metrics) are required in these type of very high-fidelity VR simulations. More importantly, these types of operator feedback require carefully validated metrics that have been shown to reliably measure operator performance. Knowing where to place a carotid stent or aortic valve is not the same as being able to place it correctly. Furthermore, assessments on the simulator must reliably and accurately reflect the physician's operative capability/skill. It must avoid creating a false sense of confidence in the operator's ability or the ease with which the procedure can be performed, all of which may be present without carefully validated metrics and critical peri-procedural performance indicators. It is the simulation capabilities of procedural rehearsal and the fidelity of performance sensing and assessment which afford metric-based feedback to the trainee that drive skill acquisition up the learning curve at a steeper rate (as shown in Figure 8).
Importance of performance benchmarks
Some generic performance metrics can be used to assess general performance in new procedures (e.g. time, contract, fluoroscopy, device size, deployment, etc.). However, specific procedure and device metrics will need to be developed and validated for new procedures as has been done for coronary24 and carotid stenting9,22,25 before those VR simulation systems can be used for high-stakes assessment of operator procedural performance (i.e. certification and credentialing). Necessary validation studies on each system/curriculum being considered for high-stakes assessment of operator skills should include: face and content validation; construct validation; discriminative validation; and finally predictive validation with randomized studies demonstrating that the metrics allow for appropriate clinical skills transfer to the live patient environment. Once validated, not only can these metrics be used for performance feedback during training, but also used to establish benchmarks of minimal performance trainees must demonstrate before progressing to perform the procedure on real patients. There is growing evidence that this method is a superior approach to training compared with training for a fixed time period or case volume in surgical procedures.16,17 Early reports suggest similar results in catheter-based procedures as well.21,25
Performance benchmarking using VR simulation has now been accepted by the FDA and used for the roll-out of a new interventional technology, e.g. CAS4 and FDA clearly sees the importance of simulation-based training in the future approval of new procedural technologies in medicine.27 Also, VR benchmarking has now become the basis of worldwide societal-based procedure training curriculum.11 Despite all this, the question that still plagues organized medicine and the cardiovascular specialists is: to what competence level and to what practical degree of procedural training will be sufficient for the safe performance of new, high-risk, and technically challenging procedures in the future? Also, does the Interventional community have the will to commit the resources required to develop the necessary high-fidelity, full-physics procedural simulators? As importantly, does Cardiology intend to develop the required validated metric-based simulation curriculum to effectively and efficiently train (to a defined benchmark) the complex new techniques/procedures of the future? If the answer to these questions is yes, then this fact will forever change the paradigm of interventional procedure training, dramatically improve overall cardiovascular procedural quality, and radically improve patient safety. Only time will tell.
Changing work patterns and the evolution of increasingly complex cardiovascular interventions has forced cardiovascular interventionists to develop a better understanding of procedural skill acquisition and learning. Virtual reality simulation offers a uniquely poised solution that can augment the learning experience. Virtual reality simulation can supplant much of the skill acquisition that was previously acquired only by repeated practice on patients with the associated risk to those patients. High-fidelity, full-physics VR simulations afford the trainee the opportunity to practice the procedure in a safe environment and receive detailed metric-based proximate feedback on their performance. It also offers the opportunity for the trainee to acquire critically important aspects of skilled performance to a quantitatively defined benchmark. This we feel will be essential for new high-risk cardiovascular procedures in the future.
Conflict of interest: none declared.
- acute cerebrovascular accidents
- implantable defibrillators
- aortic valve replacement
- cardiovascular surgical procedures
- cardiovascular system
- united states food and drug administration
- carotid angioplasty and stenting
- simulation training
- virtual reality
- transcatheter aortic-valve implantation
- american board of internal medicine
- maintenance of certification