Article Text
PDF
Statistics from Altmetric.com
- electrophysiology
- quality of healthcare
- education
- medical
- pacemaker
- artificial
- outcome assessment
- healthcare
Introduction
In cardiac implantable electronic device (CIED) implantations, as in other cardiovascular procedures, therapy outcomes and complication rates seem to correlate with the operator’s level of experience with the given intervention. Furthermore, the impact of the learning curve has been shown for novice physicians attempting an established conventional procedure1 as well as for expert operators adopting a new technique or technology.2 3 Even though a relationship seems to exist between procedure volume and clinical outcomes,4 it is not completely understood how the quality of the implant performance changes with increasing procedure experience volume. Patients should be assured of a verified and measurable quality of treatment that should not vary greatly as a function of physician experience. The European Society of Cardiology has highlighted as a priority across member countries the need for assessing skills more consistently and away from the bedside, rather than counting procedure numbers.5 Training may play a fundamental role in developing the operator’s skills to an established performance level before implementing a new therapy or technology in vivo. Unfortunately, across the world, current procedural training offered to beginners (no matter how senior or experienced) is not consistent; curricula and therapy approaches may vary with the provider and so does the assessment of ‘competence’. Many institutions continue to rely on the traditional apprenticeship model introduced by Halsted more than a century ago, where trainees learn by hands-on experience and education within a hierarchical training programme, on patients, under consultant/attending level of experienced physician/surgeon supervision.6 This approach to training is often unstructured due to patient/supervisor variabilities and lacks performance feedback that is uniform, validated, objective and transparent. Furthermore, it exposes patients to higher risk during trainees’ learning curve (and beyond, if training cannot guarantee a given level of performance).
Novel approach to CIED implant training
A novel approach to enhance the learning outcome and training of doctors to competently perform CIED implants could be metric-based (virtual reality) simulation training to proficiency. Such training would ensure that the first part of the learning curve takes place in a risk-free training environment and quantitatively ensures a predefined, objectively (rather than subjectively) assessed performance level of the trainees before proceeding to real patients. The imperative of such a systematic approach to training increases with procedural and anatomical complexity of more sophisticated interventions such as cardiac resynchronisation therapy (CRT), as well as with novel implantable technologies. The incremental challenges of CRT also lead to increased procedure-related complications, which are reported to be up to 10%–15% in large national databases and major CRT trials,7 8 and eventually worsen the patient prognosis.9 10 Finally, even when the CRT procedure is completed, approximately 30% of patients may still respond suboptimally to the treatment for reasons that are in part related to the quality of the implant.11 It is, however, difficult to determine how the different implant techniques affect the final results.
Performance metrics for a CIED implant procedure
A selected, international group of experienced CIED implanters has pioneered a method to help prepare novice implanters to perform pacing and CRT procedures. With the objective of creating a metric-based training to proficiency, the group, under the supervision of a behavioural scientist, has first developed a detailed performance characterisation of a ‘skin-to-skin’ CRT procedure following a previously described validated scientific methodology.12–14 Such depiction has been founded on existing CRT practice guidelines by professional societies,15 device manufacturer manuals and decades of implant practice among the group. Through a modified Delphi process, international expert consensus has been obtained on the resulting ‘reference’ procedure and the associated intraoperative performance ‘metrics’ (compliance with recommended steps, as well as errors to avoid), which cover every aspect of a triple chamber CIED implantation.14 Each ‘metric’ element derived from the initial task analysis has been constructed using unambiguous operational definitions (rather than descriptions), that is, specifying in detail how it should or should not be done, so that it could be scored with a high degree of reliability as either occurring or not occurring (yes/no) by an independent group of reviewers. Subsequently, construct, reliability, as well as discriminative validation evidence of these novel intraoperative metrics have been reported.16 The procedure metrics reliably distinguished between the objectively assessed performance of novice and experienced CRT implanters; furthermore, they differentiated performance levels within a group with similar experience. These performance metrics will underpin quality-assured novice implanter training. Trainees will be required to demonstrate a predefined proficiency benchmark, established based on the average performance — quantified by the metrics — of experienced implanters carrying out routine procedures. Such an approach to training ensures a more homogenous skillset, can be applied to any level of training and is known as proficiency-based progression (PBP, figure 1)12.
Proficiency-based progression training paradigm as an iterative process applied throughout and within training as well as for skill development for new procedures or devices.
In today’s medical education, trainees’ behaviours during CIED implants are typically tracked using high-level task descriptions and assessed using 7-point Likert-type scales (reviewers are asked to indicate their level of agreement with a given statement), leading to substantial intra-rater and inter-rater variability.17 Figure 2 illustrates an example of the differences between the PBP and the traditional approach in tracking the same performance of a trainee fixating a CIED lead. A direct observation of procedural skills (DOPS) scoresheet condenses the entire lead fixation task under a single descriptor (figure 2A) and records the performance using a subjective scale; a PBP scoresheet (figure 2B) instead contains detailed unambiguous performance metrics derived from expert consensus and records whether each step of the task has or has not been performed (according to the definition), and how well and safely it was performed (not performed or performed badly is scored as either an error or critical error when that tick box is available in the scoresheet).
Example of tracking the same trainee performance during a pacemaker lead fixation task: using DOPS (A, extracted from the complete procedure scoresheet) and using explicit validated PBP metrics (B, extracted from the complete procedure scoresheet). DOPS, direct observation of procedural skills; PBP, proficiency-based progression.
Prospective, randomised and blinded clinical trials performed in other interventional disciplines have demonstrated that PBP training leads to significantly (ie, 40%–60%) better intraoperative performance compared with traditional training.18–21 It has also been shown that the exact same simulation training delivered without using validated metrics generates only a minimal improvement in performance compared with traditional training.19 22 One of the main reasons for these large performance differences is that PBP affords trainees the opportunity to engage in deliberate rather than repeated practice: learners receive immediate, objective, specific, transparent feedback addressing performance gaps to reach the target proficiency level. Simulation is used as a tangible vehicle to deliver the training curriculum. The PBP training method is not meant to completely replace clinical in vivo training but aims to supplant a significant part of the initial learning curve on real patients. This way, clinical supervisors receive pretrained beginners in the operating theatre and will know, with a high degree of quantitative assurance, their clinical knowledge and technical performance level. Consequently, they will be able to optimise the clinical skill acquisition process for the trainee by ensuring the aggregation of marginal gains.
Simulation for CIED implant training
The impact of simulation in training novice implanters can be better appreciated when the role of cognition and, in particular, its ‘attention’ component is considered.12 Human beings have a limited attentional capacity, which means they can only attend to a finite amount of information or stimuli at any given time. Novices learning new implant skills must allocate significantly more attentional resources than experts to consciously monitor what their hands are doing, in addition to the spatial judgements and navigation under fluoroscopy, as well as operative decision making. This means that they quickly reach their attentional capacity threshold. At that point, any additional instruction or unexpected intraoperative event is unmanageable by this operator. Through deliberate practice, simulation facilitates automation of the psychomotor skills and spatial judgements, so attentional capacity resources are liberated for other tasks. The time it takes for this automation process to happen varies with the trainees: the more innate visiospatial, perceptual and psychomotor abilities (ie, ‘talent’) they have, the faster they will acquire the procedural skills, thus requiring fewer attentional resources for basic tasks. Therefore, a fixed amount of time spent in training or a fixed number of CIED procedures do not necessarily guarantee proficiency for everybody. Furthermore, this approach to training ensures that consultant/attending colleagues have greater bandwidth to impart procedure wisdom, tips and tricks for safer, more efficient and better performance rather than just fundamental aspects of the procedure.
Future implications for CIED implant training
A paradigm shift towards a more systematic and scientific approach to medical education and training may need to happen in order to produce more consistent, predictable performances and outcomes from future generations of practitioners.23 The reference pacing/CRT procedure and its metrics defined by the group of experienced implanters and consented by international experts represent the foundations of a more outcome-driven, proficiency-based training curriculum for novice CIED implanters. Identifying which subset of these CRT metrics best distinguish between novice and expert implantations will focus training on key steps/errors and will help set a realistic proficiency level that trainees should achieve in simulation before progressing to in vivo practice.
Simulation represents a viable and effective tool to support a structured learning curriculum and facilitate, through deliberate practice, the acquisition of the necessary knowledge and skills to perform the desired tasks at a predefined level of proficiency. As an example, figure 3A shows a porcine tissue model and the surgical equipment enabling metrics-based simulation training for the leads fixation task previously described. Likewise, figure 3B shows a full physics virtual reality simulator for leads implantation training and illustrates the moment of the deployment of the right ventricular lead screw into the heart tissue; this subtask is characterised in PBP by a sequence of explicit metrics such as ‘Confirm lead is in target position under fluoroscopy in AP and LAO views’, ‘Fully insert stylet’, ‘Make lead fixation indicators visible under fluoroscopy’ and ‘“Watch indicators spacing while turning until they are fully deployed; avoid >3 extra turns’, empowering granular tracking of trainees’ performance and timely instruction from supervisors. The PBP implant metrics are specific for the type of lead (active or passive fixation, straight or J-shaped) and the target site (right atrium, right ventricle or coronary vein); in comparison, a DOPS scoresheet condenses the entire lead positioning task under the single descriptor ‘Moves and positions pacing electrode(s) appropriately’.
Examples of metrics-based simulation training: porcine tissue model and surgical equipment for lead fixation task (A) and virtual reality simulator for lead implantation (B, lead screw deployment displayed).
The PBP educational approach allows the training to be systematic, reproducible and scientifically grounded. Learners benefit from objective, transparent, event-based and explicitly defined performance guidance and feedback that do not depend on individual faculty techniques or vary with training institution habits. Last but not least, the risk for patients is expected to be lower when treatment is provided by novices who have been pretrained and rigorously assessed in a simulated environment.
Conclusions
A scientific methodology to improve training systematicity and performance consistency at a target proficiency level exists and is applicable to CIEDs across institutions. Previous research in procedure-based medicine has already demonstrated the superiority of simulation-based PBP training in improving operator performance compared with conventional training. It is realistic to believe that a similar result may be generated by a CIED training curriculum based on the pacing/CRT metrics defined through this process.
References
Footnotes
Contributors JM: conceptualisation, validation, investigation, project administration, supervision and writing (original draft); LM, MS, AP and HR: conceptualisation, validation and writing (review and editing); AGG: conceptualisation, validation, investigation, supervision and writing (original draft).
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests AGG, LM, MS, AP and HR received personal fees outside of this work by Medtronic to support intraoperative performance metrics development. Medtronic had no influence in the definition or validation of the metrics. Outside of this work, JM facilitated intraoperative performance metrics development as part of his MSc studies in 'Technology Enhanced Learning for Health' at University College Cork, Ireland, and is also an employee of Medtronic.
Provenance and peer review Not commissioned; externally peer reviewed.