Abstract
Remote monitoring of cardiac implantable electronic devices is a growing standard; yet, remote follow-up and management of alerts represents a time-consuming task for physicians or trained staff. This study evaluates an automatic mechanism based on artificial intelligence tools to filter atrial fibrillation (AF) alerts based on their medical significance.
We evaluated this method on alerts for AF episodes that occurred in 60 pacemaker recipients. AKENATON prototype workflow includes two steps: natural language-processing algorithms abstract the patient health record to a digital version, then a knowledge-based algorithm based on an applied formal ontology allows to calculate the CHA2DS2-VASc score and evaluate the anticoagulation status of the patient. Each alert is then automatically classified by importance from low to critical, by mimicking medical reasoning. Final classification was compared with human expert analysis by two physicians. A total of 1783 alerts about AF episode >5 min in 60 patients were processed. A 1749 of 1783 alerts (98%) were adequately classified and there were no underestimation of alert importance in the remaining 34 misclassified alerts.
This work demonstrates the ability of a pilot system to classify alerts and improves personalized remote monitoring of patients. In particular, our method allows integration of patient medical history with device alert notifications, which is useful both from medical and resource-management perspectives. The system was able to automatically classify the importance of 1783 AF alerts in 60 patients, which resulted in an 84% reduction in notification workload, while preserving patient safety.
We report for the first time an automatic classification method for remote AF alerts.
We found our pilot mechanism to be feasible and safe.
Moreover, it was faster compared with non-assisted physician decisions.
This technology may help to filter non-clinically relevant alerts and highly reduce the workload for remote monitoring of AF alerts.
With the increasing use of medical sensors, such techniques will be essential in the future for timely analysis of data sent by connected health devices.
Introduction
Remote monitoring of patients treated with cardiac implantable electronic devices (CIEDs) has been shown to improve patient follow-up and outcome by enabling early detection of arrhythmias and of technical problems with the devices.1,2 However, although remote follow-up is accepted by international guidelines as a standard alternative to clinical follow-up, adoption in daily practice remains slow.3 Although legal and reimbursement issues may account for part of this inertia, the concern that remote monitoring needs new workflow patterns and leads to an increase in workload is an important and valid issue. Therefore, new organizational models or tools have to be developed. Interesting ways to reduce workload are usually based on alert filtering by trained nurses, as proposed by Ricci et al.4 in the HomeGuide registry, but such workflows still require human staff to operate, and a transition period to take place.
The AKENATON project is a French pilot study designed to test the hypothesis that filtering the importance of alerts could be performed automatically, using algorithms that represent medical knowledge and integrate the patient's clinical condition with device data. Artificial intelligence and automatic reasoning is a field with renewed interest since a few years, with application to text processing and decision support; more specifically web ontology language (OWL2) is a recommendation from world wide web consortium since 2012, which allows both knowledge representation and automatic inference by using appropriate tools.5
The aim of this study was to design a prototype mechanism for atrial fibrillation (AF) alerts (which represent the most frequent notifications in remote monitoring of CIEDs and are expected to lower patient morbidity), and to evaluate the efficacy and safety of this prototype.6
Methods
The design of the AKENATON pilot application was driven by the following scenario: an AF episode detected in a patient with a CIED implant (by means of an automatic mode switch) initiates an alert notification and triggers the automatic classification of this alert. In order to produce a clinically relevant classification of AF alerts, we needed to shift from a strictly device-centred follow-up to a patient-focused perspective, i.e. to take into account the individual clinical condition. Our method of alert classification relies on a knowledge base that includes a formal domain ontology (knowledge base). In order to assist the cardiologist in their decision-making, the system takes into account the patient's condition and extracts relevant data from the patient's records. The prototype was evaluated on a set of 60 eligible patients sourced from a cardiology department.
Patient selection
The research protocol was approved by the ethics evaluation committee of the INSERM (IORG0003254, FWA00005831) institutional review board (IRB00003888) under the reference 13-103. The study complied with the Declaration of Helsinki, and informed consent was obtained from all participants.
We selected 60 consecutive eligible patients from the retrospective database of the cardiology department of Rennes University Hospital. All included patients were adults (>18 years old). Inclusion criteria were (i) implantation with a Sorin REPLY dual-chamber pacemaker between June 2008 and June 2010, and (ii) at least one episode of sustained AF (>30 s) present in the device memory during follow-up.
Because (i) Sorin remote monitoring in CIED was not yet available by the time of the study and (ii) most remote monitoring systems do not allow the notification of an unlimited number of AF episodes for a given patient nor notification of short episodes, we simulated remote monitoring using the following method: the internal memory of each device was obtained, and alerts were generated according to the manufacturer's specifications.
For each patient, the following files were extracted from the hospital information system: (i) medical reports, including clinical reports, radiology reports, and CIED implantation reports; and (ii) laboratory test results, in particular international normalized ratio (INR) results.
Medical history analysis and automatic reasoning
Whenever an AF alert is generated, the AKENATON system performs the extraction of the data needed to analyse the patient's clinical condition (Figure 1), including administrative data (sex and age), narrative patient records for outpatient visit or inpatient stay, laboratory test results, and radiology reports.
AKENATON classification of AF alert based on patient data (details in the text). AF, atrial fibrillation; INR, international normalized ratio.
AKENATON classification of AF alert based on patient data (details in the text). AF, atrial fibrillation; INR, international normalized ratio.
Step 1
Information is extracted both from structured data (e.g. laboratory test results) and from text reports.
Information extraction from patient records was made possible because the hospital information system at Rennes University Hospital comprises a mediator that queries all the hospital data sources and integrates all the data corresponding to the patient.
Personal identifiable information such as patient name is removed from the data, then natural language-processing (NLP) techniques are used to identify key diagnoses and medications from text reports. This step includes the lexical matching of keywords, syntactic and semantic analysis, and detection of negative sentences that are frequent in clinical documents; e.g. the sentence ‘the patient had no sign of heart failure’ would be interpreted correctly as ‘the absence of heart failure’.7 The final data extracted from the patient medical records included: patient's age and sex, medical conditions and medications needed for CHA2DS2-VASc calculation used to assess the thrombo-embolic risk,8 and INR values and dates.
Step 2
Along with patient data, the duration and date of AF episodes are imported into the reasoning module. This module consists of a formal OWL2 as the means to represent the medical knowledge (concepts, relations between concepts) of the domain. Consequently, the system has the following reasoning capabilities. As a result, automatic computation of the CHA2DS2-VASc score and anticoagulation status of the patient is performed, as is the classification of the importance of the AF alert (low, medium, high, or critical).
Automatic reasoning allows computation of the available data in order to infer implicit information from a given patient record. For instance, even if diabetes is not explicitly mentioned in the patient report, a prescription of metformin is used by the system to infer ‘diabetes’ and the diabetes score is correctly added to the overall CHA2DS2-VASc score. Automatic reasoning is also used for representing formal definitions that physicians use daily: e.g. a patient taking warfarin with current INR <2 would not be considered an anticoagulated patient.
An additional benefit is that formal ontology allows representation of data at different levels of granularity, e.g. a patient with ‘lower extremity peripheral arterial disease’ will also match the less specific ‘vascular disease’ criterion, while ‘diabetes type 2’ is recognized as a form of ‘diabetes mellitus’.
Classification of the importance of atrial fibrillation alert
In order to determine the importance of AF notification, an ordinal 4-point scale was created by evaluating the necessity of a medical intervention and its urgency (Table 1). It is worth noting that our objective here is to score the alert importance rather than the thrombo-embolic risk itself. A personalized AF alert is ranked accordingly from low to critical, based on the following criteria:
The patient's thrombo-embolic risk is estimated using the CHA2DS2-VASc score. Three CHA2DS2-VASc classes were distinguished:
○ Score = 0. A score of 0 is usually assumed to reflect low or no risk, and thus such alerts are of little concern to the physician, especially if AF resumes spontaneously.
○ Score = 1. A patient presenting AF and CHA2DS2-VASc scores of ≥1 should usually be treated with continuing oral anticoagulation (OAC) therapy. Therefore, alerts from such patients’ devices should be considered of higher importance unless the patient is already being treated.8
○ Score ≥2. The European Society of Cardiology guidelines recommend that these patients should receive OAC therapy, unless their bleeding risk is very high, in order to lower the risk of thrombo-embolic events.
Duration of AF: the longest duration of AF that can be allowed before a significant increase in the patient's thrombo-embolic risk occurs is subject to debate, and may be as low as 5 min.9,10 Generating a notification for short episodes could be a problem in terms of alert burden, and needs exploration. For this reason, the threshold for alert notification in AKENATON pilot was set at 30 s. All episodes of switch mode during than 30 s were processed into a simulated alert and computed to determine its importance.
Classification algorithm used to determine the importance of AF alerts in a given patient (detail in the text)
| CHA2DS2-VASc (no OAC) | AF duration | |||
|---|---|---|---|---|
| <10 min | 10 min to 6 h | 6–24 h | >24 h | |
| 0 | Low | Low | Low | Medium |
| 1 | Low | Medium | Medium | High |
| 2+ | Low | Medium | High | Critical |
| OAC* active (any CHA2DS2-VASc score) | Low | Low | Low | Low |
| CHA2DS2-VASc (no OAC) | AF duration | |||
|---|---|---|---|---|
| <10 min | 10 min to 6 h | 6–24 h | >24 h | |
| 0 | Low | Low | Low | Medium |
| 1 | Low | Medium | Medium | High |
| 2+ | Low | Medium | High | Critical |
| OAC* active (any CHA2DS2-VASc score) | Low | Low | Low | Low |
*OAC, oral anticoagulation.
We postulated that the minimum duration for a significant AF alert is 10 min (under 10 min, alert is considered of low importance). In contrast, 6 h is the default duration threshold in most remote monitoring systems for reporting AF episodes. In addition, episodes of AF longer than 24 h are considered more important because diagnosed patients may require termination of AF by cardioversion. Therefore, 10 min, 6 h, and 24 h were considered as cut-off values for determining the levels of importance of AF alerts in the current study.
Current anticoagulation status: we defined current OAC status as current treatment with such drugs (based on medical record information), unless the last INR was strictly >2 for patients currently taking a vitamin K antagonist INRs >90 days were discarded. Because many hospitalized patients receive non-OAC regardless of their usual medication, we also disregarded this.
Evaluation
For each patient, the different intermediate results of the automatic computation were compared with those from the human reference. This reference consisted of two blinded physicians including one electrophysiologist that analysed the patient record and device alert data in order to determine the alert importance. CHA2DS2-VASc items, total score, anticoagulation therapy status according to medication and INR, duration of AF, and finally alert importance were compared between system computation and human reference.
Adequate classification of alerts, the primary endpoint, was considered when the alert importance computed by the system was the same as the alert importance assessed by the two human experts.
Statistical analysis
Cohen's κ and weighted κ were calculated in order to measure agreement between the system and the human reference. Underestimated alert importance cases were considered to bear twice the weight of overestimated alert importance cases.
Results
In the selected population of 60 patients, 1783 AF episodes corresponding to continuous mode switch >30 s were detected (mean follow-up 9 months). The number of AF episodes in patients ranged from 1 to 390 (29.7 ± 36.6); 744 episodes (42%) lasted at least 6 h, and 63 episodes (3.5%) lasted at least 24 h. Accordingly, 1783 AF alerts were processed.
Classification of alert importance
We compared the classification of alerts by AKENATON to the human reference. The importance of alerts computed by the AKENATON pilot was very close to that of human reference: 1749 of 1783 alerts (98%) were adequately classified by AKENATON (Figure 2). At the patient level, 58 of the 60 patients had all alerts correctly classified by the system. Results are summarized in Table 2: correct classification is indicated in bold and occurs when the alert importance computed by the system is the same as the human reference. It was the case for 1502 low importance alerts occurring in 49 different patients, 147 medium importance alerts in 22 patients, 88 high importance alerts in 13 patients, and 12 critically important alerts in 5 patients.
Results of AKENATON classification of importance of AF alerts given by AKENATON vs. the human reference
| AKENATON application | ||||
|---|---|---|---|---|
| Low | Medium | High | Critical | |
| Human reference | ||||
| Low | 1502 (49) | 1 (1) | ||
| Medium | 147 (22) | 31 (1) | ||
| High | 88 (13) | 2 (1) | ||
| Critical | 12 (5) | |||
| AKENATON application | ||||
|---|---|---|---|---|
| Low | Medium | High | Critical | |
| Human reference | ||||
| Low | 1502 (49) | 1 (1) | ||
| Medium | 147 (22) | 31 (1) | ||
| High | 88 (13) | 2 (1) | ||
| Critical | 12 (5) | |||
Correct classifications are shown in bold. The number of alerts is given (number of patients within brackets; detail in the text).
Distribution of AF alert importance classified by human (manual) vs. AKENATON (automatic).
Distribution of AF alert importance classified by human (manual) vs. AKENATON (automatic).
There were 34 misclassified alerts in 2 patients: 33 alerts in a single patient had their importance overestimated due to an error in the CHA2DS2-VASc calculation (score of 2 instead of 1), and 1 alert in another patient was overestimated because the OAC therapy, this patient was receiving, was missed by the language analysis tool (OAC mentioned without the treatment name).
Cohen's κ value between the system and the human reference was 0.93. Owing to the absence of cases with underestimated alert importance, weighted κ was the same.
CHA2DS2-VASc and anticoagulation status computation
In general, the CHA2DS2-VASc score computed by the system was very close to the human reference results. For the score items, 480 values were estimated by the system (8 criteria for 60 patients), of which 466 were correct (Figure 3). CHA2DS2-VASc score was the same for 46 of 60 patients with both the fully automatic and the human methods, while there was a single error on a CHA2DS2-VASc criterion for 14 patients (9 false-positive and 5 false-negative), which had little impact on the medical interpretation. CHA2DS2-VASc score classification (0, 1, or ≥2 as used in clinical practice) was correct for 58 patients (97%).
Evaluation of CHA2DS2-VASc items, total score, and anticoagulation status inferred by AKENATON vs. reference.
Evaluation of CHA2DS2-VASc items, total score, and anticoagulation status inferred by AKENATON vs. reference.
Anticoagulation was adequately evaluated in 57 patients (95%). The three patients with errors were two patients with interrupted vitamin K antagonist therapy, and one patient whose medical records mentioned OAC without detailing the drug, which was missed by the NLP technique.
Of the 17 errors found, 16 were due to errors in the first step (language processing), and it is very unlikely that these would have occurred if an electronic medical record with structured data had been available.
These few errors had little impact on medical interpretation in terms of alert importance. As a result, 1749 of 1783 alerts (98%) were adequately classified and there were no underestimations.
After anonymization, an average of 27 medical documents per patient were analysed in 52 s, and automatic reasoning took an additional 2 s (mean value). These computation times are compatible with a real-time computation triggered by alert notification.
Discussion
We have developed a prototype for remote monitoring and management of AF alerts. In this study, we aimed to assess the safety (specifically, its ability to detect which patients are most at risk of thrombo-embolic events) and efficacy (reduction in alerts and/or their classification as important or not) of the system. Both criteria are key features for encouraging acceptance of such a system by healthcare providers in the future.
Patient safety
To monitor a patient remotely, the data transmitted by the devices must be regarded as part of the patient's medical records, and must be combined with the clinical data from electronic medical record (EMRs) to guide medical decision-making in real time. We found that the automatic extraction of information from EMRs and the classification of alerts were feasible and accurate, with 98% of the 1783 alerts classified correctly. In two cases, the automatic classification of the alerts was associated with an overestimation of the CHA2DS2-VASc score. We did not find any case of underestimation of alert importance. Such results are compatible with optimal levels of patient safety. This study illustrates a novel approach that is likely to become needed much more in the future with the development of remote monitoring and the widespread adoption of EMRs.
Efficacy to reduce the notification workload
Of 1783 AF episodes, 744 lasted at least 6 h, and would probably have been reported by current remote monitoring systems (providing AF alerts are not switched off). Using the AKENATON system, only 133 of these 744 alerts would have been deemed useful and reported, thus achieving a potential reduction in AF alert quantity of 82%, while preserving 100% of the relevant notifications.
It has been suggested that physicians may need to monitor even short (5 min and more) episodes of AF in patients. Despite the fact that our automatic system evaluated all AF episodes of more than 30 s, it still produced a reduction in the number of alerts from 1783 potential alerts to 280 relevant alerts (84% reduction); the total number of alert notifications would be a third of what would be reported with current remote monitoring systems’ settings.
Atrial arrhythmias are estimated to represent 60–70% of all pacemaker alerts.11–13 Our findings suggest that the workload related to the management of pacemaker remote monitoring for AF events could only be greatly reduced by such an automatic system, without jeopardizing the clinical benefit of such monitoring. Although others have reported that trained nurses are able to reduce alerts reported to physicians by an average of 85%, we believe our automatic method could be considered as a replacement or added first line management of data.12,14 Furthermore, it could provide key elements of a patient's medical records to the nurse or physician in a timely manner.
We anticipate that such technology will be applicable to other types of alerts and devices such as ventricular arrhythmias in patients with implantable cardioverter-defibrillators (ICDs), and that it would be of benefit to electrophysiologists if they were made widely available for daily practice.
Interoperability between remote monitoring systems and EMRs
Decision support systems such as AKENATON require device and patient data together in order to work. Device data have to be obtained from a sustainable workflow, using a common data representation and convergent pipelines. Initiatives such as IDCO (Implantable Device Cardiac Observation Profile), a standard for representing data from CIEDs, and iCardea, a European project for sharing remote monitoring and patient data together across multiple manufacturers, are milestones in the process of addressing this need.15,16 Moreover, our study is a proof of concept demonstrating that new knowledge-based methods may assist physicians in interpreting real-time personalized data for the benefit of patients.
Sensors and other monitoring schemes
We support the notion that CIEDs represent the first step in using implanted and communicating devices for monitoring patients and therapy. It is expected that the integration of sensors and wireless technologies will transform medical practice in the near future. The huge amount of data generated by such monitoring devices shall not be analysed by human-based methods. We expect that approaches based on data integration and automated reasoning, such as the method presented in this paper, will be important factors in turning this data into action for the benefit of individual patients, or for mining this data in order to discover new medical knowledge.
Study limitations
In our study, AF detection was diagnosed by using a device mode switch. Mode switch criteria are subject to variation in different manufacturers’ devices and models. In addition, a mode switch may be erroneously activated by atrial lead noise (external interference, myopotential for unipolar leads, lead fracture) despite the absence of supraventricular arrhythmia. More importantly, a long AF episode may be detected as multiple shorter AF episodes; this is not really relevant to this study as it is already an issue in current alerts. Nevertheless, despite these well-recognized facts, the vast majority of mode switches still occurs appropriately, and transmission of the episodes recorded electrograms and simple algorithms that concatenate subsequent episodes would allow the recognition of these cases. Electrogram automatic interpretation was not implemented in AKENATON pilot.
In addition, we analysed simulated alert from real pacemakers data, whereas most remote monitoring is currently performed for patients with ICDs. However, we believe that AF detection results are generalizable to ICDs, and we plan to extend this proof of concept to all cardiac rhythm-management devices.
This study demonstrates a pilot system that could be of benefit for remote monitoring. Such system implementation is of course highly dependent on the infrastructure context and local constraints on the availability of patient record. Moreover, the availability of patient information relies on (i) local patient loyalty to the structure during follow-up and (ii) the ability to obtain updated clinical data. Outpatient clinical status is subject to modification that would not be acknowledged in the medical record, such as OAC suspension, new co-morbidity, ambulatory INR results, etc. The ability to obtain such data is the challenge of EMR, and goes beyond the scope of this study. In our experience, many teams uses current data from their medical records and would therefore face the same limitation as an automated system.
Finally, this was a retrospective study, and a multicentre prospective study, including health-cost measurements and clinical safety assessments, would confirm the benefit of such a system as well as its scalability beyond the prototype implementation.
Conclusion
This work demonstrates the ability of a pilot system to classify alerts and improve personalized remote monitoring of patients. In particular, our method allows integration of patient medical history with device alert notifications, which is useful both from medical and resource-management perspectives. The system was able to automatically classify the importance of 1783 AF alerts in 60 patients, which resulted in an 84% reduction in notification workload, while preserving patient safety. This method could be extremely helpful for remote monitoring of CIEDs and could be extended to other applications in the field of medical sensors. We expect to achieve further improvement and validation with the increasing use of electronic health records.
Funding
This work was supported by the Agence Nationale Pour la Recherche – Technologies pour la Santé [ANR-07-TecSan-001].
Conflict of interest: A.R. has received small consultancy fees from Medtronic, St. Jude Medical, and Sorin Group, and research grants from Medtronic, Inc. and Boston Scientific Corp. Ph.M. has received speaker honoraria and consultancy fees and research grants from St. Jude Medical, Sorin Group, Medtronic, Inc., Boston Scientific Corp., and Biotronik. L.L and C.H are employees of Sorin CRM SAS.
Acknowledgements
We thank the AKENATON project members (ALICANTE SA, CERIM, CIC-IT, INSERM U936, LIMSI, and Sorin CRM SAS) for their commitment. We also thank N. Roederer (Inovelan Co.) for his technical assistance.
