Discussion
The safety and efficiency of the Dutch NTS decision support tool were found to be poor for telephone triage at OHS-PCs in patients with acute chest discomfort, with a sensitivity of 0.73 and specificity of 0.43 for high urgency allocation of patients who had an ACS/other LTEs. Safety improved after intervention by the triage nurse and the final high urgency level had a sensitivity of 0.86 and specificity of 0.34.20 Women and men with ACS/other LTEs received equally high urgencies.
At a prevalence of ACS/other LTEs of 14.1% (11.2% and 2.9%, respectively), the positive predictive value for both the NTS and final urgency was low (0.17 and 0.18, respectively) and the negative predictive value was high (0.91 and 0.94, respectively).
Clearly, the triage system is calibrated on the safe side. Most GPs highlight over triage, but in our opinion, in the critical domain of patients with chest discomfort, under triage should receive more attention.5 19 21 It is evident that patients with an ACS and those classified with other LTEs should receive a high urgency allocation, but the NTS classified 27% of the ACS/LTE patients as low urgency, and this was still rather high for the final urgency (14%).
We were able to analyse the original and very first conversation with patients, and these conversations were assessed without knowledge of the diagnosis; the assessment of symptoms was not affected by recall or hindsight bias. Moreover, we were able to include a large sample of patients and missing outcome data did not lead to patient or call selection. We therefore are confident our results are generalizable to Dutch settings and also to similar OHS-PC settings in, for example, the UK and Scandinavian countries.1 Our results may also be generalizable to EMS settings, because the chance of having an ACS among those calling for chest discomfort is similar in the EMS setting and the OHS-PC setting.13 22
A limitation of the study is that, in 15% of calls, the overruled urgency was unclear. In these cases, an expert panel (blinded to the clinical outcome) decided on all the available information to be heard in the calls.
We could not find studies evaluating the accuracy of telephone triage in the OHS-PC among patients with chest discomfort against clinical outcomes.
A systematic review of six studies evaluating the MTS in the ED setting reported that the MTS had allocated a high urgency level in patients with ACS with a sensitivity of 0.70–0.80.23 One of these six studies also reported specificity (0.59) among patients with chest pain referred to a Portuguese ED.24 Thus, the MTS, with the advantage of face-to-face contact, seems to have a similar sensitivity and somewhat higher specificity to that in our study. Interestingly, the prevalence of ACS among the 233 patients with chest pain in the Portuguese study was lower than in our study (9.4% vs 11.2%).24 None of the studies reported positive and negative predictive values, which prevented us from comparing these to ours. The prior risk of ACS and LTEs of 14.1% changed only marginally to 17% and 18% for the NTS and final high urgency, respectively, and to 9% and 6% for the NTS and final low urgency allocation.
To the best of out knowledge, there are no accuracy studies performed in the EMS setting in which urgency allocation with EMS (112 or 911) telephone triage was compared with the presence or absence of ACS. In an observational Swedish study among 14 454 patients calling EMS for chest discomfort, 63.5% received an urgency 1 ambulance and 11.5% turned out to have an ACS (women 9% and men 14%).22 An EMS study in the USA with 3007 calls from patients with acute chest pain to the Medical Priority Dispatch System (MPDS) for decision support reported a prevalence of acute myocardial infarction of 8.7%.25
In a primary care study, the GP’s clinical risk estimate was more accurate in predicting ACS than a prediction rule (c-statistic 0.75 vs 0.66) in patients with acute chest discomfort.20 This is in line with our finding that the sensitivity of the final urgency level (including overruled cases) was higher than that of the NTS level of urgency alone (0.86 vs 0.73).
Previous studies have found that the risk of ACS in women is underestimated, a finding in contrast to our result in which women and men with ACS received similarly high urgency allocations.26 27 One of the former mentioned EMS studies also found that women and men received similarly high urgency allocations and the other EMS study did not report sex-stratified analyses.22 25
Missing an ACS is the most common reason for malpractice claims worldwide.9 28 There are no generally accepted guidelines that define what rate of missed ACS is considered acceptable, and this threshold will likely differ among different healthcare providers. A survey performed among 1029 ED doctors in the USA, New Zealand and Australia showed that they considered on average a rate of 0.1%–1% (range 0%–10%) as acceptable.29 We do not have such information from GPs or lay people, but we suspect that this will be similar in the Netherlands. Alarmingly, this number is not in line with our finding that 27% of patients with ACS/other LTEs received a NTS low urgency allocation and a 14% final low urgency allocation. Indeed, a low urgency allocation does not necessarily mean the ACS is being missed, but that an undesirable risk to patient safety has been taken with the hazard of life-threatening ventricular arrhythmias and sudden death due to severe pumping failure of the heart in those with acute myocardial infarction.30 31 On the other hand, triage nurses and their supervising GPs increased triage safety by overruling the decision support system adequately, although with significantly increasing over triage. Apparently, the ‘human factor’ is conducive to safety in the current telephone triage process.18 32 Nonetheless, potential room for improvement should be studied, both of the decision support tool itself by developing better diagnostic prediction rules as well as by improving its use by triage nurses and management.33