Article Text

Download PDFPDF

Original research
Personalised preinterventional risk stratification of mortality, length of stay and hospitalisation costs in transcatheter aortic valve implantation using a machine learning algorithm: a pilot trial
  1. Maria Zisiopoulou1,2,
  2. Alexander Berkowitsch1,
  3. Leonard Redlich1,
  4. Thomas Walther2,3,
  5. Stephan Fichtlscherer1 and
  6. David M Leistner1,2
  1. 1Cardiology and Vascular Medicine Department, Universitätsklinikum Frankfurt, Frankfurt am Main, Germany
  2. 2German Center of Cardiovascular Research (DZHK), Partner Site Rhine/Main, Frankfurt am Main, Germany
  3. 3Department of Cardiothoracic Surgery, Universitätsklinikum Frankfurt, Frankfurt am Main, Germany
  1. Correspondence to Dr Maria Zisiopoulou; Zisiopoulou{at}em.uni-frankfurt.de

Abstract

Introduction Risk stratification based on Euroscore II (ESII) is used in some centres to assist decisions to perform transcatheter aortic valve implant (TAVI) procedures. ESII is a generic, non-TAVI-specific metric, and its performance fades for mortality at follow-up longer than 30 days. We investigated if a TAVI-specific predictive model could achieve improved predictive preinterventional accuracy of 1-year mortality compared with ESII.

Patients and methods In this prospective pilot study, 284 participants with severe symptomatic aortic valve stenosis who underwent TAVI were enrolled. Standard clinical metrics (American Society of Anesthesiology (ASA), New York Heart Association and ESII) and patient-reported outcome measures (EuroQol-5 Dimension-Visual Analogue Scale, Kansas City Cardiomyopathy Questionnaire and Clinical Frailty Scale (CFS)) were assessed 1 day before TAVI. Using these data, we tested predictive models (logistic regression and decision tree algorithm (DTA)) with 1-year mortality as the dependent variable.

Results Logistic regression yielded the best prediction, with ASA and CFS as the strongest predictors of 1-year mortality. Our logistic regression model score showed significantly better prediction accuracy than ESII (area under the curve=0.659 vs 0.800; p=0.002). By translating our results to a DTA, cut-off score values regarding 1-year mortality risk emerged for low, intermediate and high risk. Treatment costs and length of stay (LoS) significantly increased in high-risk patients.

Conclusions and significance A novel TAVI-specific model predicts 1-year mortality, LoS and costs after TAVI using simple, established, transparent and inexpensive metrics before implantation. Based on this preliminary evidence, TAVI team members and patients can make informed decisions based on a few key metrics. Validation of this score in larger patient cohorts is needed.

  • Delivery of Health Care
  • Outcome Assessment, Health Care
  • Aortic Valve Stenosis

Data availability statement

Data are available upon reasonable request.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

  • Euroscore II (ESII) is currently used by clinicians to assist their decision to perform transcatheter aortic valve implant (TAVI) procedures

  • ESII is a generic, non-TAVI-specific metric, and its predictive performance fades for mortality at follow-up longer than 30 days.

WHAT THIS STUDY ADDS

  • A novel TAVI-specific logistic regression model predicts 1-year mortality better than ESII.

  • In this model, the American Society of Anesthesiology and Clinical Frailty Scale scores were much stronger predictors of 1-year mortality than ESII.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

  • Based on this TAVI-specific model, 1-year mortality, length of hospital stay and treatment costs may be predicted before TAVI procedure.

Introduction

Preprocedural prediction of outcomes after transcatheter aortic valve implant (TAVI) including survival probability, expected hospital stay and treatment costs is important for optimal treatment strategy and managerial decisions. Usually, this issue is being addressed with Euroscore1 or Society of Thoracic Surgeons (STS) risk score2 algorithms, which are based on univariate and multivariate Cox-regression analyses and receiver operating characteristic (ROC) curves.3 Patients’ subjective viewpoint and awareness may add a further perspective with information that may impact clinical decision-making.4 Patient-reported outcome measures (PROMs) capture some aspects of health and disease that are important and relevant for the patients themselves.5 Acquisition of PROM data is a rather low-cost and low-tech procedure compared with more technical, high-tech and high-cost procedures such as echocardiography and clinical chemistry markers that require a significant investment in infrastructure. As a result, trials with a high level of evidence using PROMs as metrics/parameters for the prediction of significant endpoints in TAVI candidates (eg, mortality) are needed. There are few prospective studies using PROM data for the prediction of key clinical endpoints, such as mortality, length of stay (LoS) and hospitalisation costs.6 Among PROMs, the Kansas City Cardiomyopathy Questionnaire (KCCQ), although originally developed to monitor heart failure outcomes, has been shown to provide valid information regarding TAVI outcomes.6–10

Additionally, prospectively acquired data from TAVI patients using the patient-reported EuroQol-5 Dimension-5 Levels (EQ5D5L) questionnaire values have shown great potential to be a reliable predictor of TAVI outcomes.6 Also, the easy-to-use Clinical Frailty Scale (CFS) scores have been used in various acute conditions,11 including TAVI procedures,10 12 as strong predictors of patient outcomes.

Many studies using statistical and machine learning models have been performed to predict outcomes after cardiological interventions,13 14 including TAVI.10 Among the tested techniques, decision tree algorithm (DTA) analysis has the great advantage of providing a very transparent use of the relevant metrics (eg, PROM values) and the use of quite straightforward cut-off values, which are very easy to capture and to understand from health professionals who are non-proficient in statistics.

Currently, Euroscore II (ESII) is used in some centres, including ours, to assist clinicians in the decision to perform a TAVI procedure, based on risk stratification. Nonetheless, ESII is a rather generic metric, which was not specifically developed and validated for TAVI patients. ESII is known to be non-linearly associated with long-term mortality, with progressive worsening of the model’s performances when applied to follow-up beyond 30 days. Therefore, our aim was to develop a predictive model that would show improved predictive accuracy regarding mortality specifically for TAVI patients compared with conventional ESII. Additionally, we hypothesised that such an optimised model would allow the evaluation of the length of hospital stay (LoS) and respective hospital treatment costs for TAVI patients.

Methods

A total of 284 consecutive patients with symptomatic severe aortic valve stenosis who received a TAVI implant according to the standard clinical routine care procedures between 1 March 2019 and 31 December 2021 at the cardiology department of a tertiary university medical centre and agreed to participate in this prospective study were enrolled. A total of 558 patients have been treated in our department during the aforementioned period, and 50.89% of them participated in this study. During the COVID-19 pandemic, inperson contact should be kept to a minimum. Due to this disruption, prospective recruitment was reduced significantly.

One-year mortality was assessed by means of telephone interviews with the patients or their relatives and double-checked by the use of the clinical information system, in which all death events were being registered. All aspects of this analysis were prospective. It should be mentioned that the current reimbursement policy in our country supports a minimal LoHS of equal to or more than 4 days after TAVI procedure, in order to prevent any reduction in hospital reimbursement. As a result, patients are discharged in a standard fashion on the 4th postoperative day, unless any complications occur. Regarding LoS measurement, in case of multiple readmissions within 30 days after the day of TAVI procedure, the additional length of hospitalisation(s) was added to the initial one, and hence, a cumulative LoS was used for analysis.

Inclusion criteria

Only patients suffering from a severe symptomatic degenerative aortic valve stenosis with an effective orifice area <1.0 cm2 or a mean gradient >40 mm Hg, according to the current guidelines who were scheduled for TAVI procedure, were included15 ; furthermore, every patient with TAVI procedure was discussed by the Heart Team of the Heart Center of our university medical centre based on personalised assessment of the relevant data of any individual patient according to the current guidelines.15

Exclusion criteria

Patients with acute non-compensated cardiogenic shock or haemodynamic instability in need of inotropic support, severe neurological disorders, dementia or inability to provide informed consent were excluded. Additionally, patients with an aortic prosthetic heart valve requiring a valve-in-valve procedure were excluded.

All trial procedures were in accordance with the Declaration of Helsinki for studies in humans. Written informed consent to use their PROM as well as their standard clinical care data for the purposes of this study was given by all participants. The trial was approved by the Institutional Review Board (Ethics Committee of the Faculty of Medicine, Nr. 296/16).

Collected data

The TAVI scorecard database

To standardise data acquisition, a TAVI scorecard has been implemented as described previously in the literature.6 10 16 On this scorecard, at baseline, that is, 1 day before TAVI procedure, ESII, American Society of Anesthesiology (ASA) scale, New York Heart Association (NYHA) scale as well as EuroQol-5 Dimension-Visual Analogue Scale, KCCQ and CFS were collected. The occurrence of any postprocedural complication was also documented in the scorecard.

Our TAVI scorecard database is based on several features of a management system previously coined by Kaplan and Norton.16 Nonetheless, our specific and dedicated TAVI scorecard is tailor-made for the practice of cardiology and measures key performance indicators from four perspectives: the clinical cardiologist’s perspective in the ‘quality’ domain, the cardiology department staff’s perspective in the ‘process’ domain, the patients’ perspective in the ‘outcome’ domain and the perspective of the hospital’s management in the ‘cost’ domain.

Data that have been assessed in our TAVI scorecard data bank are depicted in table 1. Euroscore includes nearly all comorbidities, and our aim was to predict the outcome based on a few key parameters/variables; we included only ESII, ASA scale, NYHA scale and a few demographic parameters (table 1).

Table 1

Baseline and outcome data

Identification of the best model based on the minimum average log-likelihood (table 2).

Table 2

Depicted is the comparison of the 1-year mortality preinterventional TAVI predictive performance among the tested algorithms (ie, logistic regression, TreeNet/decision tree algorithm and CART, Classification And Regression Tree)

As shown in table 2, logistic regression with linear and squared terms was identified to be the best model for predictability of TAVI outcome. Calculation of logistic regression model (LRM)-TAVI score with a dedicated name ‘ida-TAVId-Score’ (intelligent data-driven TAVI decisions) is presented in formulas 1 and 2:

Embedded Image(1)

where Y is calculated according to the formula:

Embedded Image(2)

where Ck is coefficient and Xk is term.

The term ‘R’ can be interpreted as the probability of reaching the endpoint. Of note, this formula is a result of a numeric experiment included in an algorithm of machine learning implemented in Minitab (LLC, State College, Pennsylvania, USA).

In brief, the algorithm tests all possible combinations of the included variables in both linear and squared form and in this way identifies the model(s) with the minimal log-likelihood.

The ROC curve for calculated R depicted in figure 1 clearly shows a quite high value of area under the curve (AUC), namely, 0.800.

Figure 1

Classification and Regression Tree, based on preinterventional risk stratification of 1-year mortality after transcatheter aortic valve implant (TAVI), based on logistic regression model-TAVI score cut-off values.

The predictors on which this LRM-TAVI (ida-TAVId) score is based are presented on table 3.

Table 3

Depicted are the independent variables that emerged as the most important predictors for 1-year mortality and their respective coefficient

The next algorithm we tested in this study is CART (Classification And Regression Tree), also implemented in the software package Minitab.17 This algorithm can be termed ‘decision tree’ as well. CART identifies subgroups in the cohort according to risk grade or expected continuous value; it was applied for the prediction of mortality and estimation of the LoS and treatment costs.

It should be mentioned that for the analysis of LoS and costs, the 12 patients who died within 30 days after the procedure were excluded.

Data are reported in n (%) or median (IQR), where appropriate. Data were analysed using Minitab or SPSS V.27 (IBM, Armonk, New York, USA), where appropriate. Regarding missing values, we used imputation algorithm methodology implemented in SPSS.

Results

1-year mortality

A total of 30 (10.6%) patients died within 12 months after the TAVI procedure. In our patient population, we found logistic regression to be the best predictor for mortality (figure 2, table 2).

Figure 2

Comparison of ROC curves calculated for score based on our logistic regression model and Euroscore II (ESII). The superiority of our transcatheter aortic valve implant (TAVI)-specific developed score (red ROC curve) compared with ESII (blue ROC curve) regarding the prediction of 1-year mortality after TAVI procedure can be appreciated ROC, receiver operating characteristic.

Figure 1 shows the decision tree for the prediction of 1-year mortality. By translating our results to DTA, clinically useful cut-off values regarding 1-year mortality risk stratification emerged: <0.02 points for low, 0.02–0.15 for intermediate and >0.15 for high risk. Treatment costs and LoS were significantly increased in high-risk participants compared with low-risk or intermediate-risk participants. Results are presented in figure 1.

This example of DTA demonstrates the identification of patient groups with low, moderate/intermediate and high risk of 1-year mortality. This analysis was performed with few standard baseline parameters and a single simple, barely time-consuming PROM, namely, CFS.

Length of hospital stay

As shown in table 1, the median value of the LoS in our TAVI patient cohort was 8 days.

Based on risk categories according to the LRM-TAVI score, LoS was depicted in box plots (figure 3).

Figure 3

Box plots depicting the distribution of length of stay (LoS)—values in transcatheter aortic valve implant participants according to their respective risk category (A). No significant difference was found between low-risk and intermediate-risk groups (B). However, LoS was quite significantly higher in high-risk patients (C).

Treatment costs

Based on risk categories according to the LRM-TAVI score, treatment costs (in Euro, €) were calculated. Results for each risk group (low, intermediate and high) are presented as box plots in figure 4.

Figure 4

Box plots of treatment costs in Euro (€) according to risk stratification group based on LRM-TAVI score (A). Significant differences were found among all three risk groups (B, C). LRM-TAVI, logistic regression model-transcatheter aortic valve implant.

Discussion

In the present study, we found that using simple metrics such as the standard ESII, ASA and NYHA scales and a single PROM (namely, CFS) as well as a standard clinical metric (namely, systolic pulmonary artery pressure) assessed preoperatively could quite strongly predict postinterventional 1-year mortality, LoS (LoS) and hospitalisation costs after TAVI. Using the clinically plausible and transparent cut-off values of this novel ida-TAVId-Score that can be used in an everyday practice setting may provide a simple means of calculation of personalised risk scores for 1-year mortality, LoS and hospitalisation costs for each individual TAVI patient.

This method, based on standard clinical data, gives clinicians a tool to provide personalised information to TAVI candidates before implantation. Based on this TAVI-specific model, patients (and their relatives) could be better informed than using the generic ESII score before the intervention regarding the risk of mortality. As a result, informed consent for TAVI patients and their families may be further individualised. Due to a better prediction of LoS, additional measures (eg, post TAVI rehabilitation) may be better prepared and planned according to the individual needs of each patient, possibly even before performing the TAVI procedure. Moreover, by self-reflecting based on the CFS questionnaires and by analysing their own conditions, TAVI patient participation and empowerment are promoted. By providing patients (and their relatives) as well as clinicians with such a risk score regarding mortality, this could potentially influence their decisions and possibly lead to a refusal of TAVI in some cases.

A major strength of this clinical trial is its prospective nature. The trial was conducted in a single centre. Therefore, there was no variability in the standard of care that could potentially confound the results. There are few prospective studies specifically designed to address the usefulness of preoperative PROMs in the prediction of mortality, LoS or cost after TAVI.6 10 Further, a significant feature of this trial is that the used metrics are quite simple and easy to capture and use in everyday clinical practice: namely, NYHA, ASA, ESII and CFS. Many centres, especially in Europe, have included the calculation of ESII in their daily routine as a standard operating procedure. CFS may be integrated into routine diagnostic workup or may be simply implemented as an additional workup with minimal additional administrative burden.

A major reason for the observed superior performance of our model compared with ESII (figure 2) may be that ESII was not specifically designed as a predictive instrument in TAVI patients. Nonetheless, a major issue to consider is that the ESII was developed to predict in-hospital mortality after cardiac surgery, and studies have shown that it cannot be considered a direct estimator of long-term risk of death as its performance fades for mortality at follow-up longer than 30 days. Hence, a major reason for the observed superior performance of our model compared with ESII may not be just because ESII was not designed as a predictive instrument in TAVI patients but because the ESII score was applied to determining the primary outcome of 1-year mortality. In addition, it turned out that even more generic instruments than the ESII, namely, ASA and NYHA scores, are quite important in predicting 1-year mortality. They are even more important than ESII in predicting 1-year mortality within the context of our model.

Other authors found similarly that NYHA IV independently predicted 3-month all-cause and cardiovascular mortality.18 In a further predictive model, baseline NYHA class could predict 1-year mortality among other outcomes.19

We could not find any reports in which the ASA classification was a significant predictor for 1-year mortality after TAVI. Previous reports have associated baseline pulmonary hypertension with higher 1-year cardiovascular mortality after TAVI.20 21

Using other machine learning methods, other authors reported a precise prediction of intrahospital mortality (AUC 0.94–0.97) after TAVI.22 By means of logistic regression, random forest and CatBoost models, AUC of the ROC for the prediction of 1-year mortality after TAVI of 0.65, 0.67 and 0.65 for the internal validation and of 0.62, 0.66 and 0.68 for the external validation, respectively, have been achieved based on the data of 1931 patients from two different centres.23 Other authors have reported that decision tree-generated predictive models for 30-day and 1-year mortality provided the most precise accuracy of 0.97 and 0.90 with the AUC-ROC curves of 0.83 and 0.71, respectively, on 30-day and 1-year mortality based on testing data from a total of 186 patients in a retrospective cohort study.24

Regarding the utility of CFS, other authors have also stressed the relevance of frailty as a significant predictor of TAVI outcomes. Albumin was the most commonly used single-dimension frailty measure, and the Fried or modified Fried phenotype was the most commonly used multidimensional frailty measure.25 Usually, the presence of three out of the following five criteria is used as a definition of frailty in clinical practice: grip strength as defined by a dedicated algorithm, 15-foot walking tests, a body mass index <20 kg/m2, Katz activity of daily living ≤4/6 and serum albumin <3.5 g/dL.12 Evidence suggests that the assessment of frailty may further improve patient outcomes after TAVI, and therefore, this measure should be an integral part of the preinterventional evaluation of TAVI patients.26

Practice-oriented simplified physician-estimated frailty measures may be a useful aid to TAVI risk stratification in everyday clinical routine until more objective frailty scales can find their way into the clinical routine.27 Many authors have used various frailty metrics at baseline as predictors of 12-month mortality, such as the 5 min walk test and hand grip strength, in accordance with Valve Academic Research Consortium 328 recommendations, but also other frailty scales, such as the Katz index, the Canadian Study of Health and Aging scale, the Elderly Mobility Scale or the Identification of Seniors at Risk scale.29

In a scoping review,26 mortality was the most common outcome examined, with CFS being predictive 87% of the time. Objective early identification of older adults with frailty using the CFS in acute care units may help target interventions to prevent complications and implement effective discharge planning.11 In addition to reflecting the degree of frailty, the CFS is a useful marker for predicting late mortality in elderly transcatheter aortic valve replacement cohorts.12

This is the first prospective study providing evidence for the usefulness of CFS as a predictor of hospitalisation costs after TAVI. The CFS has been used in a composite DTA to predict LoS after TAVI.10 Establishing standardised frailty measurements as a standard operating procedure in everyday healthcare practice to promote reporting consistency was highlighted in a former meta-analysis.25

The combination of KCCQ and estimated glomerular filtration rate (eGFR) can better predict midterm mortality than ESII alone.6 Additionally, the combination of eGFR, N-terminal pro brain natriuretic peptide (NTproBNP) and EQ5D5L can reliably predict LoS after TAVI.

In our cohort, we have been able to include only 35 participants with LoS ≤4 days after TAVI procedure. Data from more participants with a short (≤4 days) LoS are needed to see if this model would also be applicable in currently emerging, ‘fast track’ TAVI programmes.

The median duration of stay in our cohort was 8 days. This was due to the fact that many participants had a number of comorbid conditions, and they cannot be directly discharged at home; patients remain hospitalised and are directly discharged to a rehabilitation centre. Also, LoS was influenced by a readmission (see Methods section). Our model may help to more efficiently and timely organise the immediate rehabilitation after TAVI procedure by predicting LoS.

Regarding the prediction of acute hospitalisation costs during the TAVI procedure, to the best of our knowledge, no prospective targeted trials have been performed. However, retrospective data from frailty instruments reported in an administrative database have been used to develop statistical models predicting hospitalisation costs for patients undergoing TAVI procedures.25 Other studies have used postinterventional metrics to predict LoS and hospitalisation costs.30 We have used prospectively only one baseline PROM metric, namely, CFS, to predict hospitalisation costs during TAVI.

From a clinical viewpoint, DTA offers the opportunity of implementation in everyday practice, due to the use of cut-off values which are often easy to memorise, straightforward and transparent. Our approach sets the ground for standardisation and therefore comparison between different clinicians and centres.

Regarding limitations, given that this was a pilot trial and was done in a single medical centre, it should be considered a hypothesis-generating trial, and these findings should be interpreted with caution. Because of the pilot nature of the project, mortality prediction was trained on only 284 patients with only 30 mortality events. Therefore, the use of an internal validation data set was statistically meaningless. This ida-TAVId model is going to benefit from further training with a larger amount of high-quality structured (especially prospective) TAVI patient data, as the ones provided here.

Of note, our patients have been implanted with devices originating from different manufacturers (namely, Abbott, Boston Scientific, Edwards and Medtronic). The specific implant type could have had a confounding effect on the results. This topic should be further analysed.

Our model has been designed to predict outcomes (mortality, LoS and costs) based only on data existing before the TAVI intervention. As a result, procedural outcomes (eg, aortic dissection and stroke) have not been included as independent variables. Also, preinterventional CT scan features that may be associated with TAVI outcomes have not been considered in our prospective study design at all. CT scan features may be potential independent variables for future TAVI-related predictive models.

Another limitation is that PROMs can be multidimensional and can be influenced by several items beyond the disease entity. They can be influenced by participant’s age, culture and comorbidities, among others. Therefore, our model needs to be tested in other hospital contexts and geographical areas.

On the basis of this evidence, all TAVI team members, TAVI patients and their relatives can rely on just a few simple key metrics for final decision-making before the procedure. This could be done by means of implementing the ida-TAVId-Score in a form of an app-based solution. Therefore, the burden of resources (time, personnel and costs) needed for the process of meaningful data collection in order to measure meaningful changes in quality and safety within the framework of improvement projects and protocols involving TAVI patients is significantly reduced.

To test the hypotheses generated on the basis of the present data, we suggest a larger-scale multicentric validation in a larger cohort of participants in the framework of a further observation prospective study with the same design as applied in the present report. In addition, we are planning a further monocentric prospective randomised controlled study in our heart centre comparing medical management decisions in TAVI patients with and without the decision support provided by the model presented in this report.

Data availability statement

Data are available upon reasonable request.

Ethics statements

Patient consent for publication

Ethics approval

This study involves human participants and was approved by the Ethics Committee of the Faculty of Medicine (Nr. 296/16). Participants gave informed consent to participate in the study before taking part.

References

Footnotes

  • Contributors MZ had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: MZ and DML. Acquisition, analysis or interpretation of data and critical revision of the manuscript for important intellectual content: all authors. Drafting of the manuscript: MZ and AB. Statistical analysis: AB. Study supervision: MZ, SF and DML. MZ is the guarantor and accepts full responsibility for the work and/or conduct of the study, had access to the data, and controlled the decision to publish.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.