Perioperative Deaths After Mitral Valve Operations May Be Overestimated by Contemporary Risk Models

doi:10.1016/j.athoracsur.2014.05.011

The Annals of Thoracic Surgery

Volume 98, Issue 2, August 2014, Pages 605-610

Presented at the Fiftieth Annual Meeting of The Society of Thoracic Surgeons, Orlando, FL, Jan 25–29, 2014.

https://doi.org/10.1016/j.athoracsur.2014.05.011 Get rights and content

Background

Percutaneous therapies to manage mitral regurgitation are emerging as an alternative to conventional operations, especially for patients with a high estimated perioperative risk. However, contemporary risk models may not accurately reflect outcomes at reference mitral valve centers. The purpose of this study was to describe perioperative mortality rates after mitral valve operations in a contemporary cohort.

Methods

Between 2001 and 2011, 1,154 patients underwent mitral valve operations at a reference center. Of these, 851 underwent repair and 303 underwent replacement. Concomitant coronary artery bypass grafting was performed in 201 (17%). The Society of Thoracic Surgeons (STS) risk score version 2.73 and European System for Cardiac Operative Risk Evaluation (EuroSCORE) II were used to estimate the number of perioperative deaths.

Results

The observed perioperative mortality was 1.0%. The STS score was 2.3% ± 2.6% and was higher than the observed mortality rate for each of the STS subgroups (all p < 0.001). The EuroSCORE II expected mortality was 3.0% ± 3.4% and was greater than the observed mortality rate for isolated and combined procedures (both p < 0.001). The STS and EuroSCORE II provided fair death discrimination, with an area under the receiver operating characteristic curve of 0.74 and 0.67, respectively.

Conclusions

Although current risk models aid in risk stratifying patients, the contemporary perioperative mortality rate at a reference mitral valve center is significantly lower than expected. The use of alternate therapies must therefore take into consideration differences in perioperative risk based on the treating center.

Section snippets

Material and Methods

Approval was obtained from the University of Ottawa Heart Institute Human Research Ethics board to analyze death data after mitral operations.

Results

As described in Table 1, patients who underwent mitral valve repair were younger, less likely female, less likely to have diabetes preoperatively, and less likely to have had a previous sternotomy compared with patients who underwent replacement. Not surprisingly, the estimated perioperative risk of death according to the EuroSCORE II and STS risk score was higher for patients who underwent replacement compared with repair.

Although patients who underwent repair had a larger left ventricle

Comment

In this study, we examined a cohort of patients who underwent mitral operations at a reference center and found that the observed perioperative mortality was low. The EuroSCORE II and STS risk score were not well calibrated to predict patient deaths and provided only fair death discrimination in this population.

The calibration of a risk model refers to how well observed values align with expected values [20] and can be described as the observed-to-expected mortality ratio, which was lower than

References (25)

R.R. Makkar et al.
Stratification of outcomes after transcatheter aortic valve replacement according to surgical inoperability for technical versus clinical reasons
J Am Coll Cardiol
(2014)
L. Mauri et al.
4-year results of a randomized controlled trial of percutaneous repair versus surgery for mitral regurgitation
J Am Coll Cardiol
(2013)
B.H. Trichon et al.
Relation of frequency and severity of mitral regurgitation to survival among patients with left ventricular systolic dysfunction and heart failure
Am J Cardiol
(2003)
J.S. Rankin et al.
The Society of Thoracic Surgeons risk model for operative mortality after multiple valve surgery
Ann Thorac Surg
(2013)
S.M. O'Brien et al.
The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 2—isolated valve surgery
Ann Thorac Surg
(2009)
D.M. Shahian et al.
The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 3—valve plus coronary artery bypass grafting surgery
Ann Thorac Surg
(2009)
J.G. Castillo et al.
A near 100% repair rate for mitral valve prolapse is achievable in a reference center: implications for future guidelines
J Thorac Cardiovasc Surg
(2012)
D.R. Johnston et al.
Surgical repair of posterior mitral valve prolapse: implications for guidelines and percutaneous repair
Ann Thorac Surg
(2010)
J.M. Arcidi et al.
Fifteen-year experience with minimally invasive approach for reoperations involving the mitral valve
J Thorac Cardiovasc Surg
(2012)
F. Barili et al.
Reliability of new scores in predicting perioperative mortality after mitral valve surgery
J Thorac Cardiovasc Surg
(2014)

J.L. Kennedy et al.

Does the Society of Thoracic Surgeons risk score accurately predict operative mortality for patients with pulmonary hypertension?

J Thorac Cardiovasc Surg

(2013)

F. Maisano et al.

Percutaneous mitral valve interventions in the real world: early and 1-year results from the ACCESS-EU, a prospective, multicenter, nonrandomized post-approval study of the MitraClip therapy in Europe

J Am Coll Cardiol

(2013)

Cited by (16)

Validation of the European system for cardiac operative risk evaluation II in a large Taiwan cardiac surgical centre
2023, Journal of the Formosan Medical Association
The European System for Cardiac Operative Risk Evaluation (EuroSCORE II) is a well-established scoring system for predicting mortality in cardiac surgery. This system was derived predominantly from a European patient cohort; however, no validation of this system has been conducted in Taiwan. We sought to assess the performance of EuroSCORE II at a tertiary centre.
The 2161 adult patients receiving cardiac surgery between 2017 and 2020 in our institution were included.
Overall, the in-hospital mortality rate was 7.89%. The performance of EuroSCORE II was assessed using the area under the receiver operator curve (AUC) for discrimination and the Hosmer–Lemeshow (H–L) test for calibration. Data were analysed for type of surgery, risk stratification, and status of the operation.
EuroSCORE II had good discriminative power (AUC=0.854, 95% Confidence Interval (CI): 0.822–0.885) and good calibration (χ²=5.19, p=0.82) for all types of surgery except ventricular assist devices (AUC=0.618, 95% CI: 0.497–0.738). EuroSCORE II also showed good calibration for most types of surgery except coronary artery bypass surgery (CABG) combined procedure (P=0.033), heart transplantation (HT) (P=0.017), and urgent operation (P=0.041). EuroSCORE II significantly underestimated the risk for CABG combined procedure and urgent operations, and overestimated the risk for HT.
EuroSCORE II had satisfactory discrimination and calibration power to predict surgical mortality in Taiwan. However, the model is poorly calibrated for CABG combined procedure, HT, urgent operation, and, likely, lower- and higher-risk patients.
Machine learning using institution-specific multi-modal electronic health records improves mortality risk prediction for cardiac surgery patients
2023, JTCVS Open
The Society of Thoracic Surgeons risk scores are widely used to assess risk of morbidity and mortality in specific cardiac surgeries but may not perform optimally in all patients. In a cohort of patients undergoing cardiac surgery, we developed a data-driven, institution-specific machine learning–based model inferred from multi-modal electronic health records and compared the performance with the Society of Thoracic Surgeons models.
All adult patients undergoing cardiac surgery between 2011 and 2016 were included. Routine electronic health record administrative, demographic, clinical, hemodynamic, laboratory, pharmacological, and procedural data features were extracted. The outcome was postoperative mortality. The database was randomly split into training (development) and test (evaluation) cohorts. Models developed using 4 classification algorithms were compared using 6 evaluation metrics. The performance of the final model was compared with the Society of Thoracic Surgeons models for 7 index surgical procedures.
A total of 6392 patients were included and described by 4016 features. Overall mortality was 3.0% (n = 193). The XGBoost algorithm using only features with no missing data (336 features) yielded the best-performing predictor. When applied to the test set, the predictor performed well (F-measure = 0.775; precision = 0.756; recall = 0.795; accuracy = 0.986; area under the receiver operating characteristic curve = 0.978; area under the precision-recall curve = 0.804). eXtreme Gradient Boosting consistently demonstrated improved performance over the Society of Thoracic Surgeons models when evaluated on index procedures within the test set.
Machine learning models using institution-specific multi-modal electronic health records may improve performance in predicting mortality for individual patients undergoing cardiac surgery compared with the standard-of-care, population-derived Society of Thoracic Surgeons models. Institution-specific models may provide insights complementary to population-derived risk predictions to aid patient-level decision making.
Mortality and rehospitalization after mitral valve surgery as a function of age and key comorbidities
2023, American Heart Journal
Mitral valve surgery is associated with substantial perioperative risk and long-term complications. Data on long-term outcomes following surgery remain scarce and are hypothetically modified by age and comorbidities.
This Danish nationwide study included patients ≥60 years of age undergoing mitral valve surgery from 2000-2018. Patients were observed from day of surgery until outcome of interest (ie, rehospitalization or death) or maximum 1 year of follow-up. The absolute risks of outcomes were assessed, and associated factors were evaluated. Based on age and comorbidities, patients were stratified in 4 groups: low (<75 years + 0 comorbidities), low intermediate (≥75 years/1 comorbidity), high intermediate (≥75 years + 1 comorbidity/2 comorbidities), and high risk of death (≥75 years + ≥2 comorbidities).
In total, 4,202 patients (62.9% men) were identified. Within 1 year after surgery, 504 (12.0%) died and 2,456 (58.5%) were rehospitalized. Factors associated with death included older age (>75 years), chronic obstructive lung disease, heart failure, prior myocardial infarction, prior stroke, liver disease, and kidney disease. The 1-year risks of death among patients in low, low-intermediate, high-intermediate, and high risk of death were 3.6%, 10.3%, 19.6%, and 27.7%, respectively. Diabetes mellitus and chronic obstructive lung disease were associated with an increased incidence of rehospitalization, and the incidence of rehospitalization was similar among the 4 abovementioned groups (57.8%-62.8%).
Mortality and rehospitalization risks after mitral valve surgery varied substantially with age and comorbidities. High-risk patients with >25% 1-year mortality may be easily identified using readily available clinical features.
In Denmark, registry-based studies that are conducted for the sole purpose of statistics and scientific research do not require ethical approval or informed consent by law. However, the study is approved by the data responsible institute (the Capital Region of Denmark [approval number: P-2019-348]) in accordance with the general data protection regulation.
Optimizing care of patients undergoing cardiac surgery using a digital platform and a multidisciplinary team approach
2023, Journal of Thoracic and Cardiovascular Surgery
Restrictions to care access during the pandemic along with the increasing complexity of patients awaiting cardiac surgery provides unique challenges for care delivery. The University of Ottawa Heart Institute has developed a novel multidisciplinary digital platform, the Prehab Automated Follow-Up (AFU) Program, which delivers patient/caregiver teaching regarding risk factor mitigation, tracks patient symptoms, and screens for optimization using best practice guidelines. This study was conducted to quantify patient outcomes following initiation of the AFU Program.
Patients awaiting elective cardiac surgery are enrolled and screened via automated telephone conversation, according to best practice guidelines, and a Short Form-12 preoperative assessment. Following this screen, patients are referred for an in-person assessment by an appropriate multidisciplinary team member; namely, a diabetes specialist, physiotherapist, dietitian, smoking cessation counselor, social worker, vocational counselor, and/or psychologist.
Since initiation in February 2021, the AFU Program has enrolled more than 1237 patients with 508 multidisciplinary team referrals prompted by the AFU screening platform. Before program initiation, there were no multidisciplinary team referrals for preoperative optimization. Compared with patients treated between February 2020 and February 2021, there was a 2.5% decrease in hospital readmission rate within 30 days of surgery, a 0.6-day shorter hospital stay, and a 2.5% decrease in surgical site infection.
Our cardiac surgery AFU Program reduced adverse health outcomes for patients by identifying and optimizing risk factors that increased quality of patient care. The AFU Program provides patient/caregiver engagement through educational support and multidisciplinary team counseling.
Parsimonious machine learning models to predict resource use in cardiac surgery across a statewide collaborative
2022, JTCVS Open
We sought to several develop parsimonious machine learning models to predict resource utilization and clinical outcomes following cardiac operations using only preoperative factors.
All patients undergoing coronary artery bypass grafting and/or valve operations were identified in the 2015-2021 University of California Cardiac Surgery Consortium repository. The primary end point of the study was length of stay (LOS). Secondary endpoints included 30-day mortality, acute kidney injury, reoperation, postoperative blood transfusion and duration of intensive care unit admission (ICU LOS). Linear regression, gradient boosted machines, random forest, extreme gradient boosting predictive models were developed. The coefficient of determination and area under the receiver operating characteristic (AUC) were used to compare models. Important predictors of increased resource use were identified using SHapley summary plots.
Compared with all other modeling strategies, gradient boosted machines demonstrated the greatest performance in the prediction of LOS (coefficient of determination, 0.42), ICU LOS (coefficient of determination, 0.23) and 30-day mortality (AUC, 0.69). Advancing age, reduced hematocrit, and multiple-valve procedures were associated with increased LOS and ICU LOS. Furthermore, the gradient boosted machine model best predicted acute kidney injury (AUC, 0.76), whereas random forest exhibited greatest discrimination in the prediction of postoperative transfusion (AUC, 0.73). We observed no difference in performance between modeling strategies for reoperation (AUC, 0.80).
Our findings affirm the utility of machine learning in the estimation of resource use and clinical outcomes following cardiac operations. We identified several risk factors associated with increased resource use, which may be used to guide case scheduling in times of limited hospital capacity.
Meta-Analysis Comparing Established Risk Prediction Models (EuroSCORE II, STS Score, and ACEF Score) for Perioperative Mortality During Cardiac Surgery
2016, American Journal of Cardiology
Citation Excerpt :
After reviewing the references of all articles that met our inclusion criteria, the references in the meta-analysis found in our original search, and all articles citing the index papers for the core risk models using both Web of Knowledge and Scopus, we identified 1 additional article for inclusion. This left 22 articles with at least 1 comparison between the core risk models.16–37 Table 1 lists the characteristics of the 22 articles with comparisons among the 3 core risk models.
A wide variety of multivariable risk models have been developed to predict mortality in the setting of cardiac surgery; however, the relative utility of these models is unknown. This study investigated the literature related to comparisons made between established risk prediction models for perioperative mortality used in the setting of cardiac surgery. A systematic review was conducted to capture studies in cardiac surgery comparing the relative performance of at least 2 prediction models cited in recent guidelines (European System for Cardiac Operative Risk Evaluation [EuroSCORE II], Society for Thoracic Surgeons 2008 Cardiac Surgery Risk Models [STS] score, and Age, Creatinine, Ejection Fraction [ACEF] score) for the outcomes of 1-month or inhospital mortality. For articles that met inclusion criteria, we extracted information on study design, predictive performance of risk models, and potential for bias. Meta-analyses were conducted to calculate a summary estimate of the difference in AUCs between models. We identified 22 eligible studies that contained 33 comparisons among the above models. Meta-analysis of differences in AUCs revealed that the EuroSCORE II and STS score performed similarly (with a summary difference in AUC = 0.00), while outperforming the ACEF score (with summary differences in AUC of 0.10 and 0.08, respectively, p <0.05). Other metrics of discrimination and calibration were presented less consistently, and no study presented any metric of reclassification. Small sample size and absent descriptions of missing data were common in these studies. In conclusion, the EuroSCORE II and STS score outperform the ACEF score on discrimination.

View all citing articles on Scopus

View full text

Original articleAdult cardiacPerioperative Deaths After Mitral Valve Operations May Be Overestimated by Contemporary Risk Models

Background

Methods

Results

Conclusions

Section snippets

Material and Methods

Results

Comment

J Am Coll Cardiol

J Am Coll Cardiol

Am J Cardiol

Ann Thorac Surg

Ann Thorac Surg

Ann Thorac Surg

J Thorac Cardiovasc Surg

Ann Thorac Surg

J Thorac Cardiovasc Surg

J Thorac Cardiovasc Surg

J Thorac Cardiovasc Surg

J Am Coll Cardiol

Original article
Adult cardiac
Perioperative Deaths After Mitral Valve Operations May Be Overestimated by Contemporary Risk Models