Article Text
PDF
Abstract
Objective Low quality and unwarranted clinical variation harm patients and increase unnecessary costs. Effective approaches to improve clinical and economic value have been difficult. The Ochsner Health System looked to improve clinical care quality and reduce unnecessary costs in cardiology using active measurement and customised feedback.
Methods We serially measured care decisions using online, simulated cases to capture clinical details of cardiology practice and provide individual feedback. Fifty cardiologists cared for two simulated patients in each of six assessment rounds occurring 4 months apart. Simulated patients presented with heart failure (HF), coronary artery disease (CAD), supraventricular tachyarrhythmia (SVT) or valvular heart disease. Using Ochsner’s patient-level data, we performed real-world pre–post analyses of physician practice changes, patient outcomes and costs.
Results Between baseline and final rounds, overall simulated quality-of-care scores improved 14.1% (p<0.001). In the same period, we found cost-of-care variation decreased in patient-level data, with larger decreases for more severely ill patients. The total per-patient direct costs decreased $493 in SVT, $305 in HF and $55 in CAD (p<0.05 for SVT and HF). Readmission rates fell significantly for HF (from 20.0% to 11.9%) and SVT (from 14.5% to 7.8%) (both p<0.001) and non-significantly for CAD (from 13.7% to 11.3%, p=0.112). The cost avoidance/revenue generation opportunity from reduced readmissions and direct costs amounted to annual savings of $4.34 million, with no significant changes to in-hospital mortality rates (p>0.05).
Conclusions Using simulated patients to serially measure and provide individual feedback on clinical practice significantly raises quality and reduces practice variation and costs without negatively impacting outcomes.
- evidence-based medicine
- quality improvement
- clinical variation
- patient outcomes
- heart failure
- coronary artery disease
- arrhythmia
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- evidence-based medicine
- quality improvement
- clinical variation
- patient outcomes
- heart failure
- coronary artery disease
- arrhythmia
Key questions
What is already known about this subject?
Cardiovascular disease accounts for the highest proportion of direct medical costs in specialty care spending, with an estimated 44% of the US population having some form of cardiovascular disease by 2030.
To meet this increased demand, providers and payers must improve their quality while simultaneously lowering costs.
What does this study add?
Undertaken among cardiologists at Ochsner Health System, this is a report on a quality improvement study using online clinical vignettes designed to simulate real-world clinical practice and provide individualised feedback.
We demonstrate use of this vignette and feedback system improved clinical practice while simultaneously lowering readmissions rates, total direct costs and mortality rates.
How might this impact on clinical practice?
The evidence provided in this study demonstrates a method by which hospitals and hospital systems can provide the best possible care at a lower overall cost.
Introduction
Cardiovascular disease, at an estimated $189.7 billion,1 accounts for the highest proportion of direct medical costs in specialty care spending. By 2030, the total direct cost of treating these conditions will reach $1.028 trillion,1 2 with an estimated 44% of the US population projected to have some form of cardiovascular disease. In the face of rising demand and limited resources, the overwhelming issue for providers and payers is to provide the highest quality care at the lowest cost. In cardiology, research shows that value (quality/cost) for patients is driven primarily by secondary prevention efforts, eliminating unnecessary testing and delivering cost-effective care.3 4
Measuring clinical quality—in cardiology and other clinical areas—is difficult. First, patient heterogeneity makes identifying variation in practice patterns hard. There is simply too much variation among patients to tweeze out a clear picture of the practice variation by an individual physician or a group. Second, comparative data typically lack the necessary detail about the care provided. Strategies predictably employed to acquire clinical detail (eg, claims analysis, chart reviews, electronic health record [EHR] data) are cumbersome to use in value determinations.5 For a clinical measure to be truly revealing, one must be able to ascertain what clinical decisions and treatment choices were made within a specific clinical context: Was a proper work-up performed? Was the diagnosis correct? Was the care coordinated? Only with these details can the level of quality and, when combined with cost of care, the value to the patient and payer be determined.
In today’s evolving reimbursement environment and with specialty care costs rising, the Ochsner Health System (OHS) set ambitious goals to increase the value of care it provides to its patients. Instead of having mandates set by outside organisations, in 2011 OHS took the initiative to lower the cost of their care and launched the Pursuit of Value (POV). POV is a system-wide mandate aligned with Ochsner’s mission to deliver financially sustainable care. In its first year, POV realised significant savings in cardiology by focusing on reducing catheterisation and electrophysiology lab costs through a consensus-driven selection of vendors. The POV work also uncovered real-world evidence of practice variation across cardiologists in the network. Physicians, however, doubted the validity of this claims-data analyses. Still, all involved found that variation in patient acuity, site of care and complexity made comparing individual physician performance difficult.
From these lessons, OHS wanted an effective approach to further optimise cardiovascular clinical practice and lower costs. This approach needed to be case-mix-adjusted, use prospectively collected data to measure provider practice, feed the results back to providers in real time and demonstrate savings through analysis of cost data. To meet these requirements, OHS partnered with QURE Healthcare to use their simulated patients to measure and remeasure clinical practice decisions and provide customised feedback on improvement opportunities to providers. Their first project focused on cardiology, which was facing declining reimbursement and looming Centers for Medicare & Medicaid Services (CMS) bundled payments, to standardise practice within its award-winning multisite John Ochsner Heart and Vascular Institute.
Herein, we describe the results of a real-world physician engagement and care standardisation approach, designed to help Ochsner cardiologists succeed in a value-based world. The explicit project goals were to improve adherence to evidence-based practice guidelines and to lower costs. We report on the results of this engagement in four of the most common cardiac conditions: heart failure (HF), coronary artery disease (CAD), supraventricular tachyarrhythmia (SVT) and valvular heart disease (VHD). We document improvements in clinical decision-making on simulated patients and the translation of these improvements into real practice change as measured by patient-level chart data resulting in large economic returns.
Methods
Setting
OHS is Louisiana’s largest non-profit, academic, healthcare system, consisting of over 30 owned, managed and affiliated hospitals and more than 80 health centres. OHS is a member of the Ochsner Health Network, the largest clinically integrated network in the region. In 2011, OHS developed the POV initiative to minimise unwarranted clinical supply-chain variation, a leading factor in higher costs and poorer outcomes. The POV comprised an integrated team of clinical and administrative leaders who deploy the fundamentals of cost engineering, process improvement and clinical practice variation reduction.
Study participants
Employed cardiologists practising at six Ochsner locations in Southeast Louisiana were invited and agreed to participate in the Ochsner-QURE Quality Project (OQQP). This included community-based cardiologists and academic cardiologists specialising in electrophysiology, interventional cardiology, non-invasive imaging and advanced HF. Although OQQP participation was voluntary and initially not tied to financial incentives, 100% of the practising cardiologists at the six sites took part in the study. At study start, participants were asked to complete a brief survey to provide additional information on their training, years in practice, patient load, and perception of clinical variability and care quality.
Clinical focus
Every 4 months from September 2016 through April 2018, providers in the OQQP study cared for simulated cases designed to assess clinical practice variation in HF, CAD, SVT or VHD. The cases and the scoring criteria were developed from evidence-based practice guidelines, from baseline data of Ochsner performance gaps and in consultation with select Ochsner cardiology leaders.
Data
We had three sources of data for this real-world study: data scored from the care decisions made in the clinical performance and value (CPV) cases; patient-level data collected by Ochsner’s POV team responsible for tracking and monitoring clinical and financial outcomes; and Medicare cost data.
CPV vignettes
We developed 12 CPV cases for OQQP: 3 each in HF, CAD, SVT and VHD. National guidelines, peer-reviewed evidence base and Ochsner best practices were used to develop these cases and the explicit scoring criteria. There were six rounds, 4 months apart, and in each round providers cared for two new cases. No provider repeated any of the cases.
CPV vignettes are simulated cases presented in an online platform used to measure provider clinical practice. CPVs are a validated, responsive measure of clinical practice improvement and proven to measure how a provider evaluates, diagnoses and treats a wide range of diseases and conditions in real life.6 In CPVs, providers progress through a typical patient encounter comprising five domains of care: (1) history taking, (2) physical examination, (3) diagnostic work-up, (4) diagnosis and (5) designating a treatment plan with follow-up. Each CPV takes approximately 20–30 min to complete. Each completed CPV is scored against explicit evidence-based scoring criteria by two specially trained physician scorers and blinded to participant identity. Points are given for responses matching the explicit criteria, with domain and overall scores ranging from 0% to 100%.
The OQQP process consists of multiple rounds of CPV engagement. In each round, participants cared for two CPV patients, one presenting with HF or CAD and the other presenting with SVT or VHD. They received scores and individualised feedback reports for each case that compared their performance with the evidence base and with their peers. After every round, providers also participated in facilitated group feedback sessions, structured for providers to interact with each other. For select areas with high group variation, providers discuss areas for collective improvement and consensus building. Individual scores are shared with the participants in these group feedback sessions. The multiple rounds of measurement and feedback improve quality and outcomes, and standardise practice.
Ochsner patient data
Patient-level data came from Ochsner’s Decision Support team. Decision Support is responsible for tracking and monitoring both clinical and financial outcomes for POV. These data were pulled directly from Ochsner’s EPIC EHR and from Ochsner’s Decision Support tool, EPSi (Allscripts). We obtained comparative patient-level data at two discrete time intervals: (1) at baseline between September 2015 and August 2016 prior to initiation of OQQP; and (2) from September 2017 to August 2018. These data focused on patients with the same four conditions as the CPV patient simulations and feedback.
Analyses
We analysed three main outcomes: (1) overall and domain CPV improvements over time, as determined by the percentage of items participants addressed according to the evidence-based scoring criteria; (2) increased adherence to evidence-based care in actual clinical practice; and (3) the economic benefits derived from the improvements in the patient-level data. Fisher’s exact test was used for binary outcomes, and Student’s t-test was used for most continuous outcomes. Additionally, in the patient-level data, we log-normalised the cost (economic) data for analysis, and to account for different disease severity distributions between the two time periods we adjusted the baseline results to match that of the second time period. All statistical analyses were performed using Stata V.14.2.
Results
Baseline characteristics and assessment
Fifty OHS cardiologists participated in the study (table 1). Almost half the group (48%) were subspecialised, in either cardiovascular non-invasive imaging (2%), peripheral vascular disease (2%), electrophysiology (12%), advanced HF (10%) or interventional cardiology (22%). On average, each provider saw 59.9±27.4 patients per week, with community physicians averaging 73.1 patients compared with academics averaging 52.9. At the start of the OQQP, 80% of providers rated the organisation’s focus on quality and lowering unnecessary costs as either ‘good’ or ‘excellent’, but 90% saw the care provided by the group as being ‘somewhat’ to ‘highly variable’. Four participants left the study to practise outside of the Ochsner network and two cardiologists joined the practice after January 2017.
Baseline provider/practice characteristics of participants
At baseline, we observed high variability in the providers’ CPV quality-of-care scores. The overall quality score for the group averaged 56.0%±10.5% (table 2). The average clinical domain scores were highest for history taking at 68.3%±12.8% and physical exam at 89.0%±14.4%. As providers proceeded through the rest of the domains—those which required more provider judgement and input—we observed successively declining domain scores from work-up through diagnosis and treatment, with the lowest baseline domain score in treatment, 39.0%±16.9%. The average overall scores, disaggregated by condition, ranged from 50.7% for VHD, 55.0% for HF, 58.7% for SVT and 60.7% for CAD (p=0.006).
CPV scores
Changes in CPV scores
Overall, after six rounds of serial measurement and feedback, CPV quality scores improved 14.1% (p<0.001) (table 2). The greatest improvements occurred in history (+24.5%, p<0.001) and diagnosis (+22.4%, p<0.001) domains. The overall IQR increased from 50.0%–61.2% at baseline to 63.0%–76.2% in the sixth and final round of the study. In other words, the 25th percentile performers in the final round of OQQP scored 1.8% higher than the 75th percentile in the baseline round.
Scores by condition showed similarly large, statistically significant improvements in overall CPV scores: +13.2% in HF (p=0.004), +15.3% in CAD (p<0.001), +11.0% in SVT (p<0.001) and +16.7% in VHD (p<0.001). By domain, the largest increases were seen in the diagnosis of VHD (+28.9%, p<0.001) and HF (+35.5%, p<0.001), with primary diagnoses increasing by 6.6% (p=0.194) and 9.9% (p=0.337), respectively. Treatment scores in CAD (+12.4%, p=0.001) and VHD (+13.6%, p=0.002) increased dramatically as well, which is perhaps not as surprising given the treatment domain was the lowest scoring in the CPVs. In the VHD cases, for example, providers identified the need for cardiac rehabilitation for their atrial or mitral valve replacement patients at a greater rate as the rounds progressed, increasing 35% (p=0.041). Likewise, for CAD, making all appropriate medical treatment recommendations increased by 30% (p=0.039).
For treatment, which includes clinical decisions related to initial management and follow-up care, we found specific key improvements. Continuity-of-care referrals following discharge saw large gains, specifically in cardiac rehabilitation referrals for patients with CAD and HF (+39%, p=0.001) and referrals to primary care following resolution of an acute episode (+28%, p=0.007). Documentation of appropriate medication orders for statins (+10%, p=0.325) and aspirin (+10%, p=0.278) increased, although these did not achieve significance. In addition to writing guideline-directed prescriptions, providers were also more likely to appropriately stop medications. For example, discontinuing pioglitazone in a patient with diabetes presenting with HF increased by 64% (p=0.021) over the course of the study. However, in contended areas, such as amlodipine use in reduced ejection fraction HF, the cardiologists did not change their usage. Finally, an important and often overlooked area in patient involvement and patient satisfaction—discussing procedures and possible complications with patients—saw a robust 47% increase between the baseline and final rounds (p<0.001).
Did CPV improvement translate into patient-level improvement?
The patient simulations present a unique opportunity to engage providers by having them care for the same patients as their peers and compare their clinical decision-making across the group. The improvement in the simulated patients’ care is a means to an end. The goal is to improve care quality for patients and cost-effectiveness for payers and providers. We used the pre–post patient-level data from POV to determine if various work-up and treatment items improved, and in three priority economic metrics: readmission rate, in-hospital mortality and arithmetic cost per case.
In our pre–post analyses, CPV data showed increasing evidence-based troponin orders over creatine kinase to rule out an infarction. This change was corroborated in the Ochsner-provided patient-level data, with cardiologists ordering troponins 73.0% at baseline and increasing to 83.7% postintervention (p<0.001). Similarly, for coronary angiography, in the CPVs, providers ordered these unnecessarily (19.6% pre vs 8.6% post). When we checked the pre–post patient-level data, coronary angiography use had decreased from 13.3% pre to 10.4% post (p<0.001). Likewise, the CPV data showed that aspirin orders increased from 46.6% to 70.0% (p=0.012), while dual antiplatelet usage increased from 85.7% to 100% (p=0.437). Again, these CPV-measured changes corresponded to the patient-level data where aspirin usage increased from 88.5% to 92.1% (p<0.001) and dual antiplatelet orders increased from 84.8% to 90.8% (p=0.109).
We did a pre–post analysis to determine if clinical improvement led to cost reduction. We found reductions in readmissions for three of the four conditions, achieving statistical significance for the two highest volume disease areas: HF (20.0% vs 14.9%) and SVT (13.9% vs 8.9%) (p<0.001 for both) (table 3).
Real-world improvements in cost, readmissions and in-hospital mortality*
To ensure the practice improvements did not come at the expense of patient outcomes, we did a pre–post analysis of in-hospital mortality, finding non-significant decreases in overall mortality of 0.6% in SVT, 0.3% in HF and 1.3% in CAD, and an increase of 0.5% in VHD (p>0.05 for all).
Estimation of economic benefits
Our findings of improved clinical care leading to fewer readmissions generated hard dollar cost per case savings and costs avoided per readmission averted. Using POV’s EPSi data, we determined the pre–post net cost per case reductions from the CPV intervention. Examination of direct costs for the four disease conditions showed an average decrease of $280/patient, which proved significant (p=0.002). In HF, making up about 61% of the total patient population of the study, direct costs decreased by $305 per case (p=0.035). Applying the per case savings amounts to the current annualised volumes of patients at Ochsner, by condition, we determined the overall savings: $492 644 (=$493 x 999) in SVT, $949 514 (=$305 x 3114) in HF, $52 214 (=$55 x 954) in CAD and −$60 773 (=−$1351 x 45) in VHD. The total direct cost savings for the four conditions was $1 433 598 over 1 year.
We examined the pre–post variation costs for the three high-volume diseases (SVT, CAD and HF). In a log-normalised distribution of costs, we found not only did the direct cost SD decrease significantly but the variation decreased more as the disease severity worsened: for those who were classified as mild or moderately severe, the direct cost SD fell by $135 (p=0.003); in those that were classified as majorly or extremely severe, the SD decreased by $585 (p<0.001). These decreases signify a large reduction in outliers. Although the data were not sufficient for us to determine the underlying cause, greater adherence to the evidence base and improved practice standardisation (ie, greater diagnostic accuracy; better treatment) measured in this study appear to be important contributing factors.
The economic benefits of fewer readmissions come from decreasing unneeded spending on patients who are kept out of the hospital and also from created capacity leading to more admissions in a capacity-constrained system like Ochsner. We, however, only calculated the savings from decreased readmission rates as the additional revenue from any marginal admissions might not materialise in other hospital systems. For HF, at baseline, the readmission rate was 20.0%, but just over a year later this dropped to 14.9%, a reduction of 5.1 percentage points. From other studies,7 we know that the mean Medicare beneficiary per-patient readmission cost is $12 989. If we assume costs are similar across HF cases, then a 5.1% readmission reduction on a volume of 3114 cases leads to 157 avoided readmissions and $2 036 377 in avoided spending on those readmissions. Doing a similar calculation for the two other high-volume conditions (SVT and CAD), we see a cost avoidance of $564 450 for SVT (=$12 989 x [13.9%–8.9%] x 999 patients) and $304 526 for CAD (=$12 989 x [13.7%–11.3%] x 954 patients). Adding up the values in HF, SVT and CAD—and ignoring the smaller contribution in VHD—we estimate a total of $2 905 353 in savings.
The overall savings and new revenue for Ochsner from this project was $1 433 598 + $2 905 353 ≈ $4 388 950/year.
Discussion
Payer and provider methods to lower specialty care costs are at worst largely ineffective, and at best not keeping up with declining reimbursement.8 Two factors likely explain the limitations of existing efforts: the difficulty in reducing practice variation and, perhaps more fundamentally, the failure to engage physicians and change their practice.9 We introduced CPVs to OHS to explicitly engage physicians to measure clinical practice and drive clinical practice change towards evidence-based, cost-effective care.
In this real-world project, we found a temporal correlation between improvements in the CPV care and improvements in patient care, lowering of costs and generation of additional revenue for the health system. While this is not causal, the inference is strong because the improvements in the CPVs were the same as the improvement found in actual patients.
We initially observed a high degree of variability among the providers by overall quality scores and care domain that was mitigated by serial CPV measurements and individual and group feedback. After six rounds of measurement and feedback, the average performance improved significantly from 56% to 70%. The size of the Ochsner increase well exceeds CPV improvements that we have previously shown to be clinically detectable (3%–5%) in patient outcomes.10 We were fortunate to have high-quality pre–post financial data. Although we only examined a few of the potential savings, the economic returns were impressive for these four conditions. The total direct cost savings per case totalled $1.4 million, and an additional $2.9 million was generated in cost avoidance and new revenue. Other research indicates that these results are sustainable for a minimum of 5 years.11
The reasons to reduce costs and improve outcomes are legion. Declines in US cardiovascular mortality have surpassed those in other developed countries, where spending is far less. Estimates are that 20%–50% of the care provided in the USA could be considered unnecessary or wasteful.12–14 CMS and commercial payers have responded with new value-based reimbursement schemes structured to incentivise care that is evidence-based, more efficient and less costly. As the highest volume and costliest specialty, cardiology has been targeted by CMS, the Department of Veterans’ Affairs, the American Medical Association, the National Committee for Quality Assurance and notably the American College of Cardiology. Existing standardisation approaches, however, have not been adequate to meaningfully change clinical practice. Better measurement and engagement tools are needed to support cardiologists in their quest to change their practice to provide higher quality, more cost-effective care.
There are several limitations to the study. We would have liked more patient-level metrics of clinical practice to directly compare the detail available in the CPVs. Real-time data limitations, particularly of clinical practice, remain one of the most significant challenges facing quality improvement efforts everywhere. Ideally, we also would have liked a control group and not just a pre–post analysis. For obvious reasons, quite aside from issues of contamination, conducting an experiment and randomising providers to intervention or not are beyond the realm of feasibility in an institution taking care of patients and wanting to involve all of its members as they decide how the group is to practise.
This collaboration shows an opportunity to engage specialist providers, with limited burden, to achieve both a strong clinical and economic impact.
Footnotes
Contributors PO: study concept and design, and critical revision of the manuscript. CJW: data collection and writing of the manuscript. DC: study concept, and writing and critical revision of the manuscript. NS: data collection and drafting the manuscript. JD: data collection and drafting the manuscript. RR: data collection and drafting the manuscript. CV: data collection, drafting the article and critical revision of the manuscript. DP: data analysis and interpretation of results, drafting the manuscript, and critical revision of the manuscript. MT: data collection, drafting the manuscript and critical revision of the manuscript. JP: study concept and design, data analysis and interpretation of results, critical revision of the manuscript, final approval of the version to be published, and study supervision.
Funding This work was funded by a contract between Ochsner Health and QURE.
Competing interests QURE, whose intellectual property was used to prepare the cases and collect the data, was contracted by Ochsner Health. Otherwise, no other disclosures to report.
Patient consent for publication Not required.
Ethics approval The CPV data gathered were part of standard hospital monitoring of clinical quality and safety. The data were not collected for research purposes and contained no patient information. As per the Office of Research Integrity of the US Department of Health and Human Services under the US Code of Federal Regulation, 45 CFR 46, the study is exempt from Institutional Review Board review.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Data are available upon reasonable request to the corresponding author, Dr. John Peabody, jpeabody@qurehealthcare.com.