Methodologies in health services research for critical care
A systematic review of the Charlson comorbidity index using Canadian administrative databases: a perspective on risk adjustment in critical care research

https://doi.org/10.1016/j.jcrc.2004.09.007Get rights and content

Abstract

The Charlson index is commonly used for risk adjustment in critical care health services research. However, the literature supporting this methodology has not been thoroughly explored. We systematically reviewed the literature related to administrative database adaptations of the Charlson index. Our review has 3 major findings. First, 2 studies compared Canadian administrative databases with chart review for obtaining Charlson comorbidity data. Agreement between the database and chart review was substantial (κ > 0.70), and mortality prediction did not differ. Second, 5 database adaptations were identified with the Deyo and Dartmouth-Manitoba adaptations being most popular. Three studies directly compared these 2 popular adaptations and demonstrated substantial agreement (κ > 0.70) and similar predictive ability for mortality. Third, one study validated the Charlson index for critically ill patients but demonstrated that APACHE (Acute Physiology and Chronic Health Evaluation) II better discriminates inhospital mortality (area under curve 0.67 vs 0.87). Time and cost barriers prevent widespread use of physiology-based risk adjustment in population-based research. The decreased predictive ability of the Charlson index must be weighed against the advantages of using this instrument for population-based research. Future research should focus on updating the Charlson index for recent changes in the prognosis of comorbid diseases and introduction of International Statistical Classification of Diseases, 10th Revision coding of discharge abstracts.

Introduction

Administrative databases are increasingly being used to study the outcomes of critically ill patients. These databases provide an efficient means to evaluate outcomes for a large number and variety of patients over a large geographic area [1]. Because most outcomes studies are observational, the ability of administrative databases to provide true (ie, internally valid) results depends on appropriate risk adjustment for case mix. Measuring case mix is complex and includes consideration of patient age; admission diagnosis; physiological derangement; number and severity of comorbid diseases; baseline functional status; socioeconomic, cultural, and ethnic attributes; patient attitudes and preferences; and medical resource requirements [2], [3]. Each of these factors influences prognosis to a varying degree. Admission diagnosis and comorbid disease information are often the factors most readily available from administrative databases. Numerous studies have highlighted the importance of comorbid disease in determining patient outcome after critical illness [4], [5], [6]. Thus, risk adjustment for comorbid disease is an important consideration within health services research.

The Charlson index was developed to predict 1-year patient mortality using comorbidity data obtained from hospital chart review [7]. The derivation cohort was 604 medical inpatients admitted to a New York teaching hospital during 1 month in 1984. The validation cohort was 685 breast cancer patients at a Connecticut teaching hospital from 1962 to 1969. The final Charlson index score was the sum of 19 predefined comorbidities that were assigned weights of 1, 2, 3, or 6. These weights were based on the magnitude of the adjusted relative risks associated with each comorbidity in a Cox proportional hazards regression model (Table 1). Three conditions (liver disease, diabetes, and neoplasm) had different weights based on disease severity. The relative risk of 1-year mortality for each increasing point of the Charlson index was 2.3 (95% confidence interval 1.9-2.8), and the overall model was a highly significant predictor of mortality (P < .0001). At least 9 studies, representing more than 30 000 patients, have validated the Charlson index in a wide variety of diseases for numerous clinical outcomes [6].

Other researchers adapted the Charlson index to obtain comorbidity data from computerized hospital discharge abstract databases coded according to the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) to predict short-term mortality. Administrative database adaptations of the Charlson index are now commonly used for risk adjustment in critical care health services research [8], [9], [10], [11]. However, the literature supporting this method of risk adjustment in critical care has not been thoroughly reviewed. We undertook a systematic review of the literature with 3 specific objectives: (1) to assess the agreement between Canadian ICD-9 administrative databases and chart review for Charlson comorbidity data, (2) to summarize the existing administrative database adaptations of the Charlson index and compare the discriminative ability of each for predicting mortality, and (3) to compare the discriminative ability of the Charlson index versus other risk adjustment methods for predicting inhospital mortality of critically ill patients.

Section snippets

Search strategy

The literature search was conducted as of July 15, 2004, using PubMed and Ovid (release 9.1.0) software for searching the following electronic databases: MEDLINE (from 1965), EMBASE (from 1980), CINAHL (from 1982), and The Cochrane Library (issue 2, 2004). We retrieved all citations related to the Charlson index (search term: “Charlson”) and all citations related to comorbidity adjustment and critical care using medical subject heading search (MeSH) terms: comorbidity, intensive care units

Study selection

The literature search revealed 2163 citations for review: 2157 from the electronic database search and 6 from the hand search (Fig. 1). A total of 2135 citations were excluded because they were not applicable to the previously described study objectives. Abstracts for the remaining 28 studies were obtained, and an additional 12 abstracts were excluded because they were not applicable. Full articles were retrieved for the remaining 16 abstracts. Of these articles, 10 met at least 1 of the

Discussion

The Charlson index is the most widely used method for predicting patient mortality based on comorbidity data [6], [25]. This systematic review has 3 major findings based on its original objectives. First, in comparison to chart review, Canadian ICD-9–coded discharge abstract databases demonstrated a modest positive predictive value but a high negative predictive value for individual comorbidities of the Charlson index in 2 studies. The ICD-9–coded administrative databases were not designed for

Conclusion

The Charlson index and its adaptations for use with administrative databases discriminate mortality similarly. Although mortality prediction can be improved by using physiological data, this information is not generally available in administrative data sets. The decreased precision in risk adjustment using the Charlson index must be weighed against the advantages of population-based research in determining the most appropriate study design for a specific research question. Further validation of

Acknowledgment

The authors thank Dr Charles Flexner for assistance with developing the idea for this manuscript.

References (42)

  • P.S. Romano et al.

    Further evidence concerning the use of a clinical comorbidity index with ICD-9-CM administrative data

    J Clin Epidemiol

    (1993)
  • R.A. Deyo

    Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: a response

    J Clin Epidemiol

    (1993)
  • C.E. Behrendt

    Acute respiratory failure in the United States: Incidence and 31-day survival

    Chest

    (2000)
  • P.S. Romano et al.

    A comparison of administrative versus clinical data: coronary artery bypass surgery as an example. Ischemic Heart Disease Patient Outcomes Research Team

    J Clin Epidemiol

    (1994)
  • W.A. Knaus et al.

    The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults

    Chest

    (1991)
  • K.H. Humphries et al.

    Co-morbidity data in outcomes research: are clinical data derived from administrative databases a reliable alternative to chart review?

    J Clin Epidemiol

    (2000)
  • L.L. Roos et al.

    Comparing clinical information with claims data: some similarities and differences

    J Clin Epidemiol

    (1991)
  • L.L. Roos et al.

    Risk adjustment in claims-based research: The search for efficient approaches

    J Clin Epidemiol

    (1989)
  • W.A. Ghali et al.

    Risk adjustment using administrative data: Impact of a diagnosis-type indicator

    J Gen Intern Med

    (2001)
  • L.I. Iezzoni

    Risk adjustment for medical effectiveness research: an overview of conceptual and methodological considerations

    J Investig Med

    (1995)
  • S.P. Keenan et al.

    Intensive care unit admission has minimal impact on long-term mortality

    Crit Care Med

    (2002)
  • Cited by (311)

    • It's Just a Toe Amputation

      2022, Journal of Foot and Ankle Surgery
    View all citing articles on Scopus

    Dr Needham holds Clinician-Scientist Awards from the Canadian Institutes of Health Research and the University of Toronto Department of Medicine, and a Detweiler Travelling Fellowship from the Royal College of Physicians and Surgeons of Canada. Dr Pronovost is supported, in part, by the Agency for Healthcare Research and Quality (grant number U18HS11902-01). Dr Laupacis is supported by a Senior Scientist Career Award from the Canadian Institutes of Health Research.

    View full text