Methodologies in health services research for critical careA systematic review of the Charlson comorbidity index using Canadian administrative databases: a perspective on risk adjustment in critical care research☆
Introduction
Administrative databases are increasingly being used to study the outcomes of critically ill patients. These databases provide an efficient means to evaluate outcomes for a large number and variety of patients over a large geographic area [1]. Because most outcomes studies are observational, the ability of administrative databases to provide true (ie, internally valid) results depends on appropriate risk adjustment for case mix. Measuring case mix is complex and includes consideration of patient age; admission diagnosis; physiological derangement; number and severity of comorbid diseases; baseline functional status; socioeconomic, cultural, and ethnic attributes; patient attitudes and preferences; and medical resource requirements [2], [3]. Each of these factors influences prognosis to a varying degree. Admission diagnosis and comorbid disease information are often the factors most readily available from administrative databases. Numerous studies have highlighted the importance of comorbid disease in determining patient outcome after critical illness [4], [5], [6]. Thus, risk adjustment for comorbid disease is an important consideration within health services research.
The Charlson index was developed to predict 1-year patient mortality using comorbidity data obtained from hospital chart review [7]. The derivation cohort was 604 medical inpatients admitted to a New York teaching hospital during 1 month in 1984. The validation cohort was 685 breast cancer patients at a Connecticut teaching hospital from 1962 to 1969. The final Charlson index score was the sum of 19 predefined comorbidities that were assigned weights of 1, 2, 3, or 6. These weights were based on the magnitude of the adjusted relative risks associated with each comorbidity in a Cox proportional hazards regression model (Table 1). Three conditions (liver disease, diabetes, and neoplasm) had different weights based on disease severity. The relative risk of 1-year mortality for each increasing point of the Charlson index was 2.3 (95% confidence interval 1.9-2.8), and the overall model was a highly significant predictor of mortality (P < .0001). At least 9 studies, representing more than 30 000 patients, have validated the Charlson index in a wide variety of diseases for numerous clinical outcomes [6].
Other researchers adapted the Charlson index to obtain comorbidity data from computerized hospital discharge abstract databases coded according to the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) to predict short-term mortality. Administrative database adaptations of the Charlson index are now commonly used for risk adjustment in critical care health services research [8], [9], [10], [11]. However, the literature supporting this method of risk adjustment in critical care has not been thoroughly reviewed. We undertook a systematic review of the literature with 3 specific objectives: (1) to assess the agreement between Canadian ICD-9 administrative databases and chart review for Charlson comorbidity data, (2) to summarize the existing administrative database adaptations of the Charlson index and compare the discriminative ability of each for predicting mortality, and (3) to compare the discriminative ability of the Charlson index versus other risk adjustment methods for predicting inhospital mortality of critically ill patients.
Section snippets
Search strategy
The literature search was conducted as of July 15, 2004, using PubMed and Ovid (release 9.1.0) software for searching the following electronic databases: MEDLINE (from 1965), EMBASE (from 1980), CINAHL (from 1982), and The Cochrane Library (issue 2, 2004). We retrieved all citations related to the Charlson index (search term: “Charlson”) and all citations related to comorbidity adjustment and critical care using medical subject heading search (MeSH) terms: comorbidity, intensive care units
Study selection
The literature search revealed 2163 citations for review: 2157 from the electronic database search and 6 from the hand search (Fig. 1). A total of 2135 citations were excluded because they were not applicable to the previously described study objectives. Abstracts for the remaining 28 studies were obtained, and an additional 12 abstracts were excluded because they were not applicable. Full articles were retrieved for the remaining 16 abstracts. Of these articles, 10 met at least 1 of the
Discussion
The Charlson index is the most widely used method for predicting patient mortality based on comorbidity data [6], [25]. This systematic review has 3 major findings based on its original objectives. First, in comparison to chart review, Canadian ICD-9–coded discharge abstract databases demonstrated a modest positive predictive value but a high negative predictive value for individual comorbidities of the Charlson index in 2 studies. The ICD-9–coded administrative databases were not designed for
Conclusion
The Charlson index and its adaptations for use with administrative databases discriminate mortality similarly. Although mortality prediction can be improved by using physiological data, this information is not generally available in administrative data sets. The decreased precision in risk adjustment using the Charlson index must be weighed against the advantages of population-based research in determining the most appropriate study design for a specific research question. Further validation of
Acknowledgment
The authors thank Dr Charles Flexner for assistance with developing the idea for this manuscript.
References (42)
- et al.
Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives
J Clin Epidemiol
(1993) - et al.
Prediction of survival of critically ill patients by admission comorbidity
J Clin Epidemiol
(1996) - et al.
Evaluation of two competing methods for calculating Charlson's comorbidity index when analyzing short-term mortality using administrative data
J Clin Epidemiol
(1997) - et al.
How to measure comorbidity. A critical review of available methods
J Clin Epidemiol
(2003) - et al.
A new method of classifying prognostic comorbidity in longitudinal studies: development and validation
J Chronic Dis
(1987) Does blinding of readers affect the results of meta-analyses? University of Pennsylvania Meta-analysis Blinding Study Group
Lancet
(1997)- et al.
Practical considerations on the use of the Charlson comorbidity index with administrative data bases
J Clin Epidemiol
(1996) - et al.
Co-morbidity data in outcomes research: Are clinical data derived from administrative databases a reliable alternative to chart review?
J Clin Epidemiol
(2000) - et al.
Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases
J Clin Epidemiol
(1992) - et al.
Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data
J Clin Epidemiol
(1996)
Further evidence concerning the use of a clinical comorbidity index with ICD-9-CM administrative data
J Clin Epidemiol
Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: a response
J Clin Epidemiol
Acute respiratory failure in the United States: Incidence and 31-day survival
Chest
A comparison of administrative versus clinical data: coronary artery bypass surgery as an example. Ischemic Heart Disease Patient Outcomes Research Team
J Clin Epidemiol
The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults
Chest
Co-morbidity data in outcomes research: are clinical data derived from administrative databases a reliable alternative to chart review?
J Clin Epidemiol
Comparing clinical information with claims data: some similarities and differences
J Clin Epidemiol
Risk adjustment in claims-based research: The search for efficient approaches
J Clin Epidemiol
Risk adjustment using administrative data: Impact of a diagnosis-type indicator
J Gen Intern Med
Risk adjustment for medical effectiveness research: an overview of conceptual and methodological considerations
J Investig Med
Intensive care unit admission has minimal impact on long-term mortality
Crit Care Med
Cited by (311)
Associations of individual chronic diseases and multimorbidity with multidimensional frailty
2024, Archives of Gerontology and GeriatricsRisk of Personality disorders among childhood maltreatment victims: A nation-wide population-based study in Taiwan
2022, Journal of Affective DisordersIt's Just a Toe Amputation
2022, Journal of Foot and Ankle SurgeryRisk of psychiatric disorders in patients taking isotretinoin: A nationwide, population-based, cohort study in Taiwan
2022, Journal of Affective Disorders
- ☆
Dr Needham holds Clinician-Scientist Awards from the Canadian Institutes of Health Research and the University of Toronto Department of Medicine, and a Detweiler Travelling Fellowship from the Royal College of Physicians and Surgeons of Canada. Dr Pronovost is supported, in part, by the Agency for Healthcare Research and Quality (grant number U18HS11902-01). Dr Laupacis is supported by a Senior Scientist Career Award from the Canadian Institutes of Health Research.