ReviewImprovement evident but still necessary in clinical practice guideline quality: a systematic review
Introduction
Influencing almost all fields of health care, clinical practice guidelines (CPGs) aim to improve the quality, consistency, and effectiveness of care by applying evidence-based medicine and providing health care practitioners with expert summaries of the most recent evidence [1]. The purpose of CPGs is to bridge the gap between clinical research and clinical practice and should therefore be based on the best scientific evidence and developed using the most rigorous methodology. Since the 1980s, the number of CPGs has increased dramatically. However, over the past 25 years, evidence suggests that CPG quality may be highly variable, if not low in general, and the rigor with which CPGs follow standardized development methods is unsatisfactory [2], [3], [4], [5]. It was therefore prudent for a common, widely accepted, and standardized method to evaluate CPGs to be developed.
An international collaboration, the Appraisal of Guidelines, Research and Evaluation (AGREE), created a tool that can be used to evaluate the methodological quality of CPG development. The newest version, the AGREE II instrument, was released in 2010 and is the only appraisal tool that has been developed and validated internationally [6], [7]. It provides a standardized framework consisting of a semiquantitative scoring system involving 23 items over six domains of methodological quality: Scope and purpose, Stakeholder involvement, Rigor of development, Clarity of presentation, Applicability, and Editorial independence. The updated AGREE II instrument is an evolution of the original AGREE. Several changes were made and are outlined in the AGREE II technical document [8].
The AGREE II instrument and its predecessor have been prominent in the literature for over a decade, thus giving CPG developers a viable and effective framework from which to base their final product on. Unfortunately, concerns regarding suboptimal quality, a paucity of supporting evidence, the exclusion of relevant stakeholders from the development process, compromised editorial independence, and a lack of CPG applicability persist [9], [10], [11]. These concerns may be negatively affecting the uptake, utilization, and efficacy of CPGs in their health care domains [12]. The purpose of this study is to review the quality of CPGs spanning many different health care topics published since 1990 to analyze trends in the quality of guideline development and assess the potential effect of the availability of the AGREE II instrument on CPG quality.
Section snippets
Literature search and study selection
A predefined search strategy was used to obtain potentially relevant literature from the MEDLINE, EMBASE, and Web of Science Core Collection and BIOSIS databases. The search strategy used only terms relating to the AGREE II instrument and CPGs to target articles that used the AGREE II instrument to review CPGs from any medical field. In addition to database searching, a bibliographic list of studies citing the AGREE II instrument (list maintained by the AGREE trust and available for download at
Literature review
The authors retrieved 515 citations of articles using the AGREE II instrument from the AGREE Trust online database. Systematic searching of MEDLINE, EMBASE, and Web of Science Core Collection and BIOSIS databases retrieved 719 citations. EndNote X7 (Thomson Reuters) was used to remove duplicate citations, of which there were 180. Of the remaining 539 articles, 445 were published in English and of potential relevance. Full-text versions of the articles were sourced and assessed, after which
Discussion
Overall, CPG quality has improved considerably over the past 2 decades. Steady improvement was observed in all AGREE II domain scores. Domain scores for Clarity of presentation and Scope and purpose reached acceptable levels, Stakeholder involvement and Rigor of development were considered borderline, and Editorial independence and Applicability scored quite poorly. Particularly troubling are the suboptimal scores in Editorial independence and Rigor of development, as these two domains have
Acknowledgments
J.J.A. was funded by the Schulich School of Medicine Summer Research Training Program. J.C.M. was funded by a Canadian Institutes of Health Research Chair in Gender, Work and Health; and Dr James Roth Research Chair in Musculoskeletal Measurement and Knowledge Translation. Karen O'Neil contributed to editing the final article.
Contributors: J.J.A. and J.C.M. conceived the idea for this research and designed the study together with A.M.G. and R.S.I. J.J.A., A.M.G., and R.S.I. searched the
References (54)
- et al.
Practice guidelines developed by specialty societies: the need for a critical appraisal
Lancet
(2000) - et al.
Rigour of development of clinical practice guidelines for the pharmacological treatment of bipolar disorder: systematic review
J Affect Disord
(2015) - et al.
Appraisal of guidelines developed by the World Health Organization
Public Health
(2014) - et al.
Systematic review of guidelines for the physical management of osteoarthritis
Arch Phys Med Rehabil
(2014) - et al.
Quality assessment of asthma clinical practice guidelines
CHEST
(2013) - et al.
Author's specialty and conflicts of interest contribute to conflicting guidelines for screening mammography
J Clin Epidemiol
(2012) - et al.
A systematic review of recommendations and guidelines for the management of osteoarthritis: the Chronic Osteoarthritis Management Initiative of the U.S. Bone and Joint Initiative
Semin Arthritis Rheum
(2014) - et al.
An independent AGREE evaluation of the occupational medicine practice guidelines
Spine J
(2006) - et al.
Clinical experts or methodologists to write clinical guidelines?
Lancet
(2009) - et al.
A vision statement on guideline development for respiratory disease: the example of COPD
Lancet
(2009)
Clinical practice guidelines: directions for a new program
Are guidelines following guidelines?
AAP Gd Rounds
Guidelines International Network: toward international standards for clinical practice guidelines
Ann Intern Med
Failure of clinical practice guidelines to meet Institute of Medicine standards: two more decades of little, if any, progress
Arch Intern Med
Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project
Qual Saf Health Care
A systematic review of appraisal tools for clinical practice guidelines: multiple similarities and one common deficit
Int J Qual Health Care
AGREE II: advancing guideline development, reporting and evaluation in health care
Can Med Assoc J
The quality of clinical practice guidelines over the last two decades: a systematic review of guideline appraisal studies
Qual Saf Health Care
High methodologic quality but poor applicability: assessment of the AAOS guidelines using the AGREE II instrument
Clin Orthop Relat Res
Applicability of the AGREE II instrument in evaluating the development process and quality of current National Academy of Clinical Biochemistry guidelines
Clin Chem
Why don't physicians adhere to guideline recommendations in practice? An analysis of barriers among Dutch general practitioners
Implement Sci
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement
PLoS Med
Management of falls in community-dwelling older adults: clinical guidance statement from the Academy of Geriatric Physical Therapy of the American Physical Therapy Association
Phys Ther
Quality assessment of clinical practice guidelines for Chagas disease
Rev Soc Bras Med Trop
Evaluation of the quality of clinical guidelines for cardiac rehabilitation
J Cardiopulm Rehabil Prev
Critical appraisal of clinical practice guidelines in pediatric infectious diseases
Int J Clin Pharm
Rigorous method to assess quality and generalizability of clinical practice guidelines
Can J Hosp Pharm
Cited by (0)
Conflict of interest: All the authors declare that they have no conflict of interests.