- AAP —
- American Academy of Pediatrics
- AGREE —
- Appraisal of Guidelines for Research & Evaluation Instrument
- CPG —
- clinical practice guideline
- IOM —
- Institute of Medicine
Clinical practice guidelines (CPGs) have proliferated over the past 2 decades. The National Guideline Clearinghouse listed 2377 active CPGs in early January 2013, of which 1302 (54.8%) were cataloged as relevant to a pediatric age group.1 CPG production represents a response to inappropriate variation in medical practice in some instances and clinicians’ need for guidance when there is either an overwhelming volume of evidence, much of which may be of uncertain quality,2 or absence of reliable evidence, which exists for many clinical conditions.3 CPGs ideally improve health care quality and outcomes by enhancing translation of research into practice and collaborative decision-making between clinicians and patients.2 In reality, quality of many CPGs has been suspect due to limitations of evidence and, more importantly, shortcomings in processes used to develop them.4,5
National and international efforts to improve CPGs quality include publication in 2003 of an 18-item checklist by the Conference on Guideline Standardization6 and the Appraisal of Guidelines for Research & Evaluation (AGREE) instrument.7 AGREE, which underwent a minor revision in 2009 (AGREE-II), delineates 6 domains: scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability (implementability), and editorial independence (from the funding body). The Institute of Medicine (IOM) built upon this foundation in 2011, adding standards that call for external review, increased rigor of systematic reviews used to supports CPGs, periodic update (limited lifespan) of CPGs, and more robust management of conflicts of interest.2,8
The American Academy of Pediatrics (AAP) has been engaged and acting in parallel with these efforts. AAP was a leader among professional societies regarding CPGs quality with publication of a policy statement on classifying recommendations in 2004,9 which addressed quality of individual studies and aggregate evidence linked to recommendations, balance between benefits and harms of recommendations, and recommendation strength. A subsequent AAP statement on transparency in 2008 made recommendations regarding stakeholder input, conflicts of interest disclosure, evidence review processes, and clarity of recommendations.10 Endorsement of CPGs produced by other groups has included use of the Conference on Guideline Standardization checklist in recent years.
What, then, do we know about the quality of CPGs produced or endorsed by the AAP in recent years? In this issue of Pediatrics, Isaac et al11 describe the quality, based on AGREE-II metrics, of the 28 CPGs listed as active on the AAP Web site on October 1, 2011. The 14 CPGs developed by AAP committees had a mean overall score of 54% vs 47% for 14 CPGs endorsed but not developed by the AAP (P = .01). Scores <50% suggest serious deficiency. The domains of greatest deficiency (≤35%) among AAP-developed CPGs were applicability and editorial independence. Low scores on the latter likely reflect documentation rather than process deficiency. Only 1 of the evaluated CPGs developed by AAP was produced under both the 2004 and 2008 policies.12 AAP CPGs presently in development should exhibit higher quality than the results of Isaac et al11 suggest.
Isaacs et al11 confirm that little progress in quality of child health-focused CPGs was made during the past decade. The only other published analysis of pediatric CPGs evaluated 17 CPGs produced before 2004.13 Scores on the 6 AGREE domains were essentially the same as those found by Isaacs et al11. Our adult medicine counterparts have not fared better. An evaluation of 114 adult-focused CPGs published between 2006 and 2011 found little improvement in quality from the 1990s.14,15 Together these studies indicate that the more rigorous quality standards recently set forth by the IOM indeed are needed.2
What else is being done to improve AAP CPGs quality? CPGs in development are piloting an application that improves clarity of recommendations by guiding developers to address who should do what, how, and why to (for) whom under what circumstances with what level (strength) of obligation.16 This should enhance implementation of recommendations and translation of these into useful quality metrics and decision-support modules. An update of the 2004 statement by the AAP Steering Committee on Quality Improvement and Management, with relevant IOM standards in mind, is near completion.
Further improvements in quality of pediatric CPGs will face significant obstacles. Diverse expertise and much time are required to comply with existing AAP polices and new IOM standards. Professional societies involved in CPGs development, including AAP, may be less able to rely on uncompensated work traditionally supplied by their members, often medical school faculty, as financial constraints abound. Most pediatric CPGs also are produced by groups other than the AAP (only 16 [1.2%] of the 1302 listed by the National Guideline Clearinghouse in early 2013 were developed by AAP).1 AAP policies thus may have little direct influence on overall pediatric CPGs quality. Yet, when it can muster resources needed to develop CPGs, AAP should endeavor to lead by high quality example. CPGs increasingly are being used as sources for metrics of quality of care and provider performance.17 Quality of CPGs therefore should take precedent over quantity.
- Accepted January 15, 2013.
- Address correspondence to Charles R. Woods, MD, MS, Pediatric Infectious Diseases, 571 South Floyd St, Suite 321, Louisville, KY 40202. E-mail:
Opinions expressed in these commentaries are those of the author and not necessarily those of the American Academy of Pediatrics or its Committees.
FINANCIAL DISCLOSURE: The author has indicated he has no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
COMPANION PAPER: A companion to this article can be found on page 732, and online at www.pediatrics.org/cgi/doi/10.1542/peds.2012-2027.
- 1.↵National Guideline Clearinghouse. Available at: www.guideline.gov/search/advanced-search.aspx. Accessed January 9, 2013
- 2.↵Institute of Medicine. Clinical practice guidelines we can trust. In: Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E, eds. Washington, DC: The National Academies Press; 2011. Available at: www.nap.edu/catalog.php?record_id=13058. Accessed January 26, 2013
- 4.↵Institute of Medicine. In: Eden J, Wheatley B, McNeil B, Sox H, eds. Knowing What Works in Healthcare: A Roadmap for the Nation. Washington, DC: The National Academies Press; 2008
- 9.↵American Academy of Pediatrics Steering Committee on Quality Improvement and Management. Classifying recommendations for clinical practice guidelines. Pediatrics. 2004;114(3):874–877
- Shiffman RN,
- Marcuse EK,
- Moyer VA,
- et al
- Roberts KB
- Isaac A,
- Saginur M,
- Hartling L,
- Robinson JL
- Boluyt N,
- Lincke CR,
- Offringa M
- Copyright © 2013 by the American Academy of Pediatrics