Improving Newborn Screening Follow-up in Pediatric Practices: Quality Improvement Innovation Network
OBJECTIVE: To implement a 6-month quality improvement project in 15 primary care pediatric practices to improve short-term newborn screening (NBS) follow-up.
METHODS: At the start of the project, each practice completed a survey to evaluate office systems related to NBS and completed a chart audit. Practice teams were provided information about NBS and trained in quality-improvement methods, and then implemented changes to improve care. Monthly chart audits over a 6-month period were completed to assess change.
RESULTS: At baseline, almost half of practices completed assessment of infants for NBS; after 6 months, 80% of practices completed assessment of all infants. Only 2 practices documented all in-range results and shared them with parents at baseline; by completion, 10 of 15 practices documented and shared in-range results for ≥70% of infants. Use of the American College of Medical Genetics ACTion sheets, a decision support tool, increased from 1 of 15 practices at baseline to 7 of 15 at completion.
CONCLUSIONS: Practices were successful in improving NBS processes, including assessment, documentation, and communication with families. Providers perceived no increase in provider time at first visit, 2- to 4-week visit, or during first contact with the family of an infant with an out-of-range result after implementation of improved processes. Primary care practices increased their use of decision support tools after the project.
- AAP —
- American Academy of Pediatrics
- EMR —
- electronic medical record
- NBS —
- newborn screening
- OOR —
- out of range
- PCP —
- primary care provider
- PDSA —
- QuIIN —
- Quality Improvement Innovation Network
The number of conditions included in newborn screening (NBS) has grown significantly,1 with many states now screening for >50 congenital disorders through dried-blood spots and all states screening for hearing loss. Primary care providers (PCPs) for children play a central role in the process of NBS,2 including ensuring that testing is completed or parents have explicitly opted out of screening, that results are obtained in a timely manner, and that appropriate actions are taken for those with out-of-range (OOR) results, with, when necessary, confirmatory testing and the initiation of treatment. Ideally, this process would occur within the context of a medical home.3 Despite the key role of pediatric PCPs in NBS short-term follow-up, a survey of primary care pediatricians revealed that 28% did not actively seek results of NBS for their patients and presumed that “no news is good news.”4
With the support of the Maternal and Child Health Bureau of the Health Resources and Services Administration, the National Coordinating Center for the Regional Genetic and Newborn Screening Service Collaboratives has developed and maintains web-based resources, called ACTion sheets, or ACT sheets (http://www.acmg.net/AM/Template.cfm?Section=ACT_Sheets_and_Confirmatory_Algorithms&Template=/CM/HTMLDisplay.cfm&ContentID=5661). These decision support tools are endorsed by the American Academy of Pediatrics (AAP) and guide providers through preliminary responses to an OOR newborn screen. In 2008, the AAP published a clinical report3 to describe the responsibilities of PCPs related to states’ public health NBS programs, including 2 algorithms that outline a pathway through the process of NBS follow-up.
The overall goal of this project was to improve short-term NBS follow-up care through use of the ACT sheets. The evaluation was conducted with select members of the AAP’s Quality Improvement Innovation Network (QuIIN), a network of nearly 300 practicing pediatricians and staff with the mission of improving care and outcomes for children and families through the use of quality-improvement science.5
Specific aims of this project were to improve NBS processes by the conclusion of the 6-month project in participating pediatric practices so that 100% of infants received assessment at their first visit to ensure that NBS was conducted, 100% of infants with in-range NBS results had the result documented in the medical record and the results were shared with parents, and 100% of infants with an OOR result received follow-up care according to the condition specific ACT sheet.
This project was approved by the Institutional Review Board of the AAP.
Fifteen primary care practices from 11 states were recruited from the membership of the QuIIN. Each practice was led by a QuIIN member pediatrician. QuIIN members are interested or experienced in quality improvement and are considered early adopters. Team leaders had varying levels of quality-improvement experience before participating in the project (Table 1). Practices were included if they evaluated ≥20 newborns per month. Purposeful selection was used to ensure diversity in practice setting (ie, urban, suburban, rural), practice size, and state.
An Expert Group of geneticists, general pediatricians, and researchers convened under the direction of the QuIIN and developed performance measures based on the AAP Clinical Report.3 These measures are shown in Table 2.
Quality Improvement Design
Practice teams used a systematic approach to improve short-term management processes. Chart reviews were conducted at baseline and over 6 months to track changes in care processes. Specific goals for improving quality of care were set at the level of the individual patient and the practice. Practices were provided with the training, tools, and support to accomplish changes. Practice teams used Plan-Do-Study-Act (PDSA)5 cycles, which allowed for rapid improvement.
Each team of a physician and clinical and administrative staff completed a survey on practice guidelines, policies, and documentation systems. Practices estimated time spent by the provider at the first visit, 2- to 4-week visit, and at first contact for an infant with an OOR NBS result. Teams completed baseline chart reviews for 2 sets of patients: the first 10 patient charts of newborns seen for the first time during March 2010 and all charts of infants identified as having an OOR NBS result during March 2010.
Practices used findings from baseline chart reviews to select areas for change and improvement of care.
After the prework period, core team members from each practice attended an initial 1.5-day face-to-face learning session. Learning session objectives are included in Table 3. At the conclusion of this session, the participants were able to develop a strategy for improving care around short-term management of infants with OOR NBS results, apply the Model for Improvement6 to plan tests of change, and understand and implement a measurement strategy to monitor changes made.
For 6 months, teams used the AAP Clinical Report and algorithm and the American College of Medical Genetics ACT sheets as tools to improve care processes. Teams were provided with a change package with suggested ideas for change as applied to baseline findings. A manual was provided to team members for monthly chart review. On a monthly basis, the practice teams reviewed the first 10 charts of newborns in the practice who were seen for the first time in the previous month and all charts in the practice of infants identified as having an OOR NBS result during the previous month (1183 charts over the 6-month action period). Teams completed a monthly progress report including process changes and ACT sheet usage and usefulness. The QuIIN advisor, trained in improvement methodology and facilitation, facilitated improvement activities and compiled run charts based on data collected from chart reviews.
The QuIIN project team facilitated a monthly conference call with the core improvement team members from each practice. Practice teams shared results of their tests with the QuIIN team, provided feedback on the tools used, and discussed strategies and barriers. The QuIIN advisor provided coaching, and Expert Group members provided a brief educational session. General themes, including statements reflecting practice change, were assessed through the monthly progress reports and conference call notes.
In the last month of data collection, core improvement teams completed a postinventory survey assessing the existing systems in their practices and time spent at initial newborn visit, 2- to 4-week well-child visit, and at the first contact for those infants with an OOR result. Core improvement teams also completed an ACT Sheet evaluation survey, providing information on the usefulness of the ACT sheets. After the 6-month action period, core team members from 7 practices attended an optional 1.5-day face-to-face concluding learning session. Table 3 includes the learning objectives.
χ2 tests of association were used to evaluate changes within practices and across aggregated groups of patients in all practices. We considered P < .05 to be statistically significant. A repeated-measures analysis of variance was used to evaluate monthly provider ratings of impact on clinical and operational NBS processes at the practice level.
The number of physicians in participating practices ranged from 5 to 50. The range of uninsured children ranged from 0% to 10% among the 15 practices. Additional characteristics of participating practices are shown in Table 1.
Practice-level Changes in Processes of Care
Baseline activities of the practices are shown in Table 4. Only 7 practices assessed all infants for NBS at the first visit, and 8 practices documented these results in the chart and shared them with parents.
Changes in the Processes of Care
Policies and Procedures
Practices used several methods to improve processes of care, including adopting practice guidelines, appointing a practice champion, and using electronic reminders on electronic medical records (EMRs). The project provided a useful forum for practices to share information and tools. One physician commented, “After seeing another practice’s written policy, I realized our written policy should probably be more detailed so that anyone new coming to our office could easily follow how we track and document all newborn screens. We are doing it correctly in practice, but not all steps are written clearly.”
During baseline data collection, 1 of 15 practices routinely reviewed the ACT sheets for OOR results and followed these recommendations. By conclusion of the project, 7 of 15 practices had increased their rate of review of the ACT sheets; 5 of these reviewed the ACT sheet for all OOR results and followed the recommendations (Table 5). A major factor brought up by participating pediatric practices was the need for ongoing reinforcement of process change: “Constant surveillance of any protocol is necessary to minimize failures.” Practices that worked within larger networks had to depend on the schedule of network information technology departments for enhancements. EMRs could be inflexible, and it could be difficult to implement changes.
Implementing process change was a whole-practice endeavor. Nurses actively reminded physicians to talk to parents and document conversations; support staff served as practice champions to track results and ensure proper documentation. “When everyone on the entire team is involved in the process of tracking down results, the percentage of results coming back in a timely manner is improved,” noted one physician. Although not all practices used EMRs, the EMR “[guided] other practitioners to be sure to follow-up on the newborn screen and discuss it with parents.” The electronic format prompted physicians with drop-down boxes and flags.
As shown in Table 5, almost half of practices (7 of 15) successfully completed an initial newborn assessment on 100% of infants before the project period. By completion of the project, 12 of 15 practices (80%) reported assessing 100% of infants at the first visit (P = .06). One practice noted, “We have achieved our aim. A procedure is in place for tracking the newborn screens which we will continue to use. This project has helped us feel that we really accomplished something as a staff and has again shown us how to use small projects to make lasting changes.”
In documenting in-range results and sharing them with parents, only 2 practices reported success for 100% of infants before the project, and 4 of 15 practices were successful for ≥70% of their infants; by completion of the project, 5 practices were successful in this activity for 100% of their infants, and 10 of 15 practices reported this documentation/sharing for ≥70% of their infants (P = .03). The change within each practice from baseline to project completion is shown in Table 6. None of the practice characteristics documented in Table 1 appeared to be predictive of successful change. However, both practices that were successful in documenting results and sharing them with parents initially and at the conclusion of the project reported high-quality improvement knowledge at the start of the project. A pediatrician commented: “As we are becoming more skilled at disseminating information with parents regarding NBS, they have become empowered to be active participants in the health care of their children. I find parents now come to the 2- or 4-week visit asking for NBS results, which also serves as a reminder to the practitioner to review the results and share them with the family.” Thus, parents became part of the active reminder system for practices, helping to develop a family-centered medical home.
Aggregate Impact of Changes on Newborns
Fifty-seven infants from all 15 practices had OOR NBS results in the first month (prework phase) of the project, and 3 of these were given a diagnosis of a significant medical condition detected by NBS. Forty-four infants had OOR results in the final month of the project, and 4 of these had a significant medical condition detected by NBS.
As shown in Table 7, infants were more likely to be assessed for completion of NBS and have in-range results documented and shared with parents at the conclusion of the project than at baseline. There was no increase in the likelihood of a chart being flagged because a newborn had not been screened.
Families of newborns with OOR results were more likely to receive condition-specific information at the end of the project, although this difference was not statistically significant (Table 7). The appropriate ACT sheet was more likely to be used by the conclusion of the project, and recommendations were more likely to be followed (Table 7; P < .001). Confirmatory testing or consultation with a subspecialist also increased. There was no change in documentation of false OOR results for the duration of the project (52.6% vs 52.3%). Findings of testing outcomes were reported to the state NBS program more frequently at the conclusion of the project (Table 7), but these results did not reach statistical significance. Estimates of time spent at first newborn visit, 2- to 4-week well-child visit, and first contact for an infant with an OOR result during the prework phase were compared with estimates obtained at the conclusion of the project. Providers did not perceive an increase in time spent for any of these activities after implementation of improved NBS processes, but actual time spent was not documented (Table 8). Providers were asked to rate on a scale of 1 (much more difficult) to 5 (much easier) the impact of the project activities on their clinical and operational work each month. Providers reported that implemented changes significantly improved ease of NBS clinical operations over the course of the study period (Table 9; P < .0001).
This project demonstrated that primary care pediatric practices can improve short-term follow-up of NBS through quality-improvement processes. Most participating practices reported initially that they were not assessing infants’ records at the first visit to determine if NBS had been performed, and they were not documenting in-range results or communicating them with families. By the conclusion of the project, most practices achieved significant improvements in assessing records for the completion of NBS at the first visit, although documentation of results and communications with families was more difficult. The change within each practice setting from baseline to project completion in the percent of infants receiving assessments at first visit for completion of NBS was notable but did not reach statistical significance.
Practice performance related to OOR results was more difficult to assess because of the small number of cases. Most practices did not use ACT sheets before the project, and by the conclusion of the project, a significant improvement in the use of ACT sheets was achieved. When all infants with OOR results were combined across practices, ACT sheet recommendations were followed for 75% of infants at the conclusion of the project compared with 19% (all from 1 practice) during the baseline period.
The PDSA cycle offers a practice the opportunity to take a brief snapshot by selecting the first 10 charts of each month to assess practice performance on a measure of interest. Therefore, some months may yield “zero progress” by chance, particularly for rare events, such as lack of newborn screen or false OOR result. However, the act of routine assessment and communication of results among the practice reinforces awareness of office practice.
This study has several limitations. The participating practices already had an interest in quality improvement, which could limit generalizability. The process measures were not tested for validity or reliability. Outcome data related to management of infants with significant congenital disorders from either the PCP or subspecialists was beyond the scope of this study.
Providers and staff were surprised to discover gaps in their NBS short-term follow-up, particularly in ascertaining that screens had been conducted on their newborn patients. Surprisingly, providers and staff did not perceive an increase in time spent during the first visit, the 2- to 4-week well-child visit, or during first contact with the family of an infant with an OOR result by the conclusion of the project. Although practices invested significant amounts of time to learn about NBS and quality-improvement techniques, and to plan, perform, and analyze PDSA cycles, they reported no increase in time spent with families when surveyed retrospectively. This suggests that systems were put in place to deal efficiently with results of NBS and that improved NBS follow-up need not take additional PCP time.
Regardless of the specific procedures established within practices, systems need to be developed to ensure that NBS follow-up is actually done and communicated to families. Providers and staff found that other practice members needed continuous reinforcement to adhere to changes. Information technology could play a role in reinforcing change. Some practices were able to manipulate EMRs to provide prompts for pediatricians. However, some providers worked in systems with separate information technology departments and had little flexibility in institutional changes to the EMR.
A focused quality-improvement effort within a diverse group of pediatric primary care practices can improve short-term follow-up of NBS results, including communication of results to families. Decision support tools, including the ACT sheets, might be unfamiliar to primary care providers and therefore not used. An educational effort by the AAP and state NBS programs can familiarize providers with these tools, leading to providers’ utilization and more standardization of evaluation and referral for infants with OOR results. QuIIN providers could share experiences through state or regional pediatric professional networks. The experiences of this project have been incorporated into an AAP Education in Quality Improvement for Pediatric Practice course to help pediatricians improve short-term NBS follow-up and obtain Maintenance of Certification Part 4 certification.
We thank the pediatric practices that participated in this project: ABC Family Pediatricians (Trexlertown, PA); All About Children Pediatric Partners PC (West Reading, PA); All Pediatrics (Lorton, VA); Cleveland Clinic Children’s Hospital (Cleveland, OH); CMC Myers Park Pediatrics (Charlotte, NC); Community Medicine Associates (San Antonio, TX); Crozer Pediatrics (Upland, PA0; Flushing Hospital Medical Center (Flushing, NY); Greeley Medical Clinic (Greeley, CO); Mercy Pediatric Clinic (Bentonville, AR); Pediatric and Adolescent Health Partners (Midlothian, VA); Pediatric Associates, PSC (Crestview Hills, KY); Pediatric Clinic, University Health Care (Salt Lake City, UT); Priority Care Pediatrics (Kansas City, MO); and Sandhills Pediatrics (Southern Pines, NC).
- Accepted April 17, 2012.
- Address correspondence to Celia I. Kaye, MD, PhD, University of Colorado School of Medicine, Anschutz Medical Campus, 13001 East 17th Place, Room E1330, Aurora, CO 80045. E-mail:
This project was conceived and designed by Drs Kairys, Thiessen, Glade, and Thompson; Dr Hinton served as member of the Expert Group and assisted in data analysis, interpretation of results, and critical revision of the paper; Dr Neuspiel participated as practice team leader, analysis and interpretation of results, and critical revision of the manuscript; Ms Gubernick served as quality improvement advisor to the project and assisted in analysis and interpretation of results and critical revision of the manuscript; Dr Geleske served on the Expert Group, assisted in analysis and interpretation of results, and provided critical revision of the manuscript; Ms Healy coordinated activities of the Expert Group and assisted in preparation of the manuscript; Dr Kemper served on the Expert Group, assisted in analysis and interpretation of results, and participated in the preparation and review of the manuscript; Dr Saul served on the Expert Group and participated in the preparation and review of the manuscript; Dr Thompson served as a member of the Expert Group and assisted in analysis and interpretation of results and critical revision of the manuscript; Dr Kaye served on the Expert Group, wrote the initial draft of the manuscript, and assisted in analysis and interpretation of results and critical revision of the manuscript; and all authors approved this manuscript for publication.
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention, the American Academy of Pediatrics, the American College of Medical Genetics, or the Maternal and Child Health Bureau of the Health Resources and Services Administration.
FINANCIAL DISCLOSURE: Ruth S. Gubernick was a contracted consultant for this project by the American Academy of Pediatrics. The other authors disclosed no financial interests, relationships, or affiliations for the past 3 years related to the content of this article.
FUNDING: Supported in part through a cooperative agreement between the Maternal and Child Health Bureau/Health Resources and Services Administration and the American College of Medical Genetics, grant U22MC03957, and the American Academy of Pediatrics.
- American Academy of Pediatrics Newborn Screening Task Force
- American Academy of Pediatrics Newborn Screening Authoring Committee
- ↵Desposito F, Lloyd-Puryear MA, Tonniges TF, Rhein F, Mann M. Survey of pediatrician practices in retrieving statewide authorized newborn screening results. Pediatrics. 2001;108(2). Available at: www.pediatrics.org/cgi/content/full/108/2/e22
- Langley GL,
- Nolan KM,
- Nolan TW,
- Norman CL,
- Provost LP
- ↵McDowall D. Interrupted Time-Series Design. In: Jupp V, ed. The SAGE Dictionary of Social Research Methods. Available at: http://srmo.sagepub.com/view/the-sage-encyclopedia-of-social-science-reearch-methods/n446.xml)
- Copyright © 2012 by the American Academy of Pediatrics