Improving Developmental Screening Documentation and Referral Completion
Abstract
BACKGROUND AND OBJECTIVES: Screening, early identification, and referral improves outcomes for young children at risk for developmental delays. Effective developmental screening processes should include efforts to ensure referral completion and documentation of evaluation results and service eligibility in the child’s medical record. Our objectives were to improve provider documentation of actions taken after an abnormal developmental screening result and increase Early Intervention (State Part C) referrals.
METHODS: Various strategies including an electronic medical record template, monthly clinical informatics reporting, and a phone follow-up after an abnormal screening result were implemented to enhance provider documentation of screening results and improve referral actions and outcomes.
RESULTS: Of the children eligible for screening (n = 3023), 2610 (86%) were screened, with 382 (15%) scoring in the abnormal range. With phone follow-up, 50% of the abnormal screenings were referred to community resources, including 43% to Early Intervention (EI), in contrast to 20% community referrals and 13% EI referrals with the screening template only (P < .0001). Provider documentation of EI outcomes increased when screening templates and follow-up calls were implemented together (31%) as compared with using the screening template alone (15%).
CONCLUSIONS: Enhanced documentation of developmental screening efforts using screening templates and clinical informatics reporting in combination with phone follow-up after an abnormal screening result improved developmental screening outcomes, including referral rates, completed evaluations, and provider documentation of EI services. Such strategies can be effectively used in pediatric primary care settings to improve screening processes and ensure that young children access appropriate services.
- EI —
- early intervention
- EMR —
- electronic medical record
- RSU —
- referral status update
Developmental screening promotes timely referral and access to EI services needed to improve long-term outcomes.1–4 In 2006, the American Academy of Pediatrics issued a policy statement recommending routine developmental screening using an empirically validated tool at 3 visits before age 3 years.5 A variety of screening tools are used at well-child visits,6 although identification rates differ by tool.7 Standardized screening tools improve pediatric primary care providers’ ability to identify children at risk for developmental delay and increase referrals to community services.8,9
Although rates of developmental screening increased considerably between 2002 and 2009, more than half of pediatric primary care providers report not using standardized developmental screening instruments in their practice.10 Between 28% and 61% of children who have abnormal screens actually get referred for a more comprehensive evaluation.11,12 Thus, many children who have developmental delays do not receive services until their delays manifest more profoundly or when they experience significant disruptions in functional abilities.13
In August of 2007, our clinic transitioned from using a developmental milestone checklist embedded in well-child visit templates to implementing universal developmental screening with a standardized screening tool, the Ages & Stages Questionnaires®, Third Edition.14,15 We attributed implementation successes to a launch that included intensive provider and staff training, onsite support from integrated developmental specialists, and an awareness building campaign incentivizing personnel for distributing, scoring, and documenting developmental screening efforts. Within 2 months of implementation, our screening rate surpassed 80%. These screening practices had changed one year before initiating the quality improvement effort described below.
Although our implementation efforts yielded high screening rates, rates of identification, referrals, and documentation of referral completion were considerably lower, even with ongoing training efforts and availability of onsite developmental specialists. We uncovered numerous challenges to our clinic’s screening processes. Screening processes include identifying children at risk for developmental delay, referring these children for evaluation, and documenting service eligibility. We designed and implemented the quality improvement efforts described below to improve screening processes, including systematic documentation by providers of screening results and actions taken after abnormal screening results.
The goals of our quality improvement efforts were to (1) improve provider documentation of screening and referral efforts; (2) identify actions taken after abnormal screening results; (3) design an intervention to improve referral outcomes, including referral to Early Intervention (EI, the State Part C Program), referral completion, and documentation of referral outcomes; (4) evaluate the impact of the intervention by comparing referral outcomes throughout the intervention period; and (5) develop recommendations for practice change based on our findings.
Methods
Setting
The intervention was implemented in a large urban children’s hospital teaching clinic serving children at high risk and their families. The clinic is a primary training site for 42 general pediatric residents, 20 family medicine residents, 24 physician assistant students, and ∼60 medical students. The clinic provides ∼22 000 primary care visits annually and uses an electronic medical record (EMR) to document care. More than 80% of children seen are publicly insured. This population has difficulty accessing community resources, and therefore was considered likely to benefit from efforts to improve referral outcomes.
The clinic houses an integrated mental health services program staffed by child psychologists, psychiatrists, mental health clinicians, and mental health trainees. The program provides direct clinical services and conducts training and education for pediatric health professionals.
Developmental screening is conducted at all well-child visits between age 6 months and 5 years using the Ages & Stages Questionnaires®, Third Edition. When a child scores in the abnormal range in 1 or more domains, the screening results are considered abnormal. For this project, we examined screening processes during well-child visits between age 6 months and 3 years. We divided the intervention into 3 segments: (1) Screening Template Baseline (baseline), (2) Phone Follow-up plus Screening Template (phone follow-up), and (3) Screening Template Post (post). Each segment included an independent group of children, with the first screening-eligible well-child visit occurring during different date ranges; baseline fell between October 1, 2008 and February 27, 2009, phone follow-up between March 1, 2009 and March 31, 2010, and post between April 1, 2010 and December 31, 2010.
Confidential patient information was used to provide services in the context of routine clinical care. Analyses were conducted on de-identified data. The project was approved by the Institutional Review Board at the University of Colorado.
Planning the Quality Improvement Intervention
The first step of our quality improvement effort involved embedding a developmental screening template into the EMR templates for well-child visits (October 2008; seeSupplemental Appendix for sample text) to enable systematic documentation of screening and referral efforts including actions taken by providers for abnormal screening results. Providers were trained to use the template and an electronic referral form for EI services. The template also allowed us to extract bimonthly reports on screening outcomes, including (1) results of the screen abnormal (yes/no), (2) referral made (yes/no), and (3) 2-month follow-up visit scheduled (yes/no). Screening and referral outcomes and additional data obtained through chart reviews were tracked in a Microsoft Access database. Research assistants reviewed charts and documented reasons for no referral or type of referral made after an abnormal screening result (Step 2).
Data tracking revealed that although the screening templates improved documentation, information about referral outcomes was rarely documented in the medical record. In our clinic, pediatric providers were predominantly using passive monitoring, documenting evaluation and eligibility for services if and when families returned to clinic and reported outcomes to providers. When they returned for a follow-up visit, families frequently reported a breakdown between the referral from primary care and uptake of EI services, indicating no connection with the EI system, despite having been referred.
We revamped our provider education efforts to distinguish between “wait and see” and “watchful wait” approaches after discovering that the most common reason for no referral was “follow-up at next visit.” With a “watchful wait” approach, providers were trained to schedule a follow-up visit before the next well-child visit to reassess developmental progress and refer when indicated.
In reviewing the bimonthly data reports, we determined that educating providers would not adequately improve rates of referrals, particularly in cases in which families were seeing different providers at subsequent clinic visits. As a result, our team developed the phone follow-up intervention to facilitate appropriate referral after abnormal screening results and enhance provider documentation of referral outcomes.
Phone Follow-up
In March 2009, psychology fellows, who were providing integrated behavioral health services in our clinic and were trained in early childhood development, began reviewing charts and identifying children who had abnormal screening results and ambiguous documentation of referral outcomes (Step 3). Fellows initiated follow-up phone calls to families of children who had abnormal screening results and no referral or when the actions taken after an abnormal screening result were ambiguous (eg, informatics report indicated an EI referral was made but no documentation existed or the action taken by the provider did not include a referral). The purpose of the call was to discuss screening results, determine whether an EI referral had been made, and, if indicated, refer to early intervention services. Fellows conducted phone follow-ups between March 2009 and March 2010. TheSupplemental Appendix includes an EMR template that fellows used to document their follow-up calls. Fellows stopped making follow-up calls in April 2010 to allow us to study whether phone follow-up calls were improving referral outcomes enough to justify ongoing allocation of fellow time toward this effort.
Data Collection
Developmental screening template baseline data were collected between October 2008 and February 2009. Charts were abstracted in the manner described above. Data collection using the developmental screening template continued during and after the phone follow-up segment ended (April 2010 through December 2010). TheSupplemental Appendix depicts the workflow and number of participants during each segment and details the phone follow-up procedures.
Planning the Study of the Intervention
We developed a tracking system in Microsoft Access to import bimonthly informatics reports with developmental screening template data and to conduct additional medical record abstractions. Research assistants reviewed charts to determine whether appropriate referrals had been made and recorded the actions taken by providers. Charts requiring additional referral actions were imported into a second Microsoft Access database used by the fellows to document whether an intervention phone follow-up was necessary.
Administrative database reviews monitored whether phone follow-up was being conducted within one month after initial abnormal screening. Phone follow-up was completed for 74.67% of families eligible for calls (n = 45). The intervention was delivered during a single phone call. Multiple phone call attempts were made (median = 2) if families were not home or unavailable when the initial phone call was placed. Those families who were not reached by phone were mailed a letter with information about referral for EI.
The phone follow-up was designed to improve the referral outcomes for families whose children scored in the abnormal range on the developmental screener during a well-child visit. We expected phone follow-up to enhance referral outcomes, improving referral completion (ie, evaluation and eligibility determination) in the EI system. To assess the impact of the developmental screening template and the phone follow-up segment, we tracked screening and referral outcomes during baseline, the phone follow-up segment, and for 6 months after follow-up calls were no longer being made (post), comparing outcomes across segments of the intervention. The screening template baseline and post segments provided a valid sampling of routine screening practices in our clinic. We selected phone follow-up as a strategy to improve referral outcomes because it is routinely used in pediatric primary care as a mechanism for care coordination, communication, and resulting of referrals.
Methods of Evaluation
Research assistants reviewed charts for provider documentation of developmental screening processes, including actions taken on abnormal screening results, referrals, and eligibility for EI services (Step 4). Chart reviews for referral results occurred between 2 and 8 months after the initial referral was made to allow adequate time for future clinic visits and documentation of referral results.
Analysis
Comparisons between baseline, phone follow-up, and post segments were made by using Mantel-Haenszel χ2 tests and Fisher’s exact tests. Demographic variables included age, gender, ethnicity, language, and financial/insurance type. Outcome variables included screening rates, action taken by pediatric provider based on abnormal screening results, EI and other community referrals, and EMR documentation of disposition based on referral.
Results
Population Characteristics
The sample had slightly more males than females (Table 1). Approximately 53% of the sample identified as Hispanic/Latino, and 26% of families reported Spanish as their primary language. The population was primarily publicly insured (79%). Older children were more likely to be screened (P < .0001). Children who were male, older, and identified as Hispanic or Latino were more likely to have an abnormal screening result (data not shown in Table 1). There were no demographic differences in children who were identified as having an abnormal screen across segments of the intervention.
Demographics for Total Clinic Population of Children Eligible for Screening at Well-Child Visits and Comparison of QII Population by Screening Period
Developmental Screening Rates
During the quality improvement intervention period, 2610 of the 3023 (86%) children eligible for screening had documentation of screening in the EMR. Screening rates were lower during baseline (71%, n = 682/955) than during phone follow-up (98%, n = 753/770) and post (91%, n = 1175/1298) segments (P < .0001). We considered a “case” as the first well-child visit with a completed developmental screener. Overall, 15% (n = 100/682) of children scored in abnormal range during baseline, whereas rates were 14% (n = 106/753) during phone follow-up and 15% (n = 176/1175) during post.
Action Taken on Abnormal Screens
Before implementation of phone follow-ups (baseline), 20% (n = 20/100) of children who had abnormal screening results were referred to community resources. With phone follow-up, referrals increased to 50% (n = 53/106) and maintained at 51% (n = 89/176) during post (P < .0001). Table 2 shows the actions taken by primary care providers when screening results were abnormal. During baseline, 28% of providers did not make any referral and 43% recommended following up at the next visit. In contrast, far fewer providers made no referrals during phone follow-up (12%) and post (6%), and fewer, 31% and 28%, respectively, recommended following up at the next visit. Referrals to EI increased from 13% at baseline to 43% with phone follow-up and 39% post.
Action Taken on Abnormal Screening Results (n = 382)
EI Referrals
Figure 1 shows increased documentation of EI referrals with phone follow-up (42%, n = 45/106) and post (40%, n = 70/176) when compared with baseline (13%, n = 13/100; P < .0001). During baseline, no EMRs had documentation (n = 0/13) of completed EI evaluations, whereas 38% (n = 17/45; P < .01) of the phone follow-up and 19% (n = 13/70; P < .05) post had documentation of evaluation completion. With phone follow-up, 31% (n = 14/45) of primary care providers documented EI referral disposition, whereas only 15% (n = 2/13) did so at baseline. Rates of provider documentation of referral disposition were maintained after phone follow-up had stopped (31%, n = 22/70). Documentation of children who qualified for EI services was higher during phone follow-up (31%, n = 14/45) than baseline (0%, n = 1/13; P < .05) or post (13%, n = 9/70; P < .05).
EI referrals for abnormal screening results: completed evaluations and primary care provider documentation.
During the intervention period, we met regularly with representatives from the State Part C Program to improve communication of referral outcomes to primary care providers. Although the State Part C Program had been sending Referral Status Update (RSU) forms with information about referrals, the forms were not being returned to providers. Process changes included listing a single provider’s name and fax number as the point of contact for the entire clinic and having our team process the RSUs for scanning into the EMR. As a result, documentation of RSU forms in the EMR increased from 23% (n = 3/13) during baseline to 60% (n = 27/45) with phone follow-up and 79% (n = 55/70) post (overall P < .0005; data not shown). However, as seen in Fig 1, providers rarely documented the status of EI referrals, suggesting that they likely did not know about referral outcomes even when the information was stored in the EMR.
Discussion
This article describes a quality improvement intervention targeting provider documentation of screening processes and referral outcomes. Initiation of a developmental screening template in the EMR in combination with a phone follow-up increased referrals to the State Part C Program and improved provider documentation of completed EI referrals and evaluations. The intervention involved the development and implementation of a screening template, use of clinical informatics reports to collect and analyze EMR data, medical record abstraction, and telephone follow-up by postdoctoral psychology fellows. When providers were documenting using a developmental screening template, children who had abnormal screening results were less likely to be referred to EI services and had less documentation regarding referral outcomes (eg, evaluation completion, eligibility for services) in their EMRs than when providers used the screening template and families also received phone follow-up after an abnormal screening result.
The findings suggest that young children at risk for developmental delay may remain unidentified, or when identified, may not be referred for evaluation even with enhanced documentation strategies and extensive training and education efforts. Identifying developmental delays is necessary but not sufficient to ensure that appropriate referrals are made, services are obtained, and primary care providers know the outcomes of referrals. During the screening template baseline, nearly a third of providers did not take any action based on abnormal screening results and the most common action taken was to recommend a follow-up visit (43%). With phone follow-up and ongoing use of the developmental screening template, providers were less likely to use the “wait and see” approach and were more likely to make appropriate referrals. On the whole, providers were also more likely to document referral disposition during the phone follow-up segment than during baseline and post. Unfortunately, documentation of EI outcomes was missing from most charts during baseline and post, suggesting that screening templates embedded in the EMR did not sufficiently alter provider practice around resulting referrals after abnormal development screening. Even with enhanced EMR templates, providers were rarely following up with families and documenting the results of referrals for community-based EI services.
Without early identification and entry into a comprehensive and coordinated service system that includes a medical home, infants and young children who have special health care and developmental needs and their families are at significant risk for poor health and developmental outcomes.16–18 Importantly, in the context of a medical home, pediatric providers need to know about, discuss with families, and document the child’s involvement with community services. Our findings suggest that ongoing efforts may be required to promote pediatric providers’ gathering and documenting developmental screening referral outcomes.
Lessons learned from this project included the utility of creating functional templates in the EMR that were provider-friendly for clinical documentation and enabled automated data collection and reporting. Even with such strategies, we had to manually review records to extract information embedded within clinical documentation (eg, actions taken on abnormal screening results). Periodic chart reviews also yielded information about provider inconsistencies in using standardized EMR templates. Based on these reviews, we initiated quality improvement efforts to increase documentation compliance. The current project used postdoctoral fellows to conduct phone follow-up, which required allocating fellow time toward this effort and away from direct clinical service. In reviewing our results, we determined that fellow time was better spent providing direct care, as phone calls could be more efficiently and sustainably made by office staff who were responsible for care coordination and referrals. Strategies such as having all referral communication funneled through a single point of contact in the practice and using EMR features like best practice alerts would enhance tracking and improve documentation of referral outcomes in the medical record.
Among the strengths of these efforts was our ability to leverage findings at the community level. Ongoing statewide efforts related to developmental screening processes identified numerous gaps and barriers, including review and documentation of screening results, explanations to families about referral process and services, lack of follow-up post-referral, lack of communication between EI programs and primary care providers, and challenges with tracking and managing screening and referral data. Our findings have been used to improve local and state referral and tracking systems in collaboration with Colorado’s Assuring Better Child Health and Development program.
The findings were limited by a few factors. First, the intervention was conducted in a busy residency training clinic where provider experience with developmental screening processes and local resources varied greatly. Although screening rates (86% screened) suggest excellent adherence to screening protocols, trainee turnover and resident scheduling may have resulted in lack of continuity of care that impeded follow-up with families and, consequently, interfered with resulting of referrals and documenting referral outcomes. The Individuals with Disabilities Education Act19,20 laws allow for a 45-day evaluation process during which service eligibility is determined. Even with a 2-month follow-up window, many families do not return to clinic with completed evaluations and eligibility determinations within the allotted time frame. Additionally, our data were limited to information documented or abstracted from our EMR. Finally, the intervention required resources including dedicated time for clinical informatics specialists, research assistants, and clinical personnel to conduct the intervention and gather, track, and analyze data. Community pediatric settings interested in replicating this approach would need to dedicate staff and resources toward these efforts.
Conclusions
Even in a clinic with high rates of developmental screening, we found that screening, identification, referral, and the use of a developmental screening template in the EMR did not adequately facilitate access to EI services for children at risk for developmental delays. Successful developmental screening processes require resulting referrals and documenting outcomes in medical records. Primary care providers’ knowledge and documentation of referral outcomes enables providers to “close the loop,” thereby ensuring that young children receive necessary services and supports and that information about such services is provided to the child’s medical home.21 Implementing a developmental screening template in combination with phone follow-up yielded higher rates of referral and completed EI evaluations and greater documentation of referral outcomes in the patient’s medical record than use of the screening template alone.
Ideally, pediatric primary care providers who are screening and referring to EI services will also have procedures that allow them to monitor and document outcomes of developmental referrals. Ongoing efforts to improve the developmental screening process include identifying personnel to assist with gathering referral outcomes and re-referring as needed. Training and consultation around developmental screening has recently focused on helping providers systematically document reasons for not referring and implement “watchful waits” instead of “wait and see” approaches. Lastly, we continue to work at the systems level to improve communication between primary care providers and community referral sources. Developing processes that enhance information exchange, use technology and existing reporting mechanisms (eg, immunization registries), and decrease the burden on families and primary care providers will strengthen medical homes and facilitate comprehensive, coordinated services for young children.
Acknowledgments
This project was completed with in-kind support from the following organizations: The Irving Harris Program in Child Development and Infant Mental Health, Children’s Hospital Colorado, Children’s Outcomes Research Program, and Colorado Health Outcomes Program.
We are grateful to the providers, staff, and families in the Child Health Clinic at Children’s Hospital Colorado for their contributions and ongoing support.
Footnotes
- Accepted May 7, 2014.
- Address correspondence to Ayelet Talmi, PhD, Associate Professor, Departments of Psychiatry and Pediatrics, University of Colorado School of Medicine, Children’s Hospital Colorado, 13123 E 16th Ave, B130, Aurora, CO 80045. E-mail: ayelet.talmi{at}ucdenver.edu
Dr Talmi conceptualized and designed the study, supervised intervention implementation, directed data collection, analysis, and preparation, drafted the initial manuscript, and reviewed and revised the manuscript; Dr Bunik helped conceptualize the study, oversaw developmental screening implementation, and drafted the results section of the initial manuscript; Mr Asherin developed the database management strategy used to integrate study data, managed data collection efforts, compiled and prepared data tables for the manuscript, and reviewed and revised the manuscript; Mr Rannie developed and implemented the clinical informatics reporting mechanisms used to gather data from electronic medical records, programmed informatics reports, and compiled data for analysis; Dr Watlington participated in study conceptualization and developed and implemented the provider templates (data collection tools) used to gather data; Ms Beaty prepared and analyzed the data and drafted sections of the manuscript; Dr Berman participated in study conceptualization, provided ongoing feedback during study implementation and data analysis, and critically reviewed the manuscript; and all authors approved the final manuscript as submitted.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: All phases of this project were supported with foundation funding from Rose Community Foundation and The Colorado Health Foundation.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
References
- Copyright © 2014 by the American Academy of Pediatrics
