Abstract
OBJECTIVE: To determine the effectiveness of developmental screening on the identification of developmental delays, early intervention (EI) referrals, and EI eligibility.
METHODS: This randomized controlled, parallel-group trial was conducted from December 2008 to June 2010 in 4 urban pediatric practices. Children were eligible if they were <30 months old, term, without congenital malformations or genetic syndromes, not in foster care, and not enrolled in EI. Children were randomized to receive 1 of the following: (1) developmental screening using Ages and Stages Questionnaire-II (ASQ-II and Modified Checklist for Autism in Toddlers (M-CHAT) with office staff assistance, (2) developmental screening using ASQ-II and M-CHAT without office staff assistance, or (3) developmental surveillance using age-appropriate milestones at well visits. Outcomes were assessed using an intention-to-treat analysis.
RESULTS: A total of 2103 children were enrolled. Most were African-American with family incomes less than $30 000. Children in either screening arm were more likely to be identified with delays (23.0% and 26.8% vs 13.0%; P < .001), referred to EI (19.9% and 17.5% vs 10.2%; P < .001), and eligible for EI services (7.0% and 5.3% vs 3.0%; P < .001) than children in the surveillance arm. Children in the screening arms incurred a shorter time to identification, EI referral, and EI evaluation than children in the surveillance arm.
CONCLUSIONS: Children who participated in a developmental screening program were more likely to be identified with developmental delays, referred to EI, and eligible for EI services in a timelier fashion than children who received surveillance alone. These results support policies endorsing developmental screening.
- ASQ-II —
- Ages and Stages Questionnaire–II
- DS —
- developmental surveillance
- EI —
- early intervention
- M-CHAT —
- Modified Checklist for Autism in Toddlers
- MDE —
- multidisciplinary evaluation
- NS —
- without office staff support
- OS —
- office staff support
What’s Known on This Subject:
Developmental screening using standardized tools has been endorsed by professional groups to improve rates of identification and referral for young children who have developmental delays. Little is known about the effectiveness of these tools among a high-risk urban population.
What This Study Adds:
Using a randomized design, we found that a program of developmental screening improved the percentage and time to identification of developmental delay, referral, and eligibility for early intervention among a poor, racially diverse urban population of young children.
Developmental delays are frequently encountered among young children.1 Twelve percent to 16% of children are estimated to have developmental disorders.2,3 Delayed development occurs when a child does not reach important developmental milestones within an expected period of time in ≥1 domain.1 Developmental delays are associated with medical and genetic conditions, contribute to social and emotional problems, and result in poor educational and functional outcomes.4 Identifying delays and instituting early intervention (EI) services in a timely fashion are of critical importance to optimizing child well-being.5,6
Current rates of detection of developmental delays are substantially lower than their estimated prevalence.7–9 This finding suggests that clinicians underidentify young children who have delays.10 Valid and standardized tools are available to screen for delays,11–14 but in primary care settings, developmental screening is not conducted in a routine and standardized fashion.12,15,16 Less than 50% of pediatricians report screening routinely with standardized tools.17–20 Although parents can assist clinicians in identifying delays over time,21,22 clinicians have been shown to be slow to recognize delays without the aid of tools, even when signs and symptoms are apparent.10,20,23,24
To address this problem, Bright Futures, a national health care promotion initiative, has recommended that clinicians implement developmental surveillance at all well-child visits and conduct developmental screening at 9, 18, and 30 months of age.12,25 An additional recommendation included autism-specific screening at 18 and 24 months of age.26 Screening has been defined as use of standardized tools to identify and refine the risk of developmental delay; surveillance has been defined as a process of recognizing children at risk for developmental delay without the assistance of standardized tools.
Although developmental screening programs have been shown to be feasible and improve identification and referral of children who have possible delays,8,27–30 it is not clear whether screening improves the identification of delays without overidentifying normally developing children. Thus, we undertook a randomized trial of developmental screening to determine if screening improves the percentage and time to identification of developmental delay, EI referral, and EI eligibility compared with developmental surveillance. In addition, we sought to determine if office staff support (OS) for developmental screening is more effective than screening conducted by clinicians alone.
Methods
Setting
The study was a randomized controlled, parallel-group trial conducted from December 2008 to June 2010 at 4 urban primary care practices affiliated with The Children’s Hospital of Philadelphia. All attending physicians, nurse practitioners, and pediatric residents at participating practices were eligible to participate. Children were eligible if they were <30 months old, >36 weeks’ estimated gestational age, with no major congenital anomalies or genetic syndromes, not in home foster care, and not currently receiving EI services. Parents of eligible children and clinicians provided written informed consent, and only 1 eligible child per household was enrolled. The study was approved by the institutional review board at The Children's Hospital of Philadelphia.
Intervention
Caregivers completed a questionnaire regarding demographic characteristics and received randomization assignments. Randomization, performed centrally by the project statistician, was stratified according to age group (<15 or ≥15 months old) within site by using randomly permuted blocks of size 3, 6, and 9 and allocation of 1:1:1 to 3 study arms: (1) developmental screening with office support (OS); (2) developmental screening without office staff support (NS); or (3) developmental surveillance only (DS). Blinded allocation was implemented by using opaque envelopes prepared in advance of recruitment. After allocation, no further blinding occurred, and clinicians and parents were aware of treatment assignments.
In the screening arms, caregivers completed the Ages and Stages Questionnaire–II (ASQ-II) at their child’s 9-, 18-, and 30-month well-child visits and the Modified Checklist for Autism in Toddlers (M-CHAT) at their 18- and 24-month visits. The ASQ-II is a brief, 30-item parent self-report measure with 5 domains (communication, gross motor, fine motor, problem-solving, and personal/social) and a reported fourth to sixth grade reading level. The ASQ-II has a >80% sensitivity and specificity for the detection of developmental delay.31,32 A positive screen on the ASQ-II was indicated when a child’s score on any of the 5 developmental domains was <2 SDs for age. The M-CHAT is a validated, 23-item, parent self-report measure designed to detect autism with similar reading levels, sensitivity, and specificity as the ASQ-II.33 The M-CHAT contains a single domain consisting of 6 critical and 17 noncritical items; a positive screen is defined as a failure of 2 critical or any 3 total items. In the OS arm, trained office staff met with families before their visit to complete the screening tools by using props; these included blocks, cups, Cheerios, pen and paper, small bottle, string, and beads. In the NS arm, caregivers completed the tools without the aid of standardized props either by mail before their appointment or at the time of appointment check-in. Caregivers in the screening arms completed age-appropriate developmental milestones at nonscreening visits, whereas caregivers in the DS arm completed age-appropriate milestones at all well-child visits. Milestones consisted of 8 to 10 questions from 4 domains (gross motor, fine motor, communication, and personal/social) that were developed for each well-child visit.
Electronic decision support was implemented in the screening arms to remind clinicians to complete the ASQ-II at the time of a 9-month (8 months 0 days to 11 months 30 days), 18-month (17 months 0 days to 22 months 30 days), and 30-month (29 months 0 days to 36 months 30 days) well-child visits and to complete the M-CHAT at the time of an 18- and 24-month (23 months 0 days to 28 months 30 days) well-child visit. The alerts provided a link to an age-specific ASQ-II or M-CHAT version and an automated scoring grid to minimize scoring errors. Children in the DS arm and children in the screening arms at nonscreening visits completed age-appropriate milestones that were embedded in well-child templates. Clinicians could screen children in the DS arm at their discretion, but electronic and office staff supports for screening were not available.
Children who failed a screening test or milestone or whose parents had concerns regarding their development could be referred to EI services at their clinician’s discretion. Referrals occurred through completion and faxing of EI health appraisals/prescriptions or through the provision of EI telephone contacts to parents. Agreement between participating practices and EI service agencies permitted sharing of EI outcomes for study participants.
Main Outcomes
The main outcomes were the percentage of children identified as having developmental delays, referred to EI, and eligible for EI services. Children were categorized as having a delay if they failed a developmental screen (ASQ-II or MCHAT) or failed an age-appropriate milestone that would indicate at least a 25% delay in milestone attainment, the accepted clinical threshold for referral at the time of the study. Scores for ASQ-II and M-CHAT tests were determined in accordance with their test manuals.31,33 ASQ-II, M-CHAT, and milestone tests with missing items such that an overall score could not be ascertained were imputed as pass/fail by using multiple imputation models.34 Children were categorized as having an EI referral if an EI health appraisal and prescription were completed or there was written documentation of an EI referral in the medical record. Children were categorized as eligible for EI services if they demonstrated at least a 25% delay in ≥1 developmental domain on a multidisciplinary evaluation (MDE) conducted by local EI agencies.
We examined time to identification of delay, EI referral, and completion of EI referral (MDE evaluation) as secondary outcomes. Time of enrollment occurred at the time of a clinic visit, and children varied in age at the time of enrollment from newborn to 29 months. Time of delay and referral occurred at subsequent well-child visits after enrollment and was determined from electronic health records, whereas time of MDE completion was determined from EI records after referral. Time to identification of delay, time to EI referral, and time to completion of EI referral were defined as time in days from enrollment to first identification of developmental delay, EI referral, and completion of an MDE.
Analysis and Power Calculation
Based on a significance level of α = .025, a 1-sided test, a baseline detection rate of delay in participating practices of 7% by using developmental surveillance, and a 20% loss to follow-up, we estimated that a sample size of 680 in each arm would provide 80% power to detect a 5% improvement in the rates of identification between screening and surveillance arms, the minimum clinically significant difference.
We compared overall differences and pair-wise differences in proportions by using χ2 statistics and mean values by using t tests. For each outcome, Weibull accelerated failure time models were developed to compare time from randomization to identification of delay, EI referral, and EI referral completion.35 These models generated estimated relative time-to-events (accelerated failure time models), confidence bounds, and P values. We planned a priori contrasts involving the 2 intervention arms against the control arm and against each other by using Wald tests. For each model, we determined whether residual imbalances that existed after randomization caused confounding and whether treatment effect varied according to patient gender, age at enrollment, or treatment site; we found none, and we therefore report unadjusted outcomes. There was no adjustment made for clustering in the models because patients did not routinely follow up with the same clinicians. Kaplan-Meier plots (inverted) were prepared to depict outcomes from randomization to events of interest according to treatment arm. Raw data used for Kaplan-Meier plots were analyzed to compare relative times across each of the 3 treatment arms from randomization to a cumulative failure incidence and 10% incidence rate for each outcome. Main outcomes were assessed by using an intention-to-treat analysis.
Results
We enrolled all clinicians at participating sites and enrolled and randomized 2103 patients to 1 of the 3 treatment arms (Fig 1). Upon review of eligibility criteria, 11 (0.5%) patients were excluded for the following reasons: premature (<36 weeks’ estimated gestational age) (n = 3), older than 30 months at the time of enrollment (n = 3), currently receiving EI services (n = 3), and not enrolled at 1 of the participating practices (n = 2). Over the 18-month study period, 53 (2.5%) patients were lost to follow-up. There were no differences in the percentage (range: 97.3%–97.9%) or mean duration (OS: 278.6 days; NS: 281.3 days; DS: 272.5 days [P = .354]) of follow-up across the study arms.
Flow diagram of patients participating in the Translating Evidence-based Developmental Screening (TEDS) trial. A total of 2103 patients were enrolled and randomized to 1 of 3 treatment arms: (1) NS developmental screening (n = 698); (2) OS developmental screening (n = 707); and (3) DS only (n = 698). After review of eligibility criteria, 11 patients were excluded, leaving 2092 in the study. Fifty-three patients (2.5%) did not complete a well-child visit after enrollment and were lost to follow-up. The remaining 2039 patients (range: 97.3%–97.9% according to arm) had at least 1 well-child visit after enrollment and contributed data to the analysis.
Patients in the 3 arms were similar with respect to demographic characteristics (Table 1). Most were of African-American race with mean family incomes less than $30 000. There were no differences (P > .05) in demographic characteristics by practice site (data not shown).
Demographic Characteristics of Study Participants According to Study Arm
By design, a larger percentage of patients in both screening arms completed at least 1 developmental screening test than patients in the DS arm. In pair-wise comparisons, a slightly greater percentage of patients in the OS arm completed a developmental screening test (87.8% vs 81.2%; P < .001) than patients in the NS arm. Few patients (<10%) in the NS arm completed the screen before their visit. There was no difference in the percentage of patients who completed developmental milestones according to screening arm (P = .472).
Overall, 438 (20.9%) patients were identified as having developmental delays. Patients in either screening arm were more likely to be identified with delays than patients in the DS arm (Table 2). This finding was accounted for primarily by ASQ-II failures and M-CHAT failures in the 2 screening arms. The percentage of failures due to milestones was similar across all 3 arms (OS: 17.1%; NS: 15.6%; DS: 12.8% [P = .081]). There was no difference (P = .098) between the 2 screening arms in the identification of delays.
Children Who Had a Developmental Concern, Incurred an EI Referral, Completed an EI Referral, or Were Eligible for EI Services According to Study Arm
A total of 332 (15.9%) children were subsequently referred to EI service agencies. A larger percentage of children in the 2 screening arms were referred than children in the DS arm (Table 2), but there was no difference (P = .245) between the 2 screening arms. Among the 438 patients identified with delays, 254 (58.0%) were referred to EI service agencies. Patients in the OS arm were more likely to be referred if they were identified as having delays (OS: 71.6%; NS: 52.2%; DS: 45.6% [P < .001]) than patients in the 2 other arms. An additional 78 (4.7%) patients who did not fail a developmental screening or milestone assessment were referred for EI services.
A total of 170 (8.1%) children completed an MDE, and 107 (5.1%) children were found to be eligible for EI services (Table 2). Children in the screening arms were more likely to complete an MDE and be eligible for EI services than children in the DS arm. There was no difference (P = .208) between screening arms in EI eligibility. There was no difference in the percentage eligible for services among referred children (OS: 35.0%; NS: 30.5%; DS: 29.6% [P = .15]) or among children who completed an MDE (OS: 71.0%; NS: 62.7%; DS: 50.0% [P = .10]).
Using time-to-event analysis, children in either screening arm incurred a 59% to 68% shorter time to identification of delay, a 64% to 70% shorter time to EI referral, and a 24% to 32% shorter time to EI referral completion than children in the DS arm (Table 3). Children in the screening arms were identified with delays and referred earlier (P < .001), and this difference persisted over time (Fig 2). As a result, children in either screening arm had a greater cumulative incidence of identification of delay and EI referral and a shorter time to an incidence rate of 10% compared with the DS arm. There were only slight differences in EI referral completion over time (P = .03) among the 3 arms and small differences in the cumulative incidence of EI referral completion and time to a 10% incidence rate among the study arms.
Time Ratios, Cumulative Incidence, and Time to 10% Cumulative Failure for Developmental Delay, EI Referral, and EI Referral Completion According to Study Arm
Data were grouped, according to study arm, as follows: A, time to identification of developmental delay; B, time to EI referral; and C, time to completion of EI referral. Differences in time between arms were statistically significant in pair-wise comparisons: OS versus DS only (P < .001, P < .001, and P = .012, respectively), NS versus DS only (P < .001, P < .001, and P = .081), and OS versus NS (P = .098, P = .300, and P = .422).
Discussion
This randomized controlled trial to evaluate a program of developmental screening provides important new information to inform recommendations regarding developmental screening. Previous studies have reported beneficial effects of developmental screening but incurred limitations.8,27–30As a result, the US Preventive Services Task Force concluded that evidence was insufficient to recommend for or against routine developmental screening in young children.36
Our findings are consistent with previous research. Schonwald et al29 found that implementation of developmental screening by using the Parents’ Evaluation of Developmental Skills, a validated developmental screening tool, was associated with an increase in identification of developmental and behavioral problems. Hix-Small et al30 observed an increase in EI referrals among 1428 children aged 1 and 2 years after implementation of the ASQ. Van Agt et al37,38 observed similar benefit of language screening among >11 000 children in the Netherlands.
Despite these results, it has remained unclear whether screening improves early identification of children who have delays without overidentifying normally developing children.39,40 Our results extend previous findings regarding developmental screening to EI eligibility and suggest that standardized tests, when used as part of a screening program, outperform surveillance alone and improve identification of children with delays who are eligible for EI services. Our findings do not suggest that screening overidentified normally developing children when compared with surveillance alone. These results have important implications for EI services, which may anticipate increases in referrals as a result of screening programs. Our findings recognize limitations with screening, as a few children (4.7%) who passed developmental screens were referred for further evaluation. These children may have had psychosocial or biological risk factors that predispose them to developmental delays and required further evaluation. Referral decisions after developmental screening must take into consideration multiple factors that arise and not rely on screening results in isolation.41
Our results raise concerns regarding children who were identified as having delays but were never assessed for EI eligibility. Of the 438 children identified in this study as having delays through screening or surveillance, only 170 (38.8%) were referred and completed an MDE. This finding was due, in part, to a relatively low percentage of referrals among children identified with delays (58.0%) and a low percentage of completed referrals among children who were referred (51.2%) for services. Most (62.9%) children in this study who completed referrals were found eligible for services. These results suggest that screening for developmental problems must be followed by support and further intervention to facilitate referral completion.
Because our study is a pragmatic effectiveness trial, there may be limitations to these results. First, the fidelity of screening administration and scoring by participating clinicians may have been less than optimal. Second, not all children attended visits according to the recommended periodicity schedule, and clinicians had discretion over referral decisions. Thus, the optimal effect of screening on identification, referral, and eligibility may be greater than that found in this study. Third, there may have been contamination between the 2 screening arms, as office staff in the OS arm may have assisted clinicians in the NS arm with screening and referral. Fourth, parents and clinicians were not blinded to treatment assignment. This factor could have resulted in a greater or lower percentage of referrals in the screening arms depending on clinicians’ a priori beliefs regarding screening. However, there was little difference in referral rates between the screening (75.0% vs 78.9%) and nonscreening arms. Finally, the outcome measures for this study were identification, referral, and eligibility for EI. We did not address whether children’s development improved as a result of screening and early identification.
Results from this randomized trial support policy recommendations to implement developmental screening in low-income, racially diverse urban populations, where risk of developmental delay is greater and receipt of services lower.2 The results may not be generalizable to more affluent or less diverse communities that have a lower risk and greater resources to identify and treat children with delays. Insurers that provide reimbursement for developmental screening as part of a preventive services benefits package can expect to improve the early identification of children with delays. Although the use of parent-reported measures can have minimal financial impact,42 EI service agencies should anticipate a greater percentage of children seeking services as a result of screening. However, reasons for lack of follow-through with EI referrals and interventions to improve referral completion should be explored in future research.
Conclusions
Children who received developmental screening as recommended by Bright Futures were more likely to be identified with developmental delays, referred to an EI agency, and found eligible for EI services in a timelier fashion than children who received DS alone. There were few differences in outcomes between children who received DS with and without office staff support. Future efforts should focus on coordinating referral completion between primary care providers and local EI agencies.
Acknowledgments
We thank Anneliese Butler, MSW (Philadelphia Veteran’s Hospital), Casey Morris, BS, Stephanie Meltzer, BSN, Maddie Fromell, BS, Jeanne Shewmaker, BS, Deanna Morelli, BA, and Amy Baxter, BA, who assisted with subject recruitment and data collection. We also thank the staff and clinicians at the following practices affiliated with The Children's Hospital of Philadelphia who participated in the study: Primary Care–Cobb’s Creek, Primary Care–Market Street, Primary Care–South Philadelphia, and Primary Care–University City. Finally, we thank Michael Moore, Sara Molina-Robinson, and the staff at Philadelphia Health Management Corporation who provided EI data.
Footnotes
- Accepted September 14, 2012.
- Address correspondence to James Guevara, MD, MPH, 34th Street and Civic Center Boulevard, The Children's Hospital of Philadelphia, CHOP North, Room 1531, Philadelphia, PA 19104. E-mail: guevara{at}email.chop.edu
Dr Guevara conceived of the study, wrote the grant and protocol, acquired the data, analyzed and interpreted the data, drafted and revised the article, and approved of the final version; Dr Gerdes helped conceive of the study, analyzed and interpreted the data, revised the article for important intellectual content, and approved of the final version; Dr Localio and Ms Huang analyzed and interpreted the data, revised the article for important intellectual content, and approved of the final version; Dr Pinto-Martin helped conceive of the study, analyzed and interpreted the data, revised the article for important intellectual content, and approved of the final version; Dr Minkovitz helped analyze and interpret the data, revised the article for important intellectual content, and approved of the final version; and Ms Hsu, Ms Kyriakou, Ms Baglivo, Ms Kavanagh, and Dr Pati collected the data, helped analyze and interpret the data, revised the article for important intellectual content, and approved of the final version.
This work was presented in part at the annual meeting of the Pediatric Academic Societies; May 1–2, 2011; Denver, CO. The full trial protocol is available upon request from the corresponding author.
The Centers for Disease Control and Prevention and The Commonwealth Fund did not participate in the design and conduct of the study; collection, management, analysis, and interpretation of the data; nor were they involved in the preparation, review, or approval of the manuscript. Dr Guevara had full access to all the data in the study and had responsibility for the decision to submit for publication.
This trial has been registered at www.clinicaltrials.gov (identifier NCT00844246).
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: This study was financially supported by grant R18DD000345 from the Centers for Disease Control and Prevention (primary investigator: Dr Guevara) and The Commonwealth Fund (primary investigator: Dr Pati).
References
- Copyright © 2013 by the American Academy of Pediatrics
In this issue
Jump to section
Related Articles
Cited By...
- Trends in Pediatricians Developmental Screening: 2002-2016
- Screening Tools for Autism Spectrum Disorder in Primary Care: A Systematic Evidence Review
- Well-Child Care Adherence After Intrauterine Opioid Exposure
- Promoting Optimal Development: Identifying Infants and Young Children With Developmental Disorders Through Developmental Surveillance and Screening
- How universal are universal preschool health checks? An observational study using routine data from New Zealands B4 School Check
- Rating early child development outcome measurement tools for routine health programme use
- Ensuring Timely Connection to Early Intervention for Young Children With Developmental Delays
- Thailands national universal developmental screening programme for young children: action research for improved follow-up
- Parental Obesity and Early Childhood Development
- Protective factors for child development at age 2 in the presence of poor maternal mental health: results from the All Our Babies (AOB) pregnancy cohort
- Recommendations on screening for developmental delay
- Follow-up After Screening
- Screening for developmental delay among children aged 1-4 years: a systematic review
- Gestational Age and Developmental Risk in Moderately and Late Preterm and Early Term Infants
- Improving Developmental Screening Documentation and Referral Completion
- A Parent Questionnaire for Developmental Screening in Infants Born Late and Moderately Preterm
- Brief Approaches to Developmental-Behavioral Promotion in Primary Care: Updates on Methods and Technology
- Quality Developmental Screenings Are Essential to Quality Surveillance