OBJECTIVE: Using a statewide population-based data source, we describe current neonatal follow-up referral practices for high-risk infants with developmental delays throughout California.
METHODS: From a cohort analysis of quality improvement data from 66 neonatal follow-up programs in the California Children’s Services and California Perinatal Quality Care Collaborative High-Risk Infant Follow-Up Quality of Care Initiative, 5129 high-risk infants were evaluated at the first visit between 4 and 8 months of age in neonatal follow-up. A total of 1737 high-risk infants were evaluated at the second visit between 12 and 16 months of age. We calculated referral rates in relation to developmental status (high versus low concern) based on standardized developmental testing or screening.
RESULTS: Among infants with low concerns (standard score >70 or passed screen) at the first visit, 6% were referred to early intervention; among infants with high concerns, 28% of infants were referred to early intervention. Even after including referrals to other (private) therapies, 34% infants with high concerns did not receive any referrals. These rates were similar for the second visit.
CONCLUSIONS: In spite of the specialization of neonatal follow-up programs to identify high-risk infants with developmental delays, a large proportion of potentially eligible infants were not referred to early intervention.
- early intervention
- developmental assessment
- developmental delay
- health service utilization
- neonatal follow-up
- CCS —
- California Children’s Services
- CLD —
- chronic lung disease
- CPQCC —
- California Perinatal Quality Care Collaborative
- DQ —
- developmental quotient
- EI —
- early intervention
- NFU —
- neonatal follow-up
What’s Known on This Subject:
Neonatal follow-up programs are designed in part to identify developmental delays among high-risk infants after NICU hospitalization and make referrals to state-funded early intervention. Early intervention has been shown to benefit children with developmental delays.
What This Study Adds:
Many high-risk infants demonstrating developmental delays at neonatal follow-up programs are not referred to early intervention. Subspecialty clinics share responsibility with the medical home in referring children from birth through three years with developmental problems to early intervention services.
The Individuals with Disabilities Education Act, Part C, provides financial incentives for states to provide early intervention (EI) services to children from birth to 3 years of age who are experiencing developmental delays or have a diagnosed condition with a high probability of resulting in developmental delays. EI encompasses a wide range of state-funded supportive services, such as occupational therapy and educational support. Children benefit from EI with regard to adaptive functioning and quality of life.1,2 National campaigns and policy statements by the American Academy of Pediatrics have promoted the importance of early screening for developmental delays and referral to EI.3,4 No studies to our knowledge, however, have evaluated rates of EI referrals in clinical settings specifically geared toward the care of children at high risk for developmental disorders.
Neonatal follow-up (NFU) programs primarily serve children previously hospitalized in NICUs. This patient group is at increased risk for long-term health and neurodevelopmental impairments, including sensory and motor disabilities, intellectual disability, and behavioral problems.5–8 NFU programs emphasize identifying infants with developmental delays or disabilities early in life by means of standardized developmental testing or developmental screening. The programs also monitor infants with medical problems, such as vision impairment or chronic lung disease (CLD). NFU is not only valuable in assisting children and families with appropriate management plans after NICU discharge but also provides an opportunity to study quality of care outcomes that are of high interest to clinicians, policy makers, and public health officials. Population-based studies are needed to evaluate the effectiveness of NFU programs in addressing the developmental needs of high-risk infants.
The overall goal of this study was to determine EI referral rates among a population of high-risk infants demonstrating developmental delays. Given the responsibility NFU has in providing on-going monitoring of high-risk infants, we hypothesized that high-risk infants with no prior EI, who scored ≥2 SDs below the mean on a standardized developmental test, or failed a developmental screen would be referred to EI during the same visit. To establish a benchmark, we compared EI referrals rates with medical subspecialty referral rates among high-risk infants with known medical conditions. We reasoned that EI referral rates would compare favorably with rates of referrals to ophthalmology among infants with vision impairments and referrals to pulmonology among infants on home oxygen.
The state of California is an important stakeholder in the long-term outcomes of high-risk infants previously hospitalized in the NICU. More than 90% of the NICUs in California are funded in part by California Children’s Services (CCS), a statewide program for children with special health care needs. All CCS-funded NICUs are required to provide an NFU service for high-risk infants. Infants eligible for NFU must meet CCS-approved neonatal medical criteria.9 Children eligible for NFU qualify for a minimum of 3 visits before their third birthday: visit 1 at 4 to 8 months, visit 2 at 12 to 16 months, and visit 3 at 18 to 36 months of age, adjusted for prematurity.
CCS has partnered with the California Perinatal Quality Care Collaborative (CPQCC) in quality improvement activities to improve perinatal outcomes in the state. CPQCC is a network of public and private medical providers, health care purchasers, and public health professionals.10 A High-Risk Infant Follow-up Quality of Care Initiative was formed in 2006 by CPQCC and CCS to identify “quality improvement opportunities” with the goal of reducing long-term morbidity after NICU hospitalization. A real-time data-reporting system was created to track neurodevelopmental, health, and service outcomes of high-risk infants and to evaluate the effectiveness of NFU programs. It contains sociodemographic and child characteristics, service referrals and use (including EI), post-NICU medical history, neurologic examination findings, and developmental status (via a standardized test/screen). Information is abstracted onto standardized forms and entered into a centralized, Web-based reporting system managed by CPQCC. Statewide training and quarterly executive committee meetings ensure consistency, quality, and accuracy of data collection.
Study Design and Participants
We conducted a primary data analysis of the first wave of data from the High-Risk Infant Follow-up Quality of Care Initiative. This was a cross-sectional observational cohort study of high-risk infants born after January 1, 2009, who were seen at NFU visit 1 and visit 2 and were entered into the data set by February 2, 2011.
Developmental Testing or Screening
Programs have discretion on the choice of a standardized developmental test or developmental screener at each visit. The standardized test was selected over a screener if both were done on the same child. All participants who underwent standardized developmental testing received a developmental quotient (DQ) or standardized score based on standard administration and scoring procedures for that particular test. The majority (71%) used the Bayley Scales of Infant Development–3rd Edition, followed by the Capute Scales (20%), Gesell Developmental Observation-Revised (9%), and Mullen Scales of Early Learning (1%).11–14 DQ scores were dichotomized into infants with a DQ ≤70 and infants with a DQ >70 in at least 1 domain of development (eg, motor, language). We selected 70 as the cutoff because ≥2 SDs below the mean indicate severe delay on most standardized developmental tests.
All participants who underwent a developmental screener received either a “pass” or “fail” based on standard administration and scoring procedures for that particular screener. Common screeners included the Bayley-III Screening Test (52%) and the Bayley Infant Neurodevelopmental Screener (23%).15,16
We grouped infants on the basis of results from their developmental assessment: children with DQ ≤70 or failed screen were categorized as “high concern” and children with DQ >70 or passed screen were categorized as “low concern” for long-term disability.
We divided the study population into 4 birth weight groups: ≥2500 g, 1500 to 2499 g, 1000 to 1499 g, and <1000 g. Birth weight and gestational age were highly correlated (r = 0.89; P < .001). A gestational age of 40 weeks was used to determine adjusted age at developmental assessment for preterm infants born <37 weeks’ gestational age.
Neonatal Medical Problems
To characterize the severity of neonatal medical problems, we selected a set of problems as descriptors. Each problem was a dichotomous variable based on definitions determined by using CCS medical eligibility criteria for NFU.9 “Persistently unstable” includes any infant who had “prolonged hypoxia, acidemia, hypoglycemia and/or hypotension requiring pressor support.” “Other neurologic abnormality” included problems resulting in neurologic abnormalities, such as history of central nervous system infection and hypoxic ischemic encephalopathy. Children with vision impairment (eg, retinopathy of prematurity, strabismus) and home oxygen were included in the analysis of medical referrals.
All maternal/family characteristics were dichotomized: white or non-white maternal race, English as a primary or nonprimary language spoken at home, a maternal education of less than a high school degree or a high school degree or greater, and public health insurance (Medi-Cal) or private insurance.
Referral to state-funded (Part C) EI at the NFU visit was the main outcome measure coded as either “yes” or “no.” We also determined if children were referred to other therapies, such as private physical or occupational therapy.
Data were analyzed with SPSS version 19.0 for Macintosh (SPSS Inc, Chicago, IL). We used frequency tables to describe child, neonatal medical, and maternal/family characteristics of the study population and determine the proportion of infants referred to EI. We conducted χ2 analyses to determine associations between child, neonatal medical, and maternal/family factors and developmental concern. EI referral rates were compared among high-concern infants with referral rates to ophthalmology among infants with vision impairment and pulmonology among infants on oxygen support using a χ2 goodness-of-fit test with a hypothesized 80% referral success rate. Multivariate associations were performed between independent variables of interest and referral to services among infants not previously receiving EI. We created bivariate logistic models for 2 dependent variables: referral to EI (model A) and referral to other therapies (model B). Both models included child characteristics, neonatal medical, and maternal/family characteristics that were selected a priori. A P value of <.05 was set as the level of statistical significance. The Stanford University Institutional Review Board exempted this study from review because the data set contained de-identified information.
Figure 1 shows the distribution of the study population according to developmental status at visits 1 and 2. Among the 5129 infants at visit 1, 64% had a standardized developmental test. Of those, 82% had a DQ >70 and 18% had a DQ ≤70 in at least 1 domain of development. Among the 1609 infants who had a developmental screen, 89% passed and 11% failed. The remaining 5% received neither a developmental test nor screen. These proportions were similar in visit 2.
Characteristics of Infants With Developmental Assessments
Table 1 compares the child, neonatal medical, and maternal/family characteristics according to developmental status at visit 1 for those with low and high concerns. Greater proportions of infants with birth weights <1000 g and >2500 g had high concerns than those with birth weights 1000 to 1499 g and 1500 to 2500 g (χ2 = 91.9; df = 3; P < .001). There were no statistically significant differences between the <1000-g and >2500-g infants and between the 1000- to 1499-g infants and 1500- to 2499-g infants. A greater proportion of infants with high concerns had medical problems in the neonatal period when compared with infants with low concerns, including prolonged oxygen support (χ2 = 41.5; df = 1; P < .001), persistently unstable condition (χ2 = 18.3; df = 1; P < .001), intracranial hemorrhage (χ2 = 29; df = 1; P < .001), neonatal seizures (χ2 = 75.8; df = 1; P < .001), and other neurologic abnormalities (χ2 = 37.3; df = 1; P < .001). There was an excess number of infants with high concerns whose mother’s primary language was not English (χ2 = 29.7; df = 1; P < .001), whose mother had less than a high school degree education (χ2 = 15.5;df = 1; P < .001), and who had government health insurance (χ2 = 16.1; df = 1; P < .001). These differences in child, neonatal medical, and maternal/family characteristics according to developmental status in visit 1 were consistent in visit 2.
Referrals to EI
Table 2 shows the EI status of the study population at visits 1 and 2. Among infants with low concerns, 16% were already receiving EI at the time of the visit. At the conclusion of the visit, an additional 4% were referred to EI, 13% to other therapies, 3% to both EI and other therapies, and 80% to neither. In contrast, among infants with high concerns, 33% were already receiving EI by visit 1. At the conclusion of the visit, an additional 8% were referred to EI, 38% to other therapies, 20% to both EI and other therapies, and 34% to neither. Visit 2 showed a similar pattern of distribution. The difference in referral rates between children who failed a screener and children with a DQ ≤70 was not statistically different.
Referrals to Ophthalmology and Pulmonology
Among the 347 infants at visit 1 with vision impairment, 290 (84%) were referred to ophthalmology. Among the 175 infants on home oxygen at visit 1, a total of 140 (80%) were referred to pulmonology. These proportions did not differ significantly from the hypothesized value of an 80% referral rate. In contrast, the proportion of referrals to EI or other therapies among high-concern infants was significantly lower (χ2 = 66.3; df = 1; P < .001). This trend was consistent at visit 2.
Likelihood of Referral to Services
By visit 1, a total of 367 (9%) of the 4170 EI-naive infants were referred to EI. The results of the bivariate regression analyses are shown in Table 3. After adjusting for various child, neonatal medical, and maternal/family variables, high-concern status seemed to be the major predictor of referral to EI or other therapies. CLD during the neonatal period and Medi-Cal insurance were also consistent independent contributors to the likelihood of referral to EI or other therapies. In contrast, older age of assessment was associated with a decreased likelihood of referral to EI but an increased likelihood of referral to other therapies.
Many high-risk infants with a DQ ≤70 or failed developmental screen at NFU were not referred to EI. We expected higher rates of referrals at visit 2 with the rationale that providers would be less inclined to take a “wait-and-see” approach in referring developmentally delayed children >12 months of age. The proportion of nonreferrals to EI, however, was consistent with the proportion at visit 1. We were reassured that an additional 20% to 40% of infants in the high-concern group were referred to other therapies at both visits, but it is alarming that 34% to 37% received neither referrals to EI nor to other therapies.
The failure of eligible children to use EI has been well reported.17–20 In one population-based study of nationally represented children, only 10% of developmentally delayed children received EI at 24 months.19 The current study is the first, to our knowledge, to look at EI referral rates among a population of high-risk infants after NICU discharge. Although the referral rates of our population cohort are higher than what has been reported in the general population, these infants were evaluated at clinics specializing in developmental assessment and care coordination of children with or at high risk for special health care needs. Our findings indicate that NFU programs are missing opportunities to refer potentially eligible developmentally delayed infants to EI. This failure raises an important quality improvement opportunity in the support and management of infants after NICU hospitalization.
More than 30% of infants in the high-concern group had birth weights >2500 g. Previous studies report high rates of service use and unmet needs after NICU hospitalization; however, these studies used small sample sizes or focused on children born <32 weeks’ gestational age or with birth weights <1500 g.21–24 Further analysis of the data will look closely at how the >2500-g birth weight group uses services differently than their lower birth weight counterparts.
Several infants in the low-concern group (16%) were already receiving EI before visit 1. These infants may have strong advocates for parents (>50% had college-educated mothers), been made eligible for EI based on medical risk factors (eg, birth weight <1500 g), or presented with significant delays before NFU but showed rapid improvement with EI. Future analysis will look at factors associated with the referral and receipt of EI, especially among children who do not meet eligibility criteria for EI.
Several sociodemographic factors have been implicated as predictors for low EI use, such as non–English-speaking mothers and poverty.17,25,26 Wang et al20 reported higher EI enrollment among children with birth weights <1000 g who were receiving public insurance. Results from our regression analyses also showed that EI referrals were positively associated with public insurance. This finding suggests that privately insured children are not referred to state or private programs at the same rate as publically insured children. To our knowledge, this is the first study to compare referral rates between Part C and non–Part C services. Infants with CLD demonstrated a high likelihood of EI referral. CLD is a strong predictor of neurodevelopment impairment.27 Interestingly, high-concern status was predictive of EI referral even though a sizable proportion of developmentally delayed children were not referred. The regression models explained <20% of the variance, indicating that other factors, perhaps not captured by this data- reporting system, are important predictors of EI referrals.
Not knowing how many children referred to EI at NFU were actually enrolled in EI is a limitation of the current study. Longitudinal analysis, however, will allow us to estimate EI enrollment and factors associated with lack of referral. Another limitation is absent information about whether children were referred to EI outside NFU. In the general population, primary pediatric providers play a central role in referring developmentally delayed children to EI.28 Families also have the option to make referrals themselves. Some families may be declining the recommendation for referral for reasons ranging from denial of their child’s delays to mistrust of state agencies. A random survey of families in the high-concern group would provide insight as to why some children were not referred or whether someone other than the NFU provider made a referral.
Data abstraction errors are expected in any statewide study involving a heterogeneous mix of centers and programs even with training workshops and accessible technical support. Knowing the extent of these errors is not feasible without investigating beyond the existing data set (eg, chart reviews). Our confidence in the quality of our data grew, however, when EI referral rates were compared with other medical services. Referrals of infants on oxygen support to pulmonology and infants with vision impairment to ophthalmology were more consistent with expectations than referrals of developmentally delayed infants to EI. These differences suggest that quality improvement or systems issues are contributing to nonreferrals of developmentally delayed infants.
NFU providers may be continuing to take a wait-and-see approach after visit 2. Previous research suggests that pediatric providers are more likely to refer children to EI if their delays are severe, if they are >24 months of age, or have an established diagnosis.29–31 In one national survey, the mean time between identification of a developmental delay and EI referral was >5 months.32 The consequence of waiting is a shorter period for the child to benefit from state-funded EI. Part C services are available until 36 months of age when eligibility and services then shift to the school districts and operate with an Individual Education Plan rather than Individual Family Service Plan.
Major changes to California’s EI programs occurred over the course of data collection. Most significant have been massive state budget cuts to fund developmental services for children. The eligibility criteria to qualify for EI in California have changed.33,34,35 The developmental delay threshold is higher, and children who are high-risk for having a developmental disability on the basis of biomedical indicators are no longer eligible. These system changes may be influencing the practice patterns of NFU providers. Consequently, providers may be referring children to private therapies or no services at all. Qualitative methods will add valuable insight into the reasons behind why referrals are not made for high-risk infants with developmental delays. Further investigation into programmatic factors (eg, team composition, regional location) is also needed.
This article targets a specific population of children (children previously hospitalized in the NICU at-risk for developmental disorders) where there is ample research on neurodevelopmental and health outcomes but little on service outcomes. Much is to be learned about how developmental services are used by young children and their families. Although this article focused on children attending NFU programs in California, we are confident that the findings will also be informative to pediatric providers in other states who care for high-risk infants.
The American Academy of Pediatrics has promoted developmental screening and timely referrals to early intervention for children at risk for developmental disorders. This study alerts the academy and other policy makers that subspecialty programs should also be targeted for education and training about the importance of early intervention for children.
Many severely delayed high-risk infants are not being referred to EI at NFU based on these initial findings. Identifying reasons associated with lack of referrals can inform targeted interventions (eg, education, training), substantiate policy change (eg, automatic EI referrals for all high-risk infants), and help develop new standards of care. With budgets for children’s services threatened across the nation, a high priority should be made to evaluate how follow-up care is provided to this patient population. Much has been invested in their survival, but they remain at high-risk for lifelong health and neurodevelopmental impairments.
We thank Ms Erika Gray for her leadership as project manager, Ms Yuan Lin for assistance with data set preparation, and all of the NFU coordinators in California for their efforts in the abstraction and reporting of data.
- Accepted February 14, 2012.
- Address correspondence to Brian G. Tang, MD, Division of Neonatal and Developmental Medicine, Department of Pediatrics, Stanford University School of Medicine, 750 Welch Rd, Suite 315, Palo Alto, CA 94304. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: This research was supported in part by the Clinical and Translational Science Award 1UL1 RR025744 for the Stanford Center for Clinical and Translational Education and Research (Spectrum) from the National Center for Research Resources, National Institutes of Health, MCHB Grant # 77MC09796-01-00, California Children’s Services, and California Perinatal Quality Care Collaborative.
- Bailey DB Jr,
- Hebbeler K,
- Spiker D,
- Scarborough A,
- Mallik S,
- Nelson L
- ↵Centers for Disease Control and Prevention. Learn the signs. Act early. Available at: www.cdc.gov/ncbddd/actearly/. Accessed June 9, 2011
- Duby JC,
- American Academy of Pediatrics Council on Children With Disabilities
- Marlow N,
- Rose AS,
- Rands CE,
- Draper ES
- Shillingford AJ,
- Glanzman MM,
- Ittenbach RF,
- Clancy RR,
- Gaynor JW,
- Wernovsky G
- ↵Department of Health Care Services. High risk infant follow up. Available at: www.dhcs.ca.gov/services/ccs/Pages/HRIF.aspx. Accessed June 1, 2011
- Wirtschafter DD,
- Powers RJ
- ↵The Gesell Developmental Observation-Revised: An Overview. Available at: www.gesellinstitute.org/GDO-R_Overview_Document.html. Accessed March 21, 2012
- Bayley N
- Accardo PJ,
- Capute AJ
- Mullen E
- Bayley N
- Aylward GP
- Rosenberg SA,
- Zhang D,
- Robinson CC
- Hintz SR,
- Kendrick DE,
- Vohr BR,
- Poole WK,
- Higgins RD,
- National Institute of Child Health and Human Development (NICHD) Neonatal Research Network
- Leijon I,
- Finnström O,
- Sydsjö G,
- Wadsby M
- van Zeben-van der Aa DM,
- Verloove-Vanhorick SP,
- Brand R,
- Ruys JH
- ↵Hauser-Cram PW. Early intervention services. In: Carey WB, Crocker AC, Elias ER, Feldman HM, Coleman WL eds. Developmental-Behavioral Pediatrics. 4th ed. Philadelphia, PA: Elsevier; 2009:922–933
- Bailey DB Jr,
- Hebbeler K,
- Scarborough A,
- Spiker D,
- Mallik S
- ↵York A, Riccardi N. Deeper cuts to state budget expected. Los Angeles Times. November 17, 2011. Available at: http://articles.latimes.com/2011/nov/17/local/la-me-state-budget-20111117. Accessed March 22, 2012
- ↵Buchanan W. California leaders say time for cuts may be ending. San Francisco Chronicle. December 18, 2011. Available at: www.sfgate.com/cgi-bin/article.cgi?f=/c/a/2011/12/17/BADB1MD1IH.DTL&ao=all. Accessed March 22, 2012
- ↵State of California Department of Developmental Services. Budget information. Available at: www.dds.ca.gov/Budget/Home.cfm. Accessed March 21, 2012
- Copyright © 2012 by the American Academy of Pediatrics