BACKGROUND: The Parent Report of Children’s Abilities-Revised (PARCA-R) is a questionnaire for assessing cognitive and language development in very preterm infants. Given the increased risk of developmental delay in infants born late and moderately preterm (LMPT; 32–36 weeks), this study aimed to validate this questionnaire as a screening tool in this population.
METHODS: Parents of 219 children born LMPT completed the PARCA-R questionnaire and the Brief Infant Toddler Social and Emotional Assessment when children were 24 months corrected age (range, 24 months–27 months). The children were subsequently assessed by using the cognitive and language scales of the Bayley Scales of Infant and Toddler Development, Third Edition (Bayley-III).
RESULTS: An average Bayley-III, cognitive and language (CB-III) score and a total PARCA-R Parent Report Composite (PRC) score were computed. There was a large association between PRC and CB-III scores (r = 0.66, P < .001) indicating good concurrent validity. Using Youden index, the optimum PARCA-R cutoff for identifying children with moderate/severe developmental delay (CB-III scores < 80) was PRC scores < 73. This gave sensitivity 0.90 (95% confidence interval: 0.75–1.00) and specificity 0.76 (95% confidence interval: 0.70–0.82), indicating good diagnostic utility. Approximately two-thirds of the children who had a PRC score < 73 had false-positive screens. However, these children had significantly poorer cognitive and behavioral outcomes than children with true negative screens.
CONCLUSIONS: The PARCA-R has good concurrent validity with a gold standard developmental test and can be used to identify LMPT infants who may benefit from a clinical assessment. The PARCA-R has potential for clinical use as a first-line cognitive screening tool for this sizeable population of infants in whom follow-up may be beneficial.
- Bayley-III —
- Bayley Scales of Infant and Toddler Development, Third Edition
- BITSEA —
- Brief Infant-Toddler Social and Emotional Assessment
- CB-III —
- Combined Bayley-III cognitive and language score
- CI —
- confidence interval
- LAMBS —
- Late and Moderately Preterm Birth Study
- LMPT —
- late and moderately preterm (32–36 weeks’ gestation)
- MDI —
- Mental Development Index
- NPV —
- negative predictive value
- PARCA-R —
- Parent Report of Children’s Abilities-Revised
- PPV —
- positive predictive value
- PRC —
- Parent Report Composite
What’s Known on This Subject:
Children born late and moderately preterm are at increased risk of developmental problems compared with term-born peers. Screening for developmental problems in the early years may thus aid in the early identification of children at risk for adverse outcomes.
What This Study Adds:
The Parent Report of Children’s Abilities-Revised has good concurrent validity and 90% sensitivity and 76% specificity for identifying moderate/severe cognitive developmental delay in infants born late and moderately preterm. This parent questionnaire may be used as a clinical screening tool.
In contrast to infants born very preterm (<32 weeks), the outcomes of those born late and moderately preterm (LMPT; 32–36 weeks) have been relatively understudied. Recent studies suggest that these children are at greater risk of neurodevelopmental delay, cognitive deficits, behavioral problems, and special educational needs than term-born counterparts.1–3 Given that LMPT births account for ∼84% of all preterm births,4 there is likely to be a significant public health impact of even mild neurodevelopmental problems for this sizeable population.
Developmental screening in the early years may aid in identifying children at risk and in targeting intervention.5,6 Because face-to-face follow-up is likely to be too costly for all LMPT children, first-line screening to identify a subgroup of children in need of a clinical review may be pragmatic and cost-effective.7 The Parent Report of Children’s Abilities-Revised (PARCA-R), a parent questionnaire of cognitive and language development at 2 years of age, has excellent sensitivity and specificity for identifying neurodevelopmental delay among very preterm children and those with neonatal complications,8–12 enabling its use as a clinical tool and outcome measure for randomized trials.13,14
The PARCA-R may therefore be useful for developmental screening in children born LMPT, but its validity in this population has not been tested. As the developmental sequelae of LMPT birth are less prevalent15 and more subtle than those of very preterm children,16 existing cutoffs used for delineating limits of normality may not be appropriate in this population. The aim of this study was to assess the clinical utility of the PARCA-R as a first-line screening tool for identifying developmental delay in children born LMPT.
Participants and Procedure
A subsample of children was recruited from the Late and Moderately Preterm Birth Study (LAMBS), a prospective population-based study of outcomes following birth at 32+0 to 36+6 weeks of gestation in the East Midlands, United Kingdom. Children were aged 24 months 0 days to 27 months 30 days corrected for prematurity at the time of assessment. Target recruitment was a minimum of 200 infants; this sample size was determined to yield 95% confidence interval (CI) limits for the predicted optimum PARCA-R cutoff score of ±5 points from the point estimate. To ensure data were obtained for 200 infants, we aimed to recruit 250 children to allow for nonattendance and failure to complete the developmental test.
Parents were mailed a questionnaire to complete 7 to 14 days before their child reached 2 years corrected age as part of LAMBS. Parents were then contacted by telephone or e-mail to invite them to participate in the current study and, for those who expressed interest, a home visit was arranged to carry out a study assessment. Inclusion criteria for this study were that the child would be between 24 months 0 days and 27 months 30 days at the time of the developmental test to minimize the time between completion of the questionnaire and the assessment. When a home visit was arranged, a study information leaflet was posted to the parent before the home visit and written parental consent was obtained at the home visit. Parents were sent a feedback letter summarizing their child’s test results after the assessment. The study was approved by the Derbyshire National Health Service Research Ethics Committee.
The 2-year questionnaire comprised the PARCA-R, the Brief Infant-Toddler Social and Emotional Assessment (BITSEA), and questions regarding the child’s use of health care services. The PARCA-R is a parent questionnaire of children’s nonverbal cognitive and language development at 2 years of age.8,9 Nonverbal cognition is assessed by 34 items that are summed to give a nonverbal cognitive subscale score (range, 0–34). Language development is assessed with 2 subscales of the MacArthur Communicative Development Index Words and Sentences17: a 100-word vocabulary checklist (vocabulary scores range, 0–100) and 18 items to assess grammatical competence (sentence complexity scores range, 0–24). These 2 latter scores are summed to produce an overall language score (range, 0–124). The nonverbal cognition and language scores are summed to give a total Parent Report Composite (PRC; range, 0–158) score. In 2 UK studies, PRC scores <49 and <44 had good diagnostic utility for identifying moderate/severe developmental delay in infants born very preterm.8,9
The BITSEA is a parent-completed screening tool for identifying behavior problems and delays in socio-emotional competence. The Problem scale comprises 31 items and the Competence scale 11 items. Higher Total Problem scores indicate greater behavioral problems and lower Total Competence scores indicate delayed socio-emotional development. These scores were compared with published norms to identify children with delayed socio-emotional competence (scores <15th percentile) and behavior problems (scores ≥75th percentile).18
The cognitive and language scales of the Bayley Scales of Infant and Toddler Development, Third Edition (Bayley-III) were administered as the criterion measure from which standardized composite scores were derived (mean = 100, SD = 15). To facilitate comparison between the PARCA-R and the Bayley-III, an average Bayley-III cognitive and language (CB-III) score was computed. CB-III scores <80 have been recommended for classifying moderate/severe developmental delay19 to account for the well-documented underestimation of delay produced by using a conventional cutoff of Bayley-III scores <70.20–22 Three children for whom Bayley-III assessments could not be completed because of severe developmental problems were assigned the lowest Bayley-III scores for each scale (cognitive = 55; language = 47). To enable comparison with previous validation studies, predicted Bayley Scales of Infant Development-II Mental Development Index (predicted-MDI) scores were derived from Bayley-III scores by using a published algorithm.21 Bayley-III assessments were administered in a single session by 1 of 2 psychologists. Ten percent of Bayley-III assessments were scored independently by both examiners to assess interrater reliability with 97% agreement across all items. Psychologists were blind to parents’ PARCA-R responses when conducting Bayley-III assessments.
Data were double entered, verified, and analyzed by using SPSS software (version 20; IBM SPSS Statistics, IBM Corporation). Pearson’s correlations were used to assess concurrent validity of PARCA-R scores with Bayley-III scores. Effect sizes for correlations of 0.1, 0.3, and 0.5 were classified as weak, moderate, and large.23 Classifications of developmental delay on the Bayley-III and PARCA-R were cross-tabulated and predictive values of sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were calculated. To obtain the optimum PARCA-R cutoff in this LMPT population, Youden’s Index was used to identify a cutoff score for predicting a CB-III score <80 that maximized sensitivity and specificity where sensitivity exceeded 80%. Using R,24 the bootstrap percentile method (1000 samples) was used to obtain 95% CI for the optimum cutoff and each predictive value. To examine characteristics of children with false-positive screens, independent samples t tests were used to assess the differences in Bayley-III and BITSEA scores between these children and those with true negative screens. The associations between false-positive screens and health care use were also assessed by using χ2 with effect sizes calculated by using relative risk with 95% CI.
Of 1113 LMPT children, 394 were contacted in succession until at least 250 children were recruited and complete data were obtained for at least 200 children. Overall, the parents of 253 (64%) children provided consent and received home visits. Of the remaining 141 children, the parents of 49 (35%) refused participation and for 92 (65%) children home visits could not be scheduled within the age limit for inclusion in the study. Of the 253 children who received a home visit, 6 (2%) were excluded because the parent had not returned a PARCA-R, and for 16 (6%) the Bayley-III could not be administered because the child was uncooperative; the data from an additional 12 (5%) children were excluded because only 1 child from each multiple birth was included in the analyses to attenuate shared variance for these mother–infant dyads. The final sample thus comprised 219 children born LMPT.
There were no significant differences in gender, gestational age, or birth weight between children recruited and the remainder of the LMPT cohort; however, children recruited were less likely to be multiple births and were less socio-economically deprived on average compared with the remainder of the cohort (Table 1). The final sample of 219 LMPT children had a mean corrected age of 25 months 4 days (range, 24 months 0 days–27 months 24 days) and a mean chronological age of 26 months 4 days (range, 24 months 24 days–29 months 2 days) at the Bayley-III assessment.
Bayley-III scores were close to the normative mean of 100; in contrast, the mean predicted-MDI score was 0.6 SD lower than the normative mean (Table 2). Using CB-III scores <80, 20 (9%) LMPT children had developmental delay. Using predicted-MDI scores <70, 24 (11%) were classified with delay (Table 2). Because the correlation between age and PRC scores was nonsignificant (r = 0.068, P = .316), raw PARCA-R scores were used in subsequent analyses.
There were moderate to large correlations between PARCA-R PRC scores and Bayley-III scores (cognitive r = 0.45; language r = 0.71; CB-III r = 0.66; Fig 1) and predicted-MDI scores (r = 0.64). At the subscale level, there were large correlations between PARCA-R language scores and Bayley-III language scores (r = 0.71), CB-III scores (r = 0.65), and predicted-MDI scores (r = 0.63), indicating excellent concurrent validity for the language scales. For the nonverbal scale, PARCA-R scores were moderately correlated with Bayley-III cognitive scores (r = 0.38), CB-III scores (r = 0.42), and predicted-MDI scores (r = 0.45). All correlation coefficients were significant at the P < .01 level.
Existing PARCA-R cutoff scores developed in very preterm samples performed poorly in this population and were associated with low sensitivity (Table 3). An optimum cutoff of PRC scores <73 (95% CI: 69–86; area under the curve, 0.82) was identified, which was associated with 90% sensitivity and 76% specificity (Fig 2). PPV was lower than sensitivity and specificity, with only 28% of children with PRC <73 having developmental delay. However, the high NPV meant that only 2 children with developmental delay were undetected by PRC scores (Fig 1).
Exploring False-Positive and True-Positive Screens
Because 72% of children with PRC scores <73 had false-positive screens, we explored characteristics of these children to determine whether they represented an at-risk group. Independent samples t tests revealed that Bayley-III and BITSEA competence scores were significantly lower among children with false-positive screens compared with those with true-negative screens (Table 4). The lower mean Bayley-III language score for children with false-positive screens corresponded with the significantly increased use of speech and language therapists by these children.
To determine whether the PARCA-R identifies a group of children with developmental problems who are not already in receipt of health care services, we explored characteristics of children with true-positive screens. Of these 18 children, 5 (27.8%), 1 (5.6%), 3 (16.7%), and 2 (11.1%) had seen a pediatrician, physiotherapist, speech and language therapist, and occupational therapist, respectively and, overall, only 5 children (28%) had accessed health care services in the last year.
The PARCA-R has good criterion-referenced validity for detecting cognitive and language delay and can be used as a first-line screening tool for identifying LMPT-born children who may benefit from a clinical assessment. Concurrent validity was confirmed with large correlations between PARCA-R PRC and Bayley scores. These correlations were of the same order of magnitude as found in previous studies8–12 and exceed those of some other studies in which the validity of parent questionnaires of development have been assessed.25,26 The results of this study thus give weight to previous studies demonstrating the validity of the PARCA-R in preterm populations. Variations in the composition of the parent questionnaires and criterion measures, and in the selection of cutoffs for delay on the criterion measures used, may account for differences in agreement between this and previous studies of at-risk populations.
Consistent with a recent report, although the correlations between PARCA-R and Bayley-III language scores were large, the correlations between the cognitive scales were moderate in magnitude.11 Parents have the opportunity to observe their child in a variety of situations, but a test situation is time-limited and dependent on the current emotional and attentional state of the child, which may affect their performance. A structured assessment challenges all children in a similar way while parents vary in terms of whether they challenge their child’s full range of abilities with some avoiding situations their child may fail.27 This in turn influences toddlers’ motivation for attempting challenging tasks. In contrast, language development at this age is more ostensible, and parents may be better able to recognize their child’s developing language across diverse situations, hence the stronger correlation on these measures.
In the current study, the optimal PARCA-R cutoff for detecting moderate to severe developmental delay (standardized scores <−2 SD) was higher than those of previous studies.8,9,12 This may be the result of the use of different criterion measures with the Bayley-III used in this study and the second edition of the Bayley Scales used in previous studies. In addition, given the gestational age related gradient in outcomes, the cognitive and language development of LMPT infants, although poorer that term-born peers, is better than that of very preterm infants. The distribution of PARCA-R scores will therefore differ in LMPT infants as evidenced by the higher mean PRC score in this study compared with those of very preterm infants. These factors may contribute to the higher cutoff score in this population.
The optimum PARCA-R cutoff score had 90% sensitivity in this LMPT population. Although sensitivity was improved compared with studies of very preterm samples,8,9,12 specificity was marginally lower at 76% and PPV was 28%; consequently, 72% of children had false-positive screens. Other studies have revealed equally low PPVs on developmental screening tests.9,26 The number of false-positive screens could be reduced by lowering the PRC cutoff, but this would increase the number of children with problems who are undetected. The selected cutoffs are thus dependent on the objectives and available resources for further follow-up. For this study we were concerned with clinical utility in which high sensitivity relative to specificity is preferable because the benefits of detecting all children with developmental problems outweigh the disadvantages of assessing a larger proportion of children with false-positive screens. Thus we opted to maximize sensitivity.
Although overreferrals on screening tests may increase costs and raise parental anxiety, children with false-positive screens are typically an at-risk group.28,29 Our results similarly revealed that LMPT children with false-positive screens had significantly poorer cognitive, socio-emotional and language development than children with true-negative screens. It is likely, therefore, that these children would benefit from a clinical consultation to assess their risk status. Early developmental interventions have been shown to enhance cognitive and language development and academic achievement in cohorts of children encompassing subgroups of children born LMPT or with low birth weight.30,31 As such, early identification and provision of support for at-risk children may improve long-term outcomes among this population.
Only 28% of children with true-positive screens had accessed health care services over the last year. The PARCA-R therefore identified a substantial portion of children with developmental delay who were not in receipt of health care services. This underscores the value of establishing routine follow-up services for this population. The costs of applying existing very preterm follow-up models to this larger population of infants are likely to be prohibitive. Using the PARCA-R clinically, as a first-line screener, would reduce the proportion of infants requiring clinical assessment by two-thirds. Given the high NPV, clinicians could also be assured that almost every infant with developmental problems would be detected by screening.
This study draws strengths from its representative, population-based sample of children born LMPT and the use of a gold standard test for assessing developmental delay. Parents completed the PARCA-R before the assessment and psychologists were blind to parent’s responses on the questionnaire. Psychologists also achieved excellent interrater reliability on Bayley-III assessments. Moreover, there is growing awareness of the increased risk of neurodevelopmental sequelae in infants born LMPT and thus the derivation of a population-specific cutoff is warranted for enhancing developmental surveillance for these infants. Limitations of the study relate to the underrepresentation of families with socioeconomic deprivation and multiple births in the final sample. To assess 2 or more children required the assistance of another adult and a further 1 hour of testing, thus parents with twins or triplets were less willing to participate in the study. Because the PARCA-R assesses only cognitive and language development, this questionnaire may need to be supplemented with a measure of motor development for more comprehensive developmental screening.
The PARCA-R has good concurrent validity with a gold standard test of cognitive and language development and can be used as an effective first-line screening tool to identify children who are in need of further assessment. Early identification of developmental delay may facilitate the provision of support services for this population of children born LMPT who are at risk for subtle long-term difficulties.
We would like to thank all the children and parents who took part in the LAMBS-2 Study.
- Accepted April 22, 2014.
- Address correspondence to Samantha Johnson, PhD, Senior Research Fellow, Department of Health Sciences, University of Leicester, 22-28 Princess Rd West, Leicester, LE1 6TP, United Kingdom. E-mail:
Ms Blaggan was responsible for data collection, data entry and day-to-day management of the study, drafted the initial manuscript, and reviewed and revised the manuscript; Ms Guy contributed to data collection and data entry, and reviewed and revised the manuscript; Dr Boyle conceptualized and designed the study, supervised data collection and study progress, and critically reviewed and revised the manuscript; Mr Spata analyzed study data, and reviewed and revised the manuscript; Dr Manktelow conceptualized and designed the study, supervised data analyses, and critically reviewed and revised the manuscript; Dr Wolke conceptualized and designed the study, and critically reviewed and revised the manuscript; Dr Johnson conceptualized and designed the study, supervised data collection and study progress, and critically reviewed and revised the manuscript; and all authors approved the final manuscript as submitted.
The views expressed in this article are those of the authors and not necessarily those of the National Health Service, the National Institute for Health Research, or the Department of Health.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: This article presents independent research funded by the National Institute for Health Research under the Research for Patient Benefit Scheme (PB-PG-1010-23049). Mr Spata is funded by a Research Methods Fellowship award from the National Institute for Health Research.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- McGowan JE,
- Alderdice FA,
- Holmes VA,
- Johnston L
- Guevara JP,
- Gerdes M,
- Localio R,
- et al
- Martin AJ,
- Darlow BA,
- Salt A,
- et al
- Marlow N,
- Greenough A,
- Peacock JL,
- et al
- Fenson L,
- Pethick S,
- Renda C,
- Cox J,
- Dale P,
- Reznick J
- Briggs-Gowan MJ,
- Carter AS
- Computing RFfS
- Johnson S,
- Hollis C,
- Hennessy E,
- Kochhar P,
- Wolke D,
- Marlow N
- McCormick MC,
- Brooks-Gunn J,
- Buka SL,
- et al
- Noble M,
- Wright G,
- Dibben C,
- et al
- Copyright © 2014 by the American Academy of Pediatrics