OBJECTIVES: Validated questionnaires help the preventive child healthcare (PCH) system to identify psychosocial problems. This study assesses the psychometric properties and added value of the Strengths and Difficulties Questionnaire (SDQ) for the identification of psychosocial problems among preschool-aged children by PCH.
METHODS: We included 839 children (response 66%) 3 to 4 years of age undergoing routine health assessments in 18 PCH services across the Netherlands. Child healthcare professionals interviewed and examined children and parents. Before the interview, parents completed the SDQ and the Child Behavior Checklist (CBCL). We assessed the internal consistency, the scale structure, and validity (correlation coefficients, sensitivity, and specificity), with CBCL and treatment status as criteria, and the degree to which the SDQ could improve identification solely on the basis of clinical assessment.
RESULTS: The internal consistency of the SDQ total difficulties score was good (Cronbach’s α, 0.78), but it was worse for some subscales of the SDQ (range, 0.50–0.74). The area under the receiver operating characteristic curve using the CBCL as a criterion was 0.94 (95% confidence interval 0.91–0.97), and sensitivity and specificity were 0.79 and 0.93, respectively. The SDQ added information to the clinical assessment; the odds ratio was 36.48 for added information by using the CBCL as a criterion.
CONCLUSIONS: The SDQ is a valid tool for the identification of psychosocial problems in preschool-aged children by PCH. However, the low reliability of some SDQ subscales does not justify use of these subscales for decisions about further treatment.
- CBCL —
- Child Behavior Checklist
- CFA —
- confirmatory factor analysis
- CHP —
- child healthcare professional
- CI —
- confidence interval
- ITSEA —
- Infant Toddler Social and Emotional Assessment
- OR —
- odds ratio
- PCFI —
- parsimony comparative fit index
- PCH —
- preventive child healthcare
- RMSEA —
- root mean square error of approximation
- SDQ —
- Strengths and Difficulties Questionnaire
- SEM —
- structural equation modeling
- TDS —
- total difficulties score
- TPS —
- total problems score
What’s Known on This Subject:
Validated questionnaires can improve the identification of psychosocial problems among children. The Strengths and Difficulties Questionnaire (SDQ) 3-4 is a promising option. However, no studies are available that examine the psychometric properties of the SDQ parent form 3-4.
What This Study Adds:
The results of this study show that the SDQ 3-4 is a valid tool for the identification of psychosocial problems in preschool-aged children.
A large proportion of children suffer from psychosocial problems, such as social-emotional and behavioral problems.1,2 Children with psychosocial problems are likely to experience difficulties in their daily functioning. Such problems may be severe and persist over time.3 Early detection and treatment can improve the prognosis for psychosocial problems in children.4,5
In the Netherlands, preventive child healthcare (PCH) offers an ideal opportunity for the early detection of psychosocial problems among preschool-aged children, comparable to community pediatric services in the United States. In PCH, child health professionals (CHPs), ie, doctors and nurses, offer routine well-child examinations, including the early detection of psychosocial problems, to the entire Dutch population. Care is free of charge, and access is independent of insurance status, but the services do not provide treatment, in contrast to the US system.
Although the PCH system is important for the early identification of psychosocial problems, CHPs failed to identify psychosocial problems in 70% of the preschool-aged children (14 months to 4 years) with parent-reported problems on the Child Behavior Checklist (CBCL) questionnaire or Infant Toddler Social and Emotional Assessment (ITSEA). Furthermore, 5% to 7% of the children with no clinical problems on the CBCL or ITSEA were identified as having psychosocial problems by the CHPs.6,7
Validated questionnaires may improve the identification of psychosocial problems by CHPs.8 An example of such a questionnaire is the CBCL, a highly reliable and valid instrument for assessing psychosocial problems in children.9–11 However, the CBCL questionnaire is too long to be used as a routine screening questionnaire in PCH. Instruments used in PCH must be short. For preschool-aged children (3–4 years old), no brief validated questionnaire is available to support the early identification of psychosocial problems in PCH.
The Strengths and Difficulties Questionnaire (SDQ) is a brief behavioral screening questionnaire.12 This questionnaire was developed by Robert Goodman in the United Kingdom but is now available in many languages. The SDQ can be completed by parents, teachers, and the children themselves. There are 2 age versions of the SDQ parent form: SDQ 3-4 and 4-16. The psychometric properties of the SDQ Parent Form 4-16 have been shown to be good in different settings and in a number of countries,13–18 including the Netherlands.8,19–21 It has also been shown to fit PCH requirements.8,19 No studies are available that examine the psychometric properties of the SDQ Parent Form 3-4. Given the good validity of the SDQ 4-16 in many countries, the SDQ 3-4 is a promising option for use in PCH for preschool-aged children.
The aim of this study was to assess the psychometric properties (internal consistency, scale structure and validity) and the added value of the SDQ Parent Form 3-4 for the identification of psychosocial problems among preschool-aged children. The SDQ 3-4 was validated by using the following criteria: an elevated CBCL score and currently receiving treatment of psychosocial problems.
The sample was obtained in a 2-stage procedure. In the first step, all PCH services in the Netherlands were asked to participate in this study (at that time 55); 18 agreed to do so. These were spread all over the country. In the second step, each of the participating PCH services provided a random sample of children aged 36 and 45 months who were invited for their routine well-child examinations. A total of 1280 parents were asked to participate in this study: 16.1% explicitly refused to participate, 15.9% did not return the questionnaire, and 2.4% did not provide complete data on the questionnaires, resulting in a response of 839 children (65.5%). Respondents were representative of the total sample in terms of family composition, but nonresponse was higher for children of immigrant origin (27.1% compared with 18.3% for children from Dutch origin), for older children aged 42 to 52 months (39.5% compared with 22.0% for children aged 31–41 months), and for girls (34.1% compared with 28.9% for boys). The differences between the responders and nonresponders with regard to child ethnicity, age, and gender were small according to the Cohen effect size index w (range of w: 0.04–0.18). The sample was representative for the entire Dutch population, except that children of immigrant origin were underrepresented because of the sampling procedure.
The data were collected during the routine well-child examinations. The CBCL 1.5-5 and the SDQ 3-4 were mailed to parents along with the standard invitation for the well-child examination and were filled in at home. The completed questionnaires were returned to the CHP in a sealed envelope and forwarded to the research institute without being opened.
The CHP then took a routine history and physically assessed each child before answering the following questions. Does the child have a psychosocial problem at this moment (yes or no)? Does the child currently receive treatment for psychosocial problems?
The CHP also provided data about child age and gender, ethnic background, family composition, parental employment and educational level, number of siblings, and maternal and paternal age. The parental educational level was the highest level of education completed by either one of the parents. Family composition focused on the number of parents in the family (2 parents or 1 parent). The CHPs recorded these background characteristics during the assessment. They are presented in Table 1.
Currently receiving treatment of psychosocial problems and an elevated CBCL (1.5–5) total problems score (TPS) were used as the criteria for psychosocial problems. The CBCL assesses parental reports about children’s behavioral and emotional problems in the preceding 2 months. Its reliability and validity have been found to be sound, including in the Netherlands.9–11 The CBCL comprises 99 problem items that are used to compute total, internalizing, and externalizing problem scores. Children were allocated to a normal range or an elevated range by using the 90th percentile cutoff point.
We used the parent version of the SDQ 3-4.12,20,21 This instrument is quite comparable to the better known SDQ 4-16. Both questionnaires consist of 25 items relating to children’s strengths and difficulties and 8 items relating to the severity and the impact of problems. Six items of the SDQ 3-4 are phrased slightly differently than the corresponding SDQ 4-16 items. Two SDQ 4-16 items, referring to lying, cheating, and stealing, however, were removed and replaced by more age-adequate conduct problems (Appendix, items 18 and 22). Each item has to be scored on a 3-point scale (0 = not true, 1 = somewhat true, and 2 = certainly true). The SDQ consists of 5 subscales: emotional symptoms, conduct problems, hyperactivity-inattention, peer problems, and prosocial behavior. A SDQ total difficulties score (TDS) is calculated by adding up the scores for the first 4 subscales mentioned above.
The questions on the impact of problems refer to the duration, distress, social impairment, and burden for others. An impairment score was calculated by aggregating the scores for distress and social impairment. A 3-point scale is used for each item: 0 = not at all/ only a little, 1 = quite a lot, and 2 = a great deal.
The analyses assessed the psychometric properties of the SDQ and its added value for the identification of psychosocial problems. Regarding psychometric properties, we first computed the internal consistency (Cronbach’s α). Next, we examined the fit between the scale structure and the observed data with confirmatory factor analysis (CFA) using AMOS structural equation modeling (SEM). In the CFA, the models were considered to be a good fit when the parsimony comparative fit index (PCFI) was >0.90. Because the PCFI index is a strict criterion, we considered the model to be an approximate fit when the root mean square error of approximation (RMSEA) was <0.08. Items with regression weights <0.30 were considered not to be a fit.22
The validity of the SDQ TDS and impairment score was assessed with sensitivity and specificity indices, using CBCL TPS, internalizing, and externalizing problems scores and “currently receiving treatment for psychosocial problems” as the criteria. Cohen’s κ and Spearman correlation coefficients were calculated to assess the overall agreement between the SDQ and the criteria.
We determined the added value of the SDQ TDS and impairment score by assessing the degree to which the SDQ can improve the identification of children with problems based solely on clinical assessment by the CHP without knowledge of the SDQ. A logistic regression analysis was performed with the CBCL criterion measures as the dependent variable. The criterion “currently receiving treatment for psychosocial problems” was not used in these analyses, because treatment status may be expected to be known to CHPs. In the first step, the identification by a CHP was included in the analyses, and in the second step, the SDQ TDS or the SDQ impairment score was added as an independent variable. Next, we repeated these logistic regression analyses by adding the SDQ TDS in the second step and the SDQ impairment score in the third step. The identification of children with problems is most relevant for those children who are not being treated for such problems. We therefore repeated these analyses excluding children currently receiving treatment for psychosocial problems.
The mean age of the sample was 40 months (SD: 5 months). Further demographic information is presented in Table 1.
The internal consistency of the SDQ TDS was 0.78. For the 5 subscales of the SDQ, Cronbach’s α varied between 0.50 and 0.74 (Table 2).
SEM for the single-scale model produced a PCFI of 0.523 and a RMSEA of 0.079 (confidence interval [CI]: 0.075–0.084), suggesting an approximate fit for the single-scale model. Seven items had regression weights β < 0.30 (3, 6, 11, 13, 19, 23, and 24; see Appendix).
For a model with the SDQ subscales, PCFI was 0.550 and RMSEA was 0.075 (CI 0.071–0.080), also suggesting an approximate fit for this model. Two items had regression weights <0.30 (3 and 19).
The SDQ TDS and the CBCL subscales scores correlated significantly with the CBCL scores (Table 3). The highest correlation coefficient was found between the CBCL TPS and the SDQ TDS (Spearman’s r = 0.70) and the lowest was between the SDQ peer problems score and the CBCL externalizing problems score (Spearman r = 0.22).
The literature does not state a cutoff point for the SDQ 3-4. We therefore chose an appropriate cutoff point, namely the score that was associated with a specificity of at least 0.90 in our sample, using the elevated CBCL TPS as the criterion. We assessed whether different SDQ TDS cutoffs were needed for boys and girls; this was not the case. The chosen cutoff point was ≥11 for the TDS, resulting in 13.8% elevated scores. The cutoff point for the SDQ impairment scale was ≥1, resulting in 5.8% elevated scores.
Tables 4 and 5 present the Cohen’s κ, sensitivity, and specificity for all criteria. The Cohen’s κ for the SDQ TDS varied between 0.47 and 0.59 for the CBCL criteria and was 0.20 for treatment status. Sensitivity and specificity for the SDQ TDS varied from 0.64 to 0.79 and 0.92 to 0.93, respectively, for the CBCL criteria and were 0.68 and 0.88, respectively, for treatment status.
Sensitivity for the SDQ impairment score varied from 0.38 to 0.39, and specificity was 0.98 for all CBCL criteria. Sensitivity and specificity for treatment status were 0.60 and 0.96, respectively.
Repetition of these validity analyses without the items with an inadequate fit for the single-scale model in the SEM analyses yielded similar results (data not shown).
These results show that both an elevated SDQ TDS and an elevated SDQ impairment score are accompanied by an increase in the likelihood of identifying children with an elevated CBCL score (TPS, internalizing, or externalizing) compared with a clinical assessment alone by the CHP (Table 6). The adjusted odds ratios (ORs) for elevated SDQ TDS scores were significant and ranged from 14.3 to 36.5. Adjusted ORs for elevated SDQ impairment scores were significant and ranged from 14.8 to 16.4. The analyses in which the SDQ impairment score was added to the SDQ TDS showed that the adjusted ORs for elevated SDQ impairment scores were statistically significant and ranged from 5.10 to 6.88. However, the ORs for the SDQ TDS were substantially lower compared with the first added value analyses where the SDQ impairment score was not added. The latter finding indicates that the contribution of the SDQ impairment score overlaps with the contribution of the SDQ TDS.
Repetition of these analyses for children not under treatment for psychosocial problems yielded similar results (data not shown).
This study examined the psychometric properties of the SDQ 3-4 and the degree to which it enhances the early detection of psychosocial problems among preschool-aged children in PCH practice. Our findings show that the SDQ TDS discriminates between preschool-aged children with and without problems as measured by the CBCL and treatment status. The SDQ TDS is somewhat more sensitive to externalizing than to internalizing problems. The SDQ impairment score does not sufficiently discriminate between children with and without problems; sensitivity is low using CBCL TPS as the criterion. The SDQ TDS and impairment score both have added value for the identification of psychosocial problems among preschool-aged children by PCH.
However, for the criterion “treatment status,” the validity (ie, sensitivity) was slightly lower than for the criterion “CBCL.” Similar results were found in previous research.19,23,24 An explanation may be that some children with psychosocial problems never contact mental health services, whereas other children without psychosocial problems actually receive treatment from mental health services (ie, because of child-rearing challenges).
Fit With Previous Literature and Other Questionnaires Used in PCH
This study was the first study to investigate the psychometric properties and added value of the SDQ 3-4 for assessing psychosocial problems in preschool-aged children. Some studies have investigated the psychometric properties of other short questionnaires (such as Brief Infant Toddler Social Emotional Assessment and Ages and Stages Questionnaires: Social-Emotional) to detect psychosocial problems in preschool-aged children,25,26 but these studies did not assess the added value of these instruments for community pediatric services.
In general, our findings are in line with the findings of Vogels et al,8 who investigated the psychometric properties of the SDQ 4-16 in older children (ages 7–12 years) in the Netherlands. The internal consistency of the total problem scales for both age versions was very satisfying: 0.78 for the SDQ 3-4 and 0.80 for the SDQ 4-16. The internal consistency of some subscales was relatively low in both age versions. Cronbach’s α varied between 0.50 and 0.74 for the SDQ 3-4 and between 0.55 and 0.78 for the SDQ 4-16. The internal consistency of the SDQ subscales does not justify using these subscales for decisions about whether individual children require further attention. Other studies also yielded mixed support for the SDQs 5-factor structures when using CFAs.27,28 On the basis of empirical and theoretical support, Goodman et al27 proposed alternative subscales (ie internalizing and externalizing subscales instead of the original 5 subscales). We calculated internal consistencies for the subscales internalizing and externalizing, yielding Cronbach α of 0.62 and 0.72, respectively. These values show that the internal consistency improves somewhat when using 2 broader subscales instead of the 5 original subscales. Following George and Mallery’s29 rules of thumb, the internal consistency of the internalizing subscale is at the edge of acceptable, so using this scale carefully seems to be justified.
The sensitivities and the areas under the curve (which measure the accuracy of the SDQ for the detection of problems) in both SDQ versions are in the same range.8
With respect to the added value, we found that the SDQ TDS contributes in determining whether preschool-aged children have problems, after taking into account identification by PCH. The added value of the SDQ TDS is lower for the SDQ 3-4 (OR, 36.48; CI, 19.26–69.12) than for the SDQ 4-16 (OR, 65.4; CI, 24.8–172.4), but the CIs overlap, so the difference relating to added value was not statistically significant.8
Methodological factors are unlikely to have affected our results. The response rate (65.5%) was acceptable. Moreover, we used the CBCL as the validation criterion. The CBCL has been proven to be a well-validated questionnaire for behavioral and emotional problems. However, both the SDQ and the CBCL are completed by the parent, which could increase correlations slightly. Clinical assessments such as psychiatric interviews may provide additional information. Because of their complexity and high costs, they were not used as criteria in this study.
Children of immigrant origin were underrepresented in this study. Previous research has shown that CHPs have more difficulty in identifying psychosocial problems in immigrant children than in Dutch children.30,31 We therefore expect that the quality of identification in this study by CHPs (added value analyses) is actually somewhat worse than our data suggest.
Our results show that the SDQ can provide effective support for PCH in the identification of psychosocial problems among preschool-aged children.
The sample covered the entire Netherlands as the population, and the data were collected during routine PCH practice, which supports the potential generalization of our results to the entire Netherlands. We expect that our results may be generalized to a wide range of countries, as the SDQ has been shown to be cross-culturally valid and is available in a broad range of languages.
Early detection of psychosocial problems only has value when it is followed by early treatment in improving the prognosis for psychosocial problems. Examples of potentially effective (group-based) parenting programs for preschool-aged children are the Triple P-positive Parenting program,32,33 the Incredible Years Basic Parenting Program,34–36 and the cognitive-behavioral Stop Think Ask Respond program.37 These programs should be applied more often. Furthermore, more evidence needs to be collected about the long-term effectiveness of these programs.38,39
The results of this study show that the SDQ 3-4 parent form is a useful aid for the early detection of psychosocial problems. The SDQ can provide effective support for PCH in the identification of psychosocial problems among preschool-aged children. This instrument is therefore a valid tool for routine use in PCH.
|Not True||Somewhat True||Certainly True|
|1.||Considerate of other people's feelings||□||□||□|
|2.||Restless, overactive, cannot stay still for long||□||□||□|
|3.||Often complains of headaches, stomach-aches or sickness||□||□||□|
|4.||Shares readily with other children, for example toys, treats, pencils||□||□||□|
|5.||Often loses temper||□||□||□|
|6.||Rather solitary, prefers to play alone||□||□||□|
|7.||Generally well behaved, usually does what adults request||□||□||□|
|8.||Many worries or often seems worried||□||□||□|
|9.||Helpful if someone is hurt, upset or feeling ill||□||□||□|
|10.||Constantly fidgeting or squirming||□||□||□|
|11.||Has at least one good friend||□||□||□|
|12.||Often fights with other children or bullies them||□||□||□|
|13.||Often unhappy, depressed or tearful||□||□||□|
|14.||Generally liked by other children||□||□||□|
|15.||Easily distracted, concentration wanders||□||□||□|
|16.||Nervous or clingy in new situations, easily loses confidence||□||□||□|
|17.||Kind to younger children||□||□||□|
|18.||Often argumentative with adults||□||□||□|
|19.||Picked on or bullied by other children||□||□||□|
|20.||Often offers to help others (parents, teachers, other children)||□||□||□|
|21.||Can stop and think things out before acting||□||□||□|
|22.||Can be spiteful to others||□||□||□|
|23.||Gets along better with adults than with other children||□||□||□|
|24.||Many fears, easily scared||□||□||□|
|25.||Good attention span, sees tasks through to the end||□||□||□|
|26.||Overall, do you think that your child has difficulties in one or more of the following areas:||No||Yes,||Yes,||Yes,|
|Minor difficulties||Definite difficulties||Severe difficulties|
|emotions, concentration, behavior or being able to get on with other people?||□||□||□||□|
|27.||How long have these difficulties been present?||Less than a month||1–5 mo||6–12 mo||Over a year|
|28.||Do the difficulties upset or distress your child?||Not at all||Only a little||Quite a lot||A great deal|
|29.||Do the difficulties interfere with your child's everyday life in the following areas?||Not at all||Only a little||Quite a lot||A great deal|
|30.||Do the difficulties put a burden on you or the family as a whole?||Not at all||Only a little||Quite a lot||A great deal|
- Accepted September 20, 2012.
- Address correspondence to Meinou H.C. Theunissen, MSc, TNO Child Health, PO Box 2215, 2301 CE, Leiden, Netherlands. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: This research received financial support from the Netherlands Organization for Health Research and Development (ZonMw).
- Leaf PJ,
- Alegria M,
- Cohen P,
- et al
- Geeraert L,
- Van den Noortgate W,
- Grietens H,
- Onghena P
- Klein Velderman M,
- Crone MR,
- Wiefferink CH,
- Reijneveld SA
- Achenbach T,
- Rescorla L
- Goodman R,
- Ford T,
- Simmons H,
- Gatward R,
- Meltzer H
- Hawes DJ,
- Dadds MR
- Byrne BM
- Vogels T,
- Reijneveld SA,
- Brugman E,
- den Hollander-Gijsman M,
- Verhulst FC,
- Verloove-Vanhorick SP
- Briggs-Gowan MJ,
- Carter AS,
- Irwin JR,
- Wachtel K,
- Cicchetti DV
- Goodman A,
- Lamping DL,
- Ploubidis GB
- George D,
- Mallery P
- Hutchings J,
- Gardner F,
- Bywater T,
- et al
- Copyright © 2013 by the American Academy of Pediatrics