PEDIATRICS Vol. 108 No. 1 July 2001, p. e14

ELECTRONIC ARTICLE:
Obsessive-Compulsive Scale of the Child Behavior Checklist: Specificity, Sensitivity, and Predictive Power

Elliot C. Nelson, MD^*, Gregory L. Hanna, MD, James J. Hudziak, MD^§, Kelly N. Botteron, MD^*, Andrew C. Heath, DPhil^*, and Richard D. Todd, PhD, MD^*

From the ^* Department of Psychiatry, Washington University School of Medicine, St Louis, Missouri;  University of Michigan School of Medicine, Ann Arbor, Michigan; and ^§ University of Vermont School of Medicine, Burlington, Vermont.

	ABSTRACT

Top Abstract Methods Results Discussion References

Objective.  To create an obsessive-compulsive disorder subscale (OCS) of the Child Behavior Checklist (CBCL) and to determine its internal consistency, sensitivity, specificity, and positive and negative predictive power to identify obsessive-compulsive disorder (OCD) in children and adolescents.

Methods.  Three samples of equal size (n = 73) of children and adolescents, matched for age, gender, and race, were selected for these analyses: 1) a clinically ascertained OCD group, 2) a psychiatrically treated group whose records revealed no evidence of OCD, and 3) a general population control group. An OCS was created by applying factor analysis to 11 CBCL items. Examinations of internal consistency, sensitivity, specificity, and positive and negative predictive value were undertaken.

Results.  Of 11 items hypothesized to predict OCD, 8 items were retained after factor analyses (smallest factor loading: 0.49) and used to calculate OCS scores. The retained items displayed excellent internal consistency (Cronbach's  coefficient = 0.84). OCD participants had significantly higher OCS scores than either psychiatrically treated or general population control groups. With the use of the 2 cutoff scores closest to the true rate of OCD in the overall sample, sensitivity was 75.3% to 84.9%, specificity was 82.2% to 92.5%, positive predictive value was 70.5% to 83.3%, and negative predictive value was 88.2% to 91.6%.

Conclusion.  The performance of the proposed CBCL OCS compares favorably with that of the only previously studied screening instrument for OCD, the Leyton Obsessional Inventory-Child Version. Unlike the Leyton Obsessional Inventory-Child Version, the CBCL is already in widespread use as a screen for most other forms of psychopathology. As the performance of the CBCL OCS will need to be replicated in other sample populations, data with various cutoff levels are provided to enable investigators and clinicians to tailor its use to specific study populations.  Key words:  Child Behavior Checklist, obsessive-compulsive disorder, screening.

Obsessive-compulsive disorder (OCD) is a much more prevalent illness than previously appreciated. Although this premise is now widely accepted in adults,^1-5 supporting data in children and adolescents have accumulated more gradually.^6-12 Several studies of adolescents have found lifetime prevalence rates for OCD^6-12 ranging from 1.9% to 4.1% with reported rates of subclinical symptoms as high as 19%.¹¹ This work, reports on autoimmune-mediated OCD symptom exacerbations,^13-15 and follow-up studies that suggested an improved prognosis for children and adolescents who are treated for OCD^16,17 have instilled in many pediatricians a greater appreciation of the importance of diagnosing OCD.

Classic presentations (eg, compulsive handwashing) offer little challenge to even an inexperienced diagnostician. OCD symptoms, however, may include innumerable variants of obsessive thoughts and compulsive behaviors that are decidedly more challenging to recognize. In adult populations, researchers have struggled to produce standardized instruments that capture sufficiently the phenomenology of either obsessions or compulsions to enable OCD to be diagnosed with reasonable validity.^18,19

Two instruments of demonstrated utility in adults have been revised for use in children and adolescents. Like its adult counterpart, the Children's Yale-Brown Obsessional Scale (CY-BOCS) has demonstrated good reliability and validity for the rating of OCD symptom severity,²⁰ but it is not a diagnostic instrument. The Leyton Obsessional Inventory-Child Version (LOI-CV), a 20-item self-report adapted from the adult questionnaire,^6,21,22 is the only OCD screening instrument that has been studied previously. In an investigation that used a 2-stage design,^6,21 all high school students in a New Jersey county first were asked to complete a number of self-report measures, including the LOI-CV. A stratified random sample in which participants who had high scores on the LOI-CV or 1 of the other questionnaires were overrepresented was then selected for interview by clinicians who were blind to questionnaire results. The LOI-CV was found to have reasonable sensitivity and specificity (75%-88% and 77%-84%, respectively, depending on the cutoff used) but only modest positive predictive value (PPV; 15%-18%). A screening instrument should have reasonable PPV and high negative predictive value (NPV). The LOI-CV's high rate of false positives, coupled with its lack of applicability other than for OCD, may explain why it has had only limited use in research or clinical practice.

The Child Behavior Checklist (CBCL)²³ is one of the most widely used instruments in child and adolescent psychiatry and pediatrics. It is understood easily and completed readily by most parents. Available programs enable the generation of scores for 8 quantitatively derived CBCL syndrome scales. The scales are normalized for age and gender.²³ The reliability, validity, and temporal stability of the CBCL's scales have been documented thoroughly.²³ Investigators increasingly have used the CBCL in clinically characterized populations to use it as a tool for predicting Diagnostic and Statistic Manual of Mental Disorders (DSM) diagnoses.^24-28

Previous examinations that used the CBCL in children and adolescents with OCD reported consistent results with elevation of thought problem and anxious-depressed syndrome scores.^29-31 Hanna²⁹ found no significant correlation between any CBCL syndrome score and the CY-BOCS and noted that several CBCL syndrome scores differed significantly between participants with and without comorbid disruptive behavior disorders.

The current investigation uses factor analysis to examine CBCL data derived from clinically ascertained children and adolescents with OCD (n = 73) and 2 gender-, race-, and age-matched control groups: 1) psychiatrically treated participants without OCD (n = 73) and 2) general population control participants from the Vermont national sample (n = 73). Analyses were restricted to a set of 11 CBCL items that were hypothesized to be the most pertinent to the diagnosis of OCD. A CBCL obsessive-compulsive subscale (OCS) was constructed from these data. The internal consistency of these items was calculated to evaluate their interrelationships. Specificity, sensitivity, PPV and NPV were examined for various percentile cutoffs within our sample for the purpose of providing enough detail of the subscale's performance to enable its application to other samples, including potential use as a general population screening instrument for OCD.

	METHODS

Top Abstract Methods Results Discussion References

Participants

Data on children and adolescents with OCD were derived from 2 sources. All participants who presented for treatment between November 1, 1991, and June 1, 1997, to the St Louis Children's Hospital Child Psychiatry Center, a clinic that provides primary psychiatric care for individuals who reside primarily within the greater St Louis metropolitan area, were eligible for initial inclusion. The clinic routinely mailed copies of the CBCL to parents before their child's initial appointment and gave instructions to bring the completed checklist to the appointment. After institutional review board approval was obtained, billing records were used to identify 110 children and adolescents who had received a diagnosis of OCD during their treatment. No CBCL data were available for 30 of these individuals. The clinical information for the remainder was reviewed by a psychiatrist (E.N.) with adequate support found for a Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV),³² diagnosis of OCD in 45 participants. Comorbid diagnostic information was obtained from the initial clinical evaluation performed either by or under the direct supervision of a board-certified child and adolescent psychiatrist. Age limitations led to the exclusion of an additional 8 children aged 7 years or younger as the youngest individuals in the general population control group were 9 years of age (a decision was made to allow no more than a 1-year age difference for matching). Data on an additional 36 participants with OCD ascertained via presentation (beginning in 1987) for treatment and research protocols at the UCLA Neuropsychiatric Institute and the University of Michigan were made available for inclusion in the present analyses. Diagnostic and Statistical Manual of Mental Disorders, Third Edition, Revised (DSM-III-R)³³ OCD and comorbid diagnoses were confirmed by a board-certified child and adolescent psychiatrist (G.H.). The CBCL data of 24 of these participants were published previously.²⁹

A group of 73 psychiatrically ill control participants was obtained from the St Louis Children's Hospital Child Psychiatry Center. Participants were selected via a program that generated random numbers that then were compared with previously assigned unique clinic numbers. Participants who matched an OCD group member in terms of age, gender, and race were included on this basis. As this process progressed, a billing record search was used to locate control participants for individuals who were not readily matched. The exclusion criteria for the control participants were the lack of a CBCL or documentation of OCD symptoms in their medical record. Diagnostic information also was obtained for these participants from the initial clinical evaluation performed either by or under the direct supervision of a board-certified child and adolescent psychiatrist.

A general population control group was selected from the 1992 Vermont national sample.²³ These participants were chosen randomly to match OCD participants in terms of age, gender, and race. Each group consisted of 45 boys and 28 girls and included 1 Asian and 1 Hispanic boy. All other participants were white, non-Hispanic. The respective mean ages in years of the boys in the OCD group, psychiatrically ill controls, and population controls were 12.27 (SD: 2.77), 12.24 (SD: 2.80), and 12.31 (SD: 2.70), respectively. The similar values for girls were 11.96 (SD: 2.66), 11.96 (SD: 2.66), and 12.00 (SD: 2.61), respectively. Paired t tests that were performed across groups, overall and separately by gender, confirmed the lack of any significant intergroup age differences (P > .80 for all). Diagnoses of psychiatric control group participants and comorbid diagnoses of those with OCD are shown in Table 1. OCD group members had higher rates of non-OCD anxiety disorders, tic disorders, trichotillomania, and stereotypy/habit disorders and lower rates of attention deficit hyperactivity disorder, oppositional defiant or conduct disorder, and any substance abuse or dependence than psychiatrically treated control participants.

Measures

Completion of the CBCL requires parents to provide demographic information, report on their child's competencies across 7 aspects of behavior, and rate the degree to which each of 118 problems has been experienced during the past 6 months. The rating of problems in the past 6 months uses a scale with options of "0 not true," "1 somewhat or sometimes true," and "2 very true or often true." The current study used only the responses to the 118 problem items. Except for coding of "wears glasses" under "problems with eyes" and material already reported as "not listed above," parental responses were entered as circled to minimize the influence of clinical judgments and improve consistency with previous work.²⁹ Specifically, no attempt was made to verify that the parents responded correctly to the questions about obsessions and compulsions. CBCLs were entered and scored via an available scoring program that provided raw scores for the current analyses.

Data Analyses

After scoring, CBCL data were incorporated into a SAS data file, and the SAS system³⁴ was used for additional analyses. The authors chose 11 CBCL items for factor analysis. Because of omitted responses to some of these items, data from 12 individuals with OCD and 9 psychiatrically ill control participants could not be included. Cronbach's coefficient  was used to gauge the internal consistency of the selected 11 items. A 2-factor model was fitted by the method of principal factor analysis, without factor rotation. Factor scores then were calculated for each participant using these factor loadings. Normalized factor scores (OCS scores) were obtained by dividing each participant's score by the maximum score available to that individual (so that participants with missing data could be given a normalized score that corrected for the lost contribution of any unanswered items). Analysis of variance with OCS score as the dependent variable was used to examine whether there was a significant main effect for the group. Paired t tests, which used Satterthwaite's approximation to correct for inequality of variances, were used to compare OCS scores across groups. Sensitivity [true positives/(true positives + false negatives)], specificity [true negatives/(true negatives + false positives)], PPV [true positives/(true positives + false positives)], and NPV [true negatives/(true negatives + false negatives)] were calculated at various percentile levels of the presumptive OCS in the current sample.

	RESULTS

Top Abstract Methods Results Discussion References

Factor Analysis

The initial, unrotated solution retained 2 factors. The first factor explained 40.0% of the variance and had positive loading values for all 11 items that ranged from 0.4802 to 0.7369. Because the second factor explained a much smaller share of the variance and its item loading values (range: 0.2820-0.4481) all were below those of the first factor, a decision was made to retain only the unrotated first factor. Simplification of the solution was attempted with sequential removal of 3 items: "too concerned with neatness and cleanliness," the only item with a loading value below 0.50, followed by "nervous, highstrung, or tense" and "too fearful or anxious," both of which were highly correlated with the retained item "worries." The 8 remaining items had positive loading values ranging from 0.4914 to 0.7000 on the single factor (see Table 2), which explained 39.8% of the variance. Cronbach's  coefficient for the 8 retained items was 0.84. 

OCS scores, the mean normalized 8-item factor scores, for the 3 groups were as follows: OCD, 0.57 (SD: 0.24); psychiatrically treated control participants, 0.23 (SD: 0.17); and population controls, 0.11 (SD: 0.12). Analysis of variance with normalized factor score as the dependent variable revealed a significant main effect for group (F = 123.72; df = 2218; P < .0001). Paired t tests were used to compare OCS scores across groups. The OCS scores of OCD participants were higher than those of both the psychiatrically treated (t = 9.94; df = 131.3; P < .0001) and population control participants (t = 14.72; df = 106.6; P < .0001). The psychiatrically treated group's scores also were noted to be higher than those of the general population control participants (t = 4.77; df = 128.7; P < .0001).

Sensitivity, Specificity, PPV, and NPV

Sensitivity, specificity, PPV, and NPV were calculated at various percentile levels of the presumptive OCS. Comparisons are shown for the OCD group versus the other groups combined and individually (see Table 3). The OCS scores display strong PPV and NPV for all contrast groups when cutoffs at or above the 60th percentile are used.

	DISCUSSION

Top Abstract Methods Results Discussion References

This article describes how the OCS from the CBCL was created. Of the 11 items for which the authors' a priori hypothesis predicted involvement, 8 were retained after factor analyses. The smallest factor loading for these items was 0.49, and together the items displayed excellent overall internal consistency (Cronbach's coefficient = 0.84). The single retained factor explained 39.8% of the total variance. The OCD group had significantly higher OCS scores than either the psychiatrically treated or general population control participants.

The OCS demonstrated high levels of sensitivity and specificity and high diagnostic power as indicated by PPV and NPV in clinically ascertained participants with OCD versus psychiatrically treated non-OCD participants and general population control participants. With the use of the 2 cutoff scores (the 60th and 70th percentiles) that best reflected the true rate of OCD in the overall sample (one third) and comparison against the control groups, sensitivity was 75.3% to 84.9%, specificity was 82.2% to 92.5%, PPV was 70.5% to 83.3%, and NPV was 88.2% to 91.6%. Overall, the proposed CBCL OCS performance seems to compare favorably with that of the LOI-CV.^6,21

Other potential advantages offered by the proposed CBCL OCS over the LOI-CV include the following: 1) translated in 43 languages, it has been used in more than 2000 studies in 56 countries; 2) prevalence of OCS deviance can be easily estimated retrospectively using already collected CBCL data, and longitudinal stability of the OCS can be determined readily; 3) already one of the most commonly used instruments in clinical settings; 4) established utility for identifying problems other than OCD (including comorbid psychiatric illnesses commonly seen with OCD); 5) readily available computer scoring algorithms; 6) simple printed instructions enabling mailing to parents before initial office visit; 7) contains a screening question about "nervous movements or twitching" to identify additional participants who are at risk for pediatric autoimmune neuropsychiatric disorders associated with streptococcus; 8) validated collateral measures, the Teacher's Report Form and Youth Self-Report, also widely used, are available for the development of parallel scales.

One methodologic issue that deserves discussion is the diagnostic composition of our psychiatrically treated control group. We chose to select psychiatric controls randomly, with the goal of limiting any ascertainment bias to that inherent in a control group representative of our clinic population. Because OCD is classified as an internalizing disorder, it reasonably could be argued that a sample composed of fewer individuals with externalizing disorders and more with anxiety disorders might have provided a better comparison. This alternative approach would not have controlled adequately for the considerable rate of comorbid disruptive behavior disorders in our OCD group. Others have noted higher rates of these disorders in those with OCD³⁵ and observed that the aggressive behavior in those with tic disorders was seen primarily among those who were dually comorbid for OCD and attention deficit hyperactivity disorder.³⁶ Nonetheless, it will be important to examine the performance of the OCS in samples that differ in composition from that of the current report to clarify fully the extent of its overall usefulness.

The current findings must be interpreted in the context of several limitations. The OCS was evaluated using the same data on which it was developed. As such, until replication of its performance in an entirely new sample, the OCS should be used judiciously. Clinically ascertained OCD may be of greater severity than similarly diagnosed illness in either the general population or a general pediatric practice. Although details of the scale's performance at varying percentile cutoffs are provided to enable users to determine what values are most appropriate for specific populations, additional characterization of our participants (eg, CY-BOCS scores) would have been helpful. Our reliance on retrospective clinical information was a necessary but clearly suboptimal strategy given our lack of data on diagnostic reliability and validity. Although CBCL syndrome scores tend to vary by age and gender, the size of the current sample precluded an examination of this issue. Because an extremely diverse group of obsessions and compulsions are included within OCD, the idea of a single factor may seem counterintuitive. However, the decision to create a single scale was made on the basis of the factor analysis results and is conceptualized best as being consistent with the presence of shared phenomenologic elements across varying symptom presentations. It offers the additional advantage of providing a simple screening measure. The degree to which the OCS succeeds as a screen for OCD of various identified subtypes³⁷ and in samples enriched for participants with affective and other anxiety disorders will need to be examined. Some bias may been introduced by our decision not to assess the accuracy of parents' responses to CBCL questions about obsessions and compulsions. As the OCD group was clinically ascertained, their parents may have been more likely to answer these questions correctly (although within the St Louis sample, few participants had received a diagnosis before their parents' completion of the CBCL). By not recoding incorrect or unsubstantiated (ie, no example given) responses, we are likely to have inflated scoring for the 2 control groups (in which positive responses are more likely to have been incorrect). The CBCL differs from the LOI-CV in its reliance on parents as informants. Given that those with OCD often are secretive, it is possible that parents either may not be aware of all symptoms or, alternatively, might be more forthcoming. Finally, a later DSM iteration (DSM-IV) was used to diagnose OCD in St Louis than that used at UCLA (DSM-III-R). This difference is unlikely to have affected the current results because the 2 algorithms have only minor differences.

Future research will be required to determine how the proposed CBCL OCS may best be used. It is hoped that the publication of these results will stimulate necessary additional investigation. The importance of making a timely diagnosis of OCD in children and adolescents certainly has been underscored by recent reports (eg, the description of pediatric autoimmune neuropsychiatric disorders associated with streptococcus and the observation of its successful treatment^13-15 and the mounting evidence for the efficacy of serotonin reuptake inhibitors and behavior therapy in pediatric populations^16,17). We provided data on the scale's performance at varying cutoff scores to enable investigators and clinicians to tailor its use for the diagnosis of OCD to the needs imposed by their specific sample populations.

	ACKNOWLEDGMENTS

This research was supported by a National Alliance for Research on Schizophrenia and Depression Young Investigators Award (E.C.N.) and US Public Health Service Grant MH-31302 (R.D.T.).

	FOOTNOTES

Received for publication Oct 24, 2000; accepted Mar 26, 2001.

Reprint requests to (E.C.N.) Washington University School of Medicine, Department of Psychiatry, 40 N Kingshighway, Suite 1, St Louis, MO 63108. E-mail: nelsone{at}psychiatry.wustl.edu

	ABBREVIATIONS

OCD, obsessive-compulsive disorder; CY-BOCS, Children's Yale-Brown Obsessional Scale; LOI-CV, Leyton Obsessional Inventory-Child Version; PPV, positive predictive value; NPV, negative predictive value; CBCL, Child Behavior Checklist; OCS, obsessive-compulsive subscale; DSM-IV, Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition; DSM-III-R, Diagnostic and Statistical Manual of Mental Disorders, Third Edition, Revised..

	REFERENCES

Top Abstract Methods Results Discussion References

1. Karno M, Goldring JM, Sorenson SB, Burnam MA The epidemiology of obsessive-compulsive disorder in five US communities. Arch Gen Psychiatry 1988; 45:1094-1099 [Abstract/Free Full Text]
2. Chen C-N, Wong J, Lee N, Chan-Ho M-W, Lau JT-F, Fung M The Shatin Community Mental Health Survey in Hong Kong. Arch Gen Psychiatry 1993; 50:125-133 [Abstract/Free Full Text]
3. Lindal E, Stefansson JG The lifetime prevalence of anxiety disorders in Iceland as estimated by the US National Institute of Mental Health Diagnostic Interview Schedule. Acta Psychiatr Scand 1993; 88:29-34 [Medline]
4. Kolada JL, Bland RC, Newman SC Epidemiology of psychiatric disorders in Edmonton. Obsessive-compulsive disorder. Acta Psychiatr Scand Suppl 1994; 376:24-35 [Medline]
5. Weissman MM, Bland RC, Canino GJ, The cross national epidemiology of obsessive compulsive disorder. The Cross National Collaborative Group . J Clin Psychiatry 1994; 55:5-10
6. Flament MF, Whitaker A, Rapoport JL, Obsessive-compulsive disorder in adolescence: an epidemiological study. J Am Acad Child Adolesc Psychiatry 1988; 27:764-771 [Medline]
7. Whitaker A, Johnson J, Shaffer D, Uncommon troubles in young people: prevalence estimates of selected psychiatric disorders in a non referred adolescent population. Arch Gen Psychiatry 1990; 47:487-496 [Abstract/Free Full Text]
8. Zohar AH, Ratzoni G, Pauls DL, An epidemiological study of obsessive-compulsive disorder and related disorders in Israeli adolescents. J Am Acad Child Adolesc Psychiatry 1992; 6:1057-1061
9. Reinherz HZ, Giaconia RM, Lefkowitz ES, Pakiz B, Frost AK Prevalence of psychiatric disorders in a community population of older adolescents. J Am Acad Child Adolesc Psychiatry 1993; 32:369-377 [Medline]
10. Thomsen PH Obsessive-compulsive disorder in children and adolescents: self-reported obsessive-compulsive behavior in pupils in Denmark. Acta Psychiatr Scand 1993; 88:212-217 [Medline]
11. Valleni-Basile LA, Garrison CZ, Jackson KL, Frequency of obsessive-compulsive disorder in a community sample of young adolescents. J Am Acad Child Adolesc Psychiatry 1994; 33:782-791 [Medline]
12. Douglas HM, Moffitt TE, Dar R, McGee R, Silva P Obsessive-compulsive disorder in a birth cohort of 18-year-olds: prevalence and predictors. J Am Acad Child Adolesc Psychiatry 1992; 6:1057-1061
13. Swedo SE, Leonard HL, Mittleman BB, Identification of children with pediatric autoimmune neuropsychiatric disorders associated with streptococcal infections by a marker associated with rheumatic fever. Am J Psychiatry 1997; 154:110-112 [Abstract]
14. Swedo SE, Leonard HL, Garvey M, Pediatric autoimmune neuropsychiatric disorders associated with streptococcal infections: clinical description of the first 50 cases. Am J Psychiatry 1998; 155:264-271 [Abstract/Free Full Text]
15. Perlmutter SJ, Leitman SF, Garvey MA, Therapeutic plasma exchange and intravenous immunoglobulin for obsessive-compulsive disorder and tic disorders in childhood. Lancet 1999; 354:1153-1158 [CrossRef][Medline]
16. Leonard HL, Swedo SE, Lenane MC, A 2- to 7-year follow-up study of 54 obsessive-compulsive children and adolescents. Arch Gen Psychiatry 1993; 50:429-439 [Abstract/Free Full Text]
17. Thomsen PH, Mikkelsen HU Course of obsessive-compulsive disorder in children and adolescents: a prospective follow-up study of 23 Danish cases. J Am Acad Child Adolesc Psychiatry 1995; 34:1432-1440 [CrossRef][Medline]
18. Anthony JC, Folstein M, Romanoski AJ, Comparison of the lay Diagnostic Interview Schedule and a standardized psychiatric diagnosis. Arch Gen Psychiatry 1985; 42:667-675 [Abstract/Free Full Text]
19. Helzer JE, Robins LN, McEvoy LT, A comparison of clinical and Diagnostic Interview Schedule diagnoses. Arch Gen Psychiatry 1985; 42:657-666 [Abstract/Free Full Text]
20. Scahill L, Riddle MA, McSwiggin-Hardin M, Children's Yale-Brown Obsessive Compulsive Scale: reliability and validity. J Am Acad Child Adolesc Psychiatry 1997; 36:844-852 [CrossRef][Medline]
21. Berg CZ, Whitaker A, Davies M, Flament MF, Rapoport JL The survey form of the Leyton Obsessional Inventory-Child Version: norms from an epidemiological study. J Am Acad Child Adolesc Psychiatry 1988; 27:759-763 [Medline]
22. King NJ, Myerson NN, Inglis S, Ollendick TH Obsessive-compulsive behaviour in children and adolescents: a cross-sectional Australian study. J Paediatr Child Health 1995; 31:527-531 [Medline]
23. Achenbach TM. Manual for the Child Behavior Checklist/4-18 and 1991 Profile. Burlington, VT: University of Vermont, Department of Psychiatry; 1991
24. Edelbrook C, Costello AJ Convergence between statistically derived behavior problem syndromes and child psychiatric diagnoses. J Abnorm Child Psychol 1988; 16:219-231 [CrossRef][Medline]
25. Weinstein SR, Noam GG, Grimes K, Stone K, Schwab-Stone M Convergence of DSM-III diagnoses and self-reported symptoms in child and adolescent inpatients. J Am Acad Child Adolesc Psychiatry 1990; 29:627-634 [Medline]
26. Jensen PS, Salzberg AD, Richters JE, Watanabe HK Scales, diagnoses and child psychopathology: I. CBCL and DISC relationships. J Am Acad Child Adolesc Psychiatry 1993; 32:397-406 [Medline]
27. Biederman J, Wozniak J, Kiely K, CBCL clinical scales discriminate prepubertal children with structured interview-derived diagnosis of mania from those with ADHD. J Am Acad Child Adolesc Psychiatry 1995; 34:464-471 [CrossRef][Medline]
28. Biederman J, Faraone S, Mick E, Moore P, Lelon E Child Behavior Checklist findings further support comorbidity between ADHD and major depression in a referred sample. J Am Acad Child Adolesc Psychiatry 1996; 35:734-742 [CrossRef][Medline]
29. Hanna GL Demographic and clinical features of obsessive-compulsive disorder in children and adolescents. J Am Acad Child Adolesc Psychiatry 1995; 34:19-27 [CrossRef][Medline]
30. King RA, Scahill L, Vitulano LA, Schwab-Stone M, Tercyak KP, Riddle MA Childhood trichotillomania: clinical phenomenology, comorbidity, and family genetics. J Am Acad Child Adolesc Psychiatry 1995; 34:1451-1459 [CrossRef][Medline]
31. Hanna GL Trichotillomania and related disorders in children and adolescents. Child Psychiatry Hum Dev 1997; 27:255-268 [CrossRef][Medline]
32. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition. Washington, DC: American Psychiatric Association; 1994
33. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Third Edition, Revised (DSM-III-R). Washington, DC: American Psychiatric Association; 1987
34. SAS Institute Inc. SAS/STAT User's Guide, Version 6, Fourth Edition. Cary, NC: SAS Institute Inc; 1989
35. Geller DA, Biederman J, Griffin S, Jones J, Lefkowitz TR Comorbidity of juvenile obsessive-compulsive disorder with disruptive behavior disorders. J Am Acad Child Adolesc Psychiatry 1996; 35:1637-1646 [CrossRef][Medline]
36. Stephens RJ, Sandor P Aggressive behaviour in children with Tourette syndrome and comorbid attention-deficit hyperactivity disorder and obsessive-compulsive disorder. Can J Psychiatry 1999; 44:1036-1042 [Medline]
37. Leckman JF, Grice DE, Boardman J, Symptoms of obsessive-compulsive disorder. Am J Psychiatry 1997; 154:911-917 [Abstract]