Psychosocial Interventions for Child Disruptive Behaviors: A Meta-analysis
- Richard A. Epstein, PhD, MPHa,b,
- Christopher Fonnesbeck, PhDa,c,
- Shannon Potter, MLISa,
- Katherine H. Rizzone, MD, MPHd, and
- Melissa McPheeters, PhD, MPHa,e
- aInstitute for Medicine and Public Health, Evidence-Based Practice Center, and
- dDepartment of Sports Medicine, Vanderbilt University Medical Center, Nashville, Tennessee;
- bDivision of Child and Adolescent Psychiatry, Department of Psychiatry,
- Departments of cBiostatistics, and
- eHealth Policy, Vanderbilt University School of Medicine, Nashville, Tennessee
BACKGROUND: Disruptive behavior disorders are among the most common child and adolescent psychiatric disorders and associated with significant impairment.
OBJECTIVE: Systematically review studies of psychosocial interventions for children with disruptive behavior disorders.
METHODS: We searched Medline (via PubMed), Embase, and PsycINFO. Two reviewers assessed studies against predetermined inclusion criteria. Data were extracted by 1 team member and reviewed by a second. We categorized interventions as having only a child component, only a parent component, or as multicomponent interventions.
RESULTS: Sixty-six studies were included. Twenty-eight met criteria for inclusion in our meta-analysis. The effect size for the multicomponent interventions and interventions with only a parent component had the same estimated value, with a median of −1.2 SD reduction in outcome score (95% credible interval, −1.6 to −0.9). The estimate for interventions with only a child component was −1.0 SD (95% credible interval, −1.6 to −0.4).
LIMITATIONS: Methodologic limitations of the available evidence (eg, inconsistent or incomplete outcome reporting, inadequate blinding or allocation concealment) may compromise the strength of the evidence. Population and intervention inclusion criteria and selected outcome measures eligible for inclusion in the meta-analysis may limit applicability of the results.
CONCLUSIONS: The 3 intervention categories were more effective than the control conditions. Interventions with a parent component, either alone or in combination with other components, were likely to have the largest effect. Although additional research is needed in the community setting, our findings suggest that the parent component is critical to successful intervention.
- ADHD —
- attention-deficit/hyperactivity disorder
- CBCL —
- Child Behavior Checklist
- CI —
- 95% confidence interval
- ECBI —
- Eyberg Child Behavior Inventory
- EOT —
- PCIT —
- Parent–Child Interaction Therapy
- RCT —
- randomized controlled trial
- SOE —
- strength of evidence
- TAU —
- treatment as usual
- Triple P —
- Positive Parenting Program
Disruptive behavior disorders are a group of related psychiatric disorders of childhood and adolescence marked by behaviors such as temper tantrums, interpersonal aggression, defiance, and persistent impairment.1–14 Recent estimates indicate that 3.5% of children between the ages of 3 and 17 years had behavioral or conduct problems from 2005 to 2011.15 Estimates suggest that disruptive behaviors that are problematic but do not meet formal diagnostic criteria may be even more common than those meeting formal clinical diagnostic criteria.16–20 The etiology of these disorders is unknown, but temperamental, biological, and environmental factors are associated with increased risk.21–25
Although effective preventive interventions have been developed for children and adolescents at risk for these disorders, practical considerations such as training requirements and cost often pose challenges to broad implementation.26,27 The importance of preventive interventions notwithstanding, children and adolescents with clinically significant disruptive behaviors need intervention. Psychosocial interventions for children and adolescents with clinically significant disruptive behaviors are heterogeneous. Existing reviews report positive outcomes for cognitive–behavioral therapy,28,29 behavior management,30,31 and parenting interventions, either alone or in combination with family-based approaches.32–35
At the same time, data suggest that the use of general outpatient psychotherapy, either alone or in combination with psychotropic medication management, has been declining and that the use of psychotropic medications has been increasing.36–43 Changing reimbursement rates and patterns have been suggested as a possible explanation for these trends,44 but it is also possible that these trends may reflect dissatisfaction with the effectiveness of the psychosocial interventions for children and adolescents with disruptive behavior disorders that are widely available in the community.
To help address questions about the effectiveness of these interventions and provide information about which types of interventions are most likely to be best, we conducted a systematic review and meta-analysis to synthesize existing literature on the comparative effectiveness of psychosocial interventions for children and adolescents <18 years of age with disruptive behavior disorders. To accomplish this while accounting for the large number of heterogeneous psychosocial interventions, we used a Bayesian multivariate, mixed treatment (network) meta-analytic approach that classified each intervention arm of each included study into 3 groups based on whether it was a treatment-as-usual (TAU)/wait list control, an intervention with only a child component, an intervention with only a parent component, or a multicomponent intervention. The review was funded by the Agency for Healthcare Research and Quality. The full report is available at http://www.effectivehealthcare.ahrq.gov.
Protocol and Registration
Our protocol is registered with PROSPERO, an international database of prospectively registered systematic reviews in health and social care (registration #CRD42014007552).
Eligible studies focused on treating disruptive behavior disorders and included children exhibiting disruptive behaviors as a primary problem. We excluded studies of children with attention-deficit/hyperactivity disorder (ADHD) unless the specific focus of treatment was on the non-ADHD disruptive behavior, because a recent and comprehensive review of ADHD interventions already exists.45 We excluded studies of disruptive behavior secondary to conditions in which disruptive behaviors were studied as symptoms or comorbidities (eg, substance abuse, developmental delay, intellectual disability, pediatric bipolar disorder, and ADHD).
We searched the Medline medical literature database (via the PubMed interface), Embase, and PsycINFO using broad terms for psychosocial interventions, as well as interventions by name (eg, Parent–Child Interaction Therapy [PCIT], Incredible Years (IY) programs, and Positive Parenting Program [Triple P]). We also hand searched recent systematic reviews and other relevant publications to identify additional studies not captured by the database searches.
Data Extraction and Analysis
We extracted study design, the study population description, the intervention description, and baseline and outcome data on constructs of interest from eligible studies. Data were initially extracted by 1 team member and reviewed for accuracy by a second, and included studies were described.
We used Bayesian multivariate, mixed treatment (network) meta-analytic methods46–48 to use both direct and indirect evidence for comparing a large number of different specific interventions. Of the 16 different measures used to assess parent reports of child disruptive behaviors about which we extracted data, we included in our analysis only the studies that used ≥1 of the 3 most prevalent: Eyberg Child Behavior Inventory (ECBI), Intensity Subscale49,50; ECBI, Problem Subscale49,50; or the Child Behavior Checklist (CBCL), Externalizing (T-score).51
To account for the large number of different specific interventions used by the constituent studies, we classified each arm of each study according to the most salient feature of the intervention. Specifically, intervention arms were categorized as TAU/wait list control, an intervention with only a child component, an intervention with only a parent component, or a multicomponent intervention. Multicomponent interventions were defined as including ≥2 of a child component, parent component, or other component (eg, teacher, family together). Although our definition of multicomponent interventions did not require a parent component be included, all of the interventions classified as multicomponent interventions in this study contained a parent component. We also included information about the control or TAU arms of included studies. Recognizing that these intervention categories encompass different individual interventions, we modeled each component as a random effect. This allowed variation in intervention effect within each category, because of factors not explicitly modeled here. All measurement instruments shared the same study arm treatment effect in our model, but the effect was scaled by the SD of the outcome variable.
Recognizing the potential for misclassification of treatments, there was some uncertainty about the correct classification of the PCIT intervention, here included with the multicomponent interventions because the focus of the intervention is on the parent–child interaction and includes the parent and child engaged together in activities even though the clinician interacts almost exclusively with the parent. To assess the effect of our decision, we ran our models under the alternative assumption that PCIT is an intervention with only a parent component. Doing so resulted in only nominal changes to the posterior estimates of the model parameters and did not affect the overall conclusions drawn from our results.
Randomized controlled trials (RCTs) that reported baseline and end-of-treatment (EOT) means and SDs from 1 of the 3 metrics listed earlier (n = 28) were included in the meta-analysis. The response measure was defined as the EOT mean minus the baseline mean, and the associated variances were calculated as the sum of the baseline and EOT variances. The 3 outcomes were modeled jointly as a multivariate normal likelihood, with any unmeasured outcomes treated as missing data; doing so allowed the covariance between measures to be accounted for and estimated.
We included the age of participants in each study arm (broadly grouped into preschool age, school age, or teenage) in the model as a categorical covariate. We used the school age category as the baseline value because it was the most prevalent among studies. The age covariate was combined additively with the intervention component effects and control/TAU means to model the observed treatment differences relative to baseline. Although we considered age–treatment interactions, there was not enough balance between the age and treatment combinations to include them in the final model.
We gave all unknown parameters weakly informative prior distributions and estimated them by using Markov chain Monte Carlo52 methods via the PyMC 2.3 software package (Python Software Foundation).53 The model was run for 200 000 iterations, with the first 150 000 samples conservatively discarded as burn-in, leaving 50 000 for inference. Convergence diagnostics showed no evidence for lack of convergence in the 50 000 samples used for inference. We assessed model fit by using posterior predictive checks,54 which revealed no strong evidence of lack of fit.
Assessment of Study Quality and Strength of Evidence
Our primary outcomes for analysis and strength of evidence were parent reports of child disruptive behaviors as assessed using commonly used, validated measures. We used the Cochrane Risk of Bias Tool55 to assess risk of bias for RCTs and the RTI Item Bank56 to assess risk of bias for study designs other than RCTs. Two team members independently assessed each included study, with discrepancies resolved through discussion to reach consensus or adjudication by a senior reviewer. The results of these assessments were translated to low, moderate, or high risk of bias.
Two senior investigators graded the strength of the body of evidence by using methods based on the Methods Guide for Effectiveness and Comparative Effectiveness Reviews.57 The team reviewed the final strength of evidence (SOE) designation, with possible grades being high SOE (high confidence that the evidence reflects the true effect, additional research is unlikely to change estimates), moderate SOE (moderate confidence that the evidence reflects the true effect, additional research may change our confidence in the estimate of effect and may change the estimate), low SOE (low confidence that the evidence reflects the true effect, additional research is likely to change confidence in the estimate of effect and is likely to change the estimate), or insufficient SOE (evidence is unavailable or does not permit a conclusion).
Study Selection and Study Characteristics
We identified 7470 potentially relevant titles or abstracts, of which 968 proceeded to full text review. We excluded 853 studies at full text review and included 84 unique studies (in 115 publications). Of the 84 unique studies, 66 studies (59 RCTs, 7 non-RCTs) addressed psychosocial interventions (Fig 1); the rest (n = 18) addressed pharmacologic interventions, harms, or moderators of effects and are described in the full report and in a separate publication.
Table 1 describes the 66 studies58–123 by age group. Twenty-four studies were rated as high risk of bias, 34 as moderate risk of bias, and 8 as low risk of bias. About half of the studies (25) were conducted in the United States; the remaining studies were conducted in Australia (11), Canada (4), Germany (3), Ireland (2), Israel (2), Italy (1), Netherlands (5), Norway (4), Puerto Rico (1), Sweden (3), and the United Kingdom (5). The “experimental” intervention arm of each study was used to broadly categorize studies as focusing on interventions with only a child component (2 studies), only a parent component (25 studies), or on multicomponent interventions (39 studies). Specific interventions were classified into these broad categories.
Two RCTs (1 high and 1 moderate risk of bias) evaluated interventions with only a child component (1 school age and 1 adolescent). Twenty-five studies evaluated interventions with only a parent component (14 preschool age, 11 school age). Twenty-one of these 25 studies were RCTs (7 high, 12 moderate, and 2 low risk of bias); 4 were non-RCTs (3 high and 1 moderate risk of bias). Thirty-nine studies evaluated multicomponent interventions (9 preschool age, 17 school age, 13 adolescent). Thirty-six were RCTs (11 high, 19 moderate, and 6 low risk of bias); 3 were non-RCTs (2 high and 1 moderate risk of bias).
Of the 66 studies (59 RCTs, 7 non-RCTs) that addressed psychosocial interventions, 28 studies* met the additional criteria for inclusion in our meta-analysis.
Results from our meta-analysis show that all intervention categories were more effective than the TAU/control category, with high residual variability within category and overlap between categories (Fig 2, Supplemental Fig 3). The effect size for the multicomponent and parent-only intervention categories had the same estimated value, with a median of −1.2 SD reduction in outcome score (95% credible interval [CI], −1.6 to −0.9). The estimate for interventions with only a child component was −1.0 SD (95% CI, −1.6 to −0.4). The multicomponent intervention and parent-only intervention categories also had identical posterior probabilities of being the best treatment (both 43%), followed by interventions with only a child component (14%). Table 5 shows the estimated probability of remaining above the clinical cutoff for each intervention category by age group and outcome measure.
Age effects were more subtle, with an additive median effect of −0.4 SD (95% CI, −0.6 to −0.3) for preschool relative to school-age children (baseline level) and of −0.1 SD (95% CI, −0.5 to 0.2) for adolescents relative to school-age children. For example, in comparison with school-age children, preschool-age children experienced greater improvement in parent reports of child disruptive behaviors. These trends were evident across each of the outcome measures included in the analysis.
Random effect variances describe additional variation in the output beyond that accounted for by the factors included in the model. The mean estimates were 0.18 (SD 0.034; 95% CI, 0.12 to 0.25) for ECBI Intensity score, 0.17 (SD 0.038; 95% CI, 0.09 to 0.24) for ECBI Problem score, and 0.13 (SD 0.027; 95% CI, 0.08 to 0.18) for CBCL Externalizing T score.
Our review of the literature examining psychosocial interventions for child disruptive behavior included 66 studies (59 RCTs, 7 non-RCTs). Among the 66 studies, the “experimental” treatment arm of 2 studies examined interventions with only a child component, 25 studies examined interventions with only a parent component, and 39 studies examined multicomponent interventions. This categorization describes the treatment arm regarded as the experimental group. Specific named interventions were categorized accordingly, with the most common named interventions including the Incredible Years, PCIT, Triple P, and Multisystemic Therapy.
Of those 66 studies, 28 met the additional criteria for inclusion in our Bayesian multivariate, mixed treatment (network) meta-analysis. Our meta-analytic model suggested that interventions categorized as multicomponent interventions and interventions with only a parent component were approximately equivalent in their expected effectiveness (43% probability of being best treatment), whereas interventions with only a child component were estimated to be less effective (14% probability of being best). However, it should be noted that the estimate for child-only interventions was imprecise.
Our overall assessment is that there is a moderate strength of evidence supporting the effectiveness of multicomponent interventions and interventions with only a parent component, and because there were so few studies designed to evaluate interventions with only a child component, there is insufficient evidence to support interventions with only a child component. It is important to note that all interventions categorized as multicomponent interventions in this study included a parent component. Given recent trends indicating reduced use of behavioral health services and increasing use of psychotropic medications, especially for children with disruptive behavior disorders,36–39,41–43,124 we believe these findings have important policy and practice implications. For example, it might be possible for policymakers to incentivize provision of interventions including a parent component, either alone or in combination with other components, as opposed to TAU.
It should be noted that the populations studied in the articles we reviewed were mostly male and that approximately half of the studies included in the review were of school-age children. We defined a study as focusing on school-age children if it had a sample with a mean age between 5 and 12 years. We established 5 years of age as the lower bound because this is the age at which children typically begin attending kindergarten in the United States. We established 12 years of age as the upper bound because 13 years is regarded as the beginning of adolescence in casual parlance. For precisely these reasons, the age group classification has face validity in the United States but is somewhat arbitrary.
In addition to the age restriction, our definition of the target population included children with disruptive behaviors receiving treatment in health care settings. We did not restrict our study population to children meeting formal diagnostic criteria for a disruptive behavior disorder and allowed children without a diagnosed disruptive behavior disorder but with disruptive behaviors above a measure-specific threshold on well-validated measures of child disruptive behavior to be included. Furthermore, our review did not focus on preventive interventions for an at-risk population.
Additionally, the interventions most commonly examined in the studies included in this review were typically provided in academic settings and may not be widely available in real-world clinical settings. If and when they are available in real-world clinical settings, it can be difficult, if not impossible, to determine whether they are being implemented with fidelity to the intervention model. Similarly, the studies did not address the effectiveness of psychosocial interventions delivered concurrently with pharmacologic interventions. Thus, there may be limited ability to assess the applicability of our findings to clinical settings in which many children and adolescents seeking treatment of disruptive behaviors may have complex challenges125 and are increasingly likely to receive psychotropic medications.36–39,41–43,124
The literature we reviewed did not clearly identify primary outcomes or report random sequence generation or allocation concealment procedures. In addition, blinding was rarely attempted. Although there are well-recognized challenges and valid reasons for not achieving this level of control in the studies, this limitation introduces potential risk of bias to studies in this literature.
Few studies measure similar outcomes for synthesis. The lack of clearly identified primary outcomes reflects a lack of consensus on the most important outcomes. Methodologically, outcomes such as direct observation by a blinded and independent observer are arguably the most valid. However, direct observations can be expensive and are not always logistically feasible. From the perspective of patient-centered outcome research, we believe that there is a strong argument in this literature to be made in favor of the importance of parent-reported outcomes, even though it introduces a risk of bias in the absence of patient blinding because most psychosocial interventions included a parent component. Furthermore, results from mixed models are not always presented in a straightforward manner, making it very difficult to tease out effects of specific treatment approaches.
Conflict of interest is a concern in this evidence base. Many studies that evaluated a psychosocial intervention for a child disruptive behavior were conducted by the developer of the intervention or by an “intellectual descendant” of the developer. Although this conflict is understandable and akin to that of industry-sponsored clinical drug trials, the evidence from this body of literature could be strengthened with more studies replicated by researchers having no intellectual conflict of interest.
For these reasons, future research needs are both substantive and methodological. Future research should consistently and adequately describe randomization and allocation procedures and attempt blinding when feasible. Future research should clearly describe the duration of time from baseline to posttreatment and from posttreatment to follow-up and should more clearly describe results from mixed models. Future research should clearly identify the target population and address the portability of interventions from predominantly university research clinics to real-world clinical settings with implementation fidelity. Information from additional well-designed and reported head-to-head comparisons of psychosocial interventions, the effectiveness of psychosocial interventions as compared with pharmacologic interventions, and the effectiveness of combined psychosocial and pharmacologic interventions is urgently needed to aid clinical decision-making.
In conclusion, this review suggests that psychosocial interventions for children with disruptive behavior disorders that include a parent component, either alone or in combination with other intervention components, are most likely to reduce problem behaviors. Although this conclusion must be considered in light of the methodological limitations of the existing literature and current review, we believe these findings have potentially important implications. Parents need this information to make informed decisions about which treatments to seek for their children. For practitioners, these findings suggest the importance of working with parents when attempting to reduce disruptive child behaviors. This information is needed by clinicians to help them decide which interventions to be trained to deliver and to recommend to their patients. Policymakers may consider incentivizing psychosocial interventions that include a parent component to increase the delivery of interventions that have the greatest potential to improve care for these vulnerable children and families. Finally, our findings can help guide both clinicians and funders in addressing important gaps in research regarding these interventions.
The authors thank the technical experts and review team who contributed to the full report. The authors gratefully acknowledge the valuable input and guidance from Joanna Siegel, ScD, task order officer with the Agency for Healthcare Research and Quality.
Ms Tanya Surawicz, Ms Mamata Raj, and Ms Shanthi Krishnaswami contributed to the data collection and interpretation. Ms Katherine Worley coordinated and supervised data collection and assisted in the preparation of the manuscript.
- Accepted August 25, 2015.
- Address correspondence to Richard A. Epstein, PhD, MPH, Chapin Hall at the University of Chicago, 1313 East 60th St., Chicago, IL 60637. E-mail:
Dr Epstein conceptualized and designed the study and drafted the initial manuscript; Dr Fonnesbeck helped conceptualize the study, conducted the meta-analysis, and drafted the initial manuscript; Ms Potter and Drs Rizzone and McPheeters carried out the initial analyses and reviewed and revised the manuscript; and all authors approved the final manuscript as submitted.
FINANCIAL DISCLOSURE: Funded under contract 290-2012-00009-I from the Agency for Healthcare Research and Quality, US Department of Health and Human Services. The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the US Department of Health and Human Services.
FUNDING: Supported by the Agency for Healthcare Research and Quality (contract HHSA290201200009I).
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
* Refs 60, 62, 64, 66, 72, 79, 80, 83, 85, 88, 91, 95, 99–101, 103–106, 108, 109, 113, 115, 117–120, and 123.
- Copyright © 2015 by the American Academy of Pediatrics