OBJECTIVES: Although many attention-deficit/hyperactivity disorder (ADHD) care models have been studied, few have demonstrated individual-level symptom improvement. We sought to test whether complementing basic collaborative care with interventions that address common reasons for symptom persistence improves outcomes for children with inattention and hyperactivity/impulsivity.
METHODS: We conducted a randomized comparative effectiveness trial of 2 care management systems for 6- to 12-year-old children being evaluated for ADHD (n = 156). All participants received care management with decision support. Care managers in the enhanced care arm also were trained in motivational and parent management techniques to help parents engage in their child’s treatment, address their own mental health needs, and manage challenging child behaviors. We used multivariable models to assess inattention, hyperactivity/impulsivity, oppositionality, and social skills over 1 year.
RESULTS: Both treatment arms generated guideline concordant diagnostic processes in 94% of cases; 40% of children had presentations consistent with ADHD. For the entire sample, there were no differences in symptom trajectories between study arms; mean differences in change scores at 12 months were –0.14 (95% confidence interval –0.34 to 0.07) for inattention; –0.13 (–0.31 to 0.05) for hyperactivity/impulsivity; –0.09 (–0.28 to 0.11) for oppositionality; and 3.30 (–1.23 to 7.82) for social skills. Among children with ADHD-consistent presentations, enhanced arm participants experienced superior change scores for hyperactivity/impulsivity of –0.36 (–0.69 to –0.03), oppositionality –0.40 (–0.75 to –0.05), and social skills 9.57 (1.85 to 17.28).
CONCLUSIONS: Among children with ADHD-consistent presentations, addressing barriers to engagement with care and challenging child behaviors has potential to improve the effectiveness of collaborative care.
What’s Known on This Subject:
Collaborative care is known to be an effective system to manage child behavioral health conditions in the primary care setting.
What This Study Adds:
Among urban children with attention-deficit/hyperactivity disorder, using lay care managers to address barriers to engagement with care and challenging child behaviors has the potential to improve the effectiveness of conventional collaborative care.
Attention-deficit hyperactivity disorder (ADHD) affects 8% to 12% of US school-aged children.1 Far more children, however, demonstrate functionally impairing inattention or hyperactivity/impulsivity, but do not meet full diagnostic criteria for ADHD. Although most children with ADHD symptoms are managed by their primary care provider,2,3 diagnostic processes and treatment plans in this setting often fail to uphold evidence-based standards.4 To address this gap between evidence and practice, a number of ADHD care management models have been developed. These models have focused largely on communication of patient data or assistance with medical decision-making (decision support) and have succeeded in improving guideline adherence for ADHD diagnosis and management.5–7 There is a paucity of literature, however, on the impact of these care models on individual level outcomes.
One approach to improving outcomes is to augment evidence-based models of care with lay-delivered interventions designed to address specific reasons for lack of clinical improvement. This approach could be particularly important for low-income and minority children, who have limited access to mental health specialists and often receive their care in underresourced settings.8,9 Although many reasons for symptom persistence exist, 3 reasons may be particularly salient for urban children. First, engagement with behavioral health care is often problematic.10–14 Second, low-income parents, particularly those of inattentive or hyperactive/impulsive children, have an increased prevalence of depression and adult ADHD,15 which have been associated with poorer child symptom trajectories.16,17 Third, behavioral comorbidities, particularly oppositionality, are highly prevalent among low-income populations, and can limit the effectiveness of strategies designed to address core ADHD symptoms.18,19
In this context, we conducted a randomized comparative effectiveness trial to test the hypothesis that augmenting a basic collaborative care system (whereby primary care clinicians work with specialists via care manager intermediaries) with interventions to address specific reasons for protracted symptoms could improve outcomes for low-income children with inattention and hyperactivity/impulsivity. Because we were interested in real-world applicability in a Medical Home context,20 all care management processes were conducted by lay providers without formal mental health backgrounds; and study procedures, whenever possible, adhered to principles of effectiveness research.21 Because we were interested in symptomatic improvement among all children presenting to primary care with relevant behavioral concerns, we enrolled subjects at the point of initiation of an ADHD workup, as opposed to the point of diagnosis.
In our 2-arm trial, all participants received care management and decision support, based largely on evidence-based collaborative care models that have been applied to different clinical populations.22,23 Care managers in the enhanced study arm also were trained in motivational interviewing techniques to help parents engage with care for their children, and to seek care for their own behavioral health concerns.24 These care managers also were trained in Triple P’s Primary Care module, a series of techniques effective in reducing coercive parenting and behavioral problems in children.25–27 We followed families for 1 year and measured children’s inattention, hyperactivity/impulsivity, oppositionality, and social skills.
Study Design and Setting
We conducted a 2-site, parallel-group comparative effectiveness trial with 1:1 randomization. Both sites were urban primary care pediatric practices serving low-income populations: one, within an academic medical center; the other, a community health center.
We recruited 6- to 12-year-old children, whose primary care provider initiated an evaluation for ADHD based on parental report of inattentive or hyperactive/impulsive behavior. Those with existing ADHD diagnoses, documentation of autism spectrum or bipolar disorder, and those already receiving care from a pediatric neurologist, psychiatrist, or developmental-behavioral pediatrician were excluded. We restricted our sample to English- and Spanish-speaking families.
Recruitment and Enrollment
We enrolled participants from October 2010 to July 2013. To emulate real-world circumstances, primary care providers initiated evaluations for ADHD at their own discretion, and made referrals via the electronic medical record for care management. A central coordinator offered care management services, irrespective of participation in the trial. Those participating in the trial subsequently met with study staff, who obtained written informed consent.
Eligible families were allocated to “basic” or “enhanced” collaborative care according to a computer-generated randomization list. Randomization was conducted independently at each study site in randomly varying blocks of 2 and 4. The list was concealed from all study personnel in sequentially numbered, opaque, sealed envelopes. Outcome assessors, investigators, and clinicians were masked to study allocation. Independent teams of care managers provided basic and enhanced services, and were not supervised by investigators.
Both study arms were modeled on the concept of collaborative care. We retained lay care managers (bachelor’s or master’s level) without formal mental health training or clinical experience. Care managers’ responsibilities were threefold: first, they obtained symptom reports, by using the Vanderbilt ADHD Diagnostic Rating scales,28 from both parents and teachers. These scales were obtained as part of the initial diagnostic process, and the ongoing monitoring process for those with ADHD-consistent presentations. To obtain teacher ratings, care managers sent forms, along with a signed release, to a centralized school administrator who in turn contacted teachers. Second, care managers received brief training in medical history taking, and obtained clinical information from families on behavioral symptoms and social and family history. Third, care managers served as liaisons between primary care providers and a decision support panel consisting of 1 child psychiatrist (HJW), 1 developmental-behavioral pediatrician (LKH), and 1 primary care pediatrician.
Decision support focused on whether clinical presentations were consistent with ADHD (combined, inattentive, hyperactive/impulsive, or not-otherwise-specified types), or whether alternative diagnoses should be entertained. An “ADHD-consistent presentation” was defined as meeting Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, symptom criteria on both the parent and teacher Vanderbilt scales, in the absence of a plausible alternative explanation for the child’s symptoms, as determined from narrative clinical information. If an ADHD diagnosis was confirmed by the primary care provider, the decision support team provided guideline-concordant pharmacological guidance,29 and advice on monitoring and treating to a target symptom score. Decision support was phrased in nonspecific terms: for example, if a suggestion was made to start a stimulant medication, a specific medication was not named. For those with ADHD-inconsistent presentations, the decision support team provided guidance about diagnostic next steps. The electronic health record provided the medium for communication.
Enhanced care managers received additional training to address 3 common reasons for ADHD symptom persistence: ambivalence toward engagement with behavioral health care,14 parental mental health,30 and oppositional child behavior.18 To address the first 2, care managers were trained in motivational interviewing31 and followed a standardized script to resolve ambivalence toward treatment. Consistent with previous work,25 care managers used motivational interviewing principles to explore parents’ stressors, and to offer referrals to adult behavioral health services if indicated. Last, enhanced care managers were certified by a master trainer in Triple P’s Primary Care module. Whereas basic care managers conducted their work in person only until the initial diagnostic decision support meeting (and by telephone thereafter), enhanced care managers met with clients in person as often as necessary to accomplish these aims.
Intervention Training and Fidelity
All enhanced care managers underwent a 5-day training in motivational interviewing conducted by a trainer certified by the Motivational Interviewing Network of Trainers, and were judged to have appropriate skills on assessment of videotaped interactions with standardized patients. Because our study was an effectiveness trial, we took an unobtrusive approach to care manager supervision and fidelity monitoring.21 Skills were maintained through weekly supervision sessions with a social worker and through annual booster sessions with the motivational interviewing trainer. During these annual sessions, the trainer assessed skills based on a standard set of criteria consistent with best practices in motivational interviewing.31 Triple P training involved a commercially available training and certification module. We implemented Triple P as it would be in a real-world practice setting and did not deploy any additional training than provided in the original module.
Measures and Outcome Assessment
At baseline, we recorded the age and gender of all children. We chose our outcome measures to be congruent with the Multimodal Treatment of ADHD Study,32 a seminal study comparing treatment approaches for children with ADHD. We used the Swanson, Nolan, and Pelham (SNAP-IV) scale to assess behavioral symptoms. This scale, which has a range of 0 to 3, has a factor structure that includes 2 ADHD subscales (inattention and hyperactivity/impulsivity) and an oppositionality subscale.33 We used standard scores from Social Skills Rating System to assess children’s social skills.34 For adult caregivers, we assessed age, relationship to the child, marital and work status, education, and race and ethnicity. We assessed caregiver depressive symptoms by using the Quick Inventory of Depressive Symptoms35; adult ADHD symptoms, by using the Adult ADHD Self-Report Scale36; and health literacy, by using the Brief Test of Functional Health Literacy.37 For each measure, we used published clinical cutpoints.
Our primary outcome was ADHD symptoms, measured by parent SNAP-IV report, 6 and 12 months after randomization. Secondary outcomes included oppositional symptoms and social skills.
We conducted an intention-to-treat analysis, excluding 1 subject who was found to be ineligible postrandomization. To assess between-group differences in changes from baseline for inattention, hyperactivity/impulsivity, oppositionality, and social skills, we used 2-sample t-tests.
By using multivariable linear regression models, we assessed differences in symptom changes across intervention groups, accounting for relevant baseline differences, and enrollment site as a fixed effect.38 We calculated standardized effect sizes as the adjusted difference between groups divided by the pooled SD, by using the Cohen d statistic. We conducted stratified analyses to determine if there was differential impact among children whose presentations were consistent, or inconsistent, with ADHD. We assessed effect modification by ADHD-consistent presentation by entering treatment arm–by–ADHD interaction terms into the models. Because outcomes data were missing in only 9% of subjects, we employed no special techniques to handle missing data.
To explore possible intervention mechanisms of action, we supplemented the intention-to-treat analysis with an as-treated analysis, in which we divided the sample into those receiving no enhanced care sessions, those receiving 1 to 3 sessions, and those receiving 4 to 5 sessions, irrespective of random assignment. We analyzed differences in outcomes via analysis of variance, restricting this analysis to 12-month outcomes because all intervention “doses” had been delivered by this time. We used the χ2 test to assess differences between study groups in ADHD medication prescription receipt and receipt of specialty behavioral health services. Small cell sizes precluded formal mediation analyses or multivariable modeling of potential mechanistic pathways.
We estimated our sample size to generate adequate power to test a clinically significant difference across intervention arms on decreasing SNAP-IV scores over 12 months. We assumed from the Multimodal Treatment of ADHD study that at baseline, SNAP-IV scores would average 2.2 (SD 0.70).39 We assumed that among subjects in the basic arm, follow-up scores would average 1.8 at 6 months and 1.4 at 12 months; in the enhanced arm, 1.7 at 6 months and 1.1 at 12 months, differences in score trajectories considered clinically significant.42,43 We assumed a 2-sided α of 0.05, 80% power, and 10% loss to follow-up, leading to our goal of randomizing 156 subjects.
The Boston University Medical Center Institutional Review Board approved this study (Clinical Trials.gov NCT01275378).
Primary care providers referred 418 children to the central coordinator (Fig 1). Of these, 116 did not meet inclusion criteria, 100 could not be contacted, and 46 refused to participate. Of the 156 subjects in the trial, 114 were recruited from the hospital clinic. One subject in the basic care arm was found to be ineligible after randomization and excluded. Of the 155 eligible, randomized subjects, 141 (91%) were assessed at the 12-month follow-up point.
Our sample of children was predominantly male (69%), with a mean age of 8.73 years (SD 2.07). The sample was 60% black and 27% Latina. Most baseline characteristics were balanced between the 2 study groups (Table 1); however, whereas 76% of caregivers in the enhanced care arm had attained a high school diploma, 63% in the basic care arm had. Overall, randomization appeared to be effective.
Baseline child inattention, hyperactivity/impulsivity, oppositionality, and social skills scores reflected a study population with the expected behavioral challenges. Based on SNAP-IV norms, 54% of the children in the sample scored in the top 5%, nationally, for inattention; 68%, in the top 5% for hyperactivity/impulsivity; and 31% in the top 5% for oppositionality. The mean rank of children in our sample for social skills was the 23rd percentile nationally. None of these scores differed between intervention groups at baseline.
Initial steps of the ADHD diagnostic process for children in both care groups adhered well to established quality metrics, including evaluation with symptom inventories and assessment of comorbid conditions (Table 2). Overall, 40% of cases were judged to be consistent with ADHD. By design, basic care managers did not meet with families in person after initial decision support was provided; by contrast, enhanced care managers had up to 5 in-person follow-up meetings (mean, 1.58; SD 1.73) with their clients. All enhanced care managers were deemed to have maintained motivational interviewing skills throughout the study period. Thirty-seven study parents (47%) in the enhanced care arm engaged in at least 1 aspect of Triple P; 7 parents (9%) engaged in a discussion about their own mental health. There were no known adverse events in either arm.
Inattention, Hyperactivity/Impulsivity, Oppositionality, and Social Skills
At both 6 and 12 months of follow-up, mean changes in scores for inattention, hyperactivity/impulsivity, oppositionality, and social skills were similar between the 2 arms (Table 3). Among the entire study population, multivariable models estimating symptom changes from baseline through 6 and 12 months demonstrated no significant differences across study arms (Table 4).
Among children with ADHD-consistent presentations, multivariable models estimating differences in symptom changes from baseline demonstrated better trajectories among those receiving enhanced care (Table 4). At 12 months, those in the enhanced arm experienced greater improvements in hyperactivity/impulsivity (–0.36; 95% confidence interval [CI] –0.69 to –0.03), oppositionality (–0.40; 95% CI –0.75 to –0.05), and social skills (9.57; 95% CI 1.85 to 17.28) than those in the basic arm. Effect sizes for these outcomes were moderate to large: 0.57 for hyperactivity/impulsivity, 0.55 for oppositionality, and 0.69 for social skills. Treatment arm-by-ADHD interaction terms indicated statistically significant effect modification for each of these measures (P = .04 for hyperactivity/impulsivity, P = .02 for oppositionality, P = .01 for social skills).
Intervention Dose, Medication Prescription and Specialty Behavioral Health Care
An unadjusted as-treated analysis suggested better 12-month change scores for inattention (P = .07) and social skills (P = .04) among children with ADHD-consistent presentations, whose families received 1 to 3 enhanced care sessions. Whereas the proportion of children with ADHD-consistent presentations to receive specialty behavioral health services was similar across study arms, there was a clinically meaningful (but statistically nonsignificant) difference in the proportion of children with ADHD-consistent presentations to be prescribed an ADHD medication (52% in basic care versus 72% in enhanced care; P = .10).
Our study tested the comparative effectiveness of 2 collaborative care systems on urban children’s trajectories of inattention, hyperactivity/impulsivity, oppositionality, and social skills. Across the entire sample, there were no differences in outcomes between study arms. However, among children with ADHD-consistent presentations, those in the enhanced care arm experienced better symptom trajectories than those in the basic care arm. Among this subgroup, differences between arms appeared greater in the second half of the follow-up period than the first, with moderate to large effect sizes observed.
Consistent with recent novel approaches to pediatric behavioral health,23,41 we enrolled children in our trial based on presenting symptoms, as opposed to ultimate diagnoses. This family-centered approach represents a departure from much of the extant ADHD literature. Furthermore, whereas most studies to date have tested care systems against usual care, our trial is novel in studying whether augmenting collaborative care with lay-delivered strategies to address common reasons for symptom persistence improves outcomes. These dimensions, along with our emphasis on urban, low-income children, make our results relevant to primary care providers, those interested in operationalizing the principles of the Medical Home,20 and those interested in designing care for vulnerable populations.
Because our intervention has multiple facets, determining which model component was responsible for the observed differences among children with ADHD-consistent presentations is difficult. Because only 9% of enhanced care parents discussed their own mental health, it is unlikely that this component was critical. Similarly, because receipt of specialty behavioral health services was equal across intervention groups, this was also unlikely a key driver of outcomes. By contrast, although there is no direct evidence that Triple P was responsible for our results, its uptake among enhanced care families was nearly 50%, and it could well explain enhanced care’s preferential impact on hyperactivity/impulsivity. Last, among children with ADHD-consistent presentations in the enhanced care group, there was a clinically meaningful increase in ADHD medication prescriptions. Given that the motivational interviewing script deliberately focused on medication use, it is possible that motivational interviewing started a cascade of events leading to increased receptivity to ADHD medication, and that this, in turn, could have led to improved outcomes.
The principal strengths of our study are its comparative effectiveness design and real-world applicability. These particular strengths, however, introduce 2 important limitations. The first is that disentangling the impact of individual intervention components, which likely act both independently and interdependently,42,43 often requires strict standardization in intervention implementation, which is largely impossible in an effectiveness design.44 This, combined with the fact that we would not expect the benefits of enhanced care to accrue through the same pathway for all study participants, limits our ability to isolate key mediating factors of our model. Second, staying true to an effectiveness study design limits us to monitoring intervention fidelity through unobtrusive means,21 which are inherently less exacting than techniques typically used in efficacy trials.
Additionally, our study was conducted in a single geographic area. Most of our subjects attended school in a district in which there was usually not a single, easily identifiable teacher who knew the child best; therefore, we confined our outcome assessments to parent report. Although a limitation, this approach to measuring outcomes is consistent with existing studies of collaborative behavioral health care for children,41 and previous work indicates that our outcome measures tend to be stable across parent and teacher reports.39
With these limitations in mind, it appears that adding certain theory-based components to a basic collaborative care system produces no added benefit for a population of urban children with symptoms of inattention and hyperactivity/impulsivity. However, among children with presentations consistent with ADHD, using lay care managers to address barriers to engagement and challenging child behaviors could have some added benefit. Because our positive findings occurred in a study subgroup, as opposed to the main sample, these findings need to be replicated in a larger study, powered not only to confirm intervention impact, but also to decipher its mechanism of action.
We thank Megan Bair-Merritt for her thoughtful review of the manuscript.
- Accepted January 15, 2015.
- Address correspondence to Michael Silverstein, MD, MPH, Vose Hall 3, Boston Medical Center, Boston, MA 02118. E-mail:
Dr Silverstein conceptualized and designed the study, oversaw its implementation, and drafted the initial manuscript; Drs Hironaka and Walter conceptualized and designed the study, oversaw its clinical implementation, reviewed and revised the entire manuscript, and assisted in the interpretation of analyses; Dr Feinberg assisted in conceptualizing and designing the study, oversaw its clinical implementation at one site, assisted in the interpretation of analyses, and reviewed and revised the manuscript; Ms Sandler managed the data for the project, assisted in preparing the analyses, and reviewed and revised the manuscript; Ms Pellicer supervised the care managers for the project, and reviewed and revised the manuscript; Ms Chen carried out the initial analyses and interpreted the data, and reviewed and revised the manuscript; Dr Cabral carried out the multivariable analyses, interpreted the data, and reviewed and revised the manuscript; and all authors approved the final manuscript as submitted.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Health Resources and Services Administration, Maternal Child Health Bureau, grant R40MC17181.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
COMPANION PAPER: A companion to this article can be found on page e1042, online at www.pediatrics.org/cgi/doi/10.1542/peds.2015-0070.
- Stein RE,
- Horwitz SM,
- Storfer-Isser A,
- et al
- Epstein JN,
- Kelleher KJ,
- Baum R,
- et al
- Leslie LK,
- Weckerly J,
- Plemmons D,
- Landsverk J,
- Eastman S
- Epstein JN,
- Rabiner D,
- Johnson DE,
- et al
- Epstein JN,
- Langberg JM,
- Lichtenstein PK,
- Mainwaring BA,
- Luzader CP,
- Stark LJ
- Committee on Psychosocial Aspects of Child and Family Health and Task Force on Mental Health
- ↵Medical Home Initiatives for Children With Special Needs Project Advisory Committee. The American Academy of Pediatrics. The medical home. Pediatrics. 2002;110(1):184–186
- Unützer J,
- Katon W,
- Callahan CM,
- et al.,
- IMPACT Investigators. Improving Mood-Promoting Access to Collaborative Treatment
- Sanders MR,
- Markie-Dadds C,
- Tully LA,
- Bor W
- ↵Subcommittee on Attention-Deficit/Hyperactivity Disorder; Steering Committee on Quality Improvement and Management. ADHD: clinical practice guideline for the diagnosis, evaluation, and treatment of attention-deficit/hyperactivity disorder in children and adolescents. Pediatrics. 2011;128(5):1007–1022
- Miller W,
- Rollnick S
- ↵Bussing R, Fernandez M, Harwood M, et al. Parent and teacher SNAP-IV ratings of attention deficit hyperactivity disorder symptoms: psychometric properties and normative ratings from a school district sample. Assessment. 2008;15(3):317–328
- Gresham FM,
- Elliot SN
- Rush AJ,
- Trivedi MH,
- Ibrahim HM,
- et al
- Garland MR
- MTA Cooperative Group
- Lewin S,
- Glenton C,
- Oxman AD
- Campbell M,
- Fitzpatrick R,
- Haines A,
- et al
- ↵Craig P, Dieppe P, Macintyre S, et al. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655
- Copyright © 2015 by the American Academy of Pediatrics