Abstract
BACKGROUND: Although many efforts have been made to improve the quality of care delivered to children with attention-deficit/hyperactivity disorder (ADHD) in community-based pediatric settings, little is known about typical ADHD care in these settings other than rates garnered through pediatrician self-report.
METHODS: Rates of evidence-based ADHD care and sources of variability (practice-level, pediatrician-level, patient-level) were determined by chart reviews of a random sample of 1594 patient charts across 188 pediatricians at 50 different practices. In addition, the associations of Medicaid-status and practice setting (ie, urban, suburban, and rural) with the quality of ADHD care were examined.
RESULTS: Parent- and teacher-rating scales were used during ADHD assessment with approximately half of patients. The use of Diagnostic and Statistical Manual of Mental Disorders criteria was documented in 70.4% of patients. The vast majority (93.4%) of patients with ADHD were receiving medication and only 13.0% were receiving psychosocial treatment. Parent- and teacher-ratings were rarely collected to monitor treatment response or side effects. Further, fewer than half (47.4%) of children prescribed medication had contact with their pediatrician within the first month of prescribing. Most variability in pediatrician-delivered ADHD care was accounted for at the patient level; however, pediatricians and practices also accounted for significant variability on specific ADHD care behaviors.
CONCLUSIONS: There is great need to improve the quality of ADHD care received by children in community-based pediatric settings. Improvements will likely require systematic interventions at the practice and policy levels to promote change.
- AAP —
- American Academy of Pediatrics
- ADHD —
- attention-deficit/hyperactivity disorder
- DSM-IV —
- Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition
- EHR —
- electronic health record
What’s Known on This Subject:
In 2000/2001, the American Academy of Pediatrics published recommendations for attention-deficit/hyperactivity disorder (ADHD) care. According to pediatricians’ self-report of adoption of these guidelines, community-based ADHD care appears to be marginally adequate.
What This Study Adds:
Using reviews of >1500 patient charts, this study demonstrates that community-based ADHD care is not consistent with evidence-based practice. Furthermore, variability in much of community-based ADHD care is unrelated to the provider, suggesting that innovative, system-wide interventions are needed to improve ADHD care.
Most children with attention-deficit/hyperactivity disorder (ADHD) receive care from community-based pediatricians.1,2 According to pediatricians’ self-report, the quality of ADHD care delivered by community-based pediatricians is modest.3–7 For example, in a large survey (n = 1374), 78% of pediatricians reported using ADHD guidelines, 55% reported following Diagnostic and Statistical Manual of Mental Disorders (DSM) criteria to diagnose ADHD, and approximately 80% reported routinely collecting parent and teacher ratings during assessment.4 However, recent research reviewing ADHD patient charts suggests a poorer quality of care.8 Among 49 pediatricians from 8 practices, only 20% of patients were evaluated with parent or teacher rating scales during ADHD assessment. Further, none of the pediatricians used ADHD parent and teacher ratings to monitor treatment progress.8
The small number of practices sampled in Epstein et al8 did not allow for analyses of whether variability in ADHD care was associated with patients, pediatricians, and/or practices. Also, although some investigators speculated that rural location, academic affiliation, and serving patients of low socioeconomic status may affect patient care,9–15 there are few published data about the effect of practice characteristics on ADHD care. Improving care for complex, chronic conditions requires understanding the sources and predictors of variability to know which factors to address. For example, a low rate of care but with significant variability attributable to practices or pediatricians indicates that some practices and/or pediatricians can implement this care. This in turn suggests that changes at the practice or physician level (eg, adopting a policy requiring completion of teacher scales before scheduling an ADHD assessment visit) can improve these areas of care. Conversely, care behaviors for which variability can be explained only at the patient level would suggest that current systems of care and incentives for engaging in these care behaviors are not effective at targeting these behaviors. Rather, system-level interventions (eg, plan-based care coordination or pay-for-performance) may be necessary to change these care behaviors.
This study presents rates and sources of variability (eg, patient, pediatrician, or practice level) for ADHD care16–18 derived from chart reviews across 50 socioeconomically and demographically diverse pediatric practices. In addition, relations between practice characteristics and ADHD care are examined with the prediction that rural and high-poverty settings predict poorer ADHD care.
Methods
Participants and Setting
Practices were recruited from August 2010 through December 2012 to participate in a study focusing on improving the quality of community-based ADHD care. The data presented reflect rates of ADHD care before any intervention. To recruit, a mailing was sent out to 128 practices in Central and Northern Ohio that served primarily children, had at least 2 pediatricians, and did not have access to an on-site mental health professional. The first 50 pediatric practices that responded and met our inclusion criteria were selected to participate. The remaining practices either did not respond, responded late, chose not to participate because they refer out all patients for ADHD care, or declined. The 50 participating practices included 188 health care providers (184 pediatricians and 4 nurse practitioners).
Chart Reviews
We reviewed charts to assess pediatricians’ ADHD care practices. To select patient charts, we retrieved billing records with an ADHD diagnosis code during the past year. Coders randomly selected 10 patients per practitioner by selecting every nth patient from the list, where n = (number of patients on the billing query)/10. To ensure that ADHD assessment care was represented in the chart review, we attempted to include at least 5 patient charts per pediatrician with evidence of an ADHD assessment. Because these chart reviews required a review of retrospective patient charts, a waiver of consent was granted from author institutional review boards on the condition that no identifying or demographic information from the patient charts would be recorded.
The following information was extracted from each patient chart for any ADHD care between 2002 (the year after the American Academy of Pediatrics [AAP] ADHD guidelines were released) and the date of the chart review (August 2010 through December 2012): (1) presence of parent and teacher ratings during ADHD assessment, (2) documentation of whether child met Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) ADHD criteria, (3) date of ADHD diagnosis, (4) documentation of ADHD medication prescription, (5) date of initial ADHD medication prescription, (6) documentation of behavior therapy suggested, (7) dates of ADHD-related treatment visits and contacts (ie, phone, E-mail correspondence), and (8) dates of collection for parent- and teacher-completed ADHD rating scales.
Measurement of Practice Characteristics
Pediatricians reported the percentage of their patients whose primary payer was Medicaid. They also reported if their practice was affiliated with an academic medical center and classified the setting of their practice as urban, suburban, or rural.
Statistical Analyses
Patients were nested within pediatricians and pediatricians were nested within practices. We computed all descriptive estimates by modeling the multilevel nature of the data. By using multilevel models, we estimated the percentage of variability in each ADHD care variable across patients, pediatricians, and practices and statistically tested whether these estimates differed from zero.
SAS Proc Mixed (SAS Institute, Inc, Cary, NC) was used to model the continuous variables (eg, number of ADHD treatment contacts) using Kenward-Roger19 degrees of freedom for fixed-effect parameter estimate tests. Mplus version 7.11 (Muthen & Muthen, Los Angeles, CA) was used to estimate the predictor variables effects on binary response variables (ie, presence or absence of rating scale in patient chart).
In predicting treatment-related ADHD care, we used the “time to events” as indices of the timeliness of ADHD treatment. We defined duration as the number of days from when the patient was initially prescribed medication until the relevant event (eg, the collection of the teacher rating). Cox proportional hazards regression models with clustering of patients under pediatricians and using robust SEs were conducted to assess the association between practice characteristics and the times to events. For some patients, the target event had not occurred between the time of the initial prescription and the date of the chart review. For those cases, the time to the event was the time from prescribing until the time of the chart review, and the observation was right-censored in the analyses. All Cox analyses were conducted using R (version 3.01 Gentleman & Ihaka, Auckland, NZ).
Results
The mean age of the 188 health care providers was 43.5 (SD 9.5) years. The average number of years since providers finished their training was 12.9 years (SD 9.1). Most providers were white (n = 158, 86%) and women (n = 117, 64%).
Across the 188 providers, 1594 patient charts were reviewed. Of those, 1098 patient charts provided information about both ADHD assessment and ADHD treatment and 496 patient charts included information about ADHD treatment only.
Descriptives
ADHD Assessment Care
Pediatricians used parent and teacher ratings of ADHD during the ADHD diagnostic process 56.7% (SE 3.7%) and 55.5% (SE 3.6%) of the time, respectively. Evidence of children meeting DSM-IV criteria for ADHD was documented in patient charts 70.4% (SE 3.4%) of the time. Most of the variability in the assessment variables was at the patient level, but significant variability was also present at the pediatrician and practice levels (Table 1).
ADHD Care Descriptive Statistics and the Percentage of Variance Accounted for by Practices, Pediatricians, and Patients
ADHD Treatment Care
Of all children assessed for ADHD (n = 1098), pediatricians prescribed medication in 93.4% (SE 0.8%) of cases, whereas documentation that psychosocial treatment (eg, parent training, therapy) was recommended or being used by families was present in only 13.0% (SE 2.0%) of charts. Most of the variability on these 2 variables was accounted for at the patient level. However, significant variability for the psychosocial treatment variable also was also present at the pediatrician and practice levels (Table 1).
Among children with at least 30 days between prescribing medication and the chart review (n = 1518), 47.4% (SE 1.9%) had visit or phone contacts during the first month after prescribing medication. Of these contacts, 53% of patients had at least 1 office visit, 35% had at least 1 phone contact, and 12% had at least 1 visit and 1 phone contact during the first month of treatment. There were no instances of E-mail contacts during the first month of treatment. Across all children with at least 1 such contact (n = 1405), the average time from the initial prescription to first contact was 72.4 (SE 5.4) days. Times until the second and third contacts were 147.7 (SE 8.1; n = 1226) days and 226.4 (SE 10.3; n = 1007) days, respectively (Fig 1).
Timeline representing mean durations (with SEs) until first, second, and third treatment contacts and obtaining parent and teacher ratings after medication initiation.
Of those patients with at least 1 year between prescribing medication and the chart review (n = 784), children averaged 5.7 (SE 0.2) contacts in the first year of treatment. Most of these contacts were office visits (4.5 visits per year), with 1.3 per year being phone contacts. E-mail contacts (0.01 E-mails/year) were rare. Fewer contacts occurred during the second (3.1 [SE 0.2] contacts; n = 333 patients) and third years (2.8 [SE 0.2] contacts; n = 168 patients) of treatment.
Finally, with regard to monitoring treatment response with objective ratings, a minority of patient charts had evidence of parent (10.8%, SE 2.7%) or teacher ratings (7.5%, SE 1.5%) within the first year of treatment. Moreover, the average time to collection of parent (396.2 days, SE 181.6 days) or teacher ratings (362.6 days, SE 104.9 days) after prescribing medication for those with rating scales was quite long (see Fig 1).
Most of the variability in patterns of ADHD treatment contacts and follow-up practice behavior occurred at the patient level (Table 1). Significant variability also occurred at the pediatrician level and practice level for a few ADHD care treatment variables (eg, contact within the first month; Table 1).
Prediction of Evidence-Based Care
Medicaid Status
Pediatricians varied in the reported proportions of Medicaid patients in their panels (range = 0% to 99%; mean = 44.6%, SD 30.5%). Approximately 25% of pediatricians (n = 39) reported an affiliation with an academic medical center. Multilevel models were constructed to determine whether receiving Medicaid assistance (percent receiving Medicaid), a dichotomous indicator of academic affiliation, and their interaction predicted each of the selected ADHD care variables. Only psychosocial treatment demonstrated a relationship with any of the predictors. There was a significant main effect for academic affiliation and a significant Medicaid × academic affiliation interaction. As the proportion of Medicaid patients at a practice increased, rates of psychosocial treatment increased at nonacademic practices and decreased at academic practices (Fig 2). Cox models examining time-to-event variables found largely no relations between Medicaid percentage and academic affiliation (Table 2). The exceptions were that academic practices had a shorter time to the first contact. Also, having more Medicaid patients predicted a longer time to collecting parent ratings to monitor treatment response.
Graphical depiction of significant interaction between proportion of patients receiving Medicaid and academic affiliation on rates of receiving psychosocial treatment. For purposes of illustration, the proportion of patients receiving Medicaid was centered to illustrate how practices with an average proportion of Medicaid patients (44.6%; coded 0 on the x-axis of graph) compares with practices with 1 SD fewer Medicaid patients (30.5%; coded −1 on x-axis of graph) and to practices with 1 SD more Medicaid patients (58.7%; coded 1 on x-axis of graph).
Results From Regression and Cox Modeling Using the Percentage of Clinical Care Population Receiving Medicaid Assistance and Academic Affiliation to Predict Pediatrician-Delivered ADHD Care
Practice Location
Fifty-three pediatricians reported being located in urban settings and 103 self-reported as suburban and 17 as rural. Pediatricians in urban settings used medications more often than suburban pediatricians. Also, urban and rural pediatricians used psychosocial treatments more frequently than suburban pediatricians. Patients at suburban practices had shorter durations between medication initiation and the first contact than urban-based practices (Table 3).
Results From Regression and Cox Modeling Using Practice Location to Predict Pediatrician-Delivered ADHD Care
Associations Among Practice Characteristics
Practice location, percent Medicaid, and academic status of practices were strongly associated. Academic practices primarily served a Medicaid population (83.5%) compared with an average of 33.0% Medicaid patients for nonacademic practices (P < .0001). Similarly, suburban practices have relatively few Medicaid patients (26.5%) compared with rural (47.9%) and urban (78.1%) practices. Academic practices are primarily concentrated in urban settings (64.3% of urban practices), compared with 3.1% in suburban settings, and a complete absence of academic practices in rural locations (P < .0001, Fisher's exact test). This interdependence among practice characteristics prevented us from identifying independent effects (eg, multiple regressions) of any single predictor on quality of care.
Discussion
Rates of pediatrician-delivered evidence-based ADHD care across a diverse set of pediatric practices were determined by using reviews of a randomly selected sample of patient charts. Pediatricians used parent and teacher rating scales during ADHD assessment with approximately half of patients, and used DSM criteria in approximately two-thirds of patients in community-based pediatric settings. Based on information in patient charts, the vast majority of ADHD patients received medication and few received psychosocial treatment. Few pediatricians used parent and teacher rating scales to monitor treatment response and side effects (∼10%) despite AAP consensus recommendations17 to do so. In addition, fewer than half of children prescribed medication had contact with their pediatrician within the first month of treatment. Further, during the first year of medication, follow-up contacts occurred approximately every 2.5 months with lower rates of follow-up contact in subsequent years. Thus, despite the publication of ADHD consensus guidelines more than a decade ago,16,17 adoption of evidence-based ADHD care in community-based pediatric settings remains poor.
These marked deviations from recommended practice fell into 2 categories: (1) ADHD care behaviors for which significant variability occurred at the pediatrician or practice level (and thus, might be amenable to practice interventions), and (2) ADHD care behaviors for which variability was attributable primarily to patients. The first category included collection of parent and teacher rating scales during assessment, use of DSM criteria to document ADHD, prescribing psychosocial treatment to patients, contacting parents within 1 month of initiating treatment, number of contacts in the first year of treatment, and use of parent and teacher ratings to monitor treatment response. Because pediatricians and practices were associated with variability in rates of ADHD care behaviors across patients, we believe that the adoption of pediatrician- or practice-level modifications to ADHD systems of care can improve these areas of care.
The second category of ADHD care behaviors does not vary according to practice or physician. These behaviors included number of contacts in the second and third years of treatment; time to first, second, and third contacts; and time to collection of parent and teacher ratings to monitor treatment response. These patient-care activities are driven largely by patients. For pediatricians to improve these ADHD care behaviors, it seems that pediatricians would need to take on additional responsibility for patient tracking (eg, using patient registries to track contacts, appointments, and rating scale collection), and possibly use innovative technologies to prompt and facilitate patient and family engagement.20 Increased incentives (eg, pay-for-performance) or system-level interventions at the community or health plan may be necessary to promote adoption of these expensive and demanding initiatives.
We also examined practice/pediatrician characteristics (ie, percentage of Medicaid patients, academic affiliation, and practice setting) to determine whether these characteristics explain variability in practice behaviors. Only a few practice/pediatrician characteristics were associated with the quality of ADHD care. For example, consistent with other studies,10,21 Medicaid status was associated with longer times until the collection of parent ratings to monitor response. However, for other indicators of ADHD care quality, Medicaid status was not associated with worse ADHD care. Interestingly, increasing proportions of Medicaid patients were associated with higher rates of psychosocial treatment at nonacademic practices, whereas higher rates of Medicaid patients led to lower rates of psychosocial treatment at academic practices. Perhaps academic pediatricians refer within their affiliated medical center, which may have long waits for psychosocial treatment and these delays lead to lower rates of psychosocial treatment.
Practice setting also has been associated with quality of care in previous research.11,12,22,23 In particular, rural settings typically offer a poorer quality of care than metropolitan settings.23 In our study, we found no negative effects of rural setting on ADHD care behaviors. We did find that patients seen for ADHD by pediatricians in a suburban setting were less likely to be prescribed medication after an ADHD diagnosis compared with urban patients. Similarly, suburban patients had lower rates of psychosocial treatment than urban or rural patients. One possible explanation is that there was reluctance or delays (either by providers or families) in suburban settings toward beginning treatment or accepting the diagnosis. Alternatively, patients in urban settings may have presented with more severe symptoms requiring immediate initiation of treatment. The other geographic difference was a tendency for suburban practices to have shorter times to first contact after starting treatment compared with urban practices. We surmise that this difference may have resulted from the higher rates of no-shows and overall patient volume often associated with urban settings, which could have been obstacles to getting families to return for follow-up care.
This study’s findings must be interpreted in light of several study limitations. First, our chart review methodology did not include collection of personal patient data, such as age, gender, and comorbid conditions, from patient charts. Hence, the relation between patient-level data and quality of ADHD care could not be estimated. Parent preferences for ADHD care also were not collected, and thus the influence of parent preferences on patterns of ADHD care was not accounted for in the observed rates of ADHD care. Also, patient demographics were not collected and thus we relied on pediatrician-level self-report regarding patient demographics. It also is possible that chart reviews underestimated the amount of care provided.
All of the participating practices volunteered to participate in a quality improvement intervention focusing on improving ADHD care and thus may differ from typical practices. Our sample was limited both geographically (ie, central-northern Ohio) and according to specific practice characteristics (ie, practices with only 1 physician and/or access to an on-site mental health professional). Therefore, it is unclear whether our results characterize practices outside the study region or to practice types not included in this study (eg, solo practices).
Conclusions
The results of this study suggest that current pediatrician-delivered ADHD care leaves much room for improvement. Only half of children being evaluated for ADHD by community-based pediatricians received an evidence-based diagnosis using DSM criteria and standardized rating scales. The proportion of children receiving psychosocial treatment was miniscule, even though combined treatment is the most effective treatment strategy for children with ADHD.24 Moreover, almost no ADHD care follows AAP ADHD consensus guideline recommendations for treatment (eg, collection of ratings to monitor outcomes and side effects).
Although guidelines are an important first step, additional efforts, likely initiated or incentivized outside the practice, are required to improve the quality of care delivered in pediatric settings. Such efforts may take the form of quality improvement,8,25–27 clinical decision support tools,28,29 using pay-for-performance incentives,30 and/or partnering with mental health professionals.31,32 Fortunately, the US government has mandated the use of electronic health records (EHRs) and has set up incentives encouraging physicians to “meaningfully use” EHRs to improve care. Hence, one timely strategy for addressing some areas of ADHD care might be to use the EHR to prompt/remind the physician to complete ADHD care behaviors or to use Web portals29 to aid in collecting rating scales.
Footnotes
- Accepted September 11, 2014.
- Address correspondence to Jeffery N. Epstein, PhD, Cincinnati Children’s Hospital Medical Center, 3333 Burnet Ave, ML #10006, Cincinnati, OH 45229-3039. E-mail: jeff.epstein{at}cchmc.org
Dr Epstein contributed to the conceptualization and the design of the study, and drafted the initial manuscript; Drs Kelleher, Baum, Brinkman, Lichtenstein, and Langberg contributed to the conceptualization and the design of the study, and reviewed and revised the manuscript; Drs Peugh and Gardner carried out the initial analysis, and reviewed and revised the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: This research was supported by grant R01 MH083665 from the National Institute of Mental Health and grant UL1 TR000077 from the National Center for Advancing Translational Sciences of the National Institutes of Health. Drs Epstein and Brinkman are supported by grants K24MH064478 and K23MH083027 from the National Institute of Mental Health, respectively. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. Funded by the National Institutes of Health (NIH).
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
References
- Copyright © 2014 by the American Academy of Pediatrics
In this issue
Jump to section
Related Articles
Cited By...
- Predictors of Medication Continuity in Children With ADHD
- Preschool ADHD Diagnosis and Stimulant Use Before and After the 2011 AAP Practice Guideline
- Impact of a Web-Portal Intervention on Community ADHD Care and Outcomes
- Ten-Year Trends In Treatment Services For Children With Attention Deficit Hyperactivity Disorder Enrolled In Medicaid
- Variations in Mental Health Diagnosis and Prescribing Across Pediatric Primary Care Practices
- Improving Care for Children With ADHD: The Information is Just a Rating Scale Away
- Timing of the Diagnosis of Attention-Deficit/Hyperactivity Disorder and Autism Spectrum Disorder
- Collaborative Care for Children With ADHD Symptoms: A Randomized Comparative Effectiveness Trial
- Cost Saving and Quality of Care in a Pediatric Accountable Care Organization