OBJECTIVES: The objective of this study was to assess the effect of electronic health record (EHR) decision support on physician management and documentation of care for children with attention-deficit/hyperactivity disorder (ADHD).
METHODS: This study involved 79 general pediatricians in 12 pediatric primary care practices that use the same EHR who were caring for 412 children who were aged 5 to 18 years and had a previous diagnosis of ADHD. We conducted a cluster randomized trial of EHR-based decision support that included (1) clinician reminders to assess ADHD symptoms every 3 to 6 months and (2) an ADHD note template with structured fields for symptoms, treatment effectiveness, and adverse effects. The main outcome measures were (1) proportion of children with visits during the 6-month study period in which ADHD was assessed and (2) quality of documentation of ADHD assessment. Generalized estimating equations were used to control for the clustering by providers.
RESULTS: Children at intervention sites were more likely to have had a visit during the study period in which their ADHD was assessed. The ADHD template was used at 32% of visits at which patients were scheduled specifically for ADHD assessment, and its use was associated with improved documentation of symptoms, treatment effectiveness, and treatment adverse effects.
CONCLUSIONS: EHR-based decision support improved the likelihood that children with ADHD had visits for as well as care related to managing this condition. Better understanding of how to optimize provider use of the decision support and templates could promote additional improvements in care.
- attention-deficit/hyperactivity disorder
- quality of care
- electronic health records
- decision support system
- chronic disease management
WHAT'S KNOWN ON THIS SUBJECT:
Children with ADHD receive suboptimal care. EHR decision support provides a potential strategy to improve care for children with chronic conditions.
WHAT THIS STUDY ADDS:
EHR-based decision support improved the likelihood that children with ADHD had visits for and care related to their condition. Better understanding of how to optimize provider use of EHR decision support could promote greater improvement in care.
Although there is ample evidence of suboptimal quality of care for children,1,2 evidence for effective methods to improve care remains limited. Finding effective methods to improve care is particularly relevant for chronic conditions. Attention-deficit/hyperactivity disorder (ADHD) is the most common neurobehavioral disorder in children.3,4 Children with ADHD may experience academic underachievement,5 troublesome relationships,6,7 and low self-esteem. When inadequately treated, childhood ADHD can continue to have a negative impact through adolescence8 and adult life.9 Improving the quality of care for ADHD has the potential to improve quality of life.10
General pediatricians are an important source of ADHD care. The American Academy of Pediatrics (AAP) developed diagnosis11 and treatment12 guidelines for the management of ADHD in primary care. Subsequent studies have shown that although practitioners are generally aware of the guidelines, adherence to these recommendations in several areas, including establishing the diagnosis,13,14 treatment,15 and follow-up,15 remains low. In particular, inadequate treatment and follow-up often occurs.10
Several strategies for improving chronic illness care have recently emerged, including embedding decision support into electronic health records (EHRs).16,–,18 Decision support for chronic illness care includes automated reminders, templates, and registries.19 Electronic decision support has been shown to improve care in areas such as immunization20 and has some benefits over paper-based reminders, including facilitating documentation and embedded decision support.21 Similarly, the use of templates has been shown to improve documentation during clinical encounters in the outpatient setting.22 Despite these findings, few studies have examined the effectiveness of integrating electronic decision support into clinical care, specifically for use in caring for children with chronic illness.
We examined the use and effects of EHR reminders and templates in pediatric primary care on the quality of documented care for ADHD. Specifically, we hypothesized that (1) reminders would improve the likelihood that patients with ADHD would have a visit during the study period in which their ADHD was assessed and (2) the use of an ADHD documentation template would improve the quality of documented care for children with ADHD.
We conducted this study between December 2006 and July 2007 with 79 general pediatricians located at 12 pediatric primary care practices in Eastern Massachusetts. The practices included a mix of private offices and community health centers that serve both the insured and underinsured. At the time of the study, all practices were using the same EHR known as the Longitudinal Medical Record (LMR). The LMR was developed and is maintained by Partners HealthCare, an integrated health care network, and all study practices are part of the network. The LMR includes integrated functionality such as electronic note writing, medication prescribing, reminders, immunization and vital sign tracking, and access to laboratory results.
This was a cluster randomized trial in which we randomized by practice for providers to receive the intervention or serve as control. Directors of each of the practices that were approached consented for their practice to participate in the study. Block randomization was used to help ensure a balance in practice type and patient demographics. All physicians and patients at intervention practices were included in our analysis regardless of whether they used the decision support tools (intention to treat), although we excluded residents and their patients because of their irregular practice schedules. Although all study practices were part of the same integrated health care network, there was no uniform care process or tools used for managing ADHD.
We evaluated the results of the intervention on patients with ADHD, defined as (1) 5 to 18 years old; (2) ADHD on the EHR problem list or an ADHD medication (as defined by the Healthcare Effectiveness Data and Information Set [HEDIS]) on the EHR active medications list, and (3) a visit to any included physician during the study period. Although the AAP guidelines were developed for children aged 6 to 12 years, there are no comparable guidelines for older children. We included children who were aged 13 to 18 years because care at study practices was similar for all affected children through age 18 years. We excluded children with any psychiatric comorbidities noted in the EHR, as well as those who were being seen by a child psychiatrist (total n excluded = 357). The Partners Institutional Review Board approved the study.
Decision support (reminders and visit note template) was introduced to intervention clinics in September and October 2006. Physicians were notified of the new features and instructed on their use through presentations at practice meetings as well as a detailed e-mail (including screen displays). We started data collection 6 weeks after introducing the decision support to allow physicians time to become accustomed to the new features.
Each time that an intervention provider opened an eligible patient's EHR (regardless of whether during a visit), a reminder to assess ADHD symptoms and treatment effectiveness (Fig 1) appeared for all study patients who in the previous 6 months had not had either a visit documented by using a well-child template or the ADHD template designed for this study (later described). The reminder suggested that the provider schedule a visit with the patient at least every 3 to 6 months to review ADHD care. The reminder could be turned off by the provider if he or she chose any of the following from a menu of presented screen options: symptom review done today, appointment scheduled, patient refused, symptom review documented elsewhere, deferred, or other.
ADHD Note Template
We held a series of meetings with pediatricians to develop a template that included the key elements recommended by guidelines and would be considered efficient and usable. The resulting ADHD template (Fig 2) included structured fields for the key elements contained in the AAP guidelines11,12 (source of information, types of screening tools used, diagnostic criteria, assessment of patient symptoms, treatment effectiveness, and medication adverse effects). Physicians accessed the ADHD template by selecting it from a menu of available templates.
Our study had 2 primary outcomes: (1) the proportion of eligible patients who had their ADHD care assessed (“ADHD visit”) during the study period (patient-level analysis) and (2) the quality of documentation of ADHD care assessments (visit-level analysis). We reviewed all charts for documentation of ADHD care. We defined having ADHD care assessed as any mention of ADHD care in a visit note (excluding simply listing ADHD as a problem). Initial visits for the diagnosis of ADHD were not included as an eligible visit. We then reviewed all EHR notes of study patients.
For patients who had an ADHD visit during the study period, we assessed the quality of documentation by using a trained research assistant to rate each note on a 3-point scale for symptom assessment (Table 1) and a binary scale (yes/no) to assess documentation of both treatment effectiveness and treatment adverse effects. Specificity of symptoms is important both for diagnosis (recommendations 3 and 4) and for assessment of attainment of target outcomes (recommendation 2) as noted in the AAP guidelines and therefore was assessed in the visit notes. Assessment of treatment effectiveness and adverse effects is noted as important in the AAP treatment guidelines (recommendation 5) and therefore was also was assessed. For the symptom assessment scale, a higher rating indicated a more detailed description of ADHD symptoms. For treatment effectiveness and adverse effects, a “yes” rating indicated inclusion of these topics in the note. A second rater independently reviewed a 20% random sample of charts (κ = 0.76).
Finally, to assess the degree to which physicians used the decision support, we examined our data for an association between the number of times the reminder was viewed, whether an ADHD visit occurred, and whether the quality of documentation at ADHD visits varied at intervention sites according to whether the ADHD template was used. We also determined the proportion of ADHD visits in the intervention group that were documented by using the ADHD template.
We also assessed physician attitudes about how well the EHR supported them in managing care for ADHD by using a 65-item survey and focus groups. For this study, we compared responses to the question, “How helpful is the EHR in providing evidence based ADHD care?” (1 = not helpful, 7 = very helpful) among the study groups. We conducted focus groups at 4 intervention practices and included questions about the ADHD reminder and template.
We compared characteristics of patients (age, gender, race, insurance type), physicians (years since medical school graduation, months on EHR), and practice type (health center versus private practice) in the control and intervention groups by using χ2 and Student's t tests when appropriate (Table 2). We used χ2 to compare the proportion of patients who had an ADHD visit among the control and intervention patients, as well as the quality of documentation among the control and intervention practices. Generalized estimating equations were used to control for the clustering by providers, with our final adjusted model including only variables that were significantly associated (P ≤ .05) with the outcome on bivariate analyses. Our analysis of the survey results (response rate: 62%) included a comparison of the mean response (Likert-type scale) between study groups by using a t test. We extracted and reported dominant themes from our focus group data.
All analyses were conducted by using Statistical Analysis Software (SAS Institute, Cary, NC). We determined that to detect a 20% difference in ADHD visit rates between the control (50%) and intervention (70%) groups (α = .05, power 80%), we would need to have at least 73 patients in each group.
Practice and physician characteristics are displayed in Table 2. The control and intervention groups each included 3 community health centers and 3 private practices. We reviewed the EHRs of 412 randomly selected study-eligible patients (206 patients per group). Although characteristics of the control and intervention patients and physicians differed in several respects, none of these differences was statistically significant.
During the study period, a higher proportion of patients in the intervention group had an ADHD visit compared with those in the control group (71% vs 54%; odds ratio: 2.2 [95% confidence interval: 1.2–4.0]; P = .04; Table 3). The type of visit during which ADHD care was provided occurred in both well-child (28% vs 22%) and non–well-child (44% vs 34%) visits, but the differences were not statistically significant. Similarly, ADHD symptoms and treatment were documented as discussed more often during well-child visits for patients in the intervention group (78%) than in the control group (63%), but this difference was of borderline statistical significance (P = .07).
In the intervention group, the ADHD visit template was used by 14 physicians (33% of eligible) with a median use of 2 per physician (range: 1–6). There were no instances in which the template was used during anything but a visit specifically to discuss ADHD symptoms and treatment (ie, the ADHD template was not used to supplement documentation provided in a non-ADHD visit). Intervention physicians with ADHD decision support used the ADHD documentation template for 32% (29 of 90) of the non–well-child visits. No well-child visits were documented with the ADHD documentation template in addition to a well-child template. Within the intervention group, notes in which the template was used were more likely to document any assessment of symptoms (100% vs 61.3%), treatment effectiveness (96.6% vs 54.8%), and treatment adverse effects (96.6% vs 40.3%; P < .001 for each). Figure 3 provides an example of differences in documentation between the template and nontemplate documentation. Both notes received the highest scale ratings for symptom assessment and treatment.
Results from the multivariable analysis indicated that patients with physicians in the intervention group were more likely to have an ADHD visit (odds ratio: 1.9 [95% confidence interval: 1.1–3.4]) compared with those in the control group. Months on the EHR was the only patient or physician variable that was significantly associated with the outcome and included in the multivariate model. Use of the ADHD template for documenting ADHD-related visits was not related to any patient or provider characteristic. The number of times a reminder appeared for a patient during the study period was not associated with an increased likelihood of having a visit at which ADHD symptoms and treatment were discussed (P = .68).
Physicians who had access to the ADHD reminder and template were more satisfied with the EHR helping them manage ADHD compared with the physicians in the control group (4.3 vs 3.3; P = .01). Physician focus groups revealed barriers for optimal use of the decision support tool, including (1) forgetting the templates were available, (2) preferring to use templates that they created themselves, and (3) finding the templates difficult to use efficiently. They suggested that their template use may have been higher if (1) their availability within the long list of available templates was better highlighted, (2) they were introduced before having developed their own templates, and (3) the templates were simplified.
In this study, a reminder for an ADHD visit was associated with a nearly 20% increase in the proportion of patients who had a visit during the study period in which ADHD management was discussed. Furthermore, use of an EHR-based ADHD documentation template during ambulatory visits improved the quality of documentation of care provided at those visits. Although physicians agreed that these decision support tools supported the delivery of ADHD care, the decision support was followed and used less frequently than possible.
Embedding of electronic decision support in EHRs has been shown to be an effective means to improve the quality of care in numerous areas, and our study extends this finding to the care of patients with ADHD. The ADHD reminder increased the proportion of patients who had visits at which ADHD-related counseling occurred, which are related to a HEDIS quality measure.23 This finding has several important implications. First, reminders can help improve adherence to guidelines in regard to recommended interval of follow-up. For ADHD and other chronic conditions such as asthma and obesity, guidelines suggest regular review of symptoms and treatment plans to optimize care. If our results can be generalized to other chronic conditions of childhood, then reminders for periodic follow-up and assessment could significantly improve care for children with chronic conditions. If the recommended interval between follow-up changes as a result of new evidence or guidelines, then this can be easily changed in the EHR. Second, because this reminder occurs in real time (ie, when the patient's EHR is viewed), this supports the provider without requiring additional time to determine whether a patient is due for ADHD management review. We did note that the number of times a reminder appeared for a patient during the study period was not associated with an increased likelihood that a visit at which ADHD was discussed occurred. This perhaps reflects that individual providers may follow the recommended action of a reminder the first time it is viewed or not all, indicating that repeated exposure to a reminder may not have a beneficial effect.
Although physicians report using decision support when it is available and find that it improves the quality of care that they deliver,24 our study indicates that there were numerous visits during which the physicians failed to respond to the reminders or use the templates. Physician adoption of decision support will require (1) design that maximizes the utility of the decision support and (2) willingness of physicians to alter their work flow.25 In our focus groups, clinicians suggested that templates that are too comprehensive may deter their use, particularly if templates that had been used previously were brief in length. Templates need to be designed in a way that strikes a balance between providing important information for decision support while not having that information lost or underused as a result of template density and length. Furthermore, the observation that the ADHD template was not used during visits when well-child templates were used suggests that use of the information in ADHD templates during these visits may be better promoted by accessibility of the template from within the well-child template itself.
Study limitations include the small number of practices and providers. Similarly, our patient population did not included children with psychiatric comorbidities, which occur in many children with ADHD. Nevertheless, the intervention was directed at primary care physicians who are more likely to treat children in our sample. Our study was underpowered to detect differences between study groups in relation to visit type. We did not verify that what physicians recorded on the templates actually occurred. We also assumed that our results were not affected by issues related to changes in insurance for the patient or at the practices or by financial incentives during the study period. We did not measure for each patient time from their last ADHD visit but hope that given the randomization, this was equal between study groups. Finally, we did not measure the patient outcomes, so our results pertain only to care processes, namely outpatient visits and the care documented.
Structured note templates, however, have the potential to improve care. They serve as a reminder in real time. Duggan et al22 showed that well-child visit templates increased the delivery of preventive services at well-child visits as measured by both recorded and observed performance. Furthermore, templates standardize and improve documentation, which may improve interprovider communication. Although we did not measure child outcomes in our study, the increased adherence to guidelines for documentation could result in improved ADHD management. Finally, electronic templates permit rapid data collection across an entire population so that detailed information regarding such aspects of care as symptoms, treatment effectiveness, and adverse effects can be monitored at the population level for improving care.
This study suggests that EHR tools can both increase the rate at which patients with ADHD have management of their condition discussed at ambulatory visits and improve the quality of documentation of care at those visits. Their use has potential for improving care for children with ADHD and other chronic conditions. Future work should be conducted to determine how best to maximize the frequency of their use and effectiveness, including in co-managing more complex cases with specialists, as well as in combination with more patient-centered interventions.
This study was funded by Agency for Healthcare Research and Quality grant R01 HS15002.
We thank Mark Mandell, Steve Morgan, and Peter Greenspan for contributions in designing and implementing these decision support tools for our study, as well as all of the practices that participated in our study.
- Accepted May 6, 2010.
- Address correspondence to John Patrick T. Co, MD, MPH, MGH Center for Child and Adolescent Health Policy, 50 Staniford St, Suite 901, Boston, MA 02114. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
- ADHD =
- attention-deficit/hyperactivity disorder •
- AAP =
- American Academy of Pediatrics •
- EHR =
- electronic health record •
- LMR =
- Longitudinal Medical Record •
- HEDIS =
- Healthcare Effectiveness Data and Information Set
- Ferris TG,
- Dougherty D,
- Blumenthal D,
- Perrin JM
- Bloom B,
- Cohen RA,
- Freeman G
- 11.↵American Academy of Pediatrics. Subcommittee on Attention-Deficit/Hyperactivity Disorder and Committee on Quality Improvement. Clinical practice guideline: diagnosis and evaluation of the child with attention-deficit/hyperactivity disorder. American Academy of Pediatrics. Pediatrics. 2000;105(5):1158–1170
- 12.↵Clinical practice guideline: treatment of the school-aged child with attention-deficit/hyperactivity disorder. American Academy of Pediatrics. Pediatrics. 2001;108(4):1033–1044
- Leslie LK,
- Weckerly J,
- Plemmons D,
- Landsverk J,
- Eastman S
- Rushton JL,
- Fant KE,
- Clark SJ
- Menachemi N,
- Brooks RG,
- Schwalenstocker E,
- Simpson L
- Kemper AR,
- Uren RL,
- Clark SJ
- Fiks AG,
- Grundmeier RW,
- Biggs LM,
- Localio AR,
- Alessandrini EA
- Cannon DS,
- Allen SN
- Duggan AK,
- Starfield B,
- DeAngelis C
- 23.↵HEDIS 2010 summary table of measures, product lines and changes. Available at: www.ncqa.org/Portals/0/HEDISQM/HEDIS2010/2010_Measures.pdf. Accessed April 2, 2010
The Old Are Getting Older: Over the past quarter of a century, the number of people over age 85 living in the United Kingdom has doubled to 1.3 million in 2008, and makes up 2.2% of the British population, according to an article in the British Medical Journal (Wise J, June 8, 2010). The article notes that the population of people over 65 increased 18% in the same time period while those between 16 and 64 increased by only 11% and those under 16 by 5%. The article predicts that by 2031, there will be a 77% increase in those aged 75 and over and a 131% increase in those over age 85. Perhaps as inpatient pediatric admissions decrease with the advances we are making in outpatient treatments of acute and chronic diseases, we will need to loan some of those pediatric beds to the growing elderly population.
Noted by JFL, MD
- Copyright © 2010 by the American Academy of Pediatrics