Electronic Health Record–Based Decision Support to Improve Asthma Care: A Cluster-Randomized Trial
OBJECTIVE: Asthma continues to be 1 of the most common chronic diseases of childhood and affects ∼6 million US children. Although National Asthma Education Prevention Program guidelines exist and are widely accepted, previous studies have demonstrated poor clinician adherence across a variety of populations. We sought to determine if clinical decision support (CDS) embedded in an electronic health record (EHR) would improve clinician adherence to national asthma guidelines in the primary care setting.
METHODS: We conducted a prospective cluster-randomized trial in 12 primary care sites over a 1-year period. Practices were stratified for analysis according to whether the site was urban or suburban. Children aged 0 to 18 years with persistent asthma were identified by International Classification of Diseases, Ninth Revision codes for asthma. The 6 intervention-practice sites had CDS alerts imbedded in the EHR. Outcomes of interest were the proportion of children with at least 1 prescription for controller medication, an up-to-date asthma care plan, and the performance of office-based spirometry.
RESULTS: Increases in the number of prescriptions for controller medications, over time, was 6% greater (P = .006) and 3% greater for spirometry (P = .04) in the intervention urban practices. Filing an up-to-date asthma care plan improved 14% (P = .03) and spirometry improved 6% (P = .003) in the suburban practices with the intervention.
CONCLUSION: In our study, using a cluster-randomized trial design, CDS in the EHR, at the point of care, improved clinician compliance with National Asthma Education Prevention Program guidelines.
WHAT'S KNOWN ON THIS SUBJECT:
The science and understanding of how to design and implement effective CDS is evolving. Embedding CDS within the EHR, if done effectively, has the potential to improve quality of care.
WHAT THIS STUDY ADDS:
This study uses an EHR as a platform on which a CDS tool is built. This design allowed the CDS to be smoothly integrated into the clinicians' work flow, improving clinician compliance with asthma care guidelines.
Asthma is the most common chronic disease of childhood and affects more than 6 million children in the United States. Rising asthma prevalence, hospitalizations, and costs over recent decades have led to efforts to improve asthma care, such as the National Asthma Education and Prevention Program (NAEPP) guidelines published by the National Heart, Lung, and Blood Institute.1,2 The NAEPP guidelines emphasize the importance of preventive care on the basis of evidence-based principles, including (1) appropriate classification of asthma severity to initiate controller medications for persistent asthma symptoms,3 (2) monitoring asthma control through repeated symptom assessments,4,–,7 (3) education in conjunction with a written home asthma action/care plan (ACP) that guides self-management for asthma flares,8,–,10 and (4) the use of primary care office spirometry as a tool for both diagnosis and monitoring asthma control.11,–,14
We used a site-randomized trial design to determine if clinical decision support (CDS) embedded in an electronic health record (EHR) would improve clinician adherence to the NAEPP guidelines in the primary care setting. We hypothesized that this tool would help clinicians increase how often they prescribe controller medications for persistent asthma. The CDS was also designed to encourage clinicians' use of home ACPs and primary care office–based spirometry.
This study was conducted in 12 practices within the Children's Hospital of Philadelphia (CHOP) Pediatric Research Consortium. Before the study, approval was obtained from the CHOP institutional review board. All practices we approached agreed to participate in the study. We chose practices with experience using an ambulatory EHR (EpicCare [Verona, WI]). The Pediatric Research Consortium is a multistate, hospital-owned, primary care practice–based research network that includes >235 000 children and adolescents. Study practices included 4 urban teaching practices (UPs) in which fewer than 35% of the patients have private insurance and 8 suburban practices (SPs) not involved in resident reaching and in which more than 80% of the children are privately insured. One practice located in Philadelphia, Pennsylvania, was grouped with the SPs because of its lack of resident teaching.
Study Design and Patient Population
We conducted a prospective cluster-randomized trial of decision support in 12 primary care sites over a 1-year period beginning in April 2007. Children aged 2 to 18 years with asthma were identified by the presence of International Classification of Diseases, Ninth Revision (ICD-9) codes for asthma (493.00–493.92) in their chronic-problem lists or visit diagnoses. These ICD-9 codes were modified in 2001 by our institution to permit description of the patient's asthma severity (eg, mild-persistent, moderate-persistent, or severe-persistent asthma).
Preintervention Educational Program and Spirometry Training
In the 6 months before the intervention, all 12 practices participated in an educational program designed to improve asthma knowledge and communication between clinicians and patients by using a modified version of Physician Asthma Care Education (PACE), a validated program.12 Training was conducted by experienced pediatricians and occurred in 2 blocks, each lasting ∼2 hours.
Before the start of the intervention, a pediatric asthma-control tool (PACT) was introduced in the EHR to all 12 practice locations. The PACT was developed and validated by a multidisciplinary team at CHOP.13 An abbreviated subscale of the PACT designed for the office setting showed good internal consistency, good correlation with a gold standard of a subspecialist assessment of control, and excellent correlation with a previously validated asthma-related quality-of-life measure.15 All practices were prompted in the EHR to complete the PACT if one had not been filed in the EHR in the last 3 months for a patient with asthma. In the intervention practices, the PACT information was used to personalize the CDS for each individual patient during all visits to the practice.
Before the start of the intervention, nurses were trained to perform spirometry and provide education for ACPs and asthma devices. Physicians were trained on spirometry use and interpretation, including ongoing monthly case discussions in person or by telephone to review the spirometry tracings and regular quality assessments by a pediatric pulmonologist.
Intervention and Randomization
All practices had the same asthma management tools available in the EHR either passively (the control group) or actively via decision-support alerts and reminders (the intervention group).
The asthma management tools available to all practices and available in the EHR consisted of:
the PACT data-entry tool for capturing asthma symptom frequency;
standardized documentation templates to facilitate severity classification;
order sets to facilitate ordering controller medications and spirometry; and
an ACP that can be supplied to families.
The intervention-practice sites had CDS alerts and reminders activated to guide clinicians to these tools. The recommendations were personalized for each patient on the basis of information captured in the PACT and diagnosis and medication history. These alerts were defined by using the NAEPP guidelines, created by a panel of institutional experts and implemented in the EHR by using an existing decision-support framework.16 This framework presents decision-support tools prominently in the clinical workflow, but without disruptive pop-ups (Fig 1).
To balance practices with previous asthma education or involvement in resident teaching and patient characteristics, the practices were stratified according to site (UP or SP) in blocks of 2. Therefore, 4 clusters of practices were compared in the analysis: 2 control UPs, 2 intervention UPs, 4 control SPs, and 4 intervention SPs.
Outcomes were measured for all children younger than 18 years who received asthma care at 1 of the study locations. We calculated the proportion of children:
with persistent asthma with at least 1 prescription for a controller medication in each time period;
with persistent asthma with an up-to-date ACP filed in the previous year;
aged 6 to 18 years with persistent asthma with documentation of spirometry performed.
Persistent asthma was defined as mild, moderate, or severe on the basis of the NAEPP guidelines. Asthma classification was determined by the clinician's judgment. The clinicians in the intervention practices were alerted by the CDS tool to a suggested asthma-severity classification (see Fig 2) and prompted to enter it if no classification was on file.
For each site, the proportions of children with the outcomes of interest were compared with those in the time periods before introduction of the intervention CDS. The time periods used in the analysis were:
The preeducation period: from December 1, 2005, to May 30, 2006.
The education period: from October 13, 2006, to April 15, 2007. All practices participated in the same asthma care curriculum.
The first intervention period (intervention 1): from April 16, 2007, to October 15, 2007.
The second intervention period (intervention 2): from October 16, 2007, to April 15, 2008.
To determine differences, the intervention and control practices' performances were compared in the intervention 2 and education time periods. Both periods spanned winter months. Finally, unless otherwise noted, the analysis included all visits to the practices, including both health maintenance and sick visits.
Patient-level factors collected included age, gender, race and ethnicity, asthma severity, and insurance type (Table 1).
Statistical analyses followed the demands of a cluster-randomized repeated cross-sectional design in which the contrasts of interest were the relative improvement over time periods between the intervention and control sites.17,18 These contrasts were tested by means of generalized linear models that produce robust variances and P values that account for clustering of patients within sites or, conversely, the variation across the study sites in the measures of interest. Analyses of the proportions of patients as a fraction of all patients were implemented via models with logit links and binomial error structure. Contrasts of interests were tested by time-by-intervention interactions. These models were fitted with and without patient-level factors as potential confounders. We compared intervention and control practices across these patient characteristics by means of χ2 statistics and simple regressions stratified according to site and separately for the education and intervention 2 periods. These comparisons determined whether the cluster randomizations created comparable groups of children during the key periods for evaluation. In addition, we contrasted the practices over time by the same factors to assess whether patient demographics remained the same during the education and intervention 2 periods. All analyses were performed by using proc genmod in SAS 9.1 to account for the clustering within practices (SAS Institute, Inc, Cary, NC).
Patient and Clinician Characteristics
A total of 19 450 children with asthma were included in the analysis over the course of the 4 consecutive time periods of the study: preeducation, education, intervention 1, and intervention 2. The patient characteristics did not differ between the intervention and control practices in any of the time periods except for race and percentage of children with an asthma-severity classification documented (Table 1). The control SPs had significantly fewer black children and were less likely to have classified their patients' asthma than the other groups. These differences remained static in the population over time; however, we found no confounding by patient-level covariates for differences within practices over time. Other differences between the SPs and UPs were expected (ie, commercial insurance). The characteristics of the clinicians in each of the practices were comparable between the intervention cluster and the control cluster with respect to years in clinical practice and gender.
There were a total of 49 059 visits for asthma over the 4 time periods. There were no important differences in the average number of asthma visits per child between the control and intervention groups for any of the time periods studied (Table 2).
Use of the Appropriate PACT
Overall use of the PACT at the start of intervention 1 was 48% of visits; use at the start of intervention 2 was 66%. There was no significant difference between the groups in the rate at which the PACT was used. Both the intervention and control groups used the PACT as required 69% of the time, on average, during the intervention periods, which lasted from April 16, 2007, to April 15, 2008.
The average site-level effect of the CDS system over time is shown in Table 3. The P values are the comparison of intervention 2 versus the education period. There was a statistically significant increase in controller-medication prescriptions in the intervention UPs compared with control UPs (7% vs 1%, respectively; P = .006). Conversely, there was an increase in ACP use in the intervention SPs compared with control SPs (14% vs −11%; P = .03).
The proportion of patients with persistent asthma who had spirometry performed was low but increased over time in both the UPs and SPs. The intervention UP group increased the use of spirometry for its patients from 15% (87 of 586) to 24% (147 of 604) (P = .04). Among the SP groups in the intervention sites, spirometry increased from 8% to 14%, whereas in the control SP group it decreased from 8% to 1% over time (P = .003). It should be noted that for every population and metric in Table 3, the UPs had a higher proportion of compliance than the SPs.
Figure 2 shows changes at the site level for the outcomes of interest. All statistical tests used t tests that considered only the number of sites (12) and ignored the large number of subjects per site. The SPs and UPs were analyzed together. The intervention practices were always superior in their performance, although the difference in performance failed to reach statistical significance. However, the position (above or below 0) and size of the bars show that, in all cases, both the number of sites with improvement and the site-specific levels of improvement favored the intervention sites.
Practicing high-quality medical care requires incorporating guidelines, once they exist, into routine care. The adoption of practice guidelines is often a slow process taking 5 years or more from the time guidelines are agreed on to when they are incorporated into practice.19 Even when guidelines are broadly accepted they are often not followed.20,–,22 To improve adherence, researchers have assessed various interventions including clinician education, quality-improvement programs, and incentives.23,–,26 Information systems that provide support to users at the time they make decisions may enable health clinicians to accelerate adoption of guidelines and eventually close the gap between optimal and actual practice.19,28
There have been other studies that have evaluated CDS designed to improve the management of asthma.29,–,31 In 2 of the reports, researchers developed stand-alone decision-support software that required clinicians to activate the system to receive the decision support for their adult patients with asthma.29,30 There was no significant impact on process-of-care measures (including prescriptions for controller medication) or patient outcomes (eg, emergency department visits). In a third, randomized study, using a handheld computer-based decision-support program at the point of care did improve clinician adherence to guidelines but was associated with longer visits and higher fees. There was no improvement in patient outcomes.31
The science and understanding of how to design and implement effective CDS is still in its infancy.19,32,–,34 Using the EHR as a platform allows for a design that adheres to the concepts that seem most important for decision support to be effective, such as automatic prompting, speedily delivered when needed (ie, during the patient visit), smooth integration into the clinicians' workflow, and trust that the recommendations are accurate and specific for the individual patient.19,32,35,36 With attention to the concepts above, CDS has had success in improving practitioner performance in adherence to disease-management guidelines, reducing medication errors, and improving immunization rates in children.16,37,38
The EHR allows for CDS at the point of care. In this case, the CDS was designed to compare a patient's characteristics with a knowledge base, thereby guiding the clinician with patient-specific and situation-specified advice. In our study, the CDS showed mixed results, improving clinician compliance with NAEPP guidelines in the UPs with improved prescription rates for controller medications and use of spirometry, and in the SPs with improved rates for filling out an ACP for families and use of spirometry. It is important to note that the effects of the CDS were sustained over the entire year of the intervention (April 16, 2007, to April 15, 2008) without a significant decline in clinician compliance over that time.
The mixed performance of the CDS may be because of the “differences” between the UPs and SPs, as anticipated in the project design. The UPs were complying with the NAEPP guidelines at a higher rate than the SPs before the project started. The “high-performance” UPs improved their rates of prescriptions for inhaled corticosteroids, whereas the lower-performing SPs improved in both the control and intervention practices, resulting in no significant relative impact of the CDS. The opposite was true for the rates of providing an ACP to the families. The UPs were performing at such a high level for the ACP end point that the CDS alerts had no effect, whereas the CDS reminder to fill out the ACP significantly improved performance for the SPs.
Another factor, outside of the project, likely affected the performance of the UPs. Beginning on July 1, 2005, before we started this project, discussions had begun about instituting asthma care–related pay-for-performance clinician incentives in the UPs only. Note that the incentives were introduced to both the intervention and control UPs. The first evaluation period for asthma management–specific metrics (such as use of the ACP and prescriptions for inhaled corticosteroids) in the UPs was begun on October 1, 2006, which was just before the beginning of this project's education period, which was from October 13, 2006, to April 15, 2007. Given the randomized design, we are confident that despite this contemporaneous development, the CDS was responsible for significantly improving the UPs performances in both prescriptions for controller medications and use of spirometry. Pay for performance was not introduced in the SPs during the study.
This study had limitations. More practices in the randomization would have improved our ability to evaluate the impact of the CDS in terms of assessing the variability of the effect of the intervention over time across practices and in being able to generalize from the sampled sites to primary care practices in general. We found differences in patient characteristics according to intervention group, largely because of demographic and related clinical differences across practices and the inherent difficulty in balancing by patient-level factors when randomization is by cluster. Also, with only 12 practices, our power to detect differences by using summary measures at the practice level for comparisons would be limited. This weakness was somewhat offset by the number of patients evaluated across the practices. In addition, other changes were influencing performance contemporaneously with the intervention (eg, pay-for-performance incentives discussed above). Any randomization-concomitant exposures, such as performance incentives, most likely had the effect of improving the control practices' results and perhaps blunting the additional effect of the CDS. Despite this, some significant differences were found. Finally, in this project we studied clinician response to a CDS tool. The next step will be to determine if patient outcomes are improved as a result.
CONCLUSIONS AND SPECULATION
In our study, using a cluster-randomized trial design, CDS in the EHR was effective at improving clinician compliance with NAEPP guidelines. The effectiveness of the CDS, however, seemed to depend on both the specific outcome being measured and the level of practice compliance with the guidelines before the introduction of the CDS. These results suggest that, if thoughtfully introduced, CDS embedded in the EHR might shorten the interval from guideline acceptance to actual use in practice, for low-compliance practices, or might help to optimize performance in highly compliant practices.
We thank the Agency for Healthcare Research and Quality for its support (R21HS014873-01A1).
We thank the network of primary care physicians and their patients and families for their contributions to clinical research through the Pediatric Research Consortium at CHOP. In addition, we thank Valerie Kanak and James Massey for their work on this project.
- Accepted November 16, 2009.
- Address correspondence to Louis M. Bell, MD, Division of General Pediatrics, 12th Floor Northwest Tower, Children's Hospital of Philadelphia, 34th and Civic Center Boulevard, Philadelphia, PA 19104. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
- NAEPP =
- National Asthma Education and Prevention Program •
- ACP =
- asthma action/care plan •
- CDS =
- clinical decision support •
- EHR =
- electronic health record •
- CHOP =
- Children's Hospital of Philadelphia •
- UP =
- urban practice •
- SP =
- suburban practice •
- PACT =
- pediatric asthma-control tool
- 2.↵National Institutes of Health, National Heart, Lung and Blood Institute, National Asthma Education and Prevention Program. Expert Panel report 3: guidelines for the diagnosis and management of asthma. NIH publication 08–5846. Available at: www.nhlbi.nih.gov/guidelines/asthma/asthgdln.pdf. Accessed January 12, 2010
- Yawn BP,
- Brenneman SK,
- Allen-Ramey FC,
- et al
- Adams RJ,
- Fuhlbrigge A,
- Finkelstein JA,
- et al
- Lieu TA,
- Quesenberry CP Jr.,
- Capra AM,
- Sorel ME,
- Martin KE,
- Mendoza GR
- Gibson PG,
- Powell H,
- Coughlan J,
- et al
- Guevara JP,
- Wolf FM,
- Crum CM,
- Clark NJ
- Zanconato S,
- Meneghelli G,
- Braga R,
- Zacchello F,
- Baraldi E
- Cabana MD,
- Slish KK,
- Evans D,
- et al
- Fiks AG,
- Grundmeier RW,
- Biggs LM,
- Localio AR,
- Alessandrini EA
- Campbell MK,
- Elbourne DR,
- Altman DG
- Donner A,
- Klar N
- Bates DW,
- Kuperman GJ,
- Wang S,
- et al
- Warman KL,
- Johnson Silver E,
- McCourt MP,
- Stein REK
- Scarfone RJ,
- Zorc JJ,
- Capraro GA
- Eccles M,
- McColl E,
- Steen I,
- et al
- Shiffman RN,
- Freudigman M,
- Brandt CA,
- Liaw Y,
- Navedo DD
- Bates DW,
- Ebell M,
- Gotlieg E,
- et al
- Aronsky D,
- Chan KJ,
- Aug PJ
- Kemper AR,
- Uren RL,
- Clark SJ
- Walsh KE,
- Landrigan CP,
- Adams WG,
- et al
- Copyright © 2010 by the American Academy of Pediatrics