Skip to main content
Skip to main content

AAP Gateway

Advanced Search »

User menu

  • Login
  • Current Policy
  • Topic/Program Collections
  • Submit Manuscript
  • Alerts
  • Subscribe
  • aap.org

Menu

  • AAP Grand Rounds
  • AAP News
  • Hospital Pediatrics
  • NeoReviews
  • Pediatrics
  • Pediatrics in Review
  • Current AAP Policy
  • Journal CME
  • AAP Career Center
  • Pediatric Collections
  • AAP Journals Catalog

Sections

    • Login
    • Current Policy
    • Topic/Program Collections
    • Submit Manuscript
    • Alerts
    • Subscribe
    • aap.org

    Sign up for Insight Alerts highlighting editor-chosen studies with the greatest impact on clinical care.

    New! Video Abstracts -- brief videos summarizing key findings of new articles

    Know what's next when you read AAP Journals, view the new 2018 Catalog.
    Trainees,
    enter the SOPT Essay Contest for a chance to be published in Pediatrics!

    Advertising Disclaimer »

    Tools and Links

    Pediatrics
    January 2015, VOLUME 135 / ISSUE 1
    From the American Academy of Pediatrics
    Article

    Registry-Linked Electronic Influenza Vaccine Provider Reminders: A Cluster-Crossover Trial

    Melissa S. Stockwell, Marina Catallozzi, Stewin Camargo, Rajasekhar Ramakrishnan, Stephen Holleran, Sally E. Findley, Rita Kukafka, Annika M. Hofstetter, Nadira Fernandez, David K. Vawdrey
    • Article
    • Figures & Data
    • Supplemental
    • Info & Metrics
    • Comments
    Loading
    Download PDF

    Abstract

    OBJECTIVE: To determine the impact of a vaccination reminder in an electronic health record supplemented with data from an immunization information system (IIS).

    METHODS: A noninterruptive influenza vaccination reminder, based on a real-time query of hospital and city IIS, was used at 4 urban, academically affiliated clinics serving a low-income population. Using a randomized cluster-crossover design, each study site had “on” and “off” period during the fall and winter of 2011–2012. Influenza vaccination during a clinic visit was assessed for 6-month to 17-year-old patients. To assess sustainability, the reminder was active at all sites during the 2012–2013 season.

    RESULTS: In the 2011–2012 season, 8481 unique non-up-to-date children had visits. Slightly more non–up-to-date children seen when the reminder was ‘on’ were vaccinated than when ‘off’ (76.2% vs 73.8%; P = .027). Effects were seen in the winter (67.9% vs 62.2%; P = .005), not fall (76.8% vs 76.5%). The reminder also increased documentation of the reason for vaccine non-administration (68.1% vs 41.5%; P < .0001). During the 2011–2012 season, the reminder displayed for 8630 unique visits, and clinicians interacted with it in 83.1% of cases where patients required vaccination. During the 2012–2013 season, it displayed for 22 248 unique visits; clinicians interacted with it in 84.8% of cases.

    CONCLUSIONS: An IIS-linked influenza vaccination reminder increased vaccination later in the winter when fewer vaccine doses are usually given. Although the reminder did not require clinicians to interact with it, they frequently did; utilization did not wane over time.

    • immunization
    • vaccine
    • influenza
    • alert
    • reminder
    • clinical decision support

    What’s Known on This Subject:

    Frequency of influenza vaccination is low, partially because of missed opportunities to vaccinate. Barriers to implementing successful influenza vaccination reminders in the electronic health record include alert fatigue and incomplete vaccination information due to scattered records.

    What This Study Adds:

    A noninterruptive, immunization information system–linked influenza vaccination reminder can increase vaccination late in the winter when fewer vaccine doses are usually administered. Tailoring the reminder to clinicians’ needs can increase its use.

    Influenza vaccination is important both for a child’s own health and for preventing infection in households and communities.1,2 Yet coverage is far below that of other childhood vaccinations.3–6 One way to increase influenza vaccination is to reduce “missed opportunities” to vaccinate.7–10

    Computerized clinical decision support tools, such as reminders in electronic health records (EHR), may improve adherence to clinical guidelines.11–16 Several studies have reported beneficial results from electronic vaccination reminders in both adults and children17–21; however, electronic reminders may also adversely affect clinician workflow (eg, contributing to “alert fatigue”).22–25

    Reminders are dependent on accurate, timely data.26 When a decision support system generates incorrect recommendations, clinician trust in the system diminishes. One barrier to implementing successful influenza vaccination reminders is a lack of up-to-date vaccination information for a patient.27 This is a notable concern for low-income and minority children, who are especially susceptible to vaccination record fragmentation due to care from multiple providers.28,29 One can address this challenge by merging local vaccination data with data from a regional immunization information system (IIS), a population-based system that centrally collects vaccination data at a regional or state level.30 Electronic vaccination reminders in the EHR, however, have historically acted on local vaccination data.20,21

    The objective was to determine the impact on vaccine administration of EHR influenza vaccination reminders that are linked to a city IIS. A secondary goal was to assess their effect on documentation of reasons for vaccine nonadministration. We hypothesized that an electronic reminder would increase influenza vaccination and documentation of reason for nonadministration. Secondarily, we sought to assess the sustainability of reminder implementation.

    Methods

    Setting and Subjects

    In the 2011–2012 influenza season, we assessed the impact of an influenza vaccination reminder in a cluster randomized crossover trial in 4 community-based pediatric clinics affiliated with New York–Presbyterian (NYP) Hospital/Columbia University Medical Center in New York City. For the entire 2012–2013 season, the reminder was implemented at all 4 sites to assess sustainability. These clinics are staffed by a single pediatric group practice, but providers are assigned to 1 clinic for their primary patient care. They serve a predominantly Latino and publicly insured population; ∼95% are eligible for free vaccines through the Vaccines for Children program. There were no known vaccine shortages during the study. The Columbia University Medical Center Institutional Review Board approved the study with a waiver of consent.

    Each site used a common commercial EHR (Allscripts Sunrise, Allscripts, Chicago, IL), including structured visit notes. The EHR is integrated with the institution’s local vaccination registry, which includes all patients receiving care at NYP hospitals and affiliated clinics.31 The registry extracts information directly from the provider order entry/nurse sign-off EHR module. It also securely retrieves vaccination information for NYP patients from New York City’s Immunization Information System (IIS), the Citywide Immunization Registry (CIR). Reporting to the CIR is mandated for children aged <19 years old,32 and ∼94% of facilities that vaccinate children report regularly.33

    Design of the Reminder

    The reminder was designed with clinician and parent focus group and β-tester input.27 When a medical or nursing provider opened the EHR note-authoring module, the reminder automatically retrieved vaccination information from the hospital registry, which synchronized data real-time with the CIR. The reminder window appeared in a non-interruptive fashion in the top-right corner of the EHR screen, floating above the note entry form. The reminder displayed both a color-coded message based on the influenza vaccination status and the most recent seasonal and 2009 H1N1 vaccination dates (Supplemental Fig 3). An orange background indicated the child was non–up-to-date and allowed vaccine ordering using age-specific options or documentation of reason for vaccine nonadministration. Any action taken was recorded within the note’s “Plan” section. The reminder window also included a more information button, opening a window with all influenza vaccinations, allergies, and medical problems, as well as talking points based on information focus group parents suggested might help with influenza vaccination decision-making. For a child seen solely by a nurse whose family did not want vaccination, the nurse could document why the vaccine was not being ordered. If the family wanted vaccination, the child was seen by an ordering provider per existing site protocols. A green background indicated the child was up-to-date. A yellow background indicated a child was up-to-date but needed another dose after a prescribed waiting period. A red background indicated an egg allergy regardless of vaccination status.

    The reminder required no mandatory action; a provider could interact any time during note-writing or ignore it. The reminder window opened once per visit for each provider for up-to-date patients, but opened every time a note was opened for non–up-to-date patients and an action was not yet performed. At the conclusion of note-writing, if a provider neither ordered the vaccine nor documented a reason for nonadministration for a non–up-to-date patient, an additional reminder prompt was displayed.

    Care providers, including attending and resident physicians, nurse practitioners, and nurses, were given a short in-person in-service as well as visual aids. The reminder was pilot-tested at all 4 sites during a 10-week period in the 2010–2011 season.

    Intervention

    During the 2011–2012 influenza season, we conducted a cluster randomized crossover trial. To account for seasonal variations in vaccination,6,34 the season was divided into 2 phases (Fig 1), fall and winter, each with a 28-day “on” and a 28-day “off” period. Each clinic was randomized to a cluster with an on and an off period in each phase. This design allows for differences in outcomes to be attributed to the intervention, rather than due to preexisting site differences.35 During the on period, the reminder appeared when an electronic note was authored or edited for well-child, follow-up, acute care, Supplemental Nutrition Program for Women, Infants, and Children, or vaccine-only visits. During the off period, the reminder was not displayed. The switch every month and switch pattern were selected to minimize the number of transitions while allowing for sites to have both an on and an off period in the fall and winter. In the 2012–2013 season, the reminder was on for all 4 clinic sites through the season.

    FIGURE 1
    • Download figure
    • Open in new tab
    • Download powerpoint
    FIGURE 1

    Study flow diagram: 2011–2012 cluster-crossover trial. During “on” periods, the electronic influenza vaccination reminder appeared for health care providers in the electronic health record.

    Measures

    The primary outcome was the percentage of non–up-to-date children vaccinated at a visit. A secondary outcome was documentation of reason for nonadministration. Process measures were also assessed including the number of reminder firings by vaccination status, proportion of reminders acted on for non–up-to-date patients, and actions taken.

    Data Sources

    Influenza vaccination status was obtained from the hospital registry, which includes data from the local EHR as well as the CIR. To assess documentation of vaccine nonadministration, a random sample of children who remained non–up-to-date after their first visit to an MD provider during the study period was stratified by on/off status, age, site, and seasonality (fall or winter). The chart reviewer, who was blinded to on/off status, used an abstraction form to identify (1) any mention in the assessment/plan of influenza vaccine or reason for vaccine nonadministration or (2) evidence of a vaccine order, even if the vaccine was ultimately not given.

    Statistical Analyses

    For analyses related to receipt of influenza vaccination in the 2011–2012 season, the season was divided into the following analytic periods based on the on/off periods: period 1 (October 12–November 8); period 2 (November 9–December 13 excluding Thanksgiving week, a shortened week with different visit structure); period 3 (January 4–January 31); period 4: (February 1–February 28). Final vaccination status after clinic visits for non–up-to-date children aged 6 months to 17 years were compared when the reminder was on versus off using the χ2 test, and the relative risk of remaining unvaccinated (or having a missed opportunity) was determined. Children who required 2 doses in a given season were included in the analysis, focusing on the dose needed as of the start of the analytic period. The 365 children (5.2% of the total non–up-to-date) who were non–up-to-date at both on and off periods were removed from analyses to avoid any potential carryover effects. Additionally, for the 975 children (14.8% of the remaining non–up-to-date) who had multiple non–up-to-date visits, analyses were limited to the last visit in the season when the child was non–up-to-date. The last visit was chosen to best reflect the patient’s final vaccination status for the season. A sensitivity analysis was also conducted assessing the impact of the reminder at the first visit a child was seen in the analytic period; this included all children.

    Multivariable logistic models accounting for seasonality, site, and age were conducted, and interactions were assessed. Analyses were then stratified by seasonality. Fall and winter analyses included the last visit when a dose was due in the fall and winter, respectively.

    Documentation of reason for nonadministration was compared between on and off periods. In addition, to better understand the sustainability of reminder implementation, process measures were described for the 2011–2012 (October 12–March 31) and 2012–2013 (October 1–March 31) seasons.

    All analyses were performed by using SAS v.9.3 (SAS Institute, Inc, Cary, NC).

    Results

    During the analytic period in 2011–2012, 8481 unique children had visits, and median age was 6.5 years (interquartile range 2.1–10.2). Nearly two-thirds came from Spanish-speaking families, and half were female (Table 1). At the time of their first visit during the analytic period, 1478 (17.4%) had already been vaccinated that season and did not require another dose; 66 (0.8%) were vaccinated but were in need of a second dose, 21 of whom returned when they were due for that dose. A total of 6958 children had a visit in the analytic period for which they were not up-to-date, including those who were not up-to-date at their first visit (n = 6937) and the 21 indicated earlier.

    View this table:
    • View inline
    • View popup
    TABLE 1

    Characteristics of Study Population in 2011–2012

    After excluding 365 children seen in both on and off periods while non–up-to-date, most (85.2%) had only 1 visit when they were non–up-to-date, 11.8% had 2. Most (78.3%) presented for a well-child visit, with the remainder presenting for acute care (14.9%), vaccine only (6.5%), and Supplemental Nutrition Program for Women, Infants, and Children visit (<1%). There were no significant differences in gender, language, or age among those who were non–up-to-date at a visit when the reminder was on versus off.

    Overall, more non–up-to-date children seen when the reminder was on were vaccinated than when off (Table 2). In a regression model adjusted for seasonality, age, and clinic site, the impact of the reminder on reducing the number of children who remained unvaccinated, that is, had a missed opportunity, was significant (adjusted relative rates 0.90, 95% confidence interval 0.83–0.98). There were no interactions by site (P = .64)

    View this table:
    • View inline
    • View popup
    TABLE 2

    Proportion of Children Receiving Needed Influenza Vaccination During Clinic Visits When the Reminder Was On Versus Off, 2011–2012 Season

    Due to an interaction between on/off status and seasonality (P = .06), stratified analyses were conducted. Reminder effects were seen in the winter (67.9% vs 62.2%; P = .005), not the fall (76.8% vs 76.5%; Table 2). Although most visits were preventative, those whose last non–up-to-date visit in the fall was an acute care visit when the reminder was on were more likely to be vaccinated than when it was off (47.7% vs 38.9%; P = .027). In the winter, after adjusting for clinic and age, a child seen when the reminder was on for any visit was less likely to remain unvaccinated (adjusted relative rates 0.83, 95% confidence interval 0.74–0.93). In a sensitivity analysis, the impact of the reminder on the first non–up-to-date visit of any kind was significant in the winter (65.5% vs 60.2%, P = .01) but not fall.

    There were 319 children who received their first dose during the analytic period and subsequently returned for a second dose; of those, more received that dose when the alert was on versus off (73.0% vs 65.4%; P = .14). Of the 365 children who were excluded because they were seen when they were non–up-to-date at both on and off periods, more were vaccinated if their last non–up-to-date visit occurred when the alert was on vs off (48.0% vs 39.2%; P = .09), regardless of whether their previous period was on or off.

    In the 2011–2012 season, 1830 children remained non–up-to-date after their first clinic visit with an MD during the analytic period. In a random stratified sample (n = 624), documentation of the reason for vaccine nonadministration was more likely to occur when the reminder was on versus off (68.1% vs 41.5%; P < .0001; Fig 2). A similar pattern was observed in the fall (73.2% vs 45.6%; P < .0001) and winter (60.0% vs 36.6%; P < .0001).

    FIGURE 2
    • Download figure
    • Open in new tab
    • Download powerpoint
    FIGURE 2

    Proportion of unvaccinated children whose medical records contained documentation related to influenza vaccination decision-making, 2011–2012 season. All P values <.0001.

    In the 2011–2012 season, the reminder displayed 13 859 times for 8630 unique visits; 61.0% involved non–up-to-date children, for whom care providers acted on the reminder 83.1% of the time (Table 3). In the 2012–2013 season, the reminder displayed 37 379 times for 22 248 unique visits, 61.7% for non–up-to-date children, for whom care providers acted on the reminder in 84.8% of cases.

    View this table:
    • View inline
    • View popup
    TABLE 3

    Vaccination Status and Care Provider Actions While the Influenza Vaccination Reminder Was Active

    Discussion

    In this cluster randomized crossover trial, an influenza vaccination reminder in the EHR linked with city IIS data resulted in modest but significant increases in influenza vaccination, primarily in the January–February period. The majority of influenza vaccine doses nationally are given in the October–December period, and providers may be more likely to remember to offer vaccine then. Despite low national coverage, there are few increases in vaccine coverage after December.36 Yet in recent years, influenza has been present in the community through the spring, making the continued vaccination of patients later into the winter valuable.37,38 Therefore, an intervention that can have an impact on influenza vaccination after December may be particularly important. This reminder also had an effect during acute care visits in the fall, when missed opportunities may occur.39

    In EHRs, “hard-stop” alerts preventing users from proceeding until addressed, are highly unpopular with care providers. Nevertheless, hard-stop alerts are often used to enforce compliance.24,40,41 Reported override rates of non–hard-stop alerts in EHRs are as high as >90%.25,42,43 By contrast, in this study, care providers acted on the non–hard-stop reminder in the vast majority of cases; this behavior was sustained over multiple years. The success of the reminder may be attributed to involving care providers from the project’s inception.27 The reminder also fit within clinicians’ workflow and provided them with information at the time and place needed.44

    A previous study of influenza vaccine EHR reminders not linked to an IIS for children with asthma demonstrated only a 0.3% improvement in captured vaccination opportunities.21 Although it is not known if this lack of linkage to an IIS played a role, best practices should ensure that data used to trigger a clinical reminder are accurate and reliable.44 One of the key features of our reminder was the integration with a city IIS, enabling the decision support logic to use the best available vaccination information.27 This may have led to greater trust in the alert, increasing its use. All 50 states, 5 cities, and the District of Columbia operate an IIS, and ∼86% of US children ≤6 years old have vaccinations in one.30 The Centers for Medicare and Medicaid Services EHR Incentive Program provides financial incentives for meaningful use of certified EHR technology to improve patient care.45 Stage 2 of this program includes the use of clinically relevant information to identify patients who should receive reminders for preventive or follow-up care. One of the proposed measures for Stage 3 is transfer of vaccination information from an IIS to an EHR. The results of this study lend support to this Stage 2 and proposed Stage 3 measure.

    Finally, the reminder had an important impact on documentation of reasons for vaccine nonadministration. At the care-provider level, it is useful to know why a vaccine was not given at a previous visit. For example, encouraging vaccination for a child whose family previously refused the vaccine may require a different approach than for one who deferred due to illness. Improved documentation of nonvaccination may also be important at an institutional level for purposes of compliance and quality improvement.46 In this study, the most common reason for nonadministration was patient illness, followed by family refusal. It is possible that provider education regarding the acceptability of vaccinating during mild illness is needed. In addition, parents’ perceptions of influenza and the vaccine may also need to be addressed.47,48

    There were limitations to this study. The study took place in 1 low-income, urban, academic setting using a commercial EHR; results may not generalize to other environments. Latino children have higher influenza vaccination rates nationally (66.0%) versus the overall population (58.9%).6 Outcomes were assessed based on EHR documentation; the details of what actually transpired at each visit were not available. Additionally, it is possible that there may have been incomplete vaccination documentation. However, vaccine administrations are automatically transferred from the EHR to the hospital registry, and incomplete records should affect sites equally, regardless of reminder on/off status. Although we did not a priori indicate what would constitute a clinically significant difference, we believe a relative reduction in undervaccination of 10% to 17% and improvement in documentation of 61% to 64% are significant benefits, especially for an intervention that is straightforward to implement. Another limitation is that a minority (5.2%) of children had a visit during both an on and off period and were excluded from the primary analysis. Likewise, 14.8% of children had multiple visits that did not include both on and off periods. We chose to assess the last visit to best reflect the patient’s final vaccination status; we also conducted a sensitivity analysis assessing the first visit, which included all children. Also, some children required 2 doses in a given season. Children were included in the analysis for the first dose they needed during the analytic period. Finally, randomization was not performed at the individual patient level due to potential carryover effects from 1 patient to another seen by the same provider. The study used a crossover trial giving each site on and off comparison periods. We cannot, however, control for the rare possibility that the reminder sites independently had better vaccination practices at the times the reminder was on versus off. The on/off design might also have felt disruptive to providers.

    Conclusions

    An influenza vaccination reminder in an EHR linked to a regional IIS was effective in increasing vaccination, particularly later in the winter when fewer vaccine doses are usually given. Although care providers were not forced to act on the reminder, they did so the vast majority of the time. Utilization did not wane over time. The reminder also had a significant effect on documentation of why a vaccine was not administered.

    Acknowledgments

    We thank Ken Kitayama, Lauren Sonnabend, Oscar Peña, Balendu DasGupta, and Karthik Natarajan for their help with the study. We also thank NewYork–Presbyterian Hospital for its support of the EzVac Immunization Information System and NewYork–Presbyterian Hospital Ambulatory Care Network and its staff and patients.

    Footnotes

      • Accepted October 20, 2014.
    • Address correspondence to Melissa S. Stockwell, MD, MPH, Division of Child and Adolescent Health-Columbia University, 622 W. 168th Street-VC 417, New York, NY 10032. E-mail: mss2112{at}columbia.edu
    • Dr Stockwell conceptualized and designed the study, analyzed the data, and drafted the initial manuscript; Drs Catallozzi, Findley, Kukafka, Hofstetter, and Vawdrey contributed to the conceptualization and design of the study and reviewed and revised the manuscript; Mr Camargo implemented the reminder application, provided data management and data collection tools, and critically reviewed the manuscript; Dr Ramakrishnan and Mr Holleran contributed to the design of the study, aided in data analysis, and critically reviewed the manuscript; Dr Fernandez aided in data collection and critically reviewed the manuscript; and all authors approved the final manuscript as submitted.

    • This manuscript’s contents are solely the responsibility of the authors and do not necessarily represent the official views of the Agency for Healthcare Research and Quality. The funding agency had no role in the design and conduct of the study; collection, management, analysis, or interpretation of the data; or preparation, review, or approval of the manuscript.

    • This trial has been registered at www.clinicaltrials.gov (identifier NCT01146899).

    • FINANCIAL DISCLOSURE: Dr Hofstetter receives support from the Pfizer Medical Education Group for a different investigator-initiated study and also received support through the Investigator-Initiated Studies Program of Merck, Sharp & Dohme Corp for a different study; Dr Stockwell is a coinvestigator but receives no financial support; and the other authors have indicated they have no financial relationships relevant to this article to disclose.

    • FUNDING: This study was supported by grant R18HS018158 (Stockwell) from the Agency for Healthcare Research and Quality.

    • POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

    References

    1. ↵
      1. Weycker D,
      2. Edelsberg J,
      3. Halloran ME,
      4. et al
      . Population-wide benefits of routine vaccination of children against influenza. Vaccine. 2005;23(10):1284–1293pmid:15652671
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Hurwitz ES,
      2. Haber M,
      3. Chang A,
      4. et al
      . Effectiveness of influenza vaccination of day care children in reducing influenza-related morbidity among household contacts. JAMA. 2000;284(13):1677–1682pmid:11015798
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Centers for Disease Control and Prevention (CDC)
      . Vaccination coverage among children in kindergarten—United States, 2012–13 school year. MMWR Morb Mortal Wkly Rep. 2013;62(30):607–612pmid:23903595
      OpenUrlPubMed
      1. Centers for Disease Control and Prevention (CDC)
      . National and state vaccination coverage among adolescents aged 13–17 years—United States, 2012. MMWR Morb Mortal Wkly Rep. 2013;62(34):685–693pmid:23985496
      OpenUrlPubMed
      1. Centers for Disease Control and Prevention (CDC)
      . National, state, and local area vaccination coverage among children aged 19–35 months—United States, 2012. MMWR Morb Mortal Wkly Rep. 2013;62(36):733–740pmid:24025754
      OpenUrlPubMed
    4. ↵
      Centers for Disease Control and Prevention. Flu Vaccination Coverage, United States, 2013–14 Influenza Season. Available at: http://www.cdc.gov/flu/fluvaxview/coverage-1314estimates.htm. Accessed on September 29, 2014
    5. ↵
      1. Verani JR,
      2. Irigoyen M,
      3. Chen S,
      4. Chimkin F
      . Influenza vaccine coverage and missed opportunities among inner-city children aged 6 to 23 months: 2000–2005. Pediatrics. 2007;119(3). Available at: www.pediatrics.org/cgi/content/full/119/3/e580pmid:17332178
      OpenUrlAbstract/FREE Full Text
      1. Hofstetter AM,
      2. Natarajan K,
      3. Martinez RA,
      4. Rabinowitz D,
      5. Vawdrey DK,
      6. Stockwell MS
      . Influenza vaccination coverage and timeliness among children requiring two doses, 2004-2009. Prev Med. 2013;56(3-4):165–170pmid:23219757
      OpenUrlCrossRefPubMed
      1. Daley MF,
      2. Beaty BL,
      3. Barrow J,
      4. et al
      . Missed opportunities for influenza vaccination in children with chronic medical conditions. Arch Pediatr Adolesc Med. 2005;159(10):986–991pmid:16203946
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Dombkowski KJ,
      2. Davis MM,
      3. Cohn LM,
      4. Clark SJ
      . Effect of missed opportunities on influenza vaccination rates among children with asthma. Arch Pediatr Adolesc Med. 2006;160(9):966–971pmid:16953021
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      HIMSS. Clinical Decision Support. Available at: htpp://www.himss.org/library/clinical-decision-support?navItemNumber=16563. Accessed on June 25, 2014
      1. Roshanov PS,
      2. Fernandes N,
      3. Wilczynski JM,
      4. et al
      . Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ. 2013;346:f657pmid:23412440
      OpenUrlAbstract/FREE Full Text
      1. Bryan C,
      2. Boren SA
      . The use and effectiveness of electronic clinical decision support tools in the ambulatory/primary care setting: a systematic review of the literature. Inform Prim Care. 2008;16(2):79–91pmid:18713524
      OpenUrlPubMed
      1. Shea S,
      2. DuMouchel W,
      3. Bahamonde L
      . A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. J Am Med Inform Assoc. 1996;3(6):399–409pmid:8930856
      OpenUrlCrossRefPubMed
      1. Garg AX,
      2. Adhikari NK,
      3. McDonald H,
      4. et al
      . Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223–1238pmid:15755945
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Balas EA,
      2. Weingarten S,
      3. Garb CT,
      4. Blumenthal D,
      5. Boren SA,
      6. Brown GD
      . Improving preventive care by prompting physicians. Arch Intern Med. 2000;160(3):301–308pmid:10668831
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      1. McDonald CJ,
      2. Hui SL,
      3. Smith DM,
      4. et al
      . Reminders to physicians from an introspective computer medical record. A two-year randomized trial. Ann Intern Med. 1984;100(1):130–138pmid:6691639
      OpenUrlCrossRefPubMedWeb of Science
      1. Loo TS,
      2. Davis RB,
      3. Lipsitz LA,
      4. et al
      . Electronic medical record reminders and panel management to improve primary care of elderly patients. Arch Intern Med. 2011;171(17):1552–1558pmid:21949163
      OpenUrlCrossRefPubMedWeb of Science
      1. Gerard MN,
      2. Trick WE,
      3. Das K,
      4. Charles-Damte M,
      5. Murphy GA,
      6. Benson IM
      . Use of clinical decision support to increase influenza vaccination: multi-year evolution of the system. J Am Med Inform Assoc. 2008;15(6):776–779pmid:18756001
      OpenUrlCrossRefPubMed
    10. ↵
      1. Fiks AG,
      2. Grundmeier RW,
      3. Biggs LM,
      4. Localio AR,
      5. Alessandrini EA
      . Impact of clinical alerts within an electronic health record on routine childhood immunization in an urban pediatric population. Pediatrics. 2007;120(4):707–714pmid:17908756
      OpenUrlAbstract/FREE Full Text
    11. ↵
      1. Fiks AG,
      2. Hunter KF,
      3. Localio AR,
      4. et al
      . Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009;124(1):159–169pmid:19564296
      OpenUrlAbstract/FREE Full Text
    12. ↵
      1. Phansalkar S,
      2. van der Sijs H,
      3. Tucker AD,
      4. et al
      . Drug-drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records. J Am Med Inform Assoc. 2013;20(3):489–493pmid:23011124
      OpenUrlAbstract/FREE Full Text
      1. Embi PJ,
      2. Leonard AC
      . Evaluating alert fatigue over time to EHR-based clinical trial alerts: findings from a randomized controlled study. J Am Med Inform Assoc. 2012;19(e1):e145–e148pmid:22534081
      OpenUrlCrossRefPubMed
    13. ↵
      1. Kesselheim AS,
      2. Cresswell K,
      3. Phansalkar S,
      4. Bates DW,
      5. Sheikh A
      . Clinical decision support systems could be modified to reduce “alert fatigue” while still minimizing the risk of litigation. Health Aff (Millwood). 2011;30(12):2310–2317pmid:22147858
      OpenUrlAbstract/FREE Full Text
    14. ↵
      1. Isaac T,
      2. Weissman JS,
      3. Davis RB,
      4. et al
      . Overrides of medication alerts in ambulatory care. Arch Intern Med. 2009;169(3):305–311pmid:19204222
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      1. Golchin K,
      2. Roudsari A
      . Study of the effects of Clinical Decision Support System’s incorrect advice and clinical case difficulty on users’ decision making accuracy. Stud Health Technol Inform. 2011;164:13–16pmid:21335681
      OpenUrlPubMed
    16. ↵
      1. Birmingham E,
      2. Catallozzi M,
      3. Findley SE,
      4. Vawdrey DK,
      5. Kukafka R,
      6. Stockwell MS
      . FluAlert: a qualitative evaluation of providers’ desired characteristics and concerns regarding computerized influenza vaccination alerts. Prev Med. 2011;52(3–4):274–277pmid:21276811
      OpenUrlPubMed
    17. ↵
      1. Yusuf H,
      2. Adams M,
      3. Rodewald L,
      4. et al
      . Fragmentation of immunization history among providers and parents of children in selected underserved areas. Am J Prev Med. 2002;23(2):106–112pmid:12121798
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Joseph CL,
      2. Giblin PT,
      3. Kallenbach LR,
      4. Jacobsen G,
      5. Davis RM
      . Visiting multiple sites for immunization and vaccine coverage levels of preschool children in 3 urban clinics: potential indicator of record scatter? Clin Pediatr (Phila). 2002;41(4):249–256pmid:12041722
      OpenUrlAbstract/FREE Full Text
    19. ↵
      1. Centers for Disease Control and Prevention
      . Progress in immunization information systems—United States, 2012. MMWR Morb Mortal Wkly Rep. 2013;62(49):1005–1008pmid:24336133
      OpenUrlPubMed
    20. ↵
      1. Vawdrey DK,
      2. Natarajan K,
      3. Kanter AS,
      4. Hripcsak G,
      5. Kuperman GJ,
      6. Stockwell MS
      . Informatics lessons from using a novel immunization information system. Stud Health Technol Inform. 2013;192:589–593pmid:23920624
      OpenUrlPubMed
    21. ↵
      Statewide Immunization Registry, Pub Health Law, Article 21, Title 6, §2168
    22. ↵
      1. New York City Department of Health and Mental Hygiene
      . Immunization Information System Annual Report 2013. Atlanta, GA: Program Operations Branch, Immunization Services Division. Centers for Disease Control and Prevention
    23. ↵
      1. Fishbein DB,
      2. Fontanesi J,
      3. Kopald D,
      4. et al
      . Why do not patients receive influenza vaccine in December and January? Vaccine. 2006;24(6):798–802pmid:16451814
      OpenUrlCrossRefPubMedWeb of Science
    24. ↵
      1. Rietbergen C,
      2. Moerbeek M
      . The design of cluster randomized crossover trials. J Educ Behav Stat. 2011;36:472–490
      OpenUrlCrossRef
    25. ↵
      Centers for Disease Control and Prevention. National Early Season Flu Vaccination Coverage, United States, 2012–13 Flu Season. Available at: http://www.cdc.gov/flu/fluvaxview/nifs-estimates-nov2012.htm. Accessed on July 17, 2014
    26. ↵
      Centers for Disease Control and Prevention. 2013–2014 Influenza Season. Available at: http://www.cdc.gov/flu/weekly/index.htm. Accessed on June 25, 2014
    27. ↵
      Centers for Disease Control and Prevention. 2012–2013 Influenza Season Surveillance Summary. Available at: www.cdc.gov/flu/weekly/pdf/12-13 Season Summary.pdf. Accessed on June 25, 2014
    28. ↵
      1. Allred NJ,
      2. Poehling KA,
      3. Szilagyi PG,
      4. et al
      . The impact of missed opportunities on seasonal influenza vaccination coverage for healthy young children. J Public Health Manag Pract. 2011;17(6):560–564pmid:21964369
      OpenUrlCrossRefPubMed
    29. ↵
      Vawdrey DK, Chang N, Compton A, Tiase V, Hripcsak G. Impact of electronic medication reconciliation at hospital admission on clinician workflow. AMIA Annu Symp Proc. 2010;2010:822–826
    30. ↵
      1. Schnall R,
      2. Liu N,
      3. Sperling J,
      4. Green R,
      5. Clark S,
      6. Vawdrey D
      . An electronic alert for HIV screening in the emergency department increases screening but not the diagnosis of HIV. Appl Clin Inform. 2014;5(1):299–312pmid:24734140
      OpenUrlCrossRefPubMed
    31. ↵
      1. van der Sijs H,
      2. Aarts J,
      3. Vulto A,
      4. Berg M
      . Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc. 2006;13(2):138–147pmid:16357358
      OpenUrlAbstract/FREE Full Text
    32. ↵
      1. Weingart SN,
      2. Toth M,
      3. Sands DZ,
      4. Aronson MD,
      5. Davis RB,
      6. Phillips RS
      . Physicians’ decisions to override computerized drug alerts in primary care. Arch Intern Med. 2003;163(21):2625–2631pmid:14638563
      OpenUrlCrossRefPubMedWeb of Science
    33. ↵
      1. Sittig DF,
      2. Teich JM,
      3. Osheroff JA,
      4. Singh H
      . Improving clinical quality indicators through electronic health records: it takes more than just a reminder. Pediatrics. 2009;124(1):375–377pmid:19564321
      OpenUrlFREE Full Text
    34. ↵
      Centers for Disease Control and Prevention. Meaningful Use and Immunization Information Systems. Available at: http://www.cdc.gov/vaccines/programs/iis/meaningful-use/index.html. Accessed September 28, 2014
    35. ↵
      The Joint Commission. Accountability Measure List. Available at http://www.jointcommission.org/assets/1/18/2013_Accountability_Measures.pdf. Accessed on October 14, 2014
    36. ↵
      1. Freed GL,
      2. Clark SJ,
      3. Butchart AT,
      4. Singer DC,
      5. Davis MM
      . Parental vaccine safety concerns in 2009. Pediatrics. 2010;125(4):654–659pmid:20194286
      OpenUrlAbstract/FREE Full Text
    37. ↵
      1. Nowalk MP,
      2. Zimmerman RK,
      3. Lin CJ,
      4. et al
      . Parental perspectives on influenza immunization of children aged 6 to 23 months. Am J Prev Med. 2005;29(3):210–214pmid:16168870
      OpenUrlCrossRefPubMedWeb of Science
    • Copyright © 2015 by the American Academy of Pediatrics
    View Abstract
    PreviousNext

     

    Advertising Disclaimer »

    View this article with LENS
    PreviousNext
    Email

    Thank you for your interest in spreading the word on Pediatrics.

    NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

    Enter multiple addresses on separate lines or separate them with commas.
    Registry-Linked Electronic Influenza Vaccine Provider Reminders: A Cluster-Crossover Trial
    (Your Name) has sent you a message from Pediatrics
    (Your Name) thought you would like to see the Pediatrics web site.

    Alerts
    Sign In to Email Alerts with your Email Address
    Citation Tools
    Registry-Linked Electronic Influenza Vaccine Provider Reminders: A Cluster-Crossover Trial
    Melissa S. Stockwell, Marina Catallozzi, Stewin Camargo, Rajasekhar Ramakrishnan, Stephen Holleran, Sally E. Findley, Rita Kukafka, Annika M. Hofstetter, Nadira Fernandez, David K. Vawdrey
    Pediatrics Jan 2015, 135 (1) e75-e82; DOI: 10.1542/peds.2014-2616

    Citation Manager Formats

    • BibTeX
    • Bookends
    • EasyBib
    • EndNote (tagged)
    • EndNote 8 (xml)
    • Medlars
    • Mendeley
    • Papers
    • RefWorks Tagged
    • Ref Manager
    • RIS
    • Zotero
    Share
    Registry-Linked Electronic Influenza Vaccine Provider Reminders: A Cluster-Crossover Trial
    Melissa S. Stockwell, Marina Catallozzi, Stewin Camargo, Rajasekhar Ramakrishnan, Stephen Holleran, Sally E. Findley, Rita Kukafka, Annika M. Hofstetter, Nadira Fernandez, David K. Vawdrey
    Pediatrics Jan 2015, 135 (1) e75-e82; DOI: 10.1542/peds.2014-2616
    del.icio.us logo Digg logo Reddit logo Technorati logo Twitter logo CiteULike logo Connotea logo Facebook logo Google logo Mendeley logo
    Print
    PDF
    Insight Alerts
    • Table of Contents
    • Current Policy
    • Early Release
    • Current Issue
    • Past Issues
    • Editorial Board
    • Editorial Policies
    • Overview
    • Open Access
    • Pediatric Collections
    • Video Abstracts
    • Author Guidelines
    • Reviewer Guidelines
    • Submit My Manuscript

    Subjects

    • Infectious Disease
      • Infectious Disease
      • Vaccine/Immunization
    • Health Information Technology
      • Health Information Technology
    Back to top

                

    Copyright (c) 2018 by American Academy of Pediatrics

    International Access »           

    Terms of Use
    Privacy Statement
    FAQ

    AAP Pediatrics