Abstract
OBJECTIVE: To assess the impact of exchange of immunization information between an immunization information system (IIS) and an electronic health record on up-to-date rates, overimmunization, and immunization record completeness for low-income, urban children and adolescents.
METHODS: The New York City Department of Health maintains a population-based IIS, the Citywide Immunization Registry (CIR). Five community clinics in New York City implemented direct linkage of immunization data from the CIR to their local electronic health record. We compared immunization status and overimmunization in children and adolescents 19 to 35 month, 7 to 10 year, and 13 to 17 year-olds with provider visits in the 6-month period before data exchange implementation (2009; n = 6452) versus 6-months post-implementation (2010; n = 6124). We also assessed immunization record completeness with and without addition of CIR data for 8548 children and adolescents with visits in 2012–2013.
RESULTS: Up-to-date status increased from before to after implementation from 75.0% to 81.6% (absolute difference, 6.6%; 95% confidence interval [CI], 5.2% to 8.1%) and was significant for all age groups. The percentage overimmunized decreased from 8.8% to 4.7% (absolute difference, −4.1%; 95% CI, −7.8% to −0.3%) and was significant for adolescents (16.4% vs 1.2%; absolute difference, −15.2%; 95% CI, −26.7 to −3.6). Up-to-date status for those seen in 2012 to 2013 was higher when IIS data were added (74.6% vs 59.5%).
CONCLUSIONS: This study demonstrates that data exchange can improve child and adolescent immunization status. Development of the technology to support such exchange and continued focus on local, state, and federal policies to support such exchanges are needed.
- CDC —
- Centers for Disease Control and Prevention
- CI —
- confidence interval
- CIR —
- Citywide Immunization Registry
- DTaP —
- diphtheria, tetanus, and pertussis
- EHR —
- electronic health record
- Hib —
- Haemophilus influenzae type b
- HPV —
- human papillomavirus
- IIS —
- Immunization Information System
- IPV —
- inactivated polio vaccine
- MU —
- Meaningful Use
- MMR —
- measles, mumps, and rubella
- MU —
- Meaningful Use
- NYC —
- New York City
- Tdap —
- tetanus, diphtheria, and pertussis
What’s Known on this Subject:
Immunization record fragmentation places children at risk for underimmunization and overimmunization. Nearly all 50 states, 5 cities, and the District of Columbia operate an immunization information system, and >75% of US office-based physicians have adopted an electronic health record.
What This Study Adds:
We newly demonstrate that data exchange between immunization information systems and electronic health records can improve child and adolescent immunization status, allowing scarce resources to be targeted to those who are truly underimmunized.
Recent outbreaks of vaccine-preventable diseases highlight the important role of pediatricians in ensuring that their patients are fully immunized and receive all recommended vaccines.1–4 However, only 71.6% of young children in the United States have completed their primary immunization series.5 Furthermore, evidence suggests that 10% to 20% of young children receive ≥1 unnecessary, extra immunization.6–8 Fragmentation of immunization records leads to lack of timely, accurate, and complete immunization data at the point of care and can exacerbate both these problems.6,9–11 At the practice or organizational level, incomplete immunization records may also adversely affect immunization-related quality measures such as completion of the primary immunization series.9
An immunization information system (IIS), also known as an immunization registry, is a system that collects and centralizes immunization data for children and adolescents from immunization providers at a regional or state level. Nearly all 50 states, 5 cities, and the District of Columbia operate an IIS, and 86.2% of all US children <6 years old have immunizations recorded in an IIS.12 Until recently, clinicians wanting to access immunization information for their patients in an IIS generally accessed the data through a Web site at a state or local department of public health. Logging into a separate system from their electronic health record (EHR) is inefficient and impractical for clinicians.13,14 Frontline care providers are most likely to benefit from an IIS when it provides them with up-to-date immunization information at the point of patient care within their workflow, that is, in their EHR through an automated bidirectional exchange of immunization data. Although electronic transfer of immunization data from an EHR to an IIS is increasingly common, having been included in stages 1 and 2 of the federal EHR Meaningful Use (MU) financial incentive program,15 the transfer of immunization information from an IIS to the EHR is not. A requirement for IIS-to-EHR exchange was recently considered for inclusion in stage 3 of the MU program, but its impact on immunization outcomes has not been assessed.
The objective of this study was to assess the impact of IIS–EHR bidirectional exchange of immunization information on up-to-date rates, overimmunization, and immunization record completeness.
Methods
We examined data derived from 5 practices providing pediatric primary care in the New York–Presbyterian Hospital Ambulatory Care Network. The Columbia University Medical Center Institutional Review Board approved the study. The practices treat a primarily publicly insured and Latino population residing in northern Manhattan and the Bronx. The Vaccines for Children Program, a federal program that provides free vaccine to children who are unable to pay,16 provides the majority of vaccines given at the practices. The study sites use an EHR that is integrated with the hospital’s local immunization registry, EzVac,17 which contains >3.2 million immunizations administered to 461 000 patients, of whom 175 000 are children and adolescents <19 years of age. EzVac extracts information directly from the EHR, thereby providing accurate information for immunizations administered at the study sites. These sites did not have any other immunization interventions implemented during the study period.
EzVac uploads newly documented immunizations daily to the New York Citywide Immunization Registry (CIR). Established in 1996, the CIR includes 77 million immunizations given to 5.6 million people. New York City (NYC) and New York State Public Health Law require all immunizations administered to children <19 years old to be reported to the CIR within 14 days of administration.18 Because it has met high data quality standards, the CIR is 1 of 6 IIS sites chosen by the Centers for Disease Control and Prevention (CDC) as a sentinel site to monitor national immunization trends.19 Approximately 94% of facilities in NYC that vaccinate children report regularly.20
In 2009, the CIR developed a secure real-time Web service to allow immunization data to be downloaded directly into local practice site EHRs via secure standard-based data transfer and patient identification matching protocols.21 Before this Web service, electronic communication with the CIR involved transfer of immunization information from an EHR to the CIR, without the ability to download data from the CIR back into an EHR. Clinicians could access the CIR’s immunization data only through a manual patient lookup in the CIR’s online registry Web application. Because of time constraints, it was not part of the clinicians’ or other staff’s workflow at the study sites to access the CIR Web application before or at each visit.
Beginning in December 2009, we collaborated with the CIR to test and deploy the Web service. A clinician could click a “synchronize” button in EzVac to import immunizations from the CIR, with a median response time of 2 seconds.22 Duplicate data, defined as an immunization administered within 10 days of the same immunization type already in EzVac, were not imported. The method and frequency of reporting immunizations to the CIR from EzVac did not change during the study.
Impact on Coverage and Overimmunization
We assessed the impact of the capability to download immunization data by comparing the proportions of children who were up to date for their age-appropriate immunizations and those who received extra, unnecessary immunizations before and after implementation of IIS-to-EHR immunization exchange. Immunization data were collected from the EzVac immunization registry, which included immunization data from the CIR. Analyses focused on 3 sentinel age groups: 19 to 35 months, 7 to 10 years, and 13 to 17 years, reflecting the target assessment ages for the primary immunization series, school-age boosters, and adolescent immunizations.5
For both the up-to-date coverage and overimmunization analyses, the preimplementation study population included patients in the sentinel age groups with ≥1 visit to 1 of the 5 study practices in the 6-month period before implementation of the CIR Web service in December 2009 (ie, June 1–November 30, 2009). The postimplementation study population included patients meeting the same age criteria with ≥1 visit during the 6-month period 1 year after Web service implementation (June 1–November 30, 2010). The postimplementation period was selected to allow a wash-in period after initiation of the Web service and also to reflect the same months of the year as the preimplementation period. Patients in both periods were identified through the institution’s billing and registration system and were included in the period or periods in which they met inclusion criteria.
Up-to-date was defined based on the CDC’s Advisory Committee on Immunization Practices recommended age-appropriate series, as is used by the National Immunization Survey.5 For the children in the 19- to 35-month-old group, the series included completion of 4 doses of diphtheria, tetanus, and pertussis (DTaP) vaccine, 3 doses of polio vaccine (IPV), 1 dose of measles, mumps, and rubella (MMR) vaccine, 3 doses of Haemophilus influenzae type b (Hib) vaccine, 3 doses of hepatitis B vaccine, 1 dose of varicella vaccine (chickenpox), and 4 doses of pneumococcal conjugate vaccine (4:3:1:3:3:1:4 series). For 7- to 10-year-olds, up-to-date coverage included completion of the 4:3:1:3:3:1 series plus 1 booster dose each of DTaP, IPV, MMR, and varicella. These series reflect the childhood boosters needed for school entry. For the 13- to 17-year-old age group, up-to-date coverage included receipt of 1 dose of tetanus, diphtheria, and pertussis (Tdap) vaccine, 1 dose of meningococcal vaccine, and 3 doses of human papillomavirus (HPV) vaccine. HPV vaccination was calculated only for girls because the routine recommendation for boys to receive HPV vaccine occurred in 2011, after this study’s analyses.23 Secondarily, we assessed the inclusion of ≥1 HPV vaccine dose. The second dose of meningococcal vaccine, also more recently recommended (2010), was not included.24 Because we were assessing changes in true up-to-date status, immunizations found in EzVac, the CIR, or both were included in the analyses of up-to-date status for both periods.
Demographic characteristics of children seen in the preimplementation versus postimplementation periods were compared via χ2 tests. Relative risks were used to compare the proportions of children seen in the preimplementation versus postimplementation periods who were up to date by the end of the 6-month period for that group (November 2009 and November 2010). We also conducted analyses stratified by age.
Overimmunization was defined as receipt of ≥1 extra unnecessary immunization dose in the 4:3:1:3:3:1:4 series for the 19- to 35-month-old and 7- to 10-year-old children, and excess Tdap, meningococcal vaccine, or HPV vaccine for adolescents of either gender during the study periods. Analyses regarding overimmunization were limited to children who had an immunization in their CIR record at the time of the visit that was not in EzVac but would have contributed to their being up to date on their immunizations, thereby placing them at risk for overimmunization.
The proportion of at-risk children with a visit in the preimplementation versus postimplementation periods who received an extra immunization during the 6-month review periods were compared overall and stratified by age. We also compared counts of excess immunizations in the preimplementation and postimplementation groups by using the Wilcoxon rank sum test.
Impact of IIS Documentation on Completion
We examined the proportion of children who were documented as up to date for immunizations based on immunization data from EzVac versus data from both EzVac and the CIR. The study population included all children with a visit to a study site between March 1, 2012 and February 28, 2013 who were 19 to 35 months, 7 to 10 years, and 13 to 17 years as of March 1, 2013. Differences in immunization coverage (4:3:1:3:3:1:4 for 19- to 35-month-olds, 4:3:1:3:3:1 for 7- to 10-year-olds, and 1:1:3 for 13- to 17-year-olds) with EzVac-only data versus EzVac plus the CIR data were compared via a binomial test against a null rate of 10%. Data for 2013 were used because this was the most proximal complete year at the time of the study. Comparisons were made for the overall group and stratified by age. All analyses were performed in SAS version 9.3 (SAS Institute, Inc, Cary, NC).
Results
A total of 6452 children were seen in the preimplementation and 6124 in the postimplementation period. Half the children were male, nearly two-thirds came from Spanish-speaking families, and most were publicly insured (Table 1, Supplemental Table 3). There were no statistically significant differences between groups in terms of age, gender, language or insurance.
Characteristics of Study Population: Preimplementation (June–November 2009) and Postimplementation (June–November 2010) Periods of the Immunization Data Exchange Between the New York CIR and the Study Sites; 2012–2013 Cohort Assessed for Record Completion
The proportion of children and adolescents seen in the postimplementation period who were up to date on immunizations increased significantly compared with the preimplementation period for all age groups combined (absolute difference, 6.6%; 95% confidence interval [CI], 5.2% to 8.1%; relative risk, 1.09; 95% CI, 1.07 to 1.11). There was an interaction by age, but results remained significant when stratified by age (Table 2). Similarly, there was an interaction by site, and results remained significant for all but 1 site when stratified by site (Supplemental Table 4).
Effect of Data Exchange on Immunization Status and Overimmunization
Additionally, for children who had immunizations in the CIR that contributed to up-to-date status (ie, those who were at high risk of overimmunization), the percentage who were overvaccinated decreased from 8.8% to 4.7% for all age groups combined (absolute difference, −4.1%; 95% CI, -7.8% to −0.3%; relative risk, 0.54; 95% CI, 0.32 to 0.91) (Table 2). There was an interaction with age but not site. In stratified analyses by age, the absolute percentage was always lower in 2010 but statistically significantly different only in the adolescent group (absolute difference, −15.2%; 95% CI, −26.7% to −3.6%; relative risk, 0.072; 95% CI, 0.009 to 0.55) (Table 2). HPV was the most common vaccine with extra doses. Findings based on Wilcoxon rank analyses assessing the number of overimmunizations were similar to those of the χ2 (19–35 months, P = .81; 7–10 years, P = .087; 13–17 years, P = .0007; combined, P = .017), as were Wilcoxon rank analysis findings for all children and adolescents in the age groups, not just those we considered to be at risk for overimmunization (19–35 months, P = .63; 7–10 years, P = .13; 13–17 years, P = .002).
There were 8548 children seen in 2012 to 2013 (Table 1). The documented completeness of vaccine records, as reflected by the percentage of children and adolescents who were up to date, increased when the CIR data were added to EzVac data overall and for all 3 age groups (19- to 35-month-olds, 70.4% vs 83.6%; 7- to 10-year-olds, 68.2% vs 85.7%; 13- to 17-year-olds, 42.0% vs 55.9% for 3 doses of HPV vaccine; 62.6% vs 77.9% for ≥1 dose; overall, 59.5% vs 74.6% for 3 doses of HPV vaccine for adolescents and overall, 66.8% vs 82.4% for ≥1 dose; Fig 1). In all comparisons, these changes in completeness were statistically significant (P < .0001).
Changes in percentage of children and adolescents in sentinel age groups who were up to date for the recommended vaccines when immunization data from an immunization information system (IIS) was added to local electronic immunization data, 2012–2013. Note: EzVac is the immunization registry used by the hospital that is linked to the EHR. Up-to-date status for 19- to 35-month-olds refers to a 4:3:1:3:3:1:4 immunization series (4 doses of the DTaP vaccine, 3 doses of IPV, 1 dose of the MMR vaccine, ≥3 doses of the Hib vaccine, 3 doses of the hepatitis B vaccine, 1 dose of the varicella vaccine [chickenpox], and 4 doses of the pneumococcal conjugate vaccine); for 7- to 10-year-olds it refers to a 4:3:1:3:3:1 series plus 1 booster dose each of DTaP, IPV, MMR, and varicella. Pneumococcal conjugate vaccine was not included because subjects would have been born before or soon after pneumococcal conjugate vaccine was licensed in 2000. For 13- to 17-year-olds, up-to-date status refers to a *1:1:3 series (1 dose of Tdap, 1 dose of meningococcal vaccine, and 3 doses of HPV). The second dose of the meningococcal vaccine, as a booster for older adolescents only, was also recommended in 2010 and therefore was not included. **Uses 1 dose of HPV vaccine.
Discussion
The US Community Preventive Services Task Force recently published recommendations for the use of IISs and explicitly noted a critical gap in literature documenting use and utility of IIS by vaccination providers in clinical settings.25 In this study, we found that after implementation of the ability for immunization data from the New York CIR to be downloaded to a local EHR at the point of care, significant improvements in pediatric immunization coverage, a reduction in overimmunization for adolescents, and increased completeness of immunization records were observed. These findings point to the clinical benefit with regard to immunization coverage and overimmunization as a result of effective, automated bidirectional integration of EHR and IIS data. Because of the widespread requirement in the United States for reporting childhood vaccinations to a central registry, the incorporation of this bidirectional capability into EHRs was recently considered for inclusion as 1 of the MU stage 3 criteria.15 The decision was ultimately made to not finalize the proposal related to this data exchange in the MU program for 2015 to 2017 because of the time that may be needed for public health agencies and providers to develop the necessary technology. However, our data support the clinical and public health value of this criterion. Furthermore, because ∼19.5 million children <6 years old have immunizations recorded in an IIS,12 and >78% of office-based physicians have adopted an EHR system nationwide,26 our results imply that the beneficial impact of IIS-to-EHR immunization information exchange at a national scale could be substantial. As of December 2015, the CIR had 6 commercial EHR systems using bidirectional data exchange.
One potential explanation as to why immunization data exchange increased immunization coverage is that before taking action to address missing immunizations, a clinician, who may be unsure whether the immunization is really missing, may opt to wait for a family to return with a parent-held paper immunization record rather than vaccinate.27 With full immunization data available electronically at point of care, clinicians may have felt more certain that they had accurate, complete immunization data and that the child was indeed missing the immunization. It has been shown that children of parents who lack paper records are more likely to be underimmunized.27 Another possible explanation is that children’s immunization coverage naturally increased over time; however, although in this study we observed an increase in immunization coverage of 10.0 percentage points for 19- to 35-month-olds when comparing the 2010 and 2009 study periods, the National Immunization Survey immunization coverage estimates in 2010 compared with 2009 for 19- to 35-month-olds in NYC decreased by 6.6 percentage points.5
Our study also found that IIS-to-EHR information exchange had a significant impact on overimmunization for adolescents. The CDC recommends not postponing immunization if records cannot be found.8 Evidence suggests that 10% to 20% of young children receive ≥1 unnecessary, duplicate immunization.6,7 Although parent-held immunization records are valuable for notifying a provider that a child received an immunization elsewhere, parents may not remember to bring them to every care visit,28,29 underscoring the importance of having IIS information readily available in the EHR. A reason why the decrease in overimmunization was significant only among the adolescent group could be that 13- to 17-year-olds are still being actively immunized (especially for HPV), whereas 19- to 35-month-olds and 7- to 10-year-olds would not be if they have already finished the series for their age group.
Finally, this study demonstrated improved immunization documentation completeness when local and IIS data are combined. Having the combined data available in a practice or institution’s own database may be particularly important because organizations need to demonstrate compliance with immunizations to outside organizations such as insurance companies and federal incentive programs.30,31 In this study, all age groups were below Healthy People 2020 goals when only local data were considered but had coverage rates over the goal for the 2 youngest age groups and nearer the goal for the adolescent age group when the IIS data were added. This finding also supports previous studies demonstrating the utility of consolidating records within an IIS for improving immunization coverage.32–34 Having more complete records can also help organizations allocate resources more efficiently, for example, by sending targeting reminders to children who are truly underimmunized.
This study had limitations. First, the records used to assess immunization status may have been incomplete for immunizations provided at the study sites. However, the EHR automatically transfers immunization administrations to EzVac from which the data were extracted. It is also possible that additional immunizations were administered by providers who did not report to the CIR. However, reporting to the CIR is mandated in NYC, and provider participation levels have been historically high (>90%), especially after the Vaccines for Children program and CIR were linked.35 Additionally, the study design was observational rather than randomized, and the differences in coverage and overimmunization could result from differences inherent in the preimplementation and postimplementation cohorts. There were no statistically significant differences in measured demographic factors between the study populations in the 2 6-month periods; however, other unknown confounders could have played a role. Furthermore, outcomes were assessed based on vaccination status; the details of what actually transpired at each visit are not known. However, system usage logs showed that between June and November 2010, the CIR synchronize function within EzVac was used >13 000 times. Also, this study did not include analyses related to data quality, such as timeliness or accuracy. Finally, this study took place in 1 busy urban medical system, and results may not generalize to other environments. However, the effects were observed even after site stratification, supporting the generalizability.
Conclusions
This study demonstrates that data exchange can improve child and adolescent immunization status, allowing scarce resources to be targeted to those who are truly underimmunized. Additional development of the technology to support bidirectional immunization exchange and continued focus on local, state, and federal policies to support such exchanges are needed.
Acknowledgments
We thank Oscar Pena and Balendu DasGupta for their assistance with the study, and New York–Presbyterian Hospital for its support of the EzVac IIS, and New York–Presbyterian Hospital Ambulatory Care Network and its staff and patients. We thank the staff of the New York CIR and the NYC Department of Health and Mental Hygiene, especially Jane R. Zucker, MD, MSc, Amy Metroka, MSW, MPH, and Vikki Papadouka, PhD, MPH.
Footnotes
- Accepted January 28, 2016.
- Address correspondence to Melissa S. Stockwell, MD, MPH, Division of Child and Adolescent Health, Columbia University 622 W 168th St, VC 417, New York, NY 10032. E-mail: mss2112{at}columbia.edu
Presented at the 2014 annual meeting of the Pediatric Academic Societies, Washington, DC.
This manuscript’s contents are solely the responsibility of the authors and do not necessarily represent the official views of the funding agency. The funding agency had no role in the design and conduct of the study; collection, management, analysis, or interpretation of the data; or preparation, review, or approval of the manuscript.
FINANCIAL DISCLOSURE: Dr Stockwell is a coinvestigator but receives no financial support on a grant from the Pfizer Medical Education Group for an investigator-initiated study. The other authors have indicated they have no financial interests relevant to this article to disclose.
FUNDING: Supported by grant R18HS022667 (Dr Stockwell) from the Agency for Healthcare Research and Quality.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
References
- Copyright © 2016 by the American Academy of Pediatrics
