BACKGROUND: On March 22, 2002, Internet-based reports (IBRs) were added to the Vaccine Adverse Event Reporting System (VAERS) to allow rapid, expedited reporting of adverse events (AEs) in anticipation of wider use of counter-bioterrorism vaccines such as those against smallpox and anthrax.
OBJECTIVES: To evaluate the impact of IBRs on the timeliness and completeness of vaccine AE reporting.
METHODS: To evaluate timeliness and completeness, we compared the proportions of IBRs with non–Internet-based reports (NIBRs). Report interval was analyzed for timeliness and age at vaccination, birth date, and onset date for report completeness. To evaluate the impact of the smallpox vaccination program, we compared smallpox vaccine reports separately. Because influenza vaccine is the most widely used vaccine in adults each year, we compared influenza vaccine reports separately.
RESULTS: During the study period, VAERS received 54 364 NIBRs (85.8%) and 9008 IBRs (14.2%). Sixteen percent (1455) of IBRs followed smallpox vaccination. Overall, for all vaccines and for smallpox vaccine alone, IBRs had a greater proportion of completeness and a shorter report interval. The proportion of most frequently reported AEs did not differ between IBRs and NIBRs. A higher proportion of adults (18–64 years old) who received influenza vaccine chose to complete an IBR (62% vs 48%).
CONCLUSIONS: The improved timeliness and completeness of IBRs allow VAERS to more rapidly detect new or rare vaccine AEs. This important advantage is critical in times of increased public concern about vaccine safety. Clinical vaccine providers should be aware of VAERS and use IBRs whenever feasible to report vaccine AEs.
The National Childhood Vaccine Injury Act was passed in 1986; it required health professionals and vaccine manufacturers to report to the US Department of Health and Human Services specific adverse events (AEs) that occur after the administration of routinely recommended childhood vaccines.1,–,3 The act also led to the creation of the Vaccine Adverse Events Reporting System (VAERS) in 1990 under the joint administration of the Centers for Disease Control and Prevention and the US Food and Drug Administration (FDA).4 In addition to certain AEs that are required by law to be reported, voluntary reporting of additional AEs from health professionals and the general public is also encouraged.5 VAERS accepts reports directly from consumers and health care providers for events that occur at any time after vaccination. In addition, the vaccine manufacturers are required to report to VAERS serious and other medically important conditions within 15 calendar days (Code of Federal Regulations code reference).6 Manufacturer reports comprise ∼35% of all reports to VAERS.
Although VAERS usually cannot provide definitive evidence of a causal association between vaccines and particular risks, its unique role as a national spontaneous reporting system in which health care providers participate enables the early detection of potential vaccine-safety concerns.2,7,–,9 Additional objectives of VAERS include monitoring the safety of newly licensed vaccines,10 assessing potential risk factors for adverse reactions to vaccines, and ultimately improving the safety of vaccines and vaccination practices. VAERS 1-page reporting form collects demographic information about the reporter and the patient as well as a narrative and clinical descriptors regarding AEs.11 Patient information is kept confidential in accordance with federal law.12 During the period of this study, all reports, signs, symptoms, and diagnoses mentioned in the description of the AE were coded by using the FDA's Coding Symbols for Thesaurus of Adverse Reaction Terms (COSTART).13 By federal regulations, as reflected on the VAERS form, a report is considered serious if the event results in death, hospitalization, a life-threatening event, disability, or a prolonged hospital stay.6 Reporters may describe an event as serious without independent verification of the event's medical outcome. All reports classified as serious are followed up by a team of nurses to obtain clinical information from all relevant medical records.
Since the inception of VAERS in October 1990, the available reporting routes were mail, fax, or telephone. On March 25, 2002, the option of secure electronic reporting via the Internet by accessing http://secure.vaers.org/VaersDataEntryintro.htmwas added to VAERS to allow expedited reporting of AEs in anticipation of wider use of counter-bioterrorism vaccines such as those against smallpox and anthrax.14 Any person with Internet access may report vaccine AEs via this route.
In January 2003, the US federal government initiated voluntary smallpox vaccination for health care workers. To monitor the occurrence of expected AEs from previous smallpox vaccination experience and of possible new or unknown events, Internet-based report (IBR) submission was promoted to state and local health departments implementing the smallpox vaccination program as the preferred method of reporting.14 Previous analyses have documented better timeliness of electronic reporting of notifiable public health conditions compared with that of paper-based methods.15
Because IBRs were added to VAERS to increase timeliness and completeness of reporting, we evaluated report timeliness and completeness of IBRs in comparison with non–Internet-based reports (NIBRs). As a secondary study objective we analyzed demographic and epidemiologic characteristics of reports.
We selected all reports to VAERS received from March 25, 2002, to March 25, 2006. To evaluate the timeliness of reports we compared the report intervals. Report interval was defined as the period in days from date of the first onset of symptom(s) to the date that the VAERS report was received. Report completeness was assessed by using 4 key variables: vaccination date; birth date; onset date; and age at vaccination.
Comparisons of the proportion of IBRs and NIBRs were made for 6 age groups <1, 1 to 2, 3 to 6, 7 to 17, 18 to 64, and >65 years. These age groups correspond to the recommended infant, adolescent, and adult vaccination ages. We also compared the proportion of most frequently reported vaccines and most frequently reported AEs in IBRs versus NIBRs.
The US smallpox vaccination program initiated in January 2003 involved the specific promotion of IBRs14 and included wide public and vaccine-provider education on VAERS reporting. The program provided a unique opportunity to assess the impact of the introduction of IBRs as a new reporting mode. Thus, separate analyses were conducted for smallpox vaccine reports and non–smallpox vaccine reports to compare timeliness and completeness of IBRs with NIBRs (Table 1). Because of the widespread annual use of influenza vaccine, we also compared separately the proportion of influenza vaccine IBRs with NIBRs.
VAERS data were analyzed by using SAS 8 software (SAS Institute Inc, Cary, NC). To compare the proportions of key variables, we calculated 95% confidence intervals (CIs) for the ratios of proportions.
From March 25, 2002, to March 25, 2006, VAERS received a total of 9101 (13.3%) IBRs and 59 309 (86.7%) NIBRs. Sixteen percent (n = 1455) of the IBRs were after smallpox vaccine, which represents half of all smallpox reports to VAERS (Fig 1).
Overall, IBRs had shorter reporting intervals than NIBRs (eg, for the 0- to 2-day reporting interval, 26% vs 3%, ratio: 8.6 [95% CI: 8.1–9.2]) (Fig 2). This difference in reporting intervals remained similar after excluding reports after smallpox vaccination (eg, for the 0- to 2-day reporting interval, 26% vs 1.9%, ratio: 13.9 [95% CI: 12.8–15.1]). Also, comparing smallpox vaccine reports only, the report interval remained longer for NIBRs compared with that of IBRs (eg, for the 0- to 2-day reporting interval, 18.3% vs 3.8%, ratio: 4.8 [95% CI: 3.6–6.5]).
Overall, the proportion of report completeness was higher for IBRs compared with NIBRs (eg, 98.6% vs 91.5% for reported vaccination date, ratio: 1.08 [95% CI: 1.07–1.08]; and 95.9% vs 86.5% for onset date, ratio: 1.10 [95% CI: 1.10–1.11]) (Table 1). This difference remained similar after excluding reports after smallpox vaccination and comparing only smallpox vaccine reports (Table 1).
There was a difference between the 2 reporting routes with regard to vaccine-recipient age (Fig 3). Fifty-two percent of IBRs involved vaccine recipients aged 18 to 64 years compared with 31% of NIBRs (ratio: 1.7 [95% CI: 1.65–1.73]) (Fig 3). After excluding reports after smallpox vaccination, which was targeted to 18- to 64-year-old potential vaccine recipients, this difference remained 43% for IBRs versus 29% for NIBRs (ratio: 1.5 [95% CI: 1.44–1.53]). Similarly, after limiting IBRs for influenza vaccination in vaccine recipients aged 18 to 64 years, this difference became 61.3% for IBRs versus 51% for NIBRs (ratio: 1.18 [95% CI: 1.12–1.24]; data not shown). It is interesting to note that for IBRs, 19.5% of all reports were made by parents or patients compared with 5% for NIBRs (ratio: 3.8 [95% CI: 3.61–4.03]). Reporters described as “other,” which may include pharmacists and other nontraditional vaccine providers, accounted for 35% of IBRs compared with 15.7% of NIBRs (ratio: 2.23 [95% CI: 2.16–2.31]). The differences according to reporting source between IBRs and NIBRs remained similar after excluding reports after smallpox vaccination (for parents or patients, 22.2% IBRs vs 5% NIBRs, ratio: 4.4 [95% CI: 4.2–4.8] and for other reporters, 28.8% vs 14.6%, ratio: 1.98 [95% CI: 1.9–2.1]; data not shown).
Most frequently reported vaccines or vaccine combinations via IBR were those for influenza vaccine (17%) followed by smallpox vaccine (13%) and anthrax vaccine (7%), whereas the AEs after vaccines or vaccine combinations most frequently reported via NIBR were those for influenza vaccine (10%) and diphtheria, tetanus toxoid, acellular pertussis vaccine (DTaP) plus inactivated poliovirus vaccine plus measles-mumps-rubella vaccine (8.4%), and varicella vaccine (7.1%) (Table 2).
The most frequently reported AEs via IBRs were fever (23%) followed by vasodilatation (ie, a coding term usually used for redness with swelling at the injection site or flushing) (19%) and rash (16%); the most frequent AEs reported by NIBRs were hypersensitivity at the injection site (includes any injection-site reaction [eg, swelling or erythema]) (20%) followed by fever (19%) and vasodilatation (18%). After excluding reports after smallpox vaccination from the 2 reporting routes, the proportion of most frequently reported AEs remained similar to the overall data (data not shown).
Overall and for smallpox and influenza vaccines, IBRs are more timely and complete compared with NIBRs. Our findings demonstrate that IBRs to VAERS provide an important improvement when compared with the traditional paper-based reports. The timeliness of reporting may enable the Centers for Disease Control and Prevention and the FDA to attain more rapid detection of new or rare AEs. A shorter reporting interval provides a better chance for timely detection of serious or life-threatening events and, consequently, timely public health intervention.16,–,21 Report completeness is critical to the validity of VAERS reports; for example, age and onset interval can provide important information on the biological plausibility of the reported AE being associated with the administered vaccine.16,–,19 IBRs have the potential to reduce transcription and data-entry errors and result in more accurate data. For the smallpox vaccination program from 2002 to 2004, IBRs resulted in increased proportional usage of electronic reporting (50% vs 13.3% overall usage in VAERS), which in turn contributed to the timely detection of the unanticipated finding of myopericarditis in adults after smallpox vaccination.20,21
When we excluded smallpox vaccination reports (16%) from the analysis, report completeness and timeliness remained higher for IBRs compared with NIBRs. Hence, IBRs proved to be effective in accomplishing a higher proportion of completeness and timeliness independent of smallpox vaccination publicity. Overall, the types of AEs reported did not differ between IBRs and NIBRs. Similarly, there was no difference in the proportions and type of reported AEs between IBRs and NIBRs.2 The fact that IBRs after smallpox vaccination showed an overall similar higher proportion of completeness and timeliness compared with NIBRs reinforces the advantages of IBRs.
After excluding either smallpox vaccination or influenza vaccinations, the reporter type of parent and/or patient in the 18- to 64-year age groups remained almost double in IBRs. These results may reflect the fact that there is an increased usage of the Internet by working-age adults.22,–,24 However, we do not have sufficient information on parents or patients to determine if the higher proportion of reports is a result of a shift in reporting mode or an increase of vaccine usage, vaccine-safety awareness, or new vaccines in the market.2 These data could not be collected directly from VAERS reporters because of privacy and confidentiality restrictions. The significant difference in the reporting demographics suggests that IBRs may be replacing the traditional reporting methods. However, we cannot validate our assumption by using VAERS data alone; this issue warrants additional study using survey methods.
There are limitations of IBRs as currently implemented because of the current information technology infrastructure: reporters cannot directly submit supporting data such as laboratory tests results or hospital discharge notes, and they cannot send detailed follow-up information regarding the patient's condition, which in many cases includes additional important medical information.25 Manufacturers and other immunization information systems (IIS), such as immunization registries, currently cannot submit individual VAERS reports by using the Internet or other electronic data systems. A pilot project to receive electronic reports from various manufacturers began in 2007. Currently, there is no capability to directly accept reports from electronic medical records or other existing computerized medical information systems.
Recently, an expert working group formed by the International Conference on Harmonisation has agreed to extend the use of message-specification standards to support vaccine AE reporting from pharmaceutical firms.25 This message-specification agreement (sometimes referred to as the E2B) is essentially a list of standard definitions of data structure to permit ease of electronic transmission of individual case safety reports.26 The International Conference on Harmonisation guideline has been revised, and on March 2005 the FDA published its guidance to industry for electronic submission of AE reports to VAERS.25 This guidance will facilitate electronic reporting of all AEs from vaccine manufacturers to the US government and to the VAERS system in a standard format. These electronic modifications will enable the manufacturer to transmit AE reports electronically to VAERS. Electronic reporting may further decrease the report interval for all reports and, thus, improve the capability of VAERS to address potential vaccine concerns in a timely manner.
Although IBRs have resulted in improved timeliness and completeness of AE reporting, additional improvements of electronic messaging to VAERS could include acceptance of data from electronic medical records and acceptance of selected information from IIS or immunization registries.27 Such improvements will enable vaccine providers to record and report AEs directly to VAERS by using their current electronic IIS or vaccine registries. These sophisticated electronic reporting techniques could provide rapid receipt confirmation and integration with electronic medical record sources and IIS. Such enhancements will further increase efficiency and completeness of electronically submitted VAERS reports.
Pediatricians and other front-line health care providers are a key to the success of VAERS in ensuring and improving the safety of vaccines. A small number of astute reporters to VAERS triggered the withdrawal of the first rotavirus vaccine in 199916 but also ultimately contributed to the development of new and safer rotavirus vaccines.28 Knowledge and use of secure Web-based vaccine AE reporting should become part of the well-informed clinician's vaccine-safety toolkit. More information and reporting resources are available at www.vaers.hhs.gov.
We thank the following members of the Centers for Disease Control and Prevention, Atlanta, GA: Ali Rashidee, Warren Williams, Lisa Galloway, Claudia Vellozzi, Drew Baughman, Kim Ware, Sean Shadomy and Roseanne English; VAERS Working Group: Susanne Pickering, Elaine Miller, Guillermo Herrera and Frank Destefano; Immunization Safety Office, Office of the Chief Science Officer, Centers for Disease Control and Prevention, Atlanta, GA. Martha Bryant- Genevier, Dale R. Burwen, Soju Chang, Miles Braun, Hector Izurieta, Ann W. McMahon, Jane Woo, Philip Perucci, Lise Stevens, and Robert Ball; Center for Biologics Evaluation and Research, Food and Drug Administration, Rockville, Maryland. Geoffrey Evans, Health Resources and Services Administration, Rockville, Maryland.
We also thank Col John Grabenstein (Department of Defense, Washington, DC) and the staff of Constella Group, Inc (Durham, NC).
- Accepted November 29, 2010.
- Address correspondence to Penina Haber, MPH, Immunization Safety Office, Division of Healthcare Quality Promotion, Centers for Disease Control and Prevention, Mail Stop D-26, 1600 Clifton Rd, Atlanta, GA 30333. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
- AE =
- adverse event •
- VAERS =
- Vaccine Adverse Event Reporting System •
- FDA =
- Food and Drug Administration •
- IBR =
- Internet-based report •
- NIBR =
- non–Internet-based report •
- CI =
- confidence interval •
- IIS =
- immunization information system
- 1.↵National Childhood Vaccine Injury Act of the Public Health Service, §2125 (1986); codified at 42 USC §300aa-26.
- 3.↵Centers for Disease Control and Prevention. US Food and Drug Administration. Table of reportable events following vaccination. Available at: vaers.hhs.gov/resources/VAERS_Table_of_Reportable_Events_Following_Vaccination.pdf. Accessed January, 2010
- 6.↵US Food and Drug Administration. 21 CFR part 600.80. Postmarketing reporting of adverse experiences. Fed Regist. 1997;62:52252–52253. Available at: www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?fr=314.80. Accessed January, 2010
- 11.↵Centers for Disease Control and Prevention. VAERS form. Available at: www.vaers.hhs.gov/resources/vaers_form.pdf. Accessed January, 2010
- 13.↵US Food and Drug Administration. COSTART: Coding Symbol for Thesaurus of Adverse Reaction Terms. 5th ed.Rockville, MD: US Food and Drug Administration, Center for Drugs and Biologics, Division of Drug and Biological Product Experience; 1989
- Zanardi LR,
- Haber P,
- Mootrey GT,
- Niu MT,
- Wharton M
- Haber P,
- Chen RT,
- Zanardi LR,
- Mootrey GT,
- English R,
- Braun MM
- 19.↵Centers for Disease Control and Prevention. Guillain-Barré syndrome among recipients of Menactra meningococcal conjugate vaccine: United States, June–July 2005. MMWR Morb Mortal Wkly Rep. 2005;54(40):1023–1025. Available at: www.cdc.gov/mmwr/preview/mmwrhtml/mm54d1006a1.htm. Accessed January, 2010
- Arness MK,
- Eckart RE,
- Love SS,
- et al
- 24.↵Manhattan Research. Taking the pulse v5.0. Available at: www.manhattanresearch.com/products/Strategic_Advisory/ttp/default.aspx. Accessed January, 2010
- 25.↵International Conference on Harmonisation (ICH). Available at: www.ich.org/home.html. Accessed January, 2010
- 26.↵US Food and Drug Administration. Guidance for industry E2B(M): Data elements for transmission of individual case safety reports. Available at: www.fda.gov/downloads/RegulatoryInformation/Guidances/ucm129434.pdf. Accessed January, 2010
- Haber P,
- Patel M,
- Izurieta HS,
- et al
- Copyright © 2011 by the American Academy of Pediatrics