Abstract
Objectives. To compare 3 communication modes (postal, fax, and e-mail) in a rotavirus vaccine physician survey.
Methods. We used 3 communication modes to distribute a survey to physicians listed in the membership directory of the Georgia Chapter of the American Academy of Pediatrics. The directory listed 1391 members; however, 404 were deemed ineligible on the basis of their listing as a specialist, retiree, resident in training, or government public health employee. Of the 987 members expected to administer vaccines, 150 were selected randomly to receive the postal survey (postal group). Of the remaining listings, 488 (58%) of 837 listed a fax number; 150 members were selected randomly and faxed a survey (fax group). Of the remaining members, 266 (39%) of 687 had e-mail addresses listed; 150 members were selected randomly for the e-mail survey (e-mail group). A follow-up survey was sent by the same mode at 2 weeks. A final survey was sent via another mode (mixed mode) at 1 month: by fax to e-mail and postal nonresponders and by post to fax nonresponders and those without fax.
Results. Eligible respondents in the 3 survey groups were similar in their practice setting and location. Although the e-mail group had fewer median years (8 years) since medical school graduation than the fax group (19 years) and postal group (17 years), a similar percentage of responders in all groups had computers (>85%) and Internet access (≥70%) at work. However, only 39% of members listed an e-mail address in the directory. In the 2 weeks after the first mailing, 39 surveys were completed via postal mail, 50 via fax, and 16 via e-mail. In the 2 weeks after the second contact (sent at 2 weeks), 20 surveys were completed via postal mail, 15 via fax, and 17 via e-mail. The response rate after the first 2 mailings was 41% (59 of 143) for postal, 47% (65 of 137) for fax, and 26% (33 of 125) for e-mail surveys. The third and final survey (sent 1 month after the first mailing) was sent by a different (ie, mixed) mode and elicited an additional 73 responses: 19 responses (15 postal, 4 fax) from the postal group, 19 responses (18 postal, 1 fax) from the fax group, and 35 responses (15 postal, 13 fax, 7 e-mail) from the e-mail group. Twenty-three percent (9 of 40) of the e-mail and 18% (15 of 83) of the fax surveys completed were returned on the same or subsequent day they were sent, compared with none of the postal surveys. There were significant differences among the 3 groups for invalid addresses/numbers (4% postal, 8% fax, and 16% e-mail) listed in the directory. Using mixed modes as the third contact, the overall response rate increased from 39% before mixed mode to a final of 53%. On the basis of the 3 initial groups, responses to 1 of 12 rotavirus questions differed significantly.
Conclusions. Future use of e-mail surveys in selected circumstances is promising, because the majority of providers have Internet access and acknowledged interest in participating in e-mail surveys. E-mail surveys could be especially useful if rapid response time is necessary. There were fewer incomplete questions by participants who completed the e-mail survey compared with postal or fax participants. Updating membership e-mail addresses and routinely using e-mail as a communication tool should improve the ability to use e-mail surveys. There may need to be ongoing evaluations that critically evaluate providers’ responses to e-mail surveys compared with other survey modes before e-mail surveys can become a standard survey tool. In the meantime, mixed-mode surveys may be an option.
Current technology allows us to send and receive surveys by fax and e-mail/web in addition to postal mail. Postal surveys require ongoing costs for supplies, including postage and personnel time. In addition, response time is limited by the postal system, and data recording usually is performed by hand. Fax surveys require available fax machines by both the sender and the receiver. Responses may be returned by fax or mail courier, and data may be recorded manually or may be recorded by optical character recognition by some fax machines. Web surveys require computers with Internet access by both the sender and the receiver. Unless the surveyor wants a survey open to the public, a web survey must have some form of password protection. On completion, a web survey can be returned with just a keystroke and responses are recorded immediately into a database. Depending on the design of the web site, information as specific as how many minutes were spent on each question and how many questions were reviewed but not answered may be available.1 E-mail surveys (1 specific type of web survey) require that participants have an e-mail address as well as access to the Internet. The survey may be displayed in the e-mail text, as an attachment, or as a hyperlink to a web site.2
Few studies have evaluated the comparative effectiveness of postal, fax, and e-mail surveys among health care providers.3,4 This study was conducted as part of an evaluation to assess knowledge and attitudes of Georgia Chapter of the American Academy of Pediatrics (AAP) members regarding rotavirus vaccine.5 The objectives of this study were to evaluate and compare use of postal, fax, and e-mail modes of questionnaire delivery.
METHODS
Population and Design
A cross-sectional survey of members of the Georgia Chapter of the AAP was initiated in July 2001. The membership database, which is maintained by the local chapter, was last updated in January 2001. Among the 1391 listed members, 987 members were eligible to receive the survey on the basis of their self-reported activities (Fig 1). Excluded from participation because of their increased probability of not providing immunizations were specialists, residents, retirees, and government public health employees. Among the initial 987 members potentially eligible to participate, 100% had a postal address, 58% had a fax number, 39% had an e-mail address, and 27% had a fax number and an e-mail address listed in the Georgia Chapter of the AAP membership database. A sample of 450 members was selected to be mailed a survey by postal mail, fax, or e-mail. One third of the members were to receive the survey initially by postal mail (postal group), one third by fax (fax group), and one third by e-mail (e-mail group). Because all 987 eligible members had postal addresses listed in the membership database, a random sample of 150 members was selected first to be in the postal group. Of the remaining 837 members, 488 had a fax number listed in the database; 150 of these members were selected randomly to be in the fax group. Of those members not selected to receive a survey by either postal or fax modes (n = 687), 266 members had an e-mail address listed in the database. From this remaining group, 150 members were selected randomly to be in the e-mail group.
Enrollment of members of the Georgia Chapter of the AAP in the study (*may be in >1 category).
Postal surveys were mailed first class and included the 4-page survey, a cover letter from the Georgia Chapter of the AAP, and a postage return business envelope. Faxed surveys were sent with the cover letter. A prenotification e-mail message announcing the survey was e-mailed by the Georgia Chapter of the AAP to members in the e-mail group. An e-mail message that included the contents of the cover letter was sent by the author on the following day with a hyperlink to the web site where the survey could be taken. The e-mail survey design and layout were similar to the paper version. There was a continuous page of questions, which the participant could scroll through (forward or backward) to view all questions. This web site (www.zoomerang.com) hosts surveys, and only participants who were sent the e-mail could connect to the hyperlink and respond to the questionnaire. E-mail participants were assured that their e-mail address would not be used for other purposes.
Two weeks later, nonresponders were sent a reminder letter and another copy of the survey. In this article, we use the term “mailing” to indicate sending of the survey whether by postal, fax, or e-mail modes. The same mode that had been used initially to send the survey to each group was repeated (eg, fax nonresponders had another survey faxed to them).
Four weeks after the initial mailing and 2 weeks after the second mailing, a mixed-mode, or crossover, design was used for the third mailing. Nonresponders who were in the postal group and had a listed fax number were sent the survey via fax; those without a fax number were sent a third postal survey. Nonresponders in the fax group were sent a survey by postal mail. E-mail nonresponders with a listed fax number were sent the survey by fax, and those without a fax number were sent a survey by postal mail; they also were given a third opportunity to respond via e-mail. In all 3 groups, participants were given the option of returning the completed survey by mail or fax, but only those in the e-mail group could return a survey by e-mail. Each survey participant was issued a unique identifying number that was recorded on each page of the survey. A record was kept of the mode(s) used to send out the survey and the mode used by the participant to return the completed survey.
Data Collection and Analysis
The survey tool included questions about provider attitudes and practices related to rotavirus disease and vaccines. Findings related to these questions will be reported in a separate article. The survey also included questions related to computer and Internet usage, preferences for how to receive important vaccine information, and whether the participant would be willing to participate in future e-mail vaccine surveys. Demographic information obtained included practice setting, practice location, year of medical school graduation, and acceptance of Medicaid.
Attempts to correct invalid postal addresses and fax numbers were done via telephone, whereas attempts to correct invalid e-mail address were done via fax and postal mail. Data were analyzed with SAS version 8 (SAS Institute Cary, NC). Bivariate associations between provider characteristics and the mode used to respond (postal, fax, e-mail) were assessed using χ;2 and t tests. Five-point Likert scale questions were made dichotomous, strongly agree/agree versus neutral/disagree/strongly disagree or strongly preferred/preferred versus neutral/less preferred/not preferred. Only those respondents who reported that they provided immunizations and who did not self-identify as residents, retired, specialists, or government public health workers were included in the final analysis. This study was reviewed and approved by the Institutional Review Board of the Centers for Disease Control and Prevention and the Human Investigations Committee of Emory University School of Medicine.
RESULTS
Respondent Characteristics by Group (Initial Survey Mode)
Eligible respondents in the 3 survey groups were similar in their practice setting and location, with 50% to 66% working in a multipartner private pediatric office in the suburbs (Table 1). Medicaid patients were accepted by 88% to 92% of pediatricians in the 3 groups. However, the providers in the e-mail group were significantly (P < .05) younger than those in the fax group. The median years of medical school graduation for the postal, fax, and e-mail groups were 1984, 1982, and 1993, respectively. Eighty-six to 92% of providers had a computer at work. There was no significant difference in access to the Internet at work and at home among the 3 groups. Forty-one percent of members in the postal group compared with 71% in the e-mail group (P < .05) and 50% in the fax group checked their e-mail daily, whereas 10% of providers in the postal group and 9% in the fax group did not have e-mail or never checked their e-mail. Those respondents in the e-mail group were more likely than the postal group to download vaccine information from the Internet (P < .05).
Provider and Practice Characteristics by Group*
Postal Group
Ninety-five percent (143 of 150) of the randomly selected members for the postal mailing were contacted. One provider was identified as a public health worker and excluded. Six members (4%) had envelopes that were returned because of incorrect addresses and were excluded. The response rate of 55% (78 of 143) included 74 (95%) members who responded via postal mail and 4 (5%) who responded via fax (Table 2). Among the 78 respondents in this group, 59 (76%) provided immunizations and were not retired, a resident, or a specialist and were included in final analysis.
Number of Responses After Each Contact by Group
Fax Group
Contact was made with 148 (99%) of 150 members selected for the fax group. One member was identified as a public health employee and excluded. Twelve fax numbers (8%) were incorrect. However, in the third mixed-mode mailing, the previously unreachable providers were sent a postal survey and only 1 postal survey was returned with an incorrect address. The 57% (84 of 148) response rate in this group included 66 (79%) responses via fax and 18 (21%) responses via postal mail (Table 2). The median number of attempts to fax a survey successfully was 1 with a range from 1 to 11 (mean: 2). A page of the survey was missing on 7 (11%) of 66 surveys returned via fax. Five of the 7 surveys were later completed after notifying the provider of the error. The number of eligible providers included in the final analysis was 74 of 84.
E-Mail Group
The E-mail Group had 146 of 150 members who were reachable. Two pairs of providers had the same e-mail address; therefore, only 148 e-mails were sent. Twenty-three (16%) e-mail addresses were invalid; however, during the third mixed-mode mailing, only 2 members could not be reached by any mode. Of the 146 surveys sent, the response rate was 47% (68 of 146). Forty (32%) of the 125 providers who were sent an e-mail survey responded by e-mail (Table 2). Among the 63 nonresponders who were sent a postal survey in the third mailing, 15 (24%) responded by postal mail. Among the 46 nonrespondents who were sent a fax survey in the third mailing, 13 (28%) responded by fax. None of the providers who received only an e-mail survey (ie, before the third mixed-mode mailing) returned the survey via postal mail or fax. The number of eligible providers included in the final analysis and who provided immunizations was 48 of 68.
Timeline of Survey Receipt
In the 2 weeks after the first mailing, 39 surveys were completed via postal mail, 50 via fax, and 16 via e-mail (Table 2). In the 2 weeks after the second contact (sent at 2 weeks), 20 surveys were completed via postal mail, 15 via fax, and 17 via e-mail. The response rate after the first 2 mailings was 41% (59 of 143) for postal, 47% (65 of 137) for fax, and 26% (33 of 125) for e-mail surveys (Fig 2). The third and final survey (sent 1 month after the first mailing) was sent by a different (ie, mixed) mode and elicited an additional 73 responses: 19 responses (15 postal, 4 fax) from the postal group, 19 responses (18 postal, 1 fax) from the fax group, and 35 responses (15 postal, 13 fax, 7 e-mail) from the e-mail group (Table 2). Twenty-three percent (9 of 40) of the e-mail and 18% (15 of 83) of the fax surveys completed were returned on the same or subsequent day they were sent, compared with none of the postal surveys.
Response rate pre- (first and second contact) and post- (third contact) mixed-mode contact among the 3 groups.
Overall Response and Eligibility
The final response rates were 55% (78 of 143) for the postal group, 57% (84 of 148) for the fax group, and 47% (68 of 146) for the e-mail group (Fig 2). The overall provider response rate was 53% (230 of 437), and 181 (79%) were eligible. Of the 181 eligible surveys returned, 85 (47%) were completed and returned by postal mail, 65 (36%) by fax, and 31 (17%) by e-mail. We did not have sufficient information to compare characteristics of survey responders and nonresponders.
The vast majority (78%) of all providers were willing to participate in future e-mail surveys. When asked on a Likert scale how they preferred to obtain important vaccine updates via each mode (postal, fax, e-mail, web, telephone), the postal group preferred postal (81%), the fax group preferred fax (69%), and the e-mail group preferred e-mail (73%). Overall, >70% preferred postal updates.
The focus of the survey questions was on rotavirus disease and vaccine. Twelve rotavirus questions were selected before administration of the survey to determine whether the responses to these questions would be similar among providers in the postal group, fax group, and e-mail group. Responses to 1 of 12 questions were significantly different (P < .05) among the 3 groups. Although the e-mail group respondents were more recent medical school graduates, analysis stratified by age indicated that age did not explain the difference in response to this 1 question.
Completeness and Accuracy of Responses by Mode of Response
Seventy-eight of 3713 (2.1%) questions were not answered by those who responded by postal mail, 94 of 3384 (2.8%) for those who responded by fax, and 6 of 1410 (0.4%) for those who responded by e-mail (P < .001). This did not include any questions that participants were instructed to skip. Three questions in the survey had the instruction of “Select One” response. Twelve percent (28 of 237) of responses by postal mail, 9% (20 of 216) of responses by fax, and none of the e-mail responses to these 3 questions were invalid as the provider selected >1 answer (P < .005). The design of the e-mail survey did not allow the respondent to select >1 answer to this type of question.
DISCUSSION
Although most providers had access to e-mail and the Internet, the lower percentage of responses via e-mail suggests that exclusively performing an e-mail survey is not feasible at the current time. Mixed-mode survey design could serve as an important tool to solicit information from providers while introducing e-mail surveys. By using a crossover design with the third mixed-mode mailing, the response rate was increased from 39% to 53%. Twenty-two percent of participants responded to the third mixed-mode contact. Using mixed modes also may increase representativeness of the survey. However, to limit potential response bias optimally, all eligible providers in the population should be reachable by all modes being used.
The key findings of this study are summarized in Table 3. Problems with the e-mail survey included a small percent of providers (35%) with an e-mail address listed in the Georgia Chapter of the AAP directory. However, >90% of participants reported in the survey that they checked e-mail at least monthly. Therefore, many members must have had an e-mail address but did not provide this information to the Georgia Chapter of the AAP. There were also more address errors with e-mail as compared with postal and fax addresses/numbers. We are uncertain whether these errors were typographical in nature or the members had received a new e-mail address and the directory had not been updated. The survey host used in this study used cookies (electronic identifiers) to link the e-mail address of providers who were sent a survey to those who responded. This eliminated any responses from unwanted e-mail addresses; however, if an e-mail user did not have cookies enabled, then they would not have been able to respond to the survey via e-mail.
Key Findings From the 3 Survey Modes
If rapid response time is required, then a faxed survey may be more helpful than a postal or e-mail survey. E-mail surveys have the ability for rapid response as noted by the 9 providers who replied on the same or subsequent day the survey was sent. However, unless a population is accustomed to checking their e-mail messages daily, this mode will not allow for a rapid response. Fewer than one half of these providers checked their e-mail daily. Postal surveys require a minimum of 3 days to be sent and returned. This time interval will vary greatly, but most e-mail surveys are returned more quickly than postal surveys.6–9 This group of providers was already familiar with receiving informational faxes from the Georgia Chapter of the AAP.
One of 12 preselected questions had a significantly different response among the 3 study groups. This may suggest that interpretation of results from mixed-mode surveys will need to be performed with caution.9 The e-mail group graduated from medical school more recently than the fax group. There was also a substantial difference in graduation year with the e-mail and postal groups, although this did not reach significance. These differences may increase bias on the basis of the contents of the questions.
This study has several limitations. First, only members of the Georgia Chapter of the AAP who provide immunizations were surveyed; this sample may not be representative of all AAP members or all pediatricians. Second, the effect that the survey topic had on the response rate was not controlled for in this study.10,11 Also, in the Georgia Chapter of the AAP database, <50% the members listed an e-mail address; however, as reported in response to the questionnaire, >90% of providers had access to or used e-mail. This indicates that many more providers could have participated in the e-mail group. Although not evaluated, there is potential that more providers had a fax number than listed in the database as well. Ideally, we would have had a complete database and only those providers with all 3 modes listed would have been eligible to respond. This would have allowed us to test the validity of response among the 3 different modes.
Among the published studies evaluating provider response to e-mail surveys, Schleyer and Forrest evaluated3 cost and responses of a web versus postal mail survey among dentists. They reported the cost of a web-based survey as 38% less than an equivalent mail survey. Khan et al4 examined a fax survey among pharmacists using a fax data system in which the fax machine becomes an automated data-entry system using optical character recognition. The study reported a reliability of at least 0.95 for the faxed responses. University staff (but not necessarily health care providers) were surveyed by e-mail or postal mail in 2 studies in 1997. One study found a similar response rate between the 2 groups,6 but the other study resembled our study in finding a significantly lower response rate via e-mail.7 A recent study by Kleiner et al12 showed that 74% of general pediatricians had e-mail access in their office, which supports our findings.
Implications
Future use of e-mail surveys in selected circumstances is promising, because the majority of providers have Internet access and acknowledged interest in participating in e-mail surveys. The results of this e-mail survey were similar to traditional postal surveys. E-mail surveys could be especially useful if rapid response time is necessary. This is especially important now in the era when questions about smallpox and anthrax vaccine may need rapid responses. There were fewer incomplete questions by participants who completed the e-mail survey compared with postal or fax participants. Previous studies have supported this,6 but other studies have found the same number of incomplete questions with e-mail surveys.8,9 The design of e-mail/web surveys certainly plays a role in responses.13,14 The lack of handling paper, envelopes, and postage was also an advantage of the e-mail survey, and others have reported similar advantages.3 Although faxes also can be sent and returned rapidly, many fax numbers in these pediatric offices were busy throughout the day, requiring numerous attempts to complete the transmission. There is also potential to lose pages of the survey during receipt or transmission of the fax.
CONCLUSION
Updating membership e-mail addresses and routinely using e-mail as a communication tool should improve the ability to use e-mail surveys. There may need to be ongoing evaluations that critically evaluate providers’ responses to e-mail surveys compared with other survey modes before e-mail surveys can become a standard survey tool. In the meantime, mixed-mode surveys may be an option.
Acknowledgments
We thank Charles LeBaron, MD, for helpful comments throughout and the Georgia Chapter of the AAP for assistance in implementing this survey, especially Karen Townsend, RN, Rick Ward, Harry Keyserling, MD, and Martin Greenberg, MD.
Footnotes
- Received June 21, 2002.
- Accepted December 2, 2002.
- Reprint requests to (M.S.M.) National Immunization Program, 1600 Clifton Rd, Mailstop E-52, Atlanta, GA 30333. E-mail: mmassoudi{at}cdc.gov
REFERENCES
- Copyright © 2003 by the American Academy of Pediatrics
In this issue
Jump to section
Related Articles
Cited By...
- What has been the impact of Covid-19 on Safety Culture? A case study from a large metropolitan teaching hospital
- Use of Standing Orders for Vaccination Among Pediatricians
- Job type and other socio-demographic factors associated with participation in a national, cross-sectional study of Danish employees
- Primary Care Physicians' Struggle with Current Adult Pneumococcal Vaccine Recommendations
- How do general practitioners put preventive care recommendations into practice? A cross-sectional study in Switzerland and France
- Primary Care Physicians Perspectives About HPV Vaccine
- Characteristics of Physicians Who Dismiss Families for Refusing Vaccines
- Are primary care providers implementing evidence-based care for breast cancer survivors?
- Surveys and Response Rates
- Knowledge and understanding of urinary incontinence: Survey of family practitioners in northern Alberta
- Why are response rates in clinician surveys declining?
- How "should" we write guideline recommendations? Interpretation of deontic terminology in clinical practice guidelines: survey of the health services community
- Physicians' Awareness and Attitudes Toward Decision Aids for Patients With Cancer
- Tobacco-related medical education and physician interventions with parents who smoke: Survey of Canadian family physicians and pediatricians
- Variability in management of early severe sepsis
- Human Papillomavirus Vaccine Recommendations and Agreement with Mandated Human Papillomavirus Vaccination for 11-to-12-Year-Old Girls: a Statewide Survey of Texas Physicians
- Combining Web-Based and Mail Surveys Improves Response Rates: A PBRN Study From PRIME Net
- Survey Design in Orthopaedic Surgery: Getting Surgeons to Respond
- Cardiovascular risk assessment in family practice: A practice-based tool