Abstract
OBJECTIVE: We surveyed pediatricians to elicit their perceptions regarding frequency, contributing factors, and potential system- and provider-based solutions to address diagnostic errors.
METHODS: Academic, community, and trainee pediatricians (N = 1362) at 3 tertiary care institutions and 109 affiliated clinics were invited to complete the survey anonymously through an Internet survey administration service between November 2008 and May 2009.
RESULTS: The overall response rate was 53% (N = 726). More than one-half (54%) of respondents reported that they made a diagnostic error at least once or twice per month; this frequency was markedly higher (77%) among trainees. Almost one-half (45%) of respondents reported diagnostic errors that harmed patients at least once or twice per year. Failure to gather information through history, physical examination, or chart review was the most-commonly reported process breakdown, whereas inadequate care coordination and teamwork was the most-commonly reported system factor. Viral illnesses being diagnosed as bacterial illnesses was the most-commonly reported diagnostic error, followed by misdiagnosis of medication side effects, psychiatric disorders, and appendicitis. Physicians ranked access to electronic health records and close follow-up of patients as strategies most likely to be effective in preventing diagnostic errors.
CONCLUSION: Pediatricians reported making diagnostic errors relatively frequently, and patient harm from these errors was not uncommon.
WHAT'S KNOWN ON THIS SUBJECT:
Little is known about diagnostic errors in general pediatric practice, other than data from malpractice claims. We surveyed pediatricians to obtain data to understand and to reduce diagnostic errors among children.
WHAT THIS STUDY ADDS:
Pediatricians reported making diagnostic errors relatively frequently, and patient harm was not uncommon. Our study provides new data about types of diagnostic errors among children and their causes and lays the groundwork for a multifaceted approach to reduce such errors.
Errors in diagnosis constitute a sizable proportion of medical errors in the United States and are responsible for significant costs and harm.1,–,10 For instance, in the Utah-Colorado study, diagnostic breakdowns were the most-common type (36%) of preventable ambulatory adverse events that led to hospital admission.10 Although medical errors related to treatment (eg, medications or surgery) have received much-needed attention,11 errors in the diagnostic process remain relatively understudied.12 Data regarding diagnostic errors in pediatric practice settings are especially limited. To date, knowledge of these errors is limited mostly to events that result in malpractice claims. Error in diagnosis is the most-commonly identified reason (32%) in closed pediatric malpractice claims, with the highest median indemnity payments and defense expenses.13 However, diagnostic errors that result in claims may represent only a small proportion of all diagnostic errors; given their low frequency and high level of severity, they may not be representative of all types of diagnostic errors that occur in routine practice.2,14 Furthermore, underlying contributory factors for litigated diagnostic errors might differ systematically from those for nonlitigated errors.14 In short, malpractice claims have provided useful data for understanding diagnostic errors but may not represent the entire spectrum of diagnostic errors.
Errors of diagnosis occur when diagnosis is unintentionally delayed (sufficient information was available earlier), wrong (another diagnosis was made before the correct one), or missed (no diagnosis was ever made), as judged by the eventual appreciation of more-definitive information.4 Although diagnostic errors may occur when symptoms of a disease are atypical or absent, they are largely related to cognitive (eg, faulty data-gathering or clinical reasoning) and systems-related (eg, issues with policies, processes, and procedures or organizational issues) factors.2,4,15,–,28 This understanding of diagnostic errors is based largely on nonpediatric settings and populations.
The diagnostic process for children is unlikely to be entirely analogous to that for adults. The most-prevalent types of diseases and their associated diagnostic processes are different (eg, certain infections are more common and cancer is less common among children). Other unique factors in pediatrics include systems issues, patient/caregiver issues, physician training, and practice workflow, all of which are likely to influence diagnostic errors. To enhance our understanding of the relative prevalence of diagnostic errors, contributory factors, and potential preventive strategies, we designed an anonymous survey to study pediatricians' experiences with diagnostic errors. We also sought data on specific disease conditions commonly associated with diagnostic errors in pediatrics.
METHODS
Setting and Participants
To obtain diverse perspectives and solutions, we sampled 3 types of pediatricians, that is, academic pediatricians (general pediatricians and subspecialists), trainees (residents and fellows), and nonacademic, community-based pediatricians. Our sample consisted of academic pediatricians and trainees from two tertiary care institutions in Houston (Sites A and B) and one in Cincinnati (Site C). Although we invited participation from all trainees and general pediatricians at the 3 study sites, we randomly invited only 50% of pediatric subspecialists at Sites A and C, because of the large number of subspecialists at those sites. Community pediatricians were selected from large practice groups affiliated with Sites A and C and represented 109 practice locations. Institutional review board approval was obtained at all 3 study sites.
Survey Development
We were unable to find an existing survey that addressed our study questions. Building on previous empirical and theoretical research on diagnostic errors, we developed a comprehensive survey after an exhaustive review of both adult and pediatric literature. A psychometrician guided the survey development process, which included item writing and refinement by using subject matter expert input, clinician review, and pilot-testing. The content of survey items was informed by published studies, input from experts in patient safety and diagnostic errors, and feedback from a sample of pediatricians from our study sites. The survey was developed for Internet-based administration by using the resources of a user-friendly, commercial, Internet-based, survey administration service (www.SurveyMonkey.com).
After initial refinement of survey items, we convened 4 focus groups of local pediatricians to review the survey for readability, clarity, and ease of completion in a Internet-based environment. The survey was then pilot-tested with 8 general pediatricians. To understand the burden of diagnostic errors in relation to other types of errors, we inquired about frequency and potential to harm for errors related to medications, surgery, and other clinical activities in pediatrics. In addition, the final, 23-item survey (Supplemental Appendix.) assessed physicians' perceptions of (1) the most-common process breakdowns associated with diagnostic errors (eg, problems with history and examination, test ordering, performance, or interpretation, or follow-up care, as gleaned from the literature2,4,7,8,26,29,–,31); (2) the most-common contributory factors leading to diagnostic errors; (3) the most-common types of medical conditions associated with diagnostic errors; and (4) the most-useful strategies for prevention of diagnostic errors. For many questions, we used response scales with forced-choice ranking of the top 3 response choices, rather than Likert-style responses. The survey also assessed demographic information, self-reported frequency of diagnostic errors, attitudes toward discussing diagnostic errors, and previous training about diagnostic errors. The estimated completion time was 15 to 20 minutes, on the basis of pilot-testing.
Survey Administration
We identified potential participants from medical staff offices at each institution and sent each physician an e-mail invitation to complete the survey, followed by 2 e-mail reminders. As an incentive for participation, we provided an Internet link to a $10 gift certificate in the initial invitation. Participants accessed and completed the items anonymously; only the study site and pediatrician category for each participant were identified.
Data Analyses
Data were downloaded from the Internet survey administration service and analyzed by using SAS 9.2 (SAS Institute, Cary, NC). We generated descriptive statistics about the nature and frequency of diagnostic errors in aggregate and then separately for academic pediatricians, community pediatricians, and trainees. Many items asked respondents to rank first, second, and third choices from a list of possible outcomes. To calculate these rankings, we used weighted averages computed as follows: if a respondent ranked a particular choice first, then that choice was given 3 points; 2 points were given for a second choice and 1 point for a third choice, and all other choices were scored as 0. We then computed the average of all of these values for all respondents. To assess nonrandomness of ranking, we used Friedman's χ2 test to test for significant differences among choices marked by respondents (ie, whether a certain item was ranked above or below others after removal of variation among respondents). We tested the items in pairs, with the significance level adjusted for multiple comparisons. We tested for differences in response frequencies between pediatrician types and between study sites by using χ2 tests. We used linear regression analyses to examine the effects of gender, race, practice site, type of pediatrician, training related to diagnostic errors in medical school and residency, and years of clinical experience after residency on the outcome of the self-reported rate of diagnostic errors. We used a scale consisting of 7 possible frequencies; the 7 values were treated as a continuous variable. The potential covariates were examined individually for inclusion in the final model.
RESULTS
Between November 2008 and May 2009, a total of 1362 survey invitations were sent to physicians at the 3 study sites, including 569 at Site A, 194 at Site B and 599 at Site C. Invitees included 516 academic physicians, 430 trainees, and 416 community-based physicians. The overall response rate was 53%; group rates were 52% for academic physicians, 39% for community-based physicians, and 67% for trainees. However, not all respondents completed the survey in its entirety; the overall survey completion rate among respondents was 86%, and group completion rates were 87%, 88%, and 85%, respectively. The χ2 tests showed that there were no significant differences between physician types or between sites with respect to response rates.
Academic, community, and trainee groups did not differ with respect to racial/ethnic distributions (Table 1). Approximately one-fourth (27%) of respondents had been in practice for >10 years, and two-thirds (67%) performed mostly clinical duties. Approximately one-half (48%) of respondents received training about diagnostic errors in medical school and a somewhat greater number (59%) in residency; however, this training was largely informal in nature. The majority (81%) of respondents reported discussing with colleagues the diagnostic errors they had made, some, most, or all of the time.
Baseline Characteristics of Survey Respondents
Table 2 lists the average frequency rankings of clinical activities considered by pediatricians to be associated with error and potential for harm. The Friedman test showed significant differences among the ranked activities (P < .0001). Overall, pediatricians endorsed errors in medication-related activities such as prescribing and administering as being most prone to error and associated with the greatest potential for harm. Figure 1 shows the distributions (according to pediatrician category) of self-reported frequency of error (regardless of harm) and frequency of error with harm. More than one-half (54%) of respondents reported that they made a diagnostic error at least once or twice per month. However, the frequency of self-reported errors that might cause harm was lower. Only 4% of respondents reported that errors that harmed the patient occurred at least once or twice per month, 18% reported that such errors occurred at least once or twice per quarter, and 45% reported that they occurred at least once or twice per year.
Ranking of Clinical Activities Respondents Considered to Be Associated With Frequency of Error and Potential for Harm
Frequency of diagnostic errors regardless of patient harm (A) and frequency of diagnostic errors that caused patient harm (B), as self-reported by pediatricians. Percentages reported in the text are higher because they reflect cumulative frequencies.
In the regression model, nonwhite pediatricians had significantly lower self-perceived error rates and trainees had significantly higher reported rates than academic and community pediatricians. Neither gender nor number of years of experience after residency had significant effects on the self-reported diagnostic error rates. Collectively, provider characteristics explained a relatively small proportion of the variance (R2 = 0.17) in self-reported frequencies of diagnostic errors.
Diagnosis of viral illnesses as bacterial illnesses was the event most-frequently associated with diagnostic error, followed by misdiagnosis of medication side effects, psychiatric disorders, and appendicitis (Table 3). The Friedman test showed significant differences among the ranked diseases (P < .0001). Tables 4 and 5 display respondents' rankings of process breakdowns and contributory factors perceived to be associated with diagnostic errors. Among various types of process breakdowns associated with diagnostic errors, failure to gather available medical information had the highest ranking, followed by care not being sought in a timely manner by the patient or caregiver (Table 4). With regard to details of specific contributing factors, inadequate care coordination, teamwork, and/or communication across clinical settings or providers received the highest rating of all system-related factors (Table 5). Of all cognitive factors, inadequate data-gathering or work-up was ranked highest overall, although trainees ranked inadequate data assessment as a more-frequent contributing factor than data-gathering. Of miscellaneous factors leading to diagnostic errors, all groups assigned time/workload issues the highest average ranking. The Friedman test showed significant differences (P < .0001) among all ranked variables in Tables 4 and 5. We further explored the association of the factors of care not sought in a timely manner and heath literacy and found a strong positive correlation (P < .0005).
Ranking of Conditions Respondents Considered Most-Commonly Misdiagnosed in Pediatric Practice
Respondents' Ranking of Breakdowns in the Diagnostic Process Most-Commonly Associated With Diagnostic Errors
Respondents' Ranking of Most-Common Contributory Factors for Diagnostic Errors in Pediatrics
We also inquired about specific heuristic processes (important cognitive shortcuts in the face of complex situations) and biases that might affect medical decision-making. Overall, the type of bias with the highest average frequency rating was being too focused on a diagnosis or treatment plan. Another bias with a relatively high frequency rating was being misled by a normal history, physical, laboratory, or imaging study result.
Table 6 presents the ranking of selected solutions to reduce diagnostic errors. Of provider-based solutions, close follow-up of patients was ranked as being most likely to be effective by all 3 subgroups of pediatricians, followed by improving teamwork, more time in clinical encounters, and empowering patients and families to be vigilant about the possibility of diagnostic errors. For system-based solutions, access to electronic medical records that provide comprehensive clinical data was ranked highest by all 3 groups of pediatricians, followed by availability of diagnostic decision-support tools. Community-based physicians ranked increased access to and availability of consultants and experts second. The Friedman test showed significant differences among all ranked solutions (P < .0001).
Respondents' Ranking of Likely Effective Provider-Based and System-Based Solutions
DISCUSSION
We surveyed academic, community-based, and trainee pediatricians, at 3 study sites, about diagnostic errors and found several new insights that might facilitate understanding and prevention of such errors in the future. Of 6 types of medical errors, errors in diagnosis were ranked fourth in frequency and third in potential for harm. However, pediatricians reported that they made diagnostic errors rather frequently; more than one-half (54%) reported that they made a diagnostic error at least once or twice per month. Diagnostic errors that led to harm also were not infrequent; almost one-half (45%) of respondents reported that diagnostic errors that harmed patients occurred at least once or twice per year. The most-frequent diagnostic error was viral illnesses being diagnosed as bacterial illnesses, followed by misdiagnosis of medication side effects, psychiatric disorders, and appendicitis. Failures in data-gathering (history, examination, and chart review) and care delays by patients or caregivers were reported to be the most-frequent process breakdowns. Of various interventions for prevention of diagnostic errors, pediatricians gave highest rankings to close follow-up of patients and access to electronic medical records.
To our knowledge, this was the first study to assess diagnostic errors in any setting through a comprehensive survey. Our study builds on the knowledge available from malpractice literature to enhance our understanding of diagnostic errors that affect children, serving as a guide to the development of strategies to prevent these errors. According to our findings, pediatricians think that errors in diagnosis occur most commonly in a set of conditions that differ markedly from those mentioned in pediatric malpractice literature. For example, in pediatric malpractice literature, diagnoses of meningitis, appendicitis, pneumonia, and testicular torsion are cited as being missed most often.32,33 However, we found that most pediatricians thought that misdiagnosis of viral illnesses as bacterial illnesses was the most-common diagnostic error, followed by misdiagnosis of medication side effects and psychiatric diseases. None of these overlap with the top diagnostic errors found in claims files. Errors in diagnosis of medication-related side effects are relatively unexplored, in contrast to medication errors related to prescription and administration. A recent analysis of 583 physician-reported diagnostic errors in adults also revealed drug reactions or overdose as the second most-common type of diagnostic error.34 Similarly, the concept of potential misdiagnosis of psychiatric diseases in children, although not entirely new,35 has not been discussed in the context of malpractice claims and needs to be revisited with more specificity in future work. Another advantage of using this method is that pediatricians provided rich details about the most-common process breakdowns and contributory factors, some of which are much harder to determine from reviewing malpractice claims or medical records.30
We also solicited providers' rankings of proposed strategies to prevent diagnostic errors. Despite ongoing debate about the benefits of electronic health records,36 all groups of pediatricians ranked electronic records as the best system-based solution. Other strategies, such as diagnostic decision-support tools and techniques to ensure timely follow-up evaluation of certain patients,29,37 may need to be prioritized in research on preventive strategies. Pediatricians also thought that an important factor leading to diagnostic errors was the failure of patients or caregivers to seek care in a timely manner. Strategies to empower caregivers and patients should be studied and implemented in the context of patient safety research. Physicians often do not know the outcomes for patients whose diagnoses they miss; therefore, feedback pathways to relay changes in diagnosis back to the original physicians are likely to be useful.29
Our findings provide valuable data to inform ongoing efforts regarding patient safety and medical error training. The lack of emphasis on formal training regarding diagnostic errors provides opportunities to redesign curricula for both future pediatricians and practicing physicians.38,39 For instance, both academic pediatricians and trainees thought that failure to gather available medical information through history and physical examination and/or review of previous charts was a common breakdown. It may be valuable to refocus education on this much-neglected aspect of training through innovative techniques that use standardized or virtual patients and simulation.39,–,41 Training also should focus on the interplay of system-related and cognitive factors, which has been underemphasized in the literature to date. Critical thinking skills and strategies to reduce cognitive biases should be taught in the context of teamwork and system-related factors (such as time pressure) that may affect diagnostic performance.15,42,–,45 Carefully designed forums for open discussions of diagnostic errors are needed and are likely to be well received; many residents self-reported a large number of diagnostic errors, and most physicians reported that they discussed their own diagnostic errors with colleagues at least some of the time.
Our study had several limitations. Because we relied entirely on self-reported error data, rather than actual errors, and we could not determine which reported errors caused patient harm, our findings may need to be validated with other data sources. However, obtaining the perceptions of physicians (and other caregivers) about errors is important and is widely accepted as a fundamental approach to understanding and improving safety.46,47 The content and design of our survey were based largely on adult literature, although we incorporated a significant amount of feedback from pediatricians before administration. Pediatricians also may not know when they missed a diagnosis; therefore, errors may be underreported. Although perhaps our response rate was not high enough, we think this does not jeopardize the quality or generalizability of our findings. Literature findings showed that physician response rates were lower than rates for other participants and response rates have decreased over time.48,49 Published surveys of physicians have a mean response rate of only 54%, compared with 68% for nonphysicians.48 Lastly, we surveyed pediatricians only through e-mail, which might have limited participation by those lacking reliable e-mail access; however, the trainees and academic pediatricians in our sample were expected to use institutional e-mail accounts, which would limit this bias for this sample.
CONCLUSIONS
Pediatricians reported making diagnostic errors relatively frequently, and they endorsed inadequate data-gathering, poor care coordination, and patient/caregiver-related delays as prominent contributing factors. Improved follow-up of patients and access to electronic health records were perceived as the most promising potential interventions. In contrast to previous literature, our findings may be more generalizable to routine practice and may provide concrete targets for future training and interventions to prevent diagnostic errors in children.
ACKNOWLEDGMENTS
The study was supported by a National Institutes of Health Career Development Award (grant K23CA125585) to Dr Singh, the Houston Veterans Affairs Health Services Research and Development Center of Excellence (grant HFP90-020), and by a Fulbright and Jaworski Educational Award. These sources had no role in the design and conduct of the study; the collection, management, analysis, and interpretation of the data; and the preparation, review, and approval of the manuscript.
Footnotes
- Accepted February 25, 2010.
- Address correspondence to Geeta Singhal, MD, MEd, Baylor College of Medicine, Department of Pediatrics, 6621 Fannin St, Suite A.210, MC1-1481, Houston, TX 77030. E-mail: gsinghal{at}bcm.edu
The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
Funded by the National Institutes of Health (NIH).
REFERENCES
Synthetic Cell Created: Using man-made genetic instructions, scientists at the J. Craig Venter Institute recently published an article in Science Express (Gibson, DG, Glass JI, Lartigue C, et al. Creation of a bacterial cell controlled by a chemically synthesized genome. Science Express. May 20, 2010) describing their ability to create a synthetic one-cell organism that can reproduce, and thus allows man to manipulate nature. Needless to say the creation of this cell brings with it many ethical and legal questions as raised in a recent article in The Wall Street Journal (Hotz RL, May 21, 2010). David Magnus, director of the Stanford University Center for Biomedical Ethics said in the article that the synthetic cell “has the potential to transform genetic engineering” and that the research in this area “is going to explode” as a result. On the other hand, James Collins, a biomedical engineer at Boston University does not think this discovery represents the creation of an artificial life form. “I view this as an organism with a synthetic genome, not as a synthetic organism. It is tough to draw where the line is.” To no one's surprise, several companies are already trying to take advantage of the new field called synthetic biology as it combines chemistry, computer science, molecular and cellular biology and genetics to develop industrial life forms that can help make fuels, vaccines, or other commercial products.
Noted by JFL, MD
- Copyright © 2010 by the American Academy of Pediatrics
In this issue
Jump to section
Related Articles
Cited By...
- Increasing Physician Reporting of Diagnostic Learning Opportunities
- The global burden of diagnostic errors in primary care
- Patient Safety and Quality Improvement: Terminology
- Increased focus on recognizing, preventing diagnostic errors
- The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations
- Cognitive Bias in Inpatient Pediatrics
- The incidence of diagnostic error in medicine
- Advancing the research agenda for diagnostic error reduction
- Root Cause Analysis Reports Help Identify Common Factors In Delayed Diagnosis And Treatment Of Outpatients
- Use and Accuracy of Diagnostic Imaging by Hospital Type in Pediatric Appendicitis
- Challenges of making a diagnosis in the outpatient setting: a multi-site survey of primary care physicians
- Development of an evidence-based framework of factors contributing to patient safety incidents in hospital settings: a systematic review
- Electronic health record-based surveillance of diagnostic errors in primary care
- Exploring situational awareness in diagnostic errors in primary care