Preventing Provider Errors: Online Total Parenteral Nutrition Calculator
,
* Division of Neonatology, Department of Pediatrics, and Division of Health Sciences Informatics, Johns Hopkins University, Baltimore, Maryland
Division of Pediatric Pharmacy, Johns Hopkins Hospital, Baltimore, Maryland
Division of Pediatric Gastroenterology/Nutrition, Johns Hopkins Hospital, Baltimore, Maryland
 |
ABSTRACT |
|---|
 TOP ABSTRACT METHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
Objective. To describe the development of a pragmatic low-cost medical information system that reduces errors in the ordering of total parenteral nutrition (TPN) in the newborn intensive care unit at the Johns Hopkins Hospital.
Methods. We designed an online total parenteral nutrition order entry system (TPNCalculator) using Internet technologies. Utilization, impact on medical errors, and user satisfaction were evaluated prior to and immediately after introduction of TPNCalculator (intervention 1) and after 2 years (intervention 2).
Results. Total software development time was 3 weeks. The number of orders was similar during the 3 periods: 0.39 orders per patient per day (N = 557) were received compared with 0.35 and 0.43 orders per patient per day (N = 471 and N = 656) in 2 intervention periods. During the control period, an average of 10.8 errors were detected per 100 TPN orders compared with 4.2 per 100 orders in the first intervention period (61% reduction of error rate) and 1.2 per 100 orders after 2 years and some redesign of TPNCalculator (89% reduction of error rate). We found a reduction in the following types of problems (intervention 1; intervention 2): calculation errors (100%; 100%), osmolality outside the allowed range (88%; 91%), and other knowledge problems (84%; 100%). There was a 35% increase in the number of incomplete forms in the first intervention period and a 100% reduction in the second. Rapid cycle development was used in the development of this application. Users of the system were enthusiastic and supportive and compared it favorably to the previous paper-based system.
Conclusion. Low-cost, pragmatic approaches using Internet technology in the design of medical information systems can reduce medical errors and might pose a viable option for the prevention of adverse drug events.
Key Words: medication errors/classification • medication systems • hospital/organization and administration • software design • medical informatics computing
Abbreviations: ADE, adverse drug event • NICU, neonatal intensive care unit • TPN, total parenteral nutrition • JHH, Johns Hopkins Hospital
According to an estimate by the Institute of Medicine in 1999, >1 million injuries and almost 100 000 deaths may be attributed to medical errors annually.1 Most errors that occur in the prescription, dispensing, and administration of medications could have been prevented2 by redesign of the systems used to deliver medications to patients. Practical interventions that attempt to change system processes rather than people were found to be most successful in the prevention of adverse drug events (ADEs).3 Unfortunately, the underlying system failures are rarely identified and corrected,4 so that physicians, pharmacists, and nurses are often unwitting participants in the recurrence of a well-known error. The rate for potential ADE is 3 times higher in children than adults and substantially higher still for neonates in the neonatal intensive care unit (NICU).5
Adequate nutritional support for premature or sick neonates requires the daily planning, calculating, and ordering of parenteral nutrition.6 The ordering of parenteral nutrition is associated with a high incidence of medical errors and a significant potential for patient harm.7–10 and is very time-consuming (
10 minutes per patient).6
By focusing on this known problem area, the ordering of parenteral nutrition, we aimed to develop a low-budget medical information system (using an existing model and quality assurance process), evaluate its effectiveness in error reduction, and assess its acceptance by the ordering providers. We hypothesized that a pragmatic and inexpensive approach to ADEs could be found by correcting system failures with the help of Web technology.
|
METHODS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
Using a model of failure mode and effects analysis,11 we had identified the completion of the paper order form as the main source of errors. Using the paper form as a model, we designed an online total parenteral nutrition (TPN) order entry system (TPNCalculator) in 2000 and introduced it in the NICU at the Johns Hopkins Hospital (JHH) as a quality improvement instrument. TPNCalculator was created using a routine web development tool (Macromedia Cold Fusion). Its interface was designed to resemble closely the TPN order form that had been in use for >10 years at JHH and that was replaced by TPNCalculator. This allowed TPNCalculator to tap into the existing order system and quality improvement process. It also reduced the risk for potential system integration issues. Using a participant/observer as the main programmer eliminated the need for a specification process and reduced total development and testing time to
3 weeks.
Process
Residents received a brief 10-minute training before the initial use of the TPNCalculator. Through a public workstation or a desktop computer, the user (resident, nurse practitioner, pharmacist, nutritionist) accesses the TPNCalculator using individual log-in identification and password. All interactions between the TPNCalculator Server and the user’s browser are 128-bit encrypted to be compliant with the Health Insurance Portability and Accountability Act.
The user may select to enter a TPN Order for a new patient or to modify/renew an old order for an existing patient. TPNCalculator performs all necessary fluid and component calculations on the basis of provider input via an Internet browser. It contains nutritional guidelines, an osmolality calculator, and 62 rule-based alerts and reminders (Fig 1). These alerts and reminders include a check for age/weight ratio (if the child is outside the normal weight range for the calculated age, then the user is prompted), dose range alerts based on total dose, dose per weight, concentration (as indicated), and percentage of total calories (as indicated) for the included components such as glucose, lipids, protein, electrolytes, heparin, and ranitidine. Other features include automatic selection of the appropriate protein additive based on age and an osmolality alert, which is triggered only in patients without a central line. Users are reminded to add phosphorus and calcium if they have not done so and to add heparin for central line TPN. Reminders can be complex based on previous user selections; for example, if a user chooses not to order trace elements, then not only is he or she prompted to add selenium, zinc, and chromium, but also the dose ranges of these additives will now be checked. Users are prevented to submit an order unless it contains patient name and medical record number, physician name, and identification number. Other data entered by the user are validated against a possible range (eg, infusion time cannot exceed 24 hours). Patient data are automatically populated using a link to the hospital admission, discharge, and transfer system.
|
Data collected during a designated quality improvement project included the number of parenteral nutrition orders and the frequency and type of errors as identified by the pharmacist. Data were collected during a control period (October 2, 2000, to November 14, 2000) immediately before and during 2 intervention periods after (intervention 1: November 15, 2000, to December 31, 2000; intervention 2: August 27, 2002, to October 13, 2002) the implementation of TPNCalculator. Data analysis was done with
2 test. Prescribers, pharmacists, and nurses were surveyed online on their experiences and opinions in the course of using TPNCalculator. |
RESULTS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
During the control period, average patient census was 32.3. During this time, 557 TPN orders were written compared with 471 orders and 656 orders in the 2 intervention periods (average census: 30.2 and 32.4). The number of TPN orders per patient per day was not significantly different between the time periods (control: 0.39; intervention 1: 0.35; intervention 2: 0.43 orders per patient per day). During the control and the first intervention periods, only half of the orders were available to the pharmacy by order deadline at noon (51.7% control, 48.6% intervention). Prescribers most commonly cited lack of time as the cause for the delay (86%). During the first intervention period, lack of time (54%) and reduced staffing during the holidays (31%) were given as a frequent cause of delays.
During the control period, a total of 60 errors (10.8 errors per 100 orders) that required the pharmacist to contact the provider were detected compared with only 20 errors (4.2 errors per 100 orders) in the first and 8 errors (1.2 errors per 100 orders) in the second intervention period. This represents a 61% reduction in error rate (P < .01) in the first and an 89% reduction in error rate (P < .001) in the second intervention period.
Error type distribution was significantly different among the 3 groups (Fig 2). During the control period use of the paper form, prescribers were significantly more likely to order fluids that exceeded the osmolality guidelines, make calculation errors, and generate orders that demonstrated other knowledge deficiencies (eg, no heparin ordered for central line TPN). No statistical difference was noted in the number of orders with insufficient fluid amounts to allow all additives to be dissolved between the control period and both intervention periods, although there was a trend toward a reduction in intervention 2. There was an increase from 1.7 incomplete orders per 100 TPN orders in the control period to 2.3 in the intervention period (not significant). Omission of the page number on the order form (order identifier) contributed to 92% of all incomplete orders in the intervention period (Fig 3). In the second intervention period, there was a 100% reduction of omissions (P < .001).
|
|
After a 6-month period, when all stakeholders had an opportunity to interact with the TPNCalculator, 84 providers, nurses, and pharmacists were surveyed using an online questionnaire. Twenty-eight prescribers responded, including 22 postgraduate year 1 residents (100% of interns) and 6 neonatal nurse practitioners (100% of neonatal nurse practitioners). Seven percent had used TPNCalculator <5 times per week, 21% used it 5 to 9 times, 36% used it 10 to 19 times, and 36% used it >20 times per week. On a scale from 1 (very easy) to 5 (very difficult), TPNCalculator was rated 1.5. On a Likert scale (1 = strongly agree, 3 = neutral, 5 = strongly disagree), prescribers were asked to compare the TPNCalculator with the paper form (Table 1).
|
In comparison with the paper form, TPNCalculator was found to be easier to learn and to use. Prescribers believed that it protected against errors, saved time, was helpful, and constituted an improvement. Users were neutral to the statement that the TPNCalculator provided a better learning experience.
As part of the survey, prescribers were asked about the series of potential problems associated with an ordering tool. Prescribers were neutral toward the statements that TPNCalculator carried the risk of inaccurate programming (3.3), had limited scope and was not designed to handle all clinical cases (3.0), had potentially incomplete information/missing data (3.0), redirected provider priorities (3.6), and generated false expectation such as being alerted to all problems (3.0). Prescribers disagreed that TPNCalculator caused data overload in the user (4.1).
Using separate online questionnaires, we also surveyed 8 pediatric pharmacists (73% of pharmacists) and 43 neonatal nurses (42% of nurses). Comparing TPNCalculator and the paper form on a Likert scale from 1 (strongly agree) to 5 (strongly disagree), pharmacists and nurses agreed that TPNCalculator was easier to read, was easier to learn, and resulted in less need for order clarification. Pharmacists reported that TPNCalculator resulted in fewer errors, saved time, and was helpful for data entry (Table 2).
|
|
DISCUSSION |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
ADEs are common, affecting as many as 1 in every 25 hospitalized patients.12 The ADE Prevention Study Group reported that few risk factors are patient related, suggesting that rather than target ADE-prone individuals, prevention strategies should focus on improving medication systems.13
After identifying a medication ordering process with significant risk for ADEs, we designed, implemented, and evaluated a web-based medical information system (TPNCalculator) to improve the ordering process and reduce ADEs. We hypothesized that underlying system failures could be corrected with inexpensive, pragmatic problem-solving approaches. A deliberate attempt was made to address only the "broken" link, thereby preserving the majority of the existing ordering process. Furthermore, we based the interface design on existing paper forms to provide users with a familiar format and maximum comfort level. This minimized required training (usually <10 minutes) and permitted the existing order-processing flow to remain intact. The use of participant/observers (prescribers and pharmacists) as programmers, designers, and testers eliminated the need for extensive life cycle assessment (systematic set of procedures for compiling and examining the inputs and outputs of materials and energy and the associated environmental impacts directly attributable to the functioning of a product or service system throughout its life cycle), including evaluation and analysis of usage, resource utilization, patient care implication, and failure modes.11 Using this approach, the complete design process (1 Cold Fusion programmer [C.U.L.]) and quality assurance (1 nutritionist [J.M.C.] and 1 pharmacist [K.G.C.]) took 3 weeks (200 man-hours), which starkly contrasts with other ongoing clinical medical informatics projects at JHH. Modifications of TPNCalculator require minimal intervention, because any change made on the web server is immediately available to all users throughout the system.
In this study, we were able to use a close-held, multitalented, small development group to develop a rapid solution in cooperation with local information services. This model was highly effective in creating utilities for in-house support of clinicians and has in the meantime been duplicated in several other JHH projects. It must be pointed out that a development model such as ours is frequently not available to those outside the clinical process. Reasons for this may include Health Insurance Portability and Accountability Act concerns, institutional policies and interests, lack of access to data from legacy systems, and the inability to assemble a small team with all of the required skills.
Evaluation of error rates and user surveys revealed an overwhelming success. We reduced the total number of errors by 89%. Calculation errors were eliminated through TPNCalculator, and knowledge problems drastically reduced. Users of the system (including those who were involved only peripherally, eg, nurses) were enthusiastic and supportive and compared it favorably with the previous paper-based system. We experienced significant pressure to make TPNCalculator available to all units in the children’s hospital and to an affiliated hospital NICU once residents had used it during their NICU rotation. As a result, 6 months after the development of TPNCalculator, it was deployed throughout the JHH Children’s Medical & Surgical Center.
In our survey, users did not estimate the risks of TPNCalculator to be higher than in the paper system; however, experience14 has shown that the deployment of a new software tool may introduce a new type of failure mode in the form of programming/software errors. An analysis of 3140 medical device recalls conducted by the Food and Drug Administration between 1992 and 1998 revealed that 242 (7.7%) were attributable to software failures.15 Of those, 192 (79%) were caused by software defects that were introduced when changes were made to the software after its initial production and distribution. Although TPNCalculator corrected a number of system problems, in its initial version, it had its own system failure: a drastic increase in the TPN orders without order identification (page number). Once this (small and insignificant for patient safety) problem was identified, TPNCalculator was first modified to prompt the user for a page number, and in a later version, an automatic page number creation rule was added. Data from the second intervention period demonstrated that omissions had been eliminated without adding new errors. The example of the "missing page number" emphasizes that rapid cycle evaluation is able to handle software design problems through frequent series of design, evaluation, and redesign. An attempt to design a "perfect" and completely error-free application from the start would have required considerable planning, consulting, and review before implementation, leading to long cycle times, which might have derailed the process long before completion. In addition, the desire to design the "perfect" system might have resulted in an overengineered product with additional features, complicating its use and making it unattractive to users.
Federal guidelines require that medical software systems become subject to validation requirements (Title 21 Code of Federal Regulations 11.10[a]). Such systems must be validated to ensure accuracy, reliability, and consistent intended performance. Reducing total development time to 3 weeks inherently means that validation features such as requirement and feature lists are mainly implicit in the design and not necessarily expressed in detail. In our case, this was possible because the software was modeled on an existing paper form in use at JHH. The paper form implicitly determined system inputs and outputs, functionality, interface definition, user interactions, safe ranges, limits, and default values. Using domain experts for the testing allowed for testing of all potential clinical scenarios using patient data available during the development period. The robustness of the system was tested in a clinical environment by allowing users to use TPNCalculator while for a period of 3 months all calculations were duplicated in the pharmacy to ensure safety. During that period, users generated >1800 orders, which spanned the complete range of possible inputs. The existing paper order form served as a safety back-up but was never used.
Developing TPNCalculator would not have been possible without certain information infrastructure already in place at JHH. We relied on the availability of public workstations in all clinical areas where providers might order TPN. If access to computers would have required traveling (even a short distance) or a waiting period, then we suspect that users’ enthusiasm for this application would have been drastically diminished. Furthermore, the decision by JHH management to provide an Internet browser on all workstations was crucial to implementation. Allowing Internet browsing on public clinical workstations is not a universal practice, as demonstrated by differences even among the hospitals within the Johns Hopkins Health System.
We have demonstrated that pragmatic approaches to ADEs involving simple medical information systems can be successful in both reducing the number of errors and increasing user satisfaction. The approach used in the development of TPNCalculator, which could be called "find it, fix it, and forget it," identifies a specific problem and uses existing infrastructure to develop a pragmatic medical informatics solution to the problem while leaving the remaining system intact. By limiting the number of people involved in the process of problem identification, development, testing, and deployment, the used resources such as time and manpower can be drastically reduced without losing effectiveness. By shortening the time interval between the birth of an idea and its implementation, we increased the speed of innovation. Although this approach carries a greater risk for software design flaws, in our opinion, this risk can be minimized by using participant/observers in the development and is further offset by the significant gains through early implementation and cost reduction.
|
CONCLUSION |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
Low-cost, pragmatic approaches using Internet technology in the design of medical information systems might pose a viable alternative for the prevention of ADEs.
|
ACKNOWLEDGMENTS |
|---|
This work was supported by the Center for Innovation and Quality Patient Care at the Johns Hopkins Hospital.
We are grateful to Harold P. Lehmann, MD, PhD, for critical review of this manuscript.
|
FOOTNOTES |
|---|
Received for publication Jan 21, 2003; Accepted Jul 25, 2003.
Reprint requests to (C.U.L.) Eudowood Neonatal Pulmonary Division & Division of Health Sciences Informatics, Johns Hopkins University, 600 N Wolfe St, CMSC 210, Baltimore, MD 21287-3200. E-mail: clehmann{at}jhmi.edu
|
REFERENCES |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
- Kohn LT, Corrigan JM, Donaldson MS, eds. To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press; 1999
- Leape LL, Bates DW, Cullen DJ, et al. Systems analysis of adverse drug events. ADE Prevention Study Group.
JAMA.1995; 274
:35
–43
[Abstract/Free Full Text] - Leape LL, Kabcenell AI, Gandhi TK, Carver P, Nolan TW, Berwick DM. Reducing adverse drug events: lessons from a breakthrough series collaborative. Jt Comm J Qual Improv.2000; 26; 321 –331[Medline]
- Leape LL. Error in medicine.
JAMA.1994; 272
:1851
–1857
[Abstract/Free Full Text] - Kaushal R, Bates DW, Landrigan C, et al. Medication errors and adverse drug events in pediatric inpatients.
JAMA.2001; 285
:2114
–2120
[Abstract/Free Full Text] - Horn W, Popow C, Miksch S, Kirchner L, Seyfang A. Development and evaluation of VIE-PNN, a knowledge-based system for calculating the parenteral nutrition of newborn infants. Artif Intell Med.2002; 24 :217 –228[CrossRef][Web of Science][Medline]
- Hermansen MC, Kahler R, Kahler B. Data entry errors in computerized nutritional calculations. J Pediatr.1986; 109 :91 –93[Medline]
- Legler JD. Physicians’ accuracy in manual computation. MD Comput.1990; 7 :155 –159[Medline]
- Tepas JJ 3rd, Mollitt DL, String DL, Pieper P. Error in fluid and calorie calculation in the surgical neonate. J Pediatr Surg.1991; 26 :132 –134[Medline]
- Vincer MJ, Murray JM, Yuill A, Allen AC, Evans JR, Stinson DA. Drug errors and incidents in a neonatal intensive care unit. A quality assurance activity.
Am J Dis Child.1989; 143
:737
–740
[Abstract/Free Full Text] - Cohen MR, Senders J, Davis NM. Failure mode and effects analysis: a novel approach to avoiding dangerous medication errors and accidents. Hosp Pharm.1994; 29 :319 –330[Medline]
- Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med.1991; 324 :370 –376[Abstract]
- Bates DW, Miller EB, Cullen DJ, et al. Patient risk factors for adverse drug events in hospitalized patients. ADE Prevention Study Group.
Arch Intern Med.1999; 159
:2553
–2560
[Abstract/Free Full Text] - McDaniel JG. Improving system quality through software evaluation. Comput Biol Med.2002; 32 :127 –140[CrossRef][Web of Science][Medline]
- US Department Of Health and Human Services Food and Drug Administration. General Principles of Software Validation; Final Guidance for Industry and FDA Staff. Available at: www.fda.gov/cdrh/comp/guidance/938.html
PEDIATRICS (ISSN 1098-4275). ©2004 by the American Academy of Pediatrics
CiteULike 
Connotea 
Del.icio.us 
Digg 
Facebook 
Reddit 
Technorati 
Twitter What's this?
This article has been cited by other articles:
|
|
 |
|
  G. R. Kim, C. U. Lehmann, and and the Council on Clinical Information Technology Pediatric Aspects of Inpatient Health Information Technology Systems Pediatrics, December 1, 2008; 122(6): e1287 - e1296. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C Snijders, R A van Lingen, A Molendijk, and W P F Fetter Incidents and errors in neonatal intensive care: a review of the literature Arch. Dis. Child. Fetal Neonatal Ed., September 1, 2007; 92(5): F391 - F398. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. S. Gerstle, C. U. Lehmann, and the Council on Clinical Information Technology Electronic Prescribing Systems in Pediatrics: The Rationale and Functionality Requirements Pediatrics, June 1, 2007; 119(6): e1413 - e1422. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. R Miller, K. A Robinson, L. H Lubomski, M. L Rinke, and P. J Pronovost Medication errors in paediatric care: a systematic review of epidemiology and an evaluation of evidence supporting reduction strategy recommendations Qual. Saf. Health Care, April 1, 2007; 16(2): 116 - 126. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. E. Walsh, W. G. Adams, H. Bauchner, R. J. Vinci, J. B. Chessare, M. R. Cooper, P. M. Hebert, E. G. Schainker, and C. P. Landrigan Medication Errors Related to Computerized Order Entry for Children Pediatrics, November 1, 2006; 118(5): 1872 - 1879. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. A. Ghaleb, N. Barber, B. D Franklin, V. W. Yeung, Z. F Khaki, and I. C. Wong Systematic Review of Medication Errors in Pediatric Patients Ann. Pharmacother., October 1, 2006; 40(10): 1766 - 1776. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M R Miller, J S Clark, and C U Lehmann Computer based medication error reporting: insights and implications. Qual. Saf. Health Care, June 1, 2006; 15(3): 208 - 213. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 G. R. Kim, A. R. Chen, R. J. Arceci, S. H. Mitchell, K. M. Kokoszka, D. Daniel, and C. U. Lehmann Error Reduction in Pediatric Chemotherapy: Computerized Order Entry and Failure Modes and Effects Analysis Arch Pediatr Adolesc Med, May 1, 2006; 160(5): 495 - 498. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. T. Costakos Of Lobsters, Electronic Medical Records, and Neonatal Total Parenteral Nutrition Pediatrics, February 1, 2006; 117(2): e328 - e332. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R Ursprung, J E Gray, W H Edwards, J D Horbar, J Nickerson, P Plsek, P H Shiono, G K Suresh, and D A Goldmann Real time patient safety audits: improving safety every day Qual. Saf. Health Care, August 1, 2005; 14(4): 284 - 289. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 K E Walsh, R Kaushal, and J B Chessare How to avoid paediatric medication errors: a user's guide to the literature Arch. Dis. Child., July 1, 2005; 90(7): 698 - 702. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. U. Lehmann, G. R. Kim, and M. R. Miller EHRs can help drive implementation of evidence-based guidelines AAP News, October 1, 2004; 25(4): 189 - 189. [Full Text] [PDF]
|
|
eLetters:
Read all eLetters
- Electronic TPN calculators: Why aren’t we there yet?
- Dennis Costakos
- Pediatrics Online, 2 Jun 2005 [Full text]
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
