
,
,
,
,
,||

,
* Departments of Critical Care Medicine
Pediatrics
|| Clinical Research, Investigation, and Systems Modeling in Acute Illness (CRISMA) Laboratory, Department of Critical Care Medicine, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania
Department of Critical Care Medicine/Transport, Children's Hospital of Pittsburgh, Pittsburgh, Pennsylvania
| ABSTRACT |
|---|
|
|
|---|
Methods. Demographic, clinical, and mortality data were collected of all children who were admitted via interfacility transport to our regional, academic, tertiary-care level childrens hospital during an 18-month period. A commercially sold CPOE program that operated within the framework of a general, medical-surgical clinical application platform was rapidly implemented hospital-wide over 6 days during this period. Retrospective analyses of pre-CPOE and post-CPOE implementation time periods (13 months before and 5 months after CPOE implementation) were subsequently performed.
Results. Among 1942 children who were referred and admitted for specialized care during the study period, 75 died, accounting for an overall mortality rate of 3.86%. Univariate analysis revealed that mortality rate significantly increased from 2.80% (39 of 1394) before CPOE implementation to 6.57% (36 of 548) after CPOE implementation. Multivariate analysis revealed that CPOE remained independently associated with increased odds of mortality (odds ratio: 3.28; 95% confidence interval: 1.945.55) after adjustment for other mortality covariables.
Conclusions. We have observed an unexpected increase in mortality coincident with CPOE implementation. Although CPOE technology holds great promise as a tool to reduce human error during health care delivery, our unanticipated finding suggests that when implementing CPOE systems, institutions should continue to evaluate mortality effects, in addition to medication error rates, for children who are dependent on time-sensitive therapies.
Key Words: administration computer software health care delivery/access interhospital transport outcome
Abbreviations: CPOE, computerized physician order entry CHP, Children's Hospital of Pittsburgh ADE, adverse drug event PRISM, Pediatric Risk of Mortality OR, odds ratio CI, confidence interval
In their landmark report To Err is Human: Building a Safer Health System, members of the Institute of Medicine estimated that medical errors contributed to between 44000 and 98000 deaths annually in the United States.1 As a result of this report, subsequent congressional hearings, and extensive media exposure, the issue of patient safety has quickly risen to a position of highest priority among many health care organizations. Sparked by this "safety initiative," many hospitals have looked toward emerging medical information technologies, specifically computerized physician order entry (CPOE) systems, as a potential tool to reduce human error during health care delivery.
Founded by The Business Roundtable, a national association of Fortune 500 CEOs who are committed to improving public policy, the Leapfrog Group (www.leapfroggroup.org) has embraced CPOE, citing its beneficial role in reducing medication error2 as well as improving hospital resource utilization.3 With patient safety as its stated mission focus, the Leapfrog Group now actively promotes widespread CPOE implementation as 1 of its 4 benchmarks for patient safety standards.
In response to the Institute of Medicine's report and safety initiatives promoted by the Leapfrog Group, the Children's Hospital of Pittsburgh (CHP) implemented hospital-wide a commercially sold CPOE system in October 2002 to become 1 of the first children's hospitals in the United States to attain 100% CPOE status. Upperman et al4 recently reported that consistent with the experience at many other institutions, CPOE implementation at our hospital resulted in significant reductions in harmful adverse drug events (ADEs) during a 9-month study period.
However, despite CPOE's ability to reduce medication error rates, a few investigators have begun to question whether CPOE implementation necessarily results in improved patient outcome and have raised concerns regarding the Leapfrog Group's CPOE directive.5 Some have proposed that under certain circumstances, CPOE may actually foster "unintended consequences,"6 a concept recently supported by a study that described the role of CPOE in facilitating medication error risks through "systems integration failure" and "human-machine interface flaws."7 In light of reemerging uncertainty and discussion regarding the impact that CPOE might have on patient outcome, we examined mortality rates among children who were admitted via interfacility transport before and after CPOE implementation, testing the hypothesis that patient outcome would improve after this intervention.
| METHODS |
|---|
|
|
|---|
12000 annual admissions (including
3000 annual ICU admissions) and
60000 patient-days. We retrospectively examined demographic, clinical, and mortality data, extracted in accordance to Health Insurance Portability and Accountability Act regulations, for all children who were admitted to CHP via interfacility transport for specialized, tertiary-level care during an 18-month period from October 1, 2001, to March 31, 2003, using CHP's Critical Care Transport Team interfacility transport database. We chose to study this patient population because they represented a "first encounter" cohort of patients to the hospital system, requiring immediate processing of admission and stabilization orders. Severity of illness for each patient was assessed by a Pediatric Risk of Mortality (PRISM) score, which then was used to calculate cumulative, predicted mortality rates for the study population.8 Changes to health care team dynamics and the manner by which bedside care was delivered were additionally noted post hoc.
CPOE System
The CPOE system (PowerOrders; Cerner [a member of The Leapfrog Group], Kansas City, MO) that was purchased by CHP is a commercially sold "add-in" application module that operates within the software architecture of a medical information technology clinical applications platform developed by the same vendor (Millennium; Cerner). Additional modules may be integrated into the platform as they are developed and become commercially available and according to the specific needs of a particular institution.
Approximately 3 months before CPOE implementation, all hospital health care personnel were trained through a mandatory 3-hour computer tutorial and practice session. Hospital-wide implementation of CHP's CPOE system (along with its clinical applications platform) occurred over a 6-day period, reaching full operation by October 29, 2002. Designated CPOE experts were present to provide "hands-on" consultation support during the immediate postimplementation period, after which support was reduced to telephone consultation. This CPOE program provides physician "point-of-care" and decision support with alerts and reminders regarding potential drugdrug, drugallergy, and drugfood interactions in addition to potential medication errors. Physician orders are entered primarily through selecting from various order "menus" and "sub-menus" that require completion of requisite fields before orders are accepted. For example, to place an order for "cefotaxime 500 mg iv q6hr x 7 days," the physician begins by first securely logging into the CPOE system at an open computer terminal, identifies the intended patient from the patient menu, opens the order window and chooses medications, and then selects (or types) cefotaxime from the orderable search menu. Confirmation of this selection then opens a series of sub-menus that request specific fields to be filled: the dose (500), the dosing unit (mg), the route of administration (iv), the dosing frequency (q6hr), the duration (7), and the duration unit (days). Incomplete order entry fields prompt the physician to fill the missing fields before continuation of the order. After all requisite fields have been entered, the order is processed for decision support, point-of-care analysis, and potential medication errors, after which the physician is requested to confirm, override, modify, or cancel the order. The ordering of continuous intravenous infusions, respiratory therapies, laboratory studies, radiographic studies, and other clinical directives proceeds in a similar manner. All new medication orders require activation by the nurse before the pharmacist receives the actual order for processing. To facilitate the order entry process, this CPOE software program can be modified so that repetitive or frequently executed order algorithms can be saved as preprogrammed favorites, and multiple-order algorithms can be bundled into preprogrammed "order sets" that come with default selections for requisite order fields. However, no ICU-specific order sets had been programmed at the time of CPOE implementation but instead were developed over time after CPOE implementation.
Statistical Analysis
Differences between groups (before vs after CPOE implementation and survivor vs nonsurvivor) were determined by Mann-Whitney rank sum test for continuous data and by
2 or Fisher's exact tests for categorical data. Differences between observed and predicted mortality rates were determined by z statistics. To determine which factors might be independently associated with mortality, all variables whose P values were <.25 in univariate analysis were entered into a stepwise logistic regression model that also accounted for significant interactions between variables. Because incorporation of PRISM score with some of its component variables in the same model might create potential collinearity, 2 separate models, with and without the PRISM score, were fit to address this possibility. Mortality odds ratios (ORs) with 95% confidence intervals (CIs) were calculated using standard mathematical formulas. Data were analyzed using SPSS statistical software program (version 12.0; Chicago, IL).
| RESULTS |
|---|
|
|
|---|
10% of the annual admissions for the period. Demographic and clinical characteristics for this study population are shown in Table 1. The median age of these patients was 9 months, and 55.7% were male. The most common clinical conditions for admission were airway/respiratory (42.6%), infectious disease (34.9%), and central nervous system/neuromuscular (19.4%) related. Reflecting the tertiary-care referral nature of this transport population, 1102 (56.7%) children initially were admitted to an ICU, which represented
25% of the annual ICU admissions for the period.
|
2).
Overall, 75 children died during the study period, accounting for an unadjusted mortality rate of 3.86%. Unadjusted mortality rate, however, increased from 2.80% (39 of 1394) before CPOE implementation to 6.57% (36 of 548) after CPOE implementation (P < .001,
2). Observed mortality was consistently better than predicted mortality before CPOE implementation, but this association did not remain after CPOE implementation (Fig 1). Demographic and clinical characteristics for survivors and nonsurvivors are shown in Table 2. Nonsurvivors were more likely to have been younger, been premature, been admitted directly to an ICU, had higher severity-of-illness scores, or been referred for surgery. Nonsurvivors were also more likely to have had severe coma, a cardiovascular-related condition, shock, or a congenital/genetic-related condition or been referred for extracorporeal membrane oxygenation support. Nonsurvivors were less likely to have had an infectious diseaserelated condition or metabolic/renal/ingestion-related condition.
|
|
|
1 to 2 minutes per single order as compared with a few seconds previously needed to place the same order by written form. Because the vast majority of computer terminals were linked to the hospital computer system via wireless signal, communication bandwidth was often exceeded during peak operational periods, which created additional delays between each click on the computer mouse. Sometimes the computer screen seemed "frozen." This initial time burden seemed to change the organization of bedside care. Before CPOE implementation, physicians and nurses converged at the patient's bedside to stabilize the patient. After CPOE implementation, while 1 physician continued to direct medical management, a second physician was often needed solely to enter orders into the computer during the first 15 minutes to 1 hour if a patient arrived in extremis. Downstream from order entry, bedside nurses were no longer allowed to grab critical medications from a satellite medication dispenser located in the ICU because as part of CPOE implementation, all medications, including vasoactive agents and antibiotics, became centrally located within the pharmacy department. The priority to fill a medication order was assigned by the pharmacy department's algorithm. Furthermore, because pharmacy could not process medication orders until they had been activated, ICU nurses also spent significant amounts of time at a separate computer terminal and away from the bedside. When the pharmacist accessed the patient CPOE to process an order, the physician and the nurse were "locked out," further delaying additional order entry.
Before CPOE implementation, the physician expressed an intended order either through direct oral communication or by writing it at the patient's bedside (often reinforced with direct oral communication), with the latter giving the nurse a visual cue that a new order had been placed. The nurse had the opportunity to provide immediate feedback, which sometimes resulted in a necessary revision of that order. In addition, these face-to-face interactions often fostered discussions that were relevant to patient care and management. After CPOE implementation, because order entry and activation occurred through a computer interface, often separated by several bed spaces or separate ICU pods, the opportunities for such face-to-face physiciannurse communication were diminished.
| DISCUSSION |
|---|
|
|
|---|
Can CPOE Implementation Affect Bedside Care and Delivery of Time-Sensitive Therapies?
We have described a few examples of the changes that occurred after CPOE implementation in the manner by which critically ill children who were admitted through our transport system were resuscitated and stabilized. Although order delays related to the inability to "preregister" patients into the system have been resolved through CPOE programming modifications, other matters remain more problematic to address. It has been shown that additional time is needed to enter orders through CPOE as compared with written form, although some of this "lost time" may be recovered later through improved overall efficiency.15,16 We also observed the need to spend additional time upfront to enter orders through CPOE as compared with written form. In general medical-surgical wards, this "upfront time cost" may have little consequence. However, in the transport/ICU setting, where multiple, rapid-fire interventions are regularly performed, this upfront time cost might have significant patient care consequences. For some critical conditions, including shock, patient survival has been shown to be time-sensitive and dependent on successful, early resuscitation.17,18 The ongoing development of preprogrammed order sets has helped to reduce some of the upfront time cost of order entry, but it still has not eliminated the need for a second physician to be devoted solely to enter orders on the arrival of a critically ill child. Slightly downstream from order entry, nurses must continue to spend significant amounts of time at the computer terminal and away from the bedside, effectively reducing staff-to-patient ratios during this critical period. Adult and pediatric studies have consistently reported that reduced staff-to-patient ratios can have an adverse impact on outcome, particularly in patients with shock.19
We noted several changes to health care team dynamics and the manner by which bedside care was delivered to our patients after CPOE implementation. The interactions between ICU team members have remained fundamentally altered. Delays in the administration of critical medication resulting from complete centralization of pharmacy services as a consequence of CPOE implementation also remain. Before CPOE implementation, antibiotics and vasoactive drugs were administered according to national guideline-recommended timelines20,21; however, after CPOE implementation, we have found that fewer than half of the patients received critical antibiotics and vasoactive infusions within these timelines. In this regard, we continue to investigate alternative methods to reduce the time from order entry to initiation of antibiotic and vasoactive infusion therapy.
Can "Unintended Consequences" Manifest During Systems Integration?
In recent papers by Ash et al6 and Aarts et al,22 the authors advanced the concept that "unintended consequences"6 or "unpredictable outcomes"22 are inherently possible with any emergent change. Implementation of information systems, such as CPOE, is "typified by contingencies and proceed in a far from linear manner. They are part and parcel of organizational dynamics that, as a result of the complexity of the organizations of which we speak, cannot be foreseen, let alone be predicted."22 In this regard, unpredicted things did happen. For example, it was discovered that with antibiotic administration, subsequent dosing schedules were not timed according to the time of initial dose administration but rather at predetermined default times. Hence, children sometimes received the first 2 doses of an antibiotic in an unacceptably brief time interval. At the back end of antibiotic administration, default "stop order" mechanisms sometimes terminated standing antibiotic orders without physician notification or knowledge.
In a review that addressed the benefits, costs, and issues regarding CPOE, Kuperman and Gibson cautioned,23 "Computerized physician order entry is a complex undertaking and should not be the first computerized clinical system attempted by an organization. A CPOE application is more likely to be accepted if the existing clinical systems are well received." The implementation of CPOE in our hospital occurred concurrently with the implementation of its clinical applications platform. Given this simultaneous implementation, it is possible that our unanticipated finding may not have been a result of CPOE but rather the clinical application platform on which it operates. This general, medical-surgical clinical application platform was used throughout the entire hospital, including the ICUs. It is possible that the association between ICU admission and increased mortality that we observed might have been related to using a general program in an ICU environment. We note that Cerner recently developed a Critical Care Solutions application module, suggesting that industry has recognized the possibility that a general, medical-surgical clinical application program alone may be suboptimal for the ICU. It is also possible that utilization of an adult-based clinical application platform in a children's hospital may be suboptimal. A pediatric-specific application module remains to be developed.
Study Limitations
Several limitations of our study should be considered. First and foremost, inherent limitations of study design preclude any statements regarding cause and effect, and appropriate caution should be taken regarding the conclusions drawn from this retrospective study. In a single institution, it is difficult to assess the causality of increased mortality when a new intervention is given, especially when the intervention affects the administration of every drug given to every patient. This dilemma has been addressed carefully with new single-drug and single-device interventions by the Food and Drug Administration regulatory agency. Presently there is no regulatory body that evaluates the safety of computer technology in administrative medicine. Without an organized systems approach to this problem, simple-minded physician investigators can provide only conjecture. We have noted delays in administration of time-sensitive medication, increased need for physicians and nurses to be taken away from the bedside and placed at the computer terminal, and specific problems with antibiotic administration. However, accurate evaluation of CPOE will require systems-based troubleshooting with well-funded, well-designed, multicenter studies that can adequately address these questions. Second, because we have examined a unique patient population admitted through interfacility transport, our findings may not be generalizable to the hospital experience as a whole. Indeed, as we alluded to earlier, our conflicting results with that of Upperman et al4 may stem from the different patient populations studied. Still, we propose that much like drug intervention studies, the identification of subpopulations of patients who may not benefit or may even experience negative consequences from an intervention is an informative finding. Third, our observation period after CPOE implementation was brief and may simply reflect the adjustment period that commonly follows any major, sweeping change. It is possible that had we extended our study another quarter, we might have observed a return to better-than-expected outcomes. However, changes to resident and fellow coverage of the ICUs had been initiated during the second quarter of 2003 in preparation to meet the recent Residency Review Committee restrictions of resident work time. Because it was uncertain what effects the policy restricting the resident/fellow work week to <80 hours might have on patient care and outcome, the study was closed to minimize the influence of this potential confounder. We additionally note, however, that our post-CPOE observation period actually corresponds to the post-CPOE observation period by Upperman et al4 that is marked by immediate reductions in harmful ADEs. Fourth, in a related consideration, the relative imbalance between our pre- and post-CPOE observation periods raises potential confounding from seasonal variability of illness often seen in children. Although we cannot exclude this possibility, we observe that overall patient characteristics and the distribution of diagnostic categories were similar during the 2 observation periods. In addition, we note that a comparison of unadjusted mortality rates for matching 5-month periods (October 29March 31) before and after CPOE implementation reveals that mortality increased from 2.58% (16 of 621) to 6.57% (36 of 548; P = .002,
2) between these 2 matching periods. Fifth, we again consider the possibility that our finding may reflect a clinical applications program implementation and systems integration issue rather than a CPOE issue per se. Sixth, although we have attempted to control for many important mortality covariables, it remains possible that our observation that CPOE implementation is associated with increased mortality may have resulted from an unidentified confounding factor. A "regression to the mean" phenomenon cannot be discounted.
| CONCLUSION |
|---|
|
|
|---|
| ACKNOWLEDGMENTS |
|---|
Preliminary work was presented, in part, at the Society of Critical Care Medicine, 33rd Critical Care Congress; February 2025, 2004; Orlando, FL.
We thank Bradley A. Kuch, BS, RRT, for assistance with data retrieval from Children's Hospital of Pittsburgh's Critical Care Transport Team interfacility transport database and Folafoluwa O. Odetola, MD, and Frank W. Moler, MD, for manuscript suggestions.
| FOOTNOTES |
|---|
Reprint requests to (Y.Y.H.) University of Michigan Medical School, C.S. Mott Children's Hospital, Pediatric Critical Care Medicine, Mott F-6882/Box 0243, 1500 East Medical Center Dr, Ann Arbor, MI 48109. E-mail: yyhan{at}med.umich.edu
No conflict of interest declared.
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
N. M. Lorenzi, L. L. Novak, J. B. Weiss, C. S. Gadd, and K. M. Unertl Crossing the Implementation Chasm: A Proposal for Bold Action J. Am. Med. Inform. Assoc., May 1, 2008; 15(3): 290 - 296. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. M. Walker, P. Carayon, N. Leveson, R. A. Paulus, J. Tooker, H. Chin, A. Bothe Jr., and W. F. Stewart EHR Safety: The Way Forward to Safe and Effective Systems J. Am. Med. Inform. Assoc., May 1, 2008; 15(3): 272 - 277. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. E. Walsh, C. P. Landrigan, W. G. Adams, R. J. Vinci, J. B. Chessare, M. R. Cooper, P. M. Hebert, E. G. Schainker, T. J. McLaughlin, and H. Bauchner Effect of Computer Order Entry on Prevention of Serious Medication Errors in Hospitalized Children Pediatrics, March 1, 2008; 121(3): e421 - e427. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. S. Parshuram MB ChB DPh, T. To PhD, W. Seto BScPhm Pha, A. Trope MSc RPh, G. Koren MBBS, and A. Laupacis MD MSc Systematic evaluation of errors occurring during the preparation of intravenous medication Can. Med. Assoc. J., January 1, 2008; 178(1): 42 - 48. [Abstract] [Full Text] [PDF] |
||||
![]() |
F. H. Morriss Jr Adverse Medical Events in the NICU: Epidemiology and Prevention NeoReviews, January 1, 2008; 9(1): e8 - e23. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. A. Taylor, L. A. Loan, J. Kamara, S. Blackburn, and D. Whitney Medication Administration Variances Before and After Implementation of Computerized Physician Order Entry in a Neonatal Intensive Care Unit Pediatrics, January 1, 2008; 121(1): 123 - 128. [Abstract] [Full Text] [PDF] |
||||
![]() |
Committee on Pediatric Emergency Medicine Patient Safety in the Pediatric Emergency Care Setting Pediatrics, December 1, 2007; 120(6): 1367 - 1375. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. Pate Medical Errors Negatively Impact Physicians' Mental Health and Job Satisfaction AAP Grand Rounds, November 1, 2007; 18(5): 52 - 53. [Full Text] [PDF] |
||||
![]() |
W. Hollingworth, E. B. Devine, R. N. Hansen, N. M. Lawless, B. A. Comstock, J. L. Wilson-Norton, K. L. Tharp, and S. D. Sullivan The Impact of e-Prescribing on Prescriber and Staff Time in Ambulatory Care Clinics: A Time Motion Study J. Am. Med. Inform. Assoc., November 1, 2007; 14(6): 722 - 730. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. T. Holdsworth, R. E. Fichtl, D. W. Raisch, A. Hewryk, M. Behta, E. Mendez-Rico, C. L. Wong, J. Cohen, S. Bostwick, and B. M. Greenwald Impact of Computerized Prescriber Order Entry on the Incidence of Adverse Drug Events in Pediatric Inpatients Pediatrics, November 1, 2007; 120(5): 1058 - 1066. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. D. Mahoney, C. M. Berard-Collins, R. Coleman, J. F. Amaral, and C. M. Cotter Effects of an integrated clinical information system on medication safety in a multi-hospital setting Am. J. Health Syst. Pharm., September 15, 2007; 64(18): 1969 - 1977. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. I. Harrison, R. Koppel, and S. Bar-Lev Unintended Consequences of Information Technologies in Health Care An Interactive Sociotechnical Analysis J. Am. Med. Inform. Assoc., September 1, 2007; 14(5): 542 - 549. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. D. Auerbach, C. S. Landefeld, and K. G. Shojania The Tension between Needing to Improve Care and Knowing How to Do It N. Engl. J. Med., August 9, 2007; 357(6): 608 - 613. [Full Text] [PDF] |
||||
![]() |
D. W. Bates Preventing medication errors: A summary Am. J. Health Syst. Pharm., July 15, 2007; 64(14_Supplement_9): S3 - S9. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. S. Simon, T. G. Rundall, and S. M. Shortell Adoption of Order Entry with Decision Support for Chronic Care by Physician Organizations J. Am. Med. Inform. Assoc., July 1, 2007; 14(4): 432 - 439. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. S. Ash, D. F. Sittig, E. G. Poon, K. Guappone, E. Campbell, and R. H. Dykstra The Extent and Importance of Unintended Consequences Related to Computerized Provider Order Entry J. Am. Med. Inform. Assoc., July 1, 2007; 14(4): 415 - 423. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. Blumenthal and J. P. Glaser Information Technology Comes to Medicine N. Engl. J. Med., June 14, 2007; 356(24): 2527 - 2534. [Full Text] [PDF] |
||||
![]() |
I. England and D. Stewart Executive management and IT innovation in health: identifying the barriers to adoption Health Informatics Journal, June 1, 2007; 13(2): 75 - 87. [Abstract] [PDF] |
||||
![]() |
S. A. Kuiper, S. R. McCreadie, J. F. Mitchell, and J. G. Stevenson Medication errors in inpatient pharmacy operations and technologies for improvement Am. J. Health Syst. Pharm., May 1, 2007; 64(9): 955 - 959. [Full Text] [PDF] |
||||
![]() |
J. C. Crosson, P. A. Ohman-Strickland, K. A. Hahn, B. DiCicco-Bloom, E. Shaw, A. J. Orzano, and B. F. Crabtree Electronic Medical Records and Diabetes Quality of Care: Results From a Sample of Family Medicine Practices Ann. Fam. Med, May 1, 2007; 5(3): 209 - 215. [Abstract] [Full Text] [PDF] |
||||
![]() |
W. P. Welch, D. Bazarko, K. Ritten, Y. Burgess, R. Harmon, and L. G. Sandy Electronic Health Records in Four Community Physician Practices: Impact on Quality and Cost of Care J. Am. Med. Inform. Assoc., May 1, 2007; 14(3): 320 - 328. [Abstract] [Full Text] [PDF] |
||||
![]() |
E. Ammenwerth and N. de Keizer A Viewpoint on Evidence-based Health Informatics, Based on a Pilot Survey on Evaluation Studies in Health Care Informatics J. Am. Med. Inform. Assoc., May 1, 2007; 14(3): 368 - 371. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. M. Bobb, T. H. Payne, and P. A. Gross Viewpoint: Controversies Surrounding Use of Order Sets for Clinical Decision Support in Computerized Provider Order Entry J. Am. Med. Inform. Assoc., January 1, 2007; 14(1): 41 - 47. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. R. Weir, J. J.R. Nebeker, B. L. Hicken, R. Campo, F. Drews, and B. LeBar A Cognitive Task Analysis of Information Management Strategies in a Computerized Provider Order Entry Environment J. Am. Med. Inform. Assoc., January 1, 2007; 14(1): 65 - 75. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. C. Classen, A. J. Avery, and D. W. Bates Evaluation and Certification of Computerized Provider Order Entry Systems J. Am. Med. Inform. Assoc., January 1, 2007; 14(1): 48 - 55. [Abstract] [Full Text] [PDF] |
||||