 |
|
 |
|
 |
 |
|
 |
 |
 |
 |
 |
 |
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
,||,#
* Children’s Health Services Research, Indiana University School of Medicine, Indianapolis, Indiana
Robert Wood Johnson Clinical Scholars Program
Department of Pediatrics
|| Division of Biomedical Informatics
¶ School of Medicine
# Child Health Institute, University of Washington, Seattle, Washington
 |
ABSTRACT |
|---|
 TOP ABSTRACT METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Objective. To determine whether a point-of-care personal digital assistant (PDA)-based patient record and charting system could reduce the number of resident progress-note documentation discrepancies in a neonatal intensive care unit (NICU).
Design/Methods. We conducted a before-and-after trial in an academic NICU. Our intervention was a PDA-based patient record and charting system used by all NICU resident physicians over the study period. We analyzed all resident daily-progress notes from 40 randomly selected days over 4 months in both the baseline and intervention periods. Using predefined reference standards, we determined the accuracy of recorded information for patient weights, medications, and vascular lines. Logistic and Poisson regression were used in analyses to control for potential confounding factors.
Results. A total of 339 progress notes in the baseline period and 432 progress notes in the intervention period were reviewed. When controlling for covariates in the regression, there were significantly fewer documentation discrepancies of patient weights in notes written by using the PDA system (14.4%–4.4% of notes; odds ratio [OR]: 0.29; 95% confidence interval [CI]: 0.15–0.56). When using the PDA system, there were no significant changes in the numbers of notes with documentation discrepancies of medications (27.7%–17.1% of notes; OR: 0.63; 95% CI: 0.35–1.13) or vascular lines (33.6%–36.1% of notes; OR: 1.11; 95% CI: 0.66–1.87).
Conclusions. The use of our PDA-based point-of-care patient record and charting system showed a modest benefit in reducing the number of documentation discrepancies in resident daily-progress notes. Further study of PDAs in information systems is warranted before they are widely adopted.
Key Words: resident • documentation • PDA • NICU
Abbreviations: IOM, Institute of Medicine • PDA, personal digital assistant • NICU, neonatal intensive care unit • OR, odds ratio
The 1999 report by the Institute of Medicine (IOM), To Err Is Human: Building a Safer Health System, made medical error prevention an important goal of many organizations by reporting that 44 000 to 98 000 patients die each year due to medical errors.1 More than 50% of these errors were potentially preventable.2,3 Many believe that technology can be used to reduce these errors. An IOM report, Crossing the Quality Chasm: A New Health System for the 21st Century, makes this explicitly clear.4 Their 2001 report stated that safety and quality cannot improve without the use of information technology such as electronic medical records.4 Some believe that a general-purpose electronic medical record might be able to provide benefits.5–7 Focused studies have shown that specific aspects of an electronic medical record (specifically, computerized physician order entry, pharmacy systems, and clinical reminder systems) can reduce error rates and improve compliance with guidelines.8–11 This has not gone unnoticed. Investment in health care information technology rose from $6.5 billion in 1990 to more than $20 billion in 2000.12 Independent consultants, however, estimate that 25% to 50% of these costs are paying for redundant work processes.12 Some believe that new information technology is as likely to decrease efficiency as it is to increase it.13 Clearly, new systems need to be studied before being set in place.
One area of errors that remains relatively unexplored is that of documentation errors. As information is transcribed over and over, it is at higher risk for becoming corrupted. We recently found documentation discrepancies in >60% of resident daily-progress notes relating to patient weights, medications, or vascular lines.14 Those patients who were sickest and had the most information to be maintained were most susceptible to corruption of their documented data.14 Because information recorded in the chart can be central to decision-making processes, erroneous information has the potential to compromise patient safety. The limitations of paper charting often require a number of repeated transcriptions before information finally reaches the chart, potentially leading to corruption of the data. We hypothesized that a point-of-care personal digital assistant (PDA)-based patient record and charting system would reduce the number of transcriptions of data and the prevalence of documentation discrepancies.
|
METHODS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
The study protocol was approved by the University of Washington Institutional Review Board.
Definitions
To quantify the existence of and changes in documentation discrepancies, we used methods that we developed previously in an earlier study.14 These methods are described briefly below.
Resident Progress Notes
The required resident progress note is intended to summarize the patient’s changes and care plan each day. Standard of care at the study institution mandates that resident progress notes be written daily on all patients. Note templates are generated by a database storing some basic demographic and clinical data, printed, and then filled in by hand.
Documentation Discrepancies
We defined a documentation discrepancy as an occurrence when information on the progress note did not match a predefined reference standard. As in our previous study,14 we focused on 3 pieces of information: patient weights, vascular lines, and medications, all of which were recorded on the daily-progress note. We defined the reference standard for weight to be the nursing flowsheet; the reference standard for line access was a combination of the nursing flowsheet and the nursing daily-assessment sheet; the reference standard for medications was the pharmacy medication-administration record.
Vascular lines were defined as an intravenous line, peripheral or central venous catheter, umbilical vein or artery catheter, or arterial line.
Medications given in hyperalimentation were excluded. Vitamins and dietary supplements were not considered medications, because they were not consistently listed in the medication section of notes. Drip medications were counted as medications. As-needed medications that the patient did not receive that day were ignored. If, however, a patient was given an as-needed medication on a given day, then we counted that as a medication the patient was currently prescribed. Topical as-needed medications were ignored. To be correct, a medication in the progress note only had to have the correct name. Dose and frequency were not used in this analysis.
Lines and medications were considered in or prescribed if they had begun before 8:00 AM. Any that were discontinued during the day could either be documented or not documented without penalty. In each case, residents were not penalized if it was possible that their documentation was correct at any time of the day.14
We again divided medication and vascular-line documentation discrepancies into 2 groups:1 Errors of omission occurred when a resident did not record on the progress note a medication that appeared on the pharmacy medication-administration record or a line that the patient had in place;2 errors of commission occurred when a resident did record on the progress note a medication that was not on the pharmacy medication-administration record or a line that the patient did not have in place. Errors of omission could not occur when the patient was prescribed no medications or had no lines, so these patients were excluded when calculating percentages of notes with this type of discrepancy. All notes were included when calculating errors of commission.
Because the daily weights were sometimes recorded after notes had been written, we accepted as correct any weight that exactly matched, to the number of decimal places written, the weight recorded on the day of the note or the previous day. Again, in each case, the progress note was deemed accurate if the information was correct at any time of the day.
Intervention
We designed a point-of-care PDA-based client/server patient record and charting system for use in the NICU.15 This system was tested and piloted over the course of 2 years before being implemented. In November, 2001, we trained the resident physicians in the use of the system and began using it on November 16, 2001. For the next 4 months, residents used the PDA-based system exclusively for documenting and charting the care of patients. At the end of the 4-month intervention, the PDA system was removed, and the NICU returned to the previous method of documentation.
Residents would enter data into the PDAs over the course of the day, eliminating the need for pen-and-paper recording (Fig 1). Information entered into the PDA was automatically moved into appropriate databases on a personal computer, which allowed the system to generate printed progress notes, admission notes, summary sheets, and overnight signout. All progress notes were automatically populated with information in the PDA concerning medications, vascular lines, and patient weights.15 All residents were trained in the use of the system as they started their NICU rotations, and instruction manuals were distributed detailing the functions and use of the system.
|
In the 30% of charts reviewed independently by 2 people, the data collector and physician reviewer had excellent agreement for notes with errors in the documentation of weight (98%; 
= .84), lines (97%; = .92), and medications (94%; = .81).
Analysis
We recorded documentation discrepancies both by the numbers of notes and numbers of discrepancies per note. We used 
2 analysis to look for changes in the existence of documentation discrepancies in resident progress notes. We also performed regression analyses of discrepancy rates, adjusting for a number of covariates using Logistic and Poisson regression with robust error estimates clustered by patient to control for the nonindependence of notes, because each patient could have had >1 note included in our study.16 Because note writing is often a team process, we could not accurately identify or control for which resident wrote which part of each note. We calculated odds ratios (ORs) to determine, after controlling for covariates, whether the PDA system caused any significant changes in the existence of a documentation discrepancy on each note. We calculated incident rate ratios to determine how the system affected the number of each type of discrepancy on each note. We used covariates that we had investigated previously and thought might increase the likelihood of incurring a documentation error.14 These covariates included the total number of medications per patient, the number of lines per patient, day of the note (weekend or weekday), the corrected gestational age (estimated gestational age at birth plus day of life), and number of days the patient had been hospitalized. Calculations were performed by using the STATA statistical package (STATA Corporation, College Station, TX).
Using standard power calculations for detecting a difference in proportions in 2 independent groups, we determined that to detect a 10% absolute reduction in the rate of documentation discrepancies in any of our 3 areas with 80% power and an 
of .05, each group would need 321 notes for review.
|
RESULTS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
|
|
|
|
|
DISCUSSION |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
This study has limitations that warrant consideration. The design was a before-and-after trial, not a randomized, controlled trial. We found that there was no way to randomize by patient, resident, or month without compromising the integrity of patient data. We took every step to reduce possible bias, however. By using the same months for both the baseline and intervention periods, we hoped to account for secular trends, seasonal variation, and stage of resident training. Furthermore, no other systematic change in documentation was made during the intervening time period. Published firm trials such as ours have frequently used similar designs.18–21
This study also occurred in a single teaching institution. It therefore can be generalized only conservatively. This is also a study of our PDA system; other, more-robust systems might have more of an effect. However, we used a multistage development process that intimately involved end users15 and had every potential to succeed. As with our previous study,14 there may be questions about our choice of reference standards for weight, medications, and vascular lines. As mentioned previously, we do not know the extent to which documentation discrepancies such as these are associated with untoward outcomes. We only studied resident progress notes, and many would argue their relative importance in the care of patients. Many residents rely primarily on their notes, however, and would disagree. Finally, we only looked at a process measure, but we believe that the accuracy of documented information is important and worthy of study.
We hypothesized that allowing residents to enter information at the point of care would reduce the number of transcriptions and lead to fewer discrepancies in documentation. Although the accuracy of weight documentation did improve, the documentation of other areas did not. This may be due to limitations of the technology. We found it encouraging that all the residents adopted the system quickly, with everyone complying immediately after its institution. Even so, there were significant user issues that should be noted in the future planning of such systems. These issues included problems with ease of data entry, size of the PDA screen, and other hardware, software, and user issues.22 Residents found that entering numbers such as weights were relatively simple on the PDAs. Keeping track of other, more-text-intensive information was not. Whether these limitations or other factors resulted in the lack of improvement in the rates of discrepancies is not known.
These findings have implications beyond this trial, however. Millions of dollars are spent every year on new and, for the most part, untested technologies. Many groups such as the IOM believe that improved information technology is a key component of improved care. Although PDAs are becoming more popular in medicine, they have not been clearly shown to be beneficial. A recent survey found that 30% of family practice resident programs are requiring residents to use PDAs for information management.23 Although some of these have been studied in a limited setting,24,25 there is little evidence to support their widespread adoption. Even so, many are moving forward and transitioning to systems such as these. At Long Island Jewish Hospital (New Hyde Park, NY), all 80 first-year residents are now keeping records on PDAs, reportedly to reduce medical errors.26 Without rigorous study, however, there is no way to know how the PDAs are affecting error rates.
Information technology has often not lived up to its promise. According to the Standish Group in Cape Cod, Massachusetts, >30% of computer systems built internally by corporations for their employees are either canceled or rejected after completion.13 Before we blindly transition to new and perhaps exciting technologies, well-designed trials are needed to determine whether there are actual benefits or even harms.27 Additional study of PDAs in information systems is certainly warranted before they are widely adopted.
|
ACKNOWLEDGMENTS |
|---|
We thank the University of Washington, Division of Neonatal Medicine, for patience and support. We also thank Frederick P. Rivara, MD, MPH, for helpful suggestions in the preparation of this manuscript.
|
FOOTNOTES |
|---|
Address correspondence to Aaron E. Carroll, MD, MS, Riley Research, Rm 330, Indiana University School of Medicine, 699 West Dr, Indianapolis, IN 46202. E-mail: aaecarro{at}iupui.edu
The views expressed within this article are those of the authors and do not necessarily represent the views of the Robert Wood Johnson Foundation or the University of Washington.
|
REFERENCES |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
This article has been cited by other articles:
 |
|
 |
|
  D. M. Hilty, S. Benjamin, G. Briscoe, D. J. Hales, R. J. Boland, J. S. Luo, C. H. Chan, R. S. Kennedy, H. Karlinsky, D. B. Gordon, et al. APA Summit on Medical Student Education Task Force on Informatics and Technology: Steps to Enhance the Use of Technology in Education Through Faculty Development, Funding and Change Management Acad Psychiatry, December 1, 2006; 30(6): 444 - 450. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. M. Hilty, D. J. Hales, G. Briscoe, S. Benjamin, R. J. Boland, J. S. Luo, C. H. Chan, R. S. Kennedy, H. Karlinsky, D. B. Gordon, et al. APA Summit on Medical Student Education Task Force on Informatics and Technology: Learning About Computers and Applying Computer Technology to Education and Practice Acad Psychiatry, February 1, 2006; 30(1): 29 - 35. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
||
|
||
Home | My Pediatrics | Journal Information | Current Issue | Past Issues | Subscriptions & Services | Contact Us Click here for faster international access © 2004 American Academy of Pediatrics. All rights reserved. |
||
|
||
