PEDIATRICS Vol. 103 No. 1 Supplement January 1999, pp. 266-277

SECTION 2: MEASUREMENT:
Perinatal Information Systems for Quality Improvement: Visions for Today

Terri A. Slagle

From the California Pacific Medical Center, San Francisco, California.

	ABSTRACT

Top Abstract Conclusion References

Today clinical information is used for a multitude of purposes beyond patient care documentation including quality review and improvement processes, allocation of resources, budgetary and long-term planning, productivity measurement, and justification to payers for services provided. Providers in perinatal medicine are faced with the challenge of finding methods to meet these information needs. Case examples of the different approaches to collecting and using obstetric and neonatal information are described. The role of computer-based patient records is outlined and solutions available to perinatal medicine are reviewed.

Never in the history of medicine has the demand for information been so great. In the past, patient information was primarily used by the health care team directly providing care to the patient. Now clinical information is used for a multitude of additional purposes including quality review and improvement processes, allocation of resources, budgetary and long-term planning, productivity measurement, and justification to payers for services provided.¹ In 1979, 71% of hospital information system spending was on business systems for patient billing and materials management. By 1993 a dramatic shift had occurred, with 52% of health care information dollars going toward systems to handle direct clinical patient data and another 29% for miscellaneous consulting systems for use in areas other than business or direct patient documentation.² Data requests to obstetrics and neonatology come in from other hospital departments, state and federal agencies, payers, and outside accreditation agencies such as the Joint Commission on Accreditation of Healthcare Organizations (JCAHO)³ and the National Committee for Quality Assurance (NCQA).⁴ Increasingly health consumer groups like the National Business Coalition on Health,⁵ which are charged with monitoring the costs and quality of care provided, are entering the data game. These diverse information needs in an era of health care reform and decreasing health care reimbursement present a tremendous challenge to providers seeking to enter the electronic age.

There are two basic components of computerized informationthe data and the software (often a database) to store and handle that data. Health care consultant Sharon Graugnard cautions, "It is important to turn raw data, of which there is plenty, into meaningful information. Technology should be the facilitator, not the solution."⁶ To that end, many decisions need to predate the choice between internally building a database or selecting a vendor to help meet perinatal information needs. This article recounts examples of the experiences of perinatal care providers in developing and using a diverse range of systems to handle obstetric and neonatal information. In part all these efforts have touched on some of the following issues in making a choice of an information system for perinatal medicine:

1. Why collect data?

2. What data are needed to generate the desired outputs?

3. How are the data going to be collected?

4. Who is the data collection going to include?

5. Where will the information reside?

	WHY HAVE A PERINATAL INFORMATION SYSTEM?

Decisions on the issues of populations, data set size, collection methods, and software technologies should be based on the types of outputs desired. No one sets out to collect data solely for the sake of collecting data, although many have succeeded in doing just that. In a survey of tertiary neonatal intensive care units, satisfaction was highest among database users who frequently used the majority of the information they had collected.⁷ Information systems were used to compile statistical information (92% of the 237 centers with a database), generate patient care forms (38%), and support quality assurance activities (72%). With increasing frequency, a major reason to collect information on patients is to assess and improve the quality of care.^8,9

Continuous quality improvement is a concept that originated in business that enlists the entire organization to work toward measuring and improving a product (in this case, care).⁸ Accreditation organizations are increasingly focused on measuring clinical performance and have heightened the demands for information and the visibility of information systems that can serve up the data. Key programs include the ORYX initiative of JCAHO that is directed primarily toward health care institutions^3,10 and the Health Plan Employer Data and Information Set (HEDIS) of NCQA used by health maintenance organizations.⁴

The ORYX initiative was the result of a JCAHO move toward a more continuous, data-driven accreditation process. The desire for objective, comparative information resulted in a program of reporting institutional outcomes (called performance measures) to JCAHO via an outside vendor.³ The timeline for increasing the number of patients in whom performance measures are required is based on a predetermined percentage of discharges. Obstetric and nursery services remain largely hospital-based and, therefore, may satisfy the JCAHO mandated increase from 20% to 35% of yearly discharges sampled over the first 4 years of the program.¹⁰ Seeking out an acceptable vendor for the ORYX process who also meets other data needs may help justify the cost of a computerized data solution. Many companies dealing with perinatal information have made their products ORYX compliant. In addition, some enterprise-wide solutions include obstetric and infant items in the performance measures they can report to JCAHO. An updated list of ORYX vendors is available from the JCAHO Web site (http://www.jcaho.org/perfmeas/oryx/oryx_frm.htm).¹⁰

Measures tracked by health maintenance organizations (HMOs) also attempt to reflect on and compare perinatal care. The usual vehicle for these comparisons is the HEDIS measures of NCQA.^4,11 HEDIS measures referable to perinatal medicine include onset and amount of prenatal care and rates of cesarean section, vaginal birth after cesarean and infant low birth weight. HMOs are routinely judged and judge their physicians by these report cards.⁶ Most information systems designed to collect and report HEDIS measures will lack sufficient information to independently meet perinatal medicine needs. However, those seeking out a new perinatal information solution should consider whether that system will be an adjunct to any required HEDIS reporting activities.

Information technologies for use in quality assurance activities have been divided by Aranow and Coltin into four broad types based on the origin of the data they use.⁹ These technologies are 1) codification of data that use an ability to rapidly manipulate and compare coding data such as the International Classification of Diseases, 9th edition (ICD-9) and Current Procedural Terminology (CPT), 2) case-mix systems that rely on analytic techniques in their design to severity adjust and group patients or encounters by resource use, 3) computer-based records and 4) expert systems to provide rule-based procedures for decision support through the use of monitors (or alerts) for real time surveillance and off line electronic access to knowledge sources from expert "consultants." These methods are used alone or in tandem to meet the computer-based information needs of quality improvement programs. Perinatal medicine work has been concentrated in the use of coded data (see below) but expanding options using the other three categories can be expected. Case-mix systems for severity rating are beginning to be tested on obstetric¹² and neonatal¹³ patients with variable success. The expanding functionality of departmental databases to include extensive patient information is making the record more readily accessible for quality review purposes.^14,15 Although the decision-based segment of health care informatics is rapidly growing, experience with expert systems in perinatal medicine has been largely restricted to the computer-assisted interpretation of electronic fetal monitor strip data¹⁶ and management of technologies such as ventilators in the neonatal intensive care unit (NICU).¹⁷ Vendor lists for software applications in the case-mix (utilization review), electronic record, and decision support categories can be found on the Web pages of health information organizations such as Health Information and Management Systems Society ([HIMSS], http://www.himss.org/) and the Healthcare Information Systems Directory ([HISD], http://www.health-infosys-dir.com/). With continued internal goals to monitor and improve care and mounting external pressure to report outcomes the question "why collect data?" is often the easiest step in deciding to seek out a perinatal information system solution.

	WHAT DATA DO WE NEED?

A critical key to any database success is that a careful analysis of how the data are to be used (the outputs) needs to precede the decision of what data to collect (the input.)¹⁸ The amount of information collected for constructing the desired outputs and measuring maternal or neonatal outcomes varies immensely. Successful strategies have ranged from minimal data sets^19-21 to software packages that emulate a larger part of the patient record.^14,22 Satisfaction with neonatal databases depends more on the feeling the information is being used optimally than the absolute number of data elements collected. Before computers allowed the rapid selection and sorting of large amounts of information, most data sets revolved around log book functions. Although a few centers in 1989 maintained their "database" on paper (9% of neonatal database survey respondents) most have embraced the benefits offered by the availability of personal computers.⁷ The step from a paper log to a computerized one is often the first exposure to a computerized information solution.²³

A minimal data set does not necessarily mean a small number of data items but rather a strategy to collect only data that focuses on information that is critical. Maresh and colleagues²² in London reported how their minimal obstetric database of 106 essential items yielded simple statistical reports, summary data on maternal and infant stays and computer-generated birth notifications. Escobar et al¹⁹ undertook the development of a neonatal minimal data set for Northern California Kaiser after they found hospital-based legacy systems to be inadequate to 1) identify clusters of preventable adverse outcomes in a timely manner and 2) perform feasibility assessments for research trial design. After close attention to data element definition, the investigators aimed for information that could be reliably obtained by a trained research assistant. A file server model with high-speed phone lines for data exchange was used. Structured training of research assistants, error detection algorithms, and data audits optimized data quality. The successes and pitfalls of their design are provided in enough detail to assist other providers seeking to design similar data sets.

An alternative approach to minimal data sets is to "control" the content of the medical record and unlock the information historically stored in the paper chart. This expanded data approach embraces some of the concepts of electronic medical record design (see below). The following case study for a maximal data approach shows this type of information solution is a reality for today and not just a dream for the future.

A Case Study

For decades the quality improvement process for the obstetrics and neonatology services at California Pacific Medical Center (formerly Children's Hospital of San Francisco) was entirely conducted by peer review of individual cases that were hand-selected for examination. This process relied on summary administrative data that were collected after patient discharge, often were incomplete, and at times questioned by the physician reviewers. Hand-recorded delivery log books served as the source of statistical data. The overall quality of care provided was frequently overshadowed by individual bad outcomes and the committee felt that the forest (general service performance) was being lost among the trees (individual cases). In 1988, the hospital administration approved a grant for computer hardware purchase and formed a task force to investigate revising the approach to data collection to one that would support a more reliable method of monitoring care practices. A perinatologist and neonatologist spearheaded the project with the goal of developing a single integrated system to serve the needs of both disciplines.

The medical record was defined as the most reliable source of clinical data and the process of physician and nurse documentation was reengineered to include data collection as an essential purpose for the patient record. The basic philosophy was that data would be collected once and then harvested for a multitude uses. It was hypothesized that if everyone was seeing the same information continuously, a more robust and relied on data set would result. In line with a template for database success,¹⁸ the relational database was designed after the desired outputs from the system were identified. Broad-based reporting allowed the project to add stakeholders and enhanced the chance for success. A wish list with all potential inputs and outputs served as the blueprint for the initial and subsequent design effort (Fig 1). A fourth generation language relational database program (4th Dimension, ACI, Cupertino, CA) was chosen because of its ability to support queries on any data field, in a large data set, on an immediate (on the fly) basis, without the use of a programmer.

View larger version (31K): [in this window] [in a new window]   Fig. 1.   Blueprint for the ideal perinatal information system design.

Once the outputs were identified, data elements needed to generate the outputs were defined. Each piece of information was placed on a standardized chart form in one location in the medical record to be completed by the care provider deemed most likely to know the information. The fill-in-the-blank and choice list format of the forms standardized both the type and definition of the data collected. Data entry was performed from copies of the medical chart forms by the clerical personnel who were responsible for completion of the state birth certificate. The increased efficiency of birth certificate entry and printing allowed these same personnel time to input other, nonvital statistic information into the system. Using this model also guaranteed a complete set of demographic information was available in the clinical data set. A goal of having >95% of all birth certificates completed for parental signature before hospital discharge insured that the data set was always current. Data collection on the intensive care nursery population was handled initially by physician collected data sheets. These data sheets started with tracking a minimal number of diagnoses and procedures and expanded yearly as the requests for more specific data grew. Commercialization of the product has allowed the expanded functionality of creating history and physical and daily progress notes and has eliminated the need for the separate data sheets.

Examples of the type of quality improvement audits accomplished with electronic review of large data sets underscore the power of a comprehensive, integrated obstetric and neonatal relational database in the quality assessment process. Cesarean section rates were steady at about 24% until detailed, physician-specific report cards were distributed. Committee and department review of these data fueled a subsequent decrease in the cesarean delivery rate to <19%.²⁴ A policy on encouraging mothers with placenta previa to bank autologous blood in the event a transfusion would be required was deemed to be cost-ineffective after database review showed the need for transfusion to be small even in the highest risk subset of women at our center.²⁵ Electronic data review allowed us to make an educated choice between the screening methods suggested by the Centers for Disease Control and Prevention (CDC) policy statement on reducing group B streptococcal disease in infants.²⁶ The clinical profile of our population over the past 5 years revealed missed opportunities for chemoprophylaxis when a risk-based strategy was used and a downward trend in invasive disease with a data-driven move toward a culture-based protocol.

	HOW ARE THE DATA GOING TO BE COLLECTED?

The acquisition of data is a time-consuming effort and, like the Holy Grail, an ideal data set is frequently elusive. In a review on the quality of health care, Brook et al²⁷ noted that "It will never be possible to produce an error-free measure of the quality of care." Given the inherent problems with all biologic data, it is essential that consumers of health care information understand the basic principles for good data to make educated decisions on what data they want and to accurately interpret what they get.

A critical part of the design or selection process of a perinatal information system is formulating a collection process that yields the best data possible. To achieve this goal it is necessary to examine the characteristics that define data quality. The American Health Information Management Association ([AHIMA], http://www.ahima.org/) has laid out a comprehensive series of guidelines to insure high-quality data.²⁸ These key characteristics are outlined in Table 1. Accountability should be determined for who will design, coordinate, standardize, monitor and carry out data collection for each perinatal information effort.

There are a myriad of ways centers use to collect data. Data forms, trained chart abstractors, and data entry personnel have been long-standing adjuncts to data collection. Many centers adapted the techniques they first refined in research trials to their daily data collection activities. As the demands for information outpace moneys to collect and maintain large data sets, using existing hospital administrative data sets or tapping into select data kept in other hospital repositories, such as registration and laboratory systems, is playing a more prominent role in perinatal data activities. The introduction of technologies that use the efforts of care providers documenting their care in electronic record systems offers promise as an alternative means to acquire outcome data.

Administrative data found on state discharge tapes or universal billing forms represent a potential rich repository of patient information, if the data can be shown to be reliable. Uniform hospital discharge data sets (UHDDS) include patient identifiers, discharge diagnoses, procedures and financial information. There are many reports of the use of this information both locally and at the state or federal levels to measure, and in some instances, impact perinatal and neonatal care.^29-31

Rosen et al²⁹ reported on the state of Michigan's Patient Outcomes Measures Program that was based on the UHDDS. Local institutions were supplied with their data and that of peer institutions. Through this program one institution was able to show improvement in their rates of cesarean section, birth trauma, and vaginal births after cesarean section. Administrative data were used in a Voluntary Hospital Association pilot project on benchmarking.³⁰ Experience with discharge data in Ontario, Canada, raised concerns on an increase in readmission rates after the early discharge of healthy newborns.³¹

Although successes underscore the importance of administrative data sources, there may be pitfalls in using them. Case studies have questioned the validity of using ICD-9 coding for quality improvement efforts.¹⁹ Medical record documentation from which the codes are generated is often incomplete, illegible, or contains contradictions. Exact criteria and definitions for specific diagnoses often are lacking. In studies of medical record personnel-coded patient charts in both medicine³² and neonatology³³ error rates of 20% to 40% were noted for some diagnoses and procedures. The practice of "up-coding" a record to garner higher payment has been cited a source of potential error in these data sets.³⁴ In addition, even perfect ICD-9 and CPT codes may fall short of giving a full picture of the clinical situation.³⁵

One benefit of problem-oriented electronic records is that they force the care provider, generally a physician, to assign exact diagnoses to each patient encounter. This improved documentation serves to manage the content of the administrative data set. The definitions of specific diagnoses and clinical situations also can be improved through efforts to standardize medical language such as wider adoption of a more comprehensive clinical data indexing system like the Standardized Nomenclature of Medicine (SNOMED).³⁶

Despite the limitations of data sets from hospital discharge coding, success with their use in measuring care trends coupled with their wide availability makes them a frequent choice for perinatal information needs. Because physician and other care provider documentation is the cornerstone for coded data, a schema to continuously improve the quality of this data set is essential. Even if administrative data sets play no role in departmental perinatal information efforts, physicians should be reminded of the routine use of these data by local hospital administrations, state and federal agencies, networks, and payers and make a concerted effort to improve their quality. Discipline-specific coding guidelines have been shown to improve encoded data.³⁷ Those interested in improving their coding compliance are encouraged to visit the AHIMA Web site for a detailed strategy to accomplish this goal.³⁸

A means of enhancing the quality of the data is to aim for primary datadata that comes into the system directly from the source. For a great deal of medical information, especially diagnosis assignment, the source of the data should be the care provider. To help accomplish this, information systems are increasingly moving toward direct care provider entry. To date a major obstacle to this has been a reluctance, or inability, to type. Many direct entry systems use choice lists and templates to ease the process. Newer technologies such as voice recognition³⁹ and an ability to parse structured text for data acquisition⁴⁰ offer hope to reluctant computer users. Electronic data interchange standards such as Health Level 7 (HL7) are working to manage the protocols (or data definitions) for information. These standards help different applications, written by different vendors, to share information. An efficient communication and information exchange between systems will allow information in laboratory and registration systems to electronically populate selected fields of perinatal information systems and reduce data entry time and errors. Detailed facts on the HL7 system, participants, and uses can be found on the Duke University Web site dedicated to the topic (http://www.mcis.duke.edu/standards/HL7/hl7.htm).

	WHO SHOULD THE DATA SET INCLUDE?

Choosing which patients or clinical encounters to monitor is often based on the complexities and costs of data acquisition. If information on an entire population (all mothers delivering at the institution and their infants) is desired, systems based on the UHDDS are often used. These activities can be either locally based using case-mix or utilization review products, or centrally located through vendors such as the National Perinatal Information Center (NPIC), HCIA Inc, or hospital networks like the Voluntary Hospital Association.

The NPIC (Providence, RI) was designed to measure outcomes for hospitals offering care to high-risk mothers and newborns. The nonprofit organization performs research using diagnoses and procedure codes, demographics, all patient refined diagnosis-related groups, payer, and financial data from member hospitals. In use since 1985, the Perinatal Center Database contains information on >10 000 000 patient discharges and averages data on approximately 150 000 to 200 000 newborns each year.⁴¹ Enrollees have access to comparative data broken down into clinical (neonatal and obstetric), management, and quality efficiency reports.

HCIA Inc (Baltimore, MD) is a health information management company whose health care database with >325 000 000 patient discharges derived from >7000 hospitals is one of the largest in the United States.⁴² Unlike NPIC, all patient groups are included in their databases. Clinical and financial decision support applications are the mainstay of the HCIA product line. Administrative data from disparate systems is translated into universal codes (International Classification of Clinical Services) that are used to benchmark clinical performance and outcomes, profile best practices, and manage the costs and delivery of health care. Their high profile in the field is accentuated by maintaining a list of "100 Top Hospitals."

Many centers choose to target their data collection toward a selected population so that a much more detailed evaluation can be made and a more robust data set is attainable. Often the choice of the population is motivated by a specific quality of care issue and directed toward a high-risk population or procedure. Examples of perinatal information systems that target select populations include the Vermont Oxford Network (VON) that observes infants in a specific weight group, the National Nosocomial Infection Surveillance System (NNIS) of the CDC with a focus on all intensive care patients (adult, pediatric, and infant), and efforts to track a specific high-cost therapy such as extracorporeal membrane oxygenation use.

The VON is a voluntary collaborative group of neonatal intensive care units established with a goal of integrating clinical research into daily practice.⁴³ The Network database is limited to infants weighing between 401 g and 1500 g at birth and is based on paper forms collected at member sites and submitted for centralized entry. The database serves both as the nidus of information for clinical trials and a source of blinded comparative information on neonatal units and the care they provide. Members receive quarterly and annual outcome reports comparing their current data to past years and to other network sites. These reports can be used to track trends, benchmark to other institutions, and target review of mortality and morbidity cases that fall outside of the network norms. The Network's report of outcomes of 2961 very low birth weight infants (birth weight 501 g-1500 g) born in 1990 demonstrated the potential of performing high-quality, low-cost research using a well-defined minimal data set.⁴⁴ A current, international snapshot of the demographics, diagnoses, and complications of this group underscored the marked variability in the medical interventions and common clinical outcomes for these tiny patients. The report represented the first large cohort of patients cared for after the routine availability of surfactant and documented an alarmingly low use of maternal antenatal steroids of 18%. The combination of strictly university (only 11% of the centers in 1990) and community-based outcomes offered a benchmark standard for neonatal units across the world. The membership in VON is growing rapidly and in 1997 the Network reported information from 250 nurseries representing >20 000 infants to their member institutions.⁴⁵ VON has been accepted as an ORYX provider by the JCAHO.

The CDC Infection Surveillance Program includes data on patients cared for in neonatal, pediatric, and adult intensive care units. The data collection focuses on the use of invasive procedures and associated nosocomial infection rates.^46,47 Participation is open to all interested institutions with an average daily census of 100 inpatients. Centers must agree to follow the NNIS surveillance methodology, collect timely data, have a Pentium-based personal computer to enter the data into and transmit it via a modem to the centralized CDC database. There are no charges to participate and approximately 120 nurseries currently participate in the program.⁴⁷

The Extracorporeal Life Support Organization registry of extracorporeal membrane oxygenation (ECMO) use is maintained as an electronic forms-based data set and serves as an example of data collection on patients who receive a specific therapy.⁴⁸ The level of detail collected on each patient treated with ECMO is extensive and the registry regularly reports outcomes in peer reviews forums.^48,49

	WHERE WILL THE INFORMATION RESIDE?

Finally, the time to choose an information solution has arrived. The explosion in the demand for information has expanded both the number of medical information software vendors and the acceptance of hospital administrators that accessing information is part of the cost of doing business. Options for perinatal information solutions at this point include 1) build your own, 2) tap into existing research databases, 3) participate in an outside data analysis service, 4) wait and prepare for an hospital-wide computer-based patient record, or 5) seek out and purchase a department-based solution. Individual groups may decide to use only one method or a combination of them.

Developing Your Own Database

There are numerous examples of locally developed perinatal and neonatal databases serving the needs of single institutions,^15,50,51 hospital and health care provider networks^19,52 and even entire regions.²⁰ This approach to managing increasingly complex data sets not only is popular (74% of neonatal databases in use in 1989 were "homegrown") but has been highly successful in meeting perinatal information needs.⁷

Myers and Gletcher⁵³ used a perinatal database in their successful program to lower cesarean rates. Their voluntary initiative was aided by the constant review and dissemination of both department-wide and individual statistics that the computer-based information system allowed. Regionalization of perinatal information was demonstrated by Molin²⁰ in his description of southern Sweden's quality assurance activities. A locally designed database received information either from paper forms or via electronic transmission from locally based versions of the master data structure. Merging a departmental database with administrative data presents another successful model to examine perinatal information.⁵⁴ Wirtschafter et al⁵² used their internally developed perinatal information system as the springboard for designing southern California Kaiser's neonatal outcomes research activities.

In a milieu of needing information yesterday, a lengthy design phase in creating a database may be unacceptable more in terms of time rather than money. The cost of maintaining local systems may be substantial. Escobar et al¹⁹ reported 1995 annual database costs of $300 000 to maintain a minimal data set for their 11-unit network. At least half of these costs were attributable to data collection activities. A cost analysis should be considered before a "do it yourself" approach is chosen a priori as a less expensive option. As larger hospital networks are forming, some locally designed systems may become available as shared solutions with potentially lower costs.

Research Data Banks

Many local database development schemes have been the outgrowth of systems designed for a specific research project that were expanded to meet other data needs.⁷ Databases for research have been used by individual centers,¹⁹ in formal research consortiums such as the National Institutes of Child Health and Human Development (NICHD)⁵⁵ and on a membership^44,46 or voluntary⁴⁹ basis. The data for these purposes are often rigorously controlled with strict data definitions and data quality control efforts.⁵⁶

The NICHD is a federally funded research cooperative of 12 tertiary academic centers with postgraduate medical training programs. The group maintains data in a central database designed for the NICHD that is populated via modem transfer from personal computers at the individual centers. Data collection is performed by trained personnel dedicated to maintaining a detailed and accurate data set. The level of detail maintained in the system supports examining patterns of care and resource use. Tyson and members of the research network⁵⁵ demonstrated the depth of their information system in a report on outcomes of 1126 infants weighing between 501 g and 800 g at birth. Although the NICHD database is not available for general use, its extensive design and data quality can serve as a source of ideas for database developers.

Data Services

Data analysis services frequently use administrative data sets. The data originates at the hospital but is generally maintained at a centralized location. Whether reports are predefined paper reports or complete electronic data sets to query is dependent on the vendor and contract. The NPIC and HCIA Inc described earlier are examples of this type of database service. The VON represents an example of a centralized analysis process based on tightly defined minimal data sheets instead of discharge tapes. The JCAHO list of ORYX providers is a valuable source of companies offering comparative clinical outcome measurements using centralized data sets. Most hospitals already hold contracts with one or more vendors and those reviewing their perinatal information options should check with their own quality improvement, case management, and financial services to see what products are already available or being considered.

Computer-based Patient (CPR) RecordsHarvesting the Electronic Patient Record

The Institute of Medicine (IOM) published their task force report on the CPR in 1991 (http://www2.nas.edu/iom).¹ They subtitled the report "an essential technology for health care" having reached the unequivocal conclusion that electronic record-keeping is a necessity for health care to make cost sensitive advances in the quality and efficiency of care. The IOM cited five reasons why the CPR concept was ready to take off:

1. demands for patient data and health services research were growing,

2. technology was more powerful and affordable,

3. computers were recognized as a tool to enhance efficiency,

4. the population was aging and more mobile, and

5. there were pressures for health care reform.

CPR and their clinical data repositories offer the newest option for monitoring outcomes. The basic features of these systems are the storage and retrieval of patient demographics, clinical documentation, physician and nurse order entry, and retrieval and laboratory and pharmacy tracking. An ideal CPR also includes outcomes measurement, clinical research support, and quality improvement activities in its core functionality. Because of the rich repository of data, the electronic record offers an ideal medium for incorporation of clinical pathway and variance tracking. Ready access to decision support and other knowledge-based sources are a key component to the concept that a CPR improves care. The IOM's 12 essential attributes for an ideal CPR are outlined in Table 2.

There are at least 40 companies working on an electronic documentation system that could be classified as a CPR.⁵⁷ Although the installed base is small for products that are fully functioning in all aspects of a complete computer-based record, most of these companies have been health care information vendors for years. Their path of entry into the CPR field is varied and include a primary foundation in the admission, discharge, transfer (ADT) activities of patient registration and billing systems (eg, SMS, Malvern, PA, and HBO and Company, Atlanta, GA), strengths in laboratory or pharmacy-based systems (eg, Cerner Corporation, Kansas City, MO, and Meditech, Westcott, MA), a basis as a data repository or data warehouse (eg, Oacis Healthcare Systems, Greenbrae, CA) or a system built primarily around development of the CPR (eg, HealthVISION Corporation, Santa Rosa, CA). In thinking about electronic records, providers should be reminded that the CPR is a goal and not necessarily a specific product.

Hospitals throughout the world are embarking on the selection of a CPR, often with requests for proposal processes that span months to years. Templates for the successful selection of a comprehensive institution-wide solution are available in the literature^1,58,59 as are comparison feature lists of the products.⁶⁰

There are several unique aspects of perinatal medicine that must be considered before assuming a hospital-wide CPR will meet all departmental information needs. First, a CPR is slated to be a womb-to-tomb repository of patient information. In meeting the needs of all the patient populations served, the granularity of detail may be insufficient for an obstetric or neonatology service to measure its quality and outcomes. Physicians in perinatal medicine are encouraged to take an active role in their institution or network's CPR selection process to insure that all expected data needs are met. Specific examples of the desired outcomes to be measured are critical to investigate whether any given CPR product has sufficient detail to serve as a perinatal information source. Does the CPR keep the information you want to retrieve? Is the information uniformly kept in all cases so that both numerators and denominators can be measured? Remember, if critical information is stored in an electronic block of typed dictation you may not be any further ahead than you were looking at a paper chart.

Second, a primary purpose of a CPR is the rapid entry and retrieval of information on an individual patient. In contrast, quality improvement activities often require aggregate data from a specific group of patients. Does the CPR allow the full patient population to be queried for a specific attribute (eg, a birth weight range or the presence of a diagnosis such as uterine rupture)? Can the questions be asked directly of the CPR (or its repository), or must the data be exported or downloaded into a separate program? Is someone with programming expertise needed to perform a data query or export? Will a query of the repository adversely impact the speed of daily activities? If the desire is sufficient flexibility to find out the day before the board of directors are scheduled to meet how many mothers living in a particular region of the city were delivered last quarter, make sure to ask the vendor how easily this is accomplished.

Third, a CPR generally does not link one patient record to another. In perinatal medicine critical information and outcome measures for one patient (the mother) may reside entirely in another patient's chart (her infant). If records cannot be linked, is the necessary maternal information also kept in the infant's record and infant's information in the maternal record? Can maternal and infant records be matched if information is exported?

Finally, a complete CPR has a rich knowledge base either imbedded (in the form of rules) or accessible (in the form of links to the literature.) Pregnant women and sick newborns will require different rules than most hospital-based patients. Does the CPR allow rules to be written for specific populations? Within a group of patients can rules be further defined, such as different drug dosing alerts for term and very preterm infants? Is enough money budgeted for the custom rules to be developed?

As of 1991, the IOM felt that there were no CPR systems that met all their criteria (Table 2.)¹ Hewlett Packard executive Gary Eichorn commented the lack of such products in the past was because "It's not easy. The reason it has to be done now is basic survival. This is the most complex computer task I've ever seen."² A CPR implementation process is often undertaken in stages with registration, order entry, and laboratory functionality preceding provider documentation activities. The lack of mature CPR products, the high cost of a total solution (estimates range from $40 000 000-$200 000 000 to realize a complete solution)⁵⁹ and improved connectivity between information systems made possible through data exchange standards such as HL7 may be swinging the pendulum away from a single hospital-wide information systems solution toward a "best of breed" philosophy.⁵⁷

Niche Solutions in Perinatal Medicine

Obstetricians and neonatologists have attempted to harvest clinical data for non statistical reporting since the 1970s. Efforts rose out of a desire to automate discharge summary generation,^14,51 facilitate the completion of some of the multitude of forms and letters that clinical care requires,²² and the fact fetal and neonatal monitoring have made electronic information a routine part of perinatal care. Niche solutions in perinatal medicine have come from two basic routes, one founded on the strengths of provider documentation and the other from the monitor-based aspects of care.

Lowe et al¹⁴ described their group's transition from clerical staff entry of data sheets to bringing physicians and nurse practitioners to the computer to create their daily progress notes on-line. This process that began in 1988, before the advent of graphical user interfaces and other adjuncts to direct provider input, was found to be reliable and time efficient. Janik⁵⁰ showed that neonatal database information was cleaner, with a reduction in missing information and fewer data entry errors when the physicians moved to direct on-line entry in lieu of a system that previously relied on clerical entry of chart information. A report that showed that primary care physicians preferred computer-generated summaries of neonatal intensive care stays to dictated summaries further set the stage for the commercialization of neonatal information systems.⁶¹ This increased satisfaction was based on the relevance of the information provided for continuing care. Although moving care providers to the data collection source insures primary data, it may be a difficult task to accomplish. A successful action plan for getting physicians to participate in clinical information systems was presented at the 1998 HIMSS session under the title "You can lead physicians to work stations, but can you make them sign-on?"⁶²

Medical Data Systems ([MDS], Wayne, PA) was the first company to commercially market a comprehensive neonatal information system in the United States. Since then, Dr William Lowe started to distribute the database he reported on in the literature¹⁴ under the company name of MetaSoft (Charlottesville, VA), and Site of Care Systems (San Francisco, CA) began marketing the redesigned perinatal information systems started at California Pacific Medical Center. All three companies have migrated their neonatology products to a real time interface in which at least some of the database information is directly entered by physicians and nurse practitioners. A list of features of the neonatology component of these companies is presented in Table 3.

These systems generate comprehensive chart documentation including admission histories and physicals, daily progress notes, and compiled discharge summaries. A primary goal of such systems is to make select information in these patient reports accessible for later review. Electronically abstracting clinical, demographic, statistical, and outcome data from notes entered directly into a computer by the provider is only as good as the quality of the documentation. Documentation detail in these systems is encouraged through a balance of structured data collection (fields) and text.

Both MDS and Site of Care Systems offer applications for use outside of the neonatal intensive care unit. MDS began marketing efforts with a neonatal solution and have expanded into obstetrics with their Obstetric Information System. Site of Care Systems' initial efforts were in The Perinatal Data Center with the expanded daily charting for infants as an outgrowth of that product. Integrated high-risk developmental follow-up modules also have been developed by both companies. The National NeoKnowledge Network offered to MDS clients is marketed as a competitor to the VON. It compiles and compares data from all birth weight groups.

None of currently available niche solutions for neonatology purport to be electronic records, although all contain key components of a CPR (Table 2.) The current permanent repository of patient care documents remains in the paper record. Specialized departmental systems will continue to play a role in areas where a hospital-wide CPR lacks sufficient granularity of data to assess very specific questions and needs. With electronic interfacing these products could be used as the front end of the complete record by electronically transferring select data items or summary documents into the larger repository.

On labor and delivery units, perinatal information needs also may be met in part by revisions to electronic fetal heart rate (FHR) monitoring and archiving systems. Fetal monitor strips have always contained some patient identifier information on them. Expansion of this basic field information to include placement of nursing notations on the electronic strip originated in an effort to avoid the storage of paper strips solely to capture the bedside documentation of responses to FHR changes. Addition of summary prenatal, delivery and infant information completed the natural progression of FHR monitoring devices into obstetrical database management systems (ODMS). Companies who develop software for FHR monitoring are in various stages of developing and/or forming alliances to enhance the database qualities of their systems.

A framework for evaluating ODMS and an analysis of four of the five currently marketed solutions in the United States is presented in the ECRI review of health devices.⁶³ ECRI scientific, engineering, and analytic staffs perform extensive technical evaluations of products in structured and non biased manner. The basic surveillance and archiving of ODMS products can be expanded to include tracing annotations, narrative notes, nursing assessments, summary diagnoses and procedures, and basic statistical reporting. An underlying goal is to replace some or all the paper chart. The programs variably use form-based entry, flowcharts, hierarchies of menus, pop-up lists, and bar coding to facilitate data entry. Some offer integration with other information systems through HL7 interfaces or download information into external database packages to support more extensive data query activities. The full functionality of on-line charting linked to FHR monitoring devices may require a personal computer in each labor suite and an upgrade to newer versions of monitoring and archiving software. Basic information on these companies is listed in Table 4. On a similar theme, electronic bedside nursing charting linked into neonatal monitors (eg, CareVue [Hewlett Packard, Andover, MD] and NeoChart/NeoQuIc [Spacelabs, Redmond, WA]) offer an alternative source of NICU information if mechanisms to download or query the data are included with these products.

The health care information industry is a rapidly growing segment of the software industry. As noted in a Wall Street Journal article on this expansion "Much of the innovation has come from computer-literate doctors who saw their practices getting clogged by paper records."² These efforts are increasingly finding their way into the marketplace either as start-up companies or as acquisitions of larger software vendors. As the health information market expands, new vendors with perinatal medicine options and new angles on using current computerized systems are sure to appear for those not wanting to build or revise their own database.

	CONCLUSION

Top Abstract Conclusion References

A perinatal information system choice often is driven by the need to measure and improve the quality of care but is influenced by time and manpower constraints. Locally grown databases offer a custom solution but may take a lot of time, effort, and technical expertise to design and maintain. Commercially available products offer a turn-key solution but may require changing local practices and a capital budget process to purchase. As the practice of medicine today is far from static, ongoing costs to update, revise, and expand any perinatal information system should be anticipated. The perinatal literature offers many examples of successful database design, selection, and use. Providers selecting their first information system or looking for a change should use these sources in making an informed choice.

National attention on cesarean section and infant mortality rates stand as a testament to a fascination with perinatal data. Hierholzer suggested that, " ... We must view data as sand; tiny discrete particles with substance but basically without fixed structure, frequently accumulating in forms of great beauty or built into `castles' of great appeal. These temporary forms are subject to the winds of change and the waves of fad and politics. In this elemental form sand and data present great danger, able to blind and bury us."⁶⁴ Perinatal medicine has a rich history of using information to monitor, measure, and improve care. Many options are available to enhance and refine these efforts. In the end, we can hope to achieve Hierholzer's goal and fashion our sand into a lens that enhances our vision regarding the way in which we care for mothers and infants.

	APPENDIX: Web Site Addresses to Assist Perinatal Information System Selection Efforts

Governmental Agencies

AHCPR (Agency for Health Care Policy and Research): http://www.ahcpr.gov:80/clinic

Source of health care policy information

HCFA (Health Care Financing Administration): http://www.hcfa.gov

Information on governmental regulations in health care

IOM (Institute of Medicine): http://www2.nas.edu/iom

Reports on computer-based records and patient confidentiality

NNIS (National Nosocomial Infection Surveillance program of the CDC) http://www.cdc.gov/ncidod/diseases/hip/nnis.htm

A voluntary system for monitoring nosocomial infection rates

Sites for Health Care Information

AHIMA (American Health Information Management Association): http://www.ahima.org/

Excellent source of data collection standards, information, and guidelines

AMIA (American Medical Informatics Association): http://www.amia.org/

Source for conferences and presentations on information technology

CHIM (Center for Healthcare Information Management): http://www.chim.org/

Trade association for vendors and consultants

CPRI (Computer-based Patient Record Institute): http://www.cpri.org/

Information on standards in record computerization

HISD (Healthcare Information Systems Directory): http://www.health-infosys-dir.com/

Information on vendors

HIMSS (Health Information and Management Systems Society): http://www.himss.org/

Information on vendors, trade shows, and publications

HL7 standards: http://www.mcis.duke.edu/standards/HL7/hl7.htm

Extensive site for information regarding HL7-based data exchange

JCAHO (The Joint Commission on Healthcare Accreditation): http://www.jcaho.org/

Information on the ORYX program (http://www.jcaho.org/perfmeas/oryx/oryx_frm.htm is the direct ORYX Web area)

NCQA (National Committee for Quality Assurance): http://www.ncqa.org/

Information on HEDIS and HMO review

NAHDO (National Association of Health Data Organizations): http://www.nahdo.org/

Association that brings together public and private sector of health care information

Neonatology on the Web: http://www.neonatology.org/

Maintained by Dr Ray Duncan at Cedars Sinai Medical Center it offers up-to-date information on computer technology use in the NICU

Registry of State Level Efforts to Integrate Health Information: aspe.os.dhhs.gov/statereg/

Extensive source of state regulations impacting healthcare information

Selected Information System Vendors

Air-Shields: no Web site available WatchChild obstetrical data management system

First Databank: http://www.firstdatabank.com/   Markets knowledge-based systems for diagnostic decision support

HCIA: http://www.hcia.com/

Health information content company that specializes in clinical and financial decision support systems

Hewlett-Packard Company: http://www.hp.com/mpg/

OB TraceVue obstetrical data management system, CareVue products

LifeServ: http://www.lifeservtech.com/

Cygnet and Peritronics obstetrical data management systems

Marquette Medical Systems: http://www.mei.com/

QMI Quantitative Sentinel obstetrical data management system

MDS (Medical Data Systems): http://www.mdsinfo.com/

NeoKnowledge, National NeoKnowledge Network and Obstetric Information System

MetaSoft: http://www.ms-systems.com/

NeoData neonatal documentation system

NPIC (National Perinatal Information Center): http://www.npic.org/

A nationwide consortium of patient discharge information

Site of Care Systems: http://www.siteofcare.com/ The Perinatal Data Center and The Daily Baby software

Spacelabs: http://www.spacelabs.com/monitor.html

BirthNet obstetrical data management system, NeoChart and NeoQuIc

VON (Vermont Oxford Network): http://www.vtmednet.org/vtoxford/

Database for infants weighing 401 to 1500 g

The Web addresses for companies that market computer-based patient records, case management, and decision support products are available through the HIMSS and HISD vendor reference lists that can be accessed at these organizations' Web sites.

	FOOTNOTES

Disclosure:  Product names are used in examples to help guide readers into finding information on a specific type of product and are not meant to infer an endorsement. Dr Slagle is a founding partner in Site of Care Systems, a vendor reported on in this article.

Received for publication Sep 9, 1998; accepted Sep 9, 1998.

Address correspondence to Terri A. Slagle, MD, California Pacific Medical Center, 3850 California St, San Francisco, CA 94118.

	ABBREVIATIONS

JCAHO, Joint Commission on Accreditation of Healthcare Organizations; NCQA, National Committee for Quality Assurance; HEDIS, Health Plan Employer Data and Information Set; HMO, health maintenance organization; ICD-9, International Classification of Diseases, 9th edition; CPT, Current Procedural Terminology; NICU, neonatal intensive care unit; HIMSS, Health Information and Management Systems Society; HISD, Healthcare Information Systems Directory; CDC, Centers for Disease Control and Prevention; AHIMA, American Health Information and Management Association; UHDDS, uniform hospital discharge data sets; HL7, Health Level 7; NPIC, National Perinatal Information Center; VON, Vermont Oxford Network; NNIS, National Nosocomial Infection Surveillance System; ECMO, extracorporeal membrane oxygenation; NICHD, National Institutes of Child Health and Human Development; CPR, computer-based patient records; IOM, Institute of Medicine; MDS, Medical Data Systems; FHR, fetal heart rate; ODMS, obstetrical data management system.

	REFERENCES

Top Abstract Conclusion References

1. Dick RS, Steen EB, eds. The Computer-Based Patient Record: An Essential Technology for Health Care. Washington DC: National Academy Press; 1991
2. Fisher L. Diagnosis: choking on paper. Wall Street Journal. 1995;October 21:B17
3. Koss RG, Bowie MK, Sprenger SL, Loeb JM ORYX: the evolution in accreditation. J Clin Outcomes Management. 1997; 4:16-20
4. Iglehart JK The National Committee for Quality Assurance. N Engl J Med. 1996; 335:995-1000 [Full Text]
5. A conversation with Sean Sullivan. Med Interface. 1997;Feb:66-67
6. Edlin ML. Define your health plan's value with quality reporting. Health Manage Technol. 1996;July:17-19
7. Slagle TA, Gould JB Database use in neonatal intensive care units. Pediatrics. 1992; 90:959-970 [Abstract]
8. Kritchevsky SB, Simmons BP Continuous quality improvement concepts and applications for physician care. JAMA. 1991; 266:1817-1823 [Medline]
9. Aronow DB, Coltin KL Information technology applications in quality assurance and quality improvement, part 1. Jt Comm J Qual Improve. 1993; 19:402-415
10. ORYX program. JCAHO. http://www.jcaho.org/perfmeas/oryx/oryx_frm.htm (15 June 1998)
11. Jordan HS, Straus JH, Bailit MH Reporting and using health plan performance information in Massachuesetts. Jt Comm J Qual Improve. 1995; 21:167-177
12. Keeler EB, Park RE, Bell RM, Gifford DS, Keesey J Adjusting cesarean delivery rates for case mix. Health Serv Res. 1997; 32:511-528 [Medline]
13. Kotagal UR, Perlstein PH, Atherton HD, Donovan EF Acuity scores as predictors of cost-related outcomes of neonatal intensive care. J Pediatr. 1995; 126:88-93 [Medline]
14. Lowe WW, Ciszek TA, Gallaher KJ Comprehensive computerized neonatal intensive care unit data system including real-time, computer-generated daily progress notes. Pediatrics. 1992; 89:62-66 [Abstract]
15. Ohlsson O, Leeke T, Watson-MacDonell J, Forbes S Data quality assessment for perinatal events in an on-line obstetrical database (O-DB) and a health records database (HR-DB). Pediatr Res. 1997; 41:207 Abstract
16. Keith RD, Beckley S, Garibaldi JM, Westgate JA, Ifeachor EC, Greene KR A multicentre comparative study of 17 experts and an intelligent computer system for managing labour using the cardiotogram. Br J Obstet Gynaecol. 1995; 102:688-700 [Medline]
17. Snowden S, Brownlee KG, Dear PR An expert system to assist neonatal intensive care. J Med Eng Technol. 1997; 21:67-73 [Medline]
18. Jenkins D A practical introduction to databases. Biomed Instrumen Technol. 1989; 23:109-112
19. Escobar GJ, Fischer A, Kremers R, Usatin MS, Macedo AM, Gardner MN Rapid retrieval of neonatal outcomes data: The Kaiser Permanente neonatal minimum data set. Qual Manage Health Care. 1997; 5:19-33
20. Molin J A regional perinatal database in southern Swedena basis for quality assurance in obstetrics and neonatology. Acta Obstet Gynecol Scand. 1997; 164:37-39
21. Dooley SL, Freels SA, Turnock BJ Quality assessment of perinatal regionalization by multivariate analysis: Illinois 1991-1993. Obstet Gynecol. 1998; 89:193-198
22. Maresh M, Beard RW, Combe D, Selection of an obstetric data base for a microcomputer and its use for on-line production of birth notification forms, discharge summaries and perinatal audit. Br J Obstet Gynaecol. 1983; 90:227-231 [Medline]
23. Broekhuizen FF, Fox R Computerized labor and delivery logbook. A first step to a perinatal database in a university-affiliated community hospital. Am J Perinatol. 1985; 2:150-153 [Medline]
24. Main EK Reducing cesarean section rates with data-driven quality improvement activities. Pediatrics. 1999; 103(suppl):374-383
25. Regenstein AC, Main EK Placenta previa: transfusion frequency and the utility of autologous blood donation. Am J Obstet Gynecol. 1997; 176:S144 Abstract
26. Centers for Disease Control and Prevention Prevention of perinatal group B streptococcal disease: a public health perspective. MMWR. 1996; 45:1-33
27. Brook RH, McGlynn EA, Cleary PD Quality of health care: measuring quality of care. N Engl J Med. 1996; 335:996-970
28. AHIMA Data Quality Management Task Force. Issue: data quality management model. American Health Information Management Association (AHIMA). http://www.ahima.org.html (15 June 1998)
29. Rosen LS, Schroeder K, Hagan M, Acord-Szczesny J, Garavaglia M Adapting a statewide patient database for comparative analysis and quality improvement. Jt Comm J Qual Improve. 1996; 22:468-481
30. Krivenko CA, Chodroff C The analysis of clinical outcomes: getting started in benchmarking. Jt Comm J Qual Improve. 1994; 20:260-266
31. Lee KS, Perlman M, Ballantyne M, Elliott I, To T Association between duration of neonatal hospital stay and readmission rate. J Pediatr. 1995; 127:758-766 [Medline]
32. Corn RF Quality control of hospital discharge data. Med Care. 1980; 18:416-426 [Medline]
33. Slagle TA, Le HA Database information: fact or fiction. Pediatr Res. 1991; 29:266 Abstract
34. Lagnado L. Hospitals profit by "upcoding" illness. Wall Street Journal. 1997;April 17:B1
35. Chute CG, Cohn SP, Campbell KE, Oliver DE, Campbell JR The content coverage of classical systems. J Am Med Inform Assoc. 1996; 3:224-233 [Abstract]
36. Kudla KM, Rallins MC. SNOMED: a controlled vocabulary for computer-based patient records. American Health Information Management Association (AHIMA). http://www.ahima.org.html (25 June 1998)
37. Slaughter J, Willner S. A successful methodology for improving the quality of clinical data. American Health Information Management Association (AHIMA). http://www.ahima.org.html (15 June 1998)
38. Prophet S, Hammen C. Coding compliance: practical strategies for success. American Health Information Management Association (AHIMA). http://www.ahima.org.html (15 June 1998)
39. Herman S. Voice recognition and the creation of medical reports. Proceedings of the 1998 Annual HIMSS Conference; February 22-26, 1998; Orlando, FL. 1998;4:25-32
40. Hripcsak G, Friedman C, Alderson PO, DuMouchel W, Johnson SB, Clayton PD Unlocking clinical data from narrative reports: a study of natural language processing. Ann Intern Med. 1995; 122:681-688 [Medline]
41. NPIC, About the National Perinatal Information Center. http://www.npic.org.html (15 June 1998)
42. HCIA. http://www.hcia.com.html (25 June 1998)
43. Horbar JD The Vermont-Oxford Neonatal Network: integrating research and clinical practice to improve the quality of care. Semin Perinatol. 1995; 19:124-131 [Medline]
44. The investigators of the Vermont-Oxford Trials Network database project The Vermont-Oxford Trials Network: very low birth weight outcomes for 1990. Pediatrics. 1993; 91:540-546 [Abstract]
45. Vermont Oxford Network. http://www.vtmednet.org/~m012663/vtoxford.htm (August 8, 1998)
46. Gaynes RP, Martone WJ, Culver DH, Comparison of rates of nosocomial infections in neonatal intensive care units in the United States. National Nosocomial Surveillance System. Am J Med. 1991; 91(suppl):192-196
47. National nosocomial infection surveillance system (NNIS). CDC. http://www.cdc.gov/ncidod/diseases/hip/nnis.html (15 June 1998)
48. The Extracorporeal Life Support Organization. Extracorporeal Life Support at the University of Michigan. http://www.med.umich.edu/ecmo/ELSO_reg.html (28 June 1998)
49. Meyer DM, Jessen ME Results of extracorporeal membrane oxygenation in neonates with sepsis: the Extracorporeal Life Support Organization experience. J Thorac Cardiovasc Surg. 1995; 109:419-425 [Abstract/Full Text]
50. Janik DS, Swarner W, Henriksen KM, Wyman ML Computerized newborn intensive care data recording and reporting: II. an online system. J Pediatr. 1979; 94:328-330 [Medline]
51. Finer NN, Fraser AJ Neonatal data base and automated discharge summary using a personal computer and proprietary software. Pediatrics. 1985; 76:269-273 [Abstract]
52. Wirtschafter DD, Jones KR, Thomas JT Using health care outcomes to improve patient care in the NICU. J Qual Improve. 1994; 20:57-65 [Medline]
53. Myers SA, Gleicher N A successful program to lower cesarean-section rates. N Engl J Med. 1988; 319:1511-1516 [Abstract]
54. Perlstein PH, Atherton HD, Donovan EF, Richardson DK, Kotagal UR Physician variations and ancillary costs of neonatal intensive care. Health Serv Res. 1997; 32:299-311 [Medline]
55. Tyson JE, Younes N, Verter J, Wright LL Viability, morbidity, and resource use among newborns of 501- to 800-g birth weight. JAMA. 1996; 276:1645-1651 [Medline]
56. Horbar JD, Leahy KA An assessment of data quality in the Vermont-Oxford Trials Network database. Controlled Clin Trials. 1995; 16:51-61 [Medline]
57. Johnson RL, Trends in health care and health care systems. Proceedings of the 1998 Annual HIMSS Conference; February 22-26, 1998; Orlando, FL. 1998;1:323-330
58. Kannan R, Bellalouna P, Carroll T, Mikwee L, Vredeveld D, Zak M. Architecting a health care data warehouse: a template for success. Proceedings of the 1998 Annual HIMSS Conference; February 22-26, 1998; Orlando, FL. 1998;1:421-438
59. Drazen E. Why don't we have computer-based patient records? American Health Information Management Association (AHIMA). http://www.ahima.org.html (15 June 1998)
60. Borch K, CPR product hotlist. Health Management Technology. 1996;July:42-44
61. Brazy JE, Langkamp DL, Brazy ND, De Luna RF Do primary care physicians prefer dictated or computer-generated discharge summaries? Am J Dis Child. 1993; 147:986-988 [Medline]
62. Bruno CJ, Wilson M, VanFossen D. You can lead physicians to work stations, but can you make them sign-on? Proceedings of the 1998 Annual HIMSS Conference; February 22-26, 1998; Orlando, FL. 1998;1:243-1254
63. ECRI. Obstetrical data mangement systems. Health Devices. 1997;26:83-122
64. Hierholzer WJ. Health care data, the epidemiologist's sand: comments on the quantity and quality of data. Am J Med. 1991;91:3B-21-6S