Abstract
OBJECTIVE. Studies have documented high rates of inappropriate hospital use in children. We assessed the effectiveness of an audit-and-feedback intervention at reducing inappropriate hospital days on a general pediatric inpatient unit.
METHODS. A prospective observational study, using a before-and-after design, was conducted at a tertiary care pediatric hospital in Canada between March 2005 and August 2006. The appropriateness of all hospital days for all admissions was evaluated by a nurse trained in using a utilization review tool. This tool classifies hospital days as “qualified” or “nonqualified” on the basis of the nature of the inpatient services that are used. Reasons for nonqualified days were classified. The intervention consisted of (1) weekly feedback to attending physicians of which patients were nonqualified and (2) dissemination of summary reports to attending physicians. Comparisons were made between the preintervention (March 2005 to August 2005) and intervention (March 2006 to August 2006) phases.
RESULTS. The intervention was associated with a significantly lower risk of inappropriate hospital days. Of the 7246 hospital days in the 6-month intervention phase, 2413 (33%) were nonqualified versus 3859 (47%) of 8228 hospital days in the 6-month preintervention phase. A total of 7.35 hospital days would have to be reviewed, combined with weekly feedback, to prevent 1 nonqualified hospital day. The 48-hour readmission rate in the intervention phase and preintervention phase was 1.0% and 1.6%, respectively. The proportion of nonqualified days to total hospital days that were attributable to “finishing intravenous antibiotics,” “awaiting tests,” “providing nutrition,” “observation only,” “tapering treatment,” and “teaching” decreased significantly, whereas the “lack of an alternate level of care” increased significantly.
CONCLUSIONS. An audit-and-feedback intervention directed at attending physicians was associated with a lower risk of inappropriate hospital days without an increase in the readmission rate. The utilization review tool also identified processes that impact on inappropriate hospital days.
- hospital use
- hospitalization appropriateness
- health services research
- audit and feedback, quality improvement
Hospital inpatient care constitutes a large proportion of health care costs. There has been a shift of care from the inpatient to ambulatory setting in the past 2 decades to provide more efficient and effective care from both the patient's and the payer's perspectives. Several published studies have documented rates of inappropriate hospital days in pediatric hospitals around the world from 20% to 60%.1–5 These rates depend on the tool being used to assess appropriateness,6–8 population of children, country, and health care system.
Despite an ongoing shift to ambulatory care at our institution and an ongoing evolution of hospitalized care in our region, an increase in inpatient stay and census was observed. Few published studies have documented the effectiveness of interventions in reducing inappropriate hospital days9–14 including only 1 in a pediatric population.9 Audit-and-feedback interventions have been shown to be effective in improving professional practice.15,16 Thus, the objective of this study was to determine whether an audit-and-feedback intervention would reduce the inappropriate inpatient hospital days in a general pediatric inpatient unit.
METHODS
Study Setting
The study setting was the general pediatric inpatient unit at the Hospital for Sick Children, a tertiary care pediatric academic health science center affiliated with the University of Toronto (Toronto, Ontario, Canada). The study protocol was approved by the research ethics board at the Hospital for Sick Children. The general pediatric inpatient unit has a 60-bed capacity on 3 wards. The total hospital bed capacity including ICU beds and subspecialty beds is 320. It is the only freestanding pediatric hospital for the greater Toronto area with a catchment population of 5 million people. At all times, the general pediatric inpatient unit is staffed by 4 to 6 inpatient attendings from a total of 14 full-time and major part-time pediatricians. Attending physicians are on service for a minimum of 4 weeks at a time. General pediatric housestaff including fellows, residents (third year and first year), and medical students are supervised by the attendings. The division has had an active academic hospitalist group since 1995.
The Canadian health care system ensures a government-managed health insurance program funded through taxation dollars. All health care consumers in Canada have universal access to health care, with coverage for all outpatient and inpatient visits, investigations, and therapies. The consumer does not receive a bill. At the Hospital for Sick Children, Pediatricians who attend on the inpatient unit are salaried. Their salary is not linked to volume of inpatient activity. Patient management occurs without intervention from insurance companies or hospital administrators.
Utilization Review Tool
The Medical Care Appropriateness Protocol (MCAP) is a tool that provides information about hospital bed use (www.oakgroup.com). The MCAP criteria were created by a multidisciplinary team of physicians, nurses, and social workers. The process for criterion development included literature review, consultation with experts, and feedback from various stakeholders. The criteria undergo a regular review. The MCAP criteria base the appropriateness of hospitalization on the services of care provided on a day-to-day basis rather than the diagnosis or the acuity of the illness. When the MCAP criteria are applied to a particular day in the hospitalization of a patient, the medical necessity for the patient stay, “qualified” or “nonqualified,” is determined.
Nurse reviewers, who are part of the Quality and Risk Management department at the Hospital for Sick Children, were trained in using the MCAP tool for the inpatient pediatric setting. After their formal training and an initial 1-month implementation trial on the inpatient unit, tracking of inpatient hospital use started in March 2005. Evaluation occurred on each inpatient for each 24-hour hospital period. The process starts with the nurse gathering and assessing clinical information with a focus on the physician's plan of care and capturing the proposed as well as ongoing services or treatments. Relevant information is gathered from various sources, including the patient's medical chart and unit shift reports. Other resources are the service providers, such as the bedside nurse, physicians, discharge planners, nurse practitioners, and nursing leaders, as well as attending clinical handover rounds. After gathering and assessing the clinical information, the nurse reviewer then applies the appropriate MCAP criteria to determine whether the day is qualified or nonqualified. To support the decision-making process, each MCAP criterion has validating elements that identify the components of care to be delivered. Every element within the criterion must be met for the stay to be qualified. A nonqualified stay indicates that the criterion has not been met because 1 or more validating elements have not been fulfilled. For example, a hospital day for a child who has osteomyelitis and is on intravenous antibiotics and does not require monitoring or other care would be nonqualified because he or she could receive the treatment at home with appropriate nursing and ambulatory follow-up. When a nonqualified stay is determined, the reason for the stay is then classified using predetermined reason codes. The reasons codes for nonqualified stay included: awaiting tests, finishing intravenous antibiotics, providing nutrition (eg, observing for feeding tolerance for a child who does not require any other observation or care), observation only, alternative level of care not available (eg, no beds in chronic care facility), tapering treatment (eg, weaning oral steroids or pain medications for a child who requires no other care), teaching (eg, teaching caregivers on medication administration or on tube feeding), short admission (<24 hours), psychosocial/economic (eg, patient is homeless, guardianship issues), administrative (eg, awaiting funding for care), extenuating circumstances (eg, weather conditions, disaster), documentation issues (eg, delay in discharge because discharge paperwork not completed), and consultation not complete (ie, subspecialty consultation not completed and is the only reason for that hospital day).
Study Design and Intervention
This study was a prospective observational study that used a before-and-after design. The study period was from March 2005 to August 2006. Two identical 6-month periods were defined for comparison to avoid seasonal variations in the reasons for admission. The preintervention period was from March 2005 to August 2005, and the intervention period was from March 2006 to August 2006. During the entire study period, all 24-hour hospital periods on the general pediatric inpatient unit for all patients were reviewed by 1 nurse reviewer using the MCAP tool. The data and classification of patient days were recorded and managed on a computer database by using software created by the Oak Group for the MCAP tool.
The intervention began in September 2005. At the beginning, an overview of the MCAP tool was presented by the physician leader. The intervention consisted of a weekly feedback to attending physicians on the inpatient unit of the evaluations of hospital appropriateness of their patients (Fig 1). Communication to attending physicians occurred through the physician lead and nurse reviewer and consisted of (1) which of their patients had nonqualified hospital days, (2) clarification of management plans for patients (eg, additional description of reasons for the current admission, plan, and any barriers to discharge), and (3) monthly reports for the attending physicians who had been on service summarizing the qualified and nonqualified hospital days and a description of nonqualified days by reason codes. Initially, communication to attending physicians occurred in person by the physician lead on a weekly basis, and after 4 months this was switched to e-mail, again on a weekly basis. Monthly reports summarizing the qualified and nonqualified hospital days and a description of nonqualified days by reason codes were presented to nurse leaders at monthly divisional quality improvement meetings.
Audit and feedback process.
Outcomes
The primary outcome was the proportion of total hospital days that were nonqualified. Secondary outcomes included (1) proportion of total hospital days that were nonqualified by reason and (2) readmission rate within 48 hours of discharge.
Blinding
It was not feasible to blind attending physicians given the nature of the intervention. Physicians were aware that data were being collected as part of a utilization audit; however, they were not aware of the study design (ie, preintervention phase and intervention phase). Similarly, the nurse reviewer who collected the data on outcomes was unaware of the study design and as such did not know in which phase (ie, preintervention phase and intervention phase) the reviewed hospital days were.
Statistical Analysis
Descriptive statistics were used to present the following characteristics of the preintervention and intervention phases: number of admissions, number of hospital days, number of reviewed patient days, and age of patients. Comparisons were made between the preintervention period (March 2005 to August 2005) and intervention period (March 2006 to August 2006). A χ2 test or Fisher's exact test, where appropriate, was used compare the proportion of nonqualified to reviewed hospital days, proportion of nonqualified days by reason, and readmission rates between the 2 phases. The relative risk (RR) for having a nonqualified hospital day in the intervention phase compared with the preintervention phase was calculated. In this context, the relative risk is the ratio of the risk of an event (nonqualified days to total hospital days) among the exposed population (intervention phase) to the risk among the unexposed the unexposed (preintervention phase). The number of hospital days that needed to be reviewed, combined with weekly feedback to attending physicians, to prevent 1 nonqualified hospital day was calculated as the inverse of the risk difference between the 2 phases. Here, the risk difference is the difference in the proportion of nonqualified hospital days to total hospital days in the exposed (intervention) versus the unexposed (preintervention) group. The 95% confidence intervals (CIs) were calculated around all point estimates. P < .05 was considered statistically significant.
RESULTS
Tables 1 and 2 provide the characteristics of the 6-month preintervention (March to August 2005) and intervention (March to August 2006) phases. This represented all hospitalizations to the general pediatric inpatient unit during the periods and all days in hospital. There was no statistically significant difference in the age distribution between the 2 phases.
Characteristics of the Preintervention and Intervention Phases
Hospitalizations According to Age and Study Phase
The intervention was associated with a significant reduction in the proportion of nonqualified hospital days. Of the 7246 hospital days in the 6-month intervention phase, 2413 (33%) were nonqualified as were 3859 (47%) of 8228 hospital days in 6-month preintervention phase (RR: 0.71 [95% CI: 0.74–0.68]; P < .0001). There was no significant difference in the hospital readmission rate. The 48-hour readmission rate in the intervention phase and preintervention phase was 1.0% (28 of 1705) and 1.6% (15 of 1489), respectively (RR: 0.63 [95% CI: 0.34–1.2]; P = .11). The number of hospital days that would be needed to be reviewed, combined with weekly feedback, to prevent 1 nonqualified hospital day was 7.35 (95% CI: 7.34–7.37).
Table 3 summarizes nonqualified days by reason in both study phases. The intervention was associated with a significant reduction in the proportion of nonqualified days as a result of waiting for tests, finishing intravenous antibiotics, providing nutrition, observation only, tapering treatment, teaching, and short admissions (<24 hours). The intervention was not associated with a significant change in the proportion of nonqualified days for which the reason was admission for investigations, documentation issues, consultation not complete, and extenuating circumstances. The intervention was associated with a significant increase in the proportion of nonqualified days that were attributed to unavailability of alternative care levels and psychosocial/economic. The greatest RR reduction for nonqualified days was observed when the reason was tapering treatment and finishing intravenous antibiotics.
Nonqualified Days by Reason
DISCUSSION
Hospital stay accounts for a significant cost to the health care system. Furthermore, there are many indirect costs to families as well as risks while in hospital; for example, nosocomial infection and medical error. The intervention in this study consisted of an audit of all hospital days by using a utilization review tool by a trained nurse with feedback directed at attending physicians. This intervention resulted in a significant reduction of inappropriate hospital days in a general pediatric inpatient unit.
Feedback from attending physicians indicated that the intervention was widely accepted and not burdensome with respect to time; however, the intervention required daily evaluation from all inpatient admissions by a trained nurse (data collection) and a physician leader. This is not an insignificant investment in resources. This study found that in combination with a weekly feedback intervention, only 7 hospital days needed to be reviewed to prevent 1 inappropriate hospital stay. Thus, on the basis of the significant costs of inpatient hospital stay, the cost of the intervention is well justified.
Few published studies have documented the effectiveness of interventions in reducing inappropriate hospital days. Two controlled studies in Spain documented a modest benefit to physician-oriented feedback in reducing inappropriate hospital stay12,13 with absolute risk reductions of inappropriate hospital days of 8.2% and 6.4%. Two other controlled studies, including 1 in a pediatric population, used daily preset criteria for hospitalization that was completed by treating physicians as the intervention. The intervention resulted in a significant reduction in inappropriate hospital stay.9,10
Our study documented a high rate of inappropriate hospital days (49.3%) in the preintervention phase. It is important to note that the measured appropriateness of hospitalization is also a function of the instrument being used. Instruments that are more stringent, such as MCAP, are more helpful in utilization management when one is interested in using the information to change the health care processes away from the inpatient toward the ambulatory setting.10 No previous published studies have documented the proportion of inappropriate hospital days in a pediatric general inpatient unit using the utilization review tool used in this study (MCAP). In 1987, Kemper1 documented 13% inappropriate days of 285 reviewed hospital days, and in 1993 Gloor et al2 documented 29% inappropriate hospital days of 251 reviewed hospital days in general pediatric inpatient units. Both studies used the Pediatric Appropriateness Evaluation Protocol and retrospectively evaluated small numbers of general pediatric hospital days.
Inappropriate hospital days do not necessarily mean unnecessary hospital days.2 Inappropriate hospital days may result from factors related to and unrelated to physician practice or a combination of both. As such, we would not expect the audit-and-feedback intervention to have an impact on all causes of inappropriate hospital stay. For example, there may be inadequate alternate levels of care, ambulatory facilities, or home care facilities. Attending physicians would not be able to influence these processes directly; however, the use of this utilization review tool allowed for identification of reasons for inappropriate hospital days. For quality improvement purposes, we also classified reasons for inappropriate hospital stay under more specific reason codes to be able to identify specific processes that accounted for inappropriate hospital stay. Furthermore, reason codes were classified as either actionable (those that were sensitive to practice and that could be modified by the health care team) or nonactionable (those that were not sensitive to practice and depended on the local health care system). This classification facilitated interpretation of data by health care providers and administrators. This information can be used to inform hospital administrators and physicians of the deficiency in the local health care system, which may be unable to support efficient use of inpatient hospitalization.7
The proportion of nonqualified days that were attributed to awaiting tests, admitted for investigations, finishing intravenous antibiotics, observation only, providing nutrition, tapering treatment, and teaching decreased significantly during the intervention phase. Physician practice may have resulted in these decreases. Unavailability of alternate level of care facilities as a reason for inappropriate hospital stay increased significantly. During the second phase of the study, there were no longer any discharge planners on the inpatient wards as a result of an institutional restructuring. This process change may have resulted in delays or even a failure to transfer patients who were appropriate for alternate facilities with lower levels of care.
Several strategies for changing practice have been described.16,17 This study used an audit-and-feedback intervention. This method of affecting change focuses on controlling performance by external stimuli.16 The effectiveness of audit and feedback is greater when the compliance to optimal practice is low and the feedback is intense.15 For facilitation of acceptance of this intervention, several steps were taken. First, an attending physician from the group of inpatient general pediatric physicians was designated as the leader for the intervention. This facilitated communication and buy-in for the process. A benchmark goal for the group was initially set (25% nonqualified days), although individual performance was not assessed. Although feedback was sent on a weekly basis, decisions on admission and discharge were left entirely up to the attending physicians, with no pressure placed on them. Nurses and nurse leaders were also made aware of the utilization audit; however, bedside nurses did not know whether a particular patient's hospital day was qualified or nonqualified and were not provided with monthly summary reports. Nurse leaders were aware of monthly performance measures.
The strengths of this study include the large number of hospital days evaluated, prospective data collection, the use of a standardized tool to assess hospitalization appropriateness, and blinding of outcomes assessors. Several limitations to this study merit mention. It is possible that co-interventions that may have affected hospital appropriateness occurred during the study period. To our knowledge, there were no other interventions that occurred in parallel that may have resulted in a reduction of the number of inappropriate days in hospital. As noted previously, there was a loss of all discharge planners on the inpatient ward in the intervention phase; however, this would bias the results in favor of no effect for the intervention. A control group, which was lacking in this study design, could have reduced biases from potential co-interventions.18 The hospital readmission rate was tracked only for readmissions to the study hospital. It is possible that patients were readmitted to other regional hospitals, although this would be unusual from our experience.
CONCLUSIONS
This study demonstrated that an audit-and-feedback intervention significantly reduced inappropriate hospital use in a tertiary care general pediatric inpatient unit. The use of a utilization review tool was also useful in classifying reasons for inappropriate hospital use. This may help to identify deficiencies in health care practice or processes and inform future interventions aimed at improving appropriate use of hospitalization.
Footnotes
- Accepted October 18, 2007.
- Address correspondence to Sanjay Mahant, MD, FRCPC, Division of Paediatric Medicine, Paediatric Outcomes Research Team, Research Institute, Hospital for Sick Children, University of Toronto, Toronto, Ontario, Canada M5G 1X8. E-mail: sanjay.mahant{at}sickkids.ca
The authors have indicated they have no financial relationships relevant to this article to disclose.
What's Known on This Subject
Several published studies have documented rates of inappropriate hospital days in pediatric hospitals from 20% to 60%. Inappropriate hospital days lead to acute care inefficiency, increased costs, and potential morbidity to the patient. Few published studies have documented the effectiveness of interventions in reducing inappropriate hospital days.
What This Study Adds
In a general pediatric inpatient setting, an audit-and-feedback intervention aimed at attending physicians is associated with a lower risk for inappropriate hospital days.
REFERENCES
- Copyright © 2008 by the American Academy of Pediatrics
In this issue
