This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow E-mail this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My File Cabinet
Right arrow Download to citation manager
Right arrowRequest Permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via CrossRef
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Sharma, V.
Right arrow Articles by Wallace, D. D.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Sharma, V.
Right arrow Articles by Wallace, D. D.
Related Collections
Right arrow Emergency Medicine
Social Bookmarking
 Add to CiteULike   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Facebook   Add to Reddit   Add to Technorati   Add to Twitter  
What's this?

PEDIATRICS Vol. 106 No. 5 November 2000, pp. 1031-1039

Factors Influencing Infant Visits to Emergency Departments

Vidya Sharma, MBBS, MPH*, Stephen D. Simon, PhD*, Janice M. Bakewell, BADagger , Edward F. Ellerbeck, MD, MPH§, Michael H. Fox, ScDparallel , and Dennis D. Wallace, PhD§

From the * Department of Pediatrics, Children's Mercy Hospital, University of Missouri at Kansas City, Kansas City, Missouri; Dagger  Center for Health Information Management and Epidemiology, Missouri Department of Health, Jefferson City, Missouri; § Department of Preventive Medicine, University of Kansas, Kansas City, Kansas; and the parallel  Department of Health Services Administration, University of Kansas, Kansas City, Kansas.


    ABSTRACT
Top
Abstract
Methods
Results
Discussion
Conclusion
References

Objectives.  To follow the 1995 birth cohort of infants, born in the State of Missouri, through their first birthday to: 1) examine their rates of visits to emergency departments (EDs), 2) identify predictors of any ED visit, 3) examine rates of nonurgent ED visits, and 4) identify predictors of nonurgent visits.

Methods.  This was a retrospective population cohort study. Using deterministic linkage procedures, 2 databases at the Missouri Department of Health (DOH; (the patient abstract database and the birth registry database) were linked by DOH personnel. International Classification of Diseases, Ninth Revision-Clinical Modification codes for ED visits were classified as emergent, urgent, or nonurgent by 2 researchers. Eight newborn characteristics were chosen for analysis. Negative binomial regression was used to examine the rates and predictors of both total and nonurgent ED visits.

Results.  There were 935 total ED visits and 153 nonurgent ED visits per 1000 infant years. The average number of visits was .94, with 59% of infants having no visits, 21% having 1 ED visit, and 20% having 2 or more visits.Factors associated with increases in both total and nonurgent ED visits were Medicaid, self-pay, black race, rural region, presence of birth defects, and a nursery stay of >2 days. Significant interactions were found between Medicaid and race and Medicaid and rural regions on rates of ED use and nonurgent use. The highest rate of ED use, 1.8 per person year, was seen in white, rural infants on Medicaid, and the lowest rate (.4 per person year) was seen in urban white infants not on Medicaid. The highest rates of nonurgent use, .3 per person year, were among urban and rural Medicaid infants of both races and among black infants on commercial insurance. The lowest nonurgent rate, .04 per person year, was seen in white urban infants on commercial insurance.

Conclusion.  Infants in the State of Missouri have high rates of ED visits. Nonurgent visits are only a small portion of ED visits and cannot explain large variations in ED usage. Increased ED use by Medicaid patients may reflect continuing difficulties in accessing primary care.  Key words:  infants, children, emergency department, nonurgent, Medicaid.

Large volumes of patients continue to seek care in emergency departments (EDs). The 1996 National Hospital Ambulatory Medical Care Survey (NHAMCS)1 estimated an annual rate of 34.2 visits per 100 persons in the general population. Approximately 60 000 children seek emergency care each day in the United States.2 Children under the age of 15 years have a high rate of utilization at 35.1 per 100 persons and account for 23% of all ED visits. A study on ED use in Ontario, Canada3 found that 35.2% of children 0 to 5 years of age had 1 or more visits in the previous year and an adjusted odds ratio for making an ED visit of 2.45 relative to the 11- to 15-year groups. A breakdown for infants in the 0- to 1-year group was not reported.

Infants under 1 year of age represent a unique subgroup within the 0- to 5-year group. Their medical needs are quite different, ie, a fever in this age group may be more ominous and a respiratory illness may give rise to greater morbidity. Yet data documenting ED use in the first year of life are sparse. The few studies4,5 on the subject are limited in describing the population that actually presented to the ED. Evidence suggests that patterns of ED use are established as early as the first year of a child's life.6 From a societal and policy perspective, it is important to document utilization rates in this population and identify characteristics that may be helpful in predicting utilization.

The primary function of EDs is to treat patients with serious illnesses or injuries, but many ED visits involve care for nonurgent problems.7 Some children use EDs as their usual source for sick care.8 Various estimates regarding nonurgent care in children 0 to 15 years of age range from 15%9 to a high of 61%2 of all ED visits. Children in this age group are thought to have the highest rate of nonurgent care. Reasons for the disparity in estimates seem to be attributable to: 1) a lack of consensus in the literature on the definition of emergent/urgent care,10-12 2) a lack of consistency in the methodology used to determine urgency,13-16 and 3) differing perspectives among physicians, payors, and patients on the urgency of visits.17-19

The decision to seek care in an ED is complex, and its explanation involves the consideration of many factors. Some factors shown to influence choices to seek care through an ED include lack of a primary care provider, restricted office hours, limited access to weekend care, lack of transportation, exposure to violence, lack of a telephone, other barriers to care, and illness thought to be beyond the expertise of the normal caregiver.20 Data in infants regarding urgency of visit are lacking, and estimates derived from predominantly adult or older child data may not be generalizable to this population.

A consistent, reproducible method to collect data on nonurgent visits in infants must be developed to examine the complex issues surrounding nonurgent care for infants in the ED. A population perspective is essential because studies focusing only on patients who visit the ED miss out on important information to be gathered from the patients who do not visit the ED. Large administrative databases have been used to track health care utilization,21 pediatric hospitalization,22 and hospitalization trends23 in children. Strategies for linking of large databases have allowed outpatient procedures to be analyzed in large administrative databases24 as well as classifying avoidable morbidity in infants.25

We designed this study using existing databases to follow a population cohort of infants from birth through their first birthday to describe their ED utilization. In addition, we developed and used a classification for visit urgency based on diagnostic codes from the International Classification of Diseases, Ninth Revision-Clinical Modification (ICD-9-CM) for the ED visit. This procedure allowed us to examine rates of nonurgent ED visits by this cohort. The primary objectives for the study were to: 1) examine the rates of visits to the ED by this cohort of infants, 2) identify predictors of any ED visit, 3) examine the rates of nonurgent ED visits, and 4) identify predictors of nonurgent visits.

    METHODS
Top
Abstract
Methods
Results
Discussion
Conclusion
References

Study Design

This study was a retrospective population cohort study designed to examine ED utilization rates and nonurgent ED utilization rates of infants who were born in the State of Missouri and were Missouri residents during the 1995 calendar year. We used deterministic linkage procedures to construct a longitudinal dataset from 2 databases for this cohort of infants.

Databases

To identify the cohort and study the outcomes of interest, 2 databases at the Missouri Department of Health (DOH) were accessed and linked by the DOH for use in this study: 1) the patient abstract database for the calendar years 1995-1996 containing a total of 87 718 hospital records, and 2) the birth registry database for the calendar year 1995 containing a total of 72 804 records. Figure 1 represents the linkage of the databases.


View larger version (39K):
[in this window]
[in a new window]
 
Fig. 1.   Linkage procedures and results of linking databases by the DOH.

The patient abstract database contains hospital records. Hospitals in the State of Missouri are mandated by law to provide the state with data on every patient encounter in the hospital. Some of these data are sent directly to the state by the hospitals, and some are sent through the hospital association. The database included: 1) all outpatient visits, 2) all inpatient visits, 3) all ED visits, and 4) all transfers of the newborn to a higher level nursery. It excluded: 1) the birth hospitalization record, and 2) records for infants >365 days of age. Variables abstracted from the hospital records included demographic, clinical, procedural, and payment data. Demographic variables included name, age, sex, race, residence, and social security number. Clinical variables included principal diagnoses, additional diagnoses, and E codes for injury. Procedural variables included ICD-9-CM codes or all procedural codes (Current Procedural Terminology-4). Payment variables included all expected sources of payment.

The birth registry database contains all Missouri resident live births in the State of Missouri that are reported to the Bureau of Vital Records. This database included all Missouri resident live births and excluded out-of-state births and military births. Variables abstracted included demographic variables on the infant, mother, and father; clinical variables on the infant such as birth weight; presence of birth defects; diagnoses for any newborn conditions; procedural variables for any procedures performed in the nursery; and payment variables, such as expected sources of payment.

Linkage Procedures

Linkage of the 2 datasets was performed by the DOH. To develop a link between the birth registry and the hospital records dataset, the name fields in both datasets were edited, eg, all letters changed to upper case and blanks and apostrophes removed. The hospital records data were organized into sets based on last name, first name, birth date, and sex, with all records having the same values for each of these variables considered to represent the same infant.

Deterministic linkage procedures26 were followed. This method was judged suitable as the data were of relatively high quality with few ties generated. Using SAS, Version 7 (SAS, Cary, NC), a succession of linkage criteria were implemented attempting to identify the proper birth record for each hospital record. For each criterion used, over 90% of birth records have a unique combination of variables. For example, 99.9% of birth records have a unique combination of child's first name, last name, date of birth, and sex, and 92% are unique if the birth date is dropped.

Linkage began with the most stringent criteria (primary variables: exact first name, last name, birth date, and sex). Records unlinked on these criteria were passed through a series of linkage attempts in which a substitution of 1 variable was made. Secondary variables used were: phonetically simplified last name for exact last name, first 3 letters of first name for exact first name, or transposed month and day of birth for exact birth date. When a substitute variable was introduced, remaining variables were required to match exactly. Tertiary variables were zip code of residence and the name and location of the hospital recorded for the earliest hospital visit. Tertiary variables were used only to discriminate among multiple potential linkages.

After the hospital records were linked to the birth records, all visits except ED visits were excluded from the linked database as were all infants born out-of-state and all military births. All infants who died before discharge from either the birth hospital or a neonatal intensive care nursery were also excluded. All personal identifiers were then removed from the database and a single summary was made for each infant. Linked final databases provided the source of all the analysis and the infant was our unit of analysis.

Classification of Diagnoses for Urgency of Visit

ICD-9-CM codes for the ED visits were classified as emergent, urgent, or nonurgent. Classification of the codes was performed in 4 stages: 1) review of the literature on classification of ED visits, 2) a double-blinded preliminary classification, 3) discussion of disagreements, and 4) arrival at a final classification.

Review of the Literature on Classification of ED Visits

The literature provides no consensus regarding what constitutes a nonurgent visit. Different methods used have been shown to lead to discrepant conclusions as to what is an appropriate visit.16 Even the terminology is inconsistent with use of nonemergency,13,27 nonurgent,7 and inappropriate.9 In a policy statement on "Bona Fide Emergency Defined,"14 the American College of Emergency Physicians included the judgment of the prudent lay person seeking care in assessing appropriateness. In this study we combined this prudent lay-person definition with previously described implicit subjective criteria.17 These criteria consisted of asking the questions: 1) Would the patient's condition have been worse if they had not been seen for 24 hours?, and 2) Did the patient need a history, physical, or other ancillary data to rule out a condition that could have led to a worse outcome if not seen within 24 hours? The goal of the classification was to identify groups of patients of decreasing severity in each category as previously described.28

An emergent diagnosis was defined as a diagnosis that is life- or limb-threatening or a potential critical condition, which could worsen if not treated promptly. These diagnoses require immediate attention (eg, meningitis). An urgent diagnosis was defined as one that is not immediately life-threatening, but, which, if untreated, may progress to an emergent condition. Patients in this category may require examination or ancillary data to rule out conditions, which if untreated within 24 hours may lead to a worse outcome (eg, acute bronchiolitis). A nonurgent diagnosis was defined as one that could be seen and managed in a primary care setting 24 hours later without harm to the patient (eg, upper respiratory tract infection). In cases where more than 1 diagnosis was recorded for the visit, the first 3 diagnoses were categorized and the highest level of severity was used in the classification.

A preliminary double-blinded classification was performed by a pediatric emergency physician and a pediatric emergency medicine fellow. All diagnoses occurring >20 times in the database were selected for classification. All excluded diagnoses were also reviewed by a clinician to ensure that no rare but important diagnosis (ie, meningitis) was inadvertently excluded. A total of 208 different diagnoses were classified.

Disagreements were discussed between the 2 physicians to arrive at a consensus. When disagreements persisted, a final classification was obtained by using a third pediatric ED physician, this physician unaware of each physician's preliminary classification, picked 1 of the 2 choices for a final classification.

Outcome Variables

Primary outcome variables were: 1) the total rate of ED use, and 2) the rate of nonurgent ED use.

Independent Variables

Based on a literature review, 8 newborn characteristics that might account for variations in infant utilization of EDs were chosen for analysis. All variables were selected before any data analysis was performed. Variables were:

  1. 5-minute Apgar scores categorized into 2 categories: 8 or more and 7 or less. Apgar scores were designed to monitor transition from fetal to infant life but have been used as a proxy indicator of perinatal asphyxia in infants.29
  2. Insurance status at birth categorized into 3 groups: commercial, Medicaid, and self-pay groups.
  3. Sex.
  4. Race divided into 2 categories: white and black. Very small numbers of other races were excluded from the analysis.
  5. Birth weights were categorized into 3 groups consistent with those used in the literature: normal birth weight (>2500 g), low birth weight (1500-2500 g), and very low birth weight (<1500 g).
  6. Total length of stay in all nurseries to include transfer to a higher level nursery, were categorized into 5 groups: 1 to 2 days, 3 to 4 days, 5 to 7 days, 8 to 14 days, and >15 days.
  7. The place of residence was classified into 3 regions: major metro, minor metro, and nonmetro (rural) regions, based on existing Missouri regional planning commission areas.30
  8. The presence of birth defects. An infant was considered to have birth defects if any congenital anomalies (740-759 ICD-9-CM codes) were recorded in the birth record. We excluded some that we considered to be minor anomalies examples were: specified congenital anomalies of the lacrimal passages, accessory auricle, patent ductus arteriosus in infants (<2500 g or 37 weeks' gestation), absence/hypoplasia of the umbilical artery, tongue tie, Meckel's diverticulum, undescended testicle (if infant <2500 g or <37 weeks' gestation), polydactyly, cervical rib, other specified anomalies of the skin, and unspecified anomaly of the integument.

Statistical Analysis

Our modeling strategy started with simple descriptive statistics using the infant as our unit of analysis. We then examined single variable models, considered multivariable models, and finally interaction models. Our objective was to find a model that gave good predictions of total and nonurgent ED visits.

Our analysis compared annual rates of ED use in the first year of life for this population cohort. Rates were calculated using person year as a denominator. Negative binomial regression was used to predict rates of ED visits and nonurgent ED visits. A negative binomial model allows for extra variation, compared with a Poisson regression model.31 This extra variation could occur, for example, if we assume that each infant has a unique random rate of ED visits. Candidate predictor variables for these models included: Apgar scores, insurance status at birth, race, sex, birth weight, length of stay in the nursery, region of residence, and the presence of birth defects. We started with single variables, then added potential covariates sequentially to the model beginning with the most significant ones until all covariates were significant at P < .05. In the final model, interactions involving Medicaid, which seemed important from our initial descriptive analysis, were included. All interactions that were included were significant at P < .05. We conducted the statistical analysis using STATA, Version 6.0 (STATA, College Station, TX).

    RESULTS
Top
Abstract
Methods
Results
Discussion
Conclusion
References

Linkage Results

Of 87 718 hospital visits for which linkage was attempted by the DOH, 82 692 (94.3%) were linked to the birth registry and 5026 visits (6%) could not be linked (Fig 1). Linkage rates for ED visits were 94%. Successful linkage rates in the literature linking large administrative databases have ranged from 91%32 to 95%.24 A methodology combining deterministic and probabilistic linkage methods resulted in a 99% linkage.33 Ninety-two percent of linkages were made based on an exact match on the primary variables or an exact match on primary variables with a tie-breaking match on tertiary variables. Of the 82 692 hospital visits a total of 17 154 of inpatient and outpatient visits, visits by military infants, and those born out-of-state were excluded, resulting in a final database of 65 538 ED visits. Of the 72 804 birth records that were linked, a total of 2761 records were excluded because they were military births, births occurring out-of-state, or because the infants died before discharge from the hospital. Of the population of 70 043 births, 29 082 infants (41%) were in the final database and noted to have 1 or more ED visits. The remaining 40 961 infants were not in the final database and were presumed to have had no ED visits.

Agreement on Classification of Diagnoses

The database contained a total of 1414 separate diagnoses, of which 208 separate diagnoses occurred 20 or more times in the database. Consequently, project personnel classified only 208 diagnoses, relative to the urgency of visits. Visits were rated based on the highest level of urgency for any of up to 3 diagnoses recorded for each visit. When the final classification was applied to the first 3 ICD-9-CM codes of each visit, a total of 61 924 visits (94.5%) were categorized, based on 1 of these 208 diagnoses. Table 1 shows the agreement among the researchers in classifying these diagnoses as emergent, urgent, or nonurgent. The 2 researchers agreed on the classification of 172 diagnoses: they classified a total of 29 diagnoses as emergent, 105 diagnoses as urgent, and 38 diagnoses as nonurgent. There was disagreement in 36 diagnoses, which accounted for 9% of the total visits that were classified. Of these 36 diagnoses, the final classification resulted in a higher level of urgency being selected in 19 separate diagnoses and a lower level of urgency in 17 diagnoses. The final classification of the 208 diagnoses resulted in 9% of the visits being classified as emergent, 74% as urgent, and 17% as nonurgent. Examples of emergent diagnoses included meningitis, septicemia, cardiac arrest, and intussusception. Examples of urgent diagnoses included acute bronchiolitis, pneumonia, and noninfectious gastroenteritis. Nonurgent diagnoses included thrush, impacted cerumen, umbilical hernia, and impetigo. Examples of discordant diagnoses were "nasal and sinus disease not elsewhere classified," "unknown cause morbidity and mortality," and "acute nasopharyngitis."

                              
View this table:
[in this window]
[in a new window]
 

TABLE 1
Agreement Between Two Physicians on Classification of the Level of Urgency of 208 ED Diagnoses

Admission to the hospital resulted from 16% of visits for conditions classified as emergent, 6% of visits classified as urgent, and .8% of visits classified as nonurgent. There was an 83% agreement between the 2 researchers on the categories of diagnosis (n = 208). Agreement beyond that expected by chance (weighted kappa ) was .8 (95% confidence interval [CI]: .72-85). This level is considered substantial.34,35

Description of the Cohort

Characteristics of the cohort are provided in Table 2. Of infants in this cohort, 51% were male, and 83% were white. There were 39% who had Medicaid coverage at birth, 56% had commercial insurance, and 5% were self-pay; 8% of infants on Medicaid had a subsequent commercial insurance ED visit and 6% had a subsequent self-pay visit.

                              
View this table:
[in this window]
[in a new window]
 

TABLE 2
Description of the Cohort of 70 043 Infants Born in the State of Missouri in 1995

Rates of Visits to the ED

Of this cohort, 59% had no visits, 21% had 1 ED visit, and 20% had 2 or more visits. There were 935 total ED visits per 1000 infant years. The highest rates of 1.8 per person year were seen in Medicaid white infants living in rural areas. The lowest rates were noted in infants who were on commercial insurance, were white, and lived in urban areas (.4 per person year).

Predictors of Rates of ED Visits

Factors that were found to be significant (P < .001) predictors of ED use in the negative binomial regression model included insurance at birth, race, rural region of residence, the presence of birth defects, and length of stay in the nursery as shown in Table 3. Most of the very low birth weight infants stayed >2 weeks in the nursery. This colinearity makes it difficult to disentangle the effect of very low birth weight and extended length of stay in these models. None of the other variables showed as much correlation.

                              
View this table:
[in this window]
[in a new window]
 

TABLE 3
Factors Found Significant in the Multivariate Negative Binomial Regression Model Comparing Incident Rate Ratios on Total Visits to the ED*

Two interactions, Medicaid/region and Medicaid/race, were found to be significant at (P < .001) and were retained in the final model. These interactive effects are shown in Fig 2. When adjusted for birth defects, length of stay in the nursery and birth weight, rates for black and white infants on Medicaid were similar (1.7 and 1.8), but for infants in the commercial and self-pay groups, race effects seemed stronger with a 70% elevation in rates for black infants. Similarly, when adjusted for birth weight, length of stay in the nursery, and birth defects, the rates for infants on Medicaid stayed fairly similar (1.7 and 1.8) between the rural and urban areas, but for infants on commercial insurance, region effects were stronger and the rate of ED visits was 30% higher in the rural areas than in the urban areas (.4-.7). Infants in the self-pay group had a 20% higher rate in rural areas.


View larger version (14K):
[in this window]
[in a new window]
 
Fig. 2.   Interactions for Medicaid by race and Medicaid by region on total rates of ED visits adjusted for birth weight, birth defects, and length of stay in the nursery. The interactions significant at (P = .001).

Rates of Nonurgent ED Visits

Of the total cohort (n = 70 043), 12% of infants made at least 1 nonurgent visit. Of all infants making any ED visits (n = 29 082), 29% made at least 1 nonurgent visit. The rate of nonurgent ED visits was 153 per 1000 infant years. Similar high rates of .3 per person year were noted among all Medicaid infants regardless of race or region of residence. Black infants with commercial insurance, living in rural areas, and self-pay black infants from urban areas had similar high rates. The lowest rate of .04 per person year was noted in white infants who were on commercial insurance and who lived in urban areas.

Predictors of Nonurgent Visits

Factors that were found to be significant (P < .001) predictors of nonurgent ED use in the negative binomial regression model included insurance status at birth, race, rural region of residence, and the presence of birth defects as shown in Table 4. The effect of the length of stay group 3 to 4 days was significant at (P = .004) and the length of stay group 5 to 7 days was significant at (P = .001). The effect of 2 length of stay groups (8-14 days and >2 weeks) was not statistically significant (P = .08 and P = .16). This is probably because of the small sample size in these groups, but the effect was consistent with the other length of stay groups and, therefore, included in the final model.

                              
View this table:
[in this window]
[in a new window]
 

TABLE 4
Factors Found Significant in the Multivariate Negative Binomial Regression Model Comparing Incident Rate Ratios on Nonurgent Visits to the ED

Several interactions, such as Medicaid/region and Medicaid/race, were found to be significant at (P < .001) and were retained in the final model. These interactive effects are shown in Fig 3. When adjusted for birth weight, birth defects, and length of stay in the nursery, rates for black and white infants on Medicaid were similar at .3, but self-pay black infants had a 20% elevation (.1 vs .3), and a 10% elevation (.1 vs .2) was noted among black infants on commercial insurance. The interactive effects on region were less marked; when adjusted for birth weight, length of stay in the nursery, and birth defects, the rates for infants on Medicaid and commercial insurance showed a 6% elevation, and the self-pay group had a 4% elevation in rates in rural compared with urban areas.


View larger version (15K):
[in this window]
[in a new window]
 
Fig. 3.   Interactions for Medicaid by race and Medicaid by region on nonurgent rates of ED visits adjusted for birth weight, birth defects, and length of stay in the nursery. The interactions significant at (P = .001).

    DISCUSSION
Top
Abstract
Methods
Results
Discussion
Conclusion
References

This study demonstrates that infants in the State of Missouri have high rates of ED visits. Of infants in this population cohort, 41% made at least 1 ED visit, and 1 in 5 infants born in the state visited the ED 2 or more times in the first year of their life. Infants <1 year of age seem to have rates 3 times higher than any other previously reported category in the NHAMCS. Because this population seems to have unique characteristics and high rates of utilization, consideration should be given to tracking and reporting these data separately in future surveys.

Reliability and Validity of the Classification Scheme

Our study demonstrated a smaller percentage (17%) of nonurgent visits than some studies,11 but was consistent with other studies.9 We used the ICD-9-CM patient diagnostic coding system for classifying the severity of the visit. Presentation of trivial and serious diseases in infants may be similar, using presenting symptoms instead of discharge diagnosis may have resulted in a different classification of severity, but this type of data are not always consistently or reliably available. ICD-9-CM codes may not always be reliable or valid36 and may be limited by errors.37-39 Over time the documentation of these codes seems to have become more reliable. A recent study,22 which reviewed the first ICD-9-CM code compared with the first written discharge diagnosis in children, showed a 97% concordance. This validation of ICD-9-CM codes has been demonstrated in other studies.40,41 We sought to minimize some of the geographic and time biases in coding by making our comparisons over a relatively short period (1 year) and restricted to a small area (Missouri). Another measure of the validity of our classification was the proportion of children who were admitted. Admissions decreased with the severity of the category (from 16% for emergent to .8% for nonurgent visits) consistent with other studies.42-44 In contrast to other studies,15,45,46 we were able to get a substantial agreement on classification among the researchers. We believe that we were able to achieve better agreement by studying a well-defined group with a narrow age range (0-1 year). Most of the visits in this age group consisted of only a few diagnoses, which made classification simpler.

The strengths of this study include: 1) this is a population-based study measuring ED utilization based on a defined number of infants born in the state, 2) person level data were used to calculate rates of ED visits, 3) classification of severity was checked for reliability and validity by interrater reliability and rates of admission to the hospital, 4) this classification of severity allowed for the examination of patients with nonurgent visits to the ED as a subgroup, and 5) multivariable analyses allowed for adjustment of several covariates.

Caveats of this study include: 1) this is a study of associations rather than of cause and effect; 2) limitations of linking large databases apply, some patients presumed to have no visits may not have been linked, but given our high rate of linkage this number is likely to be small; 3) reliability and validity of the ICD-9-CM codes were not checked by a chart review; and 4) we only evaluated the patients' insurance status at birth.

Effect of Medicaid

The effect of Medicaid on ED utilization by children has been controversial. Some studies show increased utilization for sick care,47,11 whereas other studies have not found it to be a significant predictor.48-50 One study showed an increased utilization by commercially insured patients for minor illness and injuries, compared with self-pay and Medicaid.51

In this study, Medicaid coverage was the most important predictor of both the total and nonurgent rates of ED visits. In the entire cohort, patients on Medicaid made more visits (1.7) than did self-pay (.7) and patients on commercial insurance (.4). This is consistent with the negative binomial regression model, which showed a threefold rate for ED visits after adjusting for confounding variables. The effect for nonurgent visits is stronger than for total visits, which indicates that Medicaid has an effect on nonurgent visits above and beyond it's effect on total visits.

The data used in this study were from 1995. At that time Medicaid in Missouri was a state-run program, it was mostly a fee-for-service model with a managed care demonstration project in Jackson County. In late 1995 MC Plus, a managed care model, was put into place in some regions of the state. In 1997 Missouri expanded MC Plus to a statewide Medicaid managed care model. Eligibility for Medicaid has expanded to 225% of poverty and the numbers of Medicaid recipients in Missouri has shown a greater than twofold increase in the last decade. In 1998 there was a reduction in Medicaid enrollment attributed to the impact of Welfare reform. At the end of 1998 the State Child Health Insurance Program was started in Missouri resulting in a further expansion in Medicaid enrollment. We would expect that these expansions, which should result in reducing financial barriers and improving access, would lead to decreases in ED utilization. But the impact of Medicaid managed care on ED visits has been mixed with some studies documenting a decrease52-55and others showing no change.10,22,56 It seems that managed care alone may not affect utilization patterns in Medicaid patients. The structure of the delivery system with the availability of after-hours care and the impact of discontinuous insurance on access to and continuity of care need to be considered.

Nonurgent ED visits may represent a failure of appropriate primary care. It is important when policy strategies, such as managed care, are put in place that, along with rates of ED visits, nonurgent ED visits also be monitored to see whether the strategy results in keeping sick patients out of the ED. People generally use EDs because they consider them appropriate sites; they are convenient and other sources of care cannot be used or are inaccessible at the time they are needed. We did not analyze the time of the ED visit or the day of the week that the visit occurred; such analyses may have helped to further examine the issues of timely access to care for poor children and children on Medicaid. Recent changes in Medicaid managed care generally have not been designed to increase access to weekend and evening coverage. The growth of after-hours clinics and urgent care centers for commercially insured patients suggest that patients are not willing to wait for appointments when faced with a sick child. The question then remains as to where is the best place and who should provide these acute care nonurgent services.

One solution may be to use EDs. Separate lower acuity areas have been designed in some EDs to see nonurgent patients. One study57 estimated that the true costs of nonurgent care in the ED are very low and a cost-savings of a referral to physicians' offices on nights, weekends, and holidays would occur only if the marginal cost of providing these services was <$25 in physicians' offices. The single mother or the executive who uses the ED for their child's nonurgent care to avoid taking time off from work may indeed be using resources efficiently with a value not measured by looking at costs alone. With the expansion of information systems, it is easier to arrange follow-up and for primary care physicians to be informed about their patient's ED visit, thus preserving continuity of care.

In addition, we found some significant interactions with Medicaid and race and Medicaid and region on both the overall rates of utilization and nonurgent use. Medicaid seems to have attenuated regional and racial differences in both the total and nonurgent rates of ED visits. In this study Medicaid patients of both races had similar visit rates and rates of nonurgent use, but among the non-Medicaid groups, there seemed to be a stronger race effect with black patients having higher visit rates and nonurgent use relative to white infants.

Medicaid patients had similar total and nonurgent rates of ED use in both urban and rural areas, but the commercial group and self-pay groups seemed to show stronger regional effects. Rural infants who were either self-pay or who had commercial insurance had an elevation in their rates of nonurgent use, once again raising the issues of barriers that are greater in rural areas.

Poverty is more prevalent in rural areas and among blacks, and this increased utilization by self-pay rural and black patients may, in fact, represent utilization by a group for whom the only option would be care in the ED. In addition, persons in rural areas have less total access to care and even if a family does have insurance, health care facilities other than EDs may be unavailable.

    CONCLUSION
Top
Abstract
Methods
Results
Discussion
Conclusion
References

This study demonstrated high rates of ED utilization by this cohort of infants. Total rates of utilization and nonurgent use were affected by predisposing factors, such as region and race; enabling factors, such as Medicaid; and need factors, such as extended length of stay in the nursery and birth defects. Nonurgent visits were only a small portion of total ED visits and cannot explain large variations in ED usage. In this cohort Medicaid seemed to decrease regional and racial differences in utilization compared with the commercial and self-pay groups, but it does not seem to have created the network of primary care services or the sustained educational programs that need to occur for reductions in ED visits.56 It seems, therefore, that we have a system in place that helps in attenuating predisposing factors; we need now to design systems with the elasticity to be responsive to patients' perceptions of emergencies, while providing continuity of care.

    ACKNOWLEDGMENTS

This article was submitted by Vidya Sharma to the Department of Preventive Medicine and the Faculty of the Graduate School of the University of Kansas in partial fulfillment of the requirements for the degree of Master of Public Health.

We thank Grace Holmes, MD (Chair of Thesis Committee).

We also thank Robert D. Schremmer, MD, for his help in the classification of ED visits, and Barbara Nelson and Rachael Burke for their invaluable secretarial assistance.

    FOOTNOTES

Received for publication Jun 18, 1999; accepted Jan 18, 2000.

Reprint requests to (V.S.) the Children's Mercy Hospital, 2401 Gilham Road, Kansas City, MO 64108. E-mail: vsharma{at}cmh.edu

    ABBREVIATIONS

ED, emergency department; NHAMCS, National Hospital Ambulatory Medical Care Survey; ICD-9-CM, International Classification of Diseases, Ninth Revision-Clinical Modification; DOH, Missouri Department of Health; CI, confidence interval.

    REFERENCES
Top
Abstract
Methods
Results
Discussion
Conclusion
References
  1. Schappert SM Ambulatory care visits to physician offices, hospital outpatient departments, and emergency departments: United States, 1996. National Center for Health Statistics. Vital Health Stat 1998; 13:134
  2. McCaig LF, Stussman BJ. National Hospitals Medical Care Survey: 1996 Emergency Department Summary, Advance Data From Vital and Health Statistics. Hyattsville, MD: National Center for Health Statistics; 1997. No 245
  3. Brown EM, Goel V Factors related to emergency department use: results from the Ontario Health Survey, 1990. Ann Emerg Med 1994; 24:1083-1091 [Medline]
  4. Feigelman S, Duggan AK, Bazell CM, Correlates of emergency room utilization in the first year of life. Clin Pediatr 1990; 29:698-705
  5. Paneth N, Bell D, Stein REK Emergency room utilization in the first fifteen months of life: a randomized study. Pediatrics 1979; 63:486-490 [Abstract/Free Full Text]
  6. Fosarelli PD, DeAngelis C, Mellits ED Health services use by children enrolled in a primary care clinic: a longitudinal perspective. Pediatrics 1987; 79:196-202 [Abstract/Free Full Text]
  7. Nadel V. Emergency departments: unevenly affected by growth and change in patient use. Washington, DC: US General Accounting Office, Human Resources Division; 1993. Publication GAO/HRD 93-94
  8. St Peter R, Newacheck P, Halfon N Access to care for poor children: separate and unequal. JAMA 1992; 267:2670-2674
  9. Buesching DP, Jablonski A, Vesta E, Inappropriate emergency department visits. Ann Emerg Med 1985; 14:672-676 [CrossRef][Medline]
  10. Glotzer D, Sater A, Socolar D, Weitzman M Prior approval in the pediatric emergency room. J Pediatr 1991; 88:674-680
  11. De Angelis C, Fosarelli P, Duggan AK Use of the emergency department by children enrolled in a primary care clinic. Pediatr Emerg Care 1985; 1:61-65 [Medline]
  12. Mayefsky JH, El-Shinaway Y, Kelleher P Families who seek care for the common cold in the emergency department. J Pediatr 1991; 11:933-934
  13. Jacobs AR, Gavett JW, Weisenger MA Emergency department utilization in an urban community. JAMA 1971; 216:307-312 [Abstract/Free Full Text]
  14. American College of Emergency Physicians. Policy Statement: Bona Fide Emergency Defined. Dallas, TX: American College of Emergency Physicians; 1982. Executive Summary and Comments Adopted by the ACEP Board of Directors, October 23, 1982
  15. Lowe RA, Bindman AB Judging who needs emergency department care: A prerequisite for policy-making. Am J Emerg Med 1997; 15:133-136 [CrossRef][Medline]
  16. Lowe RA, Bindman AB, Ulrich SK, Refusing care to emergency department patients: Evaluation of published triage guidelines. Ann Emerg Med 1994; 23:286-293 [Medline]
  17. Hunt RC, Dehart KL, Allison EJ Patient and physician perception of need for emergency medical care: a prospective and retrospective analysis. Am J Emerg Med 1996; 14:635-639 [CrossRef][Medline]
  18. Gill JM, Reese CL, Diamond JJ Disagreement among health care professionals about the urgent care needs of emergency department patients. Ann Emerg Med 1996; 28:474-478 [CrossRef][Medline]
  19. Gill JM, Riley AW Non-urgent use of hospital emergency departments: urgency from the patient's perspective. J Fam Pract 1996; 42:491-496 [Medline]
  20. Kellerman AL Non-urgent emergency department visits: meeting an unmet need. JAMA 1994; 271:1953-1954 [Abstract/Free Full Text]
  21. Lozano P, Fishman P, VonKorff, et al Health care utilization and cost among children with asthma who were enrolled in a health maintenance organization. Pediatrics 1997; 99:757-764 [Abstract/Free Full Text]
  22. Gadomski A, Jenkins P, Nichols M. Impact of a Medicaid primary care provider and preventive care on pediatric hospitalizations. Pediatrics. 1998;101(3). URL: http://www.pediatrics.org/cgi/content/full/101/3/e1
  23. Perrin JM, Homer CJ, Berwick DM, Variations in the rates of hospitalization of children in three urban communities. N Engl J Med 1989; 320:1183-1187 [Abstract]
  24. Roos LL, Walld R, Wajda A, Bond R, Hartford K Record linkage strategies, outpatient procedures, and administrative data . Med Care 1996; 34:570-582 [CrossRef][Medline]
  25. McConnochie KM, Roghmann KJ, Liptak GS Avoidable morbidity in infants: a classification based on diagnoses in administrative databases. Med Care 1997; 35:237-254 [CrossRef][Medline]
  26. Roos LL, Wajda A Record linkage strategies: estimating information and evaluating approaches. Methods Inf Med 1991; 30:117-123 [Medline]
  27. Haddy RI, Schmaler ME, Epting RJ Nonemergency emergency room use in patients with and without primary care physicians . J Fam Pract 1987; 24:389-392 [Medline]
  28. Selby JV, Fireman MA, Swain MS Effects of co-payment on use of the emergency department in a health maintenance organization. N Engl J Med 1996; 334:635-641 [Abstract/Free Full Text]
  29. Hall RT, Hall FK, Daily DK High dose phenobarbital therapy in term newborn infants with severe perinatal asphyxia: a randomized, prospective study with a three-year follow-up. J Pediatr 1998; 132:345-348 [CrossRef][Medline]
  30. Missouri Department of Health, Center for Health Information Management and Epidemiology (CHIME). Missouri Vital Statistics 1997. Jefferson City, MO: Missouri Department of Health, Center for Health Information Management and Epidemiology; 1998. Publication 4.44
  31. McCullagh P, Nelder JA. Generalized Linear Models. 2nd ed. New York, NY: Chapman and Hall; 1989:198-200
  32. Waien SA Linking large administrative databases: a method for conducting emergency medical services cohort studies using existing data. Acad Emerg Med 1997; 4:1087-1095 [Medline]
  33. Herrchen B, Gould JB, Nesbitt TS Vital statistics linked birth/infant death and hospital discharge record for epidemiological studies. Comput Biomed Res 1997; 30:290-305 [CrossRef][Medline]
  34. Landis JR, Koch GG The measurement of observer agreement for categorical data. Biometrics 1977; 33:159-174 [CrossRef][Medline]
  35. Feinstein AR. Clinical Epidemiology: The Architecture of Clinical Research. Philadelphia, PA: WB Saunders; 1985:186
  36. Motheral BR, Fairman KA The use of claims databases for outcomes research: rationale, challenges, and strategies. Clin Ther 1997; 19:346-366 [CrossRef][Medline]
  37. Romano PS, Luft HS. Getting the most out of messy data: problems and approaches for dealing with large administrative data sets. In: Grady ML, Schwartz HA, eds. Medical Effectiveness Research Data Methods: Summary Report. Rockville, MD: US Department of Health and Human Services, Public Health Service, Agency for Health Care Policy and Research; 1992:57-75. AHCPR publication no. 92-0056
  38. Iezzoni LI. Risk adjustment for medical outcome studies. In: Grady ML, Schwartz HA, eds. Medical Effectiveness Research Data Methods: Summary Report. Rockville, MD: US Department of Health and Human Services, Public Health Service, Agency for Health Care Policy and Research; 1992:83-97. AHCPR publication no. 92-0056
  39. Lloyd SS, Rissing JP Physician and coding errors in patient records. JAMA 1985; 254:1330-1336 [Abstract/Free Full Text]
  40. Malenka DJ, McLerran D, Roos N, Fisher ES, Wenneberg JE Using administrative data to describe casemix: a comparison with the medical record. J Clin Epidemiol 1994; 47:1027-1032 [CrossRef][Medline]
  41. Quam L, Ellis L, Venus P, Clouse J, Taylor CG, Leatherman S Using claims data for epidemiologic research. Med Care 1993; 31:498-507 [CrossRef][Medline]
  42. Albin SL, Wassertheil-Smoller S, Jacobson S, Evaluation of emergency room triage performed by nurses. Am J Public Health 1975; 65:1063-1068 [Free Full Text]
  43. Berman DA, Coleridge ST, McMurry TA Computerized algorithm-directed triage in the emergency department. Ann Emerg Med 1989; 18:141-144 [CrossRef][Medline]
  44. Brillman JC, Doezema D, Tandbert D, Triage: limitations in predicting need for emergent care and hospital admission. Ann Emerg Med 1996; 27:493-508 [CrossRef][Medline]
  45. O'Brien GM, Shapiro MF, Woolard RW, Inappropriate emergency department use: a comparison of three methodologies for identification. Acad Emerg Med 1996; 3:252-257 [Medline]
  46. O'Brien GM, Shapiro MJ, Fagan MJ, Woolard RW, O'Sullivan PS, Stein MD Do internists and emergency physicians agree on the appropriateness of emergency department visits? J Gen Intern Med 1997; 12:188-191 [CrossRef][Medline]
  47. Wood DL, Haywoard RA, Corey CR et al Access to medical care for children and adolescents in the United States. Pediatrics 1990; 86:666-673 [Abstract/Free Full Text]
  48. Halfon N, Newacheck PW, Wood DL, Routine emergency department use for sick care by children in the United States. Pediatrics 1996; 98:28-34 [Abstract/Free Full Text]
  49. Cristoffel KK, Garside D, Tokich T Has Medicaid promoted needless pediatric emergency department use. Pediatr Emerg Care 1985; 1:22-27 [CrossRef][Medline]
  50. Weitzman M, Mooman MS, Messenger KP An after hours pediatric walk-in clinic for an entire urban community: utilization and effectiveness of follow-up care. Pediatrics 1980; 65:964-970 [Abstract/Free Full Text]
  51. Rubin MA, Bonnin MJ Utilization of the emergency department by patients with minor complaints. J Emerg Med 1995; 13:839-842 [CrossRef][Medline]
  52. Hurley RE, Fruenel DA, Taylor DE Gatekeeping the emergency department: impact of a Medicaid primary care management program. Health Care Manage Rev 1989; 14:63-71 [Medline]
  53. Bonham GE, Barbara GM Use of health care before and during citicare. Med Care 1987; 25:111-119 [Medline]
  54. Franco SM, Mitchell CK, Buzon RM Primary care physician access and gatekeeping: a key to reducing emergency department use. Clin Pediatr 1997; 36:63-68 [Abstract/Free Full Text]
  55. Rosenbach ML, Irvin C, Coulam RF Access for low income children: is health insurance enough? Pediatrics 1999; 103:1167-1174 [Abstract/Free Full Text]
  56. Gadomski AM, Perkis V, Horton L, Diverting managed Medicaid patients form pediatric emergency department use. Pediatrics 1995; 95:170-178 [Abstract/Free Full Text]
  57. Williams RM The costs of visits to emergency departments. N Engl J Med 1996; 334:642-646 [Abstract/Free Full Text]

Pediatrics (ISSN 0031 4005). Copyright ©2000 by the American Academy of Pediatrics

Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Facebook Facebook   Add to Reddit Reddit   Add to Technorati Technorati   Add to Twitter Twitter    What's this?


This article has been cited by other articles:


Home page
Arch Pediatr Adolesc MedHome page
A. D. Racine, E. M. Alderman, and J. R. Avner
Effect of Telephone Calls From Primary Care Practices on Follow-up Visits After Pediatric Emergency Department Visits: Evidence From the Pediatric Emergency Department Links to Primary Care (PEDLPC) Randomized Controlled Trial
Arch Pediatr Adolesc Med, June 1, 2009; 163(6): 505 - 511.
[Abstract] [Full Text] [PDF]


Home page
Eur J Public HealthHome page
C. Pileggi, G. Raffaele, and I. F. Angelillo
Paediatric utilization of an emergency department in Italy
Eur J Public Health, October 1, 2006; 16(5): 565 - 569.
[Abstract] [Full Text] [PDF]


Home page
Am. J. Public HealthHome page
T. J. Johnson, M. Rimsza, and W. G. Johnson
The Effects of Cost-Shifting in the State Children's Heath Insurance Program
Am J Public Health, April 1, 2006; 96(4): 709 - 715.
[Abstract] [Full Text] [PDF]


Home page
J. Nutr.Home page
J. T. Cook, D. A. Frank, S. M. Levenson, N. B. Neault, T. C. Heeren, M. M. Black, C. Berkowitz, P. H. Casey, A. F. Meyers, D. B. Cutts, et al.
Child Food Insecurity Increases Risks Posed by Household Food Insecurity to Young Children's Health
J. Nutr., April 1, 2006; 136(4): 1073 - 1076.
[Abstract] [Full Text] [PDF]


Home page
PediatricsHome page
E. Y. Friedlaender, D. M. Rubin, E. R. Alpern, D. S. Mandell, C. W. Christian, and E. A. Alessandrini
Patterns of Health Care Use That May Identify Young Children Who Are at Risk for Maltreatment
Pediatrics, December 1, 2005; 116(6): 1303 - 1308.
[Abstract] [Full Text] [PDF]


Home page
PediatricsHome page
R. D. Mistry, R. G. Hoffmann, J. S. Yauck, and D. C. Brousseau
Association Between Parental and Childhood Emergency Department Utilization
Pediatrics, February 1, 2005; 115(2): e147 - e151.
[Abstract] [Full Text] [PDF]


Home page
PediatricsHome page
I. M. Paul, T. A. Phillips, M. D. Widome, and C. S. Hollenbeak
Cost-Effectiveness of Postnatal Home Nursing Visits for Prevention of Hospital Care for Jaundice and Dehydration
Pediatrics, October 1, 2004; 114(4): 1015 - 1022.
[Abstract] [Full Text] [PDF]


Home page
J. Nutr.Home page
J. T. Cook, D. A. Frank, C. Berkowitz, M. M. Black, P. H. Casey, D. B. Cutts, A. F. Meyers, N. Zaldivar, A. Skalicky, S. Levenson, et al.
Food Insecurity Is Associated with Adverse Health Outcomes among Human Infants and Toddlers
J. Nutr., June 1, 2004; 134(6): 1432 - 1438.
[Abstract] [Full Text] [PDF]


Home page
PediatricsHome page
W. G. Johnson and M. E. Rimsza
The Effects of Access to Pediatric Care and Insurance Coverage on Emergency Department Utilization
Pediatrics, March 1, 2004; 113(3): 483 - 487.
[Abstract] [Full Text] [PDF]


Home page
Arch Pediatr Adolesc MedHome page
K. J. Dombkowski, R. Stanley, and S. J. Clark
Influence of Medicaid Managed Care Enrollment on Emergency Department Utilization by Children
Arch Pediatr Adolesc Med, January 1, 2004; 158(1): 17 - 21.
[Abstract] [Full Text] [PDF]


Home page
PediatricsHome page
G. M. Lee, J. F. Friedman, D. Ross-Degnan, P. L. Hibberd, and D. A. Goldmann
Misconceptions About Colds and Predictors of Health Service Utilization
Pediatrics, February 1, 2003; 111(2): 231 - 236.
[Abstract] [Full Text] [PDF]


Home page
Arch Pediatr Adolesc MedHome page
R. B. Hakim and D. S. Ronsaville
Effect of Compliance With Health Supervision Guidelines Among US Infants on Emergency Department Visits
Arch Pediatr Adolesc Med, October 1, 2002; 156(10): 1015 - 1020.
[Abstract] [Full Text] [PDF]


Home page
Arch Pediatr Adolesc MedHome page
J. T. Cook, D. A. Frank, C. Berkowitz, M. M. Black, P. H. Casey, D. B. Cutts, A. F. Meyers, N. Zaldivar, A. Skalicky, S. Levenson, et al.
Welfare Reform and the Health of Young Children: A Sentinel Survey in 6 US Cities
Arch Pediatr Adolesc Med, July 1, 2002; 156(7): 678 - 684.
[Abstract] [Full Text] [PDF]


Home page
Arch Pediatr Adolesc MedHome page
U. R. Kotagal, P. J. Schoettker, H. D. Atherton, R. W. Hornung, D. Bush, W. J. Pomerantz, and C. J. Schubert
Relationship Between Early Primary Care and Emergency Department Use in Early Infancy by the Medicaid Population
Arch Pediatr Adolesc Med, July 1, 2002; 156(7): 710 - 716.
[Abstract] [Full Text] [PDF]


This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow E-mail this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My File Cabinet
Right arrow Download to citation manager
Right arrowRequest Permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via CrossRef
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Sharma, V.
Right arrow Articles by Wallace, D. D.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Sharma, V.
Right arrow Articles by Wallace, D. D.
Related Collections
Right arrow Emergency Medicine
Social Bookmarking
 Add to CiteULike   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Facebook   Add to Reddit   Add to Technorati   Add to Twitter  
What's this?