BACKGROUND AND OBJECTIVES: With American children experiencing increased Medicaid coverage, it has become especially important to determine if practice patterns differ between Medicaid and non-Medicaid patients. Auditing such potential differences must carefully compare like patients to avoid falsely identifying suspicious practice patterns. We asked if we could observe differences in practice patterns between Medicaid and non-Medicaid patients admitted for asthma inside major children’s hospitals.
METHODS: A matched cohort design, studying 17 739 matched pairs of children (Medicaid to non-Medicaid) admitted for asthma in the same hospital between April 1, 2011 and March 31, 2014 in 40 Children’s Hospital Association hospitals contributing data to the Pediatric Hospital Information System database. Patients were matched on age, sex, asthma severity, and other patient characteristics.
RESULTS: Medicaid patient median cost was $4263 versus $4160 for non-Medicaid patients (P < .001). Additionally, the median cost difference (Medicaid minus non-Medicaid) between individual pairs was only $84 (95% confidence interval: 44 to 124), and the mean cost difference was only $49 (95% confidence interval: –72 to 170). The 90th percentile costs were also similar between groups ($10 710 vs $10 948; P < .07). Length of stay (LOS) was also very similar; both groups had a median stay of 1 day, with a similar percentage of patients exceeding the 90th percentile of individual hospital LOS (7.1% vs 6.7%; P = .14). ICU use was also similar (10.1% vs 10.6%; P = .12).
CONCLUSIONS: For closely matched patients within the same hospital, Medicaid status did not importantly influence costs, LOS, or ICU use.
- CHA —
- Children’s Hospital Association
- CI —
- confidence interval
- ICD-9 —
- International Classification of Diseases, Ninth Revision
- LOS —
- length of stay
- NHLBI —
- National Heart, Lung, and Blood Institute
- PHIS —
- Pediatric Hospital Information System
What’s Known on This Subject:
Asthma is the most prevalent chronic illness among children, remaining a leading cause of pediatric hospitalizations. With American children experiencing increased insurance coverage, it has become especially important to determine whether the pattern of practice differs between Medicaid and non-Medicaid patients.
What This Study Adds:
This report describes a new method to create comparable groups of patients with asthma within hospitals, thereby allowing careful comparison of practice pattern differences between Medicaid and non-Medicaid patients.
Medicaid’s payment to providers may be substantially lower than private insurance reimbursement.1–3 Therefore, it is natural to ask: Does the care of Medicaid patients differ from the care of non-Medicaid patients within the same hospital? Because asthma is the most prevalent chronic illness among children and remains a leading cause of hospitalizations among children aged 1 to 15 years in the United States,4 and because inpatient and emergency department treatment account for about one-third of all pediatric asthma-related healthcare costs,5 this study examines the variation in patterns of practice as expressed by resource use (cost), length of stay (LOS), and ICU use across Medicaid and non-Medicaid patients at 40 children’s hospitals in the United States that contributed data to the Pediatric Health Information System (PHIS) dataset. These hospitals, all members of the Children’s Hospital Association (CHA; Overland Park, KS), represent some of the most technologically sophisticated hospitals in the country. We ask whether significant practice pattern variation occurs in this common pediatric disease across closely matched Medicaid and non-Medicaid patients at these hospitals.
To accomplish this goal, we use a new methodology focusing on multivariate matching6–10 to accurately compare patient resource usage inside hospitals. Asthma admissions for Medicaid patients are paired with non-Medicaid patients within the same hospital, carefully matching on patient characteristics.
This study was approved by the institutional review board of The Children’s Hospital of Philadelphia.
Patient Population and Definitions
Data were obtained from PHIS, an administrative database that contains inpatient, emergency department, ambulatory surgery, and observation data from 41 not-for-profit, tertiary care pediatric hospitals in the United States that had complete data for the full study period at the time data were obtained. These hospitals are affiliated with the CHA. For these hospitals, we examined all patients admitted with asthma as found in the PHIS database between April 1, 2011 and March 31, 2014.
All nontransfer inpatient and observational unit nonresearch visit discharges occurring between April 1, 2011 and March 31, 2014 were considered if the patient was between 3 and 18 years old and admitted for asthma. The asthma definition was based on the presence of specific International Classification of Diseases, Ninth Revision (ICD-9) codes as shown in Table 1. Variables we matched on included: age in days, sex, common chronic conditions, asthma-affecting diagnoses, National Heart, Lung, and Blood Institute (NHLBI) diagnoses of concern,11 predicted LOS (a risk score derived from an external data set as suggested by Hansen12 [see Model for Predicted LOS section in Supplemental Information]), a propensity score to be a Medicaid patient at that hospital, and asthma severity at admission defined as moderate, severe, or critical based on treatment provided on day 0 or day 1 (see Table 1). We used only the first asthma admission in the data set for each patient. The PHIS database reports primary, secondary, and tertiary sources of payment. If the primary source of payment was Medicaid, or the secondary/tertiary sources were Medicaid with the primary as nonprivate, or the patient was indicated to have coverage under a State Children’s Health Insurance Program, then this patient was labeled as having Medicaid insurance. Otherwise, a person was flagged as not having Medicaid.
Defining Outcome and Cost Variables
Once hospital matches were complete and of good quality, hospitals were compared on the following primary variables: total inpatient cost, hospital LOS, and the percent of patients using the ICU.
Each hospital’s costs were based on their observed resource use. Billing codes were used to track resource use. The unit costs for each billing code were determined using a methodology similar to Keren et al13 with modifications as described below and in detail in the Cost Calculation Methodology section in Supplemental Information. To compare resource use, not influenced by local charges, each billing code (the basis of counting resources used at each hospital) was assigned a dollar cost using a uniform formula across all hospitals. To calculate an annual unit cost for each billing code, we used all PHIS data that was provided for all patient types admitted for asthma. Costs were adjusted to 2014 prices using US Bureau of Labor Statistics Consumer Price Index values for Medical Care items.14
The multivariate matching methodology paired very similar Medicaid and non-Medicaid patients treated in the same hospital. This design permits us to ask 3 questions: (1) In aggregate, are similar Medicaid and non-Medicaid patients treated differently inside the same hospital? (2) Is the difference between Medicaid and non-Medicaid patients different at different hospitals? (3) Are there specific hospitals for which the difference is unusually large? When matching inside hospitals, we created the maximum number of matched pairs, whether that number was limited by the number of Medicaid or non-Medicaid patients. If the quality of this match was poor at a hospital, a subset of matched patient pairs was obtained using optimal subset matching,10,15 a multivariate matching method that discards a minimal number of patients from both groups subject to conditions on the quality of the matched pairs.
We performed our matches using the R package MIPMatch.16–18 We required exact matches on asthma severity (moderate, severe, or critical). For other variables, we chose a balanced match that minimized medical distance6,7,19,20 between matched pairs at each hospital, defined using the Mahalanobis distance. Details concerning distances are provided in Supplemental Information, Details on Matching Methods.
To improve the quality of the matches within a specific hospital, we used near-fine balance10,21–24 within hospital patients; this ensured that if the hospital's Medicaid patients had, say, a 37.3% rate of upper respiratory infection that the hospital's matched non-Medicaid patients also had a upper respiratory infection rate of 37.3%, without requiring that each matched pair have the same upper respiratory infection status. It was always preferred that matches have the same patient and clinical characteristics, but for variables matched using near-fine balance, a mismatch was allowed if it could be counterbalanced in another matched pair, so overall the matched patient groups were similar on these characteristics via minimizing the Mahalanobis distance function. A mean constraint was introduced on age at admission and propensity score for being a Medicaid patient at that hospital. We also added a penalty to the Mahalanobis distance for differences in predicted LOS.
Testing Match Quality
It is important to check that the match quality is adequate. For each covariate, we examined the Medicaid versus non-Medicaid differences in means as a fraction of the SD, the so-called “standardized difference,” aiming for an absolute value of ≤0.2.10,25,26 All matching was optimized subject to balance constraints using the MIPMatch function16,27 in R.18 Statistical tests used SAS 9.2 (SAS Institute, Inc, Cary, NC) for UNIX,28 and figures were plotted using the density function in R.
All matching was completed without knowledge of outcomes, as suggested by Rubin.29,30 By matching without knowledge of outcomes, researchers are prevented from selecting the most attractive of multiple analyses.
We attempted to answer 3 questions using these matches: (1) Is there a difference in practice patterns (cost, LOS, and ICU use) between Medicaid and non-Medicaid asthma patients pooled across all PHIS hospitals? (2) Is the difference between Medicaid and non-Medicaid patients the same across hospitals? And lastly, (3) Do any individual hospitals stand out with especially large differences between Medicaid and non-Medicaid patients after adjusting for multiple testing (examining multiple individual hospitals)?
In our primary analysis, we compared cost and LOS to what is typical in that hospital, not to what is typical nationally. For example, in matched Medicaid–non-Medicaid pairs from the same hospital, the primary analysis asks whether a patient stayed longer than the median or 90th percentile in that hospital, not longer than the national median or 90th percentile. A secondary analysis looks at overall (pooled) medians and percentiles.
For continuous outcomes for the first question, we used quantile tests31,32 that determined whether each patient exceeded its own hospital’s median or 90th percentile value, then, in effect, used McNemar’s statistic31,33 to test the equality of Medicaid and non-Medicaid groups in exceeding this value. For the binary variables, ICU use, we compared matched Medicaid and non-Medicaid patients, similarly using the McNemar statistic.
We also looked at the Medicaid minus non-Medicaid paired differences in cost and LOS using the median (and its related sign test), the mean (and its related paired t test), and using the Hodges–Lehmann estimate (and its related Wilcoxon signed rank test).34 We report all 3 tests because the paired t test is destabilized by individual patients with extreme values. The Wilcoxon test is not destabilized by the tails of the distribution, unlike the t test, but it does take them into account, unlike the median.
To answer the second question, “Is the difference between Medicaid and non-Medicaid patients the same across hospitals,” we applied the Kruskal–Wallis test to the matched pair differences for cost and LOS. For the binary variable, ICU use, we applied a χ2 test of independence to the 2 × 40 table of discordant pairs.35
Finally, when looking at each hospital one at a time, we again used quantile tests as above, but with a correction for testing many hypotheses based on the Bonferroni–Holm method.36 We controlled the familywise error rate at 5% in 80 tests, testing 40 hospitals at both the median and 90th percentile of total cost, 80 = 40 × 2.
Of the 41 hospitals in PHIS, 40 hospitals were available with full patient records for the full study period. There were 859 997 patients in the PHIS data set meeting reporting requirements and therefore available for analysis; of these, there were 64 466 (or 7.5%) patients admitted for an asthma diagnosis. After excluding transfer-in patients, there were 48 903 patients. Additional exclusions for illogical departmental billing costs yielded a final sample of 48 879 patients eligible for the study.
Quality of the Matches
Of the 40 hospitals, after matching, 37 met our matching quality criteria and did not need subsetting, and 3 remaining hospitals required optimal subset matching to produce high quality matches. In these 3 hospitals, <7% of potential matches were excluded per hospital using the optimal subsetting of patients.15 The matching quality is reported in Table 2. This table displays matching quality for the covariates controlled in the match. Columns compare matched Medicaid and non-Medicaid patients. Overall, no covariate that was matched on differed significantly between the 2 patient groups (for an expanded version of this table including patients who were not used in the match, see Supplemental Table 5).
Question 1: Differences Across Matched Medicaid and Non-Medicaid Patients
We first ask the question, were there differences in outcomes across matched Medicaid and non-Medicaid patients inside the same hospital. Table 3 examines cost, LOS, and ICU use across the 17 739 matched pairs.
Hospital cost was similar between Medicaid and non-Medicaid patients. Median cost in Medicaid patients was $4263 versus $4160 in non-Medicaid patients (P < .001). A total of 9043 Medicaid patients exceeded their own hospital’s median cost (51.0%) versus 8696 (49.0%) in non-Medicaid patients (P < .001). The median difference in cost (Medicaid to non-Medicaid) between pairs was only $84 (95% confidence interval [CI]: 44 to 124; P < .001), the mean cost difference was only $49 (95% CI: −72 to 170; P = .43), and the Hodges–Lehmann estimate was $94 (95% CI: 45 to 142; P < .001). Examining higher cost patients, the 90th percentile cost in the Medicaid patients was $10 710 versus $10 948 in non-Medicaid patients (P = .14). A total of 9.7% of Medicaid patients exceeded the 90th percentile of their hospital’s costs versus 10.2% of non-Medicaid patients (P = .07). Fig 1A displays the distribution of cost for Medicaid and non-Medicaid patients using all matched patients, each group having 17 739 patients. As can be seen, cost distributions are closely overlapping. Box plots at the bottom of Fig 1A provide another description of how similar costs were between Medicaid and non-Medicaid patients. Fig 1B displays the distribution of differences in cost between each patient in a matched pair. Again, there are 17 739 differences displayed, and a box plot describing these differences is provided at the bottom of Fig 1B. The detailed distributions for all outcomes are included in the section More Detailed Outcomes of the Supplemental Information, including comparisons by cost department.
LOS was also similar between matched pairs. Both Medicaid and non-Medicaid matched patients had a median stay of 1 day, although more Medicaid patients exceeded their hospital’s median LOS (37.5%) than did non-Medicaid patients (35.6%), P < .001. On the other hand, there was no significant difference in long LOS, with the 90th percentile being 3 days for both Medicaid and non-Medicaid patients. A total of 7.1% of Medicaid patients versus 6.7% of non-Medicaid patients exceeded their own hospital’s 90th percentile LOS (P = .14). On average, Medicaid patients stayed in the hospital a tiny bit longer, with a mean difference in LOS of 0.04 days (95% CI: 0.02 to 0.07; P < .001). Although the median pair-difference in LOS was 0 days, the Medicaid LOS exceeded matched non-Medicaid LOS in 27.7% of pairs, whereas non-Medicaid LOS exceeded Medicaid LOS 25.4% of the time (P < .001).
ICU use was also similar between Medicaid and non-Medicaid patients (10.1% versus 10.6%; P = .12). Total days in the ICU was also similar between pairs. On average, Medicaid patients stayed in the ICU a tiny bit less, with a mean difference in days in the ICU of –0.01 days (95% CI: –0.02 to 0.01); P = .35). Although the median paired difference in days in the ICU was 0 days, the Medicaid days in the ICU exceeded non-Medicaid days in the ICU in 7.5% of pairs, whereas non-Medicaid days in the ICU exceeded Medicaid days in the ICU 7.9% of the time (P = .15).
A secondary analysis was performed excluding any pairs (n = 1249 [7.0%]) where the non-Medicaid patient was indicated as “self-pay.” Results were nearly identical.
Question 2: Is the Difference Between Medicaid and Non-Medicaid Patients the Same Across Hospitals?
The difference between Medicaid and non-Medicaid patients was different in different hospitals for both cost (P < .001) and LOS (P = .011). That is, the Kruskal–Wallis test was used to look at the matched pair differences in cost and LOS, and it was concluded that the variation among hospitals in these differences was too large to be attributed to chance.
Question 3: Do any Individual Hospitals Stand Out With Especially Large Differences Between Medicaid and Non-Medicaid Patients?
Our significant finding in question 2 prompted us to attempt to identify hospitals with especially large differences between Medicaid and non-Medicaid patients. Because we tested many times, twice in each of 40 hospitals, we needed to correct for testing many hypotheses. We used the Bonferroni–Holm correction to control the familywise error rate for the 2 tests of interest (median and 90th percentile cost) in 40 hospitals (P < .05/80). There were only 2 hospitals that displayed some significant differences after adjusting for multiple testing (see Outcomes for All Hospitals in Supplemental Information). In 1 hospital, called Hospital AL, 54.1% of their Medicaid patients exceeded the hospital’s median cost, whereas only 45.9% of non-Medicaid patients exceeded the same threshold (P < .001). The other hospital, called Hospital K, showed the opposite pattern: 6.0% of its Medicaid patients exceeded the hospital’s 90th percentile cost compared with 13.9% of non-Medicaid patients who exceeded the same threshold (P < .001).
From a policy perspective, our results are quite reassuring. We did not see important differences in practice patterns for nontransferred acute asthma admissions between Medicaid and non-Medicaid patients during the years 2011 to 2014 at major children’s hospitals. Because our study was large, including >17 000 pairs of patients, we did see some statistically significant differences between Medicaid and non-Medicaid patients for some outcomes, but in most cases, such differences were small in any economic or clinical sense. We also saw only slight variation across hospitals. After adjusting for multiple testing, there were only 2 hospitals that displayed significant differences in cost. Given that the differences in cost were in the opposite direction, our findings suggest that there is no consistent pattern of directionality between Medicaid and non-Medicaid patients with regard to resource use costs.
Bratton et al37 have reported that hospital LOS was greater in Medicaid patients, even after some adjustment. Our study supports the finding that Medicaid patients have slightly longer stays, although this difference was extremely small (mean of the paired differences was 0.04 days) and not clinically meaningful.
Our analysis was aimed at examining differences in practice patterns between Medicaid and non-Medicaid patients on similar patients, because we were interested in whether payment differences from different insurance plans may influence practice patterns. This is very different from asking whether Medicaid patients are more expensive because they are sicker. Where our findings differ from previous work is in comparison of matched pairs, rather than standard risk adjustment. We find very little evidence of practice pattern variation by Medicaid status. The use of multivariate matching in this study allowed us to say with confidence that, after matching, patient populations were similar in clinical presentation. Our study should serve to provide potential benchmarks for use and reimbursement standards, with implications for care and payment even when children are hospitalized outside the PHIS system.
For closely matched patients within the same hospital with the same asthma severity of illness, Medicaid status did not importantly influence costs, LOS, or ICU use. It is important to keep track of these differences to ensure that Medicaid patients are treated in a similar manner to non-Medicaid patients.
Data for this project was supplied by the CHA PHIS. The PHIS hospitals are some of the largest and most advanced children's hospitals in America and constitute the most demanding standards of pediatric service in America. We thank Traci Frank (Center for Outcomes Research, The Children’s Hospital of Philadelphia) for her assistance with this research.
- Accepted May 5, 2016.
- Address correspondence to Dr. Jeffrey H. Silber, MD, PhD, Center for Outcomes Research, The Children’s Hospital of Philadelphia, 3535 Market St, Ste 1029, Philadelphia, PA 19104. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: All phases of this study were supported by Agency for Healthcare Research and Quality grant U18-HS020508 and grant SES-1260782 from the US National Science Foundation.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Berman S,
- Dolins J,
- Tang SF,
- Yudkowsky B
- Zuckerman S,
- Williams AF,
- Stockley KE
- Hadley J,
- Holahan J
- Silber JH,
- Rosenbaum PR,
- McHugh MD, et al
- Rosenbaum PR
- National Heart, Lung, and Blood Institute, National Asthma Education and Prevention Program
- Hansen BB
- US Bureau of Labor Statistics
- R Development Core Team
- Silber JH,
- Rosenbaum PR,
- Polsky D, et al
- ↵Statistical Analytic Software System for UNIX [computer program]. Version 9.2. Cary, NC: SAS Institute Inc.; 2009
- Rosenbaum PR
- Bishop YMM,
- Fienberg SE,
- Holland PW
- Hollander M,
- Wolfe DA
- Gart JT
- Copyright © 2016 by the American Academy of Pediatrics