BACKGROUND AND OBJECTIVE: Among children’s hospitals, little is known about how barriers to electronic health record (EHR) adoption are related to meaningful use (MU) incentives. We investigated hospital success with MU incentive payments and determined associations with hospital-reported challenges and characteristics.
METHODS: A survey administered to 224 Children’s Hospital Association hospitals assessed a variety of potential challenges to achieving meaningful EHR use (eg, lack of access to capital) and specific MU criteria that would be challenging to fulfill (eg, implement clinical decision support rules). These results were combined with data on hospitals that received MU payments up to March 2014 and information on hospital characteristics. Associations between anticipated challenges, children’s hospital type, and receipt of MU incentives were evaluated in bivariate and multivariate analyses.
RESULTS: One hundred thirty-three children hospitals completed the survey (response rate 59.4%). Thirty-five percent of responding children’s hospitals received MU incentive payments. The most frequently anticipated hospital challenges included the following: exchange clinical information with other providers outside your hospital system (49%), and generate numerator and denominator data for quality reporting directly from EHR (41%). Freestanding children’s hospitals were more likely to indicate lack of relevance of MU criteria to pediatric care (odds ratio: 37.6 [95% confidence interval: 4.6–309.3]) and more likely to receive MU incentive payments (odds ratio: 26 [95% confidence interval: 5.2–130.6]).
CONCLUSIONS: As of 2014, a minority of children’s hospitals have successfully received MU incentive payments. Freestanding children’s hospitals are more likely to report MU is not relevant to pediatric care and to succeed with MU incentive payments.
What’s Known on This Subject:
Meaningful use (MU) incentive payments have been developed to encourage adoption and use of electronic health records (EHRs). Several studies have revealed children’s hospitals have unique barriers to the use of EHRs but were relatively early adopters of information technology.
What This Study Adds:
Although a minority of children’s hospitals have succeeded with MU incentives, freestanding children’s hospitals are significantly more likely to succeed. Improvement of EHRs for pediatric use should focus on information exchange, quality reporting, and MU relevance to pediatrics.
The Centers for Medicare and Medicaid Services offers incentive payments to encourage the adoption and meaningful use (MU) of electronic health records (EHRs) by providers and hospitals.1 To achieve MU in the first stage of the incentive program, EHRs must permit multiple functions, including medication reconciliation and electronic exchange of health information with patients and other facilities. The MU criteria advance over time with new measures and/or higher thresholds for older measures. Hospitals with >10% Medicaid volume can receive initial incentive payments through the Medicaid program if they Adopt-Implement-Upgrade (AIU) certified EHRs. This program has had variable success in easing access to incentive dollars for hospitals that may have limited funds (eg, safety-net hospitals) to make the initial EHR purchase.2
As early as 2005, children’s hospitals reported barriers to EHR use similar to hospitals that care for adults, such as cost and provider resistance, as well as barriers unique to pediatric care, such as products not designed for care of children.3,4 Examples of important EHR features needed to support pediatric workflows include weight-based dosing of medications, pediatric-specific drug–drug interactions, growth charts, and age-adjusted normal values for vital signs, and diagnostic test results.5–7 Despite these challenges, studies have revealed that US children’s hospitals are relatively early adopters of information technology (IT), with adoption rates that are higher than those of community hospitals focused on care of adults and similar to those of other academic teaching hospitals.3,8,9 However, to truly use EHRs in a meaningful way for children’s health care, children’s hospitals will need to invest resources to increasingly incorporate EHRs into provider, hospital, and interfacility workflow while also working with vendors to ensure products fit pediatric needs.
Limited research exists about the success of children’s hospitals in receiving MU incentives and whether specific challenges anticipated by hospitals with regard to either EHR use or MU criteria are associated with success. The purpose of this project is to determine the proportion of children’s hospitals that have succeeded with MU incentive payments and to assess the association between hospital characteristics or anticipated MU challenges and receipt of incentive payments. This information will enable identification of specific areas of concern to children’s hospitals that may enable or impede MU success.
Study Design and Participants
This is a retrospective observational study of hospitals that draws upon 3 data sources. The first of these was a survey among the 224 members of the Children’s Hospital Association (CHA) hospitals, a merger of the former National Association of Children’s Hospitals and Related Institutions and Child Health Corporation of America, that was administered from September 2011 to May 2012. The membership of this not-for-profit organization has been used to study IT within children’s health care and includes children’s hospitals, large pediatric units in medical centers, and related institutions, including those that specialize in rehabilitation.3 The survey examined various issues related to EHR adoption by children’s hospitals, including challenges of both MU of EHRs and specific MU criteria. The survey data were linked with data from the Office of the National Coordinator on MU incentives paid to hospitals from the start of incentive payments (2011) through March 2014. Additionally, both databases were merged with information on hospital characteristics from the American Hospital Association (AHA) Annual Survey. Merging the 3 data sources was performed by using the hospital name and AHA identification number. All survey respondents matched the AHA data.
The primary outcome for the study was receipt of any MU incentive payment. We considered the total number of payments through either Medicare or Medicaid (AIU and MU); freestanding children’s hospitals are eligible for the incentive program only through Medicaid.
Survey on Reported Challenges
The survey was developed by a team experienced with IT (including Drs Yu, Nakamura, and Harper) based on content extracted from the literature, including American Academy of Pediatric publications; piloted among chief information officers/chief medical information officers (CIO/CMIOs); and distributed to hospital CIO/CMIOs as noted in the CHA membership database. Respondents were requested to consider the challenges as they relate specifically to the care of children. The CIO/CMIOs were asked to specify the 2 greatest challenges to their achieving meaningful EHR use from the following list: cost (acquisition and/or maintenance); lack of access to capital; resistance to implementation from providers; lack of adequate hospital IT personnel to support implementation or maintenance; challenge of meeting MU criteria within implementation time line; lack of relevance of MU criteria to pediatric care; and inability of EHR products to meet pediatric clinical needs. Additionally, respondents were asked to select from the following list the 2 MU criteria that would be most challenging to fulfill: implement clinical decision support rules; implement computerized provider order entry (CPOE) at specified level of sophistication; exchange clinical information with other providers outside your hospital system; perform medication reconciliation across settings of care; give patients access to their data in electronic form; generate problem lists by using codified data sets; and generate numerator and denominator data for quality reporting directly from EHR.
The AHA identifies hospitals that restrict admissions primarily to children; consistent with approaches in previous research, these hospitals were considered freestanding children’s hospitals.10,11 Because all CHA members are children’s hospitals, the remaining survey respondents were considered nonfreestanding children’s hospitals. Hospital characteristics used as control variables included bed size, ownership, metropolitan status, and geographic region. Bed size is defined by number of staffed beds in the facility and was categorized as small (<200 beds), medium (200–399 beds), or large (400+ beds). Metropolitan status in AHA is categorized as rural, metropolitan, micro, or divisional. Because a large number of study hospitals were in metropolitan areas, we recoded this variable as metropolitan (N = 93) versus nonmetropolitan (N = 40). Hospital ownership was categorized as public (nonfederal governmental), for-profit, and not-for-profit. Because of the limited number of for-profit children’s hospitals, this variable was recoded as public versus nonpublic for multivariate analyses. Hospital geographic region was categorized as Northeast, Midwest, Southeast, and West.12
We determined the frequency and rank of reported barriers among all hospitals and stratified among hospitals with and without MU incentive payments. In bivariate analyses, the χ2 or Fisher’s exact test, as appropriate, was used to determine the association of incentive payments with a hospital’s anticipated MU challenges or hospital characteristics. In multivariate analyses, 2 approaches were used. During the first approach, 14 multivariate logistic regression models were performed, 1 for each of the 14 reported challenges as the dependent variable controlling for other hospital characteristics. This analysis enabled us to examine whether frequently reported challenges varied among different types of children’s hospitals. This approach to model each reported challenge separately is consistent with previous research to address multicollinearity among the various reported challenges (eg, cost [acquisition and/or maintenance], and lack of access to capital).13 For the second approach, a single multivariate logistic regression was performed on the primary outcome variable (receipt of any MU incentive payment as the dependent variable) to determine independent associations with successful MU payment to hospitals. This model was designed to determine the independent relationship of a hospital’s anticipated challenges and hospital characteristics with this primary outcome variable. Only hospital-reported challenges in the bivariate analysis with P ≤ .2 and correlation coefficient ≤0.6 were included in this model. A sensitivity analysis was performed excluding payments through the AIU mechanism to ensure the reported associations were also generalizable to the more strict MU incentive criteria. All analyses were performed by using SAS 9.4 (SAS Institute, Inc, Cary, NC).
Respondents Versus Nonrespondents
One hundred thirty-three hospitals responded to the survey (response rate 59.4%). We compared respondents and nonrespondents with regard to hospital characteristics. Of the 224 hospitals surveyed, respondents were more likely to be freestanding children’s hospitals (74% vs 51%; P < .001). However, we found no significant differences between the 2 groups in bed size, ownership, metropolitan status, or geographic region.
Top Challenges and Success With MU
Of the 133 responding hospitals, 47 (35%) received MU incentive payment as of March 2014 (Table 1). Supplemental Table 6 presents hospital payments by year. Among all hospitals, the top anticipated challenge to achieving meaningful EHR use was meeting MU criteria within their implementation time lines (36%), whereas the MU criterion that would be most challenging to fulfill was exchanging clinical information with other providers outside their hospital system (49%; Table 2). Cost (acquisition and/or maintenance) and lack of access to capital, which historically are commonly reported as top barriers, were not seen in the top 3 anticipated challenges.3 However, of the 2 hospitals that anticipated lack of access to capital would be a challenge to achieving MU, none received MU incentives during the study period.
Relationships Between Anticipated Challenges and Receipt of MU Incentives
In bivariate analysis, hospitals that received MU payments were more likely than their counterparts to indicate that lack of relevance of MU criteria to pediatric care was a barrier to MU (58% vs 19%; P < .0001; Table 2). Hospital characteristics associated with receipt of MU incentive payments included children’s hospital type (nonfreestanding children’s hospital [17%] versus freestanding [83%]; P < .001), bed size (<200 beds [34%], 200–399 beds [40%], and 400+ beds [26%]; P < .001), and hospital ownership (public [0%], not-for-profit [98%], and for profit [2%]; P < .01; Table 3).
For the multivariate analyses focusing on each challenge as the outcome, only 3 of 14 models converged with adequate fit statistics (Table 4). The lack of convergence is likely due to the relatively low numbers of children’s hospital and therefore limited number of observations. The challenges with significant overall model P values included the following: lack of relevance of MU criteria to pediatric care (P < .001), cost (acquisition and/or maintenance; P < .01), and generate numerator and denominator data for quality reporting directly from EHR (P < .05). Lack of relevance of MU criteria to pediatric care was more likely to be reported as a challenge by freestanding children’s hospitals (odds ratio [OR]: 37.6 [95% confidence interval (CI): 4.6–309.3]), controlling for other hospital characteristics. Report of cost (acquisition and/or maintenance) and generate numerator and denominator data for quality reporting directly from EHR as challenges varied by hospital bed size, independent of other hospital characteristics.
Multivariable analysis (Table 5) demonstrated that freestanding children’s hospitals were more likely to receive incentives (OR: 26 [95% CI: 5.2–130.6]), controlling for challenges and other hospital characteristics. The 2 reported challenges included in this model (based on P ≤ .2) did not demonstrate multicollinearity (correlation coefficient 0.21). The significant association between lack of relevance and MU payment became nonsignificant in this model when provider resistance was added, likely due to our limited sample size and because some of the association between MU payment and reporting lack of MU relevance can be attributed to reporting provider resistance.
Sensitivity analyses in which hospitals with only AIU payments were excluded demonstrated no change in significance (P value) or overall direction of the described associations (Supplemental Table 7).
A minority of children’s hospitals in our study, only 35%, succeeded in fulfilling MU criteria and receiving incentives as of March 2014. In contrast, 64% of all eligible hospitals received a MU payment by 2012.14 In our study, 84% of freestanding children’s hospitals received MU incentive payments and were more likely to receive incentives than nonfreestanding children’s hospitals, even when controlling for other factors. However, freestanding children’s hospitals were more likely to report lack of relevance of MU criteria to pediatric care.
Our finding that freestanding children’s hospitals were more likely to receive MU incentive payments may be due to their organizational structure and focus on a more homogeneous population (children). This could be analogous to the findings of Diana et al15 among general adult hospitals that membership in a large hospital system was associated with not receiving MU payments. Among larger hospital systems, resources for pediatric IT adoption may be constrained because pediatric and adult departments share IT bandwidth. For example, freestanding children’s hospitals may have CIO/CMIOs who can focus exclusively on child health needs. Furthermore, freestanding children’s hospitals are known to have high levels of IT adoption, which could enable them to more easily achieve MU compared with hospitals in initial phases of EHR adoption.10,15,16 Among adult hospitals, evidence is growing that incentives may have rewarded previous IT users as compared with enticing new adopters.15 An alternative or additional explanation for the greater success of freestanding children’s hospitals may be the design of the incentive programs. A blend of structure and policy could be at play as the pending penalties for Medicare may be delaying hospitals that care for more diverse populations (eg, nonfreestanding children’s hospitals) because these institutions would have to achieve MU for all patient populations or risk penalties in 2015.17
The most frequently reported challenge among all responding hospitals was exchanging clinical information with other providers outside their hospital system. The first stage of MU only requires having the capability to submit electronic data on immunizations, reportable laboratory results, and syndromic surveillance to public health agencies rather than actual submission.18 However, establishing meaningful data-sharing capability across health systems is difficult for multiple reasons, including limited EHR functionality and use in some settings, lack of interoperability, concerns about privacy and security, patient consent, and concerns about competition. Exchanging health information requires organizations to register with a health information exchange (HIE), produce a standard document for exchange (eg, Health Level-7 consolidated clinical document architecture), and send data by using standard HIE protocols. When studying MU-eligible providers, Wright et al19 noted a similar theme related to the difficulties of clinical data sharing (eg, immunization registries data) and suggest that a cross-system public health reporting infrastructure may be needed to achieve meaningful exchange of information.
The second most commonly reported challenge cited by hospitals was generating numerator and denominator data for quality reporting directly from their EHR. For this capability, an EHR must capture the necessary data elements to compute the numerator and denominator data and must be able to report these values reliably. One major constraint here is that the quality measures are often adult-centric; modifying the data definitions and capture process to accommodate child health considerations may be challenging for hospitals. Although this challenge likely exists for any hospital that cares for children attempting to meet MU criteria, the issue appears to be a priority among these children’s hospitals.
Commonly anticipated challenges in children’s hospitals appear to have evolved since Menachemi et al3 demonstrated in 2009 that the most commonly cited major barrier was vendors’ inability to deliver product or services to satisfaction. Only 16% of hospitals in the current study noted inability of EHR products to meet pediatric clinical needs as a top MU barrier, whereas the lack of relevance of MU criteria to pediatric care was the most commonly reported challenge among hospitals successful with MU incentives. This may reflect that respondents noticed improvements in EHR products for children’s care, are more capable of modifying vendor systems locally, and/or may be experiencing a shift in attitude because they are focusing on MU criteria as their primary challenges. EHR vendor products have likely improved over time, but other studies and our results continue to emphasize pediatric-specific areas for further improvement.6,20,21 Although the cause for this shift is uncertain, our results suggest that improving relevance to pediatric care could be considered a top priority for incentivizing MU among children’s hospitals as further MU objectives are developed.
We found that bed size was associated with anticipated MU challenges. Specifically, larger hospitals less frequently reported challenges with cost but more frequently reported difficulties in generating numerators/denominators for quality reporting for pediatric care. It may be that EHR implementation costs present less of a problem to large institutions as a result of economies of scale.10 Previous research has suggested among general hospitals that larger hospital bed size was associated with challenges at generating data for quality reporting.13 It is not surprising that this association would also be noted in our study among larger hospitals that care for children because applying quality measures for children is often considered complex, resource-intensive, and not fully supported by IT.6,7 In contrast, an article by Harle et al13 focused on adult hospitals suggested CPOE is the primary barrier for hospitals not meeting MU. Previous research on children’s hospitals has demonstrated they were at the forefront of CPOE adoption, and our current results suggest this experience with CPOE may be benefiting hospitals in our current era of MU incentives.8,10
Our study has several limitations. First, the survey respondents represent a diverse array of US children’s hospitals that are actively involved in EHR adoption. However, our findings may not be generalizable to other hospitals that care for children, including those in rural areas. Second, although the survey response rate is reasonable, survey respondents were more likely to be freestanding children’s hospitals. Because this group of hospitals was more likely to receive MU incentive payments, our estimates of successful payments may be overestimated. Third, the relatively small number of children’s hospitals in the United States resulted in limited power to detect differences in the challenges reported by hospitals that did and did not receive MU incentives. Some of the differences in frequencies with which challenges were cited were sizable (eg, resistance to implementation from providers; 38% vs 24%) but not statistically significant. Lastly, for nonfreestanding children’s hospitals, it is unclear whether payments were made to a hospital’s parent organization or directly to the children’s hospital. This distinction is important because receipt of incentives by the parent institution could be less likely to translate into improvements in MU of EHRs for pediatric care.
A minority of children’s hospitals succeeded in receiving MU incentive payments through March 2014. Children’s hospitals commonly identify HIE outside of their system and quality reporting as the most challenging MU objectives. Freestanding children’s hospitals were most successful in the MU program but more frequently reported that MU criteria are not relevant to children’s health care. MU criteria more focused on pediatric health care are needed to improve the relevance of the program to children’s health care.
We thank Dr. Ashish Jha and Allan Castro for their contributions to developing and administering the CHA survey tool, the CIO/CMIOs who reviewed the survey for completeness and clarity, and Myla Ebeling for her contribution to data management and processing.
- Accepted March 18, 2015.
- Address correspondence to Ronald J. Teufel II, MD, MSCR, Department of Pediatrics, Medical University of South Carolina, 135 Rutledge Ave, Charleston, SC 29425. E-mail:
Dr Teufel conceptualized and designed the study, drafted the initial manuscript, and supervised all data merges and analyses; Dr Yu participated in development of the survey, contributed to the conceptual design of the study, participated in survey administration and collection of data on incentive payments, and edited all manuscript drafts; Dr Nakamura led development of the survey, oversaw survey administration, and participated in the data analysis plan and interpretation of the results; Dr Harper participated in development of the survey, participated in survey administration, and participated in the data analysis plan and interpretation of the results; Dr Menachemi conceptualized and designed the study, participated in the initial draft of the manuscript, contributed data on hospital characteristics, and participated in the analyses and interpretation of results; and all authors approved the final manuscript as submitted.
Dr Menachemi's current affiliation is Department of Health Policy and Management, Indiana University, Indianapolis, Indiana.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Menachemi N,
- Brooks RG,
- Schwalenstocker E,
- Simpson L
- Spooner SA,
- Council on Clinical Information Technology, American Academy of Pediatrics
- ↵Spooner SA. We are still waiting for fully supportive electronic health records in pediatrics. Pediatrics. 2012;130(6). Available at: www.pediatrics.org/cgi/content/full/130/6/e1674
- Spooner SA,
- Classen DC
- Teufel RJ II,
- Kazley AS,
- Basco WT Jr
- ↵American Hospital Association. AHA annual survey database. Available at: www.ahadataviewer.com/book-cd-products/AHA-Survey/. Accessed December 23, 2014
- ↵Agency for Healthcare Research and Quality. HCUP Kids’ Inpatient Database (KID). Healthcare Cost and Utilization Project (HCUP). Rockville, MD: Agency for Healthcare Research and Quality; 2003. Available at: www.hcup-us.ahrq.gov/kidoverview.jsp. Accessed March 24, 2015
- Harle CA,
- Huerta TR,
- Ford EW,
- Diana ML,
- Menachemi N
- ↵Robert Wood Johnson Foundation. Health information technology in the United States: progress and challenges ahead. Available at: www.rwjf.org/content/dam/farm/reports/reports/2014/rwjf414891. Accessed March 24, 2015
- ↵Nakamura MM, Harper MB, Jha AK. Change in adoption of electronic health records by US children’s hospitals. Pediatrics. 2013;131(5). Available at: www.pediatrics.org/cgi/content/full/131/5/e1563
- DesRoches CM,
- Worzala C,
- Bates S
- ↵Center for Medicare & Medicaid. EHR incentive program. Available at: https://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/downloads/Hosp_CAH_MU-TOC.pdf. Accessed December 23, 2014
- Copyright © 2015 by the American Academy of Pediatrics