OBJECTIVE: To determine the association between a widely used individual-level assessment of the medical home derived from parental perception and practice-level assessment of the medical home.
METHODS: Thirty parents at each of 6 Boston-area community health centers (CHCs) were administered the 19-question medical home measure of the 2011 National Survey of Children’s Health (NSCH). Each CHC was scored in accordance with the National Committee for Quality Assurance (NCQA) 2011 Patient-Centered Medical Home Standards and Guidelines. Bivariate analysis of the independent variables against the dependent variable (the NCQA measure, assessed as both a continuous score and a categorical tier) was performed. Linear and logistical regression models accounting for the cluster design were then constructed, with the NSCH measure as the primary predictor.
RESULTS: Of the 180 parents, 52% had a medical home according to the NSCH criteria. Of the 6 CHCs, 5 were a medical home according to the NCQA Standards (2 at Tier 3, 3 at Tier 2). Regression modeling demonstrated nonsignificant associations between both the continuous and categorical (tier) NCQA scores and the NSCH assessment of the medical home, with a β of −2.80 (95% confidence interval, −7.75 to 13.35) and an adjusted odds ratio of 2.17 (95% confidence interval, 0.82 to 5.74), respectively.
CONCLUSIONS: There was no significant association between the individual-level and practice-level assessments of the medical home. Given this discrepancy, our results suggest that we may need to temper our expectations that the medical home transformation currently being implemented at the practice level will lead to the child health and health care utilization outcomes extensively demonstrated in the literature.
What’s Known on This Subject:
Medical home transformation is led by practice-level assessment, but much of the evidence supporting the medical home derives from individual-level assessment based on parental perception. The association between these 2 levels of assessment is unknown.
What This Study Adds:
Among Boston-area community health centers, there was no association between the individual- and practice-level assessments of the medical home. This highlights the need for studies supporting the child health benefits of medical home practice transformation.
The medical home was developed by the American Academy of Pediatrics as a model of patient care that is “accessible, continuous, comprehensive, family-centered, coordinated, compassionate and culturally effective.”1 Although conceived for all children, it was initially promoted nationally by the Maternal and Child Health Bureau’s Division of Services for Children With Special Health Care Needs (CSHCN).2–5 Early studies focused on that population, associating varying measures of the medical home with numerous positive health outcomes, such as decreased emergency department utilization6–8 and hospitalization rates.9–11 Recent studies in the general pediatric population have demonstrated similar positive associations with child health and health care utilization outcomes.12–14 The medical home, in part based on this evidence, is now promoted by professional medical societies15 and federal government agencies16 as an optimal model for delivering primary care.
Implementation of the medical home at the practice level is guided by standards, most notably those developed by the National Committee for Quality Assurance (NCQA)17; to date, >6700 practices nationwide have been deemed medical homes according to these standards.18 However, the majority of studies on the pediatric medical home have operationalized the medical home from the perspective of parents, that is, at the individual level. A commonly used method that operationalizes the medical home at the individual level is the National Survey of Children’s Health (NSCH), which measures the medical home by using parental response to questions designed to assess 5 of the 7 components defined by the American Academy of Pediatrics.19 To date, the NSCH has been used in >30 peer-reviewed articles20; most studies have demonstrated a beneficial impact of the medical home on child health and health care utilization outcomes.
However, little is known about how these 2 measurement methods compare, raising the possibility that pediatric practices could be implementing a model that may not yield the expected child health benefits.21,22 The objective of this study was to compare parental assessment of the medical home with practice-level assessment. We hypothesized that there would be a positive association between the 2 methods.
The study was a cross-sectional survey of parents whose children receive care at 1 of 6 Boston-area community health centers (CHCs), with an associated assessment of the CHCs as medical homes according to the NCQA method.
The 15 CHCs of the Boston HealthNet consortium range in size from ∼10 000 to >300 000 visits annually. All serve predominantly poor, minority populations.23 Four CHCs with previous NCQA Patient-Centered Medical Home (PCMH) recognition were excluded to maximize heterogeneity in NCQA scores; the first 6 eligible CHCs that responded to a recruitment e-mail were chosen for the practice-level assessment. English- and Spanish-speaking caregivers (referred to henceforth as “parents”) who brought their children to the participating CHCs for pediatric care were eligible for the individual-level assessment and were approached by a research assistant for recruitment. Parents were serially recruited at each CHC until the sample of 30 was enrolled, excluding parents <18 years of age. A $10 Target gift card was offered as remuneration.
We administered the 19-question medical home measure of the 2011 NSCH to each consented caregiver. The parent was also asked 12 questions from the 2011 NSCH about health care utilization and sociodemographic characteristics. Questions were rewritten by the study team to adapt the verbal method of the NSCH to a written survey instrument (Supplemental Appendix 1) and were administered by a research assistant in either English or Spanish. The Spanish-language questionnaire was back-translated by approved translation staff. The presence of a medical home for each child was determined based on the response of his or her caregiver to the NSCH questions according to the NSCH method. Briefly, questions for each component of the medical home were coded on an ordinal scale (never, sometimes, usually, and always) and were then recoded as numerical values. A component was considered present if the average score of the associated questions was ≥67 (ie, usually or always). The presence of a personal doctor or nurse, a usual source for sick and well care, and family-centered care were assessed for each child, and problems getting necessary referrals and care coordination needs were screened for and then assessed with additional ordinal-scaled questions. Components for which a child screened negative (eg, no need for referrals) were considered to be present; therefore, a child was deemed to have a medical home only if all needed components were present.24
The medical home status of the CHC pediatric clinics was assessed according to the NCQA PCMH checklist. The checklist consists of 28 elements divided between 6 standards: enhance access and continuity, identify and manage patient populations, plan and manage care, provide self-care support and community resources, track and coordinate care, and measure and improve performance. The assessment was performed by the study’s lead author (W.E.L.), who completed the NCQA online training modules for PCMH assessment before the study and was blinded to the individual-level assessments.17 In conjunction with the medical director of each CHC, W.E.L. completed the 2011 PCMH scoring checklist (Supplemental Appendix 2) according to the published Standards and Guidelines.17 The medical directors attested to the ability of their respective CHCs to provide the necessary documentation to support the determined score for each element. The presence of a medical home for each CHC was determined according the NCQA method as both a continuous score (0–100) and a categorical tier (Not a Medical Home and Tiers 1–3; Supplemental Appendix 2). CHCs that failed a “must-pass” element under NCQA methods were still given a continuous score based on the rest of their elements but were not placed in a tier.
The unit of measurement in the study was the parent. Because the smallest NCQA PCMH category covers a range of 16 points (Tier 3, 85–100), we determined a clinically relevant effect size to be 0.08 (ie, a range of 8 points). In previous national studies using the NSCH method, 56% to 58% of the general pediatric population reported having a medical home.12,25 Assuming a correlation between groups of 0.2 and equal cluster sizes, a power of 80%, and a significance level of 0.05, 6 clusters with a total sample size of 166 would be needed to detect a difference of 40% of an SD across comparison groups. We therefore recruited 180 subjects, 30 at each of the 6 sites.
Statistical analysis was performed with SAS software (version 9.2, SAS Institute, Inc, Cary, NC). Survey-specific procedures were used to account for the clustering of NSCH assessments at the physician and practice levels. Simple descriptive statistics were used to present the range of results from the NSCH questions compared with the results of the assessment of the CHC pediatric clinics according to the NCQA PCMH checklist. The association of the presence of a medical home between the NSCH and NCQA methods was assessed with both linear and logistic regression modeling, with the NCQA score (both continuous and tiered) as the outcome and the binary NSCH assessment as the primary predictor. The SAS procedure for logistic regression of complex survey design was adjusted to a cumulative logistical model for our analysis of the tiered (ordinal) outcome. A priori, we controlled for sociodemographic variables determined to be significant in previous work.25
Institutional Review Board
The Boston Medical Center/Boston University Medical Campus Institutional Review Board determined that this study was exempt from human studies review.
Of the 180 parent–child dyads recruited for the study, the mean child age was just over 6 years, 40.8% were black, and slight majorities were Hispanic (53.1%) and boys (54%). Most families lived in primarily English-speaking households (57.2%) of ∼4 members, with almost exactly half of families earning ∼200% of the federal poverty level. Large majorities had current health insurance (87.7%) and came from households wherein no parent had a college degree (69.4%). Table 1 lists the parent and child sociodemographics across the CHCs, all of which varied across practices. Overall, 51.6% reported their pediatric practice as a medical home, ranging from 26.1% to 70% across the CHCs.
Of the 6 CHCs recruited and examined at the practice level, 3 scored at Tier 2, with a mean score of 75.5. Two scored in the highest tier, and 1 failed a “must-pass” element and was not deemed a medical home. Table 2 lists both continuous and tier NCQA scores by CHC, including the scores for each standard. Of note, the 1 CHC not deemed a medical home by NCQA methods was reported as an NSCH-assessed medical home by 68% of the parents bringing their children to that CHC (Table 3).
Regression modeling demonstrated nonsignificant associations between both the continuous and categorical (tier) practice-level NCQA scores and the individual-level NSCH assessment of the medical home derived from parental report, with a β of −2.8 (95% confidence interval [CI] −13.4 to 7.8) and an adjusted odds ratio of 2.2 (95% CI, 0.8 to 5.7), respectively. The full models are presented in Fig 1. In independent modeling of the 6 NCQA Standards, only 1, Track and Coordinate Care, demonstrated a significant association with the NSCH assessment of the medical home (β = .4; 95% CI, .1 to .6; Tables 4 and 5).
To our knowledge this is the first study examining the association between measurements of the medical home derived from parental perception and from the practice level. Our study attempted to contribute to the evidence base on the beneficial impact of the medical home on child health and health care utilization by linking NCQA-transformed practices to the NSCH-assessed medical home, which has been previously demonstrated to be associated with positive child outcomes in the general pediatric and CSHCN populations.6,14 We believe such a link is necessary to support the continued transformation of pediatric practices into medical homes at the practice level. However, our study failed to demonstrate a significant association between the individual-level NSCH, derived from parental report, and practice-level NCQA assessments of the medical home.
The NSCH has associated the medical home with improved health care utilization patterns in both the emergency department and preventive care,13,26 decreased unmet health needs,26 and increased healthy activities.27 Similar evidence garnered largely from other individual-level assessments in both the general pediatric population and more extensively in CSHCN6,14 has led to the promotion of the medical home by a number of institutional and government organizations as an optimal model of primary care for all children.16,28
However, these individual-level assessments derived from parental report are, by definition, subjective. Therefore, a given practice can be categorized as a medical home for only some of the children it cares for. Conversely, a practice-level assessment such as the NCQA categorizes a medical home for all families within a practice regardless of the individual parent’s or caregiver’s perspective. This natural discrepancy between the individual- and practice-level measurements that we found probably results from the emphasis of the NCQA on aspects of a medical home of which a caregiver is unlikely to be aware, such as the use of electronic data to identify and track patient populations. Of the 6 “must-pass” NCQA elements, the caregiver of a healthy child may be aware only of Access and Care Management. Tellingly, the only NCQA standard associated with the NSCH in our analysis, Track and Coordinate Care, deals entirely with patient-facing elements of test and referral tracking and care coordination. Conversely, the element most closely aligned to the individual-level assessment, measurement of the Patient/Family Experience, accounts for only 4 of the 100 points of the NCQA PCMH score. Of note, the 2014 NCQA PCMH Standards have a greater family-level perspective, with the use of the Consumer Assessment of Healthcare Providers and Systems Survey, which may help better align the measures.
The failure to demonstrate an association between the NSCH and NCQA methods may be particularly significant given the mixed results of previous studies directly measuring the outcomes of NCQA PCMH certification on adult health and health care utilization outcomes. Some studies have found an association between NCQA certification and decreased emergency department utilization,29–31 and 1 also demonstrated improvement in primary care quality measures (improved blood pressure control for hypertensive patients and increased breast cancer screening).30 However, a pilot study in Pennsylvania did not demonstrate any changes in utilization or cost.21 Additional studies assessing the impact of NCQA PCMH certification along with other practice-based measurements of the medical home focusing on the pediatric population are needed.
Our study has limitations. First, for logistical reasons we did not administer the full NSCH survey. However, we administered the entire medical home section as designed, and therefore we do not expect our method to significantly affect our study’s validity. Likewise, although the CHCs recruited to our study did not submit for official NCQA PCMH scoring or provide supporting documentation, the “pre-assessment” is explicitly built into the official process and was performed according to NCQA methods, with expected correlation to official scores. Also, at the time of the study we used the most recent NCQA PCMH scoring method that was available, namely the 2011 version. Therefore, our findings may not be applicable for the 2014 and future NCQA PCMH Standards. Because of staff and logistical limitations, we obtained on-site convenience samples, which may have introduced social desirability and recall biases; despite this threat, we did not find any concordance between the parent- and practice-level assessments. Our study findings may not be generalizable because we took a small sample of parents who received pediatric care at urban CHCs; however, our sociodemographics demonstrate good heterogeneity, and the study was appropriately powered. Finally, the cross-sectional nature of our study would not capture how individual-level measures might change as CHCs go through the NCQA recognition process.
Our results suggest that medical home transformation of pediatric practices may not be associated with parental assessment of the medical home. Therefore, it is possible that NCQA-guided practice transformation may not lead to the beneficial child outcomes previously associated with the NSCH-assessed medical home at the population level. However, continued efforts to evaluate such medical home transformation are necessary given the resources being directed toward transforming pediatric practices into certified medical homes.
We thank Jennifer Gilbert, BA, for her contributions.
- Accepted December 18, 2014.
- Address correspondence to Webb E. Long, MD, MSci, University of Massachusetts Medical School/UMass Memorial Children’s Medical Center, Department of Pediatrics, Division of General Pediatrics, 55 Lake Ave North, Worcester, MA 01655. E-mail:
Dr Long conceptualized and designed the study, supervised data collection, performed the initial analyses, and drafted the initial manuscript; Dr Garg conceptualized and designed the study and reviewed and revised the manuscript; and both authors approved the final manuscript as submitted.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Supported by the Joel and Barbara Alpert Endowment for the Children of the City.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- American Academy of Pediatrics Ad Hoc Task Force on Definition of the Medical Home
- Medical Home Initiatives for Children With Special Needs Project Advisory Committee. American Academy of Pediatrics
- Sia C,
- Tonniges TF,
- Osterhus E,
- Taba S
- American Academy of Pediatrics Council on Pediatric Practice
- ↵National Center for Medical Home Implementation. History of medical home at the AAP. Available at: http://www.medicalhomeinfo.org/about/. Accessed October 18, 2014
- Christakis DA,
- Mell L,
- Koepsell TD,
- Zimmerman FJ,
- Connell FA
- Strickland BB,
- Jones JR,
- Ghandour RM,
- Kogan MD,
- Newacheck PW
- Long WE,
- Bauchner H,
- Sege RD,
- Cabral HJ,
- Garg A
- ↵American Academy of Family Physicians, American Academy of Pediatrics, American College of Physicians, American Osteopathic Association. Joint principles of the patient-centered medical home. 2007. Available at: http://www.aafp.org/dam/AAFP/documents/practice_management/pcmh/initiatives/PCMHJoint.pdf
- National Maternal and Child Health Resource Center
- National Committee for Quality Assurance
- ↵National Committee for Quality Assurance. NCQA patient-centered medical home [brochure]. 2014. Available at: www.ncqa.org/Portals/0/Programs/Recognition/PCMH/PCMH-2014_Brochure-web-1.pdf. Accessed November 12, 2014
- ↵Data Resource Center for Child and Adolescent Health. Articles. Available at: www.childhealthdata.org/action/articles. Accessed August 19, 2014
- ↵National Opinion Research Center. Findings from a site visit to Boston HealthNet of Boston, Massachusetts. Vol 2012. 2010. Accessed August 2
- Long WE,
- Bauchner H,
- Sege RD,
- Cabral HJ,
- Garg A
- Strickland BB,
- Jones JR,
- Ghandour RM,
- Kogan MD,
- Newacheck PW
- National Committee for Quality Assurance
- Rosenthal MB,
- Friedberg MW,
- Singer SJ,
- Eastman D,
- Li Z,
- Schneider EC
- Copyright © 2015 by the American Academy of Pediatrics