Using Existing Population-Based Data Sets to Measure the American Academy of Pediatrics Definition of Medical Home for All Children and Children With Special Health Care Needs
Objective. National health goals include ensuring that all children have a medical home. Historically, medical home has been determined by the presence of a usual or primary source of care, such as a pediatrician or a family physician. More recent definitions expand on this simplistic notion of medical home. A definition of medical home set forth by the American Academy of Pediatrics (AAP) includes 7 dimensions and 37 discrete concepts for determining the presence of a medical home for a child. Standardized methods to operationalize these definitions for purposes of national, state, health plan, or medical practice level reporting on the presence of medical homes for children are essential to assessing and improving health care system performance in this area. The objective of this study was to identify methods to measure the presence of medical homes for all children and for children with special health care needs (CSHCN) using existing population-based data sets.
Methods. Methods were developed for using existing population-based data sets to assess the presence of medical homes, as defined by the AAP, for children with and without special health care needs. Data sets evaluated included the National Survey of Children With Special Health Care Needs, the National Medical Expenditures Panel Survey, the Consumer Assessment of Health Plans Study Child Survey (CAHPS), and the Consumer Assessment of Health Plans Study Child Survey—Children With Chronic Conditions (CAHPS-CCC2.0H). Alternative methods for constructing measures using existing data were compared and results used to inform the design of a new method for use in the upcoming National Survey of Children's Health. Data from CAHPS-CCC2.0H are used to illustrate measurement options and variations in the overall presence of medical homes for children across managed health care plans as well as to evaluate in which areas of the AAP definition of medical home improvements may be most needed for all CSHCN.
Results. Existing surveys vary in their coverage of concepts included in the AAP definition of medical home and, therefore, in their capacity to evaluate medical home for children with and without special health care needs. Using data from CAHPS-CCC2.0H, the overall proportion of children who were enrolled in managed care health plans and met criteria for having a medical home varied from 43.9% to 74% depending on the specific scoring method selected for these items. Wide variations across health plans were observed and were most prominent in the areas of “accessible care” and “comprehensive care.” Performance was uniformly poorest in the area of “coordinated care” and for CSHCN. Although children with a personal doctor or nurse were more likely to meet the AAP criteria for having a medical home, simply having a personal doctor or nurse was not highly predictive of whether a child experienced the other core qualities of a medical home (positive predictive value: .50; negative predictive value: .59).
Conclusions. Despite differences across existing surveys and gaps in concepts represented, we believe that the AAP definition of medical home can be well represented by the small subset of concepts represented in the National Survey of Children With Special Health Care Needs and the CAHPS-CCC2.0H. A less comprehensive yet still worthwhile measure is possible using the Medical Expenditures Panel Survey. The varying degrees of empirical evidence and consensus for each of the AAP definition domains for medical home suggest the need for constructing measures that also vary in terms of criteria for determining that a child does or does not have a medical home. In addition to a simple “yes or no,” or rate-based, measure, a continuous medical “homeness” score that places a child or group of children on a continuum of medical “homeness” is also valuable. Findings indicate that health plans have an important role to play in ensuring medical homes for children in addition to medical practices and those who set policies that guide the design and delivery of health care for children. Overall, using existing population-based data, a measure of medical home that is aligned with the AAP definition is feasible to include in the annual National Healthcare Quality Report, in state reports on the quality of Medicaid, State Children's Health Insurance Program, and Title V programs as well as to evaluate performance on the Healthy People 2010 objectives and the President's New Freedom Initiative.
- medical home
- health care quality
- children with special health care needs
National health goals include ensuring that children have a medical home.1 Historically, medical home has been determined by the presence of a usual or primary source of care, such as a pediatrician or a family physician.2,3 More recent definitions expand on this simplistic notion of medical home.4,5 Specifically, in 2002, the American Academy of Pediatrics (AAP) issued a policy statement providing a detailed definition of medical home that identifies 7 dimensions and 37 discrete concepts for determining the presence of a medical home for a child. The AAP definition outlines an ideal model of care for children whereby health care is delivered or directed by well-trained physicians who provide primary preventive, acute, and chronic condition care and that this care is 1) accessible, 2) continuous, 3) comprehensive, 4) family-centered, 5) compassionate, 6) culturally effective, and 7) coordinated with specialized services provided outside the primary care setting. Agreed-on methods to measure medical home quantitatively are needed to assess and guide improvements in care for children in this area. Ideally, a common measurement approach will be used at medical practice levels and the national, state, and health care plan levels. This consistency will provide a basis for comparing and tracking performance over time and across each level of the health care system that plays a role in advancing the AAP concept of medical home for children.
Although consensus has grown regarding a definition of medical home, several challenges to standardized measurement remain, including 1) incomplete empirical evidence and consensus regarding the absolute and relative costs and benefits of discrete components of the AAP definition of medical home; 2) lack of availability of comparable sources of data for measuring certain aspects of medical home in a consistent way across children and settings of care; 3) ongoing debate and uncertainty regarding how to measure specific concepts included in the AAP definition of medical home, such as “ongoing source of care,” continuity, and coordination of care; 4) selecting which of the numerous concepts of medical home send the strongest signal and can be most feasibly and validly measured when data collection methods, such as parent surveys, must be made as parsimonious as possible; and 5) designing a specific measure construction, scoring, and grading method that sets high and realistic expectations for the implementation of the medical home model of care and maximizes the value of measures for stimulating and informing improvements.
The AAP definition of medical home greatly advances the operationalization of this important model of care for children. However, it does not address the measurement issues outlined previously and leaves a great deal of room for interpretation when translating this definition to a precise measurement method. The most historically central concept of medical home, that of having a usual source of care, provides a good illustration of this issue. The AAP definition states that care should be “delivered or directed” by “physicians who provide primary care.” This definition leaves open several issues that must be resolved for purposes of measurement. For example, decisions must be made regarding 1) whether any or only certain types of physicians qualify as providing primary care and whether a child should have only 1 physician who provides this care; 2) which pediatric clinician to which to anchor a parent and/or a child's assessment of family-centeredness and other concepts of medical home in cases in which the pediatric clinician who delivers care is different from the person who directs the care or when a child has >1 pediatric clinician who delivers and/or directs care; and 3) accounting for cases in which parents consider nonphysician clinicians to be a child's primary source of care (eg, nurse practitioners, physician assistants). Similar challenges emerge when translating other medical home concepts into a precise and parsimonious measurement method.
As work progresses to specify the measurement methods needed to measure quantitatively the presence of medical homes for children as defined by the AAP, we are fortunate that several population-based data sets that include variables relevant to many components of the AAP definition of medical home already exist: 1) the National Survey of Children with Special Health Care Needs (National Survey of CSHCN),6 2) the Medical Expenditures Panel Survey (MEPS),7 3) the Consumer Assessment of Health Plans Child Survey,8 and 4) the Consumer Assessment of Health Plans Survey— Children With Chronic Conditions (CAHPS-CCC2.0H).9 These existing surveys make measurement of medical home at the national, state, and health care plan levels possible today, even as improved methods are developed and implemented over time.10
The objective of this study was to identify methods to measure the presence of medical homes for all children and for children with special health care needs (CSHCN) using existing population-based survey data sets. Measurement options were compared, and the use of the CAHPS-CCC2.0H to compare performance across health plans is explored. Findings from this study were used to inform the development of a parsimonious set of parent survey items to measure medical home in the 2003–2004 National Survey of Children's Health (NSCH), which will provide ~2000 cases of child data per state.
Four steps compose the methods for this study. First, the AAP definition of medical home was evaluated to identify which of the 7 domains and 37 discrete concepts included in this definition are possible to measure using parent-reported survey methods. Second, any survey items relevant to each of these concepts of medical home were identified for each of the 4 parent surveys considered here (National Survey of CSHCN, 2000 MEPS, CAHPS, CAHPS-CCC2.0H). Third, alternative methods for using data based on these survey items to construct measures of medical home were specified. Methods varied in 3 ways: 1) whether and how items on the “usual source of care” or “has 1 personal doctor” criterion are used; 2) whether the measure is a “yes or no” rate-based measure (eg, creates a percentage with a medical home) or a score-based measure (eg, creates a mean medical “homeness” score on a 0–100 continuous scale); and 3) whether a child's care experience as reported by parents must meet a threshold score equivalent to a child usually or always receiving the desirable aspect of a medical home a) on average, across all the AAP definitional domains or b) on every domain.
Next, alternative measurement methods were implemented and compared using existing data. Because National Survey of CSHCN and 2000 MEPS data were not yet publicly available at the time of this analysis, results presented here are based on the CAHPS-CCC2.0H survey. Although these data are not nationally representative, they can be used to illustrate methods for measuring medical home that can be applied to these other surveys. In addition, CAHPS-CCC2.0H data allow us to examine the use of measures of medical home to compare performance across health plans, which is not possible using National Survey of CSHCN or MEPS data.
The CAHPS-CCC2.0H data collected for children in 1 state Medicaid program were used and represent 10 792 children who were aged 12 years or younger and enrolled in 1 of 19 managed care health plans. Children in the state for which data were collected are eligible for Medicaid under the Temporary Aid to Needy Families program if family income is <150% of the federal poverty level or is 185% of the federal poverty level if the child is younger than 1 year.
Findings from the 4 steps outlined above were used to guide the construction of a 13-item parent survey to measure the presence of medical homes for children with and without special health care needs for use in the upcoming NSCH (target n = 100 000). First, 49 items were specified representing each of the 7 AAP medical home definition domains and alternative ways to capture information on these domains. These items were fielded in the 2003 pretest for the NSCH, and psychometric, item-reduction, and validity analyses were conducted to arrive at the short medical home survey. The items to be used in NSCH are available by contacting the primary author of this article. Future articles will report on the analyses that led to their development, as well as findings on their use to construct a measure of medical home when data from the NSCH are available.
Construction of Measures of Medical Home Using the CAHPS-CCC2.0H
Alternative scoring methods for measuring the AAP concept of medical home were constructed using data from CAHPS-CCC2.0H. Primarily, the alternative methods vary in terms of the type of metric that is produced (eg, a percentage of children with a medical home vs a medical “homeness” 0–100 continuous score) and the level of performance expected before a child is determined to have a medical home (eg, on average, the child experiences good care across all domains of care vs the child experiences good care on every domain of care).
The CAHPS-CCC2.0H was administered using the National Committee on Quality Assurance Health Plan Employer Data and Information Set data collection specifications.11 The CAHPS-CCC2.0H data were collected across managed care health plans in a statewide Medicaid program. A total of 19 managed health care plans were sampled. Five health care plans were eliminated from this analysis because of small sample sizes and/or the inadvertent use of an administration protocol that differed from the rest of the plans. NCS Pearson administered the survey on behalf of the state's Medicaid agency, and the sample was drawn by Abt Associates.
The sampling frame for CAHPS-CCC2.0H included all child Medicaid clients who were 12 years of age or younger and were continuously enrolled in a managed care health plan for at least 6 months as of January 2002. A break in enrollment of up to 45 days was allowed. Two samples were drawn for each health plan: 1) a population-based sample of 1275 children and 2) an oversample of children whose patient encounter utilization data indicated a higher probability of having a special health care need. This latter sample was drawn using a prescreening algorithm specified in the Health Plan Employer Data and Information Set guidelines. The target sample size for the oversample population was 1650, but only 5 health care plans had sufficient enrollment to reach that goal after the population-based sample had been drawn. The remaining health care plan oversample sizes ranged from 265 to 1390. Data were combined from both samples for each plan. All results presented here are adjusted to the mean proportion of children identified as having a special health care need across the 14 health care plans. This proportion was based on responses to the CSHCN Screener, which is included within CAHPS-CCC2.0H.12,13
The survey was administered by mail with a telephone follow-up for those who did not respond to the mailed survey. A maximum of 2 mailings and 3 telephone calls were made for each child in the starting sample, with 3 weeks between mailing and telephoning. The survey was administered in English only.
The 34 CAHPS-CCC2.0H survey items relevant to the AAP definition of medical home were used to calculate alternative measures of medical home. Table 1 summarizes which CAHPS-CCC2.0H were placed into each of the AAP definitional domains and how survey items with different response options were aligned for purposes of measurement construction. As can be seen, no items were available for the continuity of care domain, and the “family-centered” and “compassionate care” domains were combined.
Three steps were required to create medical home scores for each child and subsequently to calculate the proportion of children in each health plan who did or did not meet criteria for having a medical home. First, for each child, a mean score was calculated for each of the AAP medical home domains addressed by CAHPS-CCC2.0H: accessible, family centered/compassionate, comprehensive, coordinated, and culturally effective. These mean scores ranged from 0 to 100 points. Next, the mean scores created in step 1 were used to create 2 dichotomous versions of medical home
Across All Scoring Model. Here children with an overall mean score of 75 points or above across all domains are categorized as having a medical home. Those who do not meet this criterion are categorized as not having a medical home. A score of 75 points or above is equivalent to a child on average, usually or always experiencing aspects of care associated with having a medical home. If a child does not have an average score of 75 points or above in 1 or more domains, then he or she may still meet criteria for having a medical home as long as his or her overall score across all domains is 75 or greater. A score of <75 points indicates that a child experiences aspects of a medical home less than “usually or always.”
On Each Scoring Model. Here the percentage of children with a mean score of 75 points or above on each domain are categorized as having a medical home. If a child does not have an average score of 75 points or above in 1 or more domains, then he or she does not meet criteria for having a medical home.
Finally, whether a child had 1 person whom the parent considered to be the child's personal doctor or nurse (PDN) was layered on top of each of these 2 rate-based versions of medical home described. In this way, each of the 2 rate-based measures was calculated for children who did and did not have 1 person whom the parent considered to be the child's PDN. This was done to examine the impact of this criterion and to evaluate the association between “having a usual source of care” and the more expanded AAP definition of medical home.
Conclusions derived using the alternative measurement construction methods described here were compared across 14 health care plans as well as for children with and without special health care needs. Differences in the percentage of children with a medical home using alternative methods were evaluated using odds ratios as well as χ2 tests of significance. A minimum P = .05 was set. Spearman rank correlations were calculated to evaluate the stability in the ranking of health care plan medical home scores and rates when alternative measurement methods were used. Predictive values were calculated to examine the degree to which having 1 person who is a child's PDN predicts whether that child also meets the more expanded AAP criteria for having a medical home as operationalized here using the CAHPS-CCC2.0H survey.
Eighty percent of the 37 concepts included in the AAP definition of medical home are appropriate for or specifically require parent-reported information. An example of a concept not considered appropriate for parent report is “a central database containing all pertinent information about child's care is maintained.” Most of the 37 AAP medical home concepts would require multiple parent survey items to capture the required information. As such, even if all concepts were amenable to parent report, many more than 37 survey items would be required to fully operationalize this definition.
The existing national, state, and health care plan levels survey data sets reviewed here varied widely in their coverage of the concepts included in the AAP definition of medical home and according to whether a measure of medical home could be stratified for children with and without special health care needs. A range of 11 to 28 relevant items were present across each of the 4 surveys reviewed, representing all but the continuity of care domain. The CAHPS-CCC and National Survey of CSHCN provide the most robust measurement approach (Table 2).
The 4 surveys reviewed share several of the same or very similar survey items relevant to the family-centered care domain. Three of the 4 surveys share a common method for identifying CSHCN, and 2 share items on evaluating coordination of care and culturally effective care. All other domains are measured differently across surveys. Although these differences prevent comparison in findings across surveys, a minimum measure addressing the family-centered care domain and “having a usual source of care” concept may be possible to develop for this purpose. In CAHPS-CCC2.0H, we find that the family-centered care domain is correlated .76 with the mean score of medical home. Specifically, the probability that a child whose parent reports usually or always experiencing family-centered care also meets criteria for having a medical home using the “across all” measure is .77 (positive predictive value of family-centered care: .77). Conversely, the probability that a child does not meet the medical home criteria given that they also do not usually or always experience family-centered care is .70 (negative predictive value of family-centered care: .70).
Using the survey items and response coding illustrated in Table 1, the mean medical “homeness” score for CSHCN was 80.9 and for children without special health care needs was 82.0. As shown in Table 3, the proportion of children with a medical “homeness” score of 75 points or above varies from 43.9% to 74.0%, depending on whether the “across all” scoring model or the “on each” scoring model is used and whether having 1 person who is the child's PDN is included as a criterion (Table 3). Scores for CSHCN are lower for each method and vary from 38% to 71.2%.
The proportion of children who met criteria for having a medical home also varied significantly across the 14 health care plans (P < .05). For example, using the “across all” scoring model with the PDN criterion included, plan scores ranged from 49.4% to 76.8% for all children and 45.7% to 73.0% for CSHCN. As expected, scores are uniformly lower for the more stringent “on each” scoring model and uniformly higher when the PDN criterion is not included. Again, in nearly all health care plans evaluated, CSHCN are less likely to meet the medical home criteria (Fig 1), in part, because CSHCN have more needs and, therefore, more chances for needs to be met.
Variation across health care plans was greatest for the accessible care domain (range: 58.3%–78.1%) and comprehensive care domain (range: 72.6%–91.0%), and the lowest scores across plans were consistently found in the coordinated care domain (range: 57.6%–67.3%). Variation between CSHCN and children without special health care needs was greatest on the comprehensive care and family-centered care domains. Again, scores were lowest for both groups of children in the coordinated care domain (Table 4).
Although any 1 of the 14 managed care health care plans' medical home performance scores varied widely depending on the measurement construction method used, rarely did a plan's overall performance ranking relative to other health plans change on the basis of the method used. Spearman rank correlations between methods is .98 to .99 (Table 5).
Using the “across all” scoring method, the odds are ~2 times greater that a child with 1 person whom the parent considers to be the PDN meets criteria for having a medical home compared with children whose parents do not report the presence of 1 PDN. The odds ratio between those with and without a PDN range from 1.54 to 2.40 across the domains assessed for CSHCN and 1.54 and 2.64 for children without special health care needs (Table 6).
Although children with 1 PDN are more likely to have a medical home, overall, the PDN criteria by itself, as measured in CAHPS-CCC2.0H, is not highly predictive for having a medical home using the more expanded AAP definition. Specifically, the probability that a child with 1 person as the PDN meets criteria for having a medical home using the “across all” scoring method is .50. The probability that a child who does not have 1 person as the PDN does not meet criteria for having a medical home using the “across all” scoring method is .59. It is important to note that some parents not reporting 1 person as the PDN do so because their child has more than 1 person they consider to be a PDN.
Measures of medical home are possible to construct using each of the national, state, and health care plan levels surveys and data sets evaluated here. However, with the exception of alignment in survey items used to identify CSHCN and some common items to evaluate family-centered care, these surveys vary substantially in how each of the remaining domains represented in the AAP definition of medical home are assessed. Overall, a small subset of the 37 distinct concepts included in the AAP definition are represented across surveys. However, it is unlikely that it would be feasible or even necessary for any future national, state, health care plan, or medical practice level survey to capture information about each of these concepts. We estimate that it would require a minimum of 90 survey items to do so, including a complicated sampling and survey administration process for identifying the small number of children for whom many of the concepts apply. In such a comprehensive survey on medical home, we expect that many of the items would be highly correlated, resulting in a more analytically powerful but inefficient and burdensome measurement tool.
Despite differences across existing surveys and gaps in concepts represented, the AAP definition of medical home can be well represented by the small subset of concepts represented in the National Survey of CSHCN and CAHPS-CCC2.0H. A less comprehensive yet still worthwhile measure is possible using the National MEPS. This is largely because the National MEPS includes core family-centered care items that were shown here to be highly correlated with a more comprehensive measure of medical home. In addition, the MEPS allows findings to be stratified for children with and without special health care needs using the CSHCN Screener.11,12
To advance medical home as a serious model of care for children, it will be important to track and compare the overall presence of medical homes for children as well as identify in which domains of the AAP definition of medical home improvements are most needed and for which subgroups of children. Existing survey data sets can be used to begin this process. However, over time, it will be ideal if future versions of the National Survey of CSHCN, CAHPS-CCC, and MEPS and other similar surveys are aligned in the way medical home definition domains are assessed. The survey items identified for inclusion in the NSCH may provide a useful starting point for this purpose.
In this study, considerable variation in performance across health care plans in the percentage of children who meet the overall criteria for medical home was observed. We found that health plans vary in terms of the domains in which their performance was higher versus lower. These findings indicate that some systems of care may do a better job than others in certain areas and that many lessons for improvement may be learned by further examining where performance is higher or lower across plans as well as other units of analysis, such as medical practices. These findings also indicate that variations are likely to be attributable not only to medical practices and that health plans have an important role to play in ensuring medical homes for children. Few health plans scored well in the area of coordination of care, confirming known deficiencies in the capacity and performance of the health care system in this area. Lower overall and domain-specific medical home scores for CSHCN support the focus of the Healthy People 2010 medical home objective on CSHCN.1
There are varying degrees of empirical evidence and consensus to support each of the AAP domains for medical home as recommended guidelines for care. As such, we recognize the need for constructing measures that also vary in terms of how stringently medical home is measured. Results from the “across all” and “on each” scoring models illustrated here do send a different signal regarding the proportion of children with a medical home. However, findings from this study suggest that these alternative methods do not alter conclusions regarding the relative performance across health care plans, between groups of children, or in terms of identifying in which domains of the AAP definition of medical home improvements in performance are most needed. In addition to the use of the simple “yes or no” rate-based measures constructed here, we recommend the use of a medical “homeness,” 0 to 100 continuous score in cases in which it is important to place any given child or group of children on a continuum of medical “homeness” or understand the distribution of medical “homeness” in any given population or unit of analysis.
Potential users of the medical home measure construction methods outlined here need to determine their comfort in requiring that a child experience, on average, the qualities of a medical home across all domains versus requiring that a child experience the qualities of a medical home on each domain. The latter measure would disqualify a child for not meeting criteria on 1 or more domains, whereas the former measure allows for some deficiencies in care as long as overall care, on average, usually or always meets criteria for having a medical home. Regardless of the perspective of potential users on the methods outlined here, it should be noted that seemingly small variations in the way the same survey items are used to construct a measure of medical home can lead to very different scores for an individual health care plan or group of children and must be considered carefully.
Findings verify that having a usual source of care, as measured in CAHPS-CCC2.0H, is helpful, but in no way is this medical home criterion sufficient for predicting whether a child experiences the qualities of a medical home as outlined by the AAP. Conversely, many children usually or always experience many of the qualities of a medical home even when they do not have a usual source of care as defined in CAHPS-CCC2.0H. Given these findings, the expanded definition of medical home set forth by the AAP is more likely to lead to the qualities of care represented in this definition—qualities that many may have historically assumed would be present simply by having a usual source of care.
Overall, although improvement and modifications to future versions of existing surveys are recommended, findings from this study suggest that existing data can be used to evaluate the presence of medical homes for children. It is feasible to include a measure of medical home for all children and for CSHCN in the annual National Healthcare Quality Report,13 in state reports on the quality of Medicaid, State Children's Health Insurance Program, and Title V programs as well as to evaluate performance on the Maternal and Child Health Bureau's 2010 Core Outcomes for CSHCN.14
This study was supported by the Maternal and Child Health Bureau, the David and Lucile Packard Foundation, the Centers for Disease Control and Prevention. This research was guided by an advisory group sponsored by the Maternal and Child Health Bureau. This group met twice in person and numerous times via telephone throughout the course of this and related projects on the measurement of medical home. Members included Barbara Starfield, Johns Hopkins University, Bonnie Strickland, Maternal and Child Health Bureau; Carl Cooley, Crotched Mountain Rehabilitation Center; Christopher Kus, Association of Maternal and Child Health Programs; James Bryant, Ohio State Department of Health; Jan-Hicks Thompson, Washington State Department of Health; Jeanne McAllister, Hood Center for Children & Families; Maureen Mitchell, Family Voices; Merle McPherson, Maternal and Child Health Bureau; Nora Wells, Family Voices; Paul Newacheck, University of California, San Francisco; Richard Antonelli, University of Massachusetts; Stephen J. Blumberg, National Center for Health Statistics; and Tom Tonniges, American Academy of Pediatrics.
For more information about the Child and Adolescent Health Measurement Initiatives medical home measurement work, go to www.cahmi.org/cahmi.
- Reprint requests to (C.D.B.) Kaiser Center for Health Research, 3800 N Interstate Ave, Portland, OR 97227; E-mail:
- Copyright © 2004 by the American Academy of Pediatrics