SUPPLEMENT ARTICLE |
* Child and Adolescent Health Measurement Initiative, Kaiser Center for Health Research, Portland, Oregon
| ABSTRACT |
|---|
|
|
|---|
Methods. Methods were developed for using existing population-based data sets to assess the presence of medical homes, as defined by the AAP, for children with and without special health care needs. Data sets evaluated included the National Survey of Children With Special Health Care Needs, the National Medical Expenditures Panel Survey, the Consumer Assessment of Health Plans Study Child Survey (CAHPS), and the Consumer Assessment of Health Plans Study Child SurveyChildren With Chronic Conditions (CAHPS-CCC2.0H). Alternative methods for constructing measures using existing data were compared and results used to inform the design of a new method for use in the upcoming National Survey of Children's Health. Data from CAHPS-CCC2.0H are used to illustrate measurement options and variations in the overall presence of medical homes for children across managed health care plans as well as to evaluate in which areas of the AAP definition of medical home improvements may be most needed for all CSHCN.
Results. Existing surveys vary in their coverage of concepts included in the AAP definition of medical home and, therefore, in their capacity to evaluate medical home for children with and without special health care needs. Using data from CAHPS-CCC2.0H, the overall proportion of children who were enrolled in managed care health plans and met criteria for having a medical home varied from 43.9% to 74% depending on the specific scoring method selected for these items. Wide variations across health plans were observed and were most prominent in the areas of "accessible care" and "comprehensive care." Performance was uniformly poorest in the area of "coordinated care" and for CSHCN. Although children with a personal doctor or nurse were more likely to meet the AAP criteria for having a medical home, simply having a personal doctor or nurse was not highly predictive of whether a child experienced the other core qualities of a medical home (positive predictive value: .50; negative predictive value: .59).
Conclusions. Despite differences across existing surveys and gaps in concepts represented, we believe that the AAP definition of medical home can be well represented by the small subset of concepts represented in the National Survey of Children With Special Health Care Needs and the CAHPS-CCC2.0H. A less comprehensive yet still worthwhile measure is possible using the Medical Expenditures Panel Survey. The varying degrees of empirical evidence and consensus for each of the AAP definition domains for medical home suggest the need for constructing measures that also vary in terms of criteria for determining that a child does or does not have a medical home. In addition to a simple "yes or no," or rate-based, measure, a continuous medical "homeness" score that places a child or group of children on a continuum of medical "homeness" is also valuable. Findings indicate that health plans have an important role to play in ensuring medical homes for children in addition to medical practices and those who set policies that guide the design and delivery of health care for children. Overall, using existing population-based data, a measure of medical home that is aligned with the AAP definition is feasible to include in the annual National Healthcare Quality Report, in state reports on the quality of Medicaid, State Children's Health Insurance Program, and Title V programs as well as to evaluate performance on the Healthy People 2010 objectives and the President's New Freedom Initiative.
Key Words: medical home children health care quality children with special health care needs
Abbreviations: AAP, American Academy of Pediatrics National Survey of CSHCN, National Survey of Children with Special Health Care Needs National MEPS, National Medical Expenditures Panel Survey CAHPS-CCC2.0H, Consumer Assessment of Health Plans SurveyChildren With Chronic Conditions CSHCN, children with special health care needs NSCH, National Survey of Children's Health PDN, personal doctor or nurse
National health goals include ensuring that children have a medical home.1 Historically, medical home has been determined by the presence of a usual or primary source of care, such as a pediatrician or a family physician.2,3 More recent definitions expand on this simplistic notion of medical home.4,5 Specifically, in 2002, the American Academy of Pediatrics (AAP) issued a policy statement providing a detailed definition of medical home that identifies 7 dimensions and 37 discrete concepts for determining the presence of a medical home for a child. The AAP definition outlines an ideal model of care for children whereby health care is delivered or directed by well-trained physicians who provide primary preventive, acute, and chronic condition care and that this care is 1) accessible, 2) continuous, 3) comprehensive, 4) family-centered, 5) compassionate, 6) culturally effective, and 7) coordinated with specialized services provided outside the primary care setting. Agreed-on methods to measure medical home quantitatively are needed to assess and guide improvements in care for children in this area. Ideally, a common measurement approach will be used at medical practice levels and the national, state, and health care plan levels. This consistency will provide a basis for comparing and tracking performance over time and across each level of the health care system that plays a role in advancing the AAP concept of medical home for children.
Although consensus has grown regarding a definition of medical home, several challenges to standardized measurement remain, including 1) incomplete empirical evidence and consensus regarding the absolute and relative costs and benefits of discrete components of the AAP definition of medical home; 2) lack of availability of comparable sources of data for measuring certain aspects of medical home in a consistent way across children and settings of care; 3) ongoing debate and uncertainty regarding how to measure specific concepts included in the AAP definition of medical home, such as "ongoing source of care," continuity, and coordination of care; 4) selecting which of the numerous concepts of medical home send the strongest signal and can be most feasibly and validly measured when data collection methods, such as parent surveys, must be made as parsimonious as possible; and 5) designing a specific measure construction, scoring, and grading method that sets high and realistic expectations for the implementation of the medical home model of care and maximizes the value of measures for stimulating and informing improvements.
The AAP definition of medical home greatly advances the operationalization of this important model of care for children. However, it does not address the measurement issues outlined previously and leaves a great deal of room for interpretation when translating this definition to a precise measurement method. The most historically central concept of medical home, that of having a usual source of care, provides a good illustration of this issue. The AAP definition states that care should be "delivered or directed" by "physicians who provide primary care." This definition leaves open several issues that must be resolved for purposes of measurement. For example, decisions must be made regarding 1) whether any or only certain types of physicians qualify as providing primary care and whether a child should have only 1 physician who provides this care; 2) which pediatric clinician to which to anchor a parent and/or a child's assessment of family-centeredness and other concepts of medical home in cases in which the pediatric clinician who delivers care is different from the person who directs the care or when a child has >1 pediatric clinician who delivers and/or directs care; and 3) accounting for cases in which parents consider nonphysician clinicians to be a child's primary source of care (eg, nurse practitioners, physician assistants). Similar challenges emerge when translating other medical home concepts into a precise and parsimonious measurement method.
As work progresses to specify the measurement methods needed to measure quantitatively the presence of medical homes for children as defined by the AAP, we are fortunate that several population-based data sets that include variables relevant to many components of the AAP definition of medical home already exist: 1) the National Survey of Children with Special Health Care Needs (National Survey of CSHCN),6 2) the Medical Expenditures Panel Survey (MEPS),7 3) the Consumer Assessment of Health Plans Child Survey,8 and 4) the Consumer Assessment of Health Plans Survey Children With Chronic Conditions (CAHPS-CCC2.0H).9 These existing surveys make measurement of medical home at the national, state, and health care plan levels possible today, even as improved methods are developed and implemented over time.10
The objective of this study was to identify methods to measure the presence of medical homes for all children and for children with special health care needs (CSHCN) using existing population-based survey data sets. Measurement options were compared, and the use of the CAHPS-CCC2.0H to compare performance across health plans is explored. Findings from this study were used to inform the development of a parsimonious set of parent survey items to measure medical home in the 20032004 National Survey of Children's Health (NSCH), which will provide ~2000 cases of child data per state.
| METHODS |
|---|
|
|
|---|
Next, alternative measurement methods were implemented and compared using existing data. Because National Survey of CSHCN and 2000 MEPS data were not yet publicly available at the time of this analysis, results presented here are based on the CAHPS-CCC2.0H survey. Although these data are not nationally representative, they can be used to illustrate methods for measuring medical home that can be applied to these other surveys. In addition, CAHPS-CCC2.0H data allow us to examine the use of measures of medical home to compare performance across health plans, which is not possible using National Survey of CSHCN or MEPS data.
The CAHPS-CCC2.0H data collected for children in 1 state Medicaid program were used and represent 10 792 children who were aged 12 years or younger and enrolled in 1 of 19 managed care health plans. Children in the state for which data were collected are eligible for Medicaid under the Temporary Aid to Needy Families program if family income is <150% of the federal poverty level or is 185% of the federal poverty level if the child is younger than 1 year.
Findings from the 4 steps outlined above were used to guide the construction of a 13-item parent survey to measure the presence of medical homes for children with and without special health care needs for use in the upcoming NSCH (target n = 100 000). First, 49 items were specified representing each of the 7 AAP medical home definition domains and alternative ways to capture information on these domains. These items were fielded in the 2003 pretest for the NSCH, and psychometric, item-reduction, and validity analyses were conducted to arrive at the short medical home survey. The items to be used in NSCH are available by contacting the primary author of this article. Future articles will report on the analyses that led to their development, as well as findings on their use to construct a measure of medical home when data from the NSCH are available.
Construction of Measures of Medical Home Using the CAHPS-CCC2.0H
Alternative scoring methods for measuring the AAP concept of medical home were constructed using data from CAHPS-CCC2.0H. Primarily, the alternative methods vary in terms of the type of metric that is produced (eg, a percentage of children with a medical home vs a medical "homeness" 0100 continuous score) and the level of performance expected before a child is determined to have a medical home (eg, on average, the child experiences good care across all domains of care vs the child experiences good care on every domain of care).
The CAHPS-CCC2.0H was administered using the National Committee on Quality Assurance Health Plan Employer Data and Information Set data collection specifications.11 The CAHPS-CCC2.0H data were collected across managed care health plans in a statewide Medicaid program. A total of 19 managed health care plans were sampled. Five health care plans were eliminated from this analysis because of small sample sizes and/or the inadvertent use of an administration protocol that differed from the rest of the plans. NCS Pearson administered the survey on behalf of the state's Medicaid agency, and the sample was drawn by Abt Associates.
The sampling frame for CAHPS-CCC2.0H included all child Medicaid clients who were 12 years of age or younger and were continuously enrolled in a managed care health plan for at least 6 months as of January 2002. A break in enrollment of up to 45 days was allowed. Two samples were drawn for each health plan: 1) a population-based sample of 1275 children and 2) an oversample of children whose patient encounter utilization data indicated a higher probability of having a special health care need. This latter sample was drawn using a prescreening algorithm specified in the Health Plan Employer Data and Information Set guidelines. The target sample size for the oversample population was 1650, but only 5 health care plans had sufficient enrollment to reach that goal after the population-based sample had been drawn. The remaining health care plan oversample sizes ranged from 265 to 1390. Data were combined from both samples for each plan. All results presented here are adjusted to the mean proportion of children identified as having a special health care need across the 14 health care plans. This proportion was based on responses to the CSHCN Screener, which is included within CAHPS-CCC2.0H.12,13
The survey was administered by mail with a telephone follow-up for those who did not respond to the mailed survey. A maximum of 2 mailings and 3 telephone calls were made for each child in the starting sample, with 3 weeks between mailing and telephoning. The survey was administered in English only.
The 34 CAHPS-CCC2.0H survey items relevant to the AAP definition of medical home were used to calculate alternative measures of medical home. Table 1 summarizes which CAHPS-CCC2.0H were placed into each of the AAP definitional domains and how survey items with different response options were aligned for purposes of measurement construction. As can be seen, no items were available for the continuity of care domain, and the "family-centered" and "compassionate care" domains were combined.
|
Finally, whether a child had 1 person whom the parent considered to be the child's personal doctor or nurse (PDN) was layered on top of each of these 2 rate-based versions of medical home described. In this way, each of the 2 rate-based measures was calculated for children who did and did not have 1 person whom the parent considered to be the child's PDN. This was done to examine the impact of this criterion and to evaluate the association between "having a usual source of care" and the more expanded AAP definition of medical home.
Analytic Methods
Conclusions derived using the alternative measurement construction methods described here were compared across 14 health care plans as well as for children with and without special health care needs. Differences in the percentage of children with a medical home using alternative methods were evaluated using odds ratios as well as
2 tests of significance. A minimum P = .05 was set. Spearman rank correlations were calculated to evaluate the stability in the ranking of health care plan medical home scores and rates when alternative measurement methods were used. Predictive values were calculated to examine the degree to which having 1 person who is a child's PDN predicts whether that child also meets the more expanded AAP criteria for having a medical home as operationalized here using the CAHPS-CCC2.0H survey.
| RESULTS |
|---|
|
|
|---|
The existing national, state, and health care plan levels survey data sets reviewed here varied widely in their coverage of the concepts included in the AAP definition of medical home and according to whether a measure of medical home could be stratified for children with and without special health care needs. A range of 11 to 28 relevant items were present across each of the 4 surveys reviewed, representing all but the continuity of care domain. The CAHPS-CCC and National Survey of CSHCN provide the most robust measurement approach (Table 2).
|
Using the survey items and response coding illustrated in Table 1, the mean medical "homeness" score for CSHCN was 80.9 and for children without special health care needs was 82.0. As shown in Table 3, the proportion of children with a medical "homeness" score of 75 points or above varies from 43.9% to 74.0%, depending on whether the "across all" scoring model or the "on each" scoring model is used and whether having 1 person who is the child's PDN is included as a criterion (Table 3). Scores for CSHCN are lower for each method and vary from 38% to 71.2%.
|
|
|
|
|
| DISCUSSION |
|---|
|
|
|---|
Despite differences across existing surveys and gaps in concepts represented, the AAP definition of medical home can be well represented by the small subset of concepts represented in the National Survey of CSHCN and CAHPS-CCC2.0H. A less comprehensive yet still worthwhile measure is possible using the National MEPS. This is largely because the National MEPS includes core family-centered care items that were shown here to be highly correlated with a more comprehensive measure of medical home. In addition, the MEPS allows findings to be stratified for children with and without special health care needs using the CSHCN Screener.11,12
To advance medical home as a serious model of care for children, it will be important to track and compare the overall presence of medical homes for children as well as identify in which domains of the AAP definition of medical home improvements are most needed and for which subgroups of children. Existing survey data sets can be used to begin this process. However, over time, it will be ideal if future versions of the National Survey of CSHCN, CAHPS-CCC, and MEPS and other similar surveys are aligned in the way medical home definition domains are assessed. The survey items identified for inclusion in the NSCH may provide a useful starting point for this purpose.
In this study, considerable variation in performance across health care plans in the percentage of children who meet the overall criteria for medical home was observed. We found that health plans vary in terms of the domains in which their performance was higher versus lower. These findings indicate that some systems of care may do a better job than others in certain areas and that many lessons for improvement may be learned by further examining where performance is higher or lower across plans as well as other units of analysis, such as medical practices. These findings also indicate that variations are likely to be attributable not only to medical practices and that health plans have an important role to play in ensuring medical homes for children. Few health plans scored well in the area of coordination of care, confirming known deficiencies in the capacity and performance of the health care system in this area. Lower overall and domain-specific medical home scores for CSHCN support the focus of the Healthy People 2010 medical home objective on CSHCN.1
There are varying degrees of empirical evidence and consensus to support each of the AAP domains for medical home as recommended guidelines for care. As such, we recognize the need for constructing measures that also vary in terms of how stringently medical home is measured. Results from the "across all" and "on each" scoring models illustrated here do send a different signal regarding the proportion of children with a medical home. However, findings from this study suggest that these alternative methods do not alter conclusions regarding the relative performance across health care plans, between groups of children, or in terms of identifying in which domains of the AAP definition of medical home improvements in performance are most needed. In addition to the use of the simple "yes or no" rate-based measures constructed here, we recommend the use of a medical "homeness," 0 to 100 continuous score in cases in which it is important to place any given child or group of children on a continuum of medical "homeness" or understand the distribution of medical "homeness" in any given population or unit of analysis.
Potential users of the medical home measure construction methods outlined here need to determine their comfort in requiring that a child experience, on average, the qualities of a medical home across all domains versus requiring that a child experience the qualities of a medical home on each domain. The latter measure would disqualify a child for not meeting criteria on 1 or more domains, whereas the former measure allows for some deficiencies in care as long as overall care, on average, usually or always meets criteria for having a medical home. Regardless of the perspective of potential users on the methods outlined here, it should be noted that seemingly small variations in the way the same survey items are used to construct a measure of medical home can lead to very different scores for an individual health care plan or group of children and must be considered carefully.
Findings verify that having a usual source of care, as measured in CAHPS-CCC2.0H, is helpful, but in no way is this medical home criterion sufficient for predicting whether a child experiences the qualities of a medical home as outlined by the AAP. Conversely, many children usually or always experience many of the qualities of a medical home even when they do not have a usual source of care as defined in CAHPS-CCC2.0H. Given these findings, the expanded definition of medical home set forth by the AAP is more likely to lead to the qualities of care represented in this definitionqualities that many may have historically assumed would be present simply by having a usual source of care.
Overall, although improvement and modifications to future versions of existing surveys are recommended, findings from this study suggest that existing data can be used to evaluate the presence of medical homes for children. It is feasible to include a measure of medical home for all children and for CSHCN in the annual National Healthcare Quality Report,13 in state reports on the quality of Medicaid, State Children's Health Insurance Program, and Title V programs as well as to evaluate performance on the Maternal and Child Health Bureau's 2010 Core Outcomes for CSHCN.14
| ACKNOWLEDGMENTS |
|---|
For more information about the Child and Adolescent Health Measurement Initiatives medical home measurement work, go to www.cahmi.org/cahmi.
| FOOTNOTES |
|---|
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
R. E. Benedict Quality Medical Homes: Meeting Children's Needs for Therapeutic and Supportive Services Pediatrics, January 1, 2008; 121(1): e127 - e134. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. Simpson, D. Dougherty, D. Krause, C. M. Ku, and J. M. Perrin Measuring Children's Health Care Quality American Journal of Medical Quality, March 1, 2007; 22(2): 80 - 84. [PDF] |
||||
![]() |
D. S. Mandell, M. M. Novak, and C. D. Zubritsky Factors Associated With Age of Diagnosis Among Children With Autism Spectrum Disorders Pediatrics, December 1, 2005; 116(6): 1480 - 1486. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||