ARTICLE |
a Department of Pediatrics, Franklin Square Hospital Center, Baltimore, Maryland
b Department of Pediatrics, Albert Einstein College of Medicine/Children's Hospital at Montefiore, New York, New York
c Department of Pediatrics, University of South Florida, Tampa, Florida
d Department of Pediatrics, Northwestern University, Chicago, Illinois
e Department of Epidemiology and Preventive Medicine, University of Maryland, Baltimore, Maryland
f Center for Health Care Quality, Cincinnati Children's Hospital and Medical Center, Cincinnati, Ohio
g Department of Pediatrics, Johns Hopkins University School of Medicine, Baltimore, Maryland
| ABSTRACT |
|---|
|
|
|---|
METHODS. A cross-sectional survey using the Parents Perception of Primary Care was conducted of patients enrolled at 19 national academically affiliated resident continuity practices from the Continuity Research Network. Outcome measures included mean total scale score for the Parents' Perception of Primary Care and mean scores for each primary care domain. Comparisons were made between the subset of resident patients who were older than 5 years and a previously published community survey of parents of school-age children.
RESULTS. A total of 2572 patients were enrolled with a final sample size of 2211 analyzable surveys. The sample was 37% black and 40% Hispanic; 81% of the children had Medicaid insurance; and 20% of the parents had less than a high school education. Parents rated the care that they received in resident continuity with high total scores and subscale scores, with an overall mean total scale score of 74.0. Higher scores were associated with number of visits to the provider and being able to name the resident as the primary care provider, whereas minority status was associated with lower access and communication scores. The resident sample over age 5 had higher mean scores for the total scale and every domain as compared with the community sample.
CONCLUSIONS. Parents of patients at resident continuity sites rated residents as providers of high-quality care to a socioeconomically disadvantaged population as compared with a previously published community sample. Efforts to improve resident continuity and identification may help improve care delivered in resident practices.
Key Words: resident continuity practice CORNET primary care
Abbreviations: IOM—Institute of Medicine P3C—Parents' Perception of Primary Care ACGME—Accreditation Council on Graduate Medical Education CORNET—Continuity Research Network
Providing high-quality primary care is 1 key to improving the health and well-being of children1 and has been a priority of the Institute of Medicine (IOM) since the launching of the Health Care Quality Initiative in 1996.2 Studies have demonstrated that a strong primary care infrastructure is associated with better health outcomes for children.3 The key components of primary care as defined by IOM reports as well as the American Academy of Pediatrics medical home construct include longitudinal continuity, access, contextual knowledge, comprehensiveness, communication, coordination, community orientation, and cultural competence.4,5 In addition, the IOM report emphasized the importance of the development and adoption of uniform methods and measures to monitor the performance of health, including cost, quality, patient access, and both patient and clinician satisfaction.5
The ability to measure the quality of children's health care effectively is challenging because of a number of factors, including the child's dependence on caregivers, changing development, and demographics.6,7 Although some proxy measures for quality that do not depend on parents exist, such as immunization rates and frequency of screening for lead and anemia, these measures may reflect the quality of systems rather than providers. Only recently have researchers developed instruments to assess the IOM key components of primary care attributes, which include the multifaceted interactions between primary care providers and patients.8–10 The two main surveys developed for use in pediatric populations are the Primary Care Assessment Tool8 and the Parents Perception of Primary Care (P3C).9
Measuring the quality of primary care that patients receive in pediatric resident continuity clinics is important for a variety of reasons. First, hospital-based primary care clinics, where residents are often the primary care providers, provide care to
21% of the socioeconomically disadvantaged families in the United States as well as to children with chronic health care needs.11 In addition, given the structural challenges of resident training, including limited work hours, limited weekly availability, and automatic transitions at the completion of 3 years of training, there may be concerns about a lower quality of care received by patients. Yet, resident education about general pediatric principles, continuity, the doctor–patient relationship, management of chronic diseases, assessing and counseling on family psychosocial issues, and other features of primary care are taught during the resident's continuity clinic experience. The use of an index tool for quality can provide resident-specific data that are helpful for residency programs to satisfy the Accreditation Council on Graduate Medical Education (ACGME) requirements for competency-based resident evaluation. Providing quality primary care addresses many of the core competency areas, including patient care, practice-based learning, systems-based practice, and interpersonal and communication skills. Because it has been shown that practitioners tend to practice in the manner in which they were trained during residency, the resident continuity experience embodies a critical element in the development of future pediatricians.12,13 Residents also believe that their continuity experience embodies a critical element in their development as future pediatricians.14 Although there are many studies of pediatric resident continuity experiences,11,15–17 none of these studies has assessed the quality of resident care from the family's perspective or evaluated how the continuity practices meet the American Academy of Pediatrics medical home recommendation.4
The purpose of this study was to assess parents' perception of resident primary care using a version of the P3C to determine the characteristics that are associated with better care in resident practices and to compare perception with a previously published community sample of elementary students
| METHODS |
|---|
|
|
|---|
|
Parents were informed of the purpose of the survey and then asked to identify which doctor in the practice was their primary care physician. A list of the residents in the practice was provided at some sites to assist parents to remember the name of their child's doctor. Patient–parent dyads were not excluded when they could not remember their doctor's name, because each continuity site was able to determine by computerized records at registration whether the assigned provider was a resident and whether the family had seen that provider before. Verbal consent was obtained from each parent. Institutional review board approval was obtained from each academic institution of the participating continuity practice.
Tool
The P3C survey tool, developed by 1 of the co-investigators (Dr Seid), was used. This tool yields a total score and subscale scores for 6 domains of primary care, including access, longitudinal continuity, comprehensiveness, coordination, communication, and contextual knowledge. The P3C had been field-tested in a large community sample of parents of 3371 elementary school–aged children in San Diego, CA.9 The children in this sample were selected from a nonrandom sample of 228 classes in 18 elementary schools in San Diego. Parents completed the survey in a self-administered manner. The survey instrument has demonstrated high feasibility, strong internal reliability, and construct validity.9 Additional construct validity with this initial study group has been published demonstrating that language and access to care are independently associated with higher P3C scores.18
The P3C was selected over other measures of primary care because of its short, 1-page length and because it was designed to be self-administered. The 23-question P3C was minimally revised to develop a 23-question "resident version" that was used in this study. Two access questions were removed, and 2 questions were added, including 1 about community awareness and 1 about whether the parent would recommend the provider to family members. These additional questions are similar to questions used on Starfield's Primary Care Assessment Tool.8 The revised tool was pilot tested in the continuity site of the principal investigator and the data from the pilot test was not included in this study. The main survey is shown in the Appendix except for the question on longitudinal continuity, which is in "Demographic Variables." The response scale for the instrument used a 5-point Likert scale, with the options for most questions as follows: 0 (never), 1 (sometimes), 2 (often), 3 (almost always), and 4 (always). For the 5 questions of the coordination scale, an additional response choice of NA (not applicable) was included.
Demographic Variables
All demographic questions had categorical response options. Questions assessed the child's age, gender, race/ethnicity, and type of insurance. The parent/caregiver who brought the child to the visit during the study period completed the survey and identified himself or herself as the child's mother, father, grandparent, or other (hereafter referred to as "parents" for simplification) and noted his or her educational achievement (categorized as less than high school, high school through some college, and college degree or more). The parent also categorized the child's health status on a scale ranging from poor to excellent and was asked whether the child had a chronic health condition. Parents were also asked to identify the name of their child's primary care physician, the longevity of the relationship, and the number of times the child had seen this provider for care.
The variables were dichotomized for the multivariable regression models as follows: "good health" as reported excellent, very good, or good compared with fair or poor health as the reference category; "completed high school" as high school degree or higher as compared with respondent with less than high school education as the reference category; "named resident provider" when respondent documented resident's name as compared with respondent unable to name individual resident as reference category; "Saw MD >3 times" as parents who reported 4 or more visits to the provider as compared with seeing the provider <4 times as the reference category; "Age >5 years" compared with age of the child
5 years as the reference category; and "African American," "Latino" and "other ethnicity" each compared with white race as the reference category.
Data Management
Each study site collected the surveys, removed patient identifiers, and mailed surveys to a central site for coding and entry into an Access database. Survey data were double entered into the database by site number, and data entry errors were corrected. The primary investigator had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis
Comparative Analyses
Stata 9.1 (Stata Corp, College Station, TX) was used to analyze the data. A total mean score as well as primary care domain-specific scores were calculated from the survey results and compared with the results from Seid's community sample. Surveys with fewer than 13 of the 23 P3C scale questions completed were considered incomplete and excluded from the analysis to maintain the test properties of the scale. For the complete surveys, domain and total scores were calculated for the entire sample using Seid's previously described conversion of the Likert scale results. For ease of interpretability, the scale items were transformed to a scale from 0 to 100, with 100 being best, as follows: 0 = 0, 1 = 25, 2 = 50, 3 = 75, and 4 = 100. For example, with this transformation, a score of 75 means that the care provided is "almost always" consistent with the primary care attribute. Longitudinal continuity scores were converted from the Likert scale as follows: <6 months = 0, between 6 months and 1 year = 25, between 1 and 2 years = 50, between 2 and 3 years = 75, and >3 years = 100. Computing the mean of the nonmissing values on each scale formed the total scale score, as well as the scores for each subscale.
Test characteristics for the resident scale were calculated, including percentage of missing data and internal consistency alphas for the total scale and each domain subset. Demographic data were described using tabular frequencies of categorical data. Total scale and domain scores were calculated for the entire sample (all CORNET) from the resident clinics and the subset of patients who were older than years (CORNET >5) to match better the ages for comparison with the San Diego community sample. Demographic predictors of higher scores in the entire resident sample were evaluated first by univariate comparison using mean total score and mean score for each domain as outcomes. Multiple variable linear regression models were developed using mean score as a continuous outcome. We adjusted for the following demographic variables: health status of the child, parental education, identified primary care resident, number of visits to the clinic, age, and race. All of these variables were included in the model to control for each other because each fits into the theoretical framework of factors that have an impact on quality of care. The unstandardized coefficients that resulted from the models represent the association of each variable to the outcome in each model, adjusted for all other variables in the model. In addition, to account for clustering effects of multiple sites, the "site" variable was analyzed as a random effect in all models.
Comparison was made between CORNET >5 with the originally published community sample from San Diego (referred to as San Diego subsequently), because the original community study was completed using an elementary school population. We restricted the San Diego sample for this analysis to parents who identified a person or place as a primary care provider for their child (2570 of the original 3371), because the CORNET sample had an identified place or provider. Mean scores for the full scale, as well as each domain, were compared between the 2 populations using t tests. Adjusted mean scores and 95% confidence intervals were then obtained using multiple variable linear regression models and adjusting for the common demographic variables in the 2 data sets: race, insurance status, education, and language of administration.
| RESULTS |
|---|
|
|
|---|
|
|
|
|
Comparison with San Diego
CORNET >5 patients rated residents higher than the San Diego sample in the overall score as well as in each domain. Figure 2 demonstrates the unadjusted comparison between the mean scores for all 3 groups. After adjusting for comparable demographic variables, the mean scores of CORNET >5 for the total scale and every domain except access remained significantly higher than the community sample as shown in Fig 4.
|
| DISCUSSION |
|---|
|
|
|---|
The finding that the quality of care that is delivered by pediatric residents is high along multiple dimensions is reassuring. Although the ACGME has mandated the continuity experience, no one has measured the quality. Despite concerns about the availability of pediatric house officers to their ambulatory patients given the exigencies of scheduling multiple service obligations during residency training, parents in our sample reported that their access to care at the resident site was high. They also reported that resident providers' demonstrated knowledge about their children's conditions, the comprehensiveness of what was discussed during routine health care maintenance visits, and the residents' ability to coordinate care in a longitudinal manner all were highly rated. Most gratifying for those charged with training housestaff was the finding that parents also reported high ratings for residents' communication skills.
The demographic variables that were associated with higher and lower scores provide insight into the care that resident continuity practices provide. Similar to previous publications that demonstrated health care disparities,7,15 this study showed ethnic disparities in perception of care. Compared with white patients, black patients and patients of other ethnicities in this study reported lower total scale scores, and all ethnicities reported lower scores in the access and communication domains. Although the scores were lower, the overall rating of care being "almost always" consistent with the primary care definition was not different between groups. Children with reported good health overall demonstrated lower scores in every domain except longitudinal continuity. Despite controlling for the number of visits, this perception may still be related to how often the patient sees the provider. Because healthy children need to be seen less frequently, whereas those with chronic illness are more likely to see their provider an increased number of times, parents may have more opportunities to evaluate the care and thus rate with higher scores.
The effect of older age on primary care ratings is interesting. Because of the direct relationship between age and longitudinal continuity, defined as length of time seen in the clinic, these scores were significantly higher than those for younger children. It should be noted that because residency is 3 years long, a score of 75 in the longitudinal continuity outcome is the highest achievable for the resident sample. The scores for contextual knowledge and comprehensive care were lower than those for younger children. The likely explanation for this seeming contradiction is that although the older children have most likely been seen for a longer time interval, a single resident has provided their care for at most a 3-year period and the older children are seen with much less frequency than younger infants. Increased frequency of visiting with the same provider, such as happens during infancy, may allow parents to have a better impression of the full range of services the residents can provide. Two outcomes were associated with higher total and domain scores, the parent's ability to name a resident provider, and greater number of visits. More visits, similar to length of time, allows parents more of an opportunity to assess the care that they receive, and higher scores for children whose parents could identify a resident illustrates the importance of provider continuity.
Ongoing assessment of quality of care constitutes a distinguishing feature of modern, well-functioning health care systems. This study demonstrates the feasibility of large-scale quality assessments that can and should be conducted in settings where ambulatory pediatric care is routinely provided by pediatricians in training. This type of assessment addresses the ACGME resident competency assessments such as systems-based improvement. In addition, the test characteristics of the P3C demonstrated in this study show that it is possible to obtain a broad spectrum of information related to different elements of quality from a minority population of families from urban environments throughout the United States.
Several limitations exist in this study. Despite representing clinics in a broad geographic region, this study represents a national convenience sample and we cannot determine the extent to which our findings are generalizable across all pediatric resident continuity practices in the United States. Although the response rate at each institution was relatively high, we have no information on the quality of care that was experienced by nonresponders, making it impossible to determine whether a response bias exists within the sample. The survey collection period was restricted to a 2-week period, which may not be representative of the entire population of each site or of care delivered at all times of year (may be worse in busy winter months). In addition, because the survey was administered at the site where parents bring their children for care, 2 additional sources of bias may be introduced relative to the information gathered from a community-based sample. In 1 important sense, the parents in this sample may be a self-selected group because they have demonstrated a preference for these health care environments and are more likely to believe that the quality of care that they are receiving is superior relative to parents who are surveyed at random. Even if these parents are not different from another group surveyed at random, there exists the possibility that because they were surveyed at the continuity clinic itself, the parent respondents might feel expected to rate the quality of care received as higher than had these same parents been surveyed in another location.
Although the San Diego sample is the only available comparison, the 2 samples were obtained with different methods and had significantly different demographics; therefore, the conclusions drawn by direct comparison between the 2 samples may be limited. Even if the differences detected are not as large as found in this study, the San Diego sample does represent the first and only available community benchmark and the mean scores shown by the resident sample are at least as high as 1 community. It is also unknown whether parents in California are a nationally representative sample. A final limitation is that the survey assesses parental perception of quality, without being able to measure "clinical" or "process" quality outcomes.
| CONCLUSIONS |
|---|
|
|
|---|
|
| ACKNOWLEDGMENTS |
|---|
The following are CORNET Network Participants: East Carolina University, John Olsson, MD; Indiana University, Phillip Siefken, MD; Johns Hopkins Children's Center, Janet Serwint, MD; Loma Linda University, Sharon Reisen, MD; MedPro/Maricopa Phoenix, Shawn McMahon, MD; Montifiore, Andrew Racine, MD; Nebraska University, Sheryl Pitner, MD; Northwestern University, Sandy Sanguino, MD; Oregon Health Sciences University, Cynthia Ferrell, MD, MSEd; University of Rochester/Rochester General Hospital, Lynn C. Garfunkel, MD; Rush University, Bill Stratbucker, MD; St Joseph's, Phoenix, Lilia Parra-Roide, MD; Uniformed Services, Joe Lopreiato, MD, MPH; University of Arizona, Karen Davenport, MD; University of California Davis, Richard Pan, MD, MPH; University of California Irvine, Lynn Hunt, MD; University of Florida, Allan Friedman, MD; University of South Florida, Sharon Dabrow, MD; University of Utah, Sarah Croskell, MD, MPH; Medical students: Aynslee Wells and Raequi Wilson (data entry).
| FOOTNOTES |
|---|
Address correspondence to Scott D. Krugman, MD, Department of Pediatrics, Franklin Square Hospital Center, 9000 Franklin Square Dr, Baltimore, MD 21237. E-mail: scott.krugman{at}medstar.net
The authors have indicated they have no financial relationships relevant to this article to disclose.
| REFERENCES |
|---|
|
|
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||