Objective. A measure of pediatric primary care quality that is brief, practical, reliable, and valid would be useful to patients and pediatricians, policymakers, and health system leaders. Parents have a unique perspective from which to report their experiences with their child's primary care, and these reports may be valid indicators of pediatric primary care quality. The research objective was to develop a brief parent report of their children's primary care, the Parent's Perceptions of Primary Care measure (P3C), and to test its reliability and validity as a measure of pediatric primary care quality.
Study Design. The P3C was based on the elements of primary care as defined by the Institute of Medicine. Pretesting of domain content and item clarity was accomplished via focus interviews. The P3C was developed in English and translated to Spanish, Vietnamese, and Tagalog. The 23-item P3C yields a total score, as well as subscale scores for continuity, access, contextual knowledge, communication, comprehensiveness, and coordination. The P3C was administered to 3371 parents of children in kindergarten through sixth grades in a large, urban school district.
Principal Findings. The percentage of missing values for the overall sample was 1.88%, indicating acceptable feasibility. Range of measurement, assessed via floor and ceiling effects, was moderate to good. Cronbach's coefficient α, an indicator of scale internal consistency reliability, was 0.95 for the P3C total scale. Factor analysis supported the subscale structure, and P3C scores were higher for children with health insurance, whose parents completed the survey in English, and who had a regular physician. P3C scores were positively related to parent reports of the child's health-related quality of life.
Conclusions. The P3C is a practical, reliable, and valid measure of parents' reports of pediatric primary care quality. This brief measure could be used alone, or in conjunction with other measures, to enhance outcomes and evaluate the impact of systems changes on the delivery of the main elements of primary care.
- quality of healthcare
- outcome and process assessment (healthcare)
- primary healthcare
High quality pediatric primary care is a cornerstone of efforts to improve health outcomes, control health care spending, and improve access to care.1–3 To improve the quality of pediatric primary care, a reliable and valid measure must exist. The ideal instrument is one that is brief, practical, reliable, and valid.4,,5 Given the current high rate of uninsured children6 and children without a regular source of care,7,,8 such an instrument must also be applicable regardless of insurance status and health plan membership or the presence of a regular provider of care.
Primary care, according to the Institute of Medicine (IOM), is “the provision of integrated, accessible health care services by clinicians who are accountable for addressing a large majority of personal health care needs, developing a sustained partnership with patients, and practicing in the context of family and community.”9Operational definitions of these concepts vary across researchers. Nevertheless, there is general agreement that primary care is accessible, longitudinally continuous, adequately communicated, contextual (based on a provider's accumulated knowledge of the patient and family), comprehensive, and coordinated.
Parents are in a unique position to report on the care their children receive.10–13 Indeed, some aspects of primary care (for example, accessibility, adequate communication, and contextual care) describe the parent's/patient's experience of care, rather than a specific provider behavior. Other aspects of care can be reliably reported by parents (for example longitudinal continuity, comprehensiveness). Although some have argued that patients/parents cannot report on the coordination of care,14 it could be argued that the best coordination is apparent to the patient or parent—that part of coordinating care involves communicating the process and outcomes of that coordination to the parent.
A distinction has been made between patient/parent reports of experiences with the health care delivery system and ratings of satisfaction with health care delivery.15,,16 Although both reports of experiences and ratings of satisfaction require the respondent to make evaluative responses, these 2 types of evaluations differ in the criterion against which the evaluation is made. Satisfaction ratings are evaluated against an individual's expectations and preferences, which vary widely across individuals and ultimately do not suggest ways in which the health system can be improved.16 On the other hand, reports of experiences are evaluated against a specific prescriptive criterion (for example, that care be “adequately communicated”14). As such, deviations from the criterion represent potential decrements in quality and areas for improvement.
Several measures of primary care currently exist. Starfield and Cassady's Child Health Systems Primary Care Assessment Survey16 is an interviewer-administered survey consisting of 2 parallel surveys for consumers and providers of care that assess both structural and process indicators of primary care for children. It focuses on consumers' experiences with health delivery characteristics rather than satisfaction with them, and has been shown to be a reliable and valid measure. A somewhat shorter version, the Primary Care Assessment Tool–Child Edition has been described.17 This version also requires interviewer-administration and takes approximately 25 minutes to complete. This subject burden may have contributed to the 36% response rate (27% response rate for the test-retest portion) reported. Safran's Primary Care Assessment Survey18 is a 51-item, self-administered instrument. It measures 7 domains of primary care for adults within the context of a specific clinician-patient primary care relationship. Only those respondents who report an established relationship with a primary clinician can complete the measure. The Primary Care Assessment Survey includes both reports and evaluative ratings. It has been subjected to rigorous psychometric testing and has been linked to outcomes of care.19 Flocke's Components of Primary Care Index15 is a brief (20-item), self-administered measure of adults' reports of their experiences with 4 aspects of primary care. It requires patients to report, rather than rate, their interaction with a specific physician in the context of a particular office visit. All these instruments are based on the IOM's definition of primary care and are designed as multidimensional instruments, although each measures a slightly different overlapping set of constructs. Each has been subjected to various degrees of psychometric testing, and all seem to have at least some degree of reliability and validity.
However, a need exists for an instrument that is at once a brief, practical, reliable, and valid measure of pediatric primary care that can be used irrespective of insurance status or the presence of an established clinician-patient relationship. Although others have posited that an established clinician-patient relationship is necessary for the existence of primary care, and have therefore restricted measurement of primary care to those patients with such a relationship, the fact remains that many children, both insured and uninsured, lack a regular site of care or a regular provider of care. It behooves researchers, clinicians, and policy makers to develop and use measures that encompass the experiences of these children as well.
The present study describes such an instrument, the Parent's Perceptions of Primary Care measure (P3C). Using a large, diverse, community sample, we assessed the feasibility, reliability, and validity of the P3C. Feasibility is measured in terms of missing data and the percentage of respondents with the lowest possible (floor effect) and the highest possible (ceiling effect) score. Low rates of missing data indicate that parents are willing and able to complete the survey items. Small floor and ceiling effects indicate that the instrument allows parents to respond to an adequate range of experiences. Reliability is measured in terms of internal consistency. High internal consistency indicates that items grouped into a scale are measuring a similar construct.
Validity is demonstrated by an instrument's ability to respond as expected, given specific research-based hypotheses. First, we examined the validity of the P3C subscales via factor analysis. It is hypothesized that P3C items should be associated with the a priori subscales. Second, we examined the P3C's ability to distinguish between groups thought to differ in the quality of primary care received. In this case, it is hypothesized that higher quality primary care will be reported for parents whose children have health insurance,20,,21 who complete the form in English,22,,23 and whose children have a regular health care provider.24–26 Finally, we examined the relationship between the P3C and a measure of health-related quality of life, the Pediatric Quality of Life Inventory (PedsQL).27 It is hypothesized that higher scores on the P3C will be related to higher PedsQL scores.
The P3C is based on the IOM definition of primary care. Using this definition as a criterion, the P3C was designed to measure 6 components of care which, when present, constitute high quality primary care. High scores reflect care conforming to this a priori definition. Thus, the P3C measures quality based on parent reports of their experiences, rather than ratings of satisfaction with those experiences. The P3C was designed to measure the quality of primary care received, rather than the quality of a particular provider of primary care. This was done so that the care received by children without a regular provider could also be described in relation to the IOM definition of quality primary care.
The components of primary care included in the P3C are those on which parents are thought able to report. The 6 components of primary care are defined as follows. Longitudinal continuity is defined as the parent's report of the length of time they have been bringing their children to a regular place or physician.2,,14 Access is defined as the parent's report of timely and convenient access to care for their children.14 Communication is defined as the parent's report of how well the physician listens and explains during their interactions.15 Contextual knowledge is defined as the parent's report that the physician knows his or her values and preferences about medical care issues, clearly understands his or her child's health needs, and knows the child's medical history.2 Comprehensiveness is defined as the parent's report of the extent to which a regular place and/or doctor provides care for acute and chronic problems and preventive services.14,,15 Coordination of care is defined as the parent's report of their physician's knowledge of other visits and visits to specialists, as well as the follow-up of problems through subsequent visits or phone calls.2
Item Development and Pilot Testing
Items were developed from the existing literature,14–19 from parent interviews, and were created for this measure. A team of researchers collaborated on item development. The components of primary care were divided among the researchers (M.S., L.O.B., M.Z., M.D.F., and M.N.) so that each researcher was responsible for developing questions for particular components. Multiple items were generated for each scale. Items in each scale were reviewed and discussed by the other team members to ensure appropriateness. Questions were rewritten or omitted if they were confusing, lacked clarity, or too closely resembled other items. The goal was to create parsimonious scales consisting of items that would be understood by the majority of parents in a diverse community population. This instrument was specifically developed to be appropriate for self-administration in a culturally diverse community population. All items are at or below an eighth-grade reading level.
To ensure content validity and that the measure encompasses all appropriate domains of interest, 12 groups of 3 parents each were interviewed in small-group settings at an elementary school. Parents completed pilot versions of the measure and were asked whether the measure allowed them to adequately characterize their experiences of their child's health care. Parents were asked to nominate additional domains or items and to indicate items thought to be irrelevant or redundant. Minor changes were made to the measure based on parent feedback at this stage.
To ensure item clarity and to identify and modify unclear items, the modified instrument was then piloted with 15 additional parents at an elementary school and a preschool using cognitive interviewing methodology.28,,29 Parents completed the questionnaire and then, for each item, were asked to describe in their own words what the item was asking. Discrepancies between item intent and parents' understanding were noted, and the parent was asked for suggestions as to how to reword the item. The pilot subjects easily understood the majority of the items. Several items were modified or deleted based on this feedback.
The resulting measure included 23 items measuring 6 aspects of primary care. The items, grouped by aspect of primary care, are shown in the “Appendix.”
The instructions were designed to be applicable for parents of children regardless of the presence of a regular site or source of care or the type of health care provider. They read as follows:
“The person your child sees for health care might be a general doctor, a specialist doctor, a nurse practitioner, a physician assistant, or a nurse. Although these questions ask about ”the doctor,“ please think about the person (or people) your child sees for health care.”
Except for the 2 longitudinal continuity items, the response scale for the instrument was a 5-point Likert-type scale, with the options 0 = never, 1 = sometimes, 2 = often, 3 = almost always, and 4 = always. For the coordination scale, an additional response choice of NA (not applicable) was included, and scored as a user-missing value. For ease of interpretability, these items are transformed to a 0 to 100 scale, with 100 being best, as follows: 0 = 0, 1 = 25, 2 = 50, 3 = 75, and 4 = 100. The response scale for the longitudinal continuity items was; 0 = no particular place (person), 1 = <6 months, 2 = 6 months to 1 year, 3 = 1 to 2 years, 4 = 3 to 5 years, and 5 = >5 years. These responses were rescored to be compatible with the 0 to 100 scale as follows: 0 = 0, 1 = 20, 2 = 40, 3 = 60, 4 = 80, and 5 = 100. Computing the mean of the nonmissing values on each scale formed the total scale score, as well as the scores for each subscale.
The P3C was developed in English, and then translated into Spanish, Vietnamese, and Tagalog. Translation was accomplished using forward-backward translation striving for conceptual, as opposed to syntactical equivalence and consistent language level.30–34
Respondents also completed the PedsQL, a reliable and valid measure of children's health-related quality of life.27In this study, the parent proxy-report version is used. Additionally, respondents were asked whether there was someone they would consider their child's personal doctor, and whether their child currently had health insurance.
As part of a larger study to validate a measure of quality of care for vulnerable children, the P3C was administered to parents of children in 228 classes, from kindergarten through the sixth grade, at 18 elementary schools within a large, urban school district. Schools were selected for inclusion in the study based on the presence of the 3 target languages (Spanish, Vietnamese, and Tagalog) and heterogeneity of socioeconomic status, as measured by the percentage of the student body eligible for federal free or reduced-priced lunch. Classes at schools were randomly selected within grade. Classes in which the consent rate was below 50% were dropped from the sample and resampled if additional classes at that school were available.
Project staff visited each classroom and distributed the questionnaires for students to take home to their parents. Parents signed the informed consent and completed the surveys at home, and returned them to school via the students. At several schools, sessions were held for parents with limited literacy. In these cases, bilingual staff was available to administer the survey to the parents. Phone calls were made to those parents who had missing data on any part of the larger survey.
This protocol was reviewed and approved by the institutional review board at Children's Hospital and Health Center, San Diego, California.
The overall response rate for the survey was 66%. A total of 3371 parents (77.1% mothers, 16.9% fathers, 6% percent other) completed the P3C. The sample was 50.8% girls, and diverse with respect to race/ethnicity (14.0% white, 37.6% Latino, 13.3% black, 5.9% Vietnamese, 21.3% Filipino, 6.8% Other Asian-Pacific Islander, 0.4% Native American, 0.8% other). 64.4% of respondents completed the P3C in English (27.7% Spanish, 4.2% Vietnamese, 3.6% Tagalog). Regarding educational attainment, 33.6% of mothers and 30.7% of fathers had not completed high school, while 22.2% of mothers and 19.5% of fathers had graduated from college or beyond. Parents reported that 20.2% of the children had no health insurance, 38.9% had no regular physician, and 10.8% had a chronic health condition.
Missing data were 1.88% for the sample overall (1.21% for respondents completing the survey in English and 3.06% for respondents completing the survey in another language, 1.22% in cases where the mother had completed high school and 2.55% if she had not).
The P3C Total scale shows minimal floor and ceiling effects overall, by insurance status, and by language of form. Regarding subscales, notable floor effects exist for the continuity subscale, especially for uninsured (45.1%) and for non-English speakers (36.6%). Notable ceiling effects exist for the communication subscale, especially for the insured (35.5%) and for those who completed the questionnaire in English (37.1%).
Internal Consistency Reliability
The internal consistency reliability (α) coefficients, as well as scale descriptives, of the P3C Total scale and subscales are displayed in Table 1. An α coefficient of 0.70 is recommended for group comparison, whereas the more stringent 0.90 level is recommended for individual level comparison.35 As can be seen, the internal consistency for the Total scale and for the subscales of the P3C are acceptable, for the whole sample and for the subgroup having a personal doctor. These results hold, as well, for groups with and without insurance, and for those completing the survey in English and in another language.
The factor structure of the P3C was explored using factor analysis. Table 2 displays the results of the principal components factor analysis, using oblique rotation and forcing 6 factors (the same number as the number of subscales). Factor rotation converged in 19 iterations and the solution explained a total of 77.0% of the variance. With few exceptions, the items of the 6 factors are consistent with the a priori hypothesized P3C subscales.
Table 3 displays the comparisons on mean P3C Total scale scores between groups thought to differ in the quality of primary care received. Independent sample t tests compared children with and without health insurance; children whose parents completed the survey in English or another language; and children with and without a regular physician. All comparisons were statistically significant. Multivariate analyses of variance comparing the P3C subscale means between the above factors confirmed that the subscale means also were significantly different in every comparison.
We examined the relationship between P3C and PedsQL scores. We split the sample into 3 tertiles according to P3C score and compared these 3 groups on their PedsQL scores. A 1-way analysis of variance indicated that the differences among these means were statistically significant (F(2, 3260) = 40.94, P < .001), and posthoc tests showed that the highest tertile P3C group (mean: 80.24, standard deviation [SD]: 16.29) had significantly higher PedsQL scores than did the lowest (mean: 74.29, SD: 16.91) and middle (mean: 75.07, SD: 16.78) tertiles, which were not significantly different from one another (Fig 1).
This study presents the feasibility, reliability, and validity of the P3C. Based on the IOM definition of primary care, the P3C elicits a parent's reports of certain characteristics of their child's primary care which, when present, constitute high quality care. In this diverse community sample, where a third of the mothers and 30% of the fathers had not completed high school, the low rates of missing data overall and by language indicate that English-speakers, as well as Spanish-, Vietnamese-, and Tagalog-speakers were able to respond to the P3C items. For the sample overall and for groups defined by insurance status and language spoken, the internal consistency reliability of the P3C total scale exceeded the stringent 0.90 standard required for comparisons between individuals. The internal consistency reliability of the subscales was also strong, exceeding the 0.70 standard required for group comparisons and, in some cases, approaching or exceeding 0.90. The results of the factor analysis supported the a priori conceptually derived subscales, lending validity to the subscale scores. The total scale and all subscale scores distinguished between children with and without health insurance, between those whose parents completed the P3C in English or another language, and between those with and without a regular physician. Furthermore, the P3C was related to health-related quality of life, as measured by the PedsQL. The 5- to 6-point difference in PedsQL scores between children in the group with the highest P3C scores and those in the middle and lowest groups was similar to the difference in PedsQL scores found between healthy and chronically ill children in the original validation of the PedsQL.27 These data imply that the P3C is a feasible, reliable, and valid measure of primary care characteristics, suitable for use in large, diverse community samples.
Several shortcomings exist. Test-retest reliability was not assessed for the P3C. Additional field-testing is planned to address test-retest reliability. The P3C was not compared with an extant measure of primary care. Additional research comparing the P3C to an existing measure would strengthen evidence for its validity. The P3C measures only parents' reports of their child's primary care. It does not include a provider-report form, as other measures have.16 Regarding translation, although the measure was translated using state-of-the-art methods, it was not simultaneously developed in these other languages, as has been sometimes recommended. However, the measure seems to be feasible, reliable, and valid in other languages and the present method has the advantage of creating a set of items that are parallel across language. The P3C does not specify a particular provider, and thus might not accurately represent a particular source of primary care. This was intentional, as the goal was to develop a measure that could be used irrespective of the presence of a regular provider of care. The P3C is intended to measure the quality of primary care received, rather than the quality of a specific provider of primary care. It stands to reason that at least those parents reporting a regular provider were thinking of a particular person as they completed the form, and the internal consistency for this subgroup was very similar to the overall sample. Nevertheless, additional research is still needed to determine whether the measurement properties of the P3C hold when respondents report explicitly on a specific health care provider. Finally, the relationship between P3C scores and PedsQL scores, although supportive of the construct validity of the P3C, is subject to confounding. Specifically, third variables such as socioeconomic status, race/ethnicity, language, and education, may affect both P3C and PedsQL scores. Additional research incorporating these variables is necessary to further clarify this relationship.
These results have several implications for pediatricians, patients, purchasers, and policy makers. Those pediatricians and medical groups interested in performance improvement now have a measure that can serve as an indicator of the quality of primary care experienced by parents of their patients. The instrument's brevity minimizes respondent burden, and the results presented here support the idea that it is sufficiently reliable and valid to measure changes in quality of care cross-sectionally, and perhaps even over time. Parents and those purchasing health care have, in the P3C, a common metric for evaluating the experiences of parents of patients in a particular pediatric practice or group. Such information could be useful for enrollment and purchasing decisions. Policy makers, likewise, have a tool to measure the experiences of populations of interest, such as children covered by Medicaid or the federal State Child Health Insurance Program, or minority populations who may have limited English ability.
Additional research on the P3C and its correlates is planned. A follow-up survey of the original sample is being fielded. Changes in health insurance status in the sample are anticipated, with some uninsured children becoming insured. Given extant research linking insurance status with receipt of primary care,36 measuring the resulting changes in P3C scores for this subsample will enable conclusions to be drawn regarding the sensitivity of this measure to change. A separate subsample will be asked to respond to a 2-week follow-up to examine the test–retest reliability of this instrument. Field trials in other settings will also allow evaluation of the P3C when respondents are asked to report on a specific provider. Finally, research is needed to more fully explicate the links between insurance status, vulnerability, use of primary care, quality of primary care, and health-related quality of life.
This research was supported by the Agency for Healthcare Research and Quality (Grant R01 HS10317) and the Substance Abuse and Mental Health Services Administration.
We thank the parents and students at San Diego Unified School District who generously gave their time to complete these surveys, the principals and teachers who graciously allowed us onto their campuses and into their classrooms, and Jack Campana, Rose Marie Lofgren, RN, MA, CNP, and Sandy Wright, RN, BSN, whose experience, cooperation, and sage advice enabled the project team to forge strong working relationships with the schools in this sample.
- Received September 28, 2000.
- Accepted December 15, 2000.
Reprint requests to (M.S.) Center for Child Health Outcomes, 3020 Children's Way, MC 5053, San Diego, CA 92123. E-mail:
- IOM =
- Institute of Medicine •
- PEDsQL =
- Pediatric Quality of Life Inventory •
- SD =
- standard deviation
- ↵Starfield B. Primary Care: Balancing Health Needs, Services, and Technology. New York, NY: Oxford University Press; 1998
- Varni JW,
- Seid M,
- Kurtin PS
- ↵US Department of Commerce. Census Brief: Children Without Health Insurance. Washington, DC: US Department of Commerce, Bureau of the Census; 1998
- Halfon N,
- Newacheck PW,
- Wood DL,
- St Peter RF
- Wood DL,
- Hayward RA,
- Corey CR,
- Freeman HE,
- Shapiro MF
- ↵Donaldson M, Yordy K, Lohr K, Vanselow N, eds. Primary Care: America's Health in a New Era. Washington, DC: National Academy Press; 1996
- Szilagyi PG,
- Zwanziger J,
- Rodewald LE,
- et al.
- Kempe A,
- Beaty B,
- Englund BP,
- Roark RJ,
- Hester N,
- Steiner JF
- ↵Schwarz N, Sudman N. Answering Questions: Methodology for Determining Cognitive and Communicative Processes in Survey Research. San Francisco, CA: Jossey-Bass; 1996
- ↵Nunnally JC, Bernstein IR. Psychometric Theory. 3rd ed. New York, NY: McGraw-Hill; 1994
- Copyright © 2001 American Academy of Pediatrics