 |
|
 |
|
 |
 |
|
 |
 |
 |
 |
 |
 |
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
PEDIATRICS Vol. 108 No. 2 August 2001, pp. 264-270
,
From the * Center for Child Health Outcomes, Children's
Hospital and Health Center, San Diego, California; and Objective. A measure of
pediatric primary care quality that is brief, practical, reliable, and
valid would be useful to patients and pediatricians, policymakers, and
health system leaders. Parents have a unique perspective from which to
report their experiences with their child's primary care, and these
reports may be valid indicators of pediatric primary care quality. The
research objective was to develop a brief parent report of their
children's primary care, the Parent's Perceptions of Primary Care
measure (P3C), and to test its reliability and validity as a measure of
pediatric primary care quality.
Study Design. The P3C was based on the elements of primary
care as defined by the Institute of Medicine. Pretesting of domain
content and item clarity was accomplished via focus interviews. The P3C
was developed in English and translated to Spanish, Vietnamese, and Tagalog. The 23-item P3C yields a total score, as well as subscale scores for continuity, access, contextual knowledge, communication, comprehensiveness, and coordination. The P3C was administered to 3371 parents of children in kindergarten through sixth grades in a large,
urban school district.
Principal Findings. The percentage of missing values for
the overall sample was 1.88%, indicating acceptable feasibility. Range
of measurement, assessed via floor and ceiling effects, was moderate to
good. Cronbach's coefficient Conclusions. The P3C is a practical, reliable, and valid
measure of parents' reports of pediatric primary care quality. This
brief measure could be used alone, or in conjunction with other
measures, to enhance outcomes and evaluate the impact of systems
changes on the delivery of the main elements of primary
care.
Department of
Psychiatry, University of California San Diego School of Medicine, San
Diego, California.
ABSTRACT
Top
Abstract
Methods
Results
Discussion
References
, an indicator of scale internal
consistency reliability, was 0.95 for the P3C total scale. Factor
analysis supported the subscale structure, and P3C scores were higher
for children with health insurance, whose parents completed the survey in English, and who had a regular physician. P3C scores were positively related to parent reports of the child's health-related quality of
life.
High quality pediatric primary care is a cornerstone of
efforts to improve health outcomes, control health care spending, and
improve access to care.1-3 To improve the quality of
pediatric primary care, a reliable and valid measure must exist. The
ideal instrument is one that is brief, practical, reliable, and
valid.4,5 Given the current high rate of uninsured
children6 and children without a regular source of
care,7,8 such an instrument must also be applicable
regardless of insurance status and health plan membership or the
presence of a regular provider of care.
Primary care, according to the Institute of Medicine (IOM), is "the
provision of integrated, accessible health care services by clinicians
who are accountable for addressing a large majority of personal health
care needs, developing a sustained partnership with patients, and
practicing in the context of family and community."9 Operational definitions of these concepts vary across researchers. Nevertheless, there is general agreement that primary care is accessible, longitudinally continuous, adequately communicated, contextual (based on a provider's accumulated knowledge of the patient
and family), comprehensive, and coordinated.
Parents are in a unique position to report on the care their children
receive.10-13 Indeed, some aspects of primary care (for
example, accessibility, adequate communication, and contextual care)
describe the parent's/patient's experience of care, rather than a
specific provider behavior. Other aspects of care can be reliably
reported by parents (for example longitudinal continuity,
comprehensiveness). Although some have argued that patients/parents
cannot report on the coordination of care,14 it could be
argued that the best coordination is apparent to the patient or
parent A distinction has been made between patient/parent reports of
experiences with the health care delivery system and ratings of
satisfaction with health care delivery.15,16 Although both
reports of experiences and ratings of satisfaction require the
respondent to make evaluative responses, these 2 types of evaluations
differ in the criterion against which the evaluation is made.
Satisfaction ratings are evaluated against an individual's expectations and preferences, which vary widely across individuals and
ultimately do not suggest ways in which the health system can be
improved.16 On the other hand, reports of experiences are
evaluated against a specific prescriptive criterion (for example, that
care be "adequately communicated"14). As such, deviations from the criterion represent potential decrements in quality
and areas for improvement.
Several measures of primary care currently exist. Starfield and
Cassady's Child Health Systems Primary Care Assessment
Survey16 is an interviewer-administered survey consisting
of 2 parallel surveys for consumers and providers of care that assess
both structural and process indicators of primary care for children. It
focuses on consumers' experiences with health delivery characteristics rather than satisfaction with them, and has been shown to be a reliable
and valid measure. A somewhat shorter version, the Primary Care
Assessment Tool-Child Edition has been described.17 This
version also requires interviewer-administration and takes approximately 25 minutes to complete. This subject burden may have
contributed to the 36% response rate (27% response rate for the
test-retest portion) reported. Safran's Primary Care Assessment Survey18 is a 51-item, self-administered instrument. It
measures 7 domains of primary care for adults within the context of a
specific clinician-patient primary care relationship. Only those
respondents who report an established relationship with a primary
clinician can complete the measure. The Primary Care Assessment Survey
includes both reports and evaluative ratings. It has been subjected to
rigorous psychometric testing and has been linked to outcomes of
care.19 Flocke's Components of Primary Care
Index15 is a brief (20-item), self-administered measure of
adults' reports of their experiences with 4 aspects of primary care.
It requires patients to report, rather than rate, their interaction
with a specific physician in the context of a particular office visit.
All these instruments are based on the IOM's definition of primary
care and are designed as multidimensional instruments, although each
measures a slightly different overlapping set of constructs. Each has
been subjected to various degrees of psychometric testing, and all seem
to have at least some degree of reliability and validity.
However, a need exists for an instrument that is at once a brief,
practical, reliable, and valid measure of pediatric primary care that
can be used irrespective of insurance status or the presence of an
established clinician-patient relationship. Although others have
posited that an established clinician-patient relationship is necessary
for the existence of primary care, and have therefore restricted
measurement of primary care to those patients with such a relationship,
the fact remains that many children, both insured and uninsured, lack a
regular site of care or a regular provider of care. It behooves
researchers, clinicians, and policy makers to develop and use measures
that encompass the experiences of these children as well.
The present study describes such an instrument, the Parent's
Perceptions of Primary Care measure (P3C). Using a large, diverse, community sample, we assessed the feasibility, reliability, and validity of the P3C. Feasibility is measured in terms of missing data
and the percentage of respondents with the lowest possible (floor
effect) and the highest possible (ceiling effect) score. Low rates of
missing data indicate that parents are willing and able to complete the
survey items. Small floor and ceiling effects indicate that the
instrument allows parents to respond to an adequate range of
experiences. Reliability is measured in terms of internal consistency.
High internal consistency indicates that items grouped into a scale are
measuring a similar construct.
Validity is demonstrated by an instrument's ability to respond as
expected, given specific research-based hypotheses. First, we examined
the validity of the P3C subscales via factor analysis. It is
hypothesized that P3C items should be associated with the a priori
subscales. Second, we examined the P3C's ability to distinguish between groups thought to differ in the quality of primary care received. In this case, it is hypothesized that higher quality primary
care will be reported for parents whose children have health
insurance,20,21 who complete the form in
English,22,23 and whose children have a regular health
care provider.24-26 Finally, we examined the relationship
between the P3C and a measure of health-related quality of life, the
Pediatric Quality of Life Inventory (PedsQL).27 It is
hypothesized that higher scores on the P3C will be related to higher
PedsQL scores.
Instrument Development
The P3C is based on the IOM definition of primary care. Using
this definition as a criterion, the P3C was designed to measure 6 components of care which, when present, constitute high quality primary
care. High scores reflect care conforming to this a priori definition.
Thus, the P3C measures quality based on parent reports of their
experiences, rather than ratings of satisfaction with those
experiences. The P3C was designed to measure the quality of primary
care received, rather than the quality of a particular provider of
primary care. This was done so that the care received by children
without a regular provider could also be described in relation to the
IOM definition of quality primary care.
The components of primary care included in the P3C are those on which
parents are thought able to report. The 6 components of primary care
are defined as follows. Longitudinal continuity is defined as the
parent's report of the length of time they have been bringing their
children to a regular place or physician.2,14 Access is
defined as the parent's report of timely and convenient access to care
for their children.14 Communication is defined as the
parent's report of how well the physician listens and explains during
their interactions.15 Contextual knowledge is defined as
the parent's report that the physician knows his or her values and
preferences about medical care issues, clearly understands his or her
child's health needs, and knows the child's medical
history.2 Comprehensiveness is defined as the parent's
report of the extent to which a regular place and/or doctor provides
care for acute and chronic problems and preventive
services.14,15 Coordination of care is defined as the
parent's report of their physician's knowledge of other visits and
visits to specialists, as well as the follow-up of problems through
subsequent visits or phone calls.2
Item Development and Pilot Testing
Items were developed from the existing
literature,14-19 from parent interviews, and were created
for this measure. A team of researchers collaborated on item
development. The components of primary care were divided among the
researchers (M.S., L.O.B., M.Z., M.D.F., and M.N.) so that each
researcher was responsible for developing questions for particular
components. Multiple items were generated for each scale. Items in each
scale were reviewed and discussed by the other team members to ensure
appropriateness. Questions were rewritten or omitted if they were
confusing, lacked clarity, or too closely resembled other items. The
goal was to create parsimonious scales consisting of items that would
be understood by the majority of parents in a diverse community
population. This instrument was specifically developed to be
appropriate for self-administration in a culturally diverse community
population. All items are at or below an eighth-grade reading level.
To ensure content validity and that the measure encompasses all
appropriate domains of interest, 12 groups of 3 parents each were
interviewed in small-group settings at an elementary school. Parents
completed pilot versions of the measure and were asked whether the
measure allowed them to adequately characterize their experiences of
their child's health care. Parents were asked to nominate additional
domains or items and to indicate items thought to be irrelevant or
redundant. Minor changes were made to the measure based on parent
feedback at this stage.
To ensure item clarity and to identify and modify unclear items, the
modified instrument was then piloted with 15 additional parents at an
elementary school and a preschool using cognitive interviewing
methodology.28,29 Parents completed the questionnaire and
then, for each item, were asked to describe in their own words what the
item was asking. Discrepancies between item intent and parents'
understanding were noted, and the parent was asked for suggestions as
to how to reword the item. The pilot subjects easily understood the
majority of the items. Several items were modified or deleted based on
this feedback.
The resulting measure included 23 items measuring 6 aspects of primary
care. The items, grouped by aspect of primary care, are shown in the
"Appendix."
The instructions were designed to be applicable for parents of children
regardless of the presence of a regular site or source of care or the
type of health care provider. They read as follows:
"The person your child sees for health care might be a general
doctor, a specialist doctor, a nurse practitioner, a physician assistant, or a nurse. Although these questions ask about "the doctor," please think about the person (or people) your child sees
for health care."
Except for the 2 longitudinal continuity items, the response scale for
the instrument was a 5-point Likert-type scale, with the options 0 = never, 1 = sometimes, 2 = often, 3 = almost always, and 4 = always. For the coordination scale, an additional response choice of NA (not applicable) was included, and scored as a
user-missing value. For ease of interpretability, these items are
transformed to a 0 to 100 scale, with 100 being best, as follows:
0 = 0, 1 = 25, 2 = 50, 3 = 75, and 4 = 100. The response scale for the longitudinal continuity items was; 0 = no particular place (person), 1 = <6 months, 2 = 6 months to
1 year, 3 = 1 to 2 years, 4 = 3 to 5 years, and 5 = >5
years. These responses were rescored to be compatible with the 0 to 100 scale as follows: 0 = 0, 1 = 20, 2 = 40, 3 = 60,
4 = 80, and 5 = 100. Computing the mean of the nonmissing
values on each scale formed the total scale score, as well as the
scores for each subscale.
The P3C was developed in English, and then translated into Spanish,
Vietnamese, and Tagalog. Translation was accomplished using
forward-backward translation striving for conceptual, as opposed to
syntactical equivalence and consistent language
level.30-34
Other Measures
Respondents also completed the PedsQL, a reliable and valid
measure of children's health-related quality of life.27 In this study, the parent proxy-report version is used. Additionally, respondents were asked whether there was someone they would consider their child's personal doctor, and whether their child currently had
health insurance.
Procedures
As part of a larger study to validate a measure of quality of
care for vulnerable children, the P3C was administered to parents of
children in 228 classes, from kindergarten through the sixth grade, at
18 elementary schools within a large, urban school district. Schools
were selected for inclusion in the study based on the presence of the 3 target languages (Spanish, Vietnamese, and Tagalog) and heterogeneity
of socioeconomic status, as measured by the percentage of the student
body eligible for federal free or reduced-priced lunch. Classes at
schools were randomly selected within grade. Classes in which the
consent rate was below 50% were dropped from the sample and resampled
if additional classes at that school were available.
Project staff visited each classroom and distributed the questionnaires
for students to take home to their parents. Parents signed the informed
consent and completed the surveys at home, and returned them to school
via the students. At several schools, sessions were held for parents
with limited literacy. In these cases, bilingual staff was available to
administer the survey to the parents. Phone calls were made to those
parents who had missing data on any part of the larger survey.
This protocol was reviewed and approved by the institutional review
board at Children's Hospital and Health Center, San Diego, California.
The overall response rate for the survey was 66%. A total of 3371 parents (77.1% mothers, 16.9% fathers, 6% percent other) completed
the P3C. The sample was 50.8% girls, and diverse with respect to
race/ethnicity (14.0% white, 37.6% Latino, 13.3% black, 5.9%
Vietnamese, 21.3% Filipino, 6.8% Other Asian-Pacific Islander, 0.4%
Native American, 0.8% other). 64.4% of respondents completed the P3C
in English (27.7% Spanish, 4.2% Vietnamese, 3.6% Tagalog). Regarding
educational attainment, 33.6% of mothers and 30.7% of fathers had not
completed high school, while 22.2% of mothers and 19.5% of fathers
had graduated from college or beyond. Parents reported that 20.2% of
the children had no health insurance, 38.9% had no regular physician,
and 10.8% had a chronic health condition.
Feasibility
Missing data were 1.88% for the sample overall (1.21% for
respondents completing the survey in English and 3.06% for respondents completing the survey in another language, 1.22% in cases where the
mother had completed high school and 2.55% if she had not).
The P3C Total scale shows minimal floor and ceiling effects overall, by
insurance status, and by language of form. Regarding subscales, notable
floor effects exist for the continuity subscale, especially for
uninsured (45.1%) and for non-English speakers (36.6%). Notable
ceiling effects exist for the communication subscale, especially for
the insured (35.5%) and for those who completed the questionnaire in
English (37.1%).
Internal Consistency Reliability
The internal consistency reliability ( TABLE 1
that part of coordinating care involves communicating the
process and outcomes of that coordination to the parent.
METHODS
Top
Abstract
Methods
Results
Discussion
References
RESULTS
Top
Abstract
Methods
Results
Discussion
References
) coefficients, as well
as scale descriptives, of the P3C Total scale and subscales are
displayed in Table 1. An coefficient
of 0.70 is recommended for group comparison, whereas the more stringent
0.90 level is recommended for individual level
comparison.35 As can be seen, the internal consistency for
the Total scale and for the subscales of the P3C are acceptable, for
the whole sample and for the subgroup having a personal doctor. These
results hold, as well, for groups with and without insurance, and for
those completing the survey in English and in another language.
Scale Descriptives and Internal Consistency Reliability for P3C
Validity
The factor structure of the P3C was explored using factor analysis. Table 2 displays the results of the principal components factor analysis, using oblique rotation and forcing 6 factors (the same number as the number of subscales). Factor rotation converged in 19 iterations and the solution explained a total of 77.0% of the variance. With few exceptions, the items of the 6 factors are consistent with the a priori hypothesized P3C subscales.
|
Table 3 displays the comparisons on mean P3C Total scale scores between groups thought to differ in the quality of primary care received. Independent sample t tests compared children with and without health insurance; children whose parents completed the survey in English or another language; and children with and without a regular physician. All comparisons were statistically significant. Multivariate analyses of variance comparing the P3C subscale means between the above factors confirmed that the subscale means also were significantly different in every comparison.
|
We examined the relationship between P3C and PedsQL scores. We split the sample into 3 tertiles according to P3C score and compared these 3 groups on their PedsQL scores. A 1-way analysis of variance indicated that the differences among these means were statistically significant (F(2, 3260) = 40.94, P < .001), and posthoc tests showed that the highest tertile P3C group (mean: 80.24, standard deviation [SD]: 16.29) had significantly higher PedsQL scores than did the lowest (mean: 74.29, SD: 16.91) and middle (mean: 75.07, SD: 16.78) tertiles, which were not significantly different from one another (Fig 1).
|
| |
DISCUSSION |
|---|
|
Top
Abstract
Methods
Results
Discussion
References
|
|---|
This study presents the feasibility, reliability, and validity of the P3C. Based on the IOM definition of primary care, the P3C elicits a parent's reports of certain characteristics of their child's primary care which, when present, constitute high quality care. In this diverse community sample, where a third of the mothers and 30% of the fathers had not completed high school, the low rates of missing data overall and by language indicate that English-speakers, as well as Spanish-, Vietnamese-, and Tagalog-speakers were able to respond to the P3C items. For the sample overall and for groups defined by insurance status and language spoken, the internal consistency reliability of the P3C total scale exceeded the stringent 0.90 standard required for comparisons between individuals. The internal consistency reliability of the subscales was also strong, exceeding the 0.70 standard required for group comparisons and, in some cases, approaching or exceeding 0.90. The results of the factor analysis supported the a priori conceptually derived subscales, lending validity to the subscale scores. The total scale and all subscale scores distinguished between children with and without health insurance, between those whose parents completed the P3C in English or another language, and between those with and without a regular physician. Furthermore, the P3C was related to health-related quality of life, as measured by the PedsQL. The 5- to 6-point difference in PedsQL scores between children in the group with the highest P3C scores and those in the middle and lowest groups was similar to the difference in PedsQL scores found between healthy and chronically ill children in the original validation of the PedsQL.27 These data imply that the P3C is a feasible, reliable, and valid measure of primary care characteristics, suitable for use in large, diverse community samples.
Several shortcomings exist. Test-retest reliability was not assessed for the P3C. Additional field-testing is planned to address test-retest reliability. The P3C was not compared with an extant measure of primary care. Additional research comparing the P3C to an existing measure would strengthen evidence for its validity. The P3C measures only parents' reports of their child's primary care. It does not include a provider-report form, as other measures have.16 Regarding translation, although the measure was translated using state-of-the-art methods, it was not simultaneously developed in these other languages, as has been sometimes recommended. However, the measure seems to be feasible, reliable, and valid in other languages and the present method has the advantage of creating a set of items that are parallel across language. The P3C does not specify a particular provider, and thus might not accurately represent a particular source of primary care. This was intentional, as the goal was to develop a measure that could be used irrespective of the presence of a regular provider of care. The P3C is intended to measure the quality of primary care received, rather than the quality of a specific provider of primary care. It stands to reason that at least those parents reporting a regular provider were thinking of a particular person as they completed the form, and the internal consistency for this subgroup was very similar to the overall sample. Nevertheless, additional research is still needed to determine whether the measurement properties of the P3C hold when respondents report explicitly on a specific health care provider. Finally, the relationship between P3C scores and PedsQL scores, although supportive of the construct validity of the P3C, is subject to confounding. Specifically, third variables such as socioeconomic status, race/ethnicity, language, and education, may affect both P3C and PedsQL scores. Additional research incorporating these variables is necessary to further clarify this relationship.
These results have several implications for pediatricians, patients, purchasers, and policy makers. Those pediatricians and medical groups interested in performance improvement now have a measure that can serve as an indicator of the quality of primary care experienced by parents of their patients. The instrument's brevity minimizes respondent burden, and the results presented here support the idea that it is sufficiently reliable and valid to measure changes in quality of care cross-sectionally, and perhaps even over time. Parents and those purchasing health care have, in the P3C, a common metric for evaluating the experiences of parents of patients in a particular pediatric practice or group. Such information could be useful for enrollment and purchasing decisions. Policy makers, likewise, have a tool to measure the experiences of populations of interest, such as children covered by Medicaid or the federal State Child Health Insurance Program, or minority populations who may have limited English ability.
Additional research on the P3C and its correlates is planned. A
follow-up survey of the original sample is being fielded. Changes in
health insurance status in the sample are anticipated, with some
uninsured children becoming insured. Given extant research linking
insurance status with receipt of primary care,36 measuring
the resulting changes in P3C scores for this subsample will enable
conclusions to be drawn regarding the sensitivity of this measure to
change. A separate subsample will be asked to respond to a 2-week
follow-up to examine the test-retest reliability of this instrument.
Field trials in other settings will also allow evaluation of the P3C
when respondents are asked to report on a specific provider. Finally,
research is needed to more fully explicate the links between insurance
status, vulnerability, use of primary care, quality of primary care,
and health-related quality of life.
| |
ACKNOWLEDGMENTS |
|---|
This research was supported by the Agency for Healthcare Research and Quality (Grant R01 HS10317) and the Substance Abuse and Mental Health Services Administration.
We thank the parents and students at San Diego Unified School District who generously gave their time to complete these surveys, the principals and teachers who graciously allowed us onto their campuses and into their classrooms, and Jack Campana, Rose Marie Lofgren, RN, MA, CNP, and Sandy Wright, RN, BSN, whose experience, cooperation, and sage advice enabled the project team to forge strong working relationships with the schools in this sample.
| |
FOOTNOTES |
|---|
Received for publication Sep 28, 2000; accepted Dec 15, 2000.
Reprint requests to (M.S.) Center for Child Health Outcomes, 3020 Children's Way, MC 5053, San Diego, CA 92123. E-mail: mseid{at}chsd.org
| |
ABBREVIATIONS |
|---|
IOM, Institute of Medicine; PEDsQL, Pediatric Quality of Life Inventory; SD, standard deviation.
| |
REFERENCES |
|---|
|
Top
Abstract
Methods
Results
Discussion
References
|
|---|
This article has been cited by other articles:
 |
|
 |
|
  S. D. Krugman, A. Racine, S. Dabrow, S. Sanguino, W. Meyer, M. Seid, and J. R. Serwint Measuring Primary Care of Children in Pediatric Resident Continuity Practices: A Continuity Research Network Study Pediatrics, August 1, 2007; 120(2): e262 - e271. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. Jean and C. Cyr Use of Complementary and Alternative Medicine in a General Pediatric Clinic Pediatrics, July 1, 2007; 120(1): e138 - e141. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. D. Stevens, M. Seid, and N. Halfon Enrolling Vulnerable, Uninsured but Eligible Children in Public Health Insurance: Association With Health Status and Primary Care Access Pediatrics, April 1, 2006; 117(4): e751 - e759. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
B. Zuckerman, G. D. Stevens, M. Inkelas, and N. Halfon Prevalence and Correlates of High-Quality Basic Pediatric Preventive Care Pediatrics, December 1, 2004; 114(6): 1522 - 1529. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 I. Sandler, C. Kennedy, and E. Shapiro Parental Grief and Palliative Care Require Attention Arch Pediatr Adolesc Med, June 1, 2004; 158(6): 590 - 591. [Full Text] [PDF]
|
|
|
|
 |
|
 G. D. Stevens and L. Shi Racial and Ethnic Disparities in the Primary Care Experiences of Children: A Review of the Literature Med Care Res Rev, March 1, 2003; 60(1): 3 - 30. [Abstract] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
||
|
||
Home | My Pediatrics | Journal Information | Current Issue | Past Issues | Subscriptions & Services | Contact Us Click here for faster international access © 2001 American Academy of Pediatrics. All rights reserved. |
||
|
||
