Our objective was to conduct a rigorous, structured process to create a new model of well-child care (WCC) in collaboration with a multisite community health center and 2 small, independent practices serving predominantly Medicaid-insured children. Working groups of clinicians, staff, and parents (called “Community Advisory Boards” [CABs]) used (1) perspectives of WCC stakeholders and (2) a literature review of WCC practice redesign to create 4 comprehensive WCC models for children ages 0 to 3 years. An expert panel, following a modified version of the Rand/UCLA Appropriateness Method, rated each model for potential effectiveness on 4 domains: (1) receipt of recommended services, (2) family-centeredness, (3) timely and appropriate follow-up, and (4) feasibility and efficiency. Results were provided to the CABs for selection of a final model to implement. The newly developed models rely heavily on a health educator for anticipatory guidance and developmental, behavioral, and psychosocial surveillance and screening. Each model allots a small amount of time with the pediatrician to perform a brief physical examination and to address parents' physical health concerns. A secure Web-based tool customizes the visit to parents' needs and facilitates previsit screening. Scheduled, non–face-to-face methods (text, phone) for parent communication with the health care team are also critical to these new models of care. A structured process that engages small community practices and community health centers in clinical practice redesign can produce comprehensive, site-specific, and innovative models for delivery of WCC. This process, as well as the models developed, may be applicable to other small practices and clinics interested in practice redesign.
- ACO —
- Accountable Care Organization
- CHC —
- community health center
- EP —
- expert panel
- RAM —
- Rand/UCLA Appropriateness Method
- WCC —
- well-child care
Well-child care (WCC) is the foundation of pediatric primary care in the United States. Pediatricians provide the vast majority of WCC from infancy through adolescence. Through these visits they have a unique opportunity to identify and address social, developmental, behavioral, and health issues that could have significant impact over the long term.1
Studies have shown multiple deficiencies in WCC.2–6 Pediatric providers are currently not providing many important recommended preventive and developmental services, and most parents leave visits with unmet WCC needs.2–4,7 Many of these deficiencies occur more frequently among children in low-income families.5
Practice redesign may create a more effective and efficient system of health care delivery. Proposals to improve WCC delivery include changes such as utilizing group visits, incorporating nonphysician child developmental specialists into the visit, and instituting standardized screening by nonphysician staff.8–10 Although practice redesign in large, integrated delivery systems has been described,11,12 smaller practices with ≤5 physicians provide most primary care.13 Additionally, community health centers (CHCs) are a critical source of primary care for children in low-income communities.14,15 These practices and clinics report major barriers to effective WCC16 and will need explicit processes for selecting and implementing innovative delivery models in ways that are feasible and customized for their families. One such explicit process is a community-based adaptation of the modified Delphi method known as the Rand/UCLA Appropriateness Method (RAM).17
The study objective was to conduct a rigorous, structured process to create a new WCC model in collaboration with a multisite CHC and 2 private practices serving predominantly Medicaid-insured children. To meet this objective, we combined a community-based approach with the RAM process to systematically incorporate previously proposed WCC improvements into a comprehensive, community-specific model of WCC for children ages 0 to 3 years.
We used the following 3-step process to design the new models of care for each clinic/practice.
CAB meetings: 2 working groups (henceforth Community Advisory Boards [CABs]) used data from WCC stakeholders,18–20 a systematic literature review,21 and a WCC framework designed for this study to develop 4 potential models for WCC delivery for children ages 0 to 3 years in low-income communities.
Expert panel (EP) process: these 4 potential models for WCC delivery were evaluated by a National WCC Redesign EP on 4 categories of measures using the RAM.
CAB retreat and model selection: the CABs received the EP findings to select 1 final model for WCC delivery for children ages 0 to 3 years in their clinical setting.
UCLA Institutional Review Board approval was obtained.
The study involved 2 groups: (1) a large, multisite, federally qualified Los Angeles–area CHC and (2) 2 independent Los Angeles–area pediatric primary care practices serving predominantly Medicaid-insured children. Each clinical group convened a CAB for the study; CAB members and practice characteristics are detailed in Table 1.
National WCC Practice Redesign EP
The EP consisted of 10 individuals. We requested nominations for panelists from national organizations listed in Table 1. Each organization provided nominee(s) or suggested participants for the EP. We also invited 3 nationally recognized WCC redesign experts. Panelists are listed in Table 1.
The RAM was developed to synthesize scientific literature with expert opinion. It begins with a detailed literature review and synthesis of the evidence, after which various scenarios (ie, specific options for care) are developed. An EP rates appropriateness or necessity of these scenarios in 2 rounds, first independently and then at a meeting of all panelists.17,22 Although RAM EPs have typically been used to determine appropriateness or necessity of medical procedures or to develop clinical decision-making tools, they have also been used to address health-related research questions.23 We adapted the RAM to rate the models developed by our CABs.
We performed a systematic literature review as part of the RAM. We searched PubMed using criteria to identify relevant English-language articles (January 1981 through February 2012). We considered observational studies, controlled trials, and systematic reviews evaluating efficiency and effectiveness of WCC for children ages 0 to 5 years. Interventions were organized into 3 categories: providers, formats, and locations of care. For providers, we focused on the use of nonphysician providers for WCC services. For formats, we focused on alternative ways to deliver care, such as the use of non–face-to-face formats. For locations, we considered nonclinical locations for WCC services. We used independent article review, including study quality, by 3 investigators with consensus resolution of discrepancies. Of 275 screened articles, 33 met inclusion criteria. Seventeen focused on providers, 13 on formats, 2 on locations, and 1 on a combination. A literature review summary was sent to panelists to aid in their initial ranking of the new models of care. Details of the literature review can be found elsewhere.21
Model Development Process
Step 1. CAB meetings. The CABs convened for 4 in-person meetings and a full-day retreat. The CABs developed the models by using data collected on stakeholders’ views on WCC practice redesign,16,18–20 a summary of the systematic literature review, and the WCC Framework. The WCC Framework considers alternative structures for care: nonphysician providers (eg, nurses, health educators, social workers), nontraditional formats (eg, group visits, Internet, phone), and nonclinical locations (eg, day care centers, home visits, grocery stores) for each WCC service. The CABs selected WCC practice redesign options on the basis of the literature review summary and the acceptability/preference of each stakeholder group (parents, providers, payers), prioritizing options that were strongly preferred by, or acceptable to, all 3 stakeholder groups. WCC options that were not feasible for payers (eg, day care center visits) or providers (eg, home visits), or not acceptable to parents (eg, retail-based clinics), were not selected as potential options for the new models. We created a summary document of stakeholders’ views that summarized the findings from our previous qualitative studies of parents, providers, and payers to assist the CAB in understanding these stakeholders’ views on various WCC redesign options. The selected WCC practice redesign elements were then used to create comprehensive models, including a provider, format, and location for each WCC service.
Step 2. EP process. Ratings were accomplished in 2 rounds. For the first round, panelists individually received a literature review summary and a detailed description of the CABs’ 4 models. Panelists used a 1 to 5 rating scale to individually score each model’s potential to provide better (rating = 5) or worse (rating = 1) than usual care on 4 domains: (1) receipt of recommended services, (2) family-centeredness, (3) timely and appropriate follow-up, and (4) feasibility and efficiency; they rated 16 items per model. Ranking response options were as follows: 1 = very likely worse than usual care, 2 = somewhat likely worse than usual care, 3 = about the same as usual care, 4 = somewhat likely better than usual care, and 5 = very likely better than usual care.
For the second round of ratings, panelists had a full-day in-person meeting to discuss the models, ratings, and areas of disagreement. The EP then modified the models on the basis of their discussion and individually rerated each model. Panelists focused on potential benefits and challenges of each model and were asked not to consider costs of the model17; costs were explicitly considered in the CABs’ final selection of a model.
Step 3. CAB retreat and model selection. The CABs met for a 1-day retreat to select a final model using the EP results as a guide. They used a structured process to consider EP ratings, feasibility, acceptability, and relative costs for each model with the use of a break-even analysis to reach consensus on a final model. First, the CAB reviewed and discussed the models and the EP ratings and modifications to the models. Next, break-out groups detailed model selection criteria and addressed discussion questions for each model (Table 2). Each break-out group focused on 1 model and presented its findings to the larger group. The CAB reviewed the model selection criteria for each model (feasibility, acceptability, and the relative costs) and came to consensus as to which model should be chosen for the pilot test.
Before the EP meeting, panelists’ scores from the first round were entered into a database. A personalized summary was created for each panelist, showing the distribution of responses and his or her own personal responses for each question. The moderators used a summary table showing the distribution of ratings for each question and the effectiveness designation to guide the EP discussion. After the second round of ratings, a similar summary was created. We examined the spread of rankings across panelists to determine the level of EP agreement/disagreement and the frequency of rating scores for each question to determine overall EP assessment of each model.
To ensure that the new model of care would be at minimum cost-neutral to the clinics, we conducted a break-even analysis. The main ongoing cost of the models was the health educator compensation. By using an estimate for a masters-level health educator annual salary ($47 000 with 22% benefits) and data on monthly averages of patients seen, we determined the increase in patient volume needed to cover health educator costs for the CHC.
After reviewing the qualitative summary of stakeholders’ views, the WCC framework, and the systematic literature review summary, CAB members populated the WCC framework with potential elements of a redesigned model for WCC (Table 3). They used this WCC framework to consider a range of redesign options for providers, formats, and locations of care that they wanted to potentially include in their models (not all elements made it into developed models). Many of these options specifically addressed the needs of the low-income populations that they served; these options included using nonphysicians to ensure routine comprehensive screening and community referral for a range of family psychosocial concerns (eg, homelessness, unemployment) and using non–face-to-face encounters to improve parent education and guidance without increasing the burden of office visits for families.
The 2 CABs together created 9 models that were collapsed into 4 categories of models to be evaluated by the EP. The CHC CAB developed 6 models; the private practice CAB developed 3. All models incorporated some elements from the WCC Framework areas of “providers” and “format”; elements from “location” were not chosen because the CABs determined that these were either not feasible or not acceptable to stakeholders. The models were categorized as follows: (1) group visit models, (2) individual visit provider team–based models, (3) technology-based models, and (4) mixed models. The models were meant to provide preventive care for healthy children and children with special health care needs because routine preventive care is universal. In the group visit models, a small group of parent-child dyads (∼6–8) are scheduled for age-specific well-visits as a group. A health educator leads the group in a discussion covering age-appropriate anticipatory guidance. Depending on the specific group visit model designed by the CABs, infants are examined, measured, and immunized either before the group session, after the group session, or in a rotation during the group session. In the one-on-one team-based model (which we called “Station-to-Station”), parents have a one-on-one encounter with a health educator for the bulk of the visit to receive anticipatory guidance, psychosocial screening and services, and developmental and behavioral surveillance and screening and spend a brief portion of the visit with the physician for the physical examination and parents’ medical concerns. The technology-based model uses the Internet, secure messaging, and phone to provide anticipatory guidance and psychosocial, behavioral, and developmental services; the in-person physician visit in this model is an abbreviated, problem-focused visit for the physical examination and medical concerns. The CABs envisioned this model both as a stand-alone model and as a set of technology-based tools that could be added to any of the other models. Finally, the mixed models combine periodic group sessions for age-appropriate parent education and guidance with a one-on-one brief physician visit.
The EP ranked these 4 models in 2 rounds, offering modifications to the models at the EP meeting (Table 4). Table 5 provides a summary of results; Supplemental Appendices 1, 2, 3, and 4 provide more detailed tables with the distribution of scores.
CAB Model Selection
A formal break-even analysis was performed with the CHC CAB on the basis of information from their chief financial officer. Currently, well-visits are given a 20-minute time slot (3 patients per hour or 12 patients per half-day session). For the new model to break even, the CHC would need a 50% increase in productivity during well-visit sessions, equivalent to increasing productivity to 1.5 patients per 20-minute time slot or 18 patients per session. The CAB determined that this could be accomplished by decreasing physician time in well-visits so that providers could see an additional 1.5 sick visits per hour.
For the private practices, a similar break-even analysis was not feasible due to the different payment structure for a private practice with primarily managed-care Medicaid patients (capitated payments) compared with payment structures for federally qualified health centers receiving enhanced reimbursement that includes per-visit rates (via prospective payment systems or alternative payment methods) for Medicaid managed-care patients. An increase in productivity for these private practices would not produce more revenue without broader changes in staffing mix or overall patient panels.
The CHC CAB selected the Group Visit Model. Group visits are scheduled in 2-hour blocks, with 9 patients scheduled for each block. The group session is led by the health educator with the physician available for the majority of the group session. The CAB estimated that during each 2-hour block, the pediatrician would be able to see an additional 2 sick visits. They would devote the first 2 hours of a half-day session to the group visit and the second 2 hours of the half-day session to sick/urgent visits. To break even, they would need to see 7 patients during the second 2 hours of the half-day session (1 urgent visit per 17-minute time slot) to total 18 patients per session.
The private practice CAB selected the Station-to-Station Model. These visits are scheduled in 40-minute blocks, with the physician needed for 10 minutes of that block.
In each model, the CABs determined that encounter data would be entered in “real time” in the electronic health record. Both CABs wanted to use a Web-based previsit tool to focus the visit on parents’ needs. The tool would allow parents to select priorities for their child’s visit, complete prescreening questions, and receive Web-based anticipatory guidance. It would be completed at home before the visit or on a tablet/kiosk in the waiting room or examination room, with data automatically uploaded to the electronic health record. The CABs preferred to find an existing tool to meet their needs; the EP offered suggestions.24
Both CABs also decided to include a text message service as part of the model to send weekly age-specific health messages, anticipatory guidance, and visit reminders to parents. They preferred to use an existing system that could be customized to their needs (eg, healthy-txt.com).
In collaboration with a CHC and 2 pediatric practices, we used a modified Delphi/EP process to design a new model for WCC delivery at each clinical site. The 2 newly developed models rely heavily on a trained health educator for anticipatory guidance and efficient, but comprehensive, developmental, behavioral, and psychosocial surveillance. The well-visit is considerably longer in these models of care, and parents of healthy children spend only a minority of their time with the physician at each visit. A Web-based tool to customize the visit to parents' needs and facilitate previsit screening is viewed as an essential element of all the models. Scheduled non–face-to-face methods for parent communication with the health care team are also viewed as critical to success.
In creating these models, we combined a community-based approach with a modified Delphi method. Our adaptation of the RAM is novel in 2 ways: we used the RAM (1) in conjunction with clinic-specific working groups and (2) to design a new and innovative delivery model for care. This structured process engaged small, independent practices and CHCs in clinical practice redesign; our findings from this process reveal that it can produce comprehensive, site-specific, and innovative care delivery models.
Few previous studies have focused on designing new comprehensive models for WCC. In our systematic literature review, we found tools and strategies for improving WCC delivery, but few offered a comprehensive model.21 In 1995, Zuckerman and Parker25 proposed a comprehensive, community-based system of pediatric primary care that included collaboration with early childhood educators to enhance developmental and behavioral services and a 2-generational approach that included primary care services to parents and children. Other proposed models have included e-visits and “tailored” well-visits based on the family’s needs.26 Internet-based tools to deliver anticipatory guidance have been examined as additional WCC tools.27,28 The most rigorously studied of these comprehensive models of care is Healthy Steps for Young Children. It uses a nonphysician provider to provide developmental and behavioral services to parents, including screening, assessment, and guidance. Parents discuss behavioral and developmental issues during well-visits with this Healthy Steps developmental specialist, either concomitantly or separately from the physician visit.29 In a prospective controlled study of this program, participating families discussed more anticipatory guidance topics, were more likely to have a developmental assessment, and were more likely to comply with WCC visits and immunization schedules.10,30 Program costs, however, have limited its ability to be widely adopted.31
Distinct from previous comprehensive models for redesigned WCC, the CABs had an explicit goal of decreasing reliance on physician time for routine WCC services while simultaneously providing more time for parents to discuss anticipatory guidance, psychosocial concerns, and developmental and behavioral concerns with a preventive care professional. The families served by these practices had multiple psychosocial needs often related to poverty, as well as numerous concerns regarding behavior and development.19
Delivery system design should simultaneously pursue all 3 dimensions of the Triple Aim, which includes the goal of reducing costs.32 A new model that can deliver patient-centered WCC more efficiently may meet this goal without providing direct savings or increased revenue to a private practice with primarily managed-care patients. These practices would have to rely on their ability to attract more patients or on their ability to reduce the number of physicians or nurses on the basis of increased productivity as a result of the new model. Conversely, the payment structure of the CHC provides a more direct financial benefit to increasing physician productivity in this new model. Under the Affordable Care Act, Accountable Care Organizations (ACOs) allow potential savings from reduced unnecessary emergency department and urgent care visits and hospitalizations; these savings may be realized in these new WCC models and will be particularly important in incentivizing practice redesign. Without data on these potential savings, it is unlikely that practices will have the incentives to invest in redesign. Practices that participate in ACOs can deliver and benefit from these potential cost savings; however, ACOs will need to find ways to pass on these cost savings to them.
Under these 2 new models of care that rely on the health educator for most routine WCC services outside of the physical examination, it is possible that the patient-doctor relationship may be altered or that physician satisfaction may suffer as a result of being less integral to routine WCC. However, in this team-based approach to care, the parent–health educator relationship would be as important to the parent, particularly related to nonmedical WCC needs. It is also probable that parents would have significantly longer well-visits, particularly in the group visit model. Well-visit duration has been associated with content and quality of care and parent satisfaction with care.33 In a previous study, parents with visits >20 minutes in duration were more likely to receive recommended services and more likely to have their concerns and questions heard. Although parents do not want to spend more time in the waiting or examination rooms waiting for the visit to begin, parents have expressed a desire to spend more time in visits getting their needs addressed, particularly psychosocial, behavioral, and developmental needs.19,34 Our findings from previous WCC redesign studies suggest that a WCC model that is less reliant on the physician for routine WCC services is acceptable to parents, payers, and pediatricians.16,18–20 With the advent of retail-based clinics and Internet-based care,35 a greater burden of chronic disease during childhood,36 and an increasing need for chronic care management,37 primary care pediatrics may need to adapt to a more specialty-based model similar to primary care pediatrics in many other developed nations.38,39
This study has several limitations. First, WCC models were designed specifically for practices involved in the study and may not be generalizable to others. We used a structured process that may be applicable to other practices serving predominantly Medicaid-insured children and to CHCs. Although it is not feasible to repeat the EP process that we conducted, practices/clinics interested in WCC redesign might consider using our EP results and organize similar CABs to evaluate the models and components of each model for applicability and feasibility in their own setting. Our findings may not be applicable to practices that do not serve low-income families, because parent perspectives on WCC redesign differ for samples of higher-income parents.34 Finally, we conducted a preliminary break-even analysis for the CHC but do not have actual cost or utilization data.
We used a structured process that engages small practices and CHCs in WCC redesign. Our findings suggest that such a process can produce comprehensive, site-specific, and innovative models for WCC delivery for children in low-income communities. This process, as well as the models developed, may be applicable to other practices/clinics that serve publicly insured children in implementing WCC practice redesign. These models will need to be implemented and tested in various settings to understand how they might affect child and family outcomes.
We thank the 2 pediatric practices, the community clinic, and the expert panelists who made this project possible.
- Accepted April 16, 2014.
- Address correspondence to Tumaini R. Coker, MD, MBA, Mattel Children’s Hospital, UCLA, 10833 Le Conte Ave, Room 12-467, Los Angeles, CA 90095. E-mail:
Dr Coker conceptualized and designed the study; led data acquisition, analysis, and interpretation; and drafted the initial manuscript; Ms Moreno made substantial contributions to the acquisition and analysis of data and reviewed and revised the manuscript; Drs Shekelle and Chung made substantial contributions to the study conception and design and acquisition and interpretation of data and reviewed and revised the manuscript; Dr Schuster made substantial contributions to the study conception and design and reviewed and revised the manuscript; and all authors approved the final manuscript as submitted.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: This research was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (K23-HD06267) and the Health Resources and Services Administration (R40-MC21516). Funded by the National Institutes of Health (NIH).
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Bright Futures/American Academy of Pediatrics
- Bethell C,
- Reuland CH,
- Halfon N,
- Schor EL
- Halfon N,
- Regalado M,
- Sareen H,
- et al
- Leatherman S,
- McCarthy D
- Rice RL,
- Slater CJ
- Schor EL
- Kilo CM,
- Wasson JH
- Taylor EF,
- Machta RM,
- Meyers DS,
- Genevro J,
- Peikes DN
- ↵National Center for Health Statistics. National Ambulatory Medical Care Survey: 2010 summary tables. Available at: www.cdc.gov/nchs/ahcd/web_tables. Accessed November 1, 2013
- Mooney K,
- Moreno C,
- Chung PJ,
- Elijah J,
- Coker TR
- Fitch K,
- Bernstein S,
- Aguilar M,
- et al
- Coker T,
- Casalino LP,
- Alexander GC,
- Lantos J
- Coker TR,
- Chung PJ,
- Cowgill BO,
- Chen L,
- Rodriguez MA
- Coker TR,
- Windon A,
- Moreno C,
- Schuster MA,
- Chung PJ
- ↵Dalkey NC. The Delphi Method: An Experimental Study of Group Opinion. Santa Monica, CA: RAND Corporation; 1969
- ↵van het Loo M, Kahan J. The RAND Appropriateness Method: an annotated bibliography. RAND Europe; 1999. Available at: www.rand.org/pubs/rand_europe/RE99-010.html. Accessed May 6, 2014
- ↵The Child and Adolescent Health Measurement Initiative. Well-visit planner. Published 2014. Available at: https://www.wellvisitplanner.org/Providers/Default.aspx. Accessed February 6, 2014
- Zuckerman B,
- Parker S
- Christakis DA,
- Zimmerman FJ,
- Rivara FP,
- Ebel B
- Zuckerman B,
- Parker S,
- Kaplan-Sanoff M,
- Augustyn M,
- Barth MC
- Berwick DM,
- Nolan TW,
- Whittington J
- Halfon N,
- Stevens GD,
- Larson K,
- Olson LM
- Radecki L,
- Olson LM,
- Frintner MP,
- Tanner JL,
- Stein MT
- ↵Galewitz P. Insurers embrace “virtual” doctor visits. Kaiser Health News. May 6, 2012. Available at: www.kaiserhealthnews.org/stories/2012/may/07/telemedicine.aspx. Accessed May 6, 2014
- Kuo AA,
- Inkelas M,
- Lotstein DS,
- Samson KM,
- Schor EL,
- Halfon N
- Coker TR,
- Thomas T,
- Chung PJ
- Copyright © 2014 by the American Academy of Pediatrics