Measuring the “Triple Aim” in Transition Care: A Systematic Review
- Megan Prior, BAa,b,
- Margaret McManus, MHSa,
- Patience White, MD, MAa, and
- Laurie Davidson, MLIS, MEdb
BACKGROUND AND OBJECTIVES: Without adequate support, adolescents transitioning from the pediatric to the adult health care system are at increased risk for poor health outcomes. Numerous interventions attempt to improve this transition, yet few comprehensively evaluate efficacy. To advance evaluation methods and ultimately the quality of transition services, it is necessary to understand the current state of health care transition measurement. This study examines and categorizes transition measures by using the “Triple Aim” framework of experience of care, population health, and cost of care.
METHODS: Ovid Medline and the Cumulative Index to Nursing and Allied Health Literature were searched for articles published between 1995 and 2013. Two reviewers independently screened studies and included those that evaluated the impact of a health care transition intervention. Measures were subsequently classified according to population health, experience of care, and costs of care.
RESULTS: Of the 2282 studies initially identified, 33 met inclusion criteria. Population health measures were used in 27 studies, with disease-specific measures collected most frequently. Fifteen studies measured cost, most often service utilization. Eight studies measured experience of care, with satisfaction assessed most commonly. Only 3 studies examined all 3 domains of the “Triple Aim.” Transition interventions described in the gray literature were not reviewed.
CONCLUSIONS: Transition programs are inconsistently evaluated in terms of their impact on population health, patient experience, and cost. To demonstrate improvement in the transition from pediatric to adult health care, a more robust and consistent set of measures is needed.
- transition to adult care
- population health
- patient experience
- cost of care
- quality improvement
- AHRQ —
- Agency for Health Care Research and Quality
- DM —
- diabetes mellitus
- HbA1C —
- hemoglobin A1C
- IHI —
- Institute for Health Care Improvement
Movement from the pediatric to the adult health care system is a facet of every youth’s transition to independence. While learning to manage their own medical care, many adolescents also must change providers. Decline in health over this period is pervasive: increasing obesity,1 sickle cell disease complications,2 and renal transplant loss3 are among the many documented poor health outcomes encountered during the transition to young adulthood. These outcomes are often the result of preventable lapses in adherence,4 follow-up,5 and insurance coverage.6
Preventing adverse outcomes during transition requires support, especially for youth with special health care needs. In 2011, the American Academy of Pediatrics, the American Academy of Family Physicians, and the American College of Physicians jointly developed a report on transition that includes a set of specific clinical activities recommended for all youth beginning at age 127; yet, national data reveal that most youth do not receive needed transition services.8
A variety of interventions have been used to improve health care transition support, but the evaluation of these interventions is limited. The Agency for Healthcare Research and Quality (AHRQ) recently noted that it is difficult to determine which transition interventions are most effective because of their limited evidence base.9 Yet, there is an ever-increasing demand to demonstrate impact. Demonstrating the impact of transition interventions requires a comprehensive measurement strategy with an underlying framework.
One such framework is the “Triple Aim,” a conceptual model that has been used to determine the impact of the medical home10 and to guide the redesign of pediatric practice.11 Developed by the Institute for Health Care Improvement (IHI), the Triple Aim is organized around 3 interdependent goals: (1) improve the individual experience of health care, (2) improve the health of populations, and (3) reduce the per capita costs of care.12 Starting in 2011 with the enactment of the Affordable Care Act, the Department of Health and Human Services published its National Strategy for Quality Improvement in Health Care by using the Triple Aim to guide development and measurement across the department.13 Framing transition measurement in terms of these 3 linked goals aligns transition interventions with system-wide improvements in health care.
To elucidate the current state of transition measurement, this systematic review identified published measures used to evaluate transition. These measures are categorized within the Triple Aim framework. By understanding how transition interventions have measured their impact on experience of care, population health, and cost, a more unified approach for evaluating health care transition interventions can be developed.
Ovid Medline and the Cumulative Index to Nursing and Allied Health Literature databases were searched to identify English-language articles with abstracts published between January 1995 and November 2013. The search used a combination of medical subject headings and keywords. Search terms were divided into 3 groups: population (eg, adolescent), transition (eg, transition to adult care), and study design (eg, evaluation study). The Boolean phrase “AND” was used between groups and the phrase “OR” was used within groups. The full Medline search strategy is available (Supplemental Table 4). In addition, reference lists of relevant review articles were scanned for applicable studies. All results were compiled and duplicates were removed.
Inclusion and Exclusion Criteria
Several criteria had to be met to be included in this systematic review (Supplemental Table 5). Studies that met inclusion criteria were limited to those describing an intervention for adolescents transferring from pediatric to adult outpatient health care. Studies aimed at patients with and without chronic conditions (physical, mental, and developmental) were included, whereas those looking at dentistry were excluded. Studies that met inclusion criteria needed to evaluate preintervention and postintervention data or compare the intervention group with a comparison group. Both prospective and retrospective data were accepted. Studies that reported exclusively qualitative data or that did not include a sample size were excluded.
Titles and abstracts were screened by 1 reviewer. Only articles that described an outcome of a health care transition intervention were further reviewed. Two reviewers then read the full text of the selected articles and screened them using an inclusion criteria checklist (Supplemental Table 5). A third and fourth reviewer resolved disagreements.
Risk of Bias Assessment
Because the primary interest of this review was identifying measures and not evaluating outcomes, the quality of the articles was not formally assessed. Thus, measures used in a study with a small sample size were considered as important as measures used in a larger study.
One reviewer extracted data from the included studies and another independently confirmed accuracy. For each article, the country where the study was conducted, the medical condition of the study population, and the number of subjects (n) were reported; n is defined as the number of participants for whom outcome data were obtained. Unless noted, n refers to youth or young adults. To add context to the measures, a brief description of the transition intervention and the study type also are reported (Table 1).
Data Synthesis and Analysis
The transition measures in each article were then categorized according to the framework of the “Triple Aim.” To guide the categorization of transition measures within the domains of experience, population health, and cost, IHI’s “Guide to Measuring the ‘Triple Aim,’”14 and AHRQ’s “Early Evidence on the Patient-Centered Medical Home”10 were used.
As shown in Fig 1, each measure was further categorized within the 3 Triple Aim domains consistent with IHI and AHRQ’s example measures. Experience of care measures were split into satisfaction and barriers to care. Population health measures were organized into adherence to care/guidelines, disease-specific outcomes, mortality, patient-reported outcomes (eg, quality of life, functional status), self-care skills, and process of care. Cost measures were classified according to service use, gaps in care (eg, missed appointments, time without a primary care provider), and cost per patient.
Measures or survey instruments developed by study authors, as opposed to validated tools, are noted. Study results, noted as improvement or deterioration in the Triple Aim, also are summarized as reported by the authors of each article. When reporting on utilization, increased outpatient visits were reported as cost-effective and emergency visits and inpatient visits as not cost-effective.
Results of the Literature Search
We identified 2282 articles and reviewed the full text of 104; 33 articles met inclusion criteria and were selected for the review (Fig 2). The characteristics of these articles are presented in Table 1. The transition evaluation measures, categorized within the Triple Aim framework, are presented in Table 2.
The selected articles evaluated the impact of a variety of transition interventions for youth with specific chronic conditions. Study designs were primarily pre-post or retrospective, and sample sizes were typically <100.
Thirteen studies were conducted in the United States and the remaining 18 studies took place in Canada, Europe, or Australia. All 33 studies examined transition in the context of a specific medical condition as opposed to examining transition for all youth with or without chronic conditions. Twelve studies evaluated programs for patient with diabetes mellitus (DM),15–26 4 for transplant recipients,27–30 3 each for those with arthritis31–33 and sickle cell disease,34–36 2 for patients with cystic fibrosis,37,38 and 1 study each for those with cancers,39 congenital adrenal hyperplasia,40 HIV,41 mental health conditions,42 inflammatory bowel disease,43 spina bifida,44 and kidney failure.45 One study looked at a combination of patients with traumatic brain injury, cerebral palsy, spina bifida, and degenerative muscular disorders.46 No study included youth with developmental disabilities, and only 1 examined the impact of the transition intervention on youth with a wide variety of chronic conditions.47
Although several studies reported qualitative data or surveys from parents, only 1 study surveyed parents before and after the transition intervention.33
Sample sizes ranged from 10 to 2502, with nearly a third of the studies enrolling ≤20 in their intervention group. Fourteen studies collected data before and after the intervention, but lacked a comparison group. Most studies with comparison groups used a retrospective design. Only 1 study was a randomized controlled trial.44
Although this review does not focus on the specific components of transition interventions, a brief description of each study can be found in Table 1 to provide a context for the measures that were reviewed. Transition interventions were diverse, and their study methodologies were reported on with varying levels of detail. Several interventions consisted of 1 visit, whereas others took place over the course of multiple years. Staff involvement ranged from a single transition coordinator to joint meetings between pediatric and adult teams. Some studies focused on preparing adolescents to manage their own health care, whereas others sought to improve transfer to adult providers.
Health Care Transition Measures
The measures used to evaluate health care transition interventions were widely variable. All domains of the Triple Aim were examined, although only 3 studies assessed all domains of the Triple Aim (Table 3).15,20,25
Population health measures were used more than twice as frequently as the other 2 Triple Aim domains. In the 27 studies that included these measures, disease-specific outcomes, patient-reported outcomes, and self-care skills were examined most commonly. Measures that were used less frequently included adherence to care, mortality, and process of care.
Disease-specific measures were examined by 13 studies. Because many investigators looked at youth or young adults with DM, hemoglobin A1C (HbA1C) was the disease-specific measure used most commonly.
Patient-reported health outcomes were measured in 12 studies, with quality of life measured most frequently in 5. All 5 of these studies used validated survey instruments. The 36-Item Short Form Health Survey, an instrument that measures social and physical functioning, pain, mental health, vitality, emotional and physical roles, and general perceived health was used in 2 studies.27,39 The EuroQol, which measures mobility, usual activities, pain/discomfort, and anxiety/depression, was used in 1 study.46 Validated disease-specific tools for DM (Diabetes Quality of Life Questionnaire) and arthritis (Juvenile Arthritis Quality of Life Questionnaire), which assess topics similar to the generic instruments but in a disease-specific manner, were used in 2 studies.26,31
Other patient-reported outcome measures examined single topics associated with quality of life by using validated surveys. These included activities of daily living (Barthel Index46), stress (Community Life Skills Scale44 and Perceived Stress Scale46), loneliness (UCLA Loneliness Scale47), anxiety (State/Trait Anxiety Inventory41), psychosocial skills (Personal Adjustment and Role Skills Scale,44 Proactive Attitude Scale,46 and General Self-Efficacy Scale46), disability (London Handicap Scale46), perceived health status (Nottingham Health Profile46), and self-esteem (Rosenberg Self-Esteem Scale46). Two additional patient-reported outcome measures relied on disease-specific instruments that evaluated emotional distress caused by DM (Problem Areas in Diabetes Questionnaire23) and concerns in transitioning to adult sickle cell care (Sickle Cell Transfer Questionnaire36).
The third most commonly used set of population health measures were those that evaluated self-care skills, used in 11 studies. Six of these studies used validated instruments to evaluate changes in self-care abilities or knowledge. Of the validated instruments, several gauged a youth’s general understanding of medical conditions, wellness, emergencies, medications, pharmacies, medical records, and insurance (eg, the Developmentally Based Skills Checklist,27 Denyes Self-Care Practice Instrument,44 Transition Readiness Assessment Questionnaire,47 and Self-Care Inventory R23). As opposed to assessing general self-care skills, 2 validated surveys asked disease-specific questions about sickle cell disease (Sickle Cell Disease Quiz36) and DM (Diabetes Knowledge Questionnaire 226).
Three other studies evaluated self-care skills by using project-specific surveys, rather than validated tools.16,31,41 One survey asked about DM knowledge,16 another about general self-care skills and HIV-specific knowledge,41 and the third about self-medicating, independent visits, and arthritis knowledge.31 The remaining 3 studies that measured self-care skills did not use surveys but rather recorded patients’ abilities to complete a task, such as attending a doctor visit alone,43 carrying a self-monitoring card,32 or self-adjusting an insulin dose.26
Adherence to care was measured in 4 studies. Three studies15,17,34 examined receipt of recommended screenings or medications, and 1 study evaluated transplant recipients’ fluctuations in drug levels.27
Process of care measures were evaluated in only 1 study.32 This study investigated whether the intervention improved documentation of transition-related services in the medical chart, including the age when patients began having independent visits, receipt of a transfer letter by the family, and receipt of records by the future adult provider.
Experience of Care
Of the 7 studies measuring patient satisfaction, 3 studies20,30,43 asked specifically about transition services, whereas 3 studies31,33,38 asked about satisfaction with care in general. One study asked about both areas.15 The studies evaluating satisfaction with transition support most often assessed awareness of transfer policies, appropriateness of transfer timing, and extent of shared decision-making. Three studies assessed overall satisfaction with care. Two of these studies used the same survey instrument and evaluated practice management (eg, availability of appointments), provider characteristics, and clinical processes. This survey also included a parent version, which contained additional questions on availability of parent support. The third study assessing general satisfaction with care did not include the survey instrument in the published article.
The length and scope of survey instruments to measure satisfaction varied widely. Surveys were as simple as 1 question (asking the patient to rate transition as good, sufficient, or bad15) and as complicated as a 22-item questionnaire.31,33 Each study conducted its own project-specific survey as opposed to using an existing validated survey, and only 2 studies31,33 relied on the same instrument, the Mind the Gap Scale, which has been subsequently validated for use in the juvenile idiopathic arthritis population.49
Barriers to care were evaluated in only 1 study.25 Patients were surveyed about their difficulties finding providers, scheduling appointments, establishing relationships with an adult team, and feeling comfortable in a new health care system.
Cost measures, most often reported in terms of service utilization, were used in 15 of the 33 studies. Fewer studies examined gaps in care or per capita costs.
Cost was evaluated in 3 studies. Two studies estimated the total costs of staff and services devoted to transition services,29,46 whereas the third study34 measured cost per capita for those with and without transition support.
This systematic review reveals that transition intervention studies measure impact with little consistency and without a common measurement framework to comparatively judge transition outcomes. The Triple Aim provides an explicit conceptual model. However, most studies examine only 1 domain of the Triple Aim and are seldom using the same measure. Moreover, except for those evaluating quality of life and self-care skills, few instruments have been validated. The transition field of research is in its early stages of development, and new efforts are needed to establish a consensus on a core set of measures aligned with the Triple Aim framework.
When establishing transition measures, 2 considerations are important. Evaluation strategies should include measures that are, first, applicable to all youth, and, second, reasonably attributable to transition interventions. The American Academy of Pediatrics/American Academy of Family Physicians/American College of Physicians Clinical Report7 emphasizes the importance of transition services for all adolescents, not just those with special health care needs. It is therefore important to establish a core set of measures that can be used to assess youth with and without chronic conditions. Although disease-specific measures may be of value, they cannot be used for comparison across diverse populations. Further, several measures identified by this review pertain to care in general, as opposed to transition specifically. Quality of life or satisfaction may be more appropriate to evaluate overall health care quality rather than the impact of a transition intervention. Measures that meet these 2 criteria (applicable to all youth and attributable to transition) are discussed later in this article, with recognition that they may be best used in conjunction with other measures tailored to specific diseases or research questions.
Population health evaluation strategies applicable to all youth and attributable to transition include self-care management and process of care measures. Several validated self-care assessment tools, such as the Transition Readiness Assessment Questionnaire (TRAQ)50 and the TRxANSITION Scale,51 assess a wide array of skills relevant to youth with and without chronic conditions. Process-of-care measures also can be used to evaluate transition interventions targeted at a broad population. The paucity of process measures found in this systematic review is likely because many studies were published before the release of the transition recommendations specified in the 2011 Clinical Report.7 The necessary components of transition delineated in this report were further translated into clinical tools referred to as the Six Core Elements, which include a transition policy, a method of tracking patients, a self-care assessment, a transition plan, a transfer protocol, and a method to elicit consumer feedback.52 A newly developed process measure, the Health Care Transition Process Measurement Tool, is available to monitor the implementation of these elements and can be used to evaluate the receipt of transition services by all youth.52 This tool has yet to be validated.
With respect to experience of care, several surveys are transition-specific. The 2009–2010 National Survey of Children with Special Health Care Needs asked parents 4 questions about receipt of transition assistance8; however, it is currently undergoing significant revisions. In addition, AHRQ has funded Boston Children’s Hospital to develop and test an adolescent self-reported quality measure. Finally, as part of the updated Six Core Elements, consumer transition feedback surveys were developed for use with youth, parents/caregivers, and young adults, although these instruments have not yet been validated.52
Cost indicators also can be specific to transition while remaining applicable to all youth. One such indicator is the use of primary care services by young adult patients. By measuring utilization longitudinally, researchers can correlate transition processes with eventual cost outcomes into adulthood.
Achieving consensus on a core set of transition measures will require a careful, concerted effort involving clinical, consumer, and research expertise. To facilitate this process, AHRQ has defined specific attributes important for quality measurement.53 These include importance of the measure, clinical logic, measure properties, and feasibility. Critical examination of each of these attributes will be important in developing a consensus. Several current efforts are noteworthy. AHRQ has commissioned a technical brief to describe current transition practice and outcomes to provide a framework for future research.9 Additionally, The Healthcare Transition Research Consortium has initiated a Delphi process with consumers and providers to elicit potential health care transition measures.54
Once a core set of transition measures has been developed, sharing these with clinicians and researchers is essential. Practice-based research networks are a proven method for evaluating improvement in primary care.55 Expanding funding support for transition research networks will allow measurement strategies to be implemented and tested among a broader group of youth. Building on the investment in evaluation that the federal government, private foundations, and health plans have made in the areas of medical home and transition from hospital to community-based settings, the field of pediatric to adult health care transition could benefit from similar leadership and support.
Health care transition evaluations assess Triple Aim impacts, but do so with little consistency. Seldom do transition interventions measure all 3 domains of the Triple Aim. A unified approach for evaluating health care transition should include broadly applicable measures that evaluate patient and family experience, population health, and cost. By evaluating the Triple Aim, successful health care transition interventions can be identified and implemented to benefit all youth.
The authors are grateful to Katherine Rogers for her assistance reviewing methods and Daniel Beck for his assistance screening abstracts and revising the manuscript. We are also appreciative of the support of our project officer Marie Mann.
- Accepted September 24, 2014.
- Address correspondence to Megan Prior, BA, The National Alliance to Advance Adolescent Health, 1015 18th St, NW, Ste 300, Washington, DC 20036. E-mail:
Ms Prior developed the review protocol, completed the data collection, carried out the initial analyses, drafted the initial manuscript, and reviewed and revised the manuscript; Ms McManus conceptualized the study, resolved data collection discrepancies, reviewed the initial analyses, and revised the manuscript; Dr White resolved data collection discrepancies and revised the manuscript; Ms Davidson developed the review protocol, designed the search strategy, drafted the literature review methods, and reviewed the final manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: This study was supported by cooperative agreement U39MC25729 HRSA/MCHB (“Transition Services in Adolescent Health—Healthy and Ready to Work”). The authors of this article are responsible for its contents. No statement in this article should be construed as the official position of the Health Resources and Services Administration or the Maternal and Child Health Bureau.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Copyright © 2014 by the American Academy of Pediatrics