Abstract
OBJECTIVES: To review the role of theory as a means to enhance the practice of quality improvement (QI) research and to propose a novel conceptual model focused on the operations of health care.
METHODS: Conceptual model, informed by literature review.
RESULTS: To optimize learning across QI studies requires the integration of small-scale theories (middle-range theories, theories of change) within the context of larger unifying theories. We propose that health care QI research would benefit from a theory that describes the operations of health care delivery, including the multiplicity of roles that interpersonal interactions play. The broadest constructs of the model are entry into the system, and assessment and management of the patient, with the subordinate operations of access; recognition, assessment, and diagnosis; and medical decision-making (developing a plan), coordination of care, execution of care, referral and reassessment, respectively. Interpersonal aspects of care recognize the patient/caregiver as a source of information, an individual in a cultural context, a complex human being, and a partner in their care. Impacts to any and all of these roles may impact the quality of care.
CONCLUSIONS: Such a theory can promote opportunities for moving the field forward and organizing the planning and interpretation of comparable studies. The articulation of such a theory may simultaneously provide guidance for the QI researcher and an opportunity for refinement and improvement.
- quality of health care (measurement)
- research methods
- applied research
- quality improvement (research and practice)
- health services research
- Donabedian
- Deming
- QI —
- quality improvement
There exists an explicit imperative to improve the quality of health care delivered to the American public. Two remarkable Institute of Medicine reports, To Err is Human and Crossing the Quality Chasm, codified the foundations for such work as they chronicled 2 decades of research characterizing pervasive deficits in quality.1,2 Subsequently, 9 years of reports ordered by the US Congress document large and persistent problems in quality, including disparities in quality by race, ethnicity, and income.3 The nation is coming to agreement on a number of aims for health care, including health care for children. The overarching aims, as first articulated by Donald Berwick and colleagues,4 and now adopted by the federal government, are better care, better health, and lower cost. Better care is defined as health care that is safe, timely, effective, efficient, equitable, and patient-centered.5 In sum, the Institute of Medicine and US Department of Health and Human Services reports articulate national goals. Recent legislation, including the Children’s Health Insurance Program Reauthorization Act and the Affordable Care Act, signal a new day for the study and improvement of quality. There is much activity addressing quality within and around American health care in both the public and private sectors.6–14 For children, this is most evident in the collaboration between the Centers for Medicare and Medicaid Services and Agency for Healthcare Research and Quality to develop and guide 7 Centers of Excellence and 2 state Medicaid programs in the Pediatric Quality Measures Program.
A September 2012 Institute of Medicine report summarized a flurry of quality improvement (QI) activities and pointed to new directions aimed at making the US health care delivery system more of a “learning organization.”15 The report may be summarized as articulating the perspective that, “Technology can fix what's broken in health care.”16 This optimism is consistent with urgent calls for action that fall short of empirical guidance or even a clear path to improvement. One potential path to bring action together with learning can be found in the form of applied social science.17 As specifically applied to the enhancement of performance, “improvement science” has been described as the systematic effort to find “out how to improve and make changes in the most effective way... examining the methods and factors that best work to facilitate quality improvement.”14 How to move QI forward is an ongoing subject of discussion.18–30
This article argues for the centrality of science, including rigorous theory development and testing, in moving the nation’s quality aims forward. In part, we build off of work we presented at the first 2 “Advancing the Science of Pediatric Quality Improvement Research” conferences in 2011 and 2012.31–33 We identify gaps in the current theory and practice of QI research and evaluation in health care and to suggest approaches to closing those gaps, focusing on pediatric QI as appropriate. We suggest specific designs should be matched to specific circumstances for considering health care improvement and its evaluation. Finally, we suggest how practical experience can help to build a theory of applied QI in health care that can help researchers to identify generalizable findings from QI evaluation and research, and help the nation achieve its goals.
Quality Improvement
A large stream of QI activities in health care are derived from the philosophy of total quality management and the work of Edwards Deming and Walter Shewhart.34 The modern era of QI in health care began with efforts to stimulate providers’ use of clinical practice guidelines or to help providers meet performance goals, benchmarked against data from performance measures.35–37 This era began after it became clear that merely producing clinical practice guidelines and passively disseminating them was not leading to change. Although there are some compelling examples of successful QI efforts focused on specific clinical problems at the clinical microsystem level,15,38 the current emphasis in QI is on broader systemic efforts designed to change the environment in which providers practice. This emphasis includes concepts such as systems reengineering,39 delivery system transformation, 6 σ, LEAN, and systems change.23,26 We consider QI broadly to consist of those systematic data-based activities that focus on changing the production function of health care with an aim toward improving outcomes and/or efficiency. Although the fullness of QI may be manifest in iterative cycles of measurement and improvement, we agree with Berwick18,40 that the aims are more important than the purity of the term; the empirical emergence of evidence and understanding creates opportunities to iteratively posit and test conceptual models and theories. His other point in 1996, that we need to broaden our ideas of what constitutes good science, is as relevant today as then.
In 2003, Galvin and McGlynn41 considered 3 social science models, Change Theory, Tipping Point, and Diffusion Theory, to explore why QI had not resulted in a fundamentally improved health care system. They recommended a strategy that used performance measurement to drive change. Ten years later, observers are still asking why health care quality has largely not improved,34 and attempting to develop frameworks to help guide improvement.42
A Science of QI: Theory for Practice
We are not the first to suggest the importance of science and theory in QI.43–45 The journal Implementation Science, which publishes much of the research in QI, now recommends that manuscript submitters articulate their theory of change.46 Duncan Neuhauser, a distinguished expert in medicine and health care organization, recently summarized how theory development fits with the epistemology of knowledge and the scientific method, as well as the importance of theory in QI.47 He reminds us that “the scientific method consists of theory development, making predictions (hypotheses), hypothesis testing, theory revision…” (p. 466), and notes that: “Theory is essential because it allows us to generalize to other places, times and circumstances…Theory is the way we learn from each other…” (p. 466). This approach is a clear descendant of Merton’s seminal vision of middle range theory.48
Among characteristics of good QI theory that Neuhauser and his colleagues suggest is the specification of variables and relationships among variables, use of outcome measures related to quality, numerical measurement of variables, and hypothesis testing (see Appendix). Although there may be disagreement about the defining characteristics of good theory in general, even those suggested by Neuhauser are almost never found in QI work, no less in QI research publications in the scientific literature.39,45,49–51 Thus, much of QI is a “black box,”52 providing little guidance to those eager to take a more informed approach to improving care, and casting doubt on its value as a science. Understanding substantive interactions between specified aspects of interaction and identifiable attributes of context may be critical to support intelligent generalization from one QI project to the next, another reason to integrate theory into this work.53
The imperative to develop and test theory in QI may sound daunting to those who have not been trained in the social and behavioral sciences, where theory development and hypothesis testing are mundane. The word should not intimidate: theory may be a well-specified conceptual model that predicts and explains what the intervention is, what need it is filling, what is supposed to happen as a result (the outcome), how and why the results happened on average, and, optimally, reasons for variations across sites in a multisite strategy. Evaluation scientists might label such a theory a logic model with a theory of change. Many theories are available to use as starting points in QI, or they can be combined in complementary ways. The important point is to carefully think through (and specify ahead of time) the relationships among variables, such as “A leads to B, controlling for (or accounting for) C….” 43,54,55
Research Methods for QI
Randomized controlled trials with the patient as the unit of intervention are often not optimal for health care QI studies. The unit of intervention is typically the organization (eg, clinical practice, health plan, state), although it may be the individual provider. Cluster randomized controlled trials would be optimal, and have been conducted, primarily in other countries.56–60 In the United States, randomization of providers, plans, and other QI locations is almost unheard of.61–63 Rigorous quasi-experiments or observational studies (eg, interrupted time-series, regression discontinuity), preferably using comparison groups, have been recommended,64 and will likely increase as a result of recent efforts to train more researchers in their use. Even the use of contemporaneous nonequivalent control group designs would represent significant improvement over the dominant use of post-only or pre-post studies without control groups.
The understanding of context on the results of QI interventions is essential to building this science and making it useful to frontline implementers. As social psychologists have demonstrated for more than 60 years, behavior (and outcomes) is a function of the person (eg, the provider) in his or her environment. Although this premise is now accepted in the QI world, a true understanding of context, and an ability to test the impact of contextual variables on improvement goals, is not likely to occur without development and more common use of standardized definitions and instruments. For example, the following are among the most common features hypothesized (mostly post hoc) to account for variation in the results of performance improvement strategies: leadership, quality of team work, collective mindfulness,65 resources, organizational cultures, nature of hierarchies, and communications styles. These are often taken as an article of faith but the evidence base for them is highly variable, in part because they are either undefined or defined differently in every study. Health care experts have been skeptical about the value of using concepts and instruments from other fields; nonetheless, starting points outside of the health care context can fruitfully guide the development of health care–specific tools. Once the measures are defined, they can then be used to study the heterogeneity of effects in QI.66 Using a biological metaphor, interventions have active sites and organizations have receptors and a mismatch therein can cause an otherwise effective intervention to fail in a specific context. Moreover, the effectiveness of an intervention may depend on the need manifest in the organizational context: immunizing already protected individuals will confer no additional benefit. Needs analyses may identify gaps in resources or opportunities to improve processes that may be turned into targets for directed QI initiatives. We must develop ways to deal productively in contexts replete with uncertainties.
Improving the science of improvement will take concerted effort and increased thoughtfulness in the clinical and health services research communities. Some entities have begun this process, including in the United States, the Academic Pediatric Association, the Academy for Healthcare Improvement, and the Improvement Science Research Network. In the United Kingdom, the new Health Foundation is taking major steps to build the science and contribute to achieving better care, better health, and lower costs. Others have suggested that a new international entity be formed to accelerate progress; reactions to this idea have been mixed, perhaps because new organization may be seen to threaten rather than complement existing ones.67
Still others are impatient with calls to improve and use improvement science, understandably focusing on an urgent call for action. However, action and theory are not incompatible: as it has been said, there is nothing so practical as a good theory.68 Indeed, “action research” is defined as “comparative research on the conditions and effects of various forms of social action and research leading to social action” using iterative cycles of planning, action, and measurement.69 Action research may be valuable in health care QI and other systems change research and offers significant promise as 1 foundation for intelligent change.70–73
Moving Forward: Toward a Model for Quality Improvement
We suggest a need to progress on 2 intersecting fronts: empirical work needs to document and improve care in real time; and stakeholder engagement to develop a meaning-based understanding of health care.
Structure, process, and outcome have been central to the theory of health care quality since Donabedian articulated an architecture for measuring health care quality and laid the foundation for the recent explosion of work in the field. We call on the development of new theory to complement Donabedian’s work; to advance from his strict focus on measurement and toward health care operations and improvement. By focusing on the tasks of health care in context, we may come to learn about what makes a difference in health and health care. An operations-based model supports QI activities as it elicits relationships between root causes and outcomes.
Such a model could then generate middle-range theories in the form of theories of change. These theories could be used to drive the design and implementation of interventions designed to improve quality. Such theories would be testable and support the modification of the overall theory.
One can easily imagine how the development of such theory could be leveraged to make the Plan-Do-Check-Act cycles of QI both more efficient (as it can drive the Plan and Do components) and more informative. Such a theory has the potential to simultaneously limit degrees of freedom in QI activity and challenge those working in the field to link their work in individual institutions to ideas that transcend such boundaries.74,75 This theory of practice could suggest a taxonomy of the work of health care in relatively fundamental terms that describe constructs that can both be measured and modified. Such a model could both support and rely on theories of change to advance an epistemology of QI appropriate to the health care context. An operations-based model of quality may challenge those of us struggling to improve quality to refine and develop what is proposed here in a way that enhances our ability to understand and effect the production of quality health care. We set a toe in the water here.
Current Theory
According to Donabedian, the structure of care relates to the organization of health care delivery, whether at the level of the health care system, the organizational or corporate unit, or the individual practice.70,71 An example of a structural defect in health care is the lack of accessibility of care to those who are unable to afford it. On the micro level, the inability of a patient to access a physician’s office that was not wheelchair accessible also would represent a structural impediment. Similarly, the organization of quality measurement, data infrastructure, and the nature of quality assessment or improvement activities are all structural aspects of care. The process of care includes what is done and not done to or for the patient or by members of the health care system. Traditionally this was divided into 2 categories: interpersonal and technical. The interpersonal aspects of process related to how a patient was treated on a human level: was the encounter with the health care system characterized by respect for the patient, his or her needs, desires, and privacy, and so forth. The technical aspects of health care can further be divided into medical decision-making and technical skill in implementing medical decisions. Brook and colleagues introduced the construct of appropriateness and necessity of care, based on the relative balance of risk, benefits, and importance.76–80 The recent field of patient safety has emerged to focus on the technical quality of care, specifically related to medical errors.23,81,82 The failure to deliver a planned or necessary service represents another type of medical error, as does failure to notify or provide appropriate follow-up, as might occur after an abnormal cervical cancer screen is victimized by a medical error.
It is axiomatic that, in aggregate, health care impacts health, that its processes variably impact outcomes. Process is inherently more sensitive to measure change than are outcomes. Process measurement suffers in practice because of uncertainty in the link between specific processes and specific outcomes. Uncertainty may reflect the actual relationship between process and outcomes, the measurement challenge, or the research effort (eg, far more research links processes to outcomes in adults than in children).83
Outcomes of health care are defined as what actually becomes of the patient and may represent ultimate consequences: the degree of health or illness or mortality or intermediate aspects of care such as the need for an emergency department visit. In general, the better the evidence linking the intermediate outcome to the ultimate outcomes (or the more important the intermediate outcome is considered in and of itself), the more confident one can be using intermediate outcomes to describe the quality of care.
Because outcomes are the most visible and intuitive component of quality, there has been a large move toward emphasizing outcomes in policy decisions. When doing so, one must recognize that because of the resiliency of individuals and populations, and because of the indefinite relationship between processes and outcomes, outcomes are an insensitive way to measure changes in the quality of care. For example, only rarely does a child who is not fully immunized suffer from a vaccine-preventable illness, yet the failure to fully immunize each child represents a decrement in the quality of care. This problem is amplified when outcomes are uncommon.
Operations-based Model
The improvement of health care quality is a dynamic process and QI is just now being defined rigorously in the literature.84 We propose that the work of health care can be defined in an operational and interpersonal framework to underlie QI research in much the same way that Donabedian defines quality research. Figure 1 illustrates that either through the action of an individual patient or the health care system, there emerges an awareness of a need or opportunity for health care. Subsequent to such awareness, access is or is not achieved. Once care is accessed, there comes an assessment of the extent to which there is a concern, a need, or an opportunity. Such conditions result, for example, from a symptom (CONCERN), a treatable illness (NEED), or a circumstance (eg, underimmunized 15-month-old child representing an OPPORTUNITY). The concern, need, or opportunity then needs to be assessed and defined by the clinician or health care system (eg, the provision of visit reminders for preventive health care in a proactive managed care plan). A preliminary assessment is confirmed by history or physical and a diagnosis may be made (the “diagnosis” in this respect may represent a confirmation of the need to undertake further evaluation). A further care management plan may be negotiated and implemented, with coordination and execution of processes of care. We note that this care management may be for further diagnostic work, treatment, follow-up, or some combination of those. At times, referral might represent an auxiliary or a new main path, reinitiating these constructs. With new information comes the need to reassess; recognition of a concern, need, or opportunity, or potentially the firm recognition of the identification of actionable information (even the recognition of uncertainty that requires action) may restart the cycle. In this framework, diagnosis of a state occurs when the recognition becomes actionable. Actions may represent large activities or smaller iterative data-gathering ones. The broadest constructs of the model are entry, assessment, and management, with the subordinate operations of access; recognition, assessment, and diagnosis; and medical decision-making (developing a plan), coordination of care, execution of care, referral, and reassessment, respectively.
An operational approach to organizing health care. Major tasks include entry into the health care system, assessment of needs and opportunities (including diagnosis), and management, including medical decision-making, execution and coordination, and follow-up.
Figure 2 breaks down the various interpersonal components involved in providing health care. Interpersonal tasks include eliciting relevant clinical information from the patient or family, individualizing care plans with respect for patient values and recognizing life contexts (operationalized by many in the construct of shared decision-making), developing a relationship that optimizes the healing power of the therapeutic interpersonal relationship, and developing a partnership with the patient that reinforces other aspects of respectful behavior in a way that can be operationalized as a therapeutic partnership.
Characterizing the various roles that patients/parents/caregivers may play that are developed through the interpersonal aspects of health care delivery. The task of the clinician/health care system is to cultivate each role in appropriate measure.
We articulate this model as a straw person to initiate specific discussion regarding how to reframe quality not only as an object for measurement, as was done so ably by Donabedian, but as a target for improvement. We present this theory too as an illustration of the potential to articulate paradigmatic theories that are overarching to the middle-range theories described in this article.
Conclusions
In some ways, growth and development in pediatrics provides an apt metaphor for QI research. There is movement through time and change, some intended, some not, and some inevitable despite all attempts to steer it otherwise. These are systems beyond our control, but not beyond our direction. Understanding how to increase the desired outcome when raising a child requires some understanding of how children grow, what influences them, and where they are heading. It requires a theory that accounts for the dynamic nature of being a child in a family in a community and the multitude of potential influencers. Then a focus on those influences seen as key may help a parent to raise a healthy, well-adjusted, and productive young adult.
QI research in general and particularly that for children requires an appreciation of the impact and changes over time, in terms of the structures, processes, and outcomes of care in the immediate future and across the life course. Evidence is a critical foundation for such work, but so are reasonable theories, articulated and specified for testing.
We suggest that the task of researchers working to build a thriving field of QI research includes the following:
Developing and extending the theory of health care operations and delivery through systematic inquiry that incorporates both quantitative and qualitative methods. An ideal theory will allow for the assessment of care both to an individual patient and to populations. The Institute of Medicine suggests 7 attributes of quality, including equity, that should be accounted for at the population level.
Integrating theory and empirical findings to identify key variables that affect the quality of health care. We expect that some of these variables will be clinical process variables, others structural variables (which may be as varied as systems for ongoing training, organizational attributes, and access to technology), and operational process variables. As theory identifies a causal chain, some structural and process variables may emerge as sufficiently important to serve as outcomes in their own right or as proxies for clinical or other outcomes. Longer-term clinical outcomes should be identified and may be monitored for trends across the body of change that is accomplished.
Develop valid and reliable measures that permit the assessment of key variables in a variety of settings.
Develop and assess tools and methods that improve the ability of researchers and evaluators of QI work to grow the field. An improved understanding of the epistemology of QI research and evaluation would be a critical component to both guide and assess their development.
Develop and evaluate policies that promote the conversion of information and understanding about health care QI from a private good to a public good.85,86 Although there has been a good deal of attention paid to the “business case for quality,” much less attention has been paid to how to develop sufficient value for private health care organizations that share rather than hoard information, data, and understanding. Public funds should support only QI work that is designed to provide information beyond understanding, whether a particular intervention worked in a particular organizational setting.
Develop and evaluate interventions that social science, organizational, or other theory suggest may hold promise to improve the quality of care. In developing an understanding of the impact of interventions, it will be important to assess attribution as well as association. Excellent design, triangulation through the use of complementary designs, and careful analysis, including the thoughtful incorporation of Bradford Hill’s classic principles may guide us in developing an understanding of the cause and effect of QI interventions. Attention to context may require attention to closing gaps (additive models), changing processes (multiplicative models), or both.
We have articulated a theory of quality as a target for improvement that describes clinical and interpersonal tasks that constitute our initial framing of the work of health care. These tasks occur within clinical, organizational, and community contexts that may require recognition of the “receptor sites” that define how those contexts will modify the meaning or measurement of these various operations in that context. Recognizing the active components of context on QI and how they interact with these operations will represent a fundamental task for the QI researcher. Nonetheless, by focusing on potentially observable operations of health care, this or similar frameworks offer the potential to build on the Donabedian framework to support and extend the conceptual model from the more static construct of quality measurement to the more dynamic construct of QI research.
Quality and QI research require the collaboration of excellent listeners, sophisticated methodologists, and conceptual and analytic thinkers, all coming together with open minds that are informed but not bound by theory. In this effort, the need to create a paradigm for that incorporates the benefit of multiple disciplines in pursuit of better health care can be achieved. When considering the quality of health care for children, additional complexities are added, as is the opportunity to incorporate a fundamental appreciation for the importance of the life course for children’s total health. In this effort, the need to create a paradigm for that incorporates the benefit of multiple disciplines in pursuit of better health care can be achieved.
Appendix
The variables and their relationships are clearly defined.
The resultant model or theory is generalizable to other organizations.
The theory leads to predictions that can potentially be disproved.
The theory leads to the possibility of replication.
The theory simplifies reality. Simplicity is a virtue.
The theory predicts better than other theories.
Numerical measurement of variables and their relationships are not required, but they are useful, owing to their preciseness.
The unit of analysis is defined. In [quality improvement], the unit of analysis is [often] the organization [or the provider]. In clinical research, the unit of analysis is often the patient. This has important research consequences.
The dependent outcome variable is related to quality, such as fewer errors, lower mortality, lower costs, and patient satisfaction. “This outcome variable is what makes it a theory of health care quality improvement.” [emphasis added].43(pp 466-467)
Footnotes
- Accepted December 20, 2012.
- Address correspondence to Lawrence C. Kleinman, MD, MPH, FAAP, Mount Sinai School of Medicine, Department of Health Evidence and Policy, One Gustave Levy Place, Box 1077, New York, NY 10029. E-mail: lawrence.kleinman{at}mountsinai.org
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
References
- Copyright © 2013 by the American Academy of Pediatrics
