Evaluating Implementation of the Updated Care Considerations for Duchenne Muscular Dystrophy
Care Considerations for Duchenne Muscular Dystrophy were published in 2010. However, little is known about the extent to which these considerations were implemented after publication. With this article, we provide direction on evaluating the uptake of the 2018 Duchenne Muscular Dystrophy Care Considerations. We identify key elements of care and present suggestions for their use in evaluation and research.
- CDC —
- Centers for Disease Control and Prevention
- CDCC —
- Certified Duchenne Care Center
- DMD —
- Duchenne muscular dystrophy
- MDA —
- Muscular Dystrophy Association
- MD STARnet —
- Muscular Dystrophy Surveillance and Tracking Network
- PPMD —
- Parent Project Muscular Dystrophy
The need for consistent, comprehensive, and standardized management of Duchenne muscular dystrophy (DMD) led to the publication of the 2010 DMD Care Considerations in Lancet Neurology, which was sponsored by the Centers for Disease Control and Prevention (CDC).1,2 After publication, advocacy groups disseminated the document through several means. First, Translational Research in Europe–Assessment and Treatment of Neuromuscular Diseases (TREAT-NMD) and Parent Project Muscular Dystrophy (PPMD) created a booklet, distilling the care considerations for families and caregivers.3 Second, PPMD, TREAT-NMD, and United Parent Projects Muscular Dystrophy distilled the contents into a 1-page summary, the “Imperatives for DUCHENNE MD.”4 Third, PPMD developed the Certified Duchenne Care Center (CDCC) program to operationalize the 2010 DMD Care Considerations and certify neuromuscular centers in the United States capable of delivering the comprehensive Duchenne care outlined in the 2010 DMD Care Considerations.5 Fourth, the PPMD DuchenneConnect registry, a patient-reported outcomes registry that includes data from over 4000 families, began collecting data reflecting care and services received by families from the United States and around the world. Fifth, the Muscular Dystrophy Association (MDA) launched the US Neuromuscular Disease Registry, a pilot program used to collect a wide range of provider-entered clinical data from individuals with DMD and other neuromuscular conditions who receive care at MDA Care Centers.6 MDA also launched a recertification process for all MDA care centers, which are expected to adhere to standards of care for neuromuscular disease.
Although these efforts enhanced dissemination of the care considerations to clinicians and families, little information was gathered about how well health care providers implemented the care considerations or about the impact on patient health outcomes. The knowledge is limited because no a priori organized implementation and evaluation plan was established for the 2010 DMD Care Considerations to encourage evaluations. In addition, distal patient outcomes in this population are challenging to assess.
The 3-part 2018 DMD Care Considerations, sponsored by the CDC,7–9 includes 3 new topic areas (bone health, primary and emergency care, and transitions) in addition to the 8 sections from the 2010 DMD Care Considerations, which have been updated to reflect recent clinical care best practices for DMD. The update reiterates the multidisciplinary focus of the 2010 Care Considerations and encourages anticipatory management to mitigate likely complications. New approaches for diagnostics and therapeutic interventions were added to the sections. Here, we suggest outcomes for evaluating implementation of the 2018 Care Considerations and propose methods for evaluation.
Evaluation of the First Care Considerations
Implementation is a specific set of activities intended to put a program into practice (specifically, a program that is institutionally sanctioned, formally defined, consciously planned, and intended to lead to a changed outcome).10–13 Assessments of the implementation of the 2010 Care Considerations have generally been limited to the frequency of clinic visits and specialist visits14–16 as opposed to the evaluation of the care implemented and its quality. Landfeldt et al14 examined whether patient care was consistent with the 2010 DMD Care Considerations in Germany, Italy, the United Kingdom, and the United States. Their findings from an online survey of patients and caregivers indicated that fidelity to the 2010 DMD Care Considerations varied based on country and by specialist. The authors found that a less-than-optimal proportion of patients had the recommended frequency of visits to specialists. For example, 48% to 80% of patients followed the recommended twice-yearly number of visits to a physiotherapist. For neuromuscular specialist visits, 27% to 72% followed the recommended frequency of visits. Some patients had poor access to orthotic devices.
In another evaluation of clinic visits, Andrews et al17 abstracted the medical records of 299 patients from various Muscular Dystrophy Surveillance and Tracking Network (MD STARnet) sites in Arizona, Colorado, Iowa, and Western New York.18 They examined frequency of visits to health providers during a three year period to evaluate the degree to which visits aligned with the 2010 DMD Care Considerations. The authors found that 97% of patients visited a neuromuscular provider at least once, and most (64%) met the semiannual visits suggested in the 2010 DMD Care Considerations.15 However, the authors found that frequency of visits to other specialists varied. For example, just 20% of the patients saw an endocrinologist and only 40% visited an orthopedist in the three year period examined.
An evaluation of the frequency of clinic visits can be a good indicator of whether patients are receiving care from the right specialists at the right time, but it does not reveal whether patients are receiving care that follows best practices. An assessment of greater depth would be necessary to assess quality of care. For example, Conway et al19 surveyed 6 clinic directors from MDA-supported clinics within MD STARnet to inquire about the management of care and therapies for muscular dystrophy at their clinic. The authors found some adherence to the 2010 DMD Care Considerations. For instance, as suggested, all clinics offered genetic testing, genetic counseling, and cardiac monitoring.16 However, they also found that less than half of the surveyed clinics followed considerations for neuropsychological interventions. In light of these findings, there was room for improvement with regard to adherence to the 2010 DMD Care Considerations.
The 2010 DMD Care Considerations were difficult to evaluate in part because the most critical elements were not marked as such. In the 2018 update, key elements were identified (see Figs 1–5 in part 1 of the updates for a summary7). Here, we operationalize the 2018 DMD Care Considerations to improve stakeholders’ capacity for measuring adherence and suggest ways to use the elements for evaluation and research.
The Complexity of Implementation
Our purpose in operationalizing the 2018 DMD Care Considerations by defining key considerations in measurable terms is to facilitate the evaluation of implementation, rather than provide a prescriptive strategic plan for implementation or delivery. Figures 1–5 contain the key considerations that are useful for evaluating implementation outcomes.
An understanding of implementation science is helpful in recognizing the situations in which the elements listed in Figs 1–5 may be useful. Proctor et al20 and others21 who are interested in the measurement of quality of care, distinguished implementation outcomes from service (eg, efficiency, safety, effectiveness) and patient (eg, functioning and symptomatology) outcomes. Implementation outcomes refer to the effect of conducting a new program and are measured by success in program delivery, which are accounted for by various outcomes, including acceptability, adoption, and fidelity.20 Acceptability measures the degree to which stakeholders agree or are satisfied with a program.18 The extent to which patients, their family members, and health providers agree with the considerations for care is 1 way for evaluators to assess acceptability. A high level of acceptance among stakeholders increases the likelihood of adoption. Adoption is the decision or action taken to conduct an innovation or evidence-based practice. In the case of the 2018 DMD Care Considerations, adoption could be measured by health professionals’ choice to follow them or by health institutions’ decision to alter procedures and policies to conform to them. Fidelity, or adherence, is the degree to which a program is implemented as intended by its developers.20 Health providers and institutions could choose to adopt some care considerations and not others. Appropriateness or relevance for the setting, implementation cost, feasibility or likelihood of successful implementation, penetration or coverage, and sustainability are other ways that Proctor et al20 suggest assessing implementation outcomes. We mainly address acceptability, adoption, and fidelity because these are the most relevant outcomes of implementation in the early stage of program delivery, the most practically useful, and the easiest to assess compared with other categories of implementation outcomes. Figures 1–5 is intended for use in evaluating these 3 outcomes of implementation of care considerations.
Implementation can be an end goal or a process that is intermediate to distal patient outcome goals. Stakeholders can use the key care considerations detailed here as outcomes for measuring implementation adherence or as intermediate outcomes for a program to improve the health and well-being of people with DMD.
Other evaluation issues may be considered when measuring implementation success. Proctor et al12,20 suggest that when an intervention or innovation fails, it is helpful to distinguish between treatment ineffectiveness and implementation strategy failure. An intervention will not improve patient outcomes when delivered inappropriately, nor will the intervention be effective even when delivered appropriately if it is not a good fit for the setting in which it is delivered. Context affects implementation fidelity, and it refers to the ecology of the setting in which the program is implemented. There are a multitude of contextual factors that account for successful implementation. These factors differ across sites such that different settings have varying capacity to succeed in implementing the care considerations. Adherence is influenced by the complexity of the intervention, the quality of strategies to deliver the intervention, and participants’ reception to the intervention.22 For care considerations, variables that are likely to influence the success of implementation include health providers’ attitudes toward the content, the capacity of the setting to conduct the recommended testing and procedures, the clarity and adequacy of training to conduct them, and leaders’ commitment to implementation. Financial constraints and staffing limitations are pragmatic concerns that are likely to influence adoption of and fidelity to the care considerations.
Some topic areas in the 2018 DMD Care Considerations are more specific than others, and typically those that are specific are more likely to be followed.23 The DMD Care Considerations are complex, and this may be a challenge for adoption and adherence. Malleability of intervention components and the degree of autonomy, discretion, or flexibility of users’ environments can also affect the success of an intervention.13 Flexibility in implementation as well as discretion and choice for health care professionals and individuals with DMD lead to successful implementation of the 2018 DMD Care Considerations. Ideally, health care providers will have the freedom to manage patient care as recommended in the 2018 DMD Care Considerations, and patients and families will have the means to follow them. Other factors, like readiness for change, organizational culture, leadership support, and resource availability, can facilitate or hinder implementation.22,24
Development of Care Considerations Evaluation
The care elements for evaluation are designed to provide a common set of criteria for implementing the 2018 Care Considerations in a variety of settings. An evaluation committee of staff from CDC, PPMD, MDA, and the MD STARnet project took the complete 2018 Care Considerations and identified and organized the patient outcome goals for each category of care (Table 1). Evaluability, selection of key elements, potential data sources for evaluation, methods for evaluation, and uses for the identified elements were discussed. In addition to the first papers of the 2018 Care Considerations,7–9 the committee examined other sources to operationalize the key elements (Figs 1–5), including the tools used for CDCC certification,5 the data elements from MDA’s registry,25 and the indicators used for evaluating the 2010 Care Considerations by using MD STARnet data.15,16 When the evaluation committee had agreed on the set of measurable care elements, the committee chair of each care domain vetted and approved them. The committee chairs of each care domain were the lead authors of their sections of the 2018 Care Considerations.
Despite the differences in care domains that stakeholders might choose to emphasize when adopting the care considerations, the goals for implementation (ie, providing elements of DMD care consistent with the 2018 DMD Care Considerations) should nevertheless remain the same. For example, 1 clinic may have the capability to diagnose DMD and thus should use the diagnostic elements. Another clinic may provide endocrine care and so may focus on improving evaluations for endocrine management. Health providers may choose the items most relevant to their team at their clinic. We are not suggesting that piecemeal implementation of the care considerations is acceptable. Rather, we suggest that all elements that a clinic or individual health provider is capable of providing be evaluated in agreement with Fig 1. These common indicators are designed to be tailored to the goals of each provider and program, especially given the contextual variance across settings noted previously.
Using the Key Care Considerations
Setting Benchmarks for Care
We aim to set the groundwork for evaluation with Fig 1–5 and Table 1. A typical logic model outlines the theory of change for a program. It specifies how resources, activities, and the products of these activities result in the accomplishment of stated goals. It is important for an entity engaging in change to define the common goals for that change at the outset. The domain-specific goals in Table 1 may help in recognizing the patient outcomes to which adherence to the care elements are intended to contribute.
Benchmarking is a standardized process for collecting and reporting data to allow performance comparisons across programs, sites, or organizations.26 Benchmarking also improves quality of care and reduces inequalities in health care delivery through standardization of care.23,26 Having objective and specific indicators for the 2018 Care Considerations may reduce variability of care and improve the likelihood of standardized care. The first step in benchmarking is to determine the element(s) to benchmark and quantify the reference point for assessing performance.27 For example, if a DMD clinician or their institution is interested in improving early diagnosis of DMD, an indicator for evaluation might be the proportion of patients who receive genetic testing or are offered genetic testing. The clinician or institution can set a goal of reaching a rate comparable to an agreed-on standard for early diagnosis. Clarifying care goals in the beginning allows program planners to define indicators of success more clearly.
Continuous Quality Improvement
Data are most useful when they guide action. The elements found in the 2018 Care Considerations can be used to monitor performance and continuous quality improvement. The plan-do-study-act cycle is often used to facilitate program improvement.28 It is helpful to begin with a logic model during the planning stage and identify specific elements for monitoring, plan a method for tracking them, and set benchmarks for assessing them. Multiple data sources and data collection methods may be necessary for comprehensive evaluation. Although some elements like clinic visits, steroid use, imaging studies, and other measurable activities may be readily monitored and evaluated through retrospective analysis of electronic health records, elements that are not typically entered in medical records, such as items pertaining to transition and psychosocial issues, will need to be obtained through primary data collection (eg, questionnaires, interviews, focus groups). Program evaluators are likely to find that more a priori evaluation planning is needed for nonmedical care elements (eg, elements related to quality of life, such as community participation and transitions of care across the lifespan), because these may require staff support for collection, maintenance, and monitoring of data. Closely reviewing performance on care elements during the “do” stage of the plan-do-study-act cycle (the stage during which the program plan is implemented) is critical for gauging the capacity of an individual health provider or institution to adhere to the Care Considerations. Qualitative data may provide insight when implementers fall short of benchmarks.
Implementation outcomes can be assessed by using several methods. They can be assessed at the individual or organizational level through surveys, interviews, observational study, and analysis of administrative data. PPMD can evaluate the crude adoption rate of the 2018 Care Considerations by calculating the number of care centers applying for certification divided by the total number of centers that could apply for certification. Because PPMD’s certification criteria require adherence to the care considerations, the number of care centers that achieve certification could be an indication of how well they have been adopted in those DMD care centers across the United States. Clinic experience surveys are completed annually by families and returned to each center, providing data on the care and services received and serving as the means to inform and track continuous quality improvement. Additionally, PPMD’s DuchenneConnect Registry allows families to report clinical outcomes, which could be used to gauge the influence of care considerations on patient outcomes. MDA’s US Neuromuscular Registry25 helps MDA care centers implement a quality improvement program, collecting longitudinal patient data for benchmarking clinical best practices at participating sites. MDA’s registry can be used for evaluating implementation nationally and allow tracking of adherence over time.
Evaluating adoption of the care considerations at the individual health provider level is possible but may be more challenging than evaluation at the organizational level. A self-report questionnaire could be sent to health providers who care for individuals with DMD to assess adoption and acceptability of the care considerations. Findings generated by using this research method would need to be shared with the caveat that providers may not always accurately report their care or may report their compliance with the 2018 DMD Care Considerations using their own interpretation of the content. However, using the common elements specified in this document should limit bias from misinterpretation. All assessments of adoption and acceptability should include measurements of satisfaction with, agreement with, and intention to follow the components of the considerations for care. Although the target audience for the 2018 DMD Care Considerations is health care providers, families and affected individuals will decide whether to accept the care services offered. Patient and family attitudes toward it may be a good measure of the appropriateness of the care considerations. Implementation is more likely to succeed if patients and families view the considerations for care as feasible for them to follow.
MD STARnet18 may be useful for assessing fidelity and adherence to the care considerations. MD STARnet contains rich data on provision of care for individuals with DMD, but it can only assess fidelity indirectly. Although surveillance data can evaluate the consistency between 2018 DMD Care Considerations and care provided, assessing whether changes in care resulted from adopting the care considerations may not be possible. Data on the key elements can be examined longitudinally, and any changes before and after release of the 2018 Care Considerations may be a crude indication of whether they have influenced care for individuals with DMD.
Another method for assessing adherence to the care considerations would be a longitudinal analysis of administrative data, such as claims data. One drawback to this method is that the International Classification of Diseases code for DMD is currently the same code used for other types of muscular dystrophy; therefore, patients with DMD cannot be distinguished from patients with other muscular dystrophies. Leading advocacy groups, such as PPMD and MDA, have the means to collect primary data from health care providers, affected individuals, and their families. PPMD’s CDCC program and DuchenneConnect’s patient-reported outcomes registry, along with MDA’s US Neuromuscular Registry, are able to track adoption and adherence to the key elements. Thus, these organizations offer avenues for implementation outcomes research.
Evaluation of the implementation of the 2018 DMD Care Considerations is not advisable until the document has been fully disseminated. To maximize reach, the CDC and advocacy groups plan to disseminate the 2018 DMD Care Considerations through online checklists for families, training for providers, and promotion through social media activities.
The 2018 DMD Care Considerations were developed from evidence and expert opinion, but acceptance by the larger community of DMD health care practitioners is only assumed. Practitioners may be skeptical about effectiveness, concerned about the legal consequences and cost of implementation, and uncomfortable with a perceived reduction in their autonomy to manage their patients’ care.29 Despite these concerns, it is reasonable to believe that health providers will be satisfied with the 2018 DMD Care Considerations. First, the care considerations are informed by the most current science on Duchenne care. Second, leading scholars and practitioners in their respective disciplines led the writing of each care domain. This process occurred collaboratively, with the considerations for care being driven by consensus.
Ideally, evaluation of the implementation of the care considerations would provide evidence that they lead to the improved health and well-being of patients with DMD. However, establishing a causal link between adherence to care recommendations and patient outcomes at the population level is likely to be challenging. It is probable that adherence to the 2018 DMD care considerations has a cumulative effect, in that the more the care elements are adhered to, the better the patient outcomes for health-related quality of life and prolonged survival. Longitudinal tracking of the association between adherence to elements of the care considerations and patient outcomes can inform the assessment of its impact.
Contextual factors can facilitate successful implementation of the DMD Care Considerations. Future research could seek to understand these factors, as well as the mediating and moderating factors leading to the successful delivery of the care elements as described in the considerations. Ultimately, a greater understanding of contextual influences on implementation of care recommendations will improve the sophistication of future iterations of the care considerations.
- Accepted July 26, 2018.
- Address correspondence to Katherine S. Ong, PhD, Centers for Disease Control and Prevention, 4770 Buford Hwy NE, Chamblee, GA 30341. E-mail:
The guidelines or recommendations in this article are not American Academy of Pediatrics policy and publication herein does not imply endorsement.
This publication was supported in part by the Cooperative Agreement, NU38OT000167, funded by the Centers for Disease Control and Prevention. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention or the Department of Health and Human Services.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Muscular Dystrophy Association
- Parent Project Muscular Dystrophy
- Wolf J
- Birnkrant DJ,
- Bushby K,
- Bann CM, et al; DMD Care Considerations Working Group
- Birnkrant DJ,
- Bushby K,
- Bann CM, et al; DMD Care Considerations Working Group
- Fixsen DL,
- Naoom SF,
- Blase KA,
- Friedman RM,
- Wallace F
- Pandya S,
- Trout C,
- Conway K,
- Andrews J,
- Westfield C,
- Fox D
- Pandya S,
- Trout P,
- Conway C,
- Westfield C,
- Fox D
- Andrews JG,
- Conway K,
- Westfield C, et al
- Peters DH,
- Adam T,
- Alonge O,
- Agyepong IA,
- Tran N
- Grol R,
- Dalhuijsen J,
- Thomas S,
- Veld C,
- Rutten G,
- Mokkink H
- MDA US Neuromuscular Disease Registry
- Vlăsceanu L,
- Grünberg L,
- Pârlea D
- Moen R,
- Norman C
- Conroy M,
- Shannon W
- Copyright © 2018 by the American Academy of Pediatrics