February 2016, VOLUME137 /ISSUE Supplement 2

Improving Access to Care at Autism Treatment Centers: A System Analysis Approach

  1. June Austin, RNa,
  2. Patricia Manning-Courtney, MDb,
  3. Meghan L. Johnson, MScc,
  4. Rachel Weber, MSIEa,
  5. Heather Johnson, PsyDb,
  6. Donna Murray, PhDb,d,
  7. Karen Ratliff-Schaub, MD, MBOEe,
  8. Abbey Marquette Tadlock, MSIEa, and
  9. Mark Murray, MD, MPAa
  1. aMark Murray and Associates, Sacramento, California;
  2. bCincinnati Children’s Hospital Medical Center, Cincinnati, Ohio;
  3. cNational Institute for Children’s Health Quality, Cambridge, Massachusetts;
  4. dAutism Speaks Autism Treatment Network, New York, New York; and
  5. eNationwide Children’s Hospital, Columbus, Ohio
  1. Ms Austin oversaw the intervention and acquisition of data, interpreted data to provide coaching to teams, drafted the initial manuscript, and critically reviewed and revised the manuscript; Dr Manning-Courtney led the initiative at her institution, contributed to the initial draft of the manuscript, and oversaw and completed all revisions; Ms M. Johnson oversaw the NICHQ-led Collaborative to Improve Care for Children With Autism Spectrum Disorder, conceptualized the manuscript and contributed to the initial draft, and critically reviewed and revised the manuscript; Drs H. Johnson, Manning, D. Murray, and Ratliff-Schaub led the initiative at their institutions, contributed to the initial draft of the manuscript, and critically reviewed and revised the manuscript; Ms Tadlock and Ms Weber designed the data collection instruments, supervised data collection at both sites, assisted in data interpretation, and critically reviewed and revised the manuscript; Dr M. Murray conceptualized and designed the methods, contributed to the design of the manuscript, and critically reviewed and revised the manuscript; and all authors approved the final manuscript as submitted.


OBJECTIVE: The prevalence of autism spectrum disorder is steadily increasing and placing more demands on already overburdened diagnostic and treatment systems. A thoughtful, systematic reorganization of autism service delivery may reduce delays and better meet the growing need.

METHODS: Two clinical centers in the Autism Intervention Research Network on Physical Health, Cincinnati Children’s Hospital Medical Center (CCHMC) and Nationwide Children’s Hospital (NCH), undertook a year-long access improvement project to reduce delays to care by using system analysis to identify sources of delay and to target changes by using a set of defined access principles. Although both sites addressed access, they focused on slightly different targets (reducing number of patients with autism spectrum disorders waiting for follow-up appointments at NCH and reducing delay to new diagnosis at CCHMC).

RESULTS: Both sites achieved dramatic improvements in their complex, multidisciplinary systems. A 94% reduction in number of patients on the waitlist from 99 to 6 patients and a 22% reduction in median delay for a new ongoing care appointment were realized at NCH. A 94% reduction in third next available appointment for new physician visits for children 3 to 5 years old was realized at CCHMC.

CONCLUSIONS: This article demonstrates that 2 different clinical systems improved access to care for autism diagnosis and follow-up care by identifying sources of delay and using targeted changes based on a set of access change principles. With appropriate guidance and data analysis, improvements in access can be made.

  • Abbreviations:
    autism spectrum disorder
    Autism Speaks Autism Treatment Network
    Cincinnati Children’s Hospital Medical Center
    clinical full-time effort
    Division of Developmental and Behavioral Pediatrics
    Mark Murray and Associates
    Nationwide Children’s Hospital
    National Institute for Children’s Health Quality
    The Kelly O’Leary Center for Autism Spectrum Disorders
    third next available appointment
  • The Institute of Medicine named timeliness as 1 of 6 domains of quality and an aim for health system redesign.1 Long delays to receiving care can result in suboptimal clinical outcomes, provider and patient dissatisfaction, and rises in system costs due to rework, redundancy, triaging, and managing waitlists.25 Because of the high demand for finite resources, access to pediatric specialty care is a growing concern and poses ethical challenges to providers striving to improve quality of care.6

    Children with developmental disabilities, especially those with autism spectrum disorder (ASD), experience difficulty accessing specialty care.710 Barriers to access include limited clinician resources, geographic distribution of specialists, and insurance obstacles leading to long wait times.6,11 A 2012 survey of 15 autism specialty centers across the United States found third next available appointment (TNA),12 a common measure of access in health care, for an initial diagnostic evaluation to be 14 months, and 3 months for follow-up appointment, often after weeks or months on waitlists before being scheduled (unpublished data).

    Increased public awareness, recommendation by the American Academy of Pediatrics that children be screened for ASD at 18 and 24 months,13 and health care reforms leading to increased insurance coverage14 have led to an even greater demand for autism services. If the prevalence of ASD continues to rise, from 1 in 150 in 2000 to 1 in 68 in 2010,15 there will be increasing demands on an already overburdened system. A systematic reorganization of autism service delivery is needed to reduce delays and meet growing needs.

    A systematic approach, which focuses on detailed clinical process mapping and measurement of patient demand and clinician supply at critical process points, has achieved positive results in primary care and other specialty care settings.3,12,16,17 There are no known reports that examine the use of this approach to improve access to autism diagnostic and treatment services. Ultimately, access improvement requires balancing the demand for a clinical service (ie, ASD evaluation) and the supply of that clinical service (ie, ASD evaluation appointments).


    The Autism Speaks Autism Treatment Network (ATN) consists of 14 clinical centers across North America working to develop optimal approaches to medical care for children and adolescents with ASD. As part of the larger Autism Intervention Research Network on Physical Health Collaborative to Improve Care for Children with Autism Spectrum Disorder, led by the National Institute for Children’s Health Quality (NICHQ), 2 clinical centers, Cincinnati Children’s Hospital Medical Center (CCHMC) in Cincinnati, Ohio and Nationwide Children’s Hospital (NCH) in Columbus, Ohio, participated in a year of intensive access improvement work with access specialist consultants, using a system analysis approach. Institutional review board approval was not required for this quality improvement work.


    The Division of Developmental and Behavioral Pediatrics (DDBP) includes The Kelly O’Leary Center for Autism Spectrum Disorders (TKOC), a multidisciplinary diagnostic, treatment, and research program for children with ASD. DDBP completes >12 000 outpatient visits per year and receives on average 400 new referrals per month, approximately one-third of which are for suspected ASD. Children referred for possible ASD undergo comprehensive diagnostic evaluations, including assessments by a developmental pediatrician, child psychologist, and speech language pathologist. Staffing (clinical full-time effort; CFTE) dedicated to ASD assessments is as follows: developmental behavioral pediatricians (3.8 CFTE), nurse practitioners (2.9 CFTE), psychologists (2.8 CFTE), and speech language pathologists (5.2 CFTE). In November 2012, the average wait time for children <6 years old referred to DDBP for ASD or other developmental concerns was 120 days and for children >6 years old was 400 days. As a result of prioritization and other quality improvement efforts, wait time for children <3 years old was 60 days.


    The NCH Child Development Center provides a comprehensive, coordinated model of care for children with ASD, including interdisciplinary diagnostic evaluations and ongoing care for children previously diagnosed with ASD. NCH receives ∼380 referrals for new evaluations per month, receives 40 new ongoing ASD care referrals per month, and follows 1100 patients with established ASD, including psychotropic medication management. Ongoing care is provided by registered nurses (1.4 CFTE), developmental behavioral pediatricians (0.7 CFTE), and a psychiatrist (0.2 C FTE). In November 2012, the average wait for an ongoing care appointment was 84 days.

    Based on their expertise in access to specialty care, NICHQ selected Mark Murray and Associates (MMA) as consultants to work in collaboration with teams from each site. The MMA team included a physician with a master’s degree in health administration who designed and oversaw implementation of interventions, a registered nurse knowledgeable in improvement science for ongoing coaching, and management engineers who designed collection tools and supervised data collection and interpretation.

    System Analysis Approach

    Both CCHMC and NCH applied a 5-step system analysis approach for access improvement based on the Model for Improvement, with additional emphasis on mapping patient flow through the clinical system (Table 1). Step 1 of the 5-step analysis approach required the formation of dedicated improvement teams at each site, which were formed in November 2012. The initiative was launched with a live 2-hour webinar session to provide an overview of access strategies and principles to both teams. Teams from each site then completed step 2 by setting a clear, measurable, attainable aim focused on delay reduction.

    TABLE 1

    5-Step System Analysis Approach to Access Improvement

    In January 2013, both teams began biweekly hour-long coaching webinars to address step 3, that of mapping the current patient flow through clinic systems. Teams visually represented the clinical process from time of referral to a designated end point. The maps included referral sources, triage processes, and all clinic visits that might occur as the patient moves through the system. Arrows represent the flow of patients through the clinical system demand stream. Boxes represent specific appointments and the supply of clinicians available for those appointments. Demand and supply for each appointment type are measured as a potential source of constraint (ie, a bottleneck).

    With patient flow maps completed, step 4, measuring across the systems, could be addressed. Standard access measures were applied to each “box” or appointment type in the process map. Measures applied in sequence were as follows:

    1. Delay: TNA12

    2. Caseload: number of unique patients cared for by each provider in a 12-month period

    3. Demand: number of appointments scheduled each week for new and return appointments

    4. Supply: prospective count of appointments available each week for new and return appointments

    5. Activity: number of appointments completed each week

    Teams collected measures from their electronic medical record and were trained in use of customized spreadsheets for data entry and analysis. Data were collected weekly with demand and activity reported for 1 week before, delay reported for the current week, and supply estimated from future schedules. Some manual data collection was necessary at baseline.

    Measures were later incorporated into the electronic medical record. With a measurement process in place, system analysis progressed by using data to identify constraints or bottlenecks and ideas for change. Step 5, testing changes in the clinic system to improve access, was the final step in this work and was guided by access change principles (Table 2).

    TABLE 2

    Access Change Principles



    The CCHMC access team consisted of 3 clinicians (1 MD, 1 psychologist, and 1 speech language pathologist), 1 scheduler, 1 clinical director, 1 business director, and 1 quality improvement consultant. Their aim was to improve access to TKOC ASD evaluations by reducing delay for ASD evaluation appointments. Mapping occurred during weekly 1-hour meetings, with support from MMA, and took 8 weeks to complete. Through mapping, 23 different appointment types used in the diagnosis of children with ASD were identified. Mapping and measurement of supply and demand for each appointment type also identified that overall demand for ASD diagnostic evaluations exceeded supply of appointments available. In January 2012, demand was 53 visits per month, and supply was 38 visits per month for all new TKOC physician visits. Demand for patients ages 3 to 5 years was 33 visits per month and supply was only 26, representing the major system constraint or bottleneck. To achieve balance between supply and demand, the team used change principle 5 to reduce demand for potentially unnecessary visits (Table 2). Specifically, CCHMC tested a change in the usual diagnostic process for children age 3 to 5 presenting for suspected autism by reducing the total number of physician visits from 2 (1 at the beginning of the process and 1 at the end) to 1 (just at the end of the diagnostic process). This diagnostic model was the same as had been previously tested, then adopted for children aged 19 to 36 months (a group for whom there was less delay and no waitlist, based on previous efforts to maximize access for this youngest population). This change significantly increased physician visit supply. As a result, TNA for this visit type decreased from 128 days in December 2012 to 16 days in August 2013 (Fig 1).

    FIGURE 1

    Delay: new rule-out autism visit by age group.

    The team then examined the 23 visit types identified in their system and worked to decrease system complexity and therefore reduce total number of demand streams (change principle 3). Three age groups (19–36 months, 3–5 years, and ≥6 years) were reduced to 2 age groups (<6 years and >6 years). In addition, other visit types were collapsed together when appropriate to reduce system complexity. These changes resulted in a decrease in TNA for children <6 years old from 16 to 8 days starting in September 2013 (Fig 1).

    CCHMC also addressed the lengthy waitlist for all children age 3 to 5 years by reviewing children on the waitlist to determine whether visits were still needed and by implementing a short-term backlog reduction strategy, that of offering incentive-based Saturday clinics with DDBP psychologists. Through the latter strategy, 75 children on the waitlist were served (Table 3).18

    TABLE 3

    CCHMC Delay (TNA) and Waitlist for TKOC and DDBP Appointments


    The NCH team consisted of 1 physician, 2 nurses, 2 schedulers, and 1 research coordinator/parent. The aim for NCH was to improve access to autism ongoing care clinics by minimizing delay for appointments. Mapping identified 1 demand stream with 3 different appointment types for ongoing care of established ASD: new (first visit for ongoing care), return (follow-up visit for ongoing care), and ATN follow-up (annual follow-up visit specific to children enrolled in the ATN national registry). Delay occurred both for new ongoing care appointments and for patients on a waitlist for any type of ongoing care appointment. NCH did not segment ongoing care into subgroups by age or other categories.

    System analysis revealed NCH was balanced in terms of annual demand and supply for ongoing care. The 12-month caseload of unique ongoing care patients was 698, generating 1751 visits. The 12-month prospective view of supply showed the current set of providers should be able to offer 2035 visits. However, more detailed measurement revealed that the weekly demand for new ongoing care patients was 9.6 appointments, whereas supply offered was only 7 appointments. In some cases, the supply of new ongoing care appointments (60 minutes long) was converted to 2 follow-up ongoing care appointments (30 minutes), thus reducing the supply for new ongoing care appointments. Additional supply variation as a result of holidays, clinic closures, and inconsistent enforcement of clinician expectations was identified. The team used change principle 4 (Table 2) to develop contingency plans to address variation in supply.

    Analysis revealed that providers scheduled on Mondays lost supply over the course of the year because of statutory holidays. NCH reduced supply variation by recovering 155 appointments related to holidays (affected clinicians were required to add clinics). An additional 208 appointments were recovered after annual clinic expectations were standardized and enforced. Weekly appointment activity and TNA data were shared with physicians to increase awareness about their role in variation and delay. Policies regarding number of provider clinic weeks per year were established and enforced.

    The waitlist of 99 patients created hidden delay, where patients waited for prolonged periods of time before receiving an appointment. To better assess true delay, waitlisted patients were scheduled, even if a long time away, to reveal accurate patient delays. Backlog was reduced by no longer allowing return patients to take new, 60-minute appointments and by temporarily adding new ongoing care appointment supply for 1 faculty physician and 2 fellows who volunteered extra clinic time. This time-limited (4 month) effort resulted in 25 additional new patients being seen over a 4-month period. By August 2013, NCH realized an 88% reduction in number of patients on the waitlist from 99 to 12 patients and a 47% reduction in the delay for a new appointment from 104 to 55 days (Fig 2; Table 4).

    FIGURE 2

    Delay: new to ongoing care. Note: Construction resulted in diminished availability of clinic space temporarily and inability to use available provider supply to maximum capacity. TNA and waitlist increased accordingly. Team is aware this temporary supply side variation will need to be addressed with a temporary backlog reduction plan once clinic space reopens.

    TABLE 4

    NCH Delay (TNA) and Waitlist for Ongoing Care Appointments


    An access problem is a delay problem. Delay represents the relationship between supply and demand. Many independent demand and supply factors contribute to delays:

    • Situations where demand permanently exceeds supply.

    • Complex system designs that create multiple distinct streams of work. In environments where demand varies, segmenting larger populations into smaller subsets results in delay.2

    • Variation in either demand or supply results in temporary mismatches.

    In addition to visible delays, measured as TNA, creating a waitlist hides even more days or weeks of patient waiting. Other possible delay steps include processing of referrals to assess appropriateness, urgency, and need for additional information. If waitlists or processing must occur before the appointment is scheduled, TNA does not capture the total time the patient waits to be seen.

    Delay to ASD diagnosis and treatment is well recognized but has been studied primarily from the perspective of nonclinical system-related factors (ie, the role of ethnicity, socioeconomic factors, family factors, and specific child presentation factors).19 Although detailed descriptions of the varied clinical models that exist to evaluate and diagnose children with ASD are limited or nonexistent in the literature, our experience in the ATN suggests that ASD assessment models are usually complex and consist of multiple assessments, and they are overdemanded and undersupplied, a perfect setup for delay. The Developmental Behavioral Pediatrics Research Network is studying practice variation in ASD evaluation.20 Our complex and variable systems contribute to delay.

    CCHMC clinic system complexity evolved over a number of years, in an attempt to prioritize certain populations (ie, children <3 years old), and resulted in multiple visit types, or demand streams, and thus more opportunity for delay. When these visit types were minimally collapsed, TNA improved. Specifically, fewer demand streams means fewer management decisions and fewer opportunities for bottlenecks. In general, more complex systems have more opportunities for delay. In contrast, NCH made a system design choice not to segment patients more than their 3 appointment types. By maintaining patients in 1 demand stream, they had fewer queues to manage, fewer opportunities for constraints, and better use of supply.

    At NCH, although overall demand and supply were balanced, delays were caused by week-to-week supply variation and converting supply of 1 appointment type (new ongoing care) to another (follow-up ongoing care). Supply variation deterred the group from offering the total appointments needed, 2035 over 12 months, and was addressed by standardizing clinic expectations. Late August clinic construction negatively affected TNA by reducing supply temporarily (Fig 2). Postintervention metrics were taken before the construction period because the access change principles were achieving success until this extenuating circumstance.

    Waitlists are a common strategy for managing high patient demand and delay yet are also a source of hidden delay. Both sites reduced waitlists by reviewing waitlists to ensure accuracy and to discover whether patients were still waiting or had been removed from the list (eg, moved, received services elsewhere). These patients were removed, which resulted in an accurate number of waitlisted patients. At CCHMC the waitlist for DDBP new 3- to 5-year-olds was eliminated as a result of added physician supply (result of changing clinic model) and waitlist review, as well as backlog reduction efforts by psychologists. NCH provided the remaining patients with appointments by opening the schedule further. Moving patients off the waitlist and onto the schedule may temporarily extend TNA unless the backlog is formally addressed. Any clinic using waitlists may benefit from considering these strategies.

    Both sites were able to improve access without adding new clinic staff. Improvements reported in this article are the result of system changes, which maximized capacity of existing supply. Supply was increased in a time-limited manner at NCH when existing clinicians added appointments for the purpose of backlog reduction.

    Both sites had different aims but implemented similar steps to better understand and analyze their respective systems. NCH chose to address access to ongoing care appointments for children with ASD, having previously done quality improvement work to improve access to new ASD evaluations. CCHMC chose to address access to new ASD evaluations, because access to ongoing care appointments was a less critical issue (possibly in part because pediatric nurse practitioners provided ongoing care). Data collection and analysis, which took 3 to 4 hours weekly at both sites, improved change management, allowing clinicians and clinic directors to see the necessity of either testing new clinic models, as in the case of CCHMC, or enforcing clinic attendance expectations, as in the case of NCH. Although both sites used an element of backlog reduction to address waitlists, which required added staff and clinician time, it is worth noting that both sites were able to reduce delay during the course of this work without permanently adding CFTE. Backlog reduction is a useful, short-term strategy for addressing patient demand, but effects will not be sustained if overall demand continues to exceed supply. Both sites found system analysis tremendously informative and beneficial, albeit initially somewhat overwhelming. Through regular site calls, teams also learned from each other during this study. Similar themes emerged at each site, such as the need to maximize supply (through enforced clinic guidelines or changing clinic models to increase supply) and manage waitlists. With training and support, data collection and system analysis have become second nature at both sites and have permitted sustained access improvement.


    Data on patient and provider satisfaction with changes implemented were not formally collected during this study, but they are clearly needed as access improvement continues. It should be noted that informally, both sites received positive feedback from families and referring providers about shorter wait times. Specific financial analysis of the costs and benefits of this work was also not done and would be helpful for clinic directors undertaking work of this scope and scale. Both sites observed an increase in provider utilization, which presumably would translate to increased revenue. Sites unable to garner institutional leadership, team motivation, and appropriate resources of staff time may find thislack of institutional support a limitation as well. Waitlisted patients who may have sought care elsewhere were not counted or tracked to determine their impact on waitlist changes.


    Although this work started as a 1-year project, ongoing monitoring of supply and demand is necessary to maintain access goals simply because supply and demand will always vary. Having a consistent process for collecting measures, knowing what to look for, and continuing to apply access change principles as appropriate is the new standard operating procedure at these sites. Both sites have shifted from making decisions based on opinion and conjecture toward making decisions based on data. CCHMC maintains an active access team that meets weekly to address access goals. Testing is under way at CCHMC to increase capacity by decreasing appointment lengths for select appointment types. CCHMC has also implemented strategies to improve clinician productivity, including sharing monthly productivity dashboards with clinicians. NCH is working on demand reduction strategies by initiating earlier hand-offs back to referral sources and engaging with primary care referral sites to optimize referrals.

    Understanding the financial impact of system changes, both the cost of engagement in access projects and the potential financial gain as a result of increased clinical productivity, is a critical part of maintaining this work and generalizing to other settings. Equally if not more important is the impact on all stakeholders, particularly patients and their families, clinicians, and referring providers, and efforts are under way at both sites to collect these important data.


    Although this article describes the work of 2 specific autism centers and their unique systems, the improvement methods and principles provide a strategy for similarly complex systems seeking to reduce delays while delivering high-quality care. It is incumbent on our current clinical systems to not contribute to the already well-known delays to ASD diagnosis and care. Our work suggests that current delays in accessing ASD evaluation and ongoing care services should not be universally accepted as the norm but rather can be improved by systematically analyzing, and changing, these complex and variable clinics.


    We thank Charles Homer, MD, MPH, president and CEO of NICHQ, and James Perrin, MD, director of the Clinical Coordinating Center, ATN, for their executive oversight of the collaborative project; CCHMC and NCH for their work throughout the project; and Daniel Coury, MD, Meghan Guinnee, PhD, and Marianne McPherson, PhD, who reviewed and provided feedback on this manuscript.


      • Accepted November 9, 2015.
    • Address correspondence to Patricia Manning-Courtney, MD, Cincinnati Children’s Hospital Medical Center, 3333 Burnet Ave, MLC 4002, Cincinnati, OH 45229. E-mail: patty.manning{at}
    • FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

    • FUNDING: Supported by cooperative agreement UA3 MC11054 through the US Department of Health and Human Services, Health Resources and Services Administration, Maternal and Child Health Research Program to the Massachusetts General Hospital. Conducted through the Autism Speaks Autism Treatment Network serving as the Autism Intervention Research Network on Physical Health. The views expressed in this article do not necessarily reflect the views of Autism Speaks, Inc or HRSA.

    • POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.