BACKGROUND: Vision screening is an essential element of well-child care for young children. Recently, several professional groups have recommended the use of instrument-based screening; however, studies demonstrating the effectiveness of this technique in pediatric primary care settings are lacking.
METHODS: We designed a cluster randomized quality improvement project to test the implementation of instrument-based vision screening for 3- to 5-year-old children within a pediatric primary care network. The program consisted of 12 pediatric practices randomized into phase 1 and phase 2 groups. We evaluated the effect of the intervention on completed vision screening at well-child visits, family satisfaction, and referrals to eye care specialists.
RESULTS: Instrument-based vision screening increased completed screening among 3- to 5-year-old children from 54% to 89% in the phase 1 group and from 65% to 92% in the phase 2 group. Improvement was most marked among 3-year-old children, with completed screening increasing from 39% with chart-based screening to 87% with instrument screening. Family satisfaction was higher with instrument screening. In addition, instrument screening was associated with a 15% reduction in referrals to eye care specialists.
CONCLUSIONS: Instrument-based vision screening for preschool-aged children can be effectively implemented into primary care practice, results in substantially improved rates of completed vision screening at well-child visits, and may result in a reduction in unnecessary referrals to eye care specialists. Additional research is needed regarding how best to overcome barriers to the widespread use of this technology in pediatric primary care settings, as well as its longer-term effect on referrals and the prevalence of amblyopia.
- PPOC —
- Pediatric Physicians’ Organization at Children’s
- QI —
- quality improvement
Pediatric vision screening is an essential element of well-child care for young children given the importance of adequate vision to overall cognitive and social development. Appropriate vision screening in young children can detect amblyopia or amblyogenic risk factors at a time when treatment is effective, ideally before age 5 years, as well as serious but rare eye diseases, such as cataracts and neuroblastoma.1–4 Therefore, the major professional organizations concerned with children’s vision, including the American Academy of Pediatrics, the American Academy of Ophthalmology, the American Association for Pediatrics Ophthalmology and Strabismus, and the American Association of Certified Orthoptists, have recommended vision screening beginning in early childhood for all children.5,6 To date, the standard of care has relied heavily on optotype-based screening techniques, such as picture tests (eg, Lea Symbols) and letter charts (eg, HOTV or Snellen) for distance visual acuity and the Random Dot E test or similar for stereovision (hereafter, referred to as chart-based vision screening), but research has shown that such testing is limited in real-world primary care settings by a number of factors, including patient cooperation and the relatively high level of staff training and expertise needed to perform testing appropriately.7–11 Our previous work found that even an extensive training program for primary care practices led to only small increases in successful screening rates among 4- and 5-year-old children and no improvement among 3-year-old children.12 With the current status quo, the rate of amblyopia in the United States has been stable at 2% to 3%, well above what it could be if efficient and effective primary care–based screening and referral techniques were in place.13
Given the limitations of chart-based vision screening, several professional groups have recently recommended replacing its use in young children with instrument-based screening using portable photoscreeners or autorefractors.14,15 The current generation of these devices demonstrates high sensitivity/specificity relative to a gold standard comprehensive eye exam.16,17 Furthermore, in field testing, instrument screeners show high rates of testability when performed in settings such as Head Start by using trained testers.18 To date, however, no studies have demonstrated how effectively these instruments can be implemented in busy primary care settings, which will be necessary for widespread vision screening among young children to be accomplished. Thus, we designed a quality improvement (QI) project to determine how effective the implementation of instrument-based vision screening for preschool-aged children would be in the primary care setting and what effect it would have on referrals to eye care specialists. Our principal aim was to increase successfully completed vision screening of 3- to 5-year-old children at well visits within a group of pilot practices from a baseline of ∼60% using chart-based screening to at least 85% using instrument screening.
The Pediatric Physicians’ Organization at Children’s (PPOC) is an independent practice association of >80 privately-owned pediatric practices affiliated with Boston Children’s Hospital that provides primary care to an estimated 400 000 children throughout eastern Massachusetts. We invited applications from all 56 practices that had participated in our previous QI project to optimize chart-based vision screening12 to apply to participate in the project. We received applications from 22 eligible practices and randomly selected 12 practices to participate by using random number generation. Practices were stratified by size (≤4 vs ≥5 physicians) and then randomized into phase 1 or phase 2 arms by using random number generation.
The project consisted of 3 phases: a 7-week run-in phase (November 17, 2014 to January 5, 2015), during which all 12 practices used chart-based vision screening processes consisting of visual acuity testing using Lea symbol charts and stereovision testing using Random Dot E testing; an 8-week early phase (January 6 to March 2, 2015), during which phase 1 practices began instrument-based screening while phase 2 practices continued using chart-based vision screening; and a 10-week late phase (March 3 to May 8, 2015), during which phase 1 practices continued to use instrument-based screening and phase 2 practices began using instrument screening (Fig 1).
The device we chose for instrument-based screening was the SPOT Vision Screener (Welch-Allyn, Skaneateles Falls, NY), a binocular vision screener with acceptable sensitivity and specificity versus the gold standard vision exam.19–23 Participating practices received “out-of-the-box” training, as would be provided to any new purchaser of the device, consisting of a 30-minute review by a company sales professional. We built structured text fields into each participating practice’s electronic health record, with the goal of capturing standardized results for both chart-based and instrument-based screening throughout the project.
Throughout the course of the project, we asked practices to offer parents/guardians a brief survey to capture the patient/family experience associated with the screening type used during their visit. This survey consisted of 2 questions: (1) “On a scale of 0 to 10 (0 = not confident at all; 10 = extremely confident), how confident are you that today’s vision screening correctly captured your child’s ability to see?”; and (2) “On a scale of 0 to 10 (0 = not satisfied at all; 10 = extremely satisfied), how satisfied overall are you with the vision screening your child received today?”
Electronic Health Record Analysis
All routine check-up visits for 3- to 5-year-old children seen during the study period in each of the participating practices were identified through our network’s central electronic health record database. Patients for whom it was documented that they were under the care of an eye care specialist were excluded from additional analysis. For each visit, evidence of vision screening was searched for in the structured data fields built for the study and also in any other part of the record where vision screening results may have been captured. For chart-based screening, we deemed vision screening to have been fully completed if results for visual acuity testing in each eye and stereovision were documented. For instrument screening, we deemed vision screening to be complete if a result (“Pass” or “Complete Eye Exam Recommended”) was documented.
To quantify the effect of the implementation of instrument-based screening on referrals to eye care specialists, we used administrative claims data from 2 major commercial insurance companies that share such data with PPOC. For this analysis, we compared the rate of initial visits (Current Procedural Terminology codes 99201-5, 99241-5, 92002, and 92004) to ophthalmologists and optometrists for children 3 to 5 years of age at the time of the visit from 2 years before implementation through 1 year after implementation for practices participating in the instrument-based screening project to those for control PPOC practices who were not part of the project and employed chart-based screening throughout the time period analyzed. To determine if the difference in visit trends was significant, we modeled the difference in visit rates (rate for project practices minus rate for control practices) using linear regression accounting for autocorrelation over time.24
Statistical process control analyses were performed with QI Macros (KnowWare International, Denver, CO). All other analyses were performed with SAS version 9.4 (SAS Institute, Inc, Cary, NC). This project was reviewed by the Boston Children’s Hospital Committee on Clinical Investigation and deemed to meet our institution’s definition of QI and was therefore exempt from the requirement for individual informed consent.
The 6 practices randomized to phase 1 had a median of 7 medical providers (range: 3–13) and a median patient panel size of 4771 (range: 2215–8280). The corresponding numbers for the 6 phase 2 practices were 4.5 (range: 2–15) and 3651 (range: 1957–9676). A total of 18% of patients of phase 1 practices were publicly insured (range for individual practices: 7%–52%) compared with 15% of phase 2 practices (range for individual practices: 2%–37%).
Figure 2 demonstrates the project’s principal finding, screening results by study phase. During the run-in phase, with both groups employing chart-based screening, 172 of 316 patients (54.4%) adequately completed screening in the phase 1 group, and 231 of 354 patients (65.3%) completed screening in the phase 2 group. In the early study phase, the phase 1 group switched to instrument screening with 342 of 380 children (90.0%) adequately completing screening, whereas the phase 2 group continued to use chart-based screening with 338 of 508 children (66.5%) completing screening. In the late period, with both groups using instrument screening, the phase 1 group adequately screened 484 of 545 children (88.8%), whereas the phase 2 group adequately screened 571 of 621 children (91.9%).
Statistical process control charts demonstrate that a special-cause increase in the proportion of adequately screened children occurred immediately coincident with the introduction of instrument screening in both the phase 1 and phase 2 group, and performance was maintained consistently thereafter in both groups (Fig 3). Eleven out of the 12 practices in the project experienced a statistically significant improvement in completed vision screening with instrument screening compared with chart-based screening (Table 1).
As shown in Fig 4, the switch to instrument screening had the largest effect on the screening of 3-year-old children, with an increase in adequately completed screening from 156 of 404 children (38.6%) using chart-based screening to 479 of 505 children (87.1%) using instrument screening (P < .0001). Statistically significant improvements were also seen for 4-year-old children (279 of 405 children [68.9%] using chart-based screening to 455 of 493 children [92.3%] using instrument screening; P < .0001) and 5-year-old children (306 of 369 children [82.9%] using chart-based screening to 463 of 503 children [92.0%] using instrument screening; P < .0001).
To additionally understand the reasons for the failure of patients to be screened in the instrument screening cohorts, we analyzed the documentation for such patients. Of the 1546 total patients eligible for instrument screening, 149 (9.6%) were not adequately screened. Of the 149 patients not adequately screened, there was no documentation of an attempt among 137 (8.9% of the total), and there was documentation of an unsuccessful attempt among 12 (0.8% of the total).
A brief family experience survey was fielded in a convenience sample of project participants. Families of 137 of 1178 patients (11.7%) undergoing chart-based screening completed the survey as did families of 631 of 1546 (40.8%) patients undergoing instrument screening. In response to the question “On a scale of 0 to 10 (0 = not confident at all; 10 = extremely confident), how confident are you that today’s vision screening correctly captured your child’s ability to see,” the mean among those undergoing chart-based screening was 8.2 (SD: 2.3) vs 8.6 (SD: 1.8) for those undergoing instrument screening (P = .04). In response to the question “On a scale of 0 to 10 (0 = not satisfied at all; 10 = extremely satisfied), how satisfied overall are you with the vision screening your child received today,” the mean among those undergoing chart-based screening was 8.4 (SD: 2.1) vs 8.9 (SD: 1.6) for those undergoing instrument screening (P = .02).
The impact of instrument screening on initial visits to eye care specialists is shown in Fig 5. During the preimplementation period, there was no discernible difference in the quarterly rate between project participants (82.6 visits per 1000 patient-years) and controls (81.8 visits per 1000 patient-years). In the postimplementation period, after both phase 1 and phase 2 practices instituted instrument screening plus 1 quarter of additional lag time for referrals to eye care specialists to occur, the rate of initial visits to ophthalmologists and optometrists among participants (62.1 visits per 1000 patient-years) was 15.1% lower than that of controls (73.2 visits per 1000 patient-years; P = .04 for the difference in rates over time).
This QI project, consisting of a cluster randomized implementation of instrument-based vision screening in primary care pediatric practices, demonstrated clear superiority of instrument screening over chart-based screening in terms of testability for preschool-aged children. Our principal aim of increasing the proportion of 3- to 5-year-old children successfully screened at well visits to at least 85% was achieved. Notably, improvement was achieved in 11 out of 12 participating practices with instrument screening; the 1 practice that did not experience a statistically meaningful improvement started with a relatively high level of success using chart-based screening. The fact that the improvement was nearly immediate on implementing instrument screening, was relatively uniform across practices, and was achieved with minimal training suggests that similar improvement should be achievable by other practices instituting this technology.
Improvement was observed for each age evaluated (children at 3, 4, and 5 years of age), but was most impressive among 3-year-old children. Indeed, instrument screening nearly entirely closed the testability gap between 3-year-old and 4- to 5-year-old children seen with chart-based screening.7,10,25,26 Reducing the age of successful routine screening holds the promise of detecting amblyopia or amblyogenic risk factors at earlier ages when treatment is most effective.1–4 Based on our experience with this project, the improvement seen with instrument screening relates to the success of this technique in addressing the major barriers to chart-based vision screening, primarily: limitations of preschool-aged children’s ability to focus and complete the testing; limitations of adequate distraction-free space to complete chart-based screening; and the need for staff training and expertise with chart-based screening techniques.12 Additionally, staff reported that the time to complete instrument-based screening in young children (typically <1 minute) was substantially shorter than that for chart-based screening (typically several minutes), an important consideration for busy primary care practices.
Although our project achieved an ∼90% successful screening rate with instrument screening, it is important to note that there was documentation that instrument screening was attempted but unsuccessful in only 1% of cases. In the other ∼9% of unscreened children, no attempt was documented, and we were unable to ascertain from our data whether these represented failed attempts at instrument screening or instances where no screening was attempted. Other published reports of testability in young children by using automated devices suggest that the untestable proportion is closer to 1% than 10%,16,17,20,27,28 but these studies were not from primary care settings and additional study to define the untestable proportion in clinical primary care settings would be useful. This analysis also points out that the availability of effective technology does not guarantee that a preventive care service, such as vision screening, will be reliably delivered at busy primary care visits. As we know from data on other recommended services that are not reliably performed, the human element of QI is still critical to achieve high levels of performance.29
In addition to an improvement in detecting potentially treatable vision conditions in young children, we hypothesized that instrument screening may also reduce unnecessary referrals to eye care specialists. We based this hypothesis on previously published data from our network indicating that ∼40% of children referred to eye specialists were found to have no diagnosed vision condition30; such patients were presumably referred because of an inability to cooperate with chart-based screening and/or false-positive results. Indeed, in the current project, participating practices saw a reduction in new referrals to ophthalmologists and optometrists among 3- to 5-year-old children of ∼15% after the implementation of instrument screening compared with control practices using chart-based screening. It is not clear whether the reduction in referrals we experienced would occur in other networks; such depends on their preexisting vision screening practices and referral patterns. However, if our experience with referral rates is sustained over time and replicated by others, it would suggest that instrument screening will be cost-effective for the health care system in the long run, because even a small reduction in the rate of unnecessary referrals to specialists would over time compensate for the cost of the equipment. It could also potentially lead to improved access to specialty services for children with vision conditions, especially in places underserved by pediatric eye care specialists.
The major strength of this work lies in its direct applicability to the pediatric primary care setting. The practices involved received no training on instrument screening other than the standard out-of-the-box training provided by the manufacturer’s sales representative and yet found immediate success screening young children. Thus, we believe other practices who implement instrument screening with the same or similar devices should be able to replicate our results. In our view, the principal challenges to widespread adoption of this new technology lie in the several thousand dollar start-up cost, which may be prohibitive, especially for small practices, and a lack of reimbursement for this service from many insurance companies.31 Indeed, although such instruments have been available for several years now, their use in pediatric primary care settings appears to be limited thus far.8 In addition, technical questions remain with the use of instrument-based screening that need resolution, including the risk of missing significant hyperopia that may be clinically significant.32,33 The ultimate question that remains to be answered is whether instrument-based screening among young children in the primary care setting will reduce long-term rates of amblyopia.
This QI program demonstrated significant improvement in completed vision screening at well-child visits among 3-, 4-, and 5-year-old children with instrument-based screening compared with chart-based vision screening. Additional study is needed into methods of overcoming barriers to widespread use of this new technology, its effect on rates of referral to eye care specialists, and, ultimately, its long-term effect on the prevalence of amblyopia in the population.
- Accepted March 9, 2017.
- Address correspondence to Louis Vernacchio, MD, MSc, Pediatric Physicians’ Organization at Children’s, Children’s Hospital Boston, 77 Pond Ave, Suite 205c, Brookline, MA 02445. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Funded by internal funds of the Pediatric Physicians’ Organization at Children’s.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Williams C,
- Northstone K,
- Harrad RA,
- Sparrow JM,
- Harvey I; ALSPAC Study Team
- Village EG; Committee on Practice and Ambulatory Medicine, Section on Ophthalmology. American Association of Certified Orthoptists; American Association for Pediatric Ophthalmology and Strabismus; American Academy of Ophthalmology
- Wasserman RC,
- Croft CA,
- Brotherton SE; American Academy of Pediatrics
- Miller JM,
- Lessin HR; American Academy of Pediatrics Section on Ophthalmology; Committee on Practice and Ambulatory Medicine; American Academy of Ophthalmology; American Association for Pediatric Ophthalmology and Strabismus; American Association of Certified Orthoptists
- US Preventive Services Task Force
- Vision in Preschoolers Study Group
- Dahlmann-Noor AH,
- Vrotsou K,
- Kostakis V, et al
- Schwartz RH,
- Schuman AJ,
- Wei LL
- Copyright © 2017 by the American Academy of Pediatrics