OBJECTIVE: Children with specific language impairment (SLI) face risks for social difficulties. However, the nature and developmental course of these difficulties remain unclear. Gaze behaviors have been studied by using eye tracking among those with autism spectrum disorders (ASDs). Using this method, we compared the gaze behaviors of children with SLI with those of individuals with ASD and typically developing (TD) children to explore the social perception of children with SLI.
METHODS: The eye gazes of 66 children (16 with SLI, 25 with ASD, and 25 TD) were studied while viewing videos of social interactions. Gaze behaviors were summarized with multidimensional scaling, and participants with similar gaze behaviors were represented proximally in a 2-dimensional plane.
RESULTS: The SLI and TD groups each formed a cluster near the center of the multidimensional scaling plane, whereas the ASD group was distributed around the periphery. Frame-by-frame analyses showed that children with SLI and TD children viewed faces in a manner consistent with the story line, but children with ASD devoted less attention to faces and social interactions. During speech scenes, children with SLI were significantly more fixated on the mouth, whereas TD children viewed the eyes and the mouth.
CONCLUSIONS: Children with SLI viewed social situations in ways similar to those of TD children but different from those of children with ASD. However, children with SLI concentrated on the speaker’s mouth, possibly to compensate for audiovisual processing deficits. Because eyes carry important information, this difference may influence the social development of children with SLI.
- ASD —
- autism spectrum disorder
- BSID-2 —
- Bayley Scales of Infant Development–Second Edition
- BSID-3 —
- Bayley Scales of Infant and Toddler Development–Third Edition
- MDS —
- multidimensional scaling
- SLI —
- specific language impairment
- TD —
- typically developing
What’s Known on This Subject:
Children with specific language impairment are at risk for social difficulties. However, whether this occurs adaptively as a result of language impairment or occurs as a result of an underlying deficit in social cognition remains unclear.
What This Study Adds:
We used eye tracking to explore how children with specific language impairment view social scenes. The overall gaze behavior resembled that of typically developing children. Significant attention to the speaker’s mouth may result in receiving less social-emotional information from the eyes.
Recent studies have indicated that children with specific language impairment (SLI) are at risk not only for language difficulties but also for social difficulties, such as peer rejection and poor social competence.1,2 These social difficulties have been observed in children as young as preschoolers3 and may persist through adolescence into adulthood.2,4–6 Despite evidence that these social difficulties and low social competence relate to language impairment, the nature of impaired social competence and how it develops in SLI remains unclear. That is, whether it occurs adaptively as a result of language impairment or whether it represents an underlying deficit in social cognition associated with SLI has yet to be determined.2,7
Eye tracking is a way to use eye movement to explore cognitive processes and the allocation of attention.8 By using dynamic social scenes as stimuli, atypical gaze behavior in association with social behavior has been reported in individuals with autism spectrum disorders (ASDs).9–12 Fixation time analysis has been applied in most studies, but the results have been inconsistent, possibly reflecting developmental changes or task-related differences. To distinguish the core gaze behaviors of ASD, Nakano et al examined temporospatial gaze behaviors by applying multidimensional scaling (MDS). Their results indicated a distinct difference in the gaze behaviors of individuals with ASD and those of control subjects, both in adults and in children.11
In this study, we extended the work of Nakano et al by using eye tracking to explore social perception of children with SLI through gaze behaviors. We compared the results with those obtained from children with ASD, a group known to have deficits in social cognition, and with those obtained from typically developing (TD) children. We limited the age of the children with SLI to 42 months because we hypothesized that if children with SLI developed social difficulties as an adaption, the gaze behaviors of young children with SLI should be close to those of TD children but different from those of children with ASD. Conversely, if an underlying deficit in social cognition underpinned the difficulties, the gaze behaviors of those with SLI may differ from those of TD children at an early age.
Sixty-six children participated: 16 children with SLI, 25 children with ASD, and 25 TD children. All children with SLI and ASD were recruited from our university outpatient clinic and were formally diagnosed by experienced clinicians according to Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, criteria. All children's first language was Japanese.
We relied on the same sample of children with ASD and of TD children who participated in the previous study; their characteristics were described in the previous article.11 For the ASD group, developmental age was measured with the Bayley Scales of Infant Development–Second Edition (BSID-2) for children <42 months and the Japanese Kaufmann Assessment Battery for Children for those >42 months. Levels of social behavior were measured with the orientation/engagement and emotional regulation subscales of the Behavior Rating Scales in the BSID-2. Chronological ages of TD children were matched with the developmental ages of children with ASD, under the assumption that developmental ages of TD children were consistent with their chronological age. None of the TD children had a family history of ASD or a history of developmental delay.
For the SLI group, cognitive levels and receptive and expressive language levels were measured by using the Bayley Scales of Infant and Toddler Development–Third Edition (BSID-3). Developmental age was measured by using the BSID-2. Children were diagnosed as SLI when they satisfied the following criteria: (1) lack of cognitive deficits (cognitive scores >85 on BSID-3), (2) language scores <1.25 SD of the mean (receptive and/or expressive language scaled scores <7 on BSID-3),13 (3) no clinical diagnosis of ASD or other behavioral disorders including attention-deficit/hyperactivity disorder, and (4) no hearing impairment or major physical impairment. Children with SLI were further selected on the basis of chronological age (<42 months). All participants with SLI demonstrated impairments in expressive language, and 5 demonstrated impairments in receptive language as well. Levels of social behavior for the SLI group were measured with the same subscales of Behavior Rating Scales in the BSID-2 as measured in the ASD group (see Table 1 for a summary of participant characteristics and Supplemental Table 1 for detailed scores of participants with SLI).
Because this study was designed to compare SLI participants with the participants in the previous study, we could not completely match the ages in the 3 groups. To control for the lack of matching for age, a one-way analysis of variance and a post hoc test (Ryan’s method)14 were performed. The results showed that group differences in age were not significant [F(2,63) = 0.76, P = .47, η2 = 0.02]. All participants had normal vision, and none of the children had known neurologic or genetic disorders. This study received approval from the institutional ethics committee, and all parents of participants provided written informed consent according to institutional guidelines.
Apparatus and Procedure
Video clips were displayed on a 17-inch color screen (full-screen resolution of 640 × 480 pixels). Eye movements were recorded at 120 Hz with a remote eye-tracker system (Tobii 1750, Sweden) placed below the screen. Participants were accompanied by a parent throughout the task and sat on a chair ∼60 cm from the screen by themselves or on their parent’s lap. A 5-point calibration procedure was performed before recording.
The same stimuli shown in the previous study were used.11 It was 77 seconds long and consisted of 12 short video clips excerpted from a film or TV program for children. Each clip lasted ∼6 seconds and contained 1, 2, or more human characters engaged in a social interaction, such as conversation or speaking to the audience.
Data obtained during the previous study were used for the ASD and TD groups. Because the eye data in the previous study were recorded at 50 Hz, we transformed the data for the SLI group to 50 Hz for analysis. Analyses of eye movements were performed by using the average of both eye positions at each of 3850 time points (50 Hz × 77 seconds). Eye-position data were included only when data from both eyes were available. Two additional children were tested but excluded for unsuccessful calibration and brevity (ie, <27 seconds). Average total valid viewing time for each group was 54 ± 13 seconds for the SLI group, 68 ± 7 seconds for the TD group, and 57 ± 12 seconds for the ASD group.
Full Gaze Pattern Analysis
To examine overall similarities and differences in the gaze patterns of participants, we calculated the direct distance between every pair of gaze at each of 3850 time points. If the 2 participants looked at the same point in the screen, the distance was zero. If the 2 participants looked 100 pixels apart, the distance is 100. The median of 3850 distances was taken as the distance between the pair. This procedure was repeated for each of 66C2 pairs to complete a distance matrix. Then we applied multidimensional scaling (MDS)15 to this matrix to represent every participant in a 2-dimensional plane. As a result of this procedure, participants with similar temporospatial gaze patterns were plotted close together near the center of the plane, and those with atypical gaze patterns were scattered toward the periphery. Temporospatial gaze patterns through all 12 clips were summarized on the MDS plane. We used Matlab statistics toolbox (Mathworks, Natick, Massachusetts) for calculations.
To quantify each participant’s deviation from a standard gaze pattern, the distance from the median of the entire distribution in the MDS plane (MDS distance) was calculated and compared among groups.
By using a frame-by-frame analysis, the proportions of time spent viewing major targets such as faces of major characters and objects were compared across groups. Targets had been identified in our previous study by hand in all 2327 frames.11 According to the distance between the gaze position and the target, a value from 0 to 1 was assigned by using a Gaussian function. The sum of the assigned values divided by the total viewing time was defined as the proportion of time viewing the target.
Eye and Mouth Viewing Proportion Analysis
To examine differences among groups in the allocation of attention while scanning faces, the proportions of time viewing eyes, mouth, and face (eyes, nose, mouth, and ears) were compared. We analyzed eye and mouth viewing behavior in clips 5, 9, and 11 because these clips featured a main character with his or her face large enough (>60 pixels = 3°) to discriminate eyes from mouth reliably, considering the spatial resolution of the measurement system with the spatial resolution of 10 pixels (0.5°). Furthermore, we created gaze maps for each group to better understand the distributions of gazes during speech scenes. The gaze position in each frame was corrected for changes in the position of faces in each frame by using an affine transformation and mapped onto a template frame chosen for each scene.16 Briefly, both eyes and the mouth were identified as 3 separate points in each frame, and the gaze point in the frame was represented in the oblique coordinate defined by the mouth and the 2 eyes (face coordinate). The gaze point in each frame was then mapped on the template frame according to the “face” coordinate, and the sum of the gaze positions for each group was shown as a gaze map on the template frame.
Full Gaze Pattern Analysis
The TD and SLI groups each formed a cluster near the center of the MDS plane, whereas the ASD group was scattered toward the periphery (Fig 1A). This pattern indicated that the TD and SLI groups showed similar gaze patterns within each group, whereas members of the ASD group showed atypical gaze patterns that also differed among members of that group. The average MDS distance was longest for the ASD group, which differed significantly from those of the SLI and TD groups according to the one-way analysis of variance [F(2,63) = 26.7, P < .001, η2 = 0.46] and post hoc tests (Fig 1B).
We also found a clear group difference between the positions of the SLI and TD groups on the MDS plane. Discriminant analysis on the MDS plane distinguished the SLI from the TD group, with discrimination levels of 0.81 for the SLI group and 0.84 for the TD group, indicating the presence of a distinct difference in the overall gaze patterns of the 2 groups.
Cross-sectional analysis of covariance was used to adjust the influence of age on MDS distance. MDS distance was entered as the dependent variable and group as the independent variable, and age was used for matching (chronological age for the TD and the SLI group, developmental age for the ASD group) as covariate. There was no significant effect of group × age interaction [F(2,60) = 1.0, P = .36, η2 = 0.04] or age [F(1,60) = 1.1, P = .31, η2 = 0.02] on MDS distance.
The MDS distance did not correlate with gaze duration or chronological age in any of the 3 groups. In the SLI group, no significant correlation was found with nonverbal developmental age (r = –.04), receptive verbal age (r = –.13), or expressive verbal age (r = –.25).
To clarify the cause of the between-group differences on the MDS plane, we examined proportions of participants viewing specific targets in each frame and compared the temporal profiles of the viewing proportion across the groups. We paid particular attention to clip 4, in which 2 boys talked in turn, because frame-by-frame analysis in our previous study had revealed a typical difference between the ASD and TD participants.11 TD children initially viewed the boy on the right who showed his full face to the audience (from 0.8–1 seconds), then shifted their gazes to the left boy as he looked up and said “Let’s go and see” from 1 to 1.9 seconds (Fig 2A). As soon as he finished his phrase, they shifted their gazes to the right boy again and fixated on his face until the end of his speech (“How do we go?”; 4.8 seconds). In contrast, gazes of children with ASD scattered, and they could not keep their gaze fixed until the end of the boy's speech. As a result, the greatest difference in the viewing proportion occurred toward the end of each speech (1.9 seconds for the left boy and 4.8 seconds for the right boy). Data from TD children (blue traces and dots) and children with ASD (green traces and triangles) are shown in Fig 2 A–D.
Gaze behaviors of children with SLI were generally similar to those in TD children. At 1.9 seconds, most children with SLI were looking at the boy on the left. At 4.8 seconds, they were looking at the boy on the right (red squares, Fig 2 E and F). The viewing proportions of the dominant characters during the last 0.5 seconds of the speech periods (shaded areas in Fig 2 A and B) were not significantly different between TD children and children with SLI but were different between TD children and children with ASD (Fig 2 G and H). The results show that children with SLI were paying attention to each speaker until the end of his speech as did the TD children, even though they were not able to understand what the speaker said.
Eye and Mouth Viewing Proportion Analysis
We examined if the group difference on the MSD plane could be explained by differences in eye and mouth viewing behaviors.
Compared with the TD children, children with SLI spent less time viewing the eyes (23% in SLI and 29% in TD children, Fig 3A), more time viewing the mouth (27% in SLI and 21% in TD, Fig 3B), and almost the same proportion viewing the face (69% in SLI and 70% in TD, Fig 3C). When the mouth viewing time was normalized to the face viewing time (Fig 3D), the mouth viewing time was significantly larger in children with SLI (40%) than in TD children (30%).
Cross-sectional analysis of covariance was used to adjust for the effect of age on mouth viewing time. The results showed that the main effects of the group [F(2,60) = 21.0, P < .001, η2 = 0.70] were significant, but the age [F(2,60) = 3.2, P = .08, η2 = 0.05] or the group × age interaction was not [F(2,60) = 0.01, P = .99, η2 = 0.00016]. When the interaction term was removed from the model, the effect of age still did not reach significance [F(2,62) = 3.3, P = .075, η2 = 0.05].
The results indicate that children with SLI showed even stronger mouth viewing behavior than did TD children, who had been characterized by their mouth viewing behaviors compared with TD adults.11 This strong mouth viewing behavior in children with SLI contrasted with children with ASD, who generally spent less time in viewing faces and lacked any particular preference to the mouth.
Gaze maps in Fig 4 support the group differences in the eye and mouth viewing behaviors. Figure 4 A–C is a gaze map while the girl was announcing her name in clip 11. Hotspots indicate that children with SLI concentrated on the mouth but that TD children spread their gaze more widely to include the mouth and a part of the girl’s right eye. Hotspots for children with ASD were widely distributed, with some looking at the caption below the image. Figure 4 D–F is a gaze map for clip 9, which featured a male gymnastics instructor singing a well-known nursery song in Japanese. Hotspots for children with SLI were clustered around the mouth with few reaching the eyes. Hotspots for TD children covered a broad region from the eyes to the mouth centered at the nose. Hotspots for children with ASD did not form a cluster in response to this clip. In an easy-to-understand situation, such as singing a well-known nursery song, TD children devoted substantial attention to the eyes, whereas in a novel situation, such as watching a girl announce her name for the first time, they fixated more on the mouth. This influence of the content of the scene on gaze behavior was not observed in children with SLI.
In the current study, we explored the gaze patterns of children with SLI while they viewed various dynamic social scenes and compared them with that of TD children and children with ASD. The MDS distance indicated that children with SLI showed gaze patterns similar to those of TD children, whereas children with ASD showed atypical gaze patterns. Frame-by-frame analyses revealed that children with SLI were as interested in social stimuli as were TD children, whereas children with ASD seemed inattentive to social stimuli. We also investigated the distributions of attention while viewing a speaking face and found that the SLI group spent increased time viewing the mouth.
Children with SLI and TD children viewed conversations in similar ways; they shifted their gazes from 1 speaker to another at similar timings and fixated on the speaker until the end of speech. In contrast, children with ASD looked at faces less frequently throughout a scene, and their gazes shifted out of the context of the scene.11 This inattention to social stimuli (including faces and social context) has been reported in previous studies.17,18 On the other hand, TD infants have shown preference for social stimuli from an early stage.19,20 In our study, this preference for social stimuli was conserved in young children with SLI as well as in TD children. This result indicates a distinct difference in the processing of social stimuli by children with ASD versus by those with SLI, and it may also suggest a distinct difference in the nature of these disorders.
One striking finding to emerge from this research was the increased attention paid by children with SLI to the mouth while viewing speech scenes.
In our study, TD children were fixated on both the eyes and the mouth while viewing speech scenes. They naturally shifted their gazes according to the difficulty of the speech. This result is consistent with the results of other audiovisual perception studies. Because eyes provide abundant information on emotion, identity, and social phenomena,21–23 the studies suggest that perceivers viewed the eyes to gather social and emotional information, whereas they viewed the mouth to gather visual information related to speech.24,25 This integration of auditory and visual information occurs naturally (seen in infants as young as 4 months old) and may influence language development.26,27
In contrast, children with SLI focused on the mouth of the speaker, devoting scant attention to the eyes even when viewing a performance of a well-known nursery song. It may be that processing auditory stimulus requires more effort for children with SLI than for TD children. Studies have revealed that information-processing dysfunctions at various levels (ie, auditory processing) seem to be core features of SLI.28 Norrix et al reported a weaker McGurk effect in children with SLI compared with TD peers, demonstrating the reduced influence of visual information on speech perception in SLI. They suggested that children with SLI may have difficulty in audiovisual processing as well as in auditory processing.29 Taken together with our results, this finding suggests that children with SLI may view the mouth region longer to support their weak auditory processing. However, their difficulty in audiovisual processing may not enable efficient compensation.
The MDS distance was significantly greater in children with ASD than in TD children and in those with SLI. In our previous study, the MDS distance in adults with ASD correlated with the autism-spectrum quotient30 but not with verbal IQ or performance IQ, suggesting that the MDS distance reflected, at least in part, core social deficits in ASD.11 In the current study with children, we confirmed that the MDS distance did not correlate with the chronological age or developmental age. The results in the present and previous studies suggest that the MDS distance may reflect at least a part of the core deficits in ASD and possibly impairments in the social perception of children. That the MDS distance in children with SLI was as small as that in TD children suggests that social impairments were not obvious in children with SLI. This was also confirmed by the frame-by-frame analysis of clip 4; children with SLI shifted their gazes between the 2 boys in similar ways as TD children did in accordance with the context of their speech. We expect that the MDS distance will, to some extent, help in the differentiation between children with ASD and children with SLI.
Although the mean distance from the center of the MDS plane was not significantly different between children with SLI and TD children, it is worth noting again that the 2 groups formed different clusters near the center of the MDS plane. This indicates that there was a subtle but distinct difference between the ways the 2 groups viewed the video stimuli full of social scenes at 2 to 3 years of age. Significantly increased viewing of the mouth and the lack of an efficient shift between the eyes and the mouth may be 1 explanation for the difference between the gaze patterns of the 2 groups. Furthermore, the results lead to a speculation that the limited time spent gazing at the eye region while viewing speech communication may result in less social and emotional information being obtained from the eyes, resulting in misunderstandings and poor social communication skills in the long run as the children with SLI grew older. Taken these findings in young children with SLI together, social difficulties associated with children with SLI may occur adaptively after the age of 2–3 years old, in spite of their basically normal social perception during the initial period of development.
Social perception in young children with SLI was similar but not completely equivalent to that of TD children. Longitudinal follow-up is needed to explore whether gaze behaviors and fixation patterns change as a function of development and whether these changes and the future social behavior or language development of children with SLI are related.
- Accepted January 31, 2012.
- Address correspondence to Kyoko Tanaka, MD, PhD, Department of Pediatrics and Adolescent Medicine, Juntendo University Faculty of Medicine, 2-1-1 Hongo, Bunkyo-ku, Tokyo 113-8421, Japan. E-mail:
The authors declare that all authors participated sufficiently in the work and meet the following criteria: (1) substantial contributions to conceptions and design, acquisition of data, or analysis and interpretation of data; (2) drafting the article or revising it critically for important intellectual content; and (3) final approval of the version to be published.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
- Nakano T,
- Tanaka K,
- Endo Y,
- et al
- ↵Bayley N. Bayley Scales of Infant and Toddler Development—Third Edition: Technical Manual. San Antonio, TX: Psychological Corporation; 2006
- Day RW,
- Quinn GP
- Kruskal J,
- Wish M
- Nakano T,
- Yamamoto Y,
- Kitajo K,
- Takahashi T,
- Kitazawa S
- Farroni T,
- Johnson MH,
- Menon E,
- Zulian L,
- Faraguna D,
- Csibra G
- Kuhl PK,
- Meltzoff AN
- Copyright © 2012 by the American Academy of Pediatrics