Definition

Facial perception refers to the ability to rapidly recognize and understand information from faces. The ability to perceive faces and to use that information to guide and direct behavior plays a critical role in interpreting and forming representations from the social world and in the acquisition and understanding of reciprocal social interaction. Early facial recognition research suggests that humans innately attend to faces and are able to rapidly recognize faces remarkably early in life (Goren, Sarty, & Wu, 1975). Within 9 min after birth, infants turn their head to a 90° angle to preferentially attend to faces or face-like images. At 42 min after birth, infants can imitate facial expressions; and by 2 days old, infants can recognize the face of their mother. One-month-old infants show greater pupillary dilation to faces than to nonsocial stimuli suggesting greater interest and arousal in infants when looking at faces. By the end of the first year of life, infants are able to respond to directional gaze, emotional expression, and facial gestures. These early infant preferential face-directed behaviors mark the beginning of a highly protracted development in which repertoires for the face processing system become more sophisticated over time. The examination of facial perception in typical and atypical development, such as in autism spectrum disorders (ASD), has elucidated face perception processes across behavioral, anatomical, and neurophysiological domains and revealed differences in functioning in individuals with ASD and typical development in face perception.

Historical Background

Research with children with ASD has consistently revealed deficits in facial perception surfacing within the first 3 years of life. Children with ASD show impairments in social behaviors related to the perception and processing of information from faces. These impairments include reduced eye contact, joint attention, and social orienting, deficits in the imitation of faces and in face recognition, and attenuated responses to emotional displays of others (Dawson, Webb, & McPartland, 2005).

The retrospective examination of joint attention and gaze monitoring of toddlers through home videos of first birthday parties has provided insight into facial perception in ASD. In novel and uncertain situations, typically developing young children will seek affective cues from others’ faces. Given that a child’s first birthday can be a highly social, yet novel event, the setting provides the opportunity for a range of observable social behavior. Osterling and Dawson collected videotapes of 1st year birthday parties from children who were later diagnosed with ASD and from children with typical development. The tapes were rated by coders blind to the child’s subsequent diagnostic status. The authors found that the children later diagnosed with ASD looked at and attended to faces significantly less than the typically developing children. The authors determined that the single most critical factor in recognizing early signs of ASD was a failure to look at others and attend to faces in a typical manner, at or before 12 months old (1994).

Current Knowledge

The dynamic processes related to facial perception can be examined from behavioral, neuroanatomical, and neurophysiological perspectives. Much of our understanding of facial perception has been derived from clinical work and research with individuals with ASD. These impressions suggest that individuals with autism, compared to typical peers, perform worse on face discrimination, recognition, and memory tasks when behaviorally assessed. Additionally, individuals with ASD display atypical development and anatomical activation of the brain structures involved in face processing and exhibit delayed and differential neurophysiological responses to faces.

Behavioral Perspectives on Face Perception

Very early in life, typically developing individuals show an attentional preference for faces and are equipped for the specialization of face processing. These specialized abilities involve recognizing upright faces better than inverted faces and interpreting complex interactions such as direction of gaze, emotional expressions, and facial gestures. Into middle childhood, the specialization for facial perception grows in complexity. Typically, individuals show better memory for faces than objects. They also derive meaningful social-emotional information from faces through preferential attention toward the eyes and perceive faces as a holistic unit rather than isolating individual facial features (Joseph & Tanaka, 2003). In contrast, individuals with ASD attend to inverted faces for the same amount of time as upright faces and recognize inverted faces better than typical individuals by focusing on isolated features of the face rather than the face as a whole (van der Geest, Kemner, Verbaten, & van Engeland, 2002). Further, when viewing faces, individuals with ASD fixate on the mouth region rather than the eyes. Accordingly, they exhibit better performance memory for the lower portion of the face than the upper portion of the face but show a decrease in attention to the entire internal region of the face. As a result, individuals with ASD perform in the below average range on tasks of overall face memory ability. Further, individuals with ASD show impairments in the ability to use information from faces to guide behavior, such as failing to monitor a gaze shift and follow a gaze to share attention.

Studies of facial discrimination and recognition using eye tracking suggest that individuals with ASD employ atypical strategies during facial perception tasks. First, individuals with ASD attend to other objects in the environment more often than the face. For example, a pioneering study using eye tracking found that when watching a videotape of the movie Who’s Afraid of Virginia Woolf, individuals with ASD attended two times more to the actors’ mouth, bodies, and objects in the scenes than to the actors’ eyes (Klin, Jones, Schultz, Volkmar, & Cohen, 2002). This suggests that individuals with ASD do not seek social information and cues from naturalistic exchanges in the way that typical individuals seek social information. Subsequent studies have shown that individuals with ASD utilize atypical spatial distribution fixation strategies exemplified by fixating to, or avoiding specific regions of the face, focusing on the mouth, rather than the typical direction toward the eyes.

A second atypical face perception strategy that individuals with ASD appear to use involves processing faces utilizing a piecemeal (i.e., feature-based) strategy (Chawarska & Shic, 2009). In contrast, typical individuals employ the piecemeal processing strategy when looking at objects but not faces. Typical individuals attend to faces holistically by utilizing a complex-looking strategy called configural processing (Dawson et al., 2005). Configural processing is the perception and encoding of faces as a single holistic unit by incorporating the major features (i.e., eyes, nose, mouth), in respect to other features throughout the face.

To explore the limitations of the configural strategy employed by the typical population, researchers alter the configuration of faces by inverting and partially obscuring the images.

Typical individuals are adept at distinguishing similarities between faces that are upright; however, when the face is inverted, it is more difficult to discern facial feature differences. Inversion appears to alter the perception of the face by challenging the relational facial features (i.e., eye, nose, mouth), which contests the configural processing looking strategy producing what is called the inversion effect. While inverting faces results in the inversion effect, altering a face by forcing one to focus on individual facial features results in the decomposition effect. Even though typical individuals are adept at perceiving faces, once the face is altered from its original composition (e.g., inverted or presented in sections), the accuracy in which the face is identified is challenged. By challenging the configural processing strategy in these two ways, typical individuals have to abandon a configural processing strategy when looking at faces and adopt a piecemeal strategy as if they were looking at an object. This concept provides support that face perception is a highly specialized and sensitive processing system, which when challenged, forces individuals to compensate using differential processing strategies and neuroanatomical mechanisms that are not specialized for face perception tasks.

From studies exploring the configural processing strategy, the decomposition effect and the inversion effect, researchers have found that typical individuals attend to inverted faces for longer lengths of time, are better at recognizing parts of the face when presented in the context of the entire face, and are better at recognition of the eyes compared to the mouth than their peers with ASD (van der Geest et al., 2002). When presented with the inversion face perception task, individuals with autism spend an equal amount of time looking at upright and inverted faces. This is similar to the way typical individuals attend to upright and inverted objects. By employing the object specific, piecemeal processing strategy, individuals with ASD are better at perceiving specific features of the face or partially exposed sections of the face with emphasis on local detail rather than global detail. In fact, individuals with ASD are in some cases better at feature-based matching, especially around the mouth, because they are not inhibited by the implementation of decomposition effect.

These behavioral-based face perception studies suggest that typically developing individuals use the configural processing strategy to perceive faces and the piecemeal processing to perceive objects while individuals with ASD appear to apply the piecemeal strategy to process both faces and objects. The benefits of configural processing strategies are reflected in better performance on face discrimination and recognition tasks and better memory for faces relative to objects for typical individuals. In contrast, children with ASD perform equally to typical peers and in some cases better on nonface memory recognition tasks that include objects such as buildings, animals, houses, and bicycles (Blair, Frith, Smith, Abell, & Cipolotti, 2002).

Neuroanatomical Perspectives on Face Perception

Much of our understanding of the neuroanatomical structures related to face perception has been driven by research with individuals with impairments in face perception. Studies of the brain in individuals with prosopagnosia, a disorder characterized by intact object recognition ability, but impairment in the ability to recognize faces, provide a model for face perception that can be applied to ASD. Neuroanatomical findings in individuals with prosopagnosia, who exhibit behavioral face perception impairments similar to those found in ASD, indicate damage to particular cortical regions, such as the fusiform gyrus, indicating a critical role in face perception for this neural region (De Renzi, Perani, Cariesimo, Silveri, & Fazio, 1994). By utilizing functional magnetic resonance imaging (fMRI), structural magnetic resonance imaging (MRI), and positron-emission tomography (PET), neuroimaging research has provided further evidence that occipitotemporal cortical and other neural structures are functionally abnormal in individuals with ASD during facial perception. The regions identified as playing a primary role in face processing include the fusiform gyrus, the posterior superior temporal sulcus, and the amygdala. This dynamic human face processing system that is linked to the structural encoding of facial features, processing emotional expressions, and interpreting biological movement is critical to understanding the fundamental neural circuitry of the “social brain” (Pelphrey, Morris, McCarthy, & LaBar, 2007).

Fusiform Gyrus

Research over the last decade has verified a region of the fusiform gyrus that activates more strongly to faces than any other visual stimuli and appears to be specialized for face processing and facial expression discrimination. The fusiform face area (FFA), located in the medial-lateral region of the fusiform gyrus, responds almost twice as strongly to faces than wide varieties of nonface objects (Kanwisher, 2000). However, additional research adds complexity to the role of the FFA region. After finding that the FFA is activated while viewing objects that fall in an individual’s area of expertise (e.g., car experts display FFA activation when viewing images of automobiles), Gauthier, Tarr, Anderson, and Skudlarski proposed that the FFA region is not only responsible for face perception but is responsible for the mediation of specific visual expertise. Gauthier and colleagues supported this conclusion by showing that training a group of college students to identify computer generated 3-D objects (called Greebles) results in an increase of activity of the FFA (1999).

Rather than increased activity in the fusiform gyrus when looking at faces as seen in typical development, neuroimaging findings suggest that individuals with ASD display reduced activation of the fusiform gyrus while looking at faces (Schultz et al., 2000). Further, individuals with ASD exhibit an increase in activation in the inferior temporal gyri, a surrounding anatomical region specialized for object recognition. While for typical individuals, viewing objects of expertise, such as faces, results in FFA activation, individuals with ASD show FFA activation only when looking at images of their specialized interest. While the fusiform gyrus is an area linked to facial perception in typical individuals, for individuals with ASD, this “social brain region” fails to activate to faces in the same manner.

Amygdala

While there is strong research to support the FFA role in the perception of faces, the amygdala plays a significant role in detecting emotional expressions on faces. The amygdala is a fast-responding structure that plays a critical role in notifying other brain regions of emotional arousal through a mediated reward system. This structure quickly and reliably reacts to salient environmental stimuli, especially fear, and plays a role in assigning significance and constructing social judgments, such as inferring personality characteristics from pictures of the face or facial features (Ledoux, 1996).

The allure of the amygdala in ASD research was initially recognized in postmortem cases of individuals with ASD who appeared to have limited dendritic tree development resulting in an amygdala that was small and densely packed compared to typical individuals (Bauman & Kemper, 1985). Adolphs found that individuals with ASD demonstrated similar impairments to those with focal lesions to limbic structures, including the amygdala. While individuals with ASD are able to form representations of faces and understand basic knowledge of emotions, they are unable to link the perception of faces to social judgments that are often revealed by facial expressions. Neuroimaging studies of individuals with ASD have reported hypoactivation of the amygdala when viewing images of emotional and mental states of another individual. For example, Pelphrey and colleagues demonstrated that when emotional faces were presented naturalistically through a dynamic video display, all three components of the broader social brain – the amygdala, the superior temporal sulcus, and the fusiform gyrus – were hypoactive (2007). A study examining the perception of emotionally expressive faces in individuals with amygdala and hippocampal lesions found that there was a strong correlation between the extent of amygdala lesions and hypoactivation of the FFA, suggesting that the amygdala plays a direct role on the FFA when interpreting emotionally expressive face (Vuilleumier, Armony, Driver, & Dolan, 2004).

Superior Temporal Sulcus (STS)

The STS region of the brain is responsible for the detection and processing of biological motion (Pelphrey et al., 2007). In synchrony with the amygdala and fusiform gyrus, this neural network is important for linking structural encoding of facial features and emotions to biological motion. Perception of biological motion is thought to underlie impairments in attention and imitation, skills that are strongly linked to language and social development.

Since past research examined the face processing system through the display of emotional expressions using still image stimuli, Pelphrey and colleagues (2007) were interested in how presenting dynamic facial expressions affects the face processing system. They found that individuals with ASD display reduced activation in the STS, fusiform, and amygdala when faces contain dynamic information, such as movement, relative to static snapshots of the same emotional expressive face. These results were specific to the social brain and were not observed outside of the human face processing system (i.e., STS, fusiform gyrus, and amygdala).

The STS also plays a role in comprehending intentions and detecting “errors” in social situations (Pelphrey et al., 2007). This region modulates tasks involving the attribution of intention in the context of dynamic social situations. During situations in which an individual’s actions do not fulfill the expectation (e.g., a character in a story is not looking in the direction that the viewer would think he/she should), there was an increase in blood flow to the STS in a typical observer.

However, for individuals with ASD, there was reduced activity in the STS region. The lack of observed activity in the STS during attribution tasks in ASD provides further evidence that the STS region plays a strong role in the social dysfunction in ASD.

Neurophysiological Perspectives on Facial Perception

By utilizing neurophysiological measures such as event-related potentials (ERPs), scientific studies have provided insight into the neuropsychological processes underlying face perception and face perception impairment in ASD. While fMRI studies have allowed researchers to efficiently investigate abnormal regional brain activity during facial performance tasks, electroencephalogram (EEG) and event-related potentials (ERPs) have allowed researchers to examine the temporal characteristics and strength of neural activity by assessing oscillatory activity in response to viewing faces. Unlike fMRIs, which cannot determine the timing of neuronal firing, EEG/ERPs are able to monitor the timing of the summation of a collection of neuronal postsynaptic potentials firing at the resolution of milliseconds. Most studies utilizing EEG data concentrate on the amplitude, scalp topography, and latency. Amplitude is considered a representation of the amount of neuronal resources recruited for a particular process. Scalp topography distribution differences are thought to represent the location of the originally generated neuronal activity, and the timing of information processing speed is represented through the wave-like display of troughs and peaks, known as latency. Because simply passive viewing of stimuli is needed for EEG/ERP studies, valuable information can be gathered from children with significant social communication impairments through the use of EEG/ERP to effectively and noninvasively address fundamental questions about the neural basis of face processing in individuals with ASD.

Electrophysiology studies are particularly salient in face processing research because of the discovery of a face-sensitive component that occurs with a negative slope at approximately 170 milliseconds poststimuli exposure. The N170 represents one stage in a series of stages of face processing; the P100, a positive going peak around 100 milliseconds, reflects neural activity in basic visual processing; the N170 reflects structural encoding of the face; and the N250 reflects higher order recognition such as affect or identity recognition. The N170 is recorded over the posterior temporal lobe and is greater in the right hemisphere than the left hemisphere. The N170 responds quicker to face and eye stimuli alone, rather than objects or inverted faces. Slight changes in face composition or inversion alter the latency and amplitude of this component. In typical children, the N170 undergoes a prolonged period of development and does not reach full maturation until late adolescence. Between 4 and 15 years of age, the precursor to the N170, identified as the prN170, is measured from the same electrode configuration as the N170 in adults but is more positive (does not extend beyond the baseline) than the fully maturated N170 component in adults. The prN170 in children ages 3–6 years is shown to have a faster response to faces. Similar to adults, this component is shown to have preferential responses to faces than objects.

McPartland, Dawson, Webb, and Panagiotides documented the first report of an altered N170 pattern in adolescents and adults with ASD (2004). Individuals with ASD showed a slower latency when looking at faces than furniture and failed to show the face inversion effect. Additionally, researchers observed a slower speed of information processing which disrupts early structural encoding. These differences in the timing of cortical processing suggest that individuals with ASD have abnormal circuitry or delayed neural processes that modulate the face processing system.

Children with ASD also show disruptions to this physiological measure of face processing. They demonstrate longer prN170 responses to faces than objects. By the age of 6, compared to typically developing peers who continue to show faster responses to faces over objects, children with ASD show no latency differences to faces versus objects (Webb, Dawson, Bernier, & Panagiotides, 2006). In addition, research indicates that children with ASD display structural encoding of faces that is disrupted and slowed. While typically developing children exhibit a faster prN170 response to faces than objects, individuals with ASD exhibit a slower prN170 response to faces than objects.

Differential activation patterns have been observed using ERP studies in typical children comparing familiar and unfamiliar stimuli (i.e., caregivers to strangers, familiar objects to unfamiliar objects) as early as 6 months. This suggests early preferential neurological responses to familiar over unfamiliar faces and objects (de Haan & Nelson, 1999). Typically developing infants and young children display ERP differences when viewing familiar and unfamiliar faces and objects at the P400 component recorded from posterior electrodes. Children with ASD, however, fail to show a differential P400 component when looking at familiar versus unfamiliar faces. Instead, children with ASD display an enhancement in the P400 component when looking at favorite objects versus unfamiliar objects. These results provide evidence that children with ASD have specific impairment in processing faces, but not objects.

Children with ASD also show abnormal electrophysiological responses when viewing emotional expressions. Compared to typical peers, children with ASD show a slower and more positive N300 wave in response to fear and exhibit abnormal scalp topography. This slower information processing for faces in the N300 for emotional stimuli is associated with greater severity in social domains such as joint attention and social orienting (Dawson et al., 2004).

Neurophysiological studies in early childhood provide insight into the structural and neurological mechanisms that are involved in the development of face processing. Face processing impairments examined at the neurophysiological level provide insight into social brain functioning and the face processing system in individuals with ASD from infancy and throughout development.

Future Directions

Faces hold special significance and convey valuable social information early in life. Face perception provides a foundation for the development of social cognitive skills such as sharing attention and understanding emotions. Given the prominent role of face perception in social cognition, the study of ASDs has provided insight into the behavioral, neuroanatomical, and neurophysiological components of face perception.

Behavioral studies of face perception in individuals with ASD have yielded insight into visual scanning, face processing strategies, and the role of expertise of processing information. Imaging studies have identified structures related to the processing of faces, such as the fusiform gyrus, superior temporal sulcus, and amygdala, and demonstrated that the structures of the “social brain system” involved in face perception appear to be functioning differently in ASD. Neurophysiological studies have elucidated face-specific brain waves and revealed temporal delays in the processing of faces in ASD.

The ability to process and gather information from facial expressions is crucial for successful social interactions. This unique ability allows humans to understand affective cues in uncertain situations, such as situations that may be dangerous and important for survival, and to accurately read the emotions of another individual. This ability provides attentional and directive focus through gaze monitoring of others and provides a catalyst to overtly and covertly mimic the facial expressions and affective states of others, such as states that offer the opportunity for bonding or shared reciprocal experience (Chawarska & Shic, 2009). While face perception provides immediate insight and interpretation of the social world, over time, and development, it is clear that early developing face processing skills set the foundation for the maintenance of exchanges and social relationships.