ReviewNeural systems for recognizing emotion
Introduction
Emotional signals, either visual or auditory, can be considered as aspects of both an emotional response and social communication. There are multiple schemes for categorizing emotions (Table 1). This review focuses on so-called basic emotions that can be most reliably recognized from facial expressions: happiness, surprise, fear, anger, disgust, and sadness. Most brain structures that participate in the recognition of basic emotions involve both perceptual processing — identifying the geometric configuration of facial features in order to discriminate among different stimuli on the basis of their appearance — and recognition of the emotional meaning of a stimulus — knowing that a certain expression signals fear. Recognition relies on disparate strategies. For instance, recognition of fear from a facial expression may occur by linking the perceptual properties of the facial stimulus to various knowledge-based processes. These include the knowledge components of the concept of fear, the lexical label ‘fear’, the perception of the emotional fear response (or a central representation thereof) that the stimulus triggers in the subject, or knowledge about the motor representations required to produce the expression shown in the stimulus [1••].
Section snippets
Recognition of emotion from facial expressions
A large number of different structures participate in recognizing the emotion shown in a face: the occipitotemporal cortices, amygdala, orbitofrontal cortex, basal ganglia, and right parietal cortices, among others. These structures are engaged in multiple processes and at various points in time, making it difficult to assign a single function to a structure (Fig. 1).
Visual cortices
Regions of the occipital and posterior temporal visual cortices play a critical role in perceptual processing of socially and emotionally relevant visual stimuli. Single-unit studies in monkeys, intracranial field potential studies in neurosurgical human patients 2., 3., 4. and functional imaging studies, have all provided evidence that cortical areas in the lateral parts of the inferior occipital gyrus, fusiform gyrus, and superior temporal gyrus are disproportionately important in face
The amygdala
The amygdala participates in the recognition of emotional signals via at least two classes of input mechanisms: a subcortical route via the superior colliculus and the pulvinar thalamus, and a cortical route via the visual neocortex. Structures in the subcortical route are activated both when normal subjects are shown subliminal facial expressions of fear [16], and when subjects with blindsight, due to striate cortex damage, discriminate emotional facial expressions 17., 18•.. Subliminally
Orbitofrontal cortex
Damage to the orbitofrontal cortex, especially on the right, can result in impaired recognition of emotions from the face and the voice [50]. These findings are consistent with the activation found in right orbitofrontal cortex when comparing presentations of fearful and neutral faces [51••]. In contrast to the amygdala's activation in response to passive viewing of emotional faces or gender judgements, prefrontal regions may be activated when subjects are engaged in a cognitive task requiring
Somatosensory related cortices and the basal ganglia
Following a large number of lesion studies that demonstrated a role for right frontoparietal cortices in emotion recognition, an investigation of the association between lesion location and facial emotion recognition provided evidence that somatosensory related cortices in the right hemisphere played a critical role [58•]. The study revealed a consistent pattern for all emotions: lesions in the right ventral primary and secondary somatosensory areas, and, to a lesser extent in the insula and
Summary of recognizing emotion from faces
Upon presentation of an emotionally meaningful stimulus, one might envision a first, feed-forward sweep of information processing. This would proceed along occipital and temporal neocortices and extract perceptual information from faces and, after ∼100 ms in humans, would coarsely categorize the stimulus as expressing an emotion or not, on the basis of the structural properties of the image. Amygdala and orbitofrontal cortices could participate in the processes of emotion recognition from the
Recognition of emotion from prosody
Studies with computer-generated stimuli have provided evidence supporting two theories of emotion recognition from the patterns of stress and intonation during speech. First, data corroborate the idea that one or two simple auditory cues can be used by subjects to provide much of the information about the emotion in a prosodic stimulus, such as Fo range and amplitude variation. Second, findings also verify that the total number of cues available that can influence emotion judgements is large
Recognizing emotion in other sensory modalities
Our understanding of the neural correlates of emotion recognition in sensory modalities other than vision and audition is rudimentary, in large part because none of the studies have directly investigated recognition. Single-cell responses in animals have been reported to emotionally salient olfactory and gustatory stimuli especially in amygdala and orbitofrontal cortex (see [88] for review). Olfactory 85•., 89., 90. and gustatory [91] stimuli have been found to activate the human amygdala when
Conclusions and future directions
Some open questions regarding emotion recognition are indicated in Box 1. Although the recognition of emotion from facial expressions has been extensively investigated, little is known about the neural structures participating in recognition of emotion from other visual cues, such as body posture, from auditory cues, or from the integration of cues from multiple sensory modalities. Likewise, next to nothing is known about the neural basis of recognizing ‘social’ emotions (Table 1). Explorations
Update
In humans, the emotional reactions that can be triggered by stimuli also play a role in complex aesthetic judgements. Building on lesion studies that have shown dissociations between identifying melodies or recognizing emotion from music [96], a recent functional imaging study found that highly emotional music, which resulted in ‘shivers down the spine’ in the listener, activated a set of paralimbic structures including the ventral striatum, amygdala, and orbitofrontal cortex [97]. Just how to
Acknowledgements
The author is supported, in part, by grants from the National Institute of Mental Health, National Institute of Neurological Disorders and Stroke, the EJLB Foundation and the Klingenstein Fund.
References and recommended reading
Papers of particular interest, published within the annual period of review, have been highlighted as:
• of special interest
•• of outstanding interest
References (99)
- et al.
Social perception from visual cues: role of the STS region
Trends Cognit Sci
(2000) - et al.
The distributed human neural system for face perception
Trends Cognit Sci
(2000) - et al.
Exaggerated amygdala response to masked facial stimuli in posttraumatic stress disorder: a functional MRI study
Biol Psychiatry
(2000) - et al.
Spatiotemporal stages in face and word processing. I. Depth-recorded potentials in the human occipital, temporal and parietal lobes
J Physiol
(1994) - et al.
Face processing impairments after encephalitis: amygdala damage and recognition of fear
Neuropsychologia
(1998) - et al.
Recognition of facial emotion in nine subjects with bilateral amygdala damage
Neuropsychologia
(1999) - et al.
Amygdala lesion in temporal lobe epilepsy subjects impairs associative learning of emotional facial expressions
Neuropsychologia
(2001) - et al.
Response and habituation of the human amygdala during visual processing of facial expression
Neuron
(1996) - et al.
Amygdala response to facial expressions in children and adults
Biol Psychiatry
(2001) - et al.
Face and voice expression identification in patients with emotional and behavioral changes following ventral frontal lobe damage
Neuropsychologia
(1996)
Effects of attention and emotion on face processing in the human brain. An event-related fMRI study
Neuron
Are emotions contagious? Evoked emotions while viewing emotionally expressive faces: quality, quantity, time course and gender differences
Psychiatry Res
Saying it with feeling: neural responses to emotional vocalizations
Neuropsychologia
Recognition of emotional prosody and verbal components of spoken language: an fMRI study
Cognit Brain Res
Working memory of identification of emotional vocal expressions: an fMRI study
Neuroimage
Recognition of prosody following unilateral brain lesion: influence of functional and structural attributes of prosodic contours
Neuropsychologia
Lateralization of affective prosody in brain and the callosal integration of hemispheric language functions
Brain Lang
Intact recognition of emotional prosody following amygdala damage
Neuropsychologia
Beautiful faces have variable reward value: fMRI and behavioral evidence
Neuron
Recognizing emotion from facial expressions: psychological and neurological mechanisms
Behav Cognit Neurosci Rev
Electrophysiological studies of human face perception. I. Potentials generated in occipitotemporal cortex by face and non-face stimuli
Cereb Cortex
Electrophysiological studies of human face perception. II. Response properties of face-specific potentials generated in occipitotemporal cortex
Cereb Cortex
Electrophysiological studies of human face perception. III. Effects of top-down processing on face-specific potentials
Cereb Cortex
Domain specificity in face perception
Nat Neurosci
The fusiform face area: a module in human extrastriate cortex specialized for face perception
J Neurosci
FFA: a flexible fusiform area for subordinate-level visual processing automatized by expertise
Nat Neurosci
Distributed and overlapping representation of faces and objects in ventral temporal cortex
Science
Distinct representations of eye gaze and identity in the distributed human neural system for face perception
Nat Neurosci
Rapid emotional face processing in the human right and left brain hemispheres: an ERP study
Neuroreport
Cognitive response profile of the human fusiform face area as determined by MEG
Cereb Cortex
Global and fine information coded by single neurons in the temporal visual cortex
Nature
A subcortical pathway to the right amygdala mediating ‘unseen’ fear
Proc Natl Acad Sci USA
Non-conscious recognition of affect in the absence of striate cortex
Neuroreport
Differential extrageniculostriate and amygdala responses to presentation of emotional faces in a cortically blind field
Brain
Masked presentations of emotional facial expressions modulate amygdala activity without explicit knowledge
J Neurosci
Single trial analysis of neurophysiological correlates of the recognition of complex objects and facial expressions of emotion
Brain Topogr
Fear and the human amygdala
J Neurosci
Facial emotion recognition after bilateral amygdala damage: differentially severe impairment of fear
Cognit Neuropsychol
Expression without recognition: contributions of the human amygdala to emotional communication
Psychol Sci
Knowing no fear
Proc R Soc London Ser B
Neuropsychology of fear and loathing
Nat Rev Neurosci
The human amygdala and emotion
Neuroscientist
Impaired perception of facial emotions following bilateral damage to the anterior temporal lobe
Neuropsychology
Fear, vigilance, and ambiguity: initial neuroimaging studies of the human amygdala
Curr Dir Psychol Sci
Contribution of the anteromedial temporal lobes to the evaluation of facial emotion
Neuropsychology
Emotion recognition from faces and prosody following temporal lobectomy
Neuropsychology
A differential neural response in the human amygdala to fearful and happy facial expressions
Nature
Cited by (1516)
EEG decoding for musical emotion with functional connectivity features
2024, Biomedical Signal Processing and ControlMentalizing can be Impaired in Patients with Meningiomas Originating in the Anterior Skull Base
2024, World NeurosurgeryBidirectional relationship between insular grey matter volume and alexithymia: Evidence from a longitudinal study
2023, Journal of Affective DisordersAltered social cognition in early relapsing remitting multiple sclerosis
2023, Multiple Sclerosis and Related Disorders