Review
Salience, relevance, and firing: a priority map for target selection

https://doi.org/10.1016/j.tics.2006.06.011Get rights and content

The salience map is a crucial concept for many theories of visual attention. On this map, each object in the scene competes for selection – the more conspicuous the object, the greater its representation, and the more likely it will be chosen. In recent years, the firing patterns of single neurons have been interpreted using this framework. Here, we review evidence showing that the expression of salience is remarkably similar across structures, remarkably different across tasks, and modified in important ways when the salient object is consistent with the goals of the participant. These observations have important ramifications for theories of attention. We conclude that priority – the combined representation of salience and relevance – best describes the firing properties of neurons.

Introduction

The complexity of the visual world exceeds the processing capacity of the human brain [1], which forces us to select one (or a few) object(s) in the scene for more detailed analysis at the expense of other items [2]. This act of selection provides a succinct definition of the term ‘visual attention’. In this article, we explore one basic issue surrounding this selection process – how do we choose the next object of attention 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13?

The salience map provides one conceptual framework that accounts for how the next object is selected. It consists of a topographical map of space, upon which all objects in the visual scene compete. Only the physical distinctiveness of objects factor into this competitive process and, over time, the most salient object is chosen in a winner-take-all fashion [4].

Many disciplines have converged upon the same basic idea: computational and psychological models have implemented concepts similar to the salience map when describing how an object is selected 4, 8, 10, 11, 14, 15, 16, 17, 25. Over the last several years, the spiking patterns of single neurons have been likened to the salience map as well 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29. It is both an exciting prospect and a mark of true convergence across disciplines that concepts put forward in psychological and computational models have evidence in patterns of neural activity in the brain. Now it is time to integrate what we know and use this knowledge to direct future research. Here, we attempt to meet these ends by: (i) defining salience and the salience map; (ii) using these definitions to constrain the neurophysiological evidence and then summarizing these findings; (iii) identifying features of neural data that are not considered by the salience map; and (iv) describing the ramifications of this evidence.

Section snippets

Defining salience and the salience map

The definition of salience provided in psychological and computational theories is very specific – salience refers to the physical, bottom-up distinctiveness of an object 4, 8, 11, 13, 14, 15, 16, 17. It is a relative property that depends on the relationship of one object with respect to other objects in the scene 8, 10. This property of salience is highlighted in Figure 1a: it is much easier to detect the presence of a target in a display when it possesses a distinct feature (Figure 1a left,

Where is the salience map?

These properties constrain the neurophysiological evidence that should be taken to support the salience map. First, the task should be designed to encourage the bottom-up processing of items in the scene, as opposed to goal-directed selection. This is an important constraint because many neurophysiological investigations have used the terms ‘salience’ and ‘relevance’ interchangeably 18, 19, 21, 23, 24, 29, 30. Second, the neurons should be spatially selective, but otherwise encode visual

Expression of salience in the oculomotor network

There are two mechanisms by which the salience map operates that yield measurable consequences in behavior–salience effects (defined by the task) and inhibition of return. Over the last decade, much has been learned about how salience and inhibition of return are reflected in neural activity through the use of visual search and the non-predictive cue–target task (see Box 1 for interpretations of the neural correlates described here).

The priority map: combining salience and relevance

Salience and the salience map refer to bottom-up processes in attentional selection – neither the relevance of an object nor the goals of observers play any part in this conceptual framework. Yet the terms salience and relevance are often treated as synonyms in the neurophysiological literature 18, 19, 21, 23, 24, 29, 30. This indiscrepancy might reflect the ‘top-down’ knowledge that neurophysiologists bring to the issue – the relevance of an object influences how it is processed in oculomotor

Relationship between priority and action

How we choose the next object of attention seems to be closely related to how we choose the next target of a saccade [67], as shown through behavioral evidence 68, 69, functional imaging and neuropsychological investigations 35, 70. These correspondences raise the question–what is the relationship between selective attention and oculomotor action?

Computational and psychological theories treat the salience map and action as separate processes [4]. In the brain, however, the neural

Conclusions

The salience map is a concept upon which different disciplines have converged: psychologists and computational scientists have implemented the salience map in their models of attentional selection. In this article, our primary goal was to review the neurophysiological evidence taken to support the salience map. Our secondary goal was to assess if and how evidence from neurophysiology should modify our theoretical perspectives.

With regards to the first goal, we have provided ample evidence

References (84)

  • J.T. Serences et al.

    Selective visual attention and perceptual coherence

    Trends Cogn. Sci.

    (2006)
  • E. Awh

    Visual and oculomotor selection: links, causes, and implication for spatial attention

    Trends Cogn. Sci.

    (2006)
  • G. Rizzolatti

    Reorienting attention across the horizontal and vertical meridians: evidence in favor of a premotor theory of attention

    Neuropsychologia

    (1987)
  • H. Deubel et al.

    Saccade target selection and object recognition: evidence for a common attentional mechanism

    Vision Res.

    (1996)
  • A.C. Nobre

    Covert visual spatial orienting and saccades, overlapping neural systems

    Neuroimage

    (2000)
  • T.R. Sato et al.

    Effects of stimulus-response compatibility on neural 78selection in frontal eye field

    Neuron

    (2003)
  • J.K. Tsotsos

    Attending to motion

    Comput. Vis. Image Understand.

    (2005)
  • J.K. Tsotsos

    Analyzing vision at the complexity level

    Behav. Brain Sci.

    (1990)
  • W. James

    Principles of Psychology

    (1890)
  • H.E. Egeth et al.

    Visual attention, Control, representation, and time course

    Annu. Rev. Psychol.

    (1997)
  • L. Itti et al.

    Computational modeling of visual attention

    Nat. Rev. Neurosci.

    (2001)
  • R.A. Kinchla

    Attention

    Annu. Rev. Psychol.

    (1992)
  • M.I. Posner

    Orienting of attention

    Q. J. Exp. Psychol.

    (1980)
  • M.I. Posner et al.

    The attention system of the human brain

    Annu. Rev. Neurosci.

    (1990)
  • A. Treisman

    Features and objects, the Fourteenth Bartlett Memorial Lecture

    Q. J. Exp. Psychol A.

    (1988)
  • J.M. Wolfe

    The parallel guidance of visual attention

    Curr. Dir. Psychol. Sci.

    (1992)
  • J.M. Wolfe

    Guided Search 2.0, A revised model of visual search

    Psychon. Bull. Rev.

    (1994)
  • J.M. Wolfe

    Visual search

  • H. Pashler

    Attention and performance

    Annu. Rev. Psychol.

    (2001)
  • J.M. Findlay et al.

    A model of saccade generation based on parallel processing and competitive inhibition

    Behav. Brain Sci.

    (1999)
  • J. Theeuwes

    Top-down search strategies cannot override attentional capture

    Psychon. Bull. Rev.

    (2004)
  • A. Treisman

    Features and objects in visual processing

    Sci. Am.

    (1986)
  • N.P. Bichot et al.

    Effects of similarity and history on neural mechanisms of visual selection

    Nat. Neurosci.

    (1999)
  • J.W. Bisley et al.

    Neuronal activity in the lateral intraparietal area and spatial attention

    Science

    (2003)
  • J.H. Fecteau

    Neural correlates of automatic and goal-driven biases in orienting spatial attention

    J. Neurophysiol.

    (2004)
  • J.H. Fecteau et al.

    Exploring the consequences of the previous trial

    Nat. Rev. Neurosci.

    (2003)
  • J.H. Fecteau et al.

    Correlates of capture of attention and inhibition of return across stages of visual processing

    J. Cogn. Neurosci.

    (2005)
  • M.E. Goldberg

    The role of the lateral intraparietal area of the monkey in the generation of saccades and visuospatial attention

    Ann. N. Y. Acad. Sci.

    (2002)
  • R.M. McPeek et al.

    Saccade target selection in the superior colliculus during a visual search task

    J. Neurophysiol.

    (2002)
  • R.M. McPeek et al.

    Deficits in saccade target selection after inactivation of superior colliculus

    Nat. Neurosci.

    (2004)
  • K.G. Thompson et al.

    A visual salienc map in the primate frontal eye field

    Prog. Brain Res.

    (2005)
  • R. Klein

    Inhibitory tagging system facilitates visual search

    Nature

    (1988)
  • Cited by (0)

    View full text