Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Opinion
  • Published:

A computational perspective on the neural basis of multisensory spatial representations

Nature Reviews Neuroscience volume 3, pages 741–747 (2002)Cite this article

Abstract

We argue that current theories of multisensory representations are inconsistent with the existence of a large proportion of multimodal neurons with gain fields and partially shifting receptive fields. Moreover, these theories do not fully resolve the recoding and statistical issues involved in multisensory integration. An alternative theory, which we have recently developed and review here, has important implications for the idea of 'frame of reference' in neural spatial representations. This theory is based on a neural architecture that combines basis functions and attractor dynamics. Basis function units are used to solve the recoding problem, whereas attractor dynamics are used for optimal statistical inferences. This architecture accounts for gain fields and partially shifting receptive fields, which emerge naturally as a result of the network connectivity and dynamics.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: A schematic representation of the standard theory for multisensory spatial integration and sensorimotor transformations.
Figure 2: A neural network for coordinate transformations using basis functions.
Figure 3: A recurrent basis function layer with attractor dynamics.
Figure 4: A partially shifting receptive field.
Figure 5: A schematic representation of a basis function network for reaching towards visual, auditory and tactile targets.

Similar content being viewed by others

References

  1. Spence, C. & Driver, J. Crossmodal Space and Crossmodal Attention (Oxford Univ. Press, Oxford, UK, in the press).

  2. Yuille, A. L. & Bulthoff, H. H. in Perception as Bayesian Inference (eds Knill, D. C. & Richards, W.) 123–162 (Cambridge Univ. Press, New York, 1996).

    Book  Google Scholar 

  3. Ernst, M. O. & Banks, M. S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429–433 (2002).

    Article  CAS  Google Scholar 

  4. Deneve, S., Latham, P. & Pouget, A. Efficient computation and cue integration with noisy population codes. Nature Neurosci. 4, 826–831 (2001).

    Article  CAS  Google Scholar 

  5. Pena, J. L. & Konishi, M. Auditory spatial receptive fields created by multiplication. Science 292, 249–252 (2001).

    Article  CAS  Google Scholar 

  6. Gross, C. & Graziano, M. S. A. Multiple representations of space in the brain. Neuroscientist 1, 43–50 (1995).

    Article  Google Scholar 

  7. Andersen, R. Multimodal integration for the representation of space in the posterior parietal cortex. Philos. Trans. R. Soc. Lond. B Biol Sci 352, 1421–1428 (1997).

    Article  CAS  Google Scholar 

  8. Rizzolatti, G., Luppino, G. & Matelli, M. Organization of the cortical motor system: new concepts. Electroencephalogr. Clin. Neurophysiol. 106, 283–296 (1998).

    Article  CAS  Google Scholar 

  9. Colby, C. & Goldberg, M. Space and attention in parietal cortex. Annu. Rev. Neurosci. 22, 319–349 (1999).

    Article  CAS  Google Scholar 

  10. Jay, M. F. & Sparks, D. L. Sensorimotor integration in the primate superior colliculus. I. Motor convergence. J. Neurophysiol. 57, 22–34 (1987).

    Article  CAS  Google Scholar 

  11. Duhamel, J., Bremmer, F., BenHamed, S. & Graf, W. Spatial invariance of visual receptive fields in parietal cortex neurons. Nature 389, 845–848 (1997).

    Article  CAS  Google Scholar 

  12. Stricanne, B., Andersen, R. & Mazzoni, P. Eye-centered, head-centered, and intermediate coding of remembered sound locations in area LIP. J. Neurophysiol. 76, 2071–2076 (1996).

    Article  CAS  Google Scholar 

  13. Cohen, Y. & Andersen, R. Reaches to sounds encoded in an eye-centered reference frame. Neuron 27, 647–652 (2000).

    Article  CAS  Google Scholar 

  14. Buneo, C. A., Jarvis, M. R., Batista, A. P. & Andersen, R. A. Direct visuomotor transformations for reaching. Nature 416, 632–636 (2002).

    Article  CAS  Google Scholar 

  15. Graziano, M., Hu, X. & Gross, C. Visuospatial properties of ventral premotor cortex. J. Neurophysiol. 77, 2268–2292 (1997).

    Article  CAS  Google Scholar 

  16. Knill, D. C. Ideal observer perturbation analysis reveals human strategies for inferring surface orientation from texture. Vision Res. 38, 2635–2656 (1998).

    Article  CAS  Google Scholar 

  17. Jacobs, R. A. & Fine, I. Experience-dependent integration of texture and motion cues to depth. Vision Res. 39, 4062–4075 (1999).

    Article  CAS  Google Scholar 

  18. Anastasio, T. J., Patton, P. E. & Belkacem-Boussaid, K. Using Bayes' rule to model multisensory enhancement in the superior colliculus. Neural Comput. 12, 1165–1187 (2000).

    Article  CAS  Google Scholar 

  19. Colonius, H. & Diederich, A. in Advances in Neural Information Processing Systems (eds Dietterich, T. G., Becker, S. & Ghahramani, Z.) (in the press).

  20. Poggio, T. A theory of how the brain might work. Cold Spring Harb. Symp. Quant. Biol. 55, 899–910 (1990).

    Article  CAS  Google Scholar 

  21. Pouget, A. & Sejnowski, T. A neural model of the cortical representation of egocentric distance. Cereb. Cortex 4, 314–329 (1994).

    Article  CAS  Google Scholar 

  22. Pouget, A. & Sejnowski, T. Spatial transformations in the parietal cortex using basis functions. J. Cogn. Neurosci. 9, 222–237 (1997).

    Article  CAS  Google Scholar 

  23. Pouget, A., Zhang, K., Deneve, S. & Latham, P. E. Statistically efficient estimation using population codes. Neural Comput. 10, 373–401 (1998).

    Article  CAS  Google Scholar 

  24. Deneve, S., Latham, P. & Pouget, A. Reading population codes: a neural implementation of ideal observers. Nature Neurosci. 2, 740–745 (1999).

    Article  CAS  Google Scholar 

  25. Ito, M. Neurophysiological aspects of the cerebellar motor control system. Int. J. Neurol. 7, 162–176 (1970).

    CAS  PubMed  Google Scholar 

  26. Kawato, M., Furawaka, K. & Suzuki, R. A hierarchical neural-network model for control and learning of voluntary movement. Biol. Cybern. 57, 169–185 (1987).

    Article  CAS  Google Scholar 

  27. Jordan, M. & Rumelhart, D. Forward models: supervised learning with a distal teacher. Cogn. Sci. 16, 307–354 (1992).

    Article  Google Scholar 

  28. Pouget, A. & Snyder, L. Computational approaches to sensorimotor transformations. Nature Neurosci. 3, 1192–1198 (2000).

    Article  CAS  Google Scholar 

  29. Zipser, D. & Andersen, R. A back-propagation programmed network that stimulates response properties of a subset of posterior parietal neurons. Nature 331, 679–684 (1988).

    Article  CAS  Google Scholar 

  30. Burnod, Y. et al. Visuomotor transformations underlying arm movements toward visual targets: a neural network model of cerebral cortical operations. J. Neurosci. 12, 1435–1453 (1992).

    Article  CAS  Google Scholar 

  31. Groh, J. & Sparks, D. Two models for transforming auditory signals from head-centered to eye-centered coordinates. Biol. Cybern. 67, 291–302 (1992).

    Article  CAS  Google Scholar 

  32. Salinas, E. & Abbot, L. Transfer of coded information from sensory to motor networks. J. Neurosci. 15, 6461–6474 (1995).

    Article  CAS  Google Scholar 

  33. Grossberg, S., Roberts, K., Aguilar, M. & Bullock, D. A neural model of multimodal adaptive saccadic eye movement control by superior colliculus. J. Neurosci. 17, 9706–9725 (1997).

    Article  CAS  Google Scholar 

  34. Andersen, R., Essick, G. & Siegel, R. Encoding of spatial location by posterior parietal neurons. Science 230, 456–458 (1985).

    Article  CAS  Google Scholar 

  35. Boussaoud, D., Barth, T. & Wise, S. Effects of gaze on apparent visual responses of frontal cortex neurons. Exp. Brain Res. 93, 423–434 (1993).

    Article  CAS  Google Scholar 

  36. Trotter, Y. & Celebrini, S. Gaze direction controls response gain in primary visual-cortex neurons. Nature 398, 239–242 (1999).

    Article  CAS  Google Scholar 

  37. Xing, J. & Andersen, R. A. Models of the posterior parietal cortex which perform multimodal integration and represent space in several coordinate frames. J. Cogn. Neurosci. 12, 601–614 (2000).

    Article  CAS  Google Scholar 

  38. Robinson, D. A. Implications of neural networks for how we think about brain function. Behav. Brain Sci. 15, 644–655 (1992).

    Google Scholar 

  39. Driver, J. & Spence, C. Crossmodal links in spatial attention. Phil. Trans. R. Soc. Lond. B Biol. Sci. 353, 1319–1331 (1998).

    Article  CAS  Google Scholar 

  40. Enright, J. The non-visual impact of eye orientation on eye–hand coordination. Vision Res. 35, 1611–1618 (1995).

    Article  CAS  Google Scholar 

  41. Lewald, J. & Ehrenstein, W. The effect of eye position on auditory lateralization. Exp. Brain Res. 108, 473–485 (1996).

    Article  CAS  Google Scholar 

  42. Henriques, D., Klier, E., Smith, M., Lowy, D. & Crawford, J. Gaze-centered remapping of remembered visual space in an open-loop pointing task. J. Neurosci. 18, 1583–1594 (1998).

    Article  CAS  Google Scholar 

  43. Pouget, A., Ducom, J. C., Torri, J. & Bavelier, D. Multisensory spatial representations in eye-centered coordinates. Cognition 83, B1–B11 (2002).

    Article  Google Scholar 

  44. Bertelson, P. in Cognitive Contributions to the Perception of Spatial and Temporal Events (eds Ashersleben, G., Bachmann, T. & Müsseler, J.) 347–362 (Elsevier, Amsterdam, 1999).

    Book  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. the Department of Brain and Cognitive Sciences, University of Rochester, Rochester, 14627, New York, USA

    Alexandre Pouget & Sophie Deneve

  2. the Institut des Sciences Cognitives, UMR 5015 CNRS-Université Claude Bernard Lyon 1 67, Boulevard Pinel, Bron Cedex, 69675, Lyon, France

    Sophie Deneve & Jean-René Duhamel

Authors
  1. Alexandre Pouget
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sophie Deneve
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jean-René Duhamel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexandre Pouget.

Related links

Related links

FURTHER INFORMATION

MIT Encyclopedia of Cognitive Sciences

multisensory integration

spatial perception

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pouget, A., Deneve, S. & Duhamel, JR. A computational perspective on the neural basis of multisensory spatial representations. Nat Rev Neurosci 3, 741–747 (2002). https://doi.org/10.1038/nrn914

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1038/nrn914

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing