1932

Abstract

The basal ganglia are equipped with inhibitory and disinhibitory mechanisms that enable a subject to choose valuable objects and actions. Notably, a value can be determined flexibly by recent experience or stably by prolonged experience. Recent studies have revealed that the head and tail of the caudate nucleus selectively and differentially process flexible and stable values of visual objects. These signals are sent to the superior colliculus through different parts of the substantia nigra so that the animal looks preferentially at high-valued objects, but in different manners. Thus, relying on short-term value memories, the caudate head circuit allows the subject's gaze to move expectantly to recently valued objects. Relying on long-term value memories, the caudate tail circuit allows the subject's gaze to move automatically to previously valued objects. The basal ganglia also contain an equivalent parallel mechanism for action values. Such flexible–stable parallel mechanisms for object and action values create a highly adaptable system for decision making.

[Erratum, Closure]

An erratum has been published for this article:
Basal Ganglia Circuits for Reward Value–Guided Behavior
Loading

Article metrics loading...

/content/journals/10.1146/annurev-neuro-071013-013924
2014-07-08
2024-03-28
Loading full text...

Full text loading...

/deliver/fulltext/neuro/37/1/annurev-neuro-071013-013924.html?itemId=/content/journals/10.1146/annurev-neuro-071013-013924&mimeType=html&fmt=ahah

Literature Cited

  1. Ahissar M, Hochstein S. 1996. Learning pop-out detection: specificities to stimulus characteristics. Vis. Res. 36:3487–500 [Google Scholar]
  2. Alexander GE, Crutcher MD. 1990. Functional architecture of basal ganglia circuits: neural substrates of parallel processing. Trends Neurosci. 13:266–71 [Google Scholar]
  3. Alexander GE, DeLong MR, Strick PL. 1986. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9:357–81 [Google Scholar]
  4. Anderson BA, Laurent PA, Yantis S. 2011. Value-driven attentional capture. Proc. Natl. Acad. Sci. USA 108:10367–71 [Google Scholar]
  5. Anderson BA, Yantis S. 2012. Value-driven attentional and oculomotor capture during goal-directed, unconstrained viewing. Atten. Percept. Psychophys. 74:1644–53 [Google Scholar]
  6. Ashby FG, Maddox WT. 2005. Human category learning. Annu. Rev. Psychol. 56:149–78 [Google Scholar]
  7. Awh E, Belopolsky AV, Theeuwes J. 2012. Top-down versus bottom-up attentional control: a failed theoretical dichotomy. Trends Cogn. Sci. 16:437–43 [Google Scholar]
  8. Bechara A, Damasio H, Tranel D, Damasio AR. 1997. Deciding advantageously before knowing the advantageous strategy. Science 275:1293–95 [Google Scholar]
  9. Beckstead RM, Frankfurter A. 1982. The distribution and some morphological features of substantia nigra neurons that project to the thalamus, superior colliculus and pedunculopontine nucleus in the monkey. Neuroscience 7:2377–88 [Google Scholar]
  10. Benecke R, Rothwell JC, Dick JP, Day BL, Marsden CD. 1986. Performance of simultaneous movements in patients with Parkinson's disease. Brain 109:739–57 [Google Scholar]
  11. Bhatia KP, Marsden CD. 1994. The behavioural and motor consequences of focal lesions of the basal ganglia in man. Brain 117:859–76 [Google Scholar]
  12. Bichot NP, Schall JD. 1999. Effects of similarity and history on neural mechanisms of visual selection. Nat. Neurosci. 2:549–54 [Google Scholar]
  13. Bromberg-Martin ES, Matsumoto M, Hikosaka O. 2010. Dopamine in motivational control: rewarding, aversive, and alerting. Neuron 68:815–34 [Google Scholar]
  14. Brown LL, Schneider JS, Lidsky TI. 1997. Sensory and cognitive functions of the basal ganglia. Curr. Opin. Neurobiol. 7:157–63 [Google Scholar]
  15. Brown RG, Marsden CD. 1990. Cognitive function in Parkinson's disease: from description to theory. Trends Neurosci. 13:21–29 [Google Scholar]
  16. Brown VJ, Desimone R, Mishkin M. 1995. Responses of cells in the tail of the caudate nucleus during visual discrimination learning. J. Neurophysiol. 74:1083–94 [Google Scholar]
  17. Caan W, Perrett DI, Rolls ET. 1984. Responses of striatal neurons in the behaving monkey. 2. Visual processing in the caudal neostriatum. Brain Res. 290:53–65 [Google Scholar]
  18. Caplan LR, Schmahmann JD, Kase CS, Feldmann E, Baquis G. et al. 1990. Caudate infarcts. Arch. Neurol. 47:133–43 [Google Scholar]
  19. Chelazzi L, Perlato A, Santandrea E, Della Libera C. 2013. Rewards teach visual selective attention. Vis. Res. 85:58–72 [Google Scholar]
  20. Chun MM, Jiang Y. 1998. Contextual cueing: implicit learning and memory of visual context guides spatial attention. Cogn. Psychol. 36:28–71 [Google Scholar]
  21. Chun MM, Nakayama K. 2000. On the functional role of implicit visual memory for the adaptive deployment of attention across scenes. Vis. Cogn. 7:65–81 [Google Scholar]
  22. Cohen JD, McClure SM, Yu AJ. 2007. Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philos. Trans. R. Soc. B 362:933–42 [Google Scholar]
  23. Cools AR, van den Bercken JH, Horstink MW, van Spaendonck KP, Berger HJ. 1984. Cognitive and motor shifting aptitude disorder in Parkinson's disease. J. Neurol. Neurosurg. Psychiatry 47:443–53 [Google Scholar]
  24. Crossman ERFW. 1959. A theory of the acquisition of speed-skill. Ergonomics 2:153–66 [Google Scholar]
  25. Daw ND, O'Doherty JP, Dayan P, Seymour B, Dolan RJ. 2006. Cortical substrates for exploratory decisions in humans. Nature 441:876–79 [Google Scholar]
  26. Evans JSBT. 2008. Dual-processing accounts of reasoning, judgment, and social cognition. Annu. Rev. Psychol. 59:255–78 [Google Scholar]
  27. Fernandez-Ruiz J, Wang J, Aigner TG, Mishkin M. 2001. Visual habit formation in monkeys with neurotoxic lesions of the ventrocaudal neostriatum. Proc. Natl. Acad. Sci. USA 98:4196–201 [Google Scholar]
  28. Francois C, Percheron G, Yelnik J. 1984. Localization of nigrostriatal, nigrothalamic and nigrotectal neurons in ventricular coordinates in macaques. Neuroscience 13:61–76 [Google Scholar]
  29. Goldman-Rakic PS. 1988. Topography of cognition: parallel distributed networks in primate association cortex. Annu. Rev. Neurosci. 11:137–56 [Google Scholar]
  30. Henderson JM. 2003. Human gaze control during real-world scene perception. Trends Cogn. Sci. 7:498–504 [Google Scholar]
  31. Hikosaka O, Nakahara H, Rand MK, Sakai K, Lu X. et al. 1999. Parallel neural networks for learning sequential procedures. Trends Neurosci. 22:464–71 [Google Scholar]
  32. Hikosaka O, Nakamura K, Nakahara H. 2006. Basal ganglia orient eyes to reward. J. Neurophysiol. 95:567–84 [Google Scholar]
  33. Hikosaka O, Sakamoto M, Miyashita N. 1993. Effects of caudate nucleus stimulation on substantia nigra cell activity in monkey. Exp. Brain Res. 95:457–72 [Google Scholar]
  34. Hikosaka O, Sakamoto M, Usui S. 1989. Functional properties of monkey caudate neurons. II. Visual and auditory responses. J. Neurophysiol. 61:799–813 [Google Scholar]
  35. Hikosaka O, Takikawa Y, Kawagoe R. 2000. Role of the basal ganglia in the control of purposive saccadic eye movements. Physiol. Rev. 80:953–78 [Google Scholar]
  36. Hikosaka O, Wurtz RH. 1983a. Visual and oculomotor functions of monkey substantia nigra pars reticulata. I. Relation of visual and auditory responses to saccades. J. Neurophysiol. 49:1230–53 [Google Scholar]
  37. Hikosaka O, Wurtz RH. 1983b. Visual and oculomotor functions of monkey substantia nigra pars reticulata. IV. Relation of substantia nigra to superior colliculus. J. Neurophysiol. 49:1285–301 [Google Scholar]
  38. Hikosaka O, Yamamoto S, Yasuda M, Kim HF. 2013. Why skill matters. Trends Cogn. Sci. 17:434–41 [Google Scholar]
  39. Johansson RS, Westling G, Bäckström A, Flanagan JR. 2001. Eye-hand coordination in object manipulation. J. Neurosci. 21:6917–32 [Google Scholar]
  40. Joseph JP, Boussaoud D. 1985. Role of the cat substantia nigra pars reticulata in eye and head movements. I. Neural activity. Exp. Brain Res. 57:286–96 [Google Scholar]
  41. Karni A, Sagi D. 1993. The time course of learning a visual skill. Nature 365:250–52 [Google Scholar]
  42. Kato M, Miyashita N, Hikosaka O, Matsumura M, Usui S, Kori A. 1995. Eye movements in monkeys with local dopamine depletion in the caudate nucleus. I. Deficits in spontaneous saccades. J. Neurosci. 15:912–27 [Google Scholar]
  43. Kawagoe R, Takikawa Y, Hikosaka O. 1998. Expectation of reward modulates cognitive signals in the basal ganglia. Nat. Neurosci. 1:411–16 [Google Scholar]
  44. Kehagia AA, Murray GK, Robbins TW. 2010. Learning and cognitive flexibility: frontostriatal function and monoaminergic modulation. Curr. Opin. Neurobiol. 20:199–204 [Google Scholar]
  45. Kim HF, Hikosaka O. 2013. Distinct basal ganglia circuits controlling behaviors guided by flexible and stable values. Neuron 79:1001–10 [Google Scholar]
  46. Kitama T, Ohno T, Tanaka M, Tsubokawa H, Yoshida K. 1991. Stimulation of the caudate nucleus induces contraversive saccadic eye movements as well as head turning in the cat. Neurosci. Res. 12:287–92 [Google Scholar]
  47. Knowlton BJ, Mangels JA, Squire LR. 1996. A neostriatal habit learning system in humans. Science 273:1399–402 [Google Scholar]
  48. Kording KP, Tenenbaum JB, Shadmehr R. 2007. The dynamics of memory as a consequence of optimal adaptation to a changing body. Nat. Neurosci. 10:779–86 [Google Scholar]
  49. Körding KP, Wolpert DM. 2006. Bayesian decision theory in sensorimotor control. Trends Cogn. Sci. 10:319–26 [Google Scholar]
  50. Kristjánsson A, Sigurjónsdóttir O, Driver J. 2010. Fortune and reversals of fortune in visual search: reward contingencies for pop-out targets affect search efficiency and target repetition effects. Atten. Percept. Psychophys. 72:1229–36 [Google Scholar]
  51. Kyllingsbaek S, Schneider WX, Bundesen C. 2001. Automatic attraction of attention to former targets in visual displays of letters. Percept. Psychophys. 63:85–98 [Google Scholar]
  52. Land MF. 2006. Eye movements and the control of actions in everyday life. Prog. Retin. Eye Res. 25:296–324 [Google Scholar]
  53. Lauwereyns J, Watanabe K, Coe B, Hikosaka O. 2002. A neural correlate of response bias in monkey caudate nucleus. Nature 418:413–17 [Google Scholar]
  54. Lawrence AD, Sahakian BJ, Hodges JR, Rosser AE, Lange KW, Robbins TW. 1996. Executive and mnemonic functions in early Huntington's disease. Brain 119:1633–45 [Google Scholar]
  55. Lees AJ, Smith E. 1983. Cognitive deficits in the early stages of Parkinson's disease. Brain 106:257–70 [Google Scholar]
  56. Lieberman MD. 2000. Intuition: a social cognitive neuroscience approach. Psychol. Bull. 126:109–37 [Google Scholar]
  57. Logan GD. 1985. Skill and automaticity: relations, implications, and future directions. Can. J. Psychol. 39:367–86 [Google Scholar]
  58. Lynch JC, Hoover JE, Strick PL. 1994. Input to the primate frontal eye field from the substantia nigra, superior colliculus, and dentate nucleus demonstrated by transneuronal transport. Exp. Brain Res. 100:181–86 [Google Scholar]
  59. Marewski JN, Gaissmaier W, Gigerenzer G. 2010. Good judgments do not require complex cognition. Cogn. Process. 11:103–21 [Google Scholar]
  60. Marín O, Smeets WJ, González A. 1998. Evolution of the basal ganglia in tetrapods: a new perspective based on recent studies in amphibians. Trends Neurosci. 21:487–94 [Google Scholar]
  61. Marsden CD. 1982. The mysterious motor function of the basal ganglia: the Robert Wartenberg lecture. Neurology 32:514–39 [Google Scholar]
  62. Matsumura M, Kojima J, Gardiner TW, Hikosaka O. 1992. Visual and oculomotor functions of monkey subthalamic nucleus. J. Neurophysiol. 67:1615–32 [Google Scholar]
  63. McNamara JM, Green RF, Olsson O. 2006. Bayes' theorem and its applications in animal behaviour. OIKOS 112:243–51 [Google Scholar]
  64. Middleton FA, Strick PL. 1996. The temporal lobe is a target of output from the basal ganglia. Proc. Natl. Acad. Sci. USA 93:8683–87 [Google Scholar]
  65. Middleton FA, Strick PL. 2000. Basal ganglia and cerebellar loops: motor and cognitive circuits. Brain Res. Brain Res. Rev. 31:236–50 [Google Scholar]
  66. Mink JW. 1996. The basal ganglia: focused selection and inhibition of competing motor programs. Prog. Neurobiol. 50:381–425 [Google Scholar]
  67. Mishkin M, Malamut B, Bachevalier J. 1984. Memories and habits: two neural systems. Neurobiology of Learning and Memory, ed. G Lynch, JL McGaugh, NM Weinberger 65–77 New York: Guilford [Google Scholar]
  68. Mishkin M, Ungerleider LG, Macko KA. 1983. Object vision and spatial vision: two cortical pathways. Trends Neurosci. 6:414–17 [Google Scholar]
  69. Miyachi S, Hikosaka O, Lu X. 2002. Differential activation of monkey striatal neurons in the early and late stages of procedural learning. Exp. Brain Res. 146:122–26 [Google Scholar]
  70. Miyachi S, Hikosaka O, Miyashita K, Kárádi Z, Rand MK. 1997. Differential roles of monkey striatum in learning of sequential hand movement. Exp. Brain Res. 115:1–5 [Google Scholar]
  71. Miyashita K, Rand MK, Miyachi S, Hikosaka O. 1996. Anticipatory saccades in sequential procedural learning in monkeys. J. Neurophysiol. 76:1361–66 [Google Scholar]
  72. Miyashita N, Hikosaka O, Kato M. 1995. Visual hemineglect induced by unilateral striatal dopamine deficiency in monkeys. NeuroReport 6:1257–60 [Google Scholar]
  73. Nakamura K, Hikosaka O. 2006. Role of dopamine in the primate caudate nucleus in reward modulation of saccades. J. Neurosci. 26:5360–69 [Google Scholar]
  74. Nambu A, Tokuno H, Takada M. 2002. Functional significance of the cortico-subthalamo-pallidal ‘hyperdirect’ pathway. Neurosci. Res. 43:111–17 [Google Scholar]
  75. Newell A, Rosenbloom PS. 1981. Mechanisms of skill acquisition and the law of practice. Cognitive Skills and Their Acquisition JR Anderson 1–55 Hillsdale, NJ: Erlbaum [Google Scholar]
  76. Parent A, Mackey A, Smith Y, Boucher R. 1983. The output organization of the substantia nigra in primate as revealed by a retrograde double labeling method. Brain Res. Bull. 10:529–37 [Google Scholar]
  77. Peck CJ, Jangraw DC, Suzuki M, Efem R, Gottlieb J. 2009. Reward modulates attention independently of action value in posterior parietal cortex. J. Neurosci. 29:11182–91 [Google Scholar]
  78. Rangel A, Camerer C, Montague PR. 2008. A framework for studying the neurobiology of value-based decision making. Nat. Rev. Neurosci. 9:545–56 [Google Scholar]
  79. Redgrave P, Rodriguez M, Smith Y, Rodriguez-Oroz MC, Lehericy S. et al. 2010. Goal-directed and habitual control in the basal ganglia: implications for Parkinson's disease. Nat. Rev. Neurosci. 11:760–72 [Google Scholar]
  80. Rothkopf CA, Ballard DH, Hayhoe MM. 2007. Task and context determine where you look. J. Vis. 7:1–20 [Google Scholar]
  81. Rudebeck PH, Behrens TE, Kennerley SW, Baxter MG, Buckley MJ. et al. 2008. Frontal cortex subregions play distinct roles in choices between actions and stimuli. J. Neurosci. 28:13775–85 [Google Scholar]
  82. Saint-Cyr JA, Ungerleider LG, Desimone R. 1990. Organization of visual cortical inputs to the striatum and subsequent outputs to the pallido-nigral complex in the monkey. J. Comp. Neurol. 298:129–56 [Google Scholar]
  83. Sakai K, Kitaguchi K, Hikosaka O. 2003. Chunking during human visuomotor sequence learning. Exp. Brain Res. 152:229–42 [Google Scholar]
  84. Salmon DP, Butters N. 1995. Neurobiology of skill and habit learning. Curr. Opin. Neurobiol. 5:184–90 [Google Scholar]
  85. Sato M, Hikosaka O. 2002. Role of primate substantia nigra pars reticulata in reward-oriented saccadic eye movement. J. Neurosci. 22:2363–73 [Google Scholar]
  86. Schultz W. 1998. Predictive reward signal of dopamine neurons. J. Neurophysiol. 80:1–27 [Google Scholar]
  87. Schultz W, Dayan P, Montague PR. 1997. A neural substrate of prediction and reward. Science 275:1593–99 [Google Scholar]
  88. Schwab RS, Chafetz ME, Walker S. 1954. Control of two simultaneous voluntary motor acts in normals and in parkinsonism. AMA Arch. Neurol. Psychiatry 72:591–98 [Google Scholar]
  89. Seitz AR, Kim D, Watanabe T. 2009. Rewards evoke learning of unconsciously processed visual stimuli in adult humans. Neuron 61:700–7 [Google Scholar]
  90. Sheinberg DL, Logothetis NK. 2001. Noticing familiar objects in real world scenes: the role of temporal cortical neurons in natural vision. J. Neurosci. 21:1340–50 [Google Scholar]
  91. Shiffrin RM, Schneider W. 1977. Controlled and automatic human information processing: II. Perceptual learning, automatic attending, and a general theory. Psychol. Rev. 84:127–90 [Google Scholar]
  92. Sigman M, Gilbert CD. 2000. Learning to find a shape. Nat. Neurosci. 3:264–69 [Google Scholar]
  93. Smith Y, Parent A. 1986. Differential connections of caudate nucleus and putamen in the squirrel monkey (Saimiri sciureus). Neuroscience 18:347–71 [Google Scholar]
  94. Sternberg S. 1969. Memory-scanning: mental processes revealed by reaction-time experiments. Am. Sci. 57:421–57 [Google Scholar]
  95. Takikawa Y, Kawagoe R, Itoh H, Nakahara H, Hikosaka O. 2002. Modulation of saccadic eye movements by predicted reward outcome. Exp. Brain Res. 142:284–91 [Google Scholar]
  96. Tanibuchi I, Kitano H, Jinnai K. 2009. Substantia nigra output to prefrontal cortex via thalamus in monkeys. I. Electrophysiological identification of thalamic relay neurons. J. Neurophysiol. 102:2933–45 [Google Scholar]
  97. Tatler BW, Hayhoe MM, Land MF, Ballard DH. 2011. Eye guidance in natural vision: reinterpreting salience. J. Vis. 11:5 [Google Scholar]
  98. Theeuwes J, Belopolsky AV. 2012. Reward grabs the eye: oculomotor capture by rewarding stimuli. Vis. Res. 74:80–85 [Google Scholar]
  99. Treisman AM, Gelade G. 1980. A feature-integration theory of attention. Cogn. Psychol. 12:97–136 [Google Scholar]
  100. Watanabe M, Munoz DP. 2010. Saccade suppression by electrical microstimulation in monkey caudate nucleus. J. Neurosci. 30:2700–9 [Google Scholar]
  101. Watanabe M, Munoz DP. 2011. Saccade reaction times are influenced by caudate microstimulation following and prior to visual stimulus appearance. J. Cogn. Neurosci. 23:1794–807 [Google Scholar]
  102. Wichmann T, DeLong MR. 1996. Functional and pathophysiological models of the basal ganglia. Curr. Opin. Neurobiol. 6:751–58 [Google Scholar]
  103. Wickens JR, Reynolds JN, Hyland BI. 2003. Neural mechanisms of reward-related motor learning. Curr. Opin. Neurobiol. 13:685–90 [Google Scholar]
  104. Xu-Wilson M, Zee DS, Shadmehr R. 2009. The intrinsic value of visual information affects saccade velocities. Exp. Brain Res. 196:475–81 [Google Scholar]
  105. Yamamoto S, Kim HF, Hikosaka O. 2013. Reward value-contingent changes of visual responses in the primate caudate tail associated with a visuomotor skill. J. Neurosci. 33:11227–38 [Google Scholar]
  106. Yamamoto S, Monosov IE, Yasuda M, Hikosaka O. 2012. What and where information in the caudate tail guides saccades to visual objects. J. Neurosci. 32:11005–16 [Google Scholar]
  107. Yarbus AL. 1967. Eye Movements and Vision New York: Plenum
  108. Yasuda M, Hikosaka O. 2013. Functional territories in the primate substantia nigra separately signaling flexible and stable values. Soc. Neuroscience Meet. Plann. Abstr. 291.05
  109. Yasuda M, Yamamoto S, Hikosaka O. 2012. Robust representation of stable object values in the oculomotor basal ganglia. J. Neurosci. 32:16917–32 [Google Scholar]
  110. Yeterian EH, Pandya DN. 1991. Prefrontostriatal connections in relation to cortical architectonic organization in rhesus monkeys. J. Comp. Neurol. 312:43–67 [Google Scholar]
  111. Yin HH, Knowlton BJ. 2006. The role of the basal ganglia in habit formation. Nat. Rev. Neurosci. 7:464–76 [Google Scholar]
/content/journals/10.1146/annurev-neuro-071013-013924
Loading
/content/journals/10.1146/annurev-neuro-071013-013924
Loading

Data & Media loading...

Supplemental Material

Supplementary Data

  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error