Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Double dissociation of value computations in orbitofrontal and anterior cingulate neurons

Nature Neuroscience volume 14, pages 1581–1589 (2011)Cite this article

Subjects

Abstract

Damage to prefrontal cortex (PFC) impairs decision-making, but the underlying value computations that might cause such impairments remain unclear. Here we report that value computations are doubly dissociable among PFC neurons. Although many PFC neurons encoded chosen value, they used opponent encoding schemes such that averaging the neuronal population extinguished value coding. However, a special population of neurons in anterior cingulate cortex (ACC), but not in orbitofrontal cortex (OFC), multiplexed chosen value across decision parameters using a unified encoding scheme and encoded reward prediction errors. In contrast, neurons in OFC, but not ACC, encoded chosen value relative to the recent history of choice values. Together, these results suggest complementary valuation processes across PFC areas: OFC neurons dynamically evaluate current choices relative to recent choice values, whereas ACC neurons encode choice predictions and prediction errors using a common valuation currency reflecting the integration of multiple decision parameters.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: The behavioral task and experimental contingencies.
Figure 2: Three single-neuron examples showing choice and outcome activity on probability trials during rewarded and unrewarded trials.
Figure 3: Relationship between neuronal encoding during the choice and outcome epoch.
Figure 4: Three neurons that encode the value of different choice variables.
Figure 5: Population encoding of value during the choice epoch.
Figure 6: Population analyses of neuronal activity during the outcome epoch of probability trials.
Figure 7: Relationship between encoding common value and +PE.
Figure 8: Neuronal encoding of value history.

Similar content being viewed by others

References

  1. Kennerley, S.W., Walton, M.E., Behrens, T.E., Buckley, M.J. & Rushworth, M.F. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9, 940–947 (2006).

    Article  CAS  Google Scholar 

  2. Rudebeck, P.H. et al. Frontal cortex subregions play distinct roles in choices between actions and stimuli. J. Neurosci. 28, 13775–13785 (2008).

    Article  CAS  Google Scholar 

  3. Walton, M.E., Behrens, T.E., Buckley, M.J., Rudebeck, P.H. & Rushworth, M.F. Separable learning systems in the macaque brain and the role of orbitofrontal cortex in contingent learning. Neuron 65, 927–939 (2010).

    Article  CAS  Google Scholar 

  4. Buckley, M.J. et al. Dissociable components of rule-guided behavior depend on distinct medial and prefrontal regions. Science 325, 52–58 (2009).

    Article  CAS  Google Scholar 

  5. Noonan, M.P. et al. Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex. Proc. Natl. Acad. Sci. USA 107, 20547–20552 (2010).

    Article  CAS  Google Scholar 

  6. Fellows, L.K. Deciding how to decide: ventromedial frontal lobe damage affects information acquisition in multi-attribute decision making. Brain 129, 944–952 (2006).

    Article  Google Scholar 

  7. Williams, Z.M., Bush, G., Rauch, S.L., Cosgrove, G.R. & Eskandar, E.N. Human anterior cingulate neurons and the integration of monetary reward with motor responses. Nat. Neurosci. 7, 1370–1375 (2004).

    Article  CAS  Google Scholar 

  8. Bechara, A., Damasio, A.R., Damasio, H. & Anderson, S.W. Insensitivity to future consequences following damage to human prefrontal cortex. Cognition 50, 7–15 (1994).

    Article  CAS  Google Scholar 

  9. Walton, M.E., Devlin, J.T. & Rushworth, M.F. Interactions between decision making and performance monitoring within prefrontal cortex. Nat. Neurosci. 7, 1259–1265 (2004).

    Article  CAS  Google Scholar 

  10. Daw, N.D., O'Doherty, J.P., Dayan, P., Seymour, B. & Dolan, R.J. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879 (2006).

    Article  CAS  Google Scholar 

  11. Prévost, C., Pessiglione, M., Metereau, E., Clery-Melin, M.L. & Dreher, J.C. Separate valuation subsystems for delay and effort decision costs. J. Neurosci. 30, 14080–14090 (2010).

    Article  Google Scholar 

  12. Rushworth, M.F. & Behrens, T.E. Choice, uncertainty and value in prefrontal and cingulate cortex. Nat. Neurosci. 11, 389–397 (2008).

    Article  CAS  Google Scholar 

  13. Padoa-Schioppa, C. & Assad, J.A. Neurons in the orbitofrontal cortex encode economic value. Nature 441, 223–226 (2006).

    Article  CAS  Google Scholar 

  14. Tremblay, L. & Schultz, W. Relative reward preference in primate orbitofrontal cortex. Nature 398, 704–708 (1999).

    Article  CAS  Google Scholar 

  15. Kennerley, S.W., Dahmubed, A.F., Lara, A.H. & Wallis, J.D. Neurons in the frontal lobe encode the value of multiple decision variables. J. Cogn. Neurosci. 21, 1162–1178 (2009).

    Article  Google Scholar 

  16. Roesch, M.R. & Olson, C.R. Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. J. Neurophysiol. 90, 1766–1789 (2003).

    Article  Google Scholar 

  17. O'Neill, M. & Schultz, W. Coding of reward risk by orbitofrontal neurons is mostly distinct from coding of reward value. Neuron 68, 789–800 (2010).

    Article  CAS  Google Scholar 

  18. Seo, H. & Lee, D. Behavioral and neural changes after gains and losses of conditioned reinforcers. J. Neurosci. 29, 3627–3641 (2009).

    Article  CAS  Google Scholar 

  19. Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007).

    Article  CAS  Google Scholar 

  20. Wallis, J.D. & Miller, E.K. Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task. Eur. J. Neurosci. 18, 2069–2081 (2003).

    Article  Google Scholar 

  21. Kennerley, S.W. & Wallis, J.D. Encoding of reward and space during a working memory task in the orbitofrontal cortex and anterior cingulate sulcus. J. Neurophysiol. 102, 3352–3364 (2009).

    Article  Google Scholar 

  22. Kim, S., Hwang, J. & Lee, D. Prefrontal coding of temporally discounted values during intertemporal choice. Neuron 59, 161–172 (2008).

    Article  CAS  Google Scholar 

  23. Morrison, S.E. & Salzman, C.D. The convergence of information about rewarding and aversive stimuli in single neurons. J. Neurosci. 29, 11471–11483 (2009).

    Article  CAS  Google Scholar 

  24. Sallet, J. et al. Expectations, gains, and losses in the anterior cingulate cortex. Cogn. Affect. Behav. Neurosci. 7, 327–336 (2007).

    Article  Google Scholar 

  25. Kennerley, S.W. & Wallis, J.D. Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables. Eur. J. Neurosci. 29, 2061–2073 (2009).

    Article  Google Scholar 

  26. Takahashi, Y.K. et al. The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes. Neuron 62, 269–280 (2009).

    Article  CAS  Google Scholar 

  27. Wallis, J.D. & Kennerley, S.W. Heterogeneous reward signals in prefrontal cortex. Curr. Opin. Neurobiol. 20, 191–198 (2010).

    Article  CAS  Google Scholar 

  28. Rangel, A., Camerer, C. & Montague, P.R. A framework for studying the neurobiology of value-based decision making. Nat. Rev. Neurosci. 9, 545–556 (2008).

    Article  CAS  Google Scholar 

  29. Padoa-Schioppa, C. Range-adapting representation of economic value in the orbitofrontal cortex. J. Neurosci. 29, 14004–14014 (2009).

    Article  CAS  Google Scholar 

  30. Kobayashi, S., Pinto de Carvalho, O. & Schultz, W. Adaptation of reward sensitivity in orbitofrontal neurons. J. Neurosci. 30, 534–544 (2010).

    Article  CAS  Google Scholar 

  31. Machens, C.K., Romo, R. & Brody, C.D. Flexible control of mutual inhibition: a neural model of two-interval discrimination. Science 307, 1121–1124 (2005).

    Article  CAS  Google Scholar 

  32. Fiorillo, C.D., Tobler, P.N. & Schultz, W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 1898–1902 (2003).

    Article  CAS  Google Scholar 

  33. Matsumoto, M. & Hikosaka, O. Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459, 837–841 (2009).

    Article  CAS  Google Scholar 

  34. Tobler, P.N., Fiorillo, C.D. & Schultz, W. Adaptive coding of reward value by dopamine neurons. Science 307, 1642–1645 (2005).

    Article  CAS  Google Scholar 

  35. Cai, X., Kim, S. & Lee, D. Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice. Neuron 69, 170–182 (2011).

    Article  CAS  Google Scholar 

  36. Holroyd, C.B. & Coles, M.G. The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol. Rev. 109, 679–709 (2002).

    Article  Google Scholar 

  37. Debener, S. et al. Trial-by-trial coupling of concurrent electroencephalogram and functional magnetic resonance imaging identifies the dynamics of performance monitoring. J. Neurosci. 25, 11730–11737 (2005).

    Article  CAS  Google Scholar 

  38. Platt, M.L. & Glimcher, P.W. Neural correlates of decision variables in parietal cortex. Nature 400, 233–238 (1999).

    Article  CAS  Google Scholar 

  39. Shuler, M.G. & Bear, M.F. Reward timing in the primary visual cortex. Science 311, 1606–1609 (2006).

    Article  CAS  Google Scholar 

  40. Matsumoto, M., Matsumoto, K., Abe, H. & Tanaka, K. Medial prefrontal cell activity signaling prediction errors of action values. Nat. Neurosci. 10, 647–656 (2007).

    Article  CAS  Google Scholar 

  41. Amiez, C., Joseph, J.P. & Procyk, E. Anterior cingulate error-related activity is modulated by predicted reward. Eur. J. Neurosci. 21, 3447–3452 (2005).

    Article  Google Scholar 

  42. Sul, J.H., Kim, H., Huh, N., Lee, D. & Jung, M.W. Distinct roles of rodent orbitofrontal and medial prefrontal cortex in decision making. Neuron 66, 449–460 (2010).

    Article  CAS  Google Scholar 

  43. Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005).

    Article  CAS  Google Scholar 

  44. Oliveira, F.T., McDonald, J.J. & Goodman, D. Performance monitoring in the anterior cingulate is not all error related: expectancy deviation and the representation of action-outcome associations. J. Cogn. Neurosci. 19, 1994–2004 (2007).

    Article  Google Scholar 

  45. Jessup, R.K., Busemeyer, J.R. & Brown, J.W. Error effects in anterior cingulate cortex reverse when error likelihood is high. J. Neurosci. 30, 3467–3472 (2010).

    Article  CAS  Google Scholar 

  46. Jocham, G., Neumann, J., Klein, T.A., Danielmeier, C. & Ullsperger, M. Adaptive coding of action values in the human rostral cingulate zone. J. Neurosci. 29, 7489–7496 (2009).

    Article  CAS  Google Scholar 

  47. Behrens, T.E., Woolrich, M.W., Walton, M.E. & Rushworth, M.F. Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214–1221 (2007).

    Article  CAS  Google Scholar 

  48. Montague, P.R. & Berns, G.S. Neural economics and the biological substrates of valuation. Neuron 36, 265–284 (2002).

    Article  CAS  Google Scholar 

  49. Bernacchia, A., Seo, H., Lee, D. & Wang, X.J. A reservoir of time constants for memory traces in cortical neurons. Nat. Neurosci. 14, 366–372 (2011).

    Article  CAS  Google Scholar 

  50. Padoa-Schioppa, C. & Assad, J.A. The representation of economic value in the orbitofrontal cortex is invariant for changes of menu. Nat. Neurosci. 11, 95–102 (2008).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

The project was funded by US National Institute on Drug Abuse grant R01DA19028, US National Institute of Neurological Disorders and Stroke grant P01NS040813 to J.W. and US National Institute on Mental Health training grant F32MH081521 to S.W.K. T.E.J.B. and S.W.K. are supported by the Wellcome Trust.

Author information

Authors and Affiliations

  1. Institute of Neurology, University College London, London, UK

    Steven W Kennerley & Timothy E J Behrens

  2. Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, California, USA

    Steven W Kennerley & Jonathan D Wallis

  3. Oxford Centre for Functional Magnetic Resonance Imaging of the Brain (FMRIB), University of Oxford, John Radcliffe Hospital, Oxford, UK

    Timothy E J Behrens

  4. Department of Psychology, University of California, Berkeley, Berkeley, California, USA

    Jonathan D Wallis

Authors
  1. Steven W Kennerley
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Timothy E J Behrens
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jonathan D Wallis
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

S.W.K. designed the experiment, collected and analyzed the data and wrote the manuscript. T.E.J.B. analyzed the data and edited the manuscript. J.D.W. designed the experiment, supervised the project and edited the manuscript.

Corresponding author

Correspondence to Steven W Kennerley.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–4 and Supplementary Results (PDF 3782 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kennerley, S., Behrens, T. & Wallis, J. Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nat Neurosci 14, 1581–1589 (2011). https://doi.org/10.1038/nn.2961

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.2961

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing