Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Prefrontal cortex and decision making in a mixed-strategy game

Nature Neuroscience volume 7, pages 404–410 (2004)Cite this article

Abstract

In a multi-agent environment, where the outcomes of one's actions change dynamically because they are related to the behavior of other beings, it becomes difficult to make an optimal decision about how to act. Although game theory provides normative solutions for decision making in groups, how such decision-making strategies are altered by experience is poorly understood. These adaptive processes might resemble reinforcement learning algorithms, which provide a general framework for finding optimal strategies in a dynamic environment. Here we investigated the role of prefrontal cortex (PFC) in dynamic decision making in monkeys. As in reinforcement learning, the animal's choice during a competitive game was biased by its choice and reward history, as well as by the strategies of its opponent. Furthermore, neurons in the dorsolateral prefrontal cortex (DLPFC) encoded the animal's past decisions and payoffs, as well as the conjunction between the two, providing signals necessary to update the estimates of expected reward. Thus, PFC might have a key role in optimizing decision-making strategies.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Task and behavioral performance.
Figure 2: Effects of relative expected reward (i.e., difference in value functions) and its trial-to-trial changes on the activity of prefrontal neurons.
Figure 3: Percentages of neurons encoding signals related to the animal's decision.
Figure 4: Example neuron showing a significant effect of the animal's choice in the previous trial.
Figure 5: Example neuron showing a significant effect of the reward in the previous trial.
Figure 6: Example neuron with a significant interaction between the animal's choice and its outcome in the previous trial.

Similar content being viewed by others

References

  1. Von Neumann, J. & Morgenstern, O. Theory of Games and Economic Behavior (Princeton Univ. Press, New Jersey, 1944).

    Google Scholar 

  2. Fudenberg, D. & Tirole, J. Game Theory (MIT Press, Cambridge, Massachusetts, 1991).

    Google Scholar 

  3. Dixit, A. & Skeath, S. Games of Strategy (Norton, New York, 1999).

    Google Scholar 

  4. Glimcher, P.W. Decisions, Uncertainty, and the Brain (MIT Press, Cambridge, Massachusetts, 2003).

    Book  Google Scholar 

  5. Nash, J.F. Equilibrium points in n-person games. Proc. Natl. Acad. Sci. USA 36, 48–49 (1950).

    Article  CAS  Google Scholar 

  6. Robinson, J. An iterative method of solving a game. Ann. Math. 54, 296–301 (1951).

    Article  Google Scholar 

  7. Binmore, K., Swierzbinski, J. & Proulx, C. Does minimax work? An experimental study. Econ. J. 111, 445–464 (2001).

    Article  Google Scholar 

  8. Touhey, J.C. Decision processes, expectations, and adoption of strategies in zero-sum games. Hum. Relat. 27, 813–824 (1974).

    Article  Google Scholar 

  9. O'Neill, B. Nonmetric test of the minimax theory of two-person zerosum games. Proc. Natl. Acad. Sci. USA 84, 2106–2109 (1987).

    Article  CAS  Google Scholar 

  10. Brown, J.N. & Rosenthal, R.W. Testing the minimax hypothesis: a re-examination of O'Neill's game experiment. Econometrica 58, 1065–1085 (1990).

    Article  Google Scholar 

  11. Rapoport, A. & Boebel, R.B. Mixed strategies in strictly competitive games: a further test of the minimax hypothesis. Games Econ. Behav. 4, 261–283 (1992).

    Article  Google Scholar 

  12. Rapoport, A. & Budescu, D.V. Generation of random series in two-person strictly competitive games. J. Exp. Psychol. Gen. 121, 352–363 (1992).

    Article  Google Scholar 

  13. Budescu, D.V. & Rapoport, A. Subjective randomization in one- and two-person games. J. Behav. Dec. Making 7, 261–278 (1994).

    Article  Google Scholar 

  14. Ochs, J. Games with unique, mixed strategy equilibria: an experimental study. Games Econ. Behav. 10, 202–217 (1995).

    Article  Google Scholar 

  15. Sarin, R. & Vahid, F. Predicting how people play games: a simple dynamic model of choice. Games Econ. Behav. 34, 104–122 (2001).

    Article  Google Scholar 

  16. Shachat, J.M. Mixed strategy play and the minimax hypothesis. J. Econ. Theory 104, 189–226 (2002).

    Article  Google Scholar 

  17. Walker, M. & Wooders, J. Minimax play at Wimbledon. Am. Econ. Rev. 91, 1521–1538 (2001).

    Article  Google Scholar 

  18. Fudenberg, D. & Levine, D.K. Theory of Learning in Games (MIT Press, Cambridge, Massachusetts, 1998).

  19. Mookherjee, D. & Sopher, B. Learning behavior in an experimental matching pennies game. Games Econ. Behav. 7, 62–91 (1994).

    Article  Google Scholar 

  20. Erev, I. & Roth, A.E. Predicting how people play games: reinforcement learning in experimental games with unique, mixed strategy equilibria. Am. Econ. Rev. 88, 848–881 (1998).

    Google Scholar 

  21. Camerer, C.F. Behavioral Game Theory (Princeton Univ. Press, Princeton, New Jersey, 2003).

    Google Scholar 

  22. Simon, H.A. Models of Man (Wiley, New York, 1957).

    Google Scholar 

  23. Kahneman, D., Slovic, P. & Tversky, A. Judgement Under Uncertainty: Heuristics and Biases (Cambridge Univ. Press, Cambridge, UK, 1982).

    Book  Google Scholar 

  24. O'Neill, B. Comments on Brown and Rosenthal's reexamination. Econometrica 59, 503–507 (1991).

    Article  Google Scholar 

  25. Rapoport, A. & Budescu, D.V. Randomization in individual choice behavior. Psychol. Rev. 104, 603–617 (1997).

    Article  Google Scholar 

  26. Chen, H-C., Friedman, J.W. & Thisse, J-F. Boundedly rational Nash equilibrium: a probabilistic choice approach. Games Econ. Behav. 18, 32–54 (1997).

    Article  CAS  Google Scholar 

  27. Rubinstein, A. Modeling Bounded Rationality (MIT Press, Cambridge, Massachusetts, 1998).

    Google Scholar 

  28. Sutton, R.S. & Barto, A.G. Reinforcement Learning: an Introduction (MIT Press, Cambridge, Massachusetts, 1998).

    Google Scholar 

  29. Littman, M.L. Markov games as a framework for multi-agent reinforcement learning. Machine Learning: Proc. 11th Int. Conf. pp. 157–163 (Morgan Kaufmann, San Francisco, California, 1994).

  30. Christensen, R. Log-linear Models and Logistic Regression edn. 2 (Springer-Verlag, New York, 1997).

    Google Scholar 

  31. Burnham, K.P. & Anderson, D.R. Model Selection and Multimodel Inference 2nd edn. (Springer-Verlag, New York, 2002).

    Google Scholar 

  32. Byrne, R.W. & Whiten, A. Machiavellian Intelligence: Social Expertise and the Evolution of Intellect in Monkeys, Apes and Humans (Oxford Univ. Press, Oxford, UK, 1988).

    Google Scholar 

  33. Whiten, A. & Byrne, R.W. Machiavellian Intelligence II: Extensions and Evaluations (Cambridge Univ. Press, Cambridge, UK, 1997).

    Book  Google Scholar 

  34. Schultz, W. Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1–27 (1998).

    Article  CAS  Google Scholar 

  35. Rolls, E.T. Brain and Emotion (Oxford Univ. Press, Oxford, UK, 1999).

    Google Scholar 

  36. Amador, N., Schlag-Rey, M. & Schlag, J. Reward-predicting and reward detecting neuronal activity in the primate supplementary eye field. J. Neurophysiol. 84, 2166–2170 (2000).

    Article  CAS  Google Scholar 

  37. Stuphorn, V., Taylor, T.L. & Schall, J.D. Performance monitoring by the supplementary eye field. Nature 408, 857–860 (2000).

    Article  CAS  Google Scholar 

  38. Ito, S., Stuphorn, V., Brown, J.W. & Schall, J.D. Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302, 120–122 (2003).

    Article  CAS  Google Scholar 

  39. Kawagoe, R., Takikawa, Y. & Hikosaka, O. Expectation of reward modulates cognitive signals in the basal ganglia. Nat. Neurosci. 1, 411–416 (1998).

    Article  CAS  Google Scholar 

  40. Platt, M.L. & Glimcher, P.W. Neural correlates of decision variables in parietal cortex. Nature 400, 233–238 (1999).

    Article  CAS  Google Scholar 

  41. Shidara, M. & Richmond, B.J. Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science 296, 1709–1711 (2002).

    Article  Google Scholar 

  42. Ikeda, T. & Hikosaka, O. Reward-dependent gain and bias of visual responses in primate superior colliculus. Neuron 39, 693–700 (2003).

    Article  CAS  Google Scholar 

  43. McCoy, A.N., Crowley, J.C., Haghighian, G., Dean, H.L. & Platt, M.L. Saccade reward signals in posterior cingulate cortex. Neuron 40, 1031–1040 (2003).

    Article  CAS  Google Scholar 

  44. Watanabe, M. Reward expectancy in primate prefrontal cortex. Nature 382, 629–632 (1996).

    Article  CAS  Google Scholar 

  45. Leon, M.I. & Shadlen, M.N. Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 24, 415–425 (1999).

    Article  CAS  Google Scholar 

  46. Kobayashi, S., Lauwereyns, J., Koizumi, M., Sakagami, M. & Hikosaka, O. Influence of reward expectation on visuospatial processing in macaque lateral prefrontal cortex. J. Neurophysiol. 87, 1488–1498 (2002).

    Article  Google Scholar 

  47. Roesch, M.R. & Olson, C.R. Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. J. Neurophysiol. 90, 1766–1789 (2003).

    Article  Google Scholar 

  48. Tsujimoto, S. & Sawaguchi, T. Neuronal representation of response outcome in the primate prefrontal cortex. Cereb. Cortex 14, 47–55 (2004).

    Article  Google Scholar 

  49. Wang, X.-J. Probabilistic decision making by slow reverberation in cortical circuits. Neuron 36, 955–968 (2002).

    Article  CAS  Google Scholar 

  50. Bruce, C.J., Goldberg, M.E., Bushnell, M.C. & Stanton, G.B. Primate frontal eye fields. II. Physiological and anatomical correlates of electrically evoked eye movements. J. Neurophysiol. 54, 714–734 (1985).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

We thank L. Carr, R. Farrell, B. McGreevy and T. Twietmeyer for their technical assistance, J. Swan-Stone for programming, X.-J. Wang for discussions, and B. Averbeck and J. Malpeli for critically reading the manuscript. This work was supported by the James S. McDonnell Foundation and the National Institutes of Health (NS44270 and EY01319).

Author information

Authors and Affiliations

  1. Department of Brain and Cognitive Sciences, Center for Visual Science, University of Rochester, Rochester, 14627, New York, USA

    Dominic J Barraclough, Michelle L Conroy & Daeyeol Lee

Authors
  1. Dominic J Barraclough
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michelle L Conroy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daeyeol Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daeyeol Lee.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Barraclough, D., Conroy, M. & Lee, D. Prefrontal cortex and decision making in a mixed-strategy game. Nat Neurosci 7, 404–410 (2004). https://doi.org/10.1038/nn1209

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn1209

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing