Attention-Related Pearce-Kaye-Hall Signals in Basolateral Amygdala Require the Midbrain Dopaminergic System

doi:10.1016/j.biopsych.2012.05.023

Biological Psychiatry

Volume 72, Issue 12, 15 December 2012, Pages 1012-1019

https://doi.org/10.1016/j.biopsych.2012.05.023 Get rights and content

Background

Neural activity in basolateral amygdala has recently been shown to reflect surprise or attention as predicted by the Pearce-Kaye-Hall model (PKH)—an influential model of associative learning. Theoretically, a PKH attentional signal originates in prediction errors of the kind associated with phasic firing of dopamine neurons. This requirement for prediction errors, coupled with projections from the midbrain dopamine system into basolateral amygdala, suggests that the PKH signal in amygdala may depend on dopaminergic input.

Methods

To test this, we recorded single unit activity in basolateral amygdala in rats with 6-hydroxydopamine or sham lesions of the ipsilateral midbrain region. Neurons were recorded as the rats performed a task previously used to demonstrate both dopaminergic reward prediction errors and attentional signals in basolateral amygdala neurons.

Results

We found that neurons recorded in sham lesioned rats exhibited the same attention-related PKH signal observed in previous studies. By contrast, neurons recorded in rats with ipsilateral 6-hydroxydopamine lesions failed to show attentional signaling.

Conclusions

These results indicate a linkage between the neural instantiations of the basolateral complex of the amygdala attentional signal and dopaminergic prediction errors. Such a linkage would have important implications for understanding both normal and aberrant learning and behavior, particularly in diseases thought to have a primary effect on dopamine systems, such as addiction and schizophrenia.

Section snippets

Subjects, Behavioral Apparatus, and Testing

Male Long-Evans rats served as subjects, tested at the University of Maryland, Baltimore, in accordance with School of Medicine and National Institutes of Health guidelines. Equipment and training were identical to prior experiments (3). Briefly, training used aluminum chambers. A central odor port was located above two adjacent fluid wells on a panel in the right wall of each chamber. On each trial, nosepoke into the odor port resulted in delivery of the odor cue. One of three different odors

Results

Neuronal activity in ABL was recorded while rats performed a choice task. On each trial, rats were required to nosepoke at an odor port where one of three possible odors could be presented. Odors predicted reward at one of two adjacent fluid wells situated below the odor port (Figure 1A). Odor 1 signaled that the sucrose reward would be delivered in the left well (forced-left), odor 2 signaled that it would be delivered in the right well (forced-right), and odor 3 signaled that it would be made

Discussion

The current study replicates the attentional signal in ABL previously shown in rats (3) and humans (4) and further demonstrates its dependence on the dopaminergic system. What role does dopamine play in the construction of the attentional signal in ABL? An obvious possibility is that the latter has its origin in the signed prediction error signals widely reported in midbrain DA neurons (8, 9, 10, 11, 32, 33, 34, 35, 36, 37, 38, 39). In addition to inverting the polarity of neural activity for

References (68)

E.A. Murray
The amygdala, reward and emotion
Trends Cogn Sci
(2007)
M.A. Belova et al.
Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala
Neuron
(2007)
M. Brinley-Reed et al.
Evidence that dopaminergic axons provide a dense innervation of specific neuronal subpopulations in the rat basolateral amygdala
Brain Res
(1999)
K. Loretan et al.
Dopamine modulation of spontaneous inhibitory network activity in the lateral amygdala
Neuropharmacology
(2004)
H.M. Bayer et al.
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Neuron
(2005)
H. Nakahara et al.
Dopamine neurons can represent context-dependent prediction error
Neuron
(2004)
M.L. Coco et al.
Selective activation of mesoamygdaloid dopamine neurons by conditioned stress: Attenuation by diazepam
Brain Res
(1992)
C.J. Harmer et al.
Enhanced dopamine efflux in the amygdala by a predictive, but not a non-predictive, stimulus: Facilitation by prior repeated D-amphetamine
Neuroscience
(1999)
D.J. Calu et al.
Neural correlates of variations in event processing during learning in central nucleus of amygdala
Neuron
(2010)
G. Schoenbaum et al.
Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala
Neuron
(2003)

F. Ambroggi et al.

Basolateral amygdala neurons facilitate reward-seeking behavior by exciting nucleus accumbens neurons

Neuron

(2008)

A.C. Courville et al.

Bayesian theories of conditioning in a changing world

Trends Cogn Sci

(2006)

P.R. Corlett et al.

Toward a neurobiology of delusions

Prog Neurobiol

(2010)

M. Torregrossa et al.

Aberrant learning and memory in addiction

Neurobiol Learn Mem

(2011)

M.R. Roesch et al.

Neural correlates of variations in event processing during learning in basolateral amygdala

J Neurosci

(2010)

J. Li et al.

Differential roles of human striatum and amygdala in associative learning

Nat Neurosci

(2011)

C. Herry et al.

Processing of temporal unpredictability in human and animal amygdala

J Neurosci

(2007)

K.M. Tye et al.

Amygdala neural encoding in the absence of reward during extinction

J Neurosci

(2010)

J.P. Johansen et al.

Neural substrates for expectation-modulated fear learning in the amygdala and periaqueductal gray

Nat Neurosci

(2010)

P.R. Montague et al.

A framework for mesencephalic dopamine systems based on predictive hebbian learning

J Neurosci

(1996)

J. Mirenowicz et al.

Importance of unpredictability for reward responses in primate dopamine neurons

J Neurophysiol

(1994)

W.-X. Pan et al.

Dopamine cells respond to predicted events during classical conditioning: Evidence for eligibility traces in the reward-learning network

J Neurosci

(2005)

K. D'Ardenne et al.

BOLD responses reflecting dopaminergic signals in the human ventral tegmental area

Science

(2008)

R.A. Rescorla et al.

A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement

W. Schultz et al.

A neural substrate for prediction and reward

Science

(1997)

R.S. Sutton

Learning to predict by the method of temporal difference

Machine Learning

(1988)

J.M. Pearce et al.

A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli

Psychol Rev

(1980)

N.J. Mackintosh

A theory of attention: Variations in the associability of stimuli with reinforcement

Psychol Rev

(1975)

J.M. Pearce et al.

Predictive accuracy and stimulus associability: Development of a model for Pavlovian learning

K.M. Tye et al.

Rapid strengthening of thalamo-amygdala synapses mediates cue-reward learning

Nature

(2008)

G. Schoenbaum et al.

Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning

J Neurosci

(1999)

J.J. Patton et al.

The primate amygdala represents the positive and negative value of visual stimuli during learning

Nature

(2006)

A. Fontanini et al.

Distinct subtypes of basolateral amygdala neurons reflect palatability and reward

J Neurosci

(2009)

Y. Sugase-Miyamoto et al.

Neuronal signals in the monkey basolateral amygdala during reward schedules

J Neurosci

(2005)

Cited by (33)

The Recruitment of a Neuronal Ensemble in the Central Nucleus of the Amygdala During the First Extinction Episode Has Persistent Effects on Extinction Expression
2023, Biological Psychiatry
Adaptive behavior depends on the delicate and dynamic balance between acquisition and extinction memories. Disruption of this balance, particularly when the extinction of memory loses control over behavior, is the root of treatment failure of maladaptive behaviors such as substance abuse or anxiety disorders. Understanding this balance requires a better understanding of the underlying neurobiology and its contribution to behavioral regulation.
We microinjected Daun02 in Fos-lacZ transgenic rats following a single extinction training episode to delete extinction-recruited neuronal ensembles in the basolateral amygdala (BLA) and central nucleus of the amygdala (CN) and examined their contribution to behavior in an appetitive Pavlovian task. In addition, we used immunohistochemistry and neuronal staining methods to identify the molecular markers of activated neurons in the BLA and CN during extinction learning or retrieval.
CN neurons were preferentially engaged following extinction, and deletion of these extinction-activated ensembles in the CN but not the BLA impaired the retrieval of extinction despite additional extinction training and promoted greater levels of behavioral restoration in spontaneous recovery and reinstatement. Disrupting extinction processing in the CN in turn increased activity in the BLA. Our results also show a specific role for CN PKCδ+ neurons in behavioral inhibition but not during initial extinction learning.
We showed that the initial extinction-recruited CN ensemble is critical to the acquisition-extinction balance and that greater behavioral restoration does not mean weaker extinction contribution. These findings provide a novel avenue for thinking about the neural mechanisms of extinction and for developing treatments for cue-triggered appetitive behaviors.
The basolateral amygdala and lateral hypothalamus bias learning towards motivationally significant events
2021, Current Opinion in Behavioral Sciences
Citation Excerpt :
Similarly, in another study, BLA neurons increased firing rate with an unexpected change in reward magnitude or delay, exhibiting a short burst of increased activity whether these changes meant more or less reward, or delivered over a shorter or longer timescale [24]. This is reminiscent of an unsigned attentional signal [24], strengthening the idea that BLA is concerned with the specific outcomes predicted by environmental stimuli and their motivational significance [24,25]. Altogether, these studies provide evidence to support the BLA as an associative structure important for learning about motivationally significant events by encoding sensory-specific information about these events, including their current predictive status.
Every day we are faced with a huge amount of new information. We can’t learn about everything, we have to select what to learn about. We have many systems that contribute to learning in different ways, allowing us to select the most relevant information to learn about. This review will focus on one such system, comprising the basolateral amygdala and lateral hypothalamus, which we argue works to favor learning about information most relevant to current goals. Specifically, we will discuss work that has revealed the role of the basolateral amygdala in encoding the sensory-specific aspects of rewarding information. Then, we discuss new data implicating lateral hypothalamus in biasing learning towards reward-predictive cues, and away from information distal to rewards. Finally, we offer a framework of how these regions communicate to relay this information to the midbrain dopamine system, allowing them to bias ongoing learning towards the best predictors of motivationally relevant information.
Prediction errors and valence: From single units to multidimensional encoding in the amygdala
2021, Behavioural Brain Research
Citation Excerpt :
Importantly, these signals seem to contribute to the attentional changes proposed by the amended Pearce-Hall classical conditioning model, in which changes in firing are influenced by a history of prediction errors over several trials [78,80,81]. Interestingly, however, subsequent results illustrated that these unsigned signals in BLA were partially dependent on the midbrain dopamine system, which classically signals signed reward prediction errors, suggesting that the two signals are intertwined [78]. Both of these signals are in stark contrast to firing observed in CeA in rats performing the same size-delay task, where single neuron firing in CeA contributed to signaling negative—but not positive—prediction errors.
The amygdala—one of the primary structures of the limbic system—is comprised of interconnected nuclei situated within the temporal lobe. It has a well-established role in the modulation of negative affective states, as well as in fear processing. However, its vast projections with diverse brain regions—ranging from the cortex to the brainstem—are suggestive of its more complex involvement in affective or motivational aspects of cognitive processing. The amygdala can play an invaluable role in context-dependent associative learning, unsigned prediction error learning, influencing outcome selection, and multidimensional encoding. In this review, we delve into the amygdala’s role in associative learning and outcome selection, emphasizing its intrinsic involvement in the appropriate context-dependent modulation of motivated behavior.
What is dopamine doing in model-based reinforcement learning?
2021, Current Opinion in Behavioral Sciences
Experiments have implicated dopamine in model-based reinforcement learning (RL). These findings are unexpected as dopamine is thought to encode a reward prediction error (RPE), which is the key teaching signal in model-free RL. Here we examine two possible accounts for dopamine’s involvement in model-based RL: the first that dopamine neurons carry a prediction error used to update a type of predictive state representation called a successor representation, the second that two well established aspects of dopaminergic activity, RPEs and surprise signals, can together explain dopamine’s involvement in model-based RL.
Dynamic resource allocation during reinforcement learning accounts for ramping and phasic dopamine activity
2020, Neural Networks
Citation Excerpt :
In Fig. 5B, experimental (or essential) stimuli, non-essential stimuli, and the reward are indicated by a red-, blue-, or orange-colored number, respectively. Based on previous reversal studies showing that large prediction errors occurring at the beginning of each block enhance the learning rate (Esber, et al., 2012; Li et al., 2011), we made the prediction error generated by the previous outcome adjust the learning rate (see Section 2.2). In this reversal experiment, the environment remains constant until the animal exits from the bridge, while the environment after the animal making its choice changes across blocks (Fig. 5A).
For an animal to learn about its environment with limited motor and cognitive resources, it should focus its resources on potentially important stimuli. However, too narrow focus is disadvantageous for adaptation to environmental changes. Midbrain dopamine neurons are excited by potentially important stimuli, such as reward-predicting or novel stimuli, and allocate resources to these stimuli by modulating how an animal approaches, exploits, explores, and attends. The current study examined the theoretical possibility that dopamine activity reflects the dynamic allocation of resources for learning. Dopamine activity may transition between two patterns: (1) phasic responses to cues and rewards, and (2) ramping activity arising as the agent approaches the reward. Phasic excitation has been explained by prediction errors generated by experimentally inserted cues. However, when and why dopamine activity transitions between the two patterns remain unknown. By parsimoniously modifying a standard temporal difference (TD) learning model to accommodate a mixed presentation of both experimental and environmental stimuli, we simulated dopamine transitions and compared them with experimental data from four different studies. The results suggested that dopamine transitions from ramping to phasic patterns as the agent focuses its resources on a small number of reward-predicting stimuli, thus leading to task dimensionality reduction. The opposite occurs when the agent re-distributes its resources to adapt to environmental changes, resulting in task dimensionality expansion. This research elucidates the role of dopamine in a broader context, providing a potential explanation for the diverse repertoire of dopamine activity that cannot be explained solely by prediction error.
Ventral Tegmental Dopamine Neurons Participate in Reward Identity Predictions
2019, Current Biology
Dopamine (DA) neurons in the ventral tegmental area (VTA) and substantia nigra (SNc) encode reward prediction errors (RPEs) and are proposed to mediate error-driven learning. However, the learning strategy engaged by DA-RPEs remains controversial. RPEs might imbue predictive cues with pure value, independently of representations of their associated outcome. Alternatively, RPEs might promote learning about the sensory features (the identity) of the rewarding outcome. Here, we show that, although both VTA and SNc DA neuron activation reinforces instrumental responding, only VTA DA neuron activation during consumption of expected sucrose reward restores error-driven learning and promotes formation of a new cue→sucrose association. Critically, expression of VTA DA-dependent Pavlovian associations is abolished following sucrose devaluation, a signature of identity-based learning. These findings reveal that activation of VTA- or SNc-DA neurons engages largely dissociable learning processes with VTA-DA neurons capable of participating in outcome-specific predictive learning, and the role of SNc-DA neurons appears limited to reinforcement of instrumental responses.

View all citing articles on Scopus

: Authors GRE and MRR contributed equally to this work.

View full text

Archival ReportAttention-Related Pearce-Kaye-Hall Signals in Basolateral Amygdala Require the Midbrain Dopaminergic System

Background

Methods

Results

Conclusions

Section snippets

Subjects, Behavioral Apparatus, and Testing

Results

Discussion

Trends Cogn Sci

Neuron

Brain Res

Neuropharmacology

Neuron

Neuron

Brain Res

Neuroscience

Neuron

Neuron

Neuron

Trends Cogn Sci

Prog Neurobiol

Neurobiol Learn Mem

Neural correlates of variations in event processing during learning in basolateral amygdala

J Neurosci

Differential roles of human striatum and amygdala in associative learning

Nat Neurosci

Processing of temporal unpredictability in human and animal amygdala

J Neurosci

Amygdala neural encoding in the absence of reward during extinction

J Neurosci

Neural substrates for expectation-modulated fear learning in the amygdala and periaqueductal gray

Nat Neurosci

A framework for mesencephalic dopamine systems based on predictive hebbian learning

J Neurosci

Importance of unpredictability for reward responses in primate dopamine neurons

J Neurophysiol

Dopamine cells respond to predicted events during classical conditioning: Evidence for eligibility traces in the reward-learning network

J Neurosci

BOLD responses reflecting dopaminergic signals in the human ventral tegmental area

Science

A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement

A neural substrate for prediction and reward

Science

Learning to predict by the method of temporal difference

Machine Learning

A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli

Psychol Rev

A theory of attention: Variations in the associability of stimuli with reinforcement

Psychol Rev

Predictive accuracy and stimulus associability: Development of a model for Pavlovian learning

Rapid strengthening of thalamo-amygdala synapses mediates cue-reward learning

Nature

Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning

J Neurosci

The primate amygdala represents the positive and negative value of visual stimuli during learning

Nature

Distinct subtypes of basolateral amygdala neurons reflect palatability and reward

J Neurosci

Neuronal signals in the monkey basolateral amygdala during reward schedules

J Neurosci

Archival Report
Attention-Related Pearce-Kaye-Hall Signals in Basolateral Amygdala Require the Midbrain Dopaminergic System