Striatal Dopamine DA and Learning: Do Category Learning CL data constrain computational models

1
Striatal Dopamine (DA) and Learning Do Category
Learning (CL) data constrain computational models?

• Alan Pickering
• Department of Psychology
• a.pickering_at_gold.ac.uk

2
Overview

• Classic CL findings and questions
• DA, the striatum and learning
• Generate simple hypothesis about CL deficits in
  Parkinsons Disease
• Generate simple biologically-constrained neural
  net to test hypothesis
• Simulate CL data on 2 types of matched CL tasks
• Conclusions why model fails

3
Classic Findings and Questions

• Parkinsons Disease (PD) patients are impaired at
  CL tasks.
• Why?
• -What psychological processes are
• impaired?
• -What brain regions and neuro-
• transmitters are involved?

4
Category Learning in Parkinsons Disease

• Weather task Knowlton et al, 1996

5
Category Learning in Parkinsons Disease

• Main Findings Knowlton et al, 1996

6
Key Facts

• PD involves prominent damage to the striatum
• CL may (sometimes) involve procedural/habit
  learning
• Striatal structures are part of
  cortico-striato-pallido-thalamic loops possibly
  implicated in procedural learning
• The striatum is strongly innervated by ascending
  DA projections

7
Simple Interpretation

• CL deficits in PD may arise because of damage to
  
• loss of ascending DA signals
• which compromise the functioning of (parts of)
• the striatum

8
Three Learning Processes Which Might Be
DA-Related

• Appetitive reinforcement and motivation
• DA cell firing increses/decreases provide a
  positive/negative reinforcement signal which is
  required for synaptic strengthening/ weakening
• 3-factor learning rule
• (e.g., Wickens Brown et al etc.)

9
Corticostriatal (Medium Spiny Cell) Synapse
10
DA Receptors in Striatum

• After Schultz, 1998

DA receptors Unfilled rectangles GLU receptors
Filled rectangles
11
DA-Related Processes (cont)

• Reward Prediction Error
• Midbrain DA neurons increase firing in response
  to unexpected rewards and decrease firing to
  nonoccurrence of expected rewards
• Firing change reward prediction error
• Schultz, Suri, Dickinson, Dayan etc.

12
DA Cell Recordings Evidence For Reward
Prediction Error
CUE
REWARD
13
DA-Related Processes (cont)

• Modulation of Neural Signals
• Floresco et al (2001) DA receptor activity
  serves to strengthen salient inputs while
  inhibiting weaker ones
• Also Nicola Malenka J.D.Cohen Ashby
  Cassale Salum et al Nakahara Schultz

14
Evidence For Modulation

• Nicola Malenka, 1997
• Recorded effect of DA on response of striatal
  cells to strong and weak inputs

Strong
Weak
15
Linking 3-Factor Learning Reward Prediction
Error
Striatal Cell
Cue
Reward prediction
Reward predictionerror
Reward
DA Cell
Excitatory
Inhibitory
Reinforcement
16
Simple Working Hypothesis

• CL is impaired in PD patients (and other
  DA-compromised groups) due to reduced DA
  function in striatum (tail of caudate)
• The loss of ascending DA input reduces the
  reinforcing function of the reward prediction
  error signal innervating the striatum

17
Modelling

• Biologically-constrained neural net
• Data to be simulated taken from Ashby et al
  (2003)
• Data from young and old controls (YC, OC) and PD
  patients
• Study used matched CL tasks rule-based (RB) and
  Information Integration (II)
• Ashby and colleagues believe these tasks are
  handled by distinct CL systems

18
Ashby et al II Task

• 3 of the 4 dimensions determine categories
• Not readily verbalisable

Cat A
Cat B
19
Ashby et al RB Task

• 1 dimension (background colour) determines
  category
• Readily verbalisable rule

Cat A
Cat B
20
Ashby et al Results

• Proportion failing to learning to criterion in
  200 trials

RB Task
II Task
21
RB Task Results

• Trials to criterion for learners

II Task
RB Task
22
Modelling

• Constrained by input and output connections of
  striatum (caudate)
• Learning rule based on known
• 3-factor form of synaptic plasticity in striatum
• Learning rule consistent with reward prediction
  error properties of DA neurons

23
Connections of Striatum
24
Schematic Model
Reward prediction
25
Model Learning Rule
xJout
yK
yKout
wJK
Reward prediction error, E

• When reward present, Egt0
• ?wJK kREykoutxJout
• When reward absent, Elt0
• ?wJK kNEykoutxJout

26
Modelling of Reduced DA Function

• Loss of DA input to striatum (tail of caudate)
  modelled 2 ways (with same results)-
• a) loss of modifiability of cortico-
• striatal weights
• b) proportional reduction of reward
  prediction error strength
• Mean proportion of weights modifiable-
• YC 0.8 OC 0.5 PD 0.2
• (with s.d. 0.15)

27
Modelling Process

• Found parameters which gave good fit to YC
  performance on II task and set DA parameters for
  PD to produce appropriate level of nonlearners on
  same task
• Varied OC DA values between YC and PD
• Looked at fit (with these parameters) to all
  other data cells esp. RB task

28
Modelling II Task Results
Proportion of non-learners
Trials to criterion (learners)
YC
PD
29
Modelling II Task Results
Proportion of non-learners
Trials to criterion (learners)
YC
PD
OC
30
Modelling II Task Results
Proportion of non-learners
Trials to criterion (learners)
YC
PD
OC
31
Model Results II Task

• Performance of learners in blocks of 16 trials

32
Modelling RB Task Results
Proportion of non-learners
Trials to criterion (learners)
YC
PD
OC
33
Modelling RB Task Results
Proportion of non-learners
Trials to criterion (learners)
YC
PD
OC
34
Conclusions Future Work 1

• Simplest realistic model of cortico-striatal
  learning captures only limited aspects of the CL
  data
• Bimodal nature (learn normally vs. fail) of
  data simulated only under some paramter settings
• No intermediate DA parameter settings in old
  controls which can be both PD-like for II task
  and YC-like for RB task

35
Conclusions Future Work 2

• Model challenges hypothesis under test PD (and
  OC) deficits in some CL tasks seem unlikely to be
  solely due to reduced DA-related reinforcement in
  striatum
• Findings are consistent with gt1 CL system
• Future model should add rule system (c.f. Ashbys
  COVIS)

36
Alan PickeringCL Refs 2001-

• Pickering, A.D., Gray, J.A. (2001). Dopamine,
  appetitive reinforcement, and the neuropsychology
  of human learning An individual differences
  approach. In A. Eliasz A. Angleitner (Eds.),
  Advances in individual differences research (pp.
  113-149). Lengerich, Germany PABST Science
  Publishers.