Basal Ganglia

1
Basal Ganglia
2
(No Transcript)
3
basal ganglia

• recall major DA targets, involved in movement
  motivation

4
BG Disorders

• In humans, basal ganglia dysfunction associated
  with both hypokinetic and hyperkinetic movement
  disorders
• Hypokinetic Hyperkinetic
• akinesia chorea
• bradykinesia ballism
• rigidity tics

5
A Parkinsons Brainstem
6
Parkinsons disease

• Progressive death of dopamine neurons
• Hypokinetic disorder (also tremor)
• Treated with dopamine precursor (L-Dopa) or
  agonists
• Movie

7
Huntingtons disease

• Progressive death of striatal spiny neurons
• Hyperkinetic disorder chorea
• Similar problems from subthalamic nucleus
  lesions, also Tourettes, OCD
• Treated with dopamine blockade

disease striatal degeneration
healthy
8
(No Transcript)
9
(No Transcript)
10
Medium spiny neurons

• Principal neuron type in striatum
• Recipient of corticostriatal inputs
• Extensive dendrites each receives input from
  10,000 fibers
• Unusual GABAergic (inhibitory) projections
• Also collaterals (competitive network? for
  competition based on value?)

11
Striasomes/Patch Matrix
12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
The corticostriatal projection

• Input nucleus of basal ganglia striatum
• topographic projection from entire cortex
  (including sensory, motor, associative areas)
• ultimately reciprocated
• also dopamine

Voorn et al 2004
16
(No Transcript)
17
Parkinsons disease

• Progressive death of dopamine neurons
• Hypokinetic disorder (also tremor)
• Treated with dopamine precursor (L-Dopa) or
  agonists
• Movie

18
Huntingtons disease

• Progressive death of striatal spiny neurons
• Hyperkinetic disorder chorea
• Similar problems from subthalamic nucleus
  lesions, also Tourettes, OCD
• Treated with dopamine blockade

disease striatal degeneration
healthy
19
Parkinsons treatment

• Suggested by model, STN lesions (primates) GPi
  lesions in humans alleviate PD symptoms
• huge success of animal research, modeling
• More recently, turned to reversible/tunable deep
  brain (STN) stimulation

(DeLong 1990)
20
Deep-brain stimulation for PD

• Target subthalamic nucleus (usually)
• High frequency rhythmic stimulation
• Mechanism not entirely clear

21
Model of BG disorders

• hypokinetic hyperkinetic disorders caused by
  imbalance in direct/indirect pathways (Arbin et
  al. 1989 Alexander Crutcher 1990)
• Dopamine excites striatal MSNs projecting to
  direct pathway and inhibits those projecting to
  indirect pathway (this is an oversimplification)

(DeLong 1990)
22
Model of BG disorders

• hypokinetic hyperkinetic disorders caused by
  imbalance in direct/indirect pathways (Arbin et
  al. 1989 Alexander Crutcher 1990)
• Dopamine excites striatal MSNs projecting to
  direct pathway and inhibits those projecting to
  indirect pathway (this is an oversimplification)

Hypokinetic (Parkinsons)
Hyperkinetic (Huntingtons)
(DeLong 1990)
(DeLong 1990)
23
The Dopamine Revolution
24
A Parkinsons Brainstem
25
Dopamine responses

• Burst to unexpected reward
• Response transfers to reward predictors
• Pause at time of omitted reward

Schultz et al. 1997
26
The Standard Model Reward Prediction Error
Q(t1) Q(t) ar(t1) - Q(t)
Q(t) Estimate of EU at t r Reward on
last trial
27
Bush and Mosteller
New Association Strength
Old Association Strength


Correction
28
Bush and Mosteller
New Value Estimate
Old Value Estimate


Correction
Old Value Estimate
Obtained Reward
Correction

-
29
Bush and Mosteller
Association Strength
1
2
3
4
5
6
7
8
9
10
Trial Number
30
More dopamine responses
reward following 0 predictive cue
reward following 50 predictive cue
reward following 100 predictive cue
no reward following 100 predictive cue
(Fiorillo et al 2003)
31
(No Transcript)
32
Bayer and Glimcher, 2005, 2007
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
Neuronal Population
N44
37
RPE in Humans
Specific model RPE outcome () lottery
expected value ()
38
(No Transcript)
39
Basal ganglia

• Loop organization
• Input (from cortex) striatum
• Output (back to cortex, via thalamus) globus
  pallidus (internal)

40
Direct and indirect pathways

• Parallel paths through BG
• Opposite effects on thalamus, motor ctx
• direct pathway has 2x inhibition net
  facilitation, go
• indirect pathway has 3x inhibition net
  inhibition, no-go
• Recordings
• Striatum excitation inhibition related to
  movement execution
• GPi inhibition related to movement execution
• Why have two pathways?

Alexander Crutcher 1990
41
Striatal PANs
42

• Post-Saccadic Neurons
• Class 1 Movement Just Completed
• Class 2 Reward Just Received

43

• Qi(t) Coded Before Movement
• Qchosen(t) Coded After Movement

Lau and Glimcher, 2009
44
Dopamine and plasticity

• If dopamine carries a prediction error, where
  does learning happen?
• Potentially, the cortico-striatal synapse

45
(No Transcript)
46
(No Transcript)
47
DA and corticostriatal plasticity
Wickens et al. 1996

• Three-factor learning rule? (pre/post/dopamine)
• wi,t1 wi,t edt

48
Addiction
49
If it is
The Standard RPE Model Addiction (Redish)
Q(t1) Q(t) ar(t1) - Q(t) D
D Dopamine Activation r Reward on
last trial
50
Oculomotor matching taskSearching for Action
Values
Choice
0.10
0.20
Cues
Fix
Rewards arranged using independent reward
probabilities
51
Q(t1) Q(t) ar(t1) - Q(t) D
52
Q(t1) Q(t) ar(t1) - Q(t) D
Example 1
Example 2
Stim On
Stim On
53
End
54
(No Transcript)
55
temporal-difference learning
Rescorla-Wagner Want Vn rn ? (here n
indexes trials, treated as units) Use prediction
error dn rn Vn Temporal-difference learning
(Sutton Barto) Predict cumulative future
reward Want Vt rt rt1 rt2 rt3 ?
(here t indexes time within trial) rt
Vt1 ? (clever recursive trick) Use prediction
error dt rt Vt1 Vt
56
temporal difference learning

• Temporal-difference learning (Sutton Barto)
• Want Vt rt rt1 rt2 rt3
• rt Vt1
• Use prediction error dt rt Vt1 Vt
• learn to predict cumulative future rewards rt
  rt1
• learn using what I predict at time t1 (Vt1) as
  stand in for all future rewards
• so I dont have to wait forever to learn
• learn consistent predictions based on temporal
  difference Vt1 Vt
• if Vt1 Vt, my predictions are consistent
• if Vt1 gt Vt, things got unexpectedly better
• if Vt1 lt Vt, things got unexpectedly worse
• ? and these act like reward to generate
  prediction error and learning

57
More dopamine responses
reward following 0 predictive cue
Prediction error
Vt1 0 dt rt Vt
reward following 50 predictive cue
reward following 100 predictive cue
no reward following 100 predictive cue
(Fiorillo et al 2003)
58
More dopamine responses
reward following 0 predictive cue
Same story here
Vt 0 rt 0 dt Vt1
reward following 50 predictive cue
reward following 100 predictive cue
no reward following 100 predictive cue
(Fiorillo et al 2003)
59
Dopamine responses interpreted
r(t)
V(t)
V(t1) V(t)
d(t) r(t) V(t1) V(t)
r(t)
V(t)
V(t1) V(t)
d(t) r(t) V(t1) V(t)
How should this one look?
(Schultz et al. 1997)
60
Law of Effect

• Of several responses made to the same situation,
  those which are accompanied or closely followed
  by satisfaction to the animal will, other things
  being equal, be more firmly connected with the
  situation, so that, when it recurs, they will be
  more likely to recur.
• Thorndike (1911)

policy p