Assumptions of RW model

1
Assumptions of R-W model

• helpful for the animal to know 2 things about
  conditioning
• what TYPE of event is coming
• the SIZE of the upcoming event
• Thus, classical conditioning is really learning
  about
• signals (CS's) which are PREDICTORS for
• important events (US's)
• model assumes that with each CS-US pairing 1 of 3
  things can happen
• the CS might become more INHIBITORY
• the CS might become more EXCITATORY
• there is no change in the CS
• how do these 3 rules work?
• if US is larger than expected CS excitatory
• if US is smaller than expected CS inhibitory
• if US expectations No change in CS
• The effect of reinforcers or nonreinforcers on
  the change of associative strength depends upon
• the existing associative strength of THAT CS
• AND on the associative strength of other stimuli
  concurrently present

2
More assumptions

• Explanation of how an animal anticipates what
  type of CS is coming
• direct link is assumed between "CS center" and
  "US center" e.g. between a tone center and food
  center
• assumes that STRENGTH of an event is given and
  that the conditioning situation is predicted by
  the strength of this connection
• THUS when learning is complete the strength of
  the association relates directly to the size or
  intensity of the CS
• The change in associative strength of a CS as the
  result of any given trial can be predicted from
  the composite strength resulting from all stimuli
  presented on that trial
• if composite strength is low, the ability of
  reinforcer to produce increments
  in the strength of component stimuli is HIGH
• if the composite strength is low reinforcement
  is relatively less effective (LOW)

3
More assumptions

• Can expand to extinction, or nonreinforced
  trials
• if composite associative strength of a stimulus
  compound is high, then the degree to which a
  nonreinforced presentation will produce a
  decrease in associative strength of the
  components is LARGE
• if composite associative strength is low-
  nonreinforcement effects reduced
• Yields an equation
• Vi aißj(?j-Vsum)

4
First example

• rat is subjected to conditioned suppression
  procedure
• CS (light) ---gt US (1 mA shock)
• what is associative strength?
• 1 associative strength that a 1mA shock can
  support at asymptote ( ?j )
• VL associative strength of the light (strength
  of the CS-US association)
• thus ?1 size of the observed event (actual
  shock)
• VL measure of the Subjects current
  "expectation" about the size of the shock
• VL will approach ?1 over course of conditioning

5
Second example Same rat, same procedure but
2CS's

• CS (lighttone) --gt 1 mA shock
• Determine associative strength when ?1 is
  constant
• Vsum VL VT assoc. strength of the 2 CS's
• Vsum aißj(?)
• if VL and VT equally salient
• VL 0.5aißj
• VT 0.5aißj
• VT if not equally salient VL gt VT or VL lt
  VT
• now can restate the 3 rules of conditioning
• ?j gt Vsum excitatory conditioning
• ?j lt Vsum inhibitory conditioning
• ?j Vsum no change

6
Now have the Rescorla-Wagner Model

• model makes predictions on a trial by trial basis
• for each trial predicts increase or decrement in
  associative strength for every CS present
• The equation Vi aißj(?j -Vsum)
• (1) Vi change in associative strength that
  occurs for any CS, i, on a single trial
• (2) ?j associative strength that some US, j, can
  support at
• asymptote
• (3) Vsum associative strength of the sum of the
  CS's (strength of
• CS-US pairing)
• (4) ai measure of salience of the CS (must have
  value between 0
• and 1)
• (5)ßj learning rate parameters associated with
  the US (assumes
• that different beta values may depend upon the
  particular US employed)

7
Assumptions of the formal model

• General Principle as Va increases with repeated
  reinforcement of j,
• the difference between ?a and Va decreases
• increments of Va then decrease
• produce negatively accelerated learning curve
  with asymptote of ?j
• Reinforcement of compound stimuli lots of Va
  trials, then give trials of compound Vax
• Va increases toward ?a as a result of a-alone
  presentations
• Vax then exceeds ?a
• result reinforced AX trial results in DECREMENT
  to the associative strength of a and X
  components
• as A and AX are reinforced
• increments to A occur on the reinforced A trials
• increments to A and X occur on reinforced AX
  trials
• result transfer to A of whatever associative
  strength X may have

8
The equation Vi aißj(?j-Vsum)

• Vi change in associative strength that occurs
  for any CS, i, on a single trial
• ai stimulus salience (assumes that different
  stimuli may acquire associative strength at
  different rates, despite equal reinforcement)
• ßj learning rate parameters associated with the
  US (assumes that different beta values may depend
  upon the particular US employed)
• Vsum associative strength of the sum of the
  CS's (strength of CS-US pairing)
• ?j associative strength that some CS, i, can
  support at asymptote
• In English How much you learn on a given trial
  is a function of the value of the stimulus x
  value of the reinforcer x (the absolute amount
  you can learn minus the amount you have already
  learned).

9
Acquisition

• first conditioning trial CS light US 1 ma
  Shock
• Vsum Vl no trials so Vl 0
• thus ?j-Vsum 100-0 100
• -first trial must be EXCITATORY
• BUT must consider the salience of the light ai
  1.0 and learning rate ßj 0.5
• Plug into the equatio for TRIAL 1
• Vl (1.0)(0.)(100-0) 0.5(100) 50
• thus V only equals 50 of the discrepancy
  between Aj an Vsum for the first trial
• TRIAL 2
• V1 (1.0)(0.5)(100-50) 0.5(50) 25
• Vsum (5025) 75
• TRIAL 3
• V1 (1.0)(0.5)(100-75) 0.5(25) 12.5
• Vsum (502512.5) 87.5

10
Overshadowing

• Pavlov compound CS with 1 intense CS, 1 weak
• after a number of trials found strong CS
  elicits strong CR
• weak CS elicits weak or no CR
• Rescorla-Wagner model helps to explain why
  assume
• aL light 0.2 aT tone 0.5
• ßL light 1.0 ßt tone 1.0
• Plug into equation
• Vsum Vl Vt 0 on trial 1
• Vl 0.2(1)(100-0) 20
• Vt 0.5(1)(100-0) 50
• after trial 1 Vsum 70
• TRIAL 2
• Vl 0.2(1)(100-(5020)) 6
• Vt 0.5(1)(100-(5020)) 15
• Vsum (70(615)) 91

11
Blocking

• similar explanation to overshadowing
• no matter whether VL more or less salient than
  Vt, because CS has basically absorbed all the
  assoc. strength that the CS can support
• give trials of A-alone to asymptote
• reach asymptote VL ?j 100 Vsum
• aL 1.0
• ß 0.2
• First Vt Trial Vt aß(?j-Vsum)
• Vt0.21.0(100-100)?
• No learning!

12
How could one eliminate blocking effect?

• increase the intensity of the US to 2 mA with ?j
  now equals 160
• then Vsum still equals 100 (learned to 1 mA
  shock)
• plug into the equation (assume Vl and Vt equally
  salient)
• Vt 0.2(1)(160-100) 0.2(60) 12
• Vl 0.2(1)(160-100) 0.2(60) 12
• on trial 2
• Vsum 124
• Vt 0.2(1)(160-124) 0.2(36) 7.2
• Vl 0.2(1)(160-124) 0.2(36) 7.2
• Vsum now (12414.4) 138.
• could also play around with ß

13
Can also explain why probability of reward given
CS vs no CS makes a difference

• p probability of US given the CS or No US given
  No CS
• can make up three rules
• if pax gt pa then Vx should be POSITIVE
• if pax lt pa then Vx should be NEGATIVE
• if pax pa then Vx should be ZERO
• modified formula (assume ?1 1.0 ?2 0 ß1
  .10 ß2.05 a1.10 a2.5)
• Va paß1
• ----------------------
• paß1 - (1-pa)ß2
• Vax paxß1
• ----------------------
• paxß1 - (1-pax)ß2
• Vx Vax - Va

14
PLUG IN Probability of CSa then US 0.2
Probability of CSax then US 0.8

• Va (0.2)(1.0)
• --------------------------- -10
• ((.2)(.10)) - (1-.2)(.05)
• Vax (0.8)(1.0)
• --------------------------- 11.43
• ((.8)(.10)) - (1-.8)(.05)
• Vx Vax - Va or 11.43-(-10) 21.43
• probability of US given AX greater than
  probability of US given X)

15
PLUG IN Probability of CSa then US 0.8
Probability of CSax then US 0.2

• Va (0.8)(1.0)
• --------------------------- 11.43
• ((.8)(.10)) - (1-.8)(.05)
• Vax (0.2)(1.0)
• --------------------------- -10
• ((.2)(.10)) - (1-.2)(.05)
• Vx Vax - Va or -10 - 11.43 -21.43
• probability of US given AX is less than
  probability of US given A

16
PLUG IN Probability of CSa then US 0.5
Probability of CSax then US 0.5

• Va (0.5)(1.0)
• --------------------------- 20
• ((.5)(.10)) - (1-.5)(.05)
• Vax (0.5)(1.0)
• --------------------------- 20
• ((.5)(.10)) - (1-.5)(.05)
• Vx Vax - Va or 20-20 0 (probability of AX
  A)

17
Critique of the Rescorla-Wagner Model

• R-W model really a theory about the US
  effectiveness
• says nothing about CS effectiveness
• states that an unpredicted US is effective in
  promoting learning, whereas a well-predicted US
  is ineffective
• Fails to predict the CS-pre-exposure effect
• two groups of subjects (probably rats)
• Grp I CS-US pairings Control
• Grp II CS alone CS-US pairings PRE-Expos
• pre-exposure group shows much less rapid
  conditioning than the control group
• R-W model doesn't predict any difference, because
  no conditioning trials occur when CS is
  predicted alone Vsum 0
• BUT may be that salience for the CS is
  changing
• habituation to CS
• Original R-W model implies that salience is fixed
  for any given CS
• R-W assume CS salience doesn't change
  w/experience
• these data strongly suggest CS salience DOES
  change w/experience
• Newer data supports changes salience