Chapter 8 Stimulus Control

1
Chapter 8 Stimulus Control
Prepared by Brady J. Phelps, South Dakota State
University
2
Stimulus Control

• When social or nonsocial events precede operant
  behavior and affect its occurrence, they are
  called controlling stimuli.
• A controlling stimulus (S) is said to alter the
  probability of an operant, in the sense that the
  response is more (or less) likely to occur when
  the stimulus is present.
• Discriminative stimulus (SD) - a controlling
  stimulus that sets the occasion for reinforcement
  of an operant.
• S-delta (S?) or extinction stimulus- a stimulus
  that sets the occasion for nonreinforcement or
  extinction of an operant.

3
Emitted versus occasioned
Operants can and do occur in the absence of any
eliciting stimulus, they are said to be freely
emitted. However, when an SD comes to control
occurrences of an operant, to alter its
probability of occurring, then it is said that
the SD occasions the operant. The term occasion
dictates that the operant is under the stimulus
control of an antecedent stimulus. The use of
the term occasion as a verb can be defined as
creating a situation in which something (in this
case, the operant) is especially likely to occur.
4
Differential Reinforcement and Discrimination

• When an organism makes a response in one
  situation but not in another, we say that the
  animal shows a discrimination between the
  situations. The simplest way to train a
  differential response or a discrimination is to
  reinforce an operant in one situation and
  withhold reinforcement in the other.
• Stimulus control refers to a change in behavior
  that occurs when either an SD or S? is presented.
  When an SD is presented, the probability of
  response increases when an S? is presented, the
  probability of response decreases.

5
What do these have in common?

• Visiting a restaurant when neon signs are
  illuminated?

• Running only where the footprints are shallow (or
  numerous)

6

• Cat meows only at the window with lights on

• Drive in the lane with least traffic

• Dressing up for a date

7
Answer All involve stimulus control!

• Stimulus control occurs when 1) a response occurs
  in the presence of a stimulus and 2) does not
  occur in its absence
•   organism is able to discriminate two or more
  different situations
• proof is in the behavior

8
Differential Reinforcement and Discrimination

• When an organism makes a response in one
  situation but not in another, we say that the
  animal shows a discrimination between the
  situations. The simplest way to train a
  differential response or discrimination is to
  reinforce an operant in one situation and
  withhold reinforcement in the other.
• Stimulus control refers to a change in behavior
  that occurs when either an SD or S? is presented.
  When an SD is presented, the probability of
  response increases when an S? is presented, the
  probability of response decreases.

9
How does stimulus control come about?

• Through differential reinforcement
• 1. Reinforce in the presence of one stimulus
• SD response? SR
• Red light press bar? food

Press lever
Food
10
Do not reinforce in the presence of another
stimulus

• S? response? Ext
• Purple light press bar? no food

Press lever
Extinction

• if stimulus control is demonstrated, the rat
  should press bar only when the red light is on,
  and never press when the purple light is on

11

• OR you may present an aversive stimulus as the
  consequence
• purple light press bar? shock

Shock

• purple light called S- or S? because it signals
  extinction or punisher

12
Stimulus Control and Multiple Schedules

• Behavior analysts often use multiple schedules of
  reinforcement to study stimulus control in the
  laboratory.
• On a multiple schedule, two or more simple
  schedules are presented one after the other and
  each schedule is accompanied by a distinctive
  stimulus.

13
Multiple Schedules

• Prolonged exposure to a MULT VI EXT schedule
• Discrimination Index ID (SD rate/SD rate S?)
• An ID of .50

14
Stimulus Control and Neuroscience

• Demonstrating control of behavior by
  discriminative stimuli enable researchers to
  measure what animals can see

15
Stimulus Control and Multiple Schedules

• Behavior analysts often use multiple schedules of
  reinforcement to study stimulus control in the
  laboratory.
• On a multiple schedule, two or more simple
  schedules are presented one after the other and
  each schedule is accompanied by a distinctive
  stimulus.

16
Stimulus Control

• One way to measure the stimulus control exerted
  by the SD and at any moment is to use a
  discrimination index (ID), This index compares
  the rate of response in the SD component to the
  sum of the rates in both SD and S? phases.
• ID(SDrate)/(SDrate S? rate )

17
The case of the bird-brained pigeon

• What is superstitious behavior?
• What does DRO involve and how did it solve the
  question of the bird-brained pigeon?

18
Behavioral Contrast

• Positive or negative contrast
• Operant responding is the dependent variable
• Measured on multiple schedules of reinforcement
• Two or more simple schedules that alternate
• Subject does not get to choose on which
  schedule to respond

19
Behavioral Contrast

• Positive contrast- when rate of response in an
  unchanged setting increases with a decline in
  behavior in another situation.
• Negative contrast- when rate of response
  decreases in an unaltered situation with
  increases in behavior in another setting.

20
Behavioral Contrast (cont.)

• Negative Behavioral Contrast
• A decrease in response rate in one component of a
  multiple schedule of reinforcement because the
  conditions of reinforcement in another component
  have become more favorable
• Positive Behavioral Contrast
• An increase in response rate in one component of
  a multiple schedule because the conditions of
  reinforcement in another component have become
  less favorable

21
Behavioral Contrast

• Positive contrast
• Increase in unchanged component
• Negative contrast
• Decrease in unchanged component

In this example, subjects were reinforced on the
same schedule of reinforcement in phase 1 in
phase 2, responding during the light was on the
same schedule of reinforcement but during the
noise, responding was no longer reinforced and
positive contrast occurred in responding during
the light.
22
Behavioral Contrast (cont.)

• Both positive and negative contrast is commonly
  reported
• Produced by the change in reinforcement in the
  other component
• Relative rates of reinforcement have changed

23
Why contrast?

• Positive contrast increase in unchanged
  component
• Less habituation to the reinforcer in the
  unaltered setting
• The addition of autoshaped pecks in the unaltered
  setting
• Relative rate of reinforcement is increased in
  the unaltered setting

24
The Point Behind Contrast Effects

• Contrast effects are key in demonstrating that
  reinforcement in other situations can alter
  behavior in the present environment

25
Contrast Effects
It is almost a cliché to say that when a child is
acting up at school, i.e., being disruptive,
hyperactive, etc., there must be problems at
home or something is just wrong with this kid.
Maybe, but not so fast Maybe mom and dad have
implemented some strict requirements on doing
homework at home, reducing their sons
reinforcements at home, etc. Based on what is
known about contrast, we could predict that the
childs behavior will increase in some form at
school.
26
Ben Williams on Contrast

• Strong anticipatory contrast
• Weak contrast

27
Generalization

• An organism that responds in one situation but
  not in another is said to discriminate between
  the settings.
• An organism that behaves similarly in different
  situations is said to generalize across
  circumstances.
• Generalization is common in everyday life and is
  not always a bad thing.

28
Stimulus Generalization

• Stimulus generalization occurs when an operant
  that has been reinforced in the presence of a
  specific discriminative stimulus also is emitted
  in the presence of other stimuli. The process is
  called stimulus generalization because the
  operant is emitted to new stimuli that presumably
  share common properties with the discriminative
  stimulus.
• Generalization and discrimination refer to
  differences in the precision of stimulus control.
• Discrimination refers to the precise control of
  an operant by a stimulus, and generalization
  involves less precise regulation of operant
  behavior.

29
Generalization Gradient

• A generalization gradient shows the relationship
  between the probability of response and stimulus
  value.
• The research by Guttman and Kalish (1956)

30
Stimulus Generalization as a Measure of Stimulus
Control
Training S
Pigeons were trained to peck in the presence of a
colored light of 580 nm wavelength and then
tested in the presence of other colors.
After Guttman Kalish, 1956
31
Basic Procedure

• Generalization training Responses to a stimulus
  (S or SD) are reinforced. To assess
  generalization, a variety of stimuli are
  presented, responses to none of which are
  reinforced

32
Basic Procedure
Positive Stimulus
Generalization Test
33
Basic Procedure
Stimulus Color
34

• Discrimination training Responses to one
  stimulus are reinforced, and responses to a
  second stimulus (S- or S?) are put on extinction.

35
Peak Shift

• The effects of discrimination training
• Hanson (1959)
• Control subjects with only VI training with a
  550nm as SD
• Experimental subjects with same SD but with other
  stimulus values established as S?
• The peak of the generalization gradient shifted
  away from the SD in the experimental subjects

36
Peak Shift

• Refers to the change in the peak of a
  generalization gradient away from the stimulus
  that signals extinction.

37
Peak Shift
S
S-
38
Peak Shift

• Body distortions in anorexia nervosa Anorexics
  report that a larger than normal body size must
  be avoided, is too big! and think that a
  thinner than normal body size is the ideal

S
S-
39
Experience Changes the Gradient
Novice
Expert
S
S
40
Absolute and Relative Stimulus Control

• Peak shift is an unusual effect from the point of
  view of absolute control by a stimulus.
• Absolute stimulus control means that the
  probability of response is highest in the
  presence of the stimulus value used in training.
• In fact, absolute stimulus control occurs when
  reinforcement is the only procedure used to
  establish stimulus control.
• The shift in the peak of the generalization
  gradient may reflect relative, rather than
  absolute, stimulus control. Relative stimulus
  control means that an organism responds to
  differences among the values of two or more
  stimuli.

41
Absolute and Relative Stimulus Control

• There are other ways of showing relational
  control by stimuli. To study generalization
  gradients and peak shift, the researcher usually
  arranges the presentation of SD or S? so that one
  follows the other. This is called successive
  discrimination.
• An alternative procedure is labeled simultaneous
  discrimination -the SD and the S? are presented
  at the same time and the organism responds to one
  or the other.

42
Errorless Discrimination and Fading

• When the SD and the S? are alternately presented
  as in successive discrimination, the organism
  initially makes many errors. That is, the animal
  or person continues to respond in the presence
  of the S-delta on the basis of generalization.
• As extinction and reinforcement progress, a
  differential response eventually occurs to the SD
  and S?.

43
Typical Discrimination Training

• Consider the following A pigeon is taught to
  peck a green key for food. Once this behavior is
  well established, the color on the key is changed
  to blue and pecking is not reinforced. The blue
  and green colors are alternately presented and
  the corresponding schedules of extinction or
  reinforcement are in effect. During the early
  sessions, the onset of extinction will generate
  emotional behavior that interferes with ongoing
  operant behavior.

44
Typical Discrimination Training

• Extinction is an aversive procedure. Pigeons
  flap their wings in an aggressive manner and will
  work for an opportunity to attack another bird
  during the presentation to the S? on a multiple
  schedule. Birds will peck a different key if
  pecking turns off the extinction stimulus,
  implying that the stimulus is aversive.

45
Typical Discrimination Training

• There are other problems with successive
  discrimination procedures. Because emotional
  behavior is generated, discriminative responding
  takes a long time to develop. In addition,
  spontaneous recovery of S-delta responding from
  session to session interferes with the
  acquisition of a discrimination. Finally, even
  after extensive training, birds and other
  organisms continue to make errors by responding
  in the presence of the signal for extinction.

46
Errorless Discrimination and Fading

• Errorless discrimination is successful because
  the trainer or teacher does not allow the
  organism to make mistakes by responding to the
  extinction stimulus.
• Errorless discrimination involves gradually
  introducing the S? initially at a very weak
  intensity such that responding to it is very low
  probability. Over repeated trials, the intensity
  of the S? is gradually increased. Eventually the
  S? can be presented in its full intensity and the
  subject will not respond to it. A discrimination
  between the SD and the S? was acquired without
  the errors of responding to the S?.

47
Errorless discrimination

• Training a discrimination by gradually
  introducing the s-delta such that the organism
  makes few (if any) responses to it
• Red key peck disk? reinforcement
• Green key peck disk? no reinforcement
• First introduced as a dark disk and was
  systematically faded into illumination

(Off)
SR
EXT
48
SD stays same
S-delta faded in
By trial 50
By trial 100
By trial 150
49
Result very few errors were made

• Few responses (errors) to the green key
• Under traditional procedure, responses to green
• key must occur for learning to take place
• Most (all) responses to the red key

50
Fading

• Terrace (1966)
• S? introduced early and at low intensity
• Fading procedure used, the S? was faded in
• Sherman (1965) An SD was faded out and replaced
  with another verbal prompt

51
Complex Stimulus Control

• Relational control when responding to a relative
  property is reinforced, e.g., the larger
  (smaller, greener, etc.), that is what is learned

Darker is trained, darker is learned
SR
SR
SR
52
Complex Stimulus Control

• Matching to sample
• Delayed matching to (DMTS) been used to
  investigate behavior said to reflect cognition
  and memory
• Time between the offset of the stimulus and the
  onset of the comparison is known as the retention
  interval
• Theoretically, the organism is covertly doing
  something that helps to retain the information
  about the sample

53
A diagram of a typical matching to sample
procedure with pigeons Correct responses of
matching the comparison stimulus to the
sample stimulus are reinforced. Responses to the
incorrect comparison Stimulus are followed by a
blackout of the chamber lights and extinction.
54
A matching to sample procedure as done with seals
or sea lions.
55
Grant on Forgetting

• In a variation of the DMTS procedure, during the
  delay a specific stimulus was presented
  indicating the matching comparison stimulus would
  be presented after the delay. If another stimulus
  was presented during the delay, the matching
  comparison stimulus would not be presented and no
  correct response was possible.
• Douglas Grant found that the probability of a
  correct response to the comparison stimulus could
  be reduced by presenting the stimulus for no
  correct comparison stimulus and in fact the
  correct comparison stimulus was then presented.
  This effect was termed directed forgetting and
  its effect was varied depending upon when the
  stimulus was presented in the delay interval.
• Attributing directed forgetting to stimulus
  control is a more straightforward explanation
  than to inferred disrupted rehearsal.

56
Here is a matching version of the task (DMTS)
used with pigeons.
Here the sample stimulus is presented, a delay
period is introduced, followed by the
presentation of the comparison stimuli. The
duration of the sample stimulus and the delay
duration are manipulated as variables.
57
White on Remembering

• Remembering is not so much a matter of looking
  into the past or forward into the future as it is
  of making choices at the time of remembering.
• The behavioral approach treats remembering as a
  process of discriminating the relevant events
  from alternative possibilities.
• The discrimination is not made at the time of
  encoding or learning but at the time of
  remembering.
• Remembering does not always get worse with an
  increased retention interval

58

• Discrimination learning
• Typically done with a small number of simple
  stimuli with well-defined and unambiguous
  differences between SD and S?
• Concept Formation
• Use of stimuli with more abstract defining
  features
• Positive defining features or elements (SD),
  defining the stimulus as belonging to the
  concept. A fish is defined by specific stimulus
  features if the features are present in a
  stimulus, the stimulus is a fish.
• Negative defining features or elements (S?),
  defining the stimulus as not belonging to the
  concept, a starfish is clearly not a fish.
  Neither is a whale but a whale is easily
  responded to as belonging to the concept fish.

59
Pigeons and Concept Formation

• Herrnstein - Pigeons
• Pigeons can learn to form concepts of a person as
  discriminated from non-persons, i.e., statues,
  mannequins, etc., a specific person from others,
  etc. The abstract stimulus class of person is
  readily learned.
• Pigeons can learn to form concepts of persons,
  trees, fish, etc., with a high degree of
  accuracy. Concepts such as geometric forms,
  letters, also learned.
• Pigeons appear to respond to two-dimensional
  objects (photographs) as representations of
  three-dimensional objects
• Other species also show concept formation

60
Conditional Discrimination

• An SD or S? can be defined as a function of
  additional contextual stimuli. A pigeon may have
  to learn that if the houselight of an operant
  chamber is illuminated, pecking the green
  response key delivers food. If no houselight,
  pecking the red key delivers food, Thus, if
  houselight present, green key SD, red key S?.
  If no houselight, green key S?, red key SD.
• To a sexual sadist and their masochist partner,
  no means no outside of their sex lives. But
  in the context of sexual activities, the
  masochists cries of No, no, no means Yes,
  yes, yes to the sadist. In the same context,
  pink or red (so-called safe words) will mean
  no or stop.

61

• Pigeons as quality control inspectors
• Pigeons used in search and rescue

62
Conditional Discrimination

• An SD or S? can be defined as a function of
  additional contextual stimuli. A pigeon may have
  to learn that if the houselight of an operant
  chamber is illuminated, pecking the green
  response key delivers food. If no houselight,
  pecking the red key delivers food, Thus, if
  houselight present, green key SD, red key S?.
  If no houselight, green key S?, red key SD.
• To a sexual sadist and their masochist partner,
  no means no outside of their sex lives. But
  in the context of sexual activities, the
  masochists cries of No, no, no means Yes,
  yes, yes to the sadist. In the same context,
  pink or red (so-called safe words) will mean
  no or stop.
• The rubber-hand illusion as a conditional
  discrimination

63
Stimulus Control in Your Life

• Sleeping and Insomnia- one of the most common
  causes of insomnia is poor sleep hygiene, reading
  in bed, watching TV in bed, etc., then being
  unable to fall or stay asleep in bed.
• Therapy Get out of bed to read, get out of bed
  to watch TV, if you are not falling asleep, get
  out of bed.
• Only go to bed and try to sleep when you are
  feeling tired enough to sleep.
• The bed acquires stimulus control over your
  sleeping.