PSY 369: Psycholinguistics

1
PSY 369 Psycholinguistics

• Language Comprehension
• Sentence comprehension

2
Overview of comprehension
3
Comprehending sentences
The man hit the dog with the leash.

• Theory question
• How do we know which structure to build?
• Methodological question
• What methods can we use to help us answer
  theoretical questions like these?

4
Commonly used methods

• Remember, the output of comprehension processes
  are internal mental events. What kinds of
  measures have been used to test theories?
• Cross-modal priming
• E.g., listen to a sentence, make a lexical
  decision to a visually presented word
• Comprehension measure
• Read a sentence/passage and then answer questions
  about what you read (decision time, accuracy)
• Measure how long people actually spend reading
• Line by line reading
• Word by word reading
• Using eye movement monitoring techniques
• Neuropsychological methods
• E.g., ERPs, fMRI

5
Line-by-line

• A banker is a fellow

6
Line-by-line
who lends you his umbrella
7
Line-by-line
when the sun is shining
8
Line-by-line
but wants it back
9
Line-by-line
the minute it begins to rain.
10
Line-by-line

• Problem
• Overall reading time for entire sentence or
  phrase
• need for more on-line measurements
• Timing on a smaller scope
• See effects at level of word

11
Word-by-word

• A couple of methods
• RSVP (rapid serial visual presentation)
• Moving window

12
Word-by-word

• RSVP (rapid serial visual presentation)
• Experimenter presents the sentence one word a
  time.
• Typically the experimenter controls the
  presentation rate.

13
Word-by-word

• A

14
Word-by-word
lie
15
Word-by-word
can
16
Word-by-word
travel
17
Word-by-word
halfway
18
Word-by-word
around
19
Word-by-word
the
20
Word-by-word
world
21
Word-by-word
while
22
Word-by-word
the
23
Word-by-word
truth
24
Word-by-word
is
25
Word-by-word
putting
26
Word-by-word
on
27
Word-by-word
its
28
Word-by-word
shoes.
29
Word-by-word

• Moving window

30
Word-by-word

• I xxxx xxxxx xxx xx xxxxxxxxx xxxxxxxxx xxxx xx
  xxxxxxxxx.

31
Word-by-word

• x have xxxx xxx xx xxxxxxxxx xxxxxxxxx xxxx xx
  xxxxxxxxx.

32
Word-by-word

• x xxxx never xxx xx xxxxxxxxx xxxxxxxxx xxxx xx
  xxxxxxxxx.

33
Word-by-word

• x xxxx xxxxx let xx xxxxxxxxx xxxxxxxxx xxxx xx
  xxxxxxxxx.

34
Word-by-word

• x xxxx xxxxx xxx my xxxxxxxxx xxxxxxxxx xxxx xx
  xxxxxxxxx.

35
Word-by-word

• x xxxx xxxxx xxx xx schooling xxxxxxxxx xxxx xx
  xxxxxxxxx.

36
Word-by-word

• x xxxx xxxxx xxx xx xxxxxxxxx interfere xxxx xx
  xxxxxxxxx.

37
Word-by-word

• x xxxx xxxxx xxx xx xxxxxxxxx xxxxxxxxx with xx
  xxxxxxxxx.

38
Word-by-word

• x xxxx xxxxx xxx xx xxxxxxxxx xxxxxxxxx xxxx my
  xxxxxxxxx.

39
Word-by-word

• x xxxx xxxxx xxx xx xxxxxxxxx xxxxxxxxx xxxx xx
  education.

40
Word-by-word

• A couple of methods
• RSVP (rapid serial visual presentation)
• Moving window
• Better, more on-line
• But, these measures are also a little bit
  unnatural (especially RSVP)
• e.g., Dont allow regressions (looking back)

41
Eye-movements in reading

• One of the most common measures used in sentence
  comprehension research is measuring Eye-movements

42
How the eye works

• At its center is the fovea, a pit that is most
  sensitive to light and is responsible for our
  sharp central vision.
• The central retina is cone-dominated and the
  peripheral retina is rod-dominated.

43
How the eye works

• Limitations of the visual field
• 130 degrees vertically, 180 degrees horizontally
  (including peripheral vision
• Perceptual span for reading 7-12 spaces

Clothes make the man. Naked people have little
or no influence on society.
44
How the eye works

• Within the visual field, eye movements serve two
  major functions
• Saccades to Fixations Position target objects
  of interest on the fovea
• Tracking Keep fixated objects on the fovea
  despite movements of the object or head

• Eye-movements in reading are saccadic rather than
  smooth

Clothes make the man. Naked people have little
or no influence on society.
Clothes make the man. Naked people have little
or no influence on society.
Video examples 1 2 3 4 5
45
Smooth Pursuit

• Smooth movement of the eyes for visually tracking
  a moving object
• Cannot be performed in static scenes
  (fixation/saccade behavior instead)

46
Saccades

• Saccades are used to move the fovea to the next
  object/region of interest.
• Connect fixations
• Duration 10ms - 120ms
• Very fast (up to 700 degrees/second)
• Ballistic movements (pre-programmed)
• About 150,000 saccades per day
• No visual perception during saccades
• Vision is suppressed
• Evidence that some cognitive processing may also
  be suppressed during eye-movements (Irwin, 1998)

47
Saccades
Without suppression
Move to here
With suppression
Move to here
Video example (around 5 min mark)
48
Smooth Pursuit versus Saccades

• Saccades
• Jerky
• No correction
• Up to 700 degrees/sec
• Background is not blurred (saccadic suppression)

• Smooth pursuit
• Smooth and continuous
• Constantly corrected by visual feedback
• Up to 100 degrees/sec
• Background is blurred

49
Fixations

• The eye is (almost) still perceptions are
  gathered during fixations
• 90 of the time the eye is fixated
• duration 150ms - 600ms
• In reading, the assumption is that the length of
  fixation is correlated with amount/type of
  processing being done at that point (on that
  word, at that point in the syntactic parse)

Video examples 1 2 3 4 5

• Article/video (if I can get it to work)
• Raney, G. E., Campbell, S. J., Bovee, J. C.
  (2014). Using Eye Movements to Evaluate the
  Cognitive Processes Involved in Text
  Comprehension. J. Vis. Exp. (83), e50780,
  doi10.3791/50780.

50
Measuring Eye Movements

• Purkinje Eye Tracker
• Laser is aimed at the eye.
• Laser light is reflected by cornea and lens
• Pattern of reflected light is received by an
  array of light-sensitive elements.
• Very precise
• Also measures pupil accommodation (switching
  between looking far or close)
• No head movements

51
Measuring Eye Movements

• Video-Based Systems
• Infrared camera directed at eye
• Image processing hardware determines pupil
  position and size (and possibly corneal
  reflection)
• Good spatial precision (0.5 degrees) for
  head-mounted systems
• Good temporal resolution (up to 500 Hz) possible

52
Eye movements in reading
The man hit the dog with the leash.

• Theory question
• How do we know which structure to build?
• Methodological question
• How can we use eye-movements to help us answer
  the theoretical question

53
Parsing

• The syntactic analyser or parser
• Main task To construct a syntactic structure
  from the words of the sentence as they arrive
• Main research question how does the parser make
  decisions about what structure to build?

54
Eye movements in reading
The man hit the dog with the leash.
S
NP
N
det
The
man
55
Eye movements in reading
The man hit the dog with the leash.
S
NP
VP
V
N
det
The
man
hit
56
Eye movements in reading
The man hit the dog with the leash.
S
NP
VP
V
NP
NP
N
det
N
det
The
man
hit
dog
the
57
Eye movements in reading
The man hit the dog with the leash.
S
NP
VP
V
NP
NP
Modifier
N
det
N
det
The
man
hit
dog
the
58
Eye movements in reading
The man hit the dog with the leash.
S
NP
VP
V
NP
Instrument
NP
N
det
N
det
The
man
hit
dog
the
59
Sentence Comprehension

• There are many examples of syntactic ambiguity
• A vast amount of research focuses on Garden path
  sentences
• A garden path sentence invites the listener to
  consider one possible parse, and then at the end
  forces him to abandon this parse in favor of
  another.

The horse raced past the barn fell.
Actual Newspaper Headlines

• Juvenile Court to Try Shooting Defendant
• Red tape holds up new bridge
• Miners Refuse to Work after Death
• Retired priest may marry Springsteen
• Local High School Dropouts Cut in Half
• Panda Mating Fails Veterinarian Takes Over
• Kids Make Nutritious Snacks
• Squad Helps Dog Bite Victim
• Hospitals are Sued by 7 Foot Doctors

60
Different approaches

• Immediacy Principle access the meaning/syntax of
  the word and fit it into a syntactic structure
• Serial Analysis (Modular) Build just one based
  on syntactic information and continue to try to
  add to it as long as this is still possible

61
Sentence Comprehension

• The horse raced past the barn fell.

62
Sentence Comprehension

• The horse raced past the barn fell.

S
VP
NP
V
raced
The horse
63
Sentence Comprehension

• The horse raced past the barn fell.

S
VP
NP
V
PP
P
NP
raced
past
The horse
64
Sentence Comprehension

• The horse raced past the barn fell.

S
VP
NP
V
PP
P
NP
raced
past
the barn
The horse
65
Sentence Comprehension

• The horse raced past the barn fell.

S
VP
NP
V
PP
P
NP
raced
past
the barn
The horse
fell
66
Sentence Comprehension

• The horse raced past the barn fell.

• raced is initially treated as a past tense verb

S
VP
NP
V
PP
P
NP
raced
past
the barn
The horse
67
Sentence Comprehension

• The horse raced past the barn fell.

• raced is initially treated as a past tense verb
• This analysis fails when the verb fell is
  encountered

S
VP
NP
V
PP
P
NP
raced
past
the barn
The horse
fell
68
Sentence Comprehension

• The horse raced past the barn fell.

• raced is initially treated as a past tense verb
• This analysis fails when the verb fell is
  encountered
• raced can be re-analyzed as a past participle.

S
VP
NP
V
PP
P
NP
raced
past
the barn
The horse
fell
69
Different approaches

• Immediacy Principle access the meaning/syntax of
  the word and fit it into a syntactic structure
• Serial Analysis (Modular) Build just one based
  on syntactic information and continue to try to
  add to it as long as this is still possible
• Interactive Analysis Use multiple levels (both
  syntax and semantics) of information to build the
  best structure

70
Different approaches

• Immediacy Principle access the meaning/syntax of
  the word and fit it into a syntactic structure
• Serial Analysis (Modular) Build just one based
  on syntactic information and continue to try to
  add to it as long as this is still possible
• Interactive Analysis Use multiple levels (both
  syntax and semantics) of information to build the
  best structure
• Parallel Analysis Build both alternative
  structures at the same time
• Minimal Commitment Stop building - and wait
  until later material clarifies which analysis is
  the correct one.

71
A serial model

• Formulated by Lyn Frazier (1978, 1987)
• Build trees using syntactic cues
• phrase structure rules
• plus two parsing principles
• Minimal Attachment
• Late Closure
• Go back and revise the syntax if later semantic
  information suggests things were wrong

72
A serial model

• Minimal Attachment
• Prefer the interpretation that is accompanied by
  the simplest structure.
• simplest fewest branchings (tree metaphor!)
• Count the number of nodes branching points

The girl hit the man with the umbrella.
73
Minimal attachment
S
8 Nodes
NP
VP
the girl
V
NP
Preferred
S
hit
NP
PP
NP
VP
the man
P
NP
the girl
V
NP
PP
with
the umbrella
hit
the man
P
NP
with
the umbrella
9 nodes
The girl hit the man with the umbrella.
74
A serial model

• Late Closure
• Incorporate incoming material into the phrase or
  clause currently being processed.
• OR
• Associate incoming material with the most recent
  material possible.

She said he tickled her yesterday
75
Parsing Preferences .. late closure
S
Preferred
S
np
vp
np
vp
she
v
S'
adv
she
v
S'
said
np
vp
yesterday
said
np
vp
he
v
np
he
v
np
adv
tickled
her
tickled
her
yesterday
(Both have 10 nodes, so use LC not MA)
She said he tickled her yesterday
76
Minimal attachment

• Garden path sentences

(Rayner Frazier, 83)
The spy saw the cop with a telescope.
minimal attach
Build this structure first
non-minimal attach
Build this structure first
77
Minimal attachment

• Garden path sentences

(Rayner Frazier, 83)
The spy saw the cop with a revolver.
minimal attach
Build this structure first
non-minimal attach
Build this structure first
78
MA
Non-MA
The spy saw the cop with the binoculars.. The
spy saw the cop with the revolver (Rayner
Frazier, 83)
lt- takes longer to read
79
Interactive Models

• Other factors (e.g., semantic context,
  co-occurrence of usage expectation) may provide
  cues about the likely interpretation of a sentence

• Trueswell et al (1994)

• The evidence examined by the lawyer
• The defendant examined by the lawyer

80
Interactive Models

• Other factors (e.g., semantic context,
  co-occurrence of usage expectation) may provide
  cues about the likely interpretation of a sentence

• Trueswell et al (1994)

• The evidence examined by the lawyer
• The defendant examined by the lawyer

A defendant can be examined but can also do
examining.
81
Semantic expectations

• Other factors (e.g., semantic context,
  co-occurrence of usage expectation) may provide
  cues about the likely interpretation of a sentence

• Taraban McCelland (1988)
• Expectation

• The couple admired the house with a friend but
  knew that it was over-priced.
• The couple admired the house with a garden but
  knew that it was over-priced.

82
Semantic expectations

• Taraban McCelland, 1988

• The couple admired the house with a friend but
  knew that it was over-priced.
• The couple admired the house with a garden but
  knew that it was over-priced.

The Non-MA structure may be favoured
83
What about spoken sentences?

• All of the previous research focused on reading,
  what about parsing of speech?
• Methodological limits ear analog of
  eye-movements not well developed
• Auditory moving window
• Reading while listening
• Looking at a scene while listening
• Some research on use of intonation

84
Intonation as a cue

• A Id like to fly to Davenport, Iowa on TWA.
• B TWA doesnt fly there ...
• B1 They fly to Des Moines.
• B2 They fly to Des Moines.

85
Chunking, or phrasing

• A1 I met Mary and Elenas mother at the mall
  yesterday.
• A2 I met Mary and Elenas mother at the mall
  yesterday.

86
Phrasing can disambiguate
Mary Elenas mother
mall
I met Mary and Elenas mother at the mall
yesterday
One intonation phrase with relatively flat
overall pitch range.
87
Phrasing can disambiguate
Elenas mother
mall
Mary
I met Mary and Elenas mother at the mall
yesterday
Separate phrases, with expanded pitch movements.
88
Summing up

• Is ambiguity resolution a problem in real life?
  
• Yes (Try to think of a sentence that isnt
  partially ambiguous)
• Many factors might influence the process of
  making sense of a string of words. (e.g. syntax,
  semantics, context, intonation, co-occurrence of
  words, frequency of usage, )