Lipreading and Audiovisual Discourse Comprehension by Older Adults in Favorable and Unfavorable Cond

1
Lipreading and Audiovisual Discourse
Comprehension by Older Adults in Favorable and
Unfavorable Conditions

• Nancy Tye-Murray, Ph.D.
• Washington University School of Medicine
• St. Louis, MO

2
Hearing and Vision Decline with Age

• 30-60 of people aged 65 yrs and older have some
  degree of hearing loss.
• 10-30 have vision impairment, even with
  corrective lenses.

3
Auditory Speech Recognition Declines with Age

• Age-related differences in speech perception are
  observed even when younger and older adults are
  matched for hearing ability.

4
Vision-only and Auditory-Visual Speech
Recognition Also Declines with Age

• Results from our 2000-2005 experiments, consonant
  and sentence stimuli
• 50 older adults, 50 younger adults, all have
  normal hearing (similar findings for older adults
  who have hearing loss)
• (Note in particular poor performance for
  sentences)

5
Auditory Enhancement

• AV V-only/ 1 V
• (A performance is controlled)

6
Results for A enhancement (No sig. differences
cons p.200, sent p .242)
7
Integration Enhancement

• Based on probability formula
• Designed to account for the enhancement, or
  benefit, associated with integrating two channels
  of information
• Indexes AV integration ability by first removing
  the amount of AV performance that could be
  accounted for by unimodal scores (A and V-Only),
  then normalizing for the amount of improvement
  that could have been achieved.

8
Integration Enhancement
B
A

• A Amount AV improved beyond the prediction
  1-(1-A)(1-V)
• B Amount that cold have been improved (AV -
  1-(1-A)(1-V))

9
Integration Enhancement (No sig. differences cons
p.062, sent p .429
10
Goals of the Present Investigation

• 1. To predict older persons abilities to
  lipread in favorable and unfavorable conditions
  (i.e., vision-only).
• 2. To examine older persons abilities to
  comprehend discourse under both favorable and
  unfavorable conditions in an audiovisual
  condition to compare their performance to young
  adults and to identify predictors of
  performance.
• Favorable Good sound, good visual clarity (as
  when one has good hearing and good vision)
• Unfavorable Degraded sound, degraded visual
  clarity (as when one has hearing and vision
  impairment)

11
Predicting Lipreading Performance in Older Adults

• In favorable and unfavorable conditions

12
Previous Research with Adults

• Variables that appear not to be predictive
  verbal intelligence, educational achievement,
  acquired hearing loss, personality, visual memory
• Variables that may be predictive working
  memory, processing speed, verbal inference-making
  (e.g., how well a person can complete sentences
  that have missing words) (Andersson, Lyxell,
  Rönnberg, Spens, 2001)

13
Methods

• 93 Older adults (age appropriate hearing loss)
• Avg 74.7 yrs. (SD 5.7)
• Avg PTA Better Ear 23.1 dB (SD11.0)
• 38 Young adults (normal hearing)
• Avg 22.7 yrs. (SD 2.2)
• Avg PTA Better Ear 3.2 dB (SD6.1)
• Build-A-Sentence (BAS) Test

14
BAS
15
Attractive Features of the BAS

• Avoid floor effects associated with most sentence
  tests of lipreading
• Ensure that the same stimuli are used across
  conditions e.g., A,V, AV Favorable and
  Unfavorable

16
BAS
17
BAS
18
Scoring

• Frames correct
• Words in any position correct
• Words in position correct

19
Predictor Variables (Older Adults Only)

• a. Pure-tone average in better ear
• b. Working memory letter number span odd-even
  digit span
• c. Processing speed lexical decision speed
  rhyme judgment
• d. Word knowledge Vocabulary subset of Weschler
  Adult Intelligence Scale (WAIS-V) e.g., Define
  the word colony and the definition subtest of
  the Kaufman Adolescent and Adult Intelligence
  Test (KAIT) e.g., BR_W_
• e. Age at time of testing

20
Working Memory

• Letter number span score entails presenting a
  series of interspersed letters and numbers e.g.,
  D-B-3-F-5-9-W-2. The participants task is to
  repeat back the letters in alphabetic order and
  then the numbers in ascending order.
• Odd-even digit span score entails presenting a
  string of digits to the participant, e.g.,
  3-7-2-5-8-4. After each digit, the participant
  must declare whether it is odd or even. After
  the string is presented, the participant repeats
  back as many digits as possible in the order of
  presentation. Long and short strings are
  randomly presented. The score is the longest
  string that the participant correctly repeats.

21
Processing Speed

• Lexical decision time is the average amount of
  time, in milliseconds, it takes a participant to
  determine if a written word is a real English
  word or not, e.g., blug
• Rhyme judgment time is the average amount of
  time, in milliseconds, it takes a participant to
  determine if two written English words rhyme.
  The two words are always orthographically
  different e.g., blue and flew

22
Working Memory andReaction Time Results, Older
Adults
Reaction Time
Working Memory
23
Performance for V-only BAS in good and poor
(i.e., favorable and unfavorable)
80
70
60
50
Older
Avg Percent Correct
40
Young
30
20
10
0
BAS Easy V
BAS Hard V
24
Results of Hierarchical Regression Analyses
(Older Adults)

• Favorable
• 16.9 of the variance was accounted for by age
  an additional 6.7 was accounted for by working
  memory.
• Unfavorable
• 10.1 of the variance was accounted for by age
  no other variables were predictive

25
Why Study Discourse Comprehension?

• Daily encounters with verbal information are both
  visual (i.e., reading text) and auditory or
  auditory-visual (spoken language comprehension).
  Research has established that as we age, we
  become less efficient readers. Very little is
  known about how listening comprehension is
  affected by aging. Declines in both hearing and
  vision are only suggestive.
• Most face-to-face communication entails
  understanding discourse e.g., instructions,
  directions, news, sermons, lectures, so it is
  ecologically valid to do so.

26
Measuring Discourse Comprehension in Favorable
and Unfavorable Conditions

• Auditory signal Conditions are set to either
  favorable or unfavorable. Six-talker babble
  is added and the signal/noise ratio is varied to
  establish good performance (s/n 5) or poor
  (s/n ratio -5).
• Visual signal Contrast is set to either
  favorable (high) or unfavorable (low).

27
LISN Discourse Comprehension Test

• Stimulus types lectures, interviews, narratives
  (LISN Lectures, Interviews, Spoken Narratives)
• Question types information, inference,
  integration
• Four forms each form includes two of each
  stimulus type for a total of 6 passages each

28
Scoring

• Total percent correct
• Question type percent correct
• Passage type percent correct

29
LISN Development

• Selection of passages (real transcripts from such
  sources as NPR). Common knowledge? Gender
  specific? Age specific?
• Recording of passages by professional actors
• Assessment of questions, e.g., can they be
  answered without hearing passage? Can they be
  answered after hearing the passages? Are
  questions rated as being informative,
  integrative, or interferential?
• List equivalency and test-retest reliability
• Floor and ceiling effects
• Performance on LISN versus performance on other
  tests such as reading comprehension (e.g., WAIS,
  Woodcock Johnson) and other measures of listening
  comprehension, such as recorded SAT passages and
  Nelson Denney

30
Sample, good-good Eisenhower (5 s/n, 0
contrast degradation )
31
Sample Questions

• 1. What would the speaker probably
  say about his placement in the military?
• A) It was very mundane work.
• B) It was an entertaining position to take.
• C) It was constantly emotional.
• D) It was hard work.
• 2. Who was C. Merriman Smith?
• A) Head of Secret Service.
• B) Colonel of the U.S. Air Force.
• C) President of the White House press corps.
• D) A dignitary who was there to meet president
  Nixon.
• 3. What is the speakers claim to fame?
• A) He actually met Richard Nixon.
• B) He ordered C. Merriman Smith to move his car.
• C) He was part of the chain of command that
  informed Nixon of his unzipped fly.
• D) He was inside the cordon when Nixon was
  greeting the dignitaries.

32
Answers and what kind of question each one is

• 1-Inference
• B) It was a very entertaining position to
  take.
• 2-Information
• C) President of the White House press corps.
• 3-Integration
• C) He was part of the chain of command that
  informed Nixon of his unzipped fly.

33
Sample, poor-poor Rutgers University (-5
s/n ratio 98 contrast degradation)
34
Discourse Comprehension

• Comparison between young and old adults
• Predicting performance of older adults

35
Participants

• Older Participants
• 54 persons with age appropriate hearing,
  pure-tone averages (M 27.8, SD10.8). Ranged in
  age between 66 and 88 yrs (M 74.9, SD 6.0).
• Younger Participants
• 38 persons with clinically normal hearing.
  Ranged in age between 18 and 27 yrs (M22.7). PTA
  lt 25 dB (M 3.2).

36
Results for the LISN Older Adults
RM ANOVA sig dif for Age (plt.001), sig dif for
Condition (plt.001), no interaction
37
Predictor Variables

• a. Pure-tone-average
• b. Vision-only speech recognition BAS
  (favorable)
• c. Integration enhancement BAS (favorable for
  the easy, then unfavorable for the hard)
• d. Working memory (letter-number span odd-even
  digit span)
• e. Processing speed (rhyme judgment test word
  decision test)
• f. Word knowledge
• h. Age at time of testing

38
Results of Hierarchical Regression Analyses,
Older Persons Only

• Favorable
• 26.1 of the variance was accounted for by the
  word knowledge score no other variables were
  predictive.
• Unfavorable
• 13.4 of the variance was accounted for by the
  PTA 6.4 additional variance was accounted for
  by lipreading ability no other variables were
  predictive.

39
Conclusion

• When viewing conditions are favorable, age and
  working memory comes into play as a factor that
  influences lipreading performance in older
  adults.
• When viewing conditions are unfavorable, age is
  the only predictive variable.
• When viewing and listening conditions are
  favorable, vocabulary knowledge comes into play
  during discourse comprehension.
• When viewing and listening conditions are
  unfavorable, listening and lipreading abilities
  come into play.

40
Interpretation

• When viewing (and listening) conditions are
  unfavorable, more attention must be directed at
  simply understanding the words in the sentences
  or passages and working memory and/or vocabulary
  knowledge has less impact.

41
Project Outcomes

• Better understanding of how age affects
  speechreading performance and ones ability to
  integrate what is seen with what is heard.
• Clinical tools for assessing integration ability
  (BAS) and discourse comprehension (LISN).
• Better understanding of why some older persons
  perform better on comprehension tasks than do
  others.

42
Acknowledgements

• NIH NIA RO1 and NIH NIDCD RO1
• Collaborators Mitch Sommers, Brent Spehar,
  Sandra Hale, and Joel Myerson