Title: Lipreading and Audiovisual Discourse Comprehension by Older Adults in Favorable and Unfavorable Cond
1Lipreading and Audiovisual Discourse
Comprehension by Older Adults in Favorable and
Unfavorable Conditions
- Nancy Tye-Murray, Ph.D.
- Washington University School of Medicine
- St. Louis, MO
2Hearing and Vision Decline with Age
- 30-60 of people aged 65 yrs and older have some
degree of hearing loss. - 10-30 have vision impairment, even with
corrective lenses.
3Auditory Speech Recognition Declines with Age
- Age-related differences in speech perception are
observed even when younger and older adults are
matched for hearing ability.
4Vision-only and Auditory-Visual Speech
Recognition Also Declines with Age
- Results from our 2000-2005 experiments, consonant
and sentence stimuli - 50 older adults, 50 younger adults, all have
normal hearing (similar findings for older adults
who have hearing loss) - (Note in particular poor performance for
sentences)
5Auditory Enhancement
- AV V-only/ 1 V
- (A performance is controlled)
6Results for A enhancement (No sig. differences
cons p.200, sent p .242)
7Integration Enhancement
- Based on probability formula
- Designed to account for the enhancement, or
benefit, associated with integrating two channels
of information - Indexes AV integration ability by first removing
the amount of AV performance that could be
accounted for by unimodal scores (A and V-Only),
then normalizing for the amount of improvement
that could have been achieved.
8Integration Enhancement
B
A
- A Amount AV improved beyond the prediction
1-(1-A)(1-V) - B Amount that cold have been improved (AV -
1-(1-A)(1-V))
9Integration Enhancement (No sig. differences cons
p.062, sent p .429
10Goals of the Present Investigation
- 1. To predict older persons abilities to
lipread in favorable and unfavorable conditions
(i.e., vision-only). - 2. To examine older persons abilities to
comprehend discourse under both favorable and
unfavorable conditions in an audiovisual
condition to compare their performance to young
adults and to identify predictors of
performance. - Favorable Good sound, good visual clarity (as
when one has good hearing and good vision) - Unfavorable Degraded sound, degraded visual
clarity (as when one has hearing and vision
impairment)
11Predicting Lipreading Performance in Older Adults
- In favorable and unfavorable conditions
12Previous Research with Adults
- Variables that appear not to be predictive
verbal intelligence, educational achievement,
acquired hearing loss, personality, visual memory - Variables that may be predictive working
memory, processing speed, verbal inference-making
(e.g., how well a person can complete sentences
that have missing words) (Andersson, Lyxell,
Rönnberg, Spens, 2001)
13Methods
- 93 Older adults (age appropriate hearing loss)
- Avg 74.7 yrs. (SD 5.7)
- Avg PTA Better Ear 23.1 dB (SD11.0)
- 38 Young adults (normal hearing)
- Avg 22.7 yrs. (SD 2.2)
- Avg PTA Better Ear 3.2 dB (SD6.1)
- Build-A-Sentence (BAS) Test
14BAS
15Attractive Features of the BAS
- Avoid floor effects associated with most sentence
tests of lipreading - Ensure that the same stimuli are used across
conditions e.g., A,V, AV Favorable and
Unfavorable
16BAS
17BAS
18Scoring
- Frames correct
- Words in any position correct
- Words in position correct
19Predictor Variables (Older Adults Only)
- a. Pure-tone average in better ear
- b. Working memory letter number span odd-even
digit span - c. Processing speed lexical decision speed
rhyme judgment - d. Word knowledge Vocabulary subset of Weschler
Adult Intelligence Scale (WAIS-V) e.g., Define
the word colony and the definition subtest of
the Kaufman Adolescent and Adult Intelligence
Test (KAIT) e.g., BR_W_ - e. Age at time of testing
20Working Memory
- Letter number span score entails presenting a
series of interspersed letters and numbers e.g.,
D-B-3-F-5-9-W-2. The participants task is to
repeat back the letters in alphabetic order and
then the numbers in ascending order. - Odd-even digit span score entails presenting a
string of digits to the participant, e.g.,
3-7-2-5-8-4. After each digit, the participant
must declare whether it is odd or even. After
the string is presented, the participant repeats
back as many digits as possible in the order of
presentation. Long and short strings are
randomly presented. The score is the longest
string that the participant correctly repeats.
21Processing Speed
- Lexical decision time is the average amount of
time, in milliseconds, it takes a participant to
determine if a written word is a real English
word or not, e.g., blug - Rhyme judgment time is the average amount of
time, in milliseconds, it takes a participant to
determine if two written English words rhyme.
The two words are always orthographically
different e.g., blue and flew
22Working Memory andReaction Time Results, Older
Adults
Reaction Time
Working Memory
23Performance for V-only BAS in good and poor
(i.e., favorable and unfavorable)
80
70
60
50
Older
Avg Percent Correct
40
Young
30
20
10
0
BAS Easy V
BAS Hard V
24Results of Hierarchical Regression Analyses
(Older Adults)
- Favorable
- 16.9 of the variance was accounted for by age
an additional 6.7 was accounted for by working
memory. - Unfavorable
- 10.1 of the variance was accounted for by age
no other variables were predictive
25Why Study Discourse Comprehension?
- Daily encounters with verbal information are both
visual (i.e., reading text) and auditory or
auditory-visual (spoken language comprehension).
Research has established that as we age, we
become less efficient readers. Very little is
known about how listening comprehension is
affected by aging. Declines in both hearing and
vision are only suggestive. - Most face-to-face communication entails
understanding discourse e.g., instructions,
directions, news, sermons, lectures, so it is
ecologically valid to do so.
26Measuring Discourse Comprehension in Favorable
and Unfavorable Conditions
- Auditory signal Conditions are set to either
favorable or unfavorable. Six-talker babble
is added and the signal/noise ratio is varied to
establish good performance (s/n 5) or poor
(s/n ratio -5). - Visual signal Contrast is set to either
favorable (high) or unfavorable (low).
27LISN Discourse Comprehension Test
- Stimulus types lectures, interviews, narratives
(LISN Lectures, Interviews, Spoken Narratives) - Question types information, inference,
integration - Four forms each form includes two of each
stimulus type for a total of 6 passages each
28Scoring
- Total percent correct
- Question type percent correct
- Passage type percent correct
29LISN Development
- Selection of passages (real transcripts from such
sources as NPR). Common knowledge? Gender
specific? Age specific? - Recording of passages by professional actors
- Assessment of questions, e.g., can they be
answered without hearing passage? Can they be
answered after hearing the passages? Are
questions rated as being informative,
integrative, or interferential? - List equivalency and test-retest reliability
- Floor and ceiling effects
- Performance on LISN versus performance on other
tests such as reading comprehension (e.g., WAIS,
Woodcock Johnson) and other measures of listening
comprehension, such as recorded SAT passages and
Nelson Denney
30Sample, good-good Eisenhower (5 s/n, 0
contrast degradation )
31Sample Questions
- 1. What would the speaker probably
say about his placement in the military? - A) It was very mundane work.
- B) It was an entertaining position to take.
- C) It was constantly emotional.
- D) It was hard work.
- 2. Who was C. Merriman Smith?
- A) Head of Secret Service.
- B) Colonel of the U.S. Air Force.
- C) President of the White House press corps.
- D) A dignitary who was there to meet president
Nixon. - 3. What is the speakers claim to fame?
- A) He actually met Richard Nixon.
- B) He ordered C. Merriman Smith to move his car.
- C) He was part of the chain of command that
informed Nixon of his unzipped fly. - D) He was inside the cordon when Nixon was
greeting the dignitaries. -
32Answers and what kind of question each one is
- 1-Inference
- B) It was a very entertaining position to
take. - 2-Information
- C) President of the White House press corps.
- 3-Integration
- C) He was part of the chain of command that
informed Nixon of his unzipped fly.
33Sample, poor-poor Rutgers University (-5
s/n ratio 98 contrast degradation)
34Discourse Comprehension
- Comparison between young and old adults
- Predicting performance of older adults
35Participants
- Older Participants
- 54 persons with age appropriate hearing,
pure-tone averages (M 27.8, SD10.8). Ranged in
age between 66 and 88 yrs (M 74.9, SD 6.0). - Younger Participants
- 38 persons with clinically normal hearing.
Ranged in age between 18 and 27 yrs (M22.7). PTA
lt 25 dB (M 3.2).
36Results for the LISN Older Adults
RM ANOVA sig dif for Age (plt.001), sig dif for
Condition (plt.001), no interaction
37Predictor Variables
- a. Pure-tone-average
- b. Vision-only speech recognition BAS
(favorable) - c. Integration enhancement BAS (favorable for
the easy, then unfavorable for the hard) - d. Working memory (letter-number span odd-even
digit span) - e. Processing speed (rhyme judgment test word
decision test) - f. Word knowledge
- h. Age at time of testing
38Results of Hierarchical Regression Analyses,
Older Persons Only
- Favorable
- 26.1 of the variance was accounted for by the
word knowledge score no other variables were
predictive. - Unfavorable
- 13.4 of the variance was accounted for by the
PTA 6.4 additional variance was accounted for
by lipreading ability no other variables were
predictive.
39Conclusion
- When viewing conditions are favorable, age and
working memory comes into play as a factor that
influences lipreading performance in older
adults. - When viewing conditions are unfavorable, age is
the only predictive variable. - When viewing and listening conditions are
favorable, vocabulary knowledge comes into play
during discourse comprehension. - When viewing and listening conditions are
unfavorable, listening and lipreading abilities
come into play.
40Interpretation
- When viewing (and listening) conditions are
unfavorable, more attention must be directed at
simply understanding the words in the sentences
or passages and working memory and/or vocabulary
knowledge has less impact.
41Project Outcomes
- Better understanding of how age affects
speechreading performance and ones ability to
integrate what is seen with what is heard. - Clinical tools for assessing integration ability
(BAS) and discourse comprehension (LISN). - Better understanding of why some older persons
perform better on comprehension tasks than do
others.
42Acknowledgements
- NIH NIA RO1 and NIH NIDCD RO1
- Collaborators Mitch Sommers, Brent Spehar,
Sandra Hale, and Joel Myerson