Enhancing speech perception: timeontask

1
Enhancing speech perception time-on-task

• Arthur Boothroyd
• Presentation to Aural Rehabilitation conference,
  Tampa FL.
• March 2009
• Preparation supported, in part, by NIDRR grant
  H1343E98 to Gallaudet University.

2
Greetings from San Diego
Blue sky (usually)
Much water(always)
3
Four questions

• How do we understand what people say?
• How does altered hearing affect this ability?
• How can we adapt to altered hearing?
• Can formal training help?

4
Question 1

• How do we understand what people say?

5
Understanding what others say
Seems easy and spontaneous
Meaning
Meaning
Hearing
Sound
Speech
6
Not so fast! Its more complicated
Shared environment Physical/Social/Language
Talker
Listener
7
Not so fast! Its more complicated
Shared environment Physical/Social/Language
Talker
Listener
Intent
8
Not so fast! Its more complicated
Shared environment Physical/Social/Language
Talker
Listener
Intent
Hearing
Immediate context Physical/Social/Language
9
Not so fast! Its more complicated
Shared environment Physical/Social/Language
Talker
Listener
Intent
Hearing
Immediate context Physical/Social/Language
10
Not so fast! Its more complicated
Shared environment Physical/Social/Language
Talker
Listener
Intent
Hearing
Immediate context Physical/Social/Language
11
The purpose of language is not always to convey
meaning

• Bob said Hi Jim! Hows it going?.
• That was a greeting followed by an enquiry
  about my current status.
• Bob probably wants to borrow money.

Men use speech to concealtheir thoughts.
(Voltaire)
12
Key factors
Under- standing what people say
13
Key factors
1.Sensory evidence
2.Contextual evidence
Under- standing what people say
3. Listener knowledge
4.Listener skill
14
1.Sensory evidence

• Depends on
• Talker Clarity Distance Visibility
• Room Noise Reverberation
• Listener Hearing Vision Attention Working
  memory

15

• Depends on
• Information in the context World
  (situational) Social (people) Language topical
  semantic sentential lexical phonological
• Listener knowledge

2.Contextual evidence
16

• Is implicit and includes
• World knowledge
• People knowledge
• Language knowledge Sound system Vocabulary Gram
  mar Usage

3. Listener knowledge
17

• Involves
• Sustained attention
• Selective attention
• Sensory/contextual balance
• Optimizing speed while minimizing error
• Adaptability
• Working memory
• Rapid retrieval
• Multi-tasking

Maintaining focus
Ignoring noise and other distractions
Using only what is needed
Keeping up with the talker (2 to 3 words/sec)
More use of sensory evidence for challenging
content
Many decisions to be made after the fact
- of possible decisions from knowledge base
4.Listener skill
e.g. Considering implications Preparing response
The talker sets the speed
18
Q1. How do we understand what people say?

• We make our best guess
• - from both sensory and contextual evidence
• - drawing on our world, social, and language
  knowledge
• - and using a complex set of skills
• - in order to keep up with the talker while
  making as few errors as possible.

19
Question 2

• How does altered hearing affect our ability to
  understand what people say?

20
Primary impact of altered hearing
1.Sensory evidence
2.Contextual evidence
Under- standing what people say
3. Listener knowledge
4.Listener skill
21
Loss of sensory evidence

• Reduced sensitivity
• Reduced clarity (out of focus)
• Reduced range of best hearing
• Increased effect of noise

Mostly solved by hearing aids
Not solved by hearing aids
Partially solved by modern hearing aids
Minimally solved by modern hearing aids
22
Reduced range of best hearing?
23
Aggravating factor in the elderly
1.Sensory evidence
2.Contextual evidence
Under- standing what people say
3. Listener knowledge
4.Listener skill
4.Listener skill
24

• Involves
• Sustained attention
• Selective attention
• Sensory/contextual balance
• Optimizing speed while minimizing error
• Adaptability
• Working memory
• Rapid retrieval
• Multi-tasking

4.Listener skill
Aging research confirms declines in Working
memory and processing speed (Park, D. and
Schwarz, N., 2000. Cognitive aging. Taylor and
Francis. Philadelphia, PA.
25
10 elderly subjects in study of
context dependence and channel-capacity
26
(No Transcript)
27
Reported benefit of FM mic vs. age and aided
performance in noise
28
Q2. How does altered hearing affect our ability
to understand what people say?

• Reduced audibility
• Reduced clarity
• Increased noise susceptibility
• Reduced range of best hearing
• In the elderly- diminished working memory-
  slowed processing.

29
Question 3

• How can we adapt to altered hearing?

30
Optimize sensory evidence Enhance listener skill
2.Contextual evidence
1.Sensory evidence
Under- standing what people say
3. Listener knowledge
4.Listener skill
31
Enhancing sensory evidence?

• Appropriate hearing aid
• Remove or avoid noise
• Minimize distance
• Exploit lipreading
• Optimize visibility (lighting)
• Coach the talker ? visibility, clarity (not
  volume), low speed
• Clarify topic

1.Sensory evidence
32
Enhancing Skills?

• Sustained attention
• Selective attention
• Sensory/contextual balance
• Optimizing speed while minimizing error
• Adaptability
• Working memory
• Rapid retrieval
• Multi-tasking

?
4.Listener skill
33
Enhancing Skills?

• Time-on-task (aka)
• Practice, Practice, Practice
• Use it or lose it

• And a little more talker coaching

4.Listener skill
34
What is the task?
Confirm Repair
35
Q3. How can we adapt to altered hearing?

• Optimize sensory evidence
• Enhance skills through time-on-task
• Exercise control of the listening situation

36
Question 4

• Can formal training speed adaptation to altered
  hearing?

37
Enhancing time-on-task1. Every-day communication
----- Not really a new job. They just changed the
name. I am still doing the same thing. ----- Yes.
But they had to change my title so they could
give me a raise.
Hi Bill! I hear you got a new job
----- ----- ----- The same thing? -----
Topic
Confirmation
38
Enemies of time-on-task

• Discouragement (too difficult)
• Fear of failure (embarrassing)
• Lack of confidence (self doubt)
• Passivity (reluctance to control, confirm, or
  repair)
• Withdrawal (tuning out)
• Avoidance (staying home)
• Talking too much (no need to hear)
• Pretense (smile and nod)

39
Enhancing time-on-task2. Formal training

• Some options
• Individual
• Group
• Computer-assisted
• Significant-other assisted

• Requirements
• Non-threatening
• Provide feedback
• Tailored
• Reassuring
• Fun/interesting

Cost-effective
40
Some computer-assisted options
41
Got Evidence?
42
Rehabilitation of adult cochlear implantees (NY,
early 1980s)
Extra Time-on task
Time-on task
100 80 60 40 20 0
Vowel Consonant training
Sentence Level training
No treatment
No treatment
1
2
3
4
43
(No Transcript)
44
Single adult with hearing loss
Lipreading plus tactile device
Lipreading alone
45
15 Hearing adults using CasperSent lipreading
only
50
Group 1 No feedback
Group 2 Full text
Group 3 Partial text with 2nd and 3rd try
46
50 40 30 20 10 0
words recognized at first presentation
No feedback
5 10 15
20 25 Sentence set

• Group 3 appeared to learn faster but not when
  time-on-task was taken into account

50 40 30 20 10 0
0 20 40 50
70 75 Estimated time-on-task
in minutes
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
(No Transcript)
51
Are data like these worth a hill of beans?
The data are encouraging but not overwhelming
Improved performance
The ultimate criteria
Every day communi-cation
Quality of life
Training task
Other tasks
Carry-over or generalization
52
Q4. Can formal training speed adaptation to
altered hearing?

• A qualified Yes (by increasing time-on-task?)
• Strong evidence of improved performance on
  training tasks
• Some evidence of carry-over to other tasks
• Big need for evidence of carry-over to every day
  communication and quality of life
• Big need for understanding of underlying
  mechanisms

53
Concluding comments

• Hearing loss ? reduced sensory evidence
• Best aid ? partial restoration of sensory
  evidence
• Adaptation requires enhanced skillsAttention,
  speed, context, adaptability, memory, control,
  verification, repair, confidence
• Critical factor? time-on-task
• For some? every day communication enough
• For others? supplementation needed
• - combining counseling, face-to-face practice,
  and, perhaps, computer-assisted practice
• Carry-over from formal training to every-day
  communication ? probable but not yet certain

54

• 
  Bibliography on perception and learning
• Abrams H, Hnath-Chisolm T, Guerreiro S, Ritterman
  S. (1992). The effects of intervention strategy
  on self-perception of hearing handicap. Ear Hear.
  13371-377.
• Boothroyd, A. (1987). CASPER Computer-assisted
  speech perception evaluation and training. Proc
  10th annual Conf Rehab Technol. Washington, D.C.,
  Association for the Advancement of Rehabilitation
  Technology, 734-736.
• Boothroyd, A., Hanin, L., Waltzman, S. (1987).
  Development of speech perception skills in
  cochlear implantees. Proc 10th Annual Conf Rehab
  Technol, Association for the Advancement of
  Rehabilitation Technology, 428-430.
• Boothroyd, A., Nittrouer, S. (1988). Mathematical
  treatment of context effects in phoneme and word
  recognition. J Acoust Soc Amer, 84, 101-114.
• Boothroyd, A. (1993). Speech perception,
  sensorineural hearing loss, and hearing aids. In
  Studebaker G and Hochberg I (Ed.), Acoustical
  factors affecting hearing aid performance. MA
  Allyn and Bacon Boston.
• Boothroyd, A (2003). Hearing Aids and Room
  Acoustics. Hearing J, 56(10), 10-17.
• Boothroyd, A. (2007). Adult aural rehabilitation
  What is it and does it work? Trends Amp, In
  press.
• Chisolm TH, Abrams HB, McArdle R. (2004).
  Short-and long-term outcomes of audiological
  rehabilitation. Ear Hear. 25464-77.
• Hawkins D. (2005). Effectiveness of
  counseling-based adult group aural rehabilitation
  programs a systematic review of the literature.
  J Am Acad Audiol. 16485-93.
• Mackersie, C.L., Boothroyd, A., Minnear, D.
  (2001). Evaluation of the Computer-Assisted
  Speech Perception Test (CASPA). J Amer Acad
  Audiol. 12, 390-396.
• Nittrouer, S., Boothroyd, A. (1990). Context
  effects in phoneme and word recognition by young
  children and older adults. J Acoust Soc Amer.
  87, 2705-715.
• Rubinstein, A., Boothroyd, A. (1987). The effect
  of two approaches to auditory training on speech
  recognition by hearing-impaired adults. J Speech
  Hear Res, 30, 153-160.
• Sweetow RW, Palmer CV. (2005). Efficacy of
  individual auditory training in adults a
  systematic review of the evidence. J Am Acad
  Audiol. 16494-504.
• Sweetow RW, Henderson Sabe J. (2006). The need
  for and development of an adaptive listening and
  communication enhancement (LACE) program. J Am
  Acad Audiol. 17538-58.
• Sweetow RW (guest editor) (2007). Auditory
  training. Semin Hear. 28 (2).

55

• Bibliography
  on age effects
• Boothroyd A (2004). Hearing aid accessories for
  adults the remote FM microphone. Ear and
  Hearing, 25, 22-33.
• Cobb FE, Jacobson GP, Newman CW, Krestschmer LW,
  Donnelly KA (1993). Age-associated degeneration
  of backward masking task performance evidence of
  declining temporal resolution abilities in normal
  listeners. Audiology, 32, 260-271.
• Dubno JR, Horwitz AR, Ahlstrom JB (2003). Benefit
  of modulated maskers for speech recognition by
  younger and older adults with normal hearing.
  JASA, 111, 2897-907.
• Dubno JR, Ahlstrom JB, Horwitz AR (2002).
  Spectral contributions to the benefit from
  spatial separation of speech and noise. JSHR, 45,
  1297-310.
• Dubno JR, Horwitz AR, Ahlstrom JB (2003).
  Recovery from prior stimulation masking of
  speech by interrupted noise for younger and older
  adults with normal hearing. JASA, 113, 2084-2094.
• Gross JH, Poth EA, Peters RW (1994). Masking
  level differences for tones and speech in elderly
  listeners with relatively normal audiograms.
  JSHR, 37, 422-428.
• Park, D. Schwarz, N. (2000). Cognitive aging.
  Taylor and Francis, Philadelphia, PA.
• Pichora-Fuller, M.k. (2003). Cognitive aging and
  auditory information processing. Int J Audiol. 42
  (suppl. 2) 84-91. Strouse A, Ashmead DH, Ohde
  RN, Grantham DW (1998). Temporal processing in
  the aging auditory system. JASA, 104, 2385-2399.
• Snell KB, Mapes FM, Hickman ED,Frisina DR (2002).
  Word recognition in competing babble and the
  effects of age, temporal processing, and absolute
  sensitivity. JASA, 112, 720-726.
• Willot JF (1991). Aging and the auditory system.
  San Diego. Singular Press.
• Wingfield A, Tun Pa (2001). Spoken language
  comprehension in older adults interactions
  between sensory and cognitive change in normal
  aging. Semin Hear, 22, 287-301.

56

• Some links re
  computer-based training programs
• Seeing and Hearing Speech. http//www.seeingspeech
  .com
• LACE. http//www.neurotone.com/
• CAST. http//www.hei.org/news/presskits/ci/ciqanda
  .htm
• CasperSent. http//www.hearingresearch.org/SOS/Abs
  tracts.htm
• http//www.slhs.sdsu.edu/aboothro/CasperSent
• CasperWord (also known as CASPA).
• http//www.slhs.sdsu.edu/aboothro/CASPA50
• CasperCon (also known as FCSpac).
  http//www.hearingresearch.org/Projects/Boothroyd.
  htm
• http//www.slhs.sdsu.edu/aboothro/CasperCon