Catia Cucchiarini - PowerPoint PPT Presentation

About This Presentation
Title:

Catia Cucchiarini

Description:

Quantitative assessment of second language learners' fluency in read. and spontaneous speech ... Restarts: repetitions initial parts of words. Results: ... – PowerPoint PPT presentation

Number of Views:101
Avg rating:3.0/5.0
Slides: 31
Provided by: ccucch
Category:

less

Transcript and Presenter's Notes

Title: Catia Cucchiarini


1
Catia Cucchiarini
  • Quantitative assessment of second language
    learners fluency in read
  • and spontaneous speech
  • Radboud University Nijmegen

2
Context
  • Research on automatic assessment of oral
    proficiency
  • in Dutch as a second language

3
Fluency
  • Important construct in evaluation of second
    language proficiency
  • Also relevant for pathological speech

4
Fluency
  • Frequently applied notion, but not clearly
    defined
  • Various interpretations
  • Overall language proficiency
  • Oral command of a language
  • Temporal aspect of oral proficiency

5
Two experiments
  • Exp1 read speech
  • Exp2 spontaneous speech
  • Human fluency judgements
  • related to
  • Objective temporal measures (CSR)

6
Aim of these experiments
  • To explore the relationship between objective
    properties of speech and perceived fluency in
    read and spontaneous speech, with a view to
    determining whether such quantitative measures
    can be used to develop objective fluency tests.

7
Method speakers
  • Exp1 60 non-native speakers
  • 3 proficiency levels
  • beginner (PL1)
  • intermediate (PL2)
  • advanced (PL3)
  • different mother tongues
  • different gender

8
Method speakers
  • Exp2 60 non-native speakers
  • 2 proficiency levels
  • beginner level (BL)
  • intermediate level (IL)
  • different mother tongues
  • different gender

9
Method speech material
  • Exp1
  • 2 sets of 5 phonetically rich sentences
  • read over the telephone

10
Method speech material
  • Exp2
  • existing test of Dutch as a second language
    (DSL) è Profieltoets
  • 8 items from BL version
  • short tasks, 15 s to answer
  • candidates can answer immediately
  • 8 items from IL version
  • long tasks, 30 s to answer
  • candidates have to reflect to provide motivations

11
Method raters
  • Exp1
  • 3 phoneticians (PH)
  • 3 speech therapists (ST1)
  • 3 speech therapists (ST2)
  • Exp2
  • 5 DSL teachers for BL (RBL)
  • 5 DSL teachers for IL (RIL)

12
Method automatic scoring
  • Speech orthographically transcribed
  • CSR 38 monophones lexicon
  • Viterbi alignment of speech signals and
    orthographic transcriptions
  • Segmentation at phone level

13
Method some definitions
  • silent pause a stretch of silence of no less
    than 200 ms
  • dur1 duration speech without pauses (s)
  • dur2 duration speech with pauses (s)

14
Method objective measures
  • Primary variables
  • art phones / dur1
  • ros phones / dur2
  • ptr 100 dur1 / dur2
  • mlr mean phones between 2 pauses
  • mlp mean length silent pauses
  • dsp tot. dur. sil. pauses / (dur2 / 60)
  • sp sil. pauses / (dur2 / 60)

15
Method objective measures
  • Secondary variables
  • fp filled pauses / (dur2 / 60)
  • disf disfluencies / (dur2 / 60)

16
Method fluency ratings
  • Sentences scored on fluency on the basis of a
    ten-point scale
  • Raters received no special training

17
Method rating procedure
  • Exp1 each group of raters judged speakers of
    different proficiency levels
  • Exp2 each group of raters judged speakers of the
    same proficiency level

18
Results reliability
19
Results raw fluency ratings
20
Results objective measures
21
Results disfluencies
  • Repetitions exact repetitions of words
  • Repairs corrections
  • Restarts repetitions initial parts of words

22
Results disfluencies
23
Results correlations
24
Results correlations
25
Discussion
  • Reliable fluency scoring is possible
  • Fluency scores related to task performed
  • Role objective variables in rs and ss
  • similarities weak relation sec. var. / fluency
  • differences varying roles prim. var.

26
Discussion
  • Read speech
  • strong relation art, ros, ptr, sp, dsp, mlr
  • weaker relation mlp
  • ? for perceived fluency pause freq. more
    important than pause length
  • ? two factors important fluency rs
  • articulation rate
  • pause frequency

27
Discussion
  • Spontaneous speech
  • strong relation ros, ptr, sp, dsp, mlr
  • weaker relation art, mlp
  • ? possibly higher freq. pauses effaces
    importance art
  • ? fluency in ss particularly related to var.
    that contain info on pause freq.

28
Conclusions
  • Reliable fluency scoring by human raters is
    possible
  • Objective fluency scoring is possible
  • Fluency scores vary with speech type
  • Fluency scores vary with task performed

29
Conclusions
  • Read speech fluency scores strongly related to
    art and pause frequency
  • Spontaneous speech fluency scores strongly
    related to pause frequency and distribution
  • Expert fluency ratings can be predicted more
    accurately on the basis of objective measures in
    rs than in ss

30
Conclusions
  • Temporal measures of fluency may be used to
    develop objective fluency tests
  • Selection of variables to be employed should be
    dependent on material investigated and task
    performed
Write a Comment
User Comments (0)
About PowerShow.com