Computermediated Oral Assessment and Its Implications for Teaching Speaking to Thai Students

1
Computer-mediated Oral Assessment and Its
Implications for Teaching Speaking to Thai
Students
Kanchana Prapphal and Prakaikaew
Opanon-amata Chulalongkorn University Language
Institute
2
The Computerized Oral ProficiencyTest (COPI)

• Developed by the Center for Applied Linguistics

• OPI (Oral Proficiency Interview)?
• SOPI (Simulated Oral Proficiency Interview,
  tape-mediated)?
• COPI (Computerized Oral Proficiency Test)

3
Characteristics of COPI

• Non-linear
• Different tasks
• Computer-mediated
• Easier feedback for examinees
• Electronic storage of examinees performances
• Electronic storage of rating data
• (Malabonga et al., 2000)
  
  

4
Advantages of COPI

• Time saving
• Automatic computation of score
• Report generation
• Data-saving in electronic form
• Efficient
• Easy-to-use format
• Flexible for raters

5
Chula Context

• Chula students should be able to
• communicate in English
• use the computer for their academic work as
  well as for their future career

6
Objectives of the Study
1. Assess the students speaking ability
by using the computer 2. Investigate the
students communication strategies
7
Research Questions
1. Can computer-mediated oral assessment (CU
COA) assess students oral proficiency?
2. What communication strategies do the students
use in taking the CU-COA test?
8
Subjects

• 30 first year students (16 male and 14 female)
  from the Faculties of Science, Education and
  Political Science
• They studied English for 8 years.
• Their equated TOEFL scores ranged from 370-640.
  

9
Instrument

• The Chulalongkorn University Computer-mediated
  Oral Assessment (CU-COA)
• There are 4 sections
• - Introduction
• - Section 1 (Give personal information.)
• - Section 2 (Give information about familiar
  topics.)
• - Section 3 (Describe future plans.)
• Rating scale 6 levels

10
Rating Scale

• Level 6 Excellent speaker of English
• Level 5 Very good speaker of English
• Level 4 Good speaker of English
• Level 3 Fair speaker of English
• Level 2 Poor speaker of English
• Level 1 Very poor speaker of English

11
Descriptors of the Scale

• Example
• Level 6
• -employs native-like prosody
• -uses complex grammatical structures
• - rarely makes a mistake
• - links ideas coherently

12
Data Collection

• October-November 2005

• 7 raters (28-59 years old)

• Rater training before the test administration

13
Data Analysis

• Correlation coefficients
• among raters
• They range from .514 to .884.
• All correlation coefficients are significant at
  the .01 level.
• One rater is the least reliable. (lacking the
  computer skills)

14
Results

• Grand mean 3.198
• SD .918
• Correlation with CU-TEP .879

15
Conclusion
Research Question 1 Can CU-COA assess
students oral proficiency? The CU-COA can
assess their oral ability and can classify them
into different levels according to their
proficiency levels.
16
Research Question 2
What communication strategies do the students use
in taking the CU-COA test ?
17
Communication Strategies

• Nakatani(2005), Dornyei Scott(1997) and
  Bialystok (1990)
• Achievement or compensatory strategies
• Reduction or avoidance strategies

18
Achievement Strategies
Good learner behaviors Use various ways to
reach the original goal by means of whatever
resources are available
19
Avoidance Strategies
Low-proficiency learners behaviors Avoid
solving a communication problem and give up on
conveying their message
20
Analysis of Speech Production

• Four recorded oral tests selected

• Six communication
• strategies (Nakatani 2006)

21
1. Social Affective Strategies

• Try to control their own anxiety
• Try to enjoy the process of oral communication
• Are willing to encourage themselves to use
  English
• Risk making mistakes

22
1. Social Affective Strategies

23
1. Social Affective Strategies
Utterances of the level 5 student I think I
have a lot of friends because Im a friendly girl
and Im very lively.But I have one best friend.
Her name is Prae. We knew each other from 11th
grade from high school. She was very nice and
very sincere and we still keep in touch now
although shes in another college. But she will
remain my best friend.
24
1. Social Affective Strategies
Utterances of the level 2 student
..Yes, I have a lot of friends..(A long
pause of 40 seconds)... Korawit is my best
friend(Silence until times up)
25
2. Message Abandonment Strategies

• Give up their attempt to communicate
• Leave the message unfinished

26
2. Message Abandonment Strategies

27
3. Fluency-oriented Strategies

• Pay attention to the pronunciation
• Pay attention to clarity of their speech
• Try to make the speech flow

28
3. Fluency-oriented Strategies
29
4. Accuracy-oriented Strategies

• Desire to speak English accurately
• Pay attention to forms
• Seek grammatical accuracy by self-correcting

30
4. Accuracy-oriented Strategies

31
4. Accuracy-oriented Strategies
Utterances of the level 5 student Well, now ,
Im studying to be a computer programmer at the
Faculty of Science in the field of computer
science at Chulalongkorn University. So,
definitely, I want to be a programmer in the
future. But after I graduate from Chulalongkorn
University, I might probably go for a masters
degree, I suppose.(Continuing with correct
English.And then I would think about
.studying moremaybe at MIT. Thats where I
want to go although my Dad graduated from
Harvard. Maybe he wants me to go to Harvard as
well. That willThat I will see in the next four
yearsYes, but but my main future plan is to
graduate and be a professional programmer. Yes.
32
4. Accuracy-oriented Strategies
Utterance of the level 2 student (A pause of
about 8 seconds)I want to teacherI want much
money and big car.I wanta big house.(A very
long pause of 1 minute and 40 seconds).I want
to marriage.
33
5. Message Reduction and Alteration Strategies

• Reduce an original message
• Simplify their utterances
• Tend to use familiar words
• Avoid taking the risk of using new or unfamiliar
  words

34
5. Message Reduction and Alteration Strategies

35
5. Message Reduction and Alteration Strategies
Utterances of the level 5 student OK, about
newspaper. My family will receiveah..Thai Rat,
I think.But I dont really usually read them
because Im more in the Internet world. I read
news from the Internet because they are more
updated. You know up to date, sorry
36
5. Message Reduction and Alteration Strategies
Utterances of the level 2 student Manager
(pronounced with completely Thai accent)
newspaper.(Silence until the time is up.)
37
6. Attempts to think in English

• Students of high proficiency
• Can produce a natural flow of speech
• Tend to think in English
• Students of low proficiency
• Tend to think in their native language
• Construct the English sentences

38
6. Attempts to think in English
39
6. Attempts to think in English
Utterances of the level 5 student Well,
basically we watch the TV together..and we
watch..you know..after dinner. First we have
dinner together as a family and then e go and
watch TV together and each of us have our
favorite TV show and every night we get to pick
the show we want to see and the rest of the
family will watch the show with you and
sometimes we go out of the city..and that just
about it.
40
6. Attempts to think in English
Utterances of the level 2 student I watch
television with my parent ..(A long pause of 30
seconds)...I make.useful..things with my
parent.
41
Implications on Teaching Speaking
Goals of Teaching Speaking All students should
be able to talk accurately and fluently about
themselves, express feelings or opinions about
things in their daily lives.
42
Implications on Teaching Speaking
Speaking Test Teaching device Personal
Interview Role play
43
Personal Interview
Conducted constantly Willingness to
speak Fluency Accuracy
(Myint 1993)
44
Role Play
Expose students to conversational
English Encourage students to participate
physically Dramatize a specific situation Real
world of conversational English
(Bogart 2006)
45
Conclusion

• The CU-COA can
• Assess Thai students speaking ability
• Differentiate the students into various levels of
  speaking ability
• Illustrate different communication strategies