As slippery as an eel? Assessing speaking and writing Part One - PowerPoint PPT Presentation

Loading...

PPT – As slippery as an eel? Assessing speaking and writing Part One PowerPoint presentation | free to download - id: 4b7b17-YWMyM



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

As slippery as an eel? Assessing speaking and writing Part One

Description:

As slippery as an eel? Assessing speaking and writing Part One lle T rk University of Tartu Estonian Defence Forces 23rd CSW, Tampere, 27-29 March 2009 – PowerPoint PPT presentation

Number of Views:90
Avg rating:3.0/5.0
Slides: 31
Provided by: communicat89
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: As slippery as an eel? Assessing speaking and writing Part One


1
As slippery as an eel? Assessing speaking and
writing Part One
  • Ülle Türk
  • University of Tartu
  • Estonian Defence Forces
  • 23rd CSW, Tampere, 27-29 March 2009

2
Testing writing?
  • Fill in the gaps with suitable words so that the
    text is true for you.
  • Re-write the text in the future tense.
  • Fill in the form with the information given in
    the box.
  • Read the letter and write an answer.
  • Write an essay on the topic Why study English?
  • Study the pictures, put them in the order you
    think best and write the story.
  • Read the text and write a short summary of it.
  • You bought a new dictionary yesterday, but found
    later that several pages were missing. Write a
    letter to the manager of the shop informing him
    of the problem and telling him what you want him
    to do about it.
  • Read the basic facts about Australian history and
    then write a short report.

3
Questions
  • What is it exactly that we assess when we say we
    assess students speaking and writing skills?
  • How do we arrive at a common understanding of
    what is good writing, what is a good oral
    presentation or what constitutes good spoken or
    written communication?

4
Terms
  • Assessment
  • Formal ? informal
  • Continuous ? fixed-point
  • Formative ? summative
  • Testing
  • Achievement
  • Proficiency
  • Diagnostic
  • Placement
  • High-stakes ? low-stakes

5
Assessment/ test quality
  • Validity
  • Reliability
  • Authenticity
  • Washback
  • Practicality

6
Validity definitions
  • A good test needs to be valid. It must test
    what it is meant to test.
  • an integrated evaluative judgement of the degree
    to which empirical evidence and theoretical
    rationales support the adequacy and
    appropriateness of inferences and actions based
    on test scores.
  • S. A. Messick (Validity in R. L. Linn (ed.)
    Educational Measurement. 1989, p. 13)

7
Validity
  • Does the test match the curriculum, or its
    specifications?
  • Is the test based adequately on a relevant and
    acceptable theory?
  • Does the test yield results similar to those from
    a test known to be valid for the same audience
    and purpose?
  • Does the test predict a learners future
    achievements?

8
Validity
  • Content validity
  • Construct validity
  • Criterion-related validity
  • Predictive validity
  • Construct validity is indeed the unifying concept
    that integrates criterion and content
    considerations into a common framework for
    testing rational hypotheses about theoretically
    relevant relationships.
  • Messick, S. A. Test validity and the ethics of
    assessment. American Psychologist 35, 1980, p.
    1015

9
Threats to test validity
  • construct irrelevant variance
  • construct under-representation

10
Factors affecting validity
  • Lack of specifications
  • Lack of training of item/ test writers
  • Lack of / unclear criteria for marking
  • Lack of piloting/ pre-testing
  • Lack of detailed analysis of items/ tasks
  • Lack of feedback to candidates and teachers

11
Communicative competence 1
  • Canale Swain (1980), Bachman (1990) language
    knowledge types
  • Linguistic knowledge
  • Discourse knowledge
  • Sociolinguistic knowledge

12
Grabe Kaplan (1996) Model of Writing
  • Components of language knowledge relevant to
    writing
  • linguistic knowledge written code, morphology,
    vocabulary, syntax
  • discourse knowledge cohesion, structure, genre
  • sociolinguistic knowledge functional uses of
    writing, register, situational parameters
  • Influential in teaching and testing of writing
    (e.g., Weigle, 2002)

13
Communicative competence 2
  • Bachman Palmer (1996) communicative language
    ability
  • Language knowledge
  • Strategic competence

14
Douglas (2000) Specific Purpose Language Ability
  • Language knowledge
  • grammatical knowledge
  • textual knowledge
  • rhetorical organization
  • cohesion
  • functional knowledge
  • sociolinguistic knowledge
  • Strategic competence
  • assessment
  • goal setting
  • planning
  • control of execution
  • Background knowledge
  • discourse domain

15
Common European Framework of Reference for
Languages Learning, teaching, assessment (2001)
  • General competences
  • Communicative language competences

16
General competences
Declarative knowledge Skills and know-how Existential competence Ability to learn
knowledge of the world practical skills (social, living, vocational/ professional, leisure skills) attitudes language and communicative awareness
knowledge of the world practical skills (social, living, vocational/ professional, leisure skills) motivation language and communicative awareness
sociocultural knowledge practical skills (social, living, vocational/ professional, leisure skills) values language and communicative awareness
sociocultural knowledge practical skills (social, living, vocational/ professional, leisure skills) beliefs cognitive styles general phonetic awareness and skills
intercultural awareness practical skills (social, living, vocational/ professional, leisure skills) beliefs cognitive styles general phonetic awareness and skills
intercultural awareness intercultural skills personality factors study skills
intercultural awareness intercultural skills personality factors heuristic skills
17
Communicative language competences
Linguistic competences Sociolinguistic competences Pragmatic competences
lexical competence linguistic markers of social relations discourse competence
grammatical competence politeness conventions functional competence
semantic competence expressions of folk wisdom
phonological competence register differences
orthographic competence dialect and accent
orthoepic competence
18
Communicative language activities and strategies
  • productive activities and strategies
  • receptive activities and strategies
  • interactive activities and strategies
  • mediating activities and strategies
  • non-verbal communication
  • practical actions
  • paralinguistics
  • paratextual features

19
Oral production
  • public address (information, instructions, etc.)
  • addressing audiences (speeches at public
    meetings, university lectures, sermons,
    entertainment, sports commentaries, sales
    presentations, etc.)
  • reading a written text aloud
  • speaking from notes, or from a written text, or
    from visual aids
  • acting out a rehearsed role
  • speaking spontaneously
  • singing

20
Spoken interaction
  • transactions
  • casual conversation
  • informal discussion
  • formal discussion,
  • debate
  • interview,
  • negotiation
  • co-planning
  • practical goal-oriented co-operation.

21
Oral mediation
  • simultaneous interpretation (conferences,
    meetings, formal speeches, etc.)
  • consecutive interpretation (speeches of welcome,
    guided tours, etc.)
  • informal interpretation
  • of foreign visitors in own country
  • of native speakers when abroad
  • in social and transactional situations for
    friends
  • family, clients, foreign guests, etc.
  • of signs, menus, notices, etc.

22
CEFR levels
  • The Common European Framework of Reference
    (Council of Europe 2001) defines communicative
    proficiency
  • At six levels, arranged in three bands
  • A1 A2
  • B1 B2
  • C1 C2
  • in relation to six skills listening, reading,
    spoken interaction, spoken production, written
    interaction, written production
  • in the form of can do statements

23
Getting to know the levels
  • The self-assessment grid is not enough
  • More specific scales
  • CEFR Ch 4 descriptors of communicative
    activities
  • CEFR Ch 5 descriptors of linguistic competence
  • The ELP (European Language Portfolio)
  • Manual Relating Language Examinations to the
    Common European Framework of Reference for
    Languages Learning, Teaching, Assessment (CEFR)

24
I can deal with most situations likely to arise
whilst travelling in an area where the language
is spoken. I can enter unprepared into
conversation on topics that are familiar, of
personal interest or pertinent to everyday life
(e.g. family, hobbies, work, travel and current
events).
Self-assessment grid (CEFR and standard adult
passport)
25
CercleS ELP goal-setting and self-assessment
checklists
26
Questions to ask
  • What competences should my students have in
  • Spoken interaction
  • Spoken production
  • Written interaction
  • Written production
  • What tasks should they be able to perform to
    demonstrate their mastery of the competences?
  • How well should they be able to perform them?

27
Reliability
  • A test needs to be reliable. It must produce
    consistent results at different times.
  • NB! A test that is not reliable cannot, by
    definition, be valid.

28
Reliability
  • If I take the test again tomorrow, will I get the
    same result?
  • If I take a different version of the test, will I
    get the same result?
  • If the test had had different items, would I have
    got the same result?
  • Do all markers agree on the mark I got?
  • If the same marker marks my test paper again
    tomorrow, will I get the same result?

29
Factors affecting reliability
  • Poor administration conditions noise, lighting,
    cheating
  • Lack of information beforehand
  • Lack of specifications
  • Lack of marker training
  • Lack of standardisation
  • Lack of monitoring

30
References
  • Bachman, Lyle F. (1990) Fundamental
    Considerations in Language Testing, Oxford
    University Press, Oxford.
  • Bachman, Lyle F. and Palmer, Adrian (1996)
    Language Testing in Practice, Oxford University
    Press, Oxford.
  • Cushing Weigle, Sara (2002) Assessing Writing,
    Cambridge University Press, Cambridge.
  • Douglas, Dan (2000) Assessing Languages for
    Specific Purposes, Cambridge University Press,
    Cambridge.
About PowerShow.com