TASK-BASED ASSESSMENT IN LANGUAGE LEARNING PROGRAMS: PIECES OF THE PUZZLE

1
TASK-BASED ASSESSMENT IN LANGUAGE LEARNING
PROGRAMS PIECES OF THE PUZZLE

• Geoff Brindley
• Macquarie University,
• Sydney, Australia

2
Structure of the presentation

• 1. Introduction
• 2. Task-based assessment Definitions and
• features
• 3. Implementing TBA Differing stakeholder
• perspectives
• -Theoretical perspectives
• -Educational versus managerial perspectives
• 4. The need for assessment literacy
• 5. Conclusion

3
Defining task-based assessment

• the process of evaluating, in relation to a
  set of explicitly stated criteria, the quality of
  the communicative performances elicited from
  learners as part of goal-directed,
  meaning-focused language use requiring the
  integration of skills and knowledge (Brindley,
  1994)
• UK
• -Teacher assessment to be used for statutory
  reporting at 11 and

4
Positive features of TBA

• Teachers' and learners' attention is able to be
  more focused on language as a tool for real world
  communication rather than on language knowledge
  as an end in itself, thus potentially improving
  student motivation.
• Assessment can be integrated into the learning
  process through the use of attainment targets
  which are directly linked to course content and
  objectives.

5
Positive features of TBA (contd)

• Learners can obtain useful diagnostic feedback on
  their progress and achievement since explicit
  performance criteria are provided against which
  they can compare their performances.
• Better communication between users of assessment
  information and educational institutions can be
  established through the use of various forms of
  outcome reporting which are couched in
  performance terms and are hence intelligible to
  non-specialists.

6
The shift to teacher-led assessment

• UK
• -Teacher assessment given greater importance
• following review of National Curriculum
  Assessment
• -Teacher assessment used for reporting from
• 2007 in Wales
• -Government Excellence and Enjoyment report
  (DfES
• 2003) endorses greater role for teachers in
  assessment
• -Tomlinson Report (2002) endorses use of teacher
• assessment for national reporting

7
The shift to teacher-led assessment (contd)

• USA
• -Widespread adoption of authentic TBA
• in schools adult education
• -Use of authentic teacher-conducted
• assessments in some high stakes contexts (eg
• Kentucky, Nebraska)

8
Differing theoretical perspectives on TBA What
is the construct?

• Weak versus strong view of TBA
• in the strong sense, tasks will represent
  real-world tasks, and performance will be judged
  on real-world criteria, that is, the fulfilment
  of the task set (McNamara 1996)

9
Differing theoretical approaches to TBA (contd)

• In second language performance tests in the weak
  sense, the focus is on language performance.The
  candidate is required to perform on a task which
  may represent tasks he or she may subsequently
  face in the real world however, the capacity to
  perform the task is not the actual focus of the
  assessment

10
The problem of generalizability

• While the weak view is likely to assess
  underlying language skills in ways which are
  relatively broadly generalizable, the strong
  view is likely to produce judgments which are
  more authentic and relevant to the real life
  situations towards which candidates may be
  moving. These judgments about the quality of
  performance may not, however, be replicable in
  other contexts (Wigglesworth 2008).

11
Some other unresolved theoretical issues in TBA

• The difficulty of difficulty
• How can task difficulty be defined measured?
• What are the factors affecting task difficulty?
• Can task difficulty be modelled and predicted?
• (Bachman, 2002, 2007 Brindley Slatyer, 2002
  Brown
• et al, 2002 Elder et al 2002 Ellis, 2008
  Robinson,
• 2001 Skehan, 1998, 2001 Tavakoli, 2009)

12
Educational versus managerial perspectives

• Whereas politicians and government officials
  tend to see assessment as a tool for implementing
  and managing policy, teachers and educationists
  are primarily concerned with ways in which it can
  be used for the improvement of learning
  (Brindley, 2008)

13
Formative TBA The educational policy maker
perspective

• Effective assessment for all pupils should
• -recognise what pupils can do and reward
  achievement
• -be based on different kinds of evidence
• -be a valid reflection of what has been taught or
• covered in class
• -be reliable in terms of enabling someone else to
  repeat
• the assessment and obtain comparable results
• -be manageable, both in terms of the time needed
  to
• complete the task, and in providing results which
  can
• be reported or passed on to other teachers (DfES,
• 20032)

14
The formative purpose of TBA rhetoric or reality?

• No reference to
• monitoring learners emerging language awareness
  and development achievement is highlighted
• assessment as integrated within instructional
  discourse. Taught or covered suggests one-off
  measurement focused assessment
• formative assessment as an on-going process Leung
  and Rea-Dickins, 2007)

15
Popular beliefs about testing assessment

• There is a test for every population/purpose
• Norm-referenced tests are a fair and objective
  measure of student ability
• Standardized testing raises standards
• All tests have pass marks (usually 50)
• Teachers cant be trusted to do their own
  assessment

16
The media critics

• Formative assessment also embraces a
  developmental approach to learning, based on the
  argument that "students develop and learn at
  different rates and in different ways"
• The result? Instead of pass or fail, student
  progress or lack of progress is clouded by such
  politically correct terms as beginning,
  established, consolidating or emerging, solid,
  comprehensive.
• Instead of students facing regular examinations
  with consequences for failure, as do those
  students in stronger performing education systems
  overseas, students are automatically promoted
  from year to year, even though many have not
  mastered the basics (Donnelly 2005)

17
The politicians

• The reports I saw allowed for the teacher to
  assess students from a range of choices
  usually, consolidating, sometimes, and not yet.
  What kind of nonsense is this?
• The educational experts with whom I seem to be
  in constant battle, give me the constant refrain
  of outcomes assessment. The ranking of students
  against one another is opposed by teacher
  advocates. Try telling that to parents. Worse
  still, what do they think happens in the real
  world?
• (Nelson 2005)

18

• Politicians are simple people they like simple
  people they like simple choices and clear
  guidance.
• (Yes Minister)

19
The role of assessment literacy

• ..training for assessment literacy entails an
  appropriate balance of technical know-how,
  practical skills, theoretical knowledge, and
  understanding of principles but all firmly
  contextualized within a sound understanding of
  the role and function of assessment within
  education and society (Taylor, 2009)

20
Assessment literacy for all

• an appropriate level of assessment literacy
  needs to be nurtured not just among engineers and
  technicians who are actively involved in test
  development or research activities, or even among
  applied linguists and language teachersbut more
  broadly in the public domain if a better
  understanding of the function and values of
  assessment tools and their outcomes is to be
  realized throughout society (Taylor, 2009)

21
Assessment and the language teacher What skills
knowledge do they need?

• Teachers should be skilled in choosing assessment
  methods appropriate for instructional decisions.
• Teachers should be skilled in administering,
  scoring and interpreting the results of both
  externally-produced and teacher-produced
  assessment methods

22
Assessment and the language teacher What skills
knowledge do they need?

• Teachers should be skilled in developing, using
  and evaluating valid student grading procedures
  which use student assessments.
• Teachers should be skilled in communicating
  assessment results to students, educational
  decision makers and other concerned stakeholders.

23
Assessment and the language teacher What skills
knowledge do they need?

• Teachers should be skilled in using assessment
  results when making decisions about individual
  students, planning teaching, developing
  curriculum, and institutional improvement.
• Teachers should be skilled in recognizing
  unethical, illegal, and otherwise inappropriate
  assessment methods and uses of assessment
  information

24
Components of assessment literacy (Bailey
Brown, Brindley, 2001, Inbar-Lourie, 2008,
Taylor, 2009)

• The social and political context of assessment
• Defining and describing language proficiency
• Constructing and evaluating tests and assessment
  tasks
• Assessment in the curriculum
• Putting assessment into practice

25
Professional development on assessment some
guiding principles (Brindley 2001)

• Involve the whole system
• Capitalize on existing practices
• Recognise and address the reality and constraints
  affecting assessment
• Encourage a research orientation
• Plan for change

26
Teacher involvement in collaborative
test/assessment task development

• the discussion time and the time to sit down
  with people and discuss something has been very
  valuable. And then Im discussing with other
  people who are also interested in these things.
  And they ask me about tasks sometimes, they bring
  me a task and say What do you think about this?

27
Teacher involvement in collaborative
test/assessment task development

• After doing this project, I realized that
  setting a test paper is not an easy task. Instead
  of testing what is easy to test, we have to
  construct tests that we really discover how
  successful the learning experiences had been for
  the students rather than to show in what respects
  they had been deficient (Teacher participant
  cited in Coniam 2009).

28
And in conclusion..

• We believe that a new breed of assessment
  literate educators with extensive experience in
  classrooms will play an important role in
  implementing assessment policies that truly
  support student learning.
• (Lukin et al 2004)