Feasibility, ChildFriendliness and Positive Washback a tricky trio for testing Young Learners

1
Feasibility, Child-Friendliness and Positive
Washback a trickytrio for testing Young
Learners

• Shelagh Rixon
• Associate Professor, Centre for Applied
  Linguistics
• University of Warwick UK

2
Overview

• Introduction
• What children know about testing
• What children feel about testing
• What we think we know about what children should
  be able to do in a Foreign Language
• The need for testing to be a Child-Friendly
  experience
• Levels and expectations
• Types of testing task
•  

3
Introduction

• Focus on testing more than assessment
• Sources of data insight and general ideas
• Childrens MFL tests
• Childrens MFL assessment
• Assessment and testing of primary-aged children
  in general
• Prevailing language teaching methodologies and
  materials for primary-aged children
• Talking to children (research interviews and
  informal conversation)

4
What children know about testing

• Child You know what that is?
• Me a semi-colon?
• Child Thats Level 5, that is.

5
What children know about testing

• Child (reading aloud) And there were some
  exquisite carvings Whats exquisite?
• Me (explanation and examples)
• Child (with satisfaction) Thatll be a Level 5
  word, then. (writes it in her Word Book)

6
What children know about testing

• I, amusedly, shared this with her teacher. A
  week later the teacher told me
• B just complimented me on my exquisite necklace
  
• A happy ending then? A treasured, mark-bearing,
  word used in real life or a nasty piece of
  mark-grubbing engineering?

7
Why are UK children acting so overt-smart?

• Washback of the National Curriculum tests in
  Literacy ( English) where every child should
  reach Level 4 by the time they take the Year 6
  tests near the end of their primary career.
• Teachers know, parents know, and children know
  (wall displays of the punctuation pyramid,
  lists of powerful verbs etc.) what are the
  external signs of a Level 4 achievement. They
  also know what it takes to go beyond the
  ordinary and get a Level 5.

8

• However, it does not necessarily push the
  pedagogy in the direction of effective teaching
  of the judgements that need to be made before
  e.g. a semi-colon is selected.
• Top tips and token-ism as in semi-colon worship
  are a danger.

9
What children feel about testing

• Children have views and conceptions of
  assessment in general and particularly of formal
  testing. These views seem to be rarely closely
  researched.

10
Research with Italian 9- year olds

• Rixon (Rea-Dickins and Rixon, 1999) Focus group
  interviews in L1 with children who knew me well.
• Topic school-based assessment. How their
  teachers judged their English
• Findings
• only pencil and paper tests counted.
• English was seen as a rather small body of
  knowledge which could be easily covered by
  age 11.

11
Research with UK Year 6 pupils

• In the case of tests, it seems imperative to
  research childrens perspectives. I found very
  little in English.
• an honourable exception
• 'I'll Be a Nothing' Structure, Agency and the
  Construction of Identity through Assessment, by
  Diane Reay and Dylan Wiliam British Educational
  Research Journal 1999

12
Reay and William 1999

• Tracey I think even now, at night times I think
  about it and I think I'm going to get them.
• Diane You think about your SATs at night time?
• Tracey Yeah, lots. When I'm in bed, because I've
  got stars on my ceiling, I'm hoping and I look up
  and I go, 'I know I'm gonna get there'. And my
  mum goes, 'Who's talking in there?' And I goes,
  'Nothing mum'.
• Diane So what are you hoping?
• Tracey Um, I think about a three. I dunno. I
  don't think I'll get a five. I'm hoping to get a
  five. When I look at the stars I hope I'll get a
  five.

13
Talking to Norma

• Diane Norma, why are you worried about SATS now?
  
• Norma Well, it seems like I'll get no points or
  I won't be able to do it, too hard or something.
• Diane What would it mean to get no points?
• Norma Well instead of being level three I'll be
  a nothing and do badly-very badly
• Diane What makes you think that? Have you been
  practising?
• Norma No, like I analyse ... I know I worry
  about loads of things.

14

• Diane Like what?
• Norma I don't know, I just worry about things
  and my mum is going to take me to a special
  aromatherapy lady, or something like that. I
  don't know, but she said something about that
  because I am always panicking and I've been
  worrying about when it's SATs.

15
These are supposed to be non-threatening tests
that help evaluate the school rather than decide
the fate of the child!

• Stuart What if I get level one?
• Diane You won't get level one. Honestly, I'm
  quite positive you won't get level one.
• Stuart I might in English, since Mrs O'Brien
  told us about that boy messing up his chances
  I've been worried about it 'cause it's the sort
  of thing I could do.

16
Most FL testing boards have taken issues like
these very seriously

• E.g. Juliet Wilson Cambridge ESL Research Notes
  Nov 2005
• Ethical issues in the testing of young learners
• there are also those who have fundamental
  reservations and concerns not only about the
  number of tests that children now face but also
  about the very existence of formal English
  language tests for 712 year olds. These may be
  seen as essentially undesirable and even
  detrimental to the learning process.
• whether it is possible to create international
• English language tests for children that
  provide an accurate
• assessment but will also have a positive impact
  on their learning.

17
However

• In spite of best intentions and careful design
  to make tests child-friendly, there may be
  pressures and influences that as with the
  National Curriculum tests serve to render an
  innocuous test a source of anxiety and late night
  ceiling-staring by some children.
• parents, peers, anxious teachers keen to prove
  the worth of their school?

18
 What we think we know about what children
should be able to do in a Foreign Language

• The lack of empirical study of what is feasible
  and appropriate for most children in their own
  language is often criticised (Sealey, 1999).
• Similarly, for Foreign Language learning in
  instructional situations it could be that we are
  in a circular relationship between what is
  customarily taught and what is therefore
  currently seen as a feasible testing goal.

19

• In the Workshops I called this not pushing
  the pedagogy
• Some UK primary school childrens work in French
  and the way the grades it is given are described
  is perhaps interesting to look at
• http//curriculum.qca.org.uk/uploads/MFL-3375_tcm8
  -7144.JPG

20
Kelly and Daniel Mon Monstre

• The teacher asked the pupils to design a monster
  and then to write some simple sentences to
  describe it. The work brought together three
  areas of learning colours, animals, and parts of
  the body, all of which had been practised
  thoroughly through a range of oral work.
• The teacher drew several strange animals as an
  oral stimulus and asked the pupils to describe
  them, recording their suggestions on the board.
  The pupils used this model to design their own
  monsters and write their descriptions. All pupils
  took part in this activity, although some pupils
  had their sentences written for them and only had
  to complete the description with single words.
• Activity Objectives
• To use previously learnt language in an original
  and creative way.
• Commentary
• This activity provided an opportunity for all
  pupils to use language creatively at a level
  suited to them.

21
QCAs commentary on Daniel

• The teacher wrote out the sentences on Daniel's
  piece of work and Daniel had to fill in the
  missing word. This is characteristic of level 1
  in writing.

22
QCAs commentary on Kelly

• Kelly was able to work more independently. She
  has gone beyond the model sentences and has used
  aids to produce short sentences on her own. This
  is characteristic of level 3 in writing.

23
Emma and Shaun talking together

• The commentary reads
• The pupils had conversations in pairs using a
  range of questions and answers that they had
  thoroughly practised in class. The pupils
  performed at different levels, and some were able
  to answer questions but not to ask them.
• Activity Objectives
• To use previously learnt language from memory in
  a conversation.

24
Emma and Shaun
25
QCAs commentary on Emma and Shaun

• Although both Emma and Shaun were a little
  hesitant to start with, they were able to ask and
  respond to a range of questions. They used the
  third person correctly and were able to give
  quite a lot of information about themselves and
  their family. Both pupils showed they can use
  familiar language independently and creatively,
  and both spoke with good pronunciation. Their
  ability to take part in a simple structured
  conversation is characteristic of level 4 in
  speaking.

26
My commentary. Yes but

• the objective was stated as
• To use previously learnt language from memory in
  a conversation.
• Was this a conversation?
• Sounds like rehearsed construction of chunks.
• Level 4 is the highest level of achievement
• and remember the QCA levels were developed
  originally for KS3 (11 to 14 year olds)

27
QCA Levels 3 and 4 descriptions

• Level 3
• Pupils take part in brief prepared tasks of at
  least two or three exchanges, using visual or
  other cues to help them initiate and respond.
  They use short phrases to express personal
  responses for example, likes, dislikes and
  feelings. Although they use mainly memorised
  language, they occasionally substitute items of
  vocabulary to vary questions or statements. 
• Level 4
• Pupils take part in simple structured
  conversations of at least three or four
  exchanges, supported by visual or other cues.
  They are beginning to use their knowledge of
  grammar to adapt and substitute single words and
  phrases. Their pronunciation is generally
  accurate and they show some consistency in their
  intonation.

28
How some YL of English testing boards do it
differently

• An adult interlocutor with one child is
  preferred the child is not required to initiate
  in an interaction. Is this more likely to allow
  for spontaneous responses than a Kelly and Shaun
  style conversation?
• The content of the conversation is known but not
  entirely predictable, since there are syllabus
  details but the examiner has scope for different
  initiations.

29
Example Trinity Graded Examinations in Spoken
English (for children)

• The exam consists of an unscripted one-to-one
  conversation with the examiner (held in a
  designated examination room) during which the
  students demonstrate that they can communicate in
  English.
• Grades 1 to 3 5 to 7 minutes. Unscripted but
  very full models for preparation are available

30
Example London Tests of English Assessment of
Test Performance

• 0 is the lowest possible mark and 5 is the
  highest possible mark.
• Board Game
• 1. Student uses appropriate vocabulary and
  grammar sufficiently accurately for the level.
• (5 marks) ______
• 2. Student pronounces sounds and words acceptably
  well to achieve understanding.
• (5 marks) ______

31
Example London Tests of English

• Individual Activity
• 3. Student uses appropriate vocabulary and
  grammar sufficiently accurately for the level.
• (5 marks) ______
• 4. Student pronounces sounds and words acceptably
  well to achieve understanding.
• (5 marks) ______
• Total Mark (out of 20) _____

32
The issue of skills coverage and priorities

• Most international boards have put enormous
  resources of expertise and finance into ensuring
  that oral language is given a very high profile
• (in useful contrast to the situation revealed by
  Rea-Dickins and Rixon (1999) with regard to
  teachers in class testing practices)

33
Example London Tests of English

• TEST FORMAT for BREAKTHROUGH (Level 4)
• The BREAKTHROUGH written test consists of a
  one-hour fifteen minute paper which tests
  listening,
• reading and writing. There is also a separate
  speaking test.
• Listening Tasks
• 30 Marks
• There are two listening tasks. For each one,
  candidates listen to a short recorded passage and
  complete a task. Each passage is played twice.
  The following task types are used
• multiple choice
• short written answers

34
Example London Tests of English

• Reading and Writing Tasks
• 50 Marks
• There are four reading and writing tasks. The
  following task types are used
• dialogue completion
• matching utterances and social situations
• filling in gaps in a text
• writing a narrative based on pictures
• Speaking
• 20 Marks
• Learners will be tested by participating in a
  board game activity and performing an individual
  activity.

35
The issue of reading comprehension

• In some tests reading comprehension is scarcely
  present or missing entirely. E.g. The London
  Tests of English for children at Level 4
• Reading and Writing Tasks
• 50 Marks
• There are four reading and writing tasks. The
  following task types are used
• dialogue completion
• matching utterances and social situations
• filling in gaps in a text
• writing a narrative based on pictures

36
Why might some areas of language receive less
attention from particular testing bodies?

• There will be respectable rationales for
  different test emphases but clearly choices made
  for a particular context may push pedagogy in
  the private or public sector or else leave it
  undisturbed
• The expectations of reading in course materials
  for Young Learners of English is a major issue.
  Literacy practices in countries outside Europe in
  which international English tests are taken can
  be very limited. Is this an issue for the
  testers?
• Is this the case for other languages which are
  taught to Young Learners?

37
The issue of tests of speaking

• As we have seen, in some tests of English,
  speaking is foregrounded, but this means
  considerable outlay of finance and effort for
  examiner training, travel, moderation .
• Consideration of childrens characteristics has
  led in several cases to avoiding child-child
  interaction as too conducive to unreliable
  performances. A costly option, therefore in
  terms of personnel.

38
Different types of criteria for judging speaking
performance

• atomistic criteria used for elements of speech
  structures used, pronunciation
• Global judgement of success in communication
  performance with regard to an adult
  conversation-leader
• (Initiation by child or child-child interaction
  seems to be off the current scales)

39
Avoiding the glass ceiling

• A clear example of the glass ceiling might be
  in some assessments of speaking but particularly
  in tests of reading.
• Many of the childrens tests or assessment
  scales that I have examined reward what can be
  called efficiency in the find and lift the
  answer strategy, which focuses on understanding
  a small local area of a text

40
Weir and Khalifa 2008

• A cognitive processing approach towards defining
  reading comprehension
• Research notes 31 Feb 2008
•  
• http//www.cambridgeesol.org/rs_notes/rs_nts31.pdf
  
• They looked at PET and FCE (B1 and B2)

41
Emmas favourite doll

• My name is Betty and I have a little sister
  called Emma. She has lots of dolls, but her
  favourite one is called Daisy. Mum and Dad gave
  it to her when she was a baby and she takes it
  everywhere with her. She takes it to school and
  to her bedroom and when we sit down to eat, the
  doll always sits next to Emma.
• Last Sunday, our family went to the park to have
  a picnic. We took our dog, Treasure, with us and
  of course, Emma took Daisy too. There were a lot
  of people in the park because it was sunny. We
  found a place near the lake to have our picnic.
  After lunch, Emma and I went on the swings. After
  a few minutes, Emma said to me, Betty, I want
  Daisy on the swing with me. Can you go and get
  her for me? OK! I answered. But when I went
  back to our picnic, Daisy wasnt there. Mum! I
  shouted, weve lost Daisy! Dad looked in all
  the bags and Mum and I looked under our sweaters
  and other things, but we couldnt find her. I
  went to tell Emma the bad news, but when I got
  there, I saw Treasure. He carried Daisy carefully
  in his mouth. Look! said Emma, Treasure has
  brought Daisy to play with me. Hes very kind.

42
The questions

• Emma got the doll when she .
• Daisy always sits Emma when she eats.
• The family had a in the park last
  Sunday
• The park was full of people because ..
• Emma and Betty played on .. after lunch
• Mum and Betty looked everywhere but they . the
  doll
• brought the doll to Emma

43
Push the pedagogy towards a less local view of
reading with children?

• My question with this type of test is whether it
  would not be worth including each time a bonus
  inference or other discourse level item.
• This would be transparently publicised as a known
  element in the test so that there is a chance
  that gradually both teachers and course materials
  writers might begin to take account of this level
  of textual understanding.

44
An example of a bonus glass-ceiling busting
comprehension across sentences question?

• e.g. Was Emma worried about her doll?
• (OK a simple no or yes could be a guess, but
  it would require the child to do more than to
  paraphrase or seek a clue within single
  sentences).

45
Weir and Khalifa 2008

• Comprehend across sentences
• (which fits my doll bonus question)
• fits with B1 and B2, rather than the A1 and A2
  level that I have been assuming for most YL
  tests, but this is where I might personally hope
  that a small venture into the B levels could have
  a very beneficial pedagogy-pushing effect for
  reading.

46
So possibly like the exquisite and semi-colon
children

• Im interested in a small hole in the glass
  ceiling that says
• Thats Level X that is
• and encourages teachers to go for that extra
  expectation real rather than rehearsed
  conversations, comprehension that goes beyond
  the single sentence level, whatever will move
  childrens language learning closer to a
  reasonable expectation of real achievement.

47

• Thanks for listening

48
Useful Reading