A Fresh Look at Assessment

1
Re-designing assessment to enhance learning
University of Kent
Phil Race BSc PhD PGCE FCIPD ILTM Visiting
Professor, Assessment, learning and
teaching Leeds Metropolitan University Friday, 06
November 2009
2
Re-designing assessment to enhance learning

• In this session, well explore various approaches
  to moving assessment forward from being the weak
  link in the chain (as students tell us in the
  National Student Survey), in particular looking
  at how we can re-design assessment to deliver
  more and better formative feedback to students,
  while at the same time speeding up the feedback
  and reducing the time it takes to mark their
  work.
• Downloadable handout material will include
  Making Feedback Work Chapter 5 of Making
  Learning Happen (Phil Race, 2005, London Sage),
  and extracts from Making Teaching Work (Phil
  Race and Ruth Pickford, 2007, London Sage,
  forthcoming).

3
Handout materials

• Chapter 5 from Making Learning Happen (2005).
• (Ill put the main slides Ive used up on my
  website www.phil-race.com by Monday, and leave
  them there for a few days I dont believe in
  3-per-page PowerPoint slides theres a short
  discussion about this on my website too).

4
Making Learning Happen Phil Race 2005, Sage,
London
5
Intended learning outcomes

• By the end of this session, you should be able
  to
• Explore the role feedback plays in student
  learning.
• Re-design assessment to give more students better
  feedback in less time.
• Find out what your students think of the feedback
  they receive, and now they use it.

6
Face-to-face one-to-one feedback activity

• Please work in pairs, moving around the room,
  talking to different people using the script
  which follows
• The script
• A Hello.
• B Hello.
• A You are late.
• B I know.
• Try to do it completely differently each time.

7
The power of face-to-face communication

• When explaining assessment criteria to students,
  and when linking these to evidence of achievement
  of the intended learning outcomes, we need to
  make the most of face-to-face whole group
  contexts and,,,
• Tone of voice
• Body language
• Facial expression
• Eye contact
• The chance to repeat things
• The chance to respond to puzzled looks
• Some things cant work nearly so well just on
  paper or on screens.

8
Keeping it simple?

• Everything should be made as simple as possible,
• but not simpler.
• (Albert Einstein, 1879-1955).

9
Timing is critical

• Feedback only really works after weve got
  students to do something.
• Feedback on something theyve actually done is
  far more powerful than feedback on something
  theyve merely thought.

10
Starting points

• On a post-it, please write your personal
  continuation of
• Getting feedback to students would be much
  better for me if only I
• Please stick all the post-its up as directed.

11
The place of feedback in how students learn

• Feedback is a vital step in learning.

12
Five factors for successful learning (from
Chapter 2 of Making Learning Happen

• learning by doing
• learning from feedback
• wanting to learn
• needing to learn
• making sense - digesting, getting ones head
  round it

13
Traditional views...

• active experimentation
• concrete experience
• reflective observation
• abstract conceptualisation

14
Is it a cycle?
Active Experimentation
Concrete Experience
Abstract Conceptualisation
Reflective Observation
Coffield et al
15
(No Transcript)
16
Coffield et al on Kolb

• Kolb clearly believes that learning takes place
  in a cycle and that learners should use all four
  phases of that cycle to become effective. Popular
  adaptations of his theory (for which he is not,
  of course, responsible) claim, however, that all
  four phases should be tackled and in order. The
  manual for the third version of the LSI is
  explicit on this point You may begin a learning
  process in any of the four phases of the learning
  cycle. Ideally, using a well-rounded learning
  process, you would cycle through all the four
  phases. However, you may find that you sometimes
  skip a phase in the cycle or focus primarily on
  just one (Kolb 1999, 4). But if Wierstra and de
  Jongs (2002) analysis, which reduces Kolbs
  model to a one-dimensional bipolar structure of
  reflection versus doing, proves to be accurate,
  then the notion of a learning cycle may be
  seriously flawed.

17
Coffield et al on Kolb (2004)

• Finally, it may be asked if too much is being
  expected of a relatively simple test which
  consists of nine (1976) or 12 (1985 and 1999)
  sets of four words to choose from. What is
  indisputable is that such simplicity has
  generated complexity, controversy and an enduring
  and frustrating lack of clarity.
• Frank Coffield, David Moseley, Elaine Hall and
  Kathryn Ecclestone 2004, Learning styles and
  pedagogy in post-16 learning a systematic and
  critical review London, Learning Skills Research
  Centre, now LSN, I keep a link on my website to
  the LSN website from which the pdf file can be
  downloaded (free).

18
Ripples on a pond.
Wanting/ Needing
From Making learning happen
19
Ripples on a pond.
Wanting/ Needing
Doing
20
Ripples on a pond.
Wanting/ Needing
Doing
Digesting
21
Ripples on a pond.
Wanting/ Needing
Doing
Digesting
Feedback
22
Fishing for feedback?

• Feedback is like fish.
• If it is not used quickly, it becomes useless.
• (Sally Brown).
• Give a man a fish,
• Feed him for a day.
• Teach a man to fish,
• Feed him for a lifetime.
• (Chinese proverb).

23
Getting feedback to students quickly

• The next two slides are about a way of giving
  students feedback on their work within 24 hours
  of them doing it.
• There are three or more yes, but what ifs with
  this idea, but please hold these for a couple of
  minutes.

24
Speeding up marking by feedback at the point of
submission

• There are ways of giving very quick feedback to a
  large group of students at the point of
  submitting their work for assessment.
• For example, collecting their work for marking at
  the start of a whole-group session, then
  immediately issuing a pre-prepared 1-page handout
  to the class covering
• Likely mistakes
• Features of a good answer
• Frequently needed explanations
• All the better if you can spend a few minutes
  de-briefing the whole group and talking them
  through the handout
• adding tone-of-voice, facial expression, body
  language, emphasis, and so on to the feedback.

25
How this saves time

• Since many students will have done the work in
  the last 24 hours before handing it in, youre
  giving them feedback while they still remember
  what they were doing.
• You then waste far less writing the same old
  things on one piece of work after another,
  regarding frequently occurring mistakes
• You can make your comments relate more to each
  individual piece of work
• This means when students get their marked work
  back with feedback, they are more likely to use
  it, as its personal to them.
• Also, theyve by then already had the chance to
  make sense of their own piece of work in the
  light of the generic feedback you gave them when
  they handed it in.

26
Feedback, and

• Feed-forward

27
Feedback (including feed forward) works well when
it

• Motivates students helps them to want to learn
• Helps students to identify what they need to do
  next
• Helps students to take action to improve their
  learning
• Helps students to get their heads around what
  they are learning
• Helps them to make realistic evaluations of their
  own abilities and achievements
• Helps them to reflect on their past work in ways
  they can use to improve their future work.

Wanting/ Needing
Doing
Digesting
Feedback
28
Feedback works badly when it

• Saps students confidence
• Directs students activities in inappropriate
  directions
• Fails to articulate with learning outcomes
• Fails to relate clearly to evidence of
  achievement of the assessment criteria
• Relates only to what is easy to assess rather
  than what is at the heart of learning
• Focuses on failings rather than achievements.

29
Why does formative feedback cause us problems?

• If the issues surrounding giving feedback
  effectively and efficiently were straightforward,
  they would have been solved long ago!
• Nevertheless, a growing body of research
  indicates that giving effective feedback is the
  most powerfully positive thing we can do to
  foster and maintain student learning.
• When we talk to unhappy students, minimal or
  over-negative feedback is often the biggest
  bugbear and the NSS bears out that this is
  widespread at present.

30
Auditing assessment and feedback

• We know from the NSS that weve got a problem
  with assessment and feedback, and theres no
  better time to start tackling it than now.
• Theres too much summative assessment in our
  systems, and not enough opportunity for
  feed-forward.
• We need to streamline assessment, and make the
  feedback associated with assessment work better
  for ourselves and for our students.

31
Evidencing our good practice

• Feedback to students, when written (or
  word-processed) is one of the most accessible
  indicators of the quality of our teaching.
• Such evidence is looked at very thoroughly by
  external agencies, professional bodies, funding
  council, external examiners.
• And they usually talk to students, who know best
  how well (or badly) our feedback actually works.

32
Seven key questions about feedback

• What do we say to students about their work?
• How do we say it?
• Do they take any notice?
• How much does it help their learning payoff?
• How well does it relate to students evidence of
  achievement of the intended learning outcomes?
• How well does it help them to achieve their next
  learning outcomes?
• How efficient is it for us?

33
Life is too short to

• Spend time and energy writing feedback which
  wont actually be used by students (sometimes not
  even collected by them)
• Write feedback just for external examiners to
  see.
• Approach giving feedback only in the read-write
  dimension, when many students gain more from it
  through auditory, or visual, or kinaesthetic
  channels.
• (see Neil Flemings excellent (and free!) VARK
  work on www.vark-learn.com).

34

How to give more and better formative feedback to
more students in less time!
35
High learning payoff for students
Highly efficient for us
Not highly efficient for us
Low learning payoff for students
36
Feedback on paper

• Handwritten comments on returned assessed work
• Word-processed comments about assessed work, e.g.
  front sheets and so on
• Model answers
• Assignment return sheets
• Code letters or numbers written onto students
  work
• Word-processed overall reports on the work of a
  class

37
Face-to-face feedback

• Face-to-face to whole lecture groups
• Face-to-face to individual students
• Face-to-face to groups of students

38
E-feedback and more

• Emailed comments on students individual work
• Overall comments delivered electronically through
  a computer conference
• Computer-delivered feedback
• Peer-group feedback associated with
  peer-assessment
• Feedback you give students who have self-assessed
  their work.

39
Exercise on ways of giving students feedback

• Think of the main ways you give feedback to your
  students. Think also of other ways they get
  feedback without you being involved.
• Make a list of all these ways your students get
  feedback on their learning.
• Now in groups, write each feedback method on a
  separate post-it, and place the post-its on a
  flipchart as follows

40
5
25
High learning payoff
We need to be making better use of the
feedback processes in this quadrant
16
4
Highly efficient
Not highly efficient
3
2
4
5
1
2
4
Low learning payoff
We need to be making much less use of the
feedback processes in this quadrant
1
1
41
Kent feedback methods

• High scoring
• 25 peer feedback
• 25 peer assessment
• 25 verbal feedback in group setting
• 25 display of work to whole group leading
  discussion based on assessment criteria
• 16 Web CT student discussion
• 16 group discussion

• Low scoring
• 4 parental
• 2 correcting all errors in essays
• 2 marks
• 2 formal exam results
• 1 grapevine
• 1Written on script or feedback sheet

42
Tyne Met feedback methods

• High scoring
• 25 self-assessment with target setting
• 25 Self assessment against criteria
• 25 peer feedback
• 22 self-reflection and assessment
• 20 client comments on work
• 20 peer feedback
• 16 peer assessment
• 16 group feedback immediately after an
  assessment
• 16 Informal feedback in monitoring in class

• Low scoring
• 5 written feedback
• 5 empty praise, e.g. youve all done very well
• 4 front sheets
• 2 summative feedback at the end of the course
• 1 end of term reports

43
Curtin University feedback methods

• High scoring
• 25 verbal instant feedback on activities e.g. in
  prac class
• 25 summary feedback to whole class
• 20 students mark each others work providing
  feedback on marking scheme
• 20 next class put up examples on screen
• 20 poster presentation in tutorials
• 16 small group feedback by tutor in seminar

• Low scoring
• 9 quiz e.g. paper-based mcq
• 4 written feedback on individuals work
• 3 as above
• 4 providing good and bad examples of work before
  it is due
• 1 exams without feedback

44
South East Essex College feedback methods

• High scoring
• 25 body language
• 25 peer assessment, critique
• 25 timetabled tutorial 11
• 20 managed peer discussion
• 20 class discussion
• 20 electronic feedback sheets
• 20 on-line guided learning and assessment
• 18 tutor-led group critique

• Low scoring
• 5 written feedback on front sheets
• 1 written feedback on front sheets
• 1 written feedback with grade
• 4 just a mark
• 1 written assignment front sheets
• 6 comments on feedback sheet
• 4 written feedback
• 4 email comments on drafts

45
Kingston University feedback methods

• High scoring
• 25 repeat exercises
• 25 service users in simulations
• 25 group discussion
• 23 group feedback from us to them
• 22 generic comments
• 20 one-to-one feedback in labs
• 20 products give instant feedback
• 20 poster presentations
• 16 face-to-face between sessions
• 16 diagnostic tests

• Low scoring
• 5 just a grade
• 2.5 grade
• 2 written comments
• 1 written comments
• 6 exam marks

46
Syddansk Universitet feedback methods

• High scoring
• 25 feedback during active learning during
  teaching
• 20 oral presentation with instant feedback from
  teacher and class
• 20 peer review students critiquing paper
• 15 students in tutor-less groups reviewing their
  learning and making summaries
• 12 discussion session and solutions on Blackboard

• Low scoring
• 4/3/2 normal written exam with mark only

47
University of Central Lancashire feedback methods

• High scoring
• 25 student evaluation where they analyse and
  review their own work
• 20 gold star
• One-to one meetings before handing in assignment
• 20 one-to-one meetings at students request
• 20 individual presentations with immediate
  feedback
• 18 peers discussing their own work
• 12 web-based feedback

• Low scoring
• 2 written comments using codes front sheet
• 1 cover sheets
• 4 email comments on individual drafts
• 3 individual written comments on text

48
De Montfort feedback methods

• High scoring
• 25 visual and verbal feedback on best practice
  from tutors one-to-many
• 25 students talking to each other
• 25 work placement
• 20 several-to-one viva
• 20 common pitfalls and what a good one looks like
  one to many
• 20 external competitions
• 16 peer-assessment feedback

• Low scoring
• 1 PDP 1-1tutorial
• 1 module evaluation forms
• 4 individual critiques
• 4 general feedback on blackboard
• 2 written feedback sheets in a box
• 5 generic written feedback sent out

49
University of Aalborg methods

• High scoring
• 20 students discussing group work
• 20 ditto
• 20 peer assessment
• 20 students reviewing each other
• 16 peer assessment

• Low scoring
• 2 individual oral examination
• 3 comments on assignment sheets
• 3 written feedback on exercises
• 3 giving marks to exercises
• 5 face-to-face feedback, and written comments

50
Leeds Met feedback methods

• High scoring
• 25 verbal feedback to whole group
• 25 general feedback to the whole group with
  individual comments
• 25 instant feedback by self assessment
• 16 reflective journal self evaluation
• 16 peer feedback
• 14 whole group verbal feedback

• Low scoring
• 1 written individual (ditto)
• 5 no feedback

51
Newcastle College highest and lowest scoring
methods

• High scoring
• 25 ILP review, written and verbal
• 20 subject support lessons with group
• 20 individual peer feedback
• 16 handouts on common errors
• 15 verbal face-to-face one-to-one
• 12 tutorial groups face-to-face

• Low scoring
• 1 end of year individual reports
• 3 end of term individual reports
• 4 returned work
• 4 emails to individuals
• 10 verbal face-to-face
• 10 written one-to-one

52
Leeds Met feedback methods

• High scoring
• 25 in class presentation with instant feedback /
  peer
• 25 critique previous students essays in class
• 20 checklist for a presentation self/peer
• 18 feedback loop with directed activities
• 16 open tutorial in class time

• Low scoring
• 3 down to 1 summative assignment sheets after
  modules
• 5 exam with just a mark
• 1 written scripts

53
Leeds Met feedback methods

• High scoring
• 25 peer-assessment
• 25 practice-based learning
• 25 good immediate electronic feedback,
  comment-bank with marks later
• 20 students mark anonymous essays using comment
  banks related to assessment criteria
• 20 practice-based learning
• 20 discussion groups online

• Low scoring
• 4 written on assignments
• 5 formal feedback sheets
• 5 warnings in lectures regarding past groups
  performances bollocking
• 2 module grades
• 1 marks when seen first on feedback sheets

54
Teesside highest and lowest scores

• Highest
• 25 feedback from patients/clients
• 20 self-assessment
• 20 email
• 16 collaborative feedback with mentors in
  practice
• 16 face-to-face in groups
• 16 computer-based self-assessment
• 16 handwritten comments on draft work
• 16 handwritten sheets during presentations

• Lowest
• 2 model answers on electronic discussion board
• 4 written feedback on past exams
• 8 written comments in portfolios

55
Maynooth top and bottom scoring feedback methods

• 25 discussion
• 20 e-learning with instant feedback
• 20 individual peer assessment
• 16 supervised peer-assessment
• 16 Coded grades with email discussion

• 5 giving marks only
• 4 grades with oral correction
• 4 written exams

56
University of South Australia top and bottom
scoring feedback methods

• Top scoring methods
• 25 emails to group
• 25 peer assessment
• 20 moderated self assessment
• 20 group peer review
• 16 model answers
• 14 criterion based written feedback

• Bottom scoring methods
• 6 handwritten on assignments
• 5 single word comment
• 4 exams
• 3 just a mark
• 3 grabbing comments from a bank
• 3 Web conferencing

57
Belfast most favoured feedback processes

• 25 multiple choice(computer-based, used for
  feedback and not testing)
• 20 work experience feedback
• 20 opportunity for discussion one to one
• 20 peer assessment
• 16 self evaluation
• 16 feedback session to a large group
• 12 feedback during an activity
• --------------------------------
• 6 written one to one
• 2 exam
• 1 writing negative comments
• 1 just giving it a mark

58
Feedback should be

• Timely the sooner the better.
• Personal and individual each student is still a
  person.
• Articulate message getting across first time
  round.
• Empowering warming up learning, not dampening
  it down.
• Manageable for us but also for them.
• Developmental opening doors, not closing them.

59
Finding out what works for students

• Ask them!
• (e.g. adapt the various checklists for students
  in Making Learning Happen)

60
Back to our intended outcomes

• Do you now feel better able to
• 2 hands very much better, one hand somewhat
  better, no hands no better)
• Explore the role feedback plays in student
  learning?
• Re-design assessment to give more students better
  feedback in less time?
• Find out what your students think of the feedback
  they receive, and now they use it?

61
Action planning statements

• One thing Im going to do is
• One idea Im taking away is
• Im going to think more about
• I have found out that
• Id like to know
• In future, Im not going to

Friday, 06 November 2009
62
Thank you www.Phil-Race.com e-mail
p.race_at_leedsmet.ac.uk