Beyond Self and Peer Assessment

1

• Beyond Self and Peer Assessment
• David Boud

2
Outline

• Part 1.
• What do we know about self and peer assessment?
• Part 2.
• How can we think differently about assessment
  developing informed judgement?
• Part 3.
• How can we involve students as active agents in
  making judgements?

3
What is self assessment?

• the involvement of students in identifying
  standards and/or criteria to apply to their work
  and making judgements about the extent to which
  they have met these criteria and standards.

4
What does this imply?

• Students may not be the only judges
• Self assessment may be facilitated or left to
  chance
• There may be different understandings of
  involvement and making judgments of learning
• There may be different practices involved
• Self assessment doesnt necessarily involve
  student involvement in summative assessment
• Self assessment is not done in isolation from
  others or from standards

5
What do we know about self assessment?

• Self assessment is a necessary skill for learning
• within the course
• after the course
• Learners can be realistic in making self
  assessments
• inexperienced learners and those new to an area
  tend tp overrate themselves
• advanced learners are more realistic and can tend
  to underrate themselves
• Context strongly influences ratings
• incentives for students to overrate tend to work
• Practice in making judgements improves self
  assessment
• One-off uses of self assessment dont have much
  impact

6
What else do we know?

• Self assessment is not a method or technique, it
  simply represents who is the main agent in making
  judgements
• Self assessment is best if not used in isolation
  from students considering other input
• eg. of peers, of teachers, etc
• Self assessment practices are extraordinarily
  varied and must be designed to fit the
  circumstances
• of the subject
• of the stage of development of the learner

7
Five qualitatively different ways of experiencing
student self-assessment

• Category A as ensuring students behavioural
  compliance
• Category B as allowance for students contingent
  judgments of their knowledge
• Category C as providing feedback on students
  judgments of requisite standards in the program
  of study
• Category D as developing students judgments of
  their proficiency in the program of study
• Category E as sustaining students ability to
  self-assess beyond the program of study
• (Tan 2006)

8
Self assessment for formative and summative
purposes

• Self assessment for formative purposes can be
  fostered by learning tasks
• In limited circumstances student marks can be
  used for summative purposes

9
Self/peer assessment may be used for grading when

• there is a high trust, high integrity learning
  environment
• students are rewarded for high integrity marking
• marks are moderated by staff so that deviations
  need to be justified
• blind peer marking is used as a check
• random staff marking is used as a check
• students have had ample opportunity to practice
  and develop their skills
• criteria have been sufficiently unambiguously
  defined to minimise misinterpretation of grade
  boundaries
• effort is explicitly excluded as a criterion

10
Conditions in which self marking may be justified

• when students are new to the knowledge domain and
  cannot yet recognise good work
• when it is a preliminary stage to self assessment
  proper
• when it is used in association with distancing
  devices to help students look afresh at their
  work
• when rating scales used do not have connotations
  of what is socially desirable
• when the sub-components of the task, not global
  marking are emphasised
• when all scales and points on scales are explicit

11
Key features self assessment

• Active involvement in process, not following a
  recipe
• Students involved in determining criteria, not
  just self-marking
• Link to learning outcomes
• May involve peers at some stage
• Emphasis on informing judgment

12
Is self assessment flawed?

• Recent medical education literature points to
  the limitations of self assessment. How should we
  regard what they show?
• Based upon meta analyses that typically
  demonstrate moderate correlations between
  self-judgement and those of teachers (0.3-0.4)
• Many of the empirical studies on which they are
  based are limited and were not designed to
  calibrate self assessment.
• Use multiple sources of feedback to calibrate
• Self assessment alone has, of course,
  considerable limitations for summative assessment
  purposes.
• However, there is no choice but to persist with
  improving self assessment if learning is about
  developing judgement.

13
Peer assessment research 1

• Peer assessment seems adequately reliable and
  valid in a wide variety of applications, although
  virtually all of the current literature considers
  reliability of marks or grades rather than more
  detailed, formative assessment. Levels of
  acceptability to students are varied and do not
  seem to be a function of actual reliability.
  Students find peer assessment through tests,
  marks, or grades demanding but anxiety reducing.
  Learning gains in terms of test performance,
  skill performance, or subjective measures are
  frequently reported. (p. 268)
• Topping, K. (1998) Peer assessment between
  students in colleges and universities, Review of
  Educational Research, 68, 3, 249-276.

14
Peer assessment research 2

• Peer assessment and feedback of a more
  detailed, open-ended nature have been associated
  with improved confidence and better presentation
  and appraisal skills. The relatively high number
  and quality of studies of peer assessment of
  writing suggest outcomes at least as good as
  teacher assessment, and sometimes better. peer
  assessment of group and project work has been
  positive in terms of student perceptions.
  Similarly, peer assessment of professional skills
  shows adequate reliability but limited outcome
  data, often in participant perceptions. However,
  these again show outcomes at least equivalent to
  teacher assessment. (p. 268)
• Topping, K. (1998) Peer assessment between
  students in colleges and universities, Review of
  Educational Research, 68, 3, 249-276.

15
Using peer assessment

• Use formatively in conjunction with self
  assessment
• Peer feedback without summative elements can be
  used very widely
• Peer assessment with a summative flavour must be
  used carefully otherwise it can inhibit the very
  learning it seeks to promote

16
Key features peer assessment

• As before for self assessment
• Focus on peer assessment when communicating ideas
  to others is important
• Use guidelines for giving and receiving feedback
• Focus on qualitative peer feedback, downplay or
  eliminate ratings and grading

17
Giving and receiving feedback

• Offering feedback
• Be realistic
• Be specific
• Be sensitive to the persons goals
• Be timely
• Be descriptive
• Be consciously non-judgmental
• Dont compare
• Be diligent
• Be direct
• Be positive
• Be aware

• Receiving feedback
• Be explicit
• Be attentive
• Be aware
• Be silent

18
3. Returning to assessment generally

• A. What to consider about assessment in higher
  education
• B. Contrasting models of educational assessment
• C. Thinking about developing students judgement
• D. Implications of viewing assessment as about
  informing judgement

19
A. What to consider in assessment in higher
education

• Assessment is about judgement.
• Currently judging learning outcomes against
  standards
• Assessment must contribute to learning
• for learning now
• for future learning
• Assessment is about both informing students
  judgements as well as making judgements on them
• Summative assessment alone is to risky
• Students must necessarily be involved in
  assessment
• Assessment is a key influence in their formation
  and they are active subjects.

20
B. Contrasting models of educational assessment

• Scientific measurement model
• Practice derived from theory
• Knowledge is a given for practical purposes
• Knowledge is impersonal and context free
• Discipline-driven
• Deals with structured problems

• Judgemental model
• Practice and theory (loosely) symbiotic
• Knowledge is understood as provisional
• Knowledge is a human construct and reflects
  context
• Problem-driven
• Deals with unstructured problems
• (Hager and Butler 1996)

21
C. Thinking about developing judgement

• Students must develop the capacity to make
  judgments about their own learning
• Otherwise they cannot be effective learners now
  or in the future
• We can never provide them with as much or as
  detailed feedback as students need.
• Some kinds of feedback inhibit judgment through
  fostering dependency and compliance.
• Capacity for self assessment is central to
  informing judgment
• But simply adding self assessment activities is
  not sufficient.
• Communities of judgment beyond ourselves need to
  be engaged with (peers, practitioners,
  professional bodies).

22
D. Assessment as informing student judgement
implies

• Always look to what the consequences of
  assessment are for learning
• Focus on fostering reflexivity and
  self-regulation through every aspect of a course,
  not just assessment tasks
• Recognise the variety of contexts in which
  learning occurs and is utilisedjudgement is not
  independent of context
• Stage opportunities for developing informed
  judgement throughout programs
• Assessment must be integrated with learning and
  integrated within the program and over time

23
The problem of judgement

• Judgement is more elusive than it appears
• Making judgements are context specific and
  context dependent
• There are intrinsic biases in making judgements
• Judgement is always a subjective act, especially
  when the acts of people are judged
• Getting to self-assessment is essential to
  judgement, but it is always flawed

24
Students as active agents in developing judgment

• Student agency
• Communities of judgment
• Self assessment
• Role of peers

25
Why involve students?

• How can we justify not involving them?
• But what does it mean to involve them?
• Developing judgment is about more than acquiring
  knowledge and skills, it involves practice in
  discernment.
• More opportunities for this are needed.
• Students learn a lot through contributing to the
  learning of others
• engaging with criteria, formulating ideas, taking
  account of the other
• Students are a massively under-utilised resource
  at a time of resource constraints

26
Figure 1. Elements of informed judgement.
27
Some practices for involving students

• Dont mistake practices for purposes!
• Not the practice but the purpose that counts.
• Self assessment
• Peer assessment
• Hybrids
• Ultimately all assessment must be about
  informing the judgment of learners as it is only
  they who can learn.

28
Workshop task

• 1. Identify an assessment activity you wish to
  modify to enhance its contribution to learning in
  the longer term and developing student judgement.
  Make notes about what you will change.
• 2. In groups of three. Take turns in sharing your
  planned assessment activity and getting feedback.
• 3. Identify an issue that has arisen to bring
  back to the group.

29
Issues arising from the workshop task
30
Practicalities

1. Common features for any assessment innovation
2. Choosing appropriate tasks and processes
3. Giving and receiving feedback

31
Common features in any assessment innovation

• Assume students will be resistant, if they are
  not then theyre probably not behaving rationally
• Never underestimate the importance of providing a
  compelling rationale and reiterating it in
  different ways
• Act confidently. Listen to their concerns, but
  dont change what you are doing unless they
  suggest a better way of doing it.
• Be much more explicit than you imagine must be
  necessary, give full guidelines/deadlines, etc.
  in writing
• Reassure students that cheating/ collusion will
  be detected if grading is involved
• Be prepared to discuss tangible benefits to them

32
B. Choosing appropriate tasks and processes

• ask learners to make judgments on matters on
  which it is reasonable for them to do so
• use cues for success which are embedded in the
  content as much as possible
• choose situations in which there are external
  sources of judgment which can be drawn upon or,
  multiple sources of evidence are available
• avoid situations in which criteria for success
  are matters of opinion or taste
• avoid incentives for mis-assessment
• choose specific and concrete rather than the
  global and abstract task
• limit the number of criteria which must be
  considered simultaneously at first
• develop detailed guidelines on how the process is
  to be undertaken

33
Assessment is not enough

• We cant consider assessment separately from
  teaching and learning processes. All are about
  informing judgment.
• Alignment between and integration of learning
  activities is needed
• Choosing assessment practices chooses what
  students will learn

34
The new agenda

• Not just constructive alignment, but alignment of
  assessment now with long term learning goals
• Breakdown the binary between assessment and
  pedagogy
• Revisit assessment from the perspective of
  lifelong learning
• Revisit pedagogy from the perspective of lifelong
  assessment

35
References

• Boud, D. (1995). Enhancing Learning through Self
  Assessment. London Kogan Page.
• Boud, D. (2000). Sustainable assessment
  rethinking assessment for the learning society.
  Studies in Continuing Education, 22, 2, 151-167.
• Boud, D. and Falchikov, N. (2006). Aligning
  assessment with long term learning, Assessment
  and Evaluation in Higher Education, 31, 4,
  399-413.
• Boud, D. and Falchikov, N. (Eds.) (2007)
  Rethinking Assessment in Higher Education
  Learning for the Longer Term. London Routledge.
• Falchikov, N. (2005). Improving Assessment
  through Student Involvement. London Routledge.
• Gibbs, G. (2006). How assessment frames student
  learning. In Clegg, K. and Bryan, C. (Eds.)
  Innovative Assessment in Higher Education.
  London Routledge.