Enhancing Assessment and Feedback - PowerPoint PPT Presentation

1 / 56
About This Presentation
Title:

Enhancing Assessment and Feedback

Description:

Enhancing Assessment and Feedback Principles and Practice David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement (CAPLE] – PowerPoint PPT presentation

Number of Views:204
Avg rating:3.0/5.0
Slides: 57
Provided by: France190
Category:

less

Transcript and Presenter's Notes

Title: Enhancing Assessment and Feedback


1
  • Enhancing Assessment and Feedback
  • Principles and Practice
  • David Nicol
  • Professor of Higher Education
  • Centre for Academic Practice and Learning
    Enhancement (CAPLE
  • Director of REAP and PEER projects
    (www.reap.ac.uk)
  • University of Strathclyde, Scotland
  • ATN Australia 20th October 2011

2
National Student Survey (UK) Assessment
feedback (2008)
No Survey Statement England Scotland Northern Ireland
5. The criteria used in marking have been clear in advance 69 69 69
6. Assessment arrangements and marking have been fair 74 74 73
7. Feedback on my work has been prompt 56 51 53
8. I received detailed comments on my work 61 52 52
9. Feedback on my work has helped clarify things I did not understand 56 51 50
22 Overall, I am satisfied with the quality of the course 82 86 83
3
Plan
  • Background
  • Re-engineering Assessment Practices (REAP)
    project 1m
  • Concepts and example of practice
  • An institutional viewpoint
  • PEER project

4
Background
  • Departments and faculties supporting educational
    improvement projects.
  • REAP project big implementation across 3
    universities
  • Policy/strategy led development of policy in
    assessment and feedback based on REAP
  • Students Feedback as dialogue campaign
  • PEER project JISC funded 50k
  • HE Sector Project facilitator for QAA Scotland
    on AF
  • Research/publications assessment, learning,
    change
  • See www.reap.ac.uk

5
REAP Re-engineering Assessment Practices
  • Scottish Funding Council for Universities (1m
    project)
  • 3 Universities - Strathclyde, Glasgow Glasgow
    Caledonian
  • Large 1st year classes (160-600 students)
  • A range of disciplines (19 modules 6000
    students)
  • Many technologies online tests, simulations,
    discussion boards, e-portfolios, e-voting,
    peer/feedback software, VLE, online-offline
  • Learning quality and teaching efficiencies
  • Assessment for learner self-regulation
  • www.reap.ac.uk

6
Background (1)
  • Gibbs, G. Simpson, C (2004) Conditions under
    which assessment supports students learning,
    Learning and Teaching in Higher Education, 1,
    3-31.
  • See
  • Formative Assessment in Science Teaching (FAST)
    project at http//www.open.ac.uk/science/fdtl

7
Gibbs and Simpson (2004)
  • Assessment tasks Conditions 1-4
  • Capture sufficient study time (in and out of
    class)
  • Are spread out evenly across timeline of study
  • Lead to productive activity (deep vs surface)
  • Communicate clear and high expectations
  • i.e concern here is with time on task how much
    work students do - their active engagement in
    study

8
Background (2)
  • Literature Review
  • Nicol, D. Macfarlane-Dick, D. (2006). Formative
    assessment and self-regulated learning A model
    and seven principles of good feedback practice.
    Studies in Higher Education, 34 (1), 199-218
  • Background
  • Student Enhanced Learning through Effective
    Feedback SENLEF project funded by HE Academy
  • REAP project www.reap.ac.uk

9
Rethinking assessment and feedback
  • 1. Consider self and peers as much as the teacher
    as sources of assessment and feedback
  • Tap into different qualities than teacher can
    provide
  • Saves time
  • Provides considerable learning benefits (lifelong
    learning)
  • 2. Focus on every step of the cycle
  • Understanding the task criteria
  • Applying what was learned in action
  • 3. Not just written feedback
  • Also verbal, computer, vicarious, formal and
    informal

10
Seven principles of good feedback
  • Good feedback
  • Clarifies what good performance is (goals,
    criteria, standards).
  • Facilitates the development of reflection and
    self-assessment in learning
  • Delivers high quality information to students
    that enables them to self-correct
  • Encourages student-teacher and peer dialogue
    around learning
  • Encourages positive motivational beliefs self
    esteem
  • Provides opportunities to act on feedback
  • Provides information to teachers that can be used
    to help shape their teaching (making learning
    visible)
  • Source Nicol and Macfarlane-Dick (2006)

11
Principle 1 Clarify what good performance is
(the context of dialogue)
EMPOWERMENT/ SELF-REGULATION
  • Students create criteria
  • Students add own criteria
  • Students identify criteria from samples of work
  • Exemplars of different performance levels
    provided
  • Students rephrase criteria in own words
  • Provide document with criteria

ENGAGEMENT
12
Two meta principles
  • 1. Meta-PRINCIPLE 1 time and effort on task
    (structured engagement) i.e. steers on how much
    work to do and when Gibbs and Simpson 4
    conditions
  • 2. Meta-PRINCIPLE 2 developing learner
    self-regulation (empowerment/self-regulation) i.e
    steers to encourage ownership of learning the
    seven principles discussed above.
  • Key task for teacher is to balance 1 and 2

13
  • Example
  • Psychology

14
Psychology
  • 560 first year students
  • 6 topic areas (e.g. personality, classical
    conditioning), 48 lectures, 4 tutorials, 12
    practicals
  • Assessment 2 x MCQs (25), tutorial attendance
    (4), taking part in experiment (5), essay exam
    (66)

15
Problems identified
  • No practice in writing skills but required in the
    exam
  • More detail provided in lectures than mentioned
    in exams (not enough independent reading)
  • No feedback except on Multiple Choice Questions
    (percent correct)
  • Didnt want to increase staff workload

16
Psychology Redesign
  • Discussion board in Learning Management System
  • Students in 85 discussion groups of 7-8, same
    groups throughout year
  • Also open discussion board for class
  • Friday lectures cancelled discover material
    themselves
  • Series of online tasks

17
Structure of group tasks
  • 6 cycles of 3 weeks (one cycle x major course
    topic)
  • First week light written task (e.g. define
    terms) 7 short answers (all answer)
  • Second week guided reading
  • Week three deep written task students
    collaborate in writing a 700-800 word essay on
    the same topic.
  • Within each week
  • The Monday lecture introducing material
  • Immediately after lecture, task posted online
    for delivery the following Monday
  • Model answers (selected from students) posted for
    previous weeks task

18
The teaching role
  • Participation in the discussions was compulsory
    but not marked (in subsequent years there was 2
    mark for participation)
  • Course leader provided general feedback to the
    whole class often motivational
  • He encouraged students to give each other
    feedback
  • The group discussions were not moderated but
    monitored for participation

19
An example of deep task
  • The Task 800 word essay
  • Assess the strengths and weaknesses of Freuds
    and Eysencks theories of personality. Are the
    theories incompatible?
  • readings suggested
  • questions provided which all students should try

20
Relation to the Gibbs Simpsons four
assessment conditions
  1. Tasks require significant study out of class
    (condition 1)
  2. Tasks are distributed across topics and weeks
    (condition 2)
  3. They move students progressively to deeper levels
    of understanding (condition 3)
  4. There are explicit goals and progressive increase
    in challenge (condition 4)

21
Relation to 7 feedback principles
  • Standard format and model answers provide
    progressive clarification of expectations
    (principle 1)
  • Students encouraged to self-assess against model
    answer (principle 2)
  • Course leader provides motivational and
    meta-level feedback and selects model answers
    (principle 3)
  • Online peer discussion aimed at reaching
    consensus is core feature of design about
    response (principle 4)
  • Focus on learning not just marks, sense of
    control/challenge enhanced motivation
    (principle5)
  • Repeated cycle of topics and tasks provide
    opportunities to act on feedback (principle 6)
  • VLE captures all interactions allowing course
    leader to monitor progress and adapt teaching
    (principle 7)

22
Benefits
  • Students worked exceptionally hard
  • Written responses of exceedingly high standard
  • Students took responsibility for learning
  • High levels of motivation atmosphere in class
    improved
  • Online interactions showed powerful scaffolding
    and community building
  • Feedback with 560 students through peer and
    self-feedback (model answers)
  • Easy for tutors to monitor participation
  • Improved mean exam performance (up from 51-59,
    plt0.01) weaker students benefit most

23
Online postings/interaction
  • 24,362 messages posted by groups for essay tasks
  • Average number of postings per student 44.3
  • 1067 postings to general open discussion forum
  • Students set up online study groups for other
    subjects
  • Structured tasks online triggered important
    socio-cognitive processes

24
Has it worked?
25
(No Transcript)
26
(No Transcript)
27
  • Questions and Discussion

28
Why use principles
  1. Provides a framework for operationalizing the big
    idea the development of learner self-regulation
  2. Helps translate the research into accessible
    guidelines for teaching practice
  3. Ensures change is educationally driven
  4. Enables connections to be made across innovations
    in different disciplines
  5. Provides a common language to talk about
    innovation and for dissemination
  6. Can add to the evaluation process measures
  7. Helps identify where technology can leverage
    benefits

29
Guidelines for Implementation
  1. A single principle or many?
  2. Tight-loose maintain fidelity to the principles
    (tight) but encourage disciplines to develop
    their own techniques of implementation (loose)
  3. Balance teacher feedback with peer and
    self-generated feedback
  4. The more actively engaged students are, the
    better the course design

30
Developments since REAP
  1. Principles of Assessment and Feedback approved by
    University Senate and embedded in policy (2008)
  2. Use of principles to inform curriculum renewal
    and Quality Assurance processes
  3. Feedback as Dialogue campaign to gain
    commitment of students
  4. PEER Project (Peer Evaluation in Education Review)

31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
Peer Review in Education Evaluation PEER
  • The aims of the PEER project are to
  • Review evidence base for peer review
  • Develop educational designs for peer review (and
    self-review)
  • Identify software support for peer review
  • Pilot implementations of peer review with large
    student numbers
  • Produce guidelines for higher education why do
    it, how to do it, pitfalls and solutions and
    software possibilities.
  • see http//www.reap.ac.uk/peer.aspx

35
Peer feedback augmenting teacher feedback
  • Increasing quantity and variety of feedback
  • No extra workload on teacher with software (e.g.
    PeerMark, Aropa)
  • More timely e.g. collaborative projects
  • Simulates professional life reconciling
    different feedback perspectives

36
The argument
  • Not enough attention has been focused on the
    potential of peer feedback not just as a way of
    increasing the quantity and quality of the
    feedback students receive, but as a way of giving
    students practice in constructing feedback.

37
The focus
  • Scenarios where students make evaluative
    judgements about the work of peers and provide a
    feedback commentary, usually written
  • Not talking about scenarios involving
  • ...informal feedback in collaborative tasks
  • .....students evaluating each others
    contribution to group working
  • .....students grading/marking each others work,
    although some rating might be part of peer
    design

38
Benefits of feedback construction (1)
  • Constructivist rather than telling paradigm
  • High-level cognitive activity students cannot
    easily be passive
  • Students actively exercise assessment criteria
    from many perspectives
  • Writing commentaries develops deep disciplinary
    expertise
  • See many approaches and learn that quality can be
    produced in different ways
  • Shifts responsibility to student puts them in
    role of assessor exercising critical judgement

39
Benefits of feedback construction (2)
  • Learn to assess own work as exactly the same
    skills are involved
  • Develops capacity to make evaluative judgements
    - a fundamental requirement for life beyond
    university. Also, this capacity underpins all
    graduate attribute development (Nicol, 2010)
  • Nicol, D (2011) Developing students ability to
    construct feedback, Published by QAA for Higher
    Education, UK

40
Example 1Peer feedback
  • Laboratory work Gibbs 2011 www.testa.ac.uk/resour
    ces
  • Reason poor quality of lab reports in science
  • Strategy students organised in groups and
    produce poster to represent their lab report
  • Peer process hang poster in class and all
    students individually walk round analyse posters
    and write feedback on them (e.g post-its)
    questions, suggestions, inaccuracies etc
  • Result significant improvements in lab work and
    reporting, positive competition in class,
    students did not want to look bad.

41
Example 2 Engineering Design
  • Peer Project case study
  • DM 100 Design 1 first-year class
  • Dr Avril Thomson, Course Leader, Design
    Manufacturing and Engineering Management (DMEM),
    University of Strathclyde
  • avril.thomson_at_strath.ac.uk
  • Caroline Breslin, Learning Technology Adviser,
    University of Srathclyde
  • caroline.breslin_at_strath.ac.uk

42
Example 2 Design 1
  • 82 first-year students
  • Design a product theme eating and resting in
    the city
  • Research in groups (in city, in library etc.)
  • Individually produce a Product Design
    Specification (PDS) detailed requirements for
    and constraints on design (rationale,
    performance,standards, manufacturing etc)
  • Given a PDS exemplar from another domain to show
    whats required (stainless steel hot water
    cylinder)
  • Online learning environment Moodle and PeerMark
    part of Turnitin suite

43
Product Design SpecificationTypical Headings
Rationale Performance Environment Packaging Maintenance Weight and size Disposal Shipping Processes Time Scale Patents Product costs Aesthetics Ergonomics Materials Testing Quality/reliability Competition Marketing Life in service
44
DM 100 Design 1
  • Peer review task
  • Individually, each student peer-reviewed and
    provided feedback on the draft PDS of two other
    students
  • Criteria completeness, convincingness of
    rationale, specificity of values (performance)
    and one main suggestion for improvements with
    reasons
  • Students used experience, giving and receiving
    feedback, to update PDS which comprises part of a
    Folio
  • Students self-review Folio and meet/discuss with
    tutor
  • Peer review not assessed directly but 10 marks
    for professionalism which included participation
    in peer review.

45
Peer review rubric DM 100
  • Do you feel the PDS is complete in the range of
    headings covered? If no, can you suggest any
    headings that would contribute towards the
    completeness of the PDS and explain why they are
    important?
  • Is the PDS specific enough? Does it specify
    appropriate target values or ranges of values?
    Please suggest aspects that would benefit from
    further detail and explain.
  • To what extent do you think the rationale is
    convincing for the PDS? Can you make any
    suggestions as to how it might be more
    convincing? Please explain.
  • Can you identify one main improvement that could
    be made to the PDS? Provide reason(s) for your
    answer.

46
Evaluation
  • Online survey completed by 64 students
  • Course work marks compared to previous years
  • Focus group interviews
  • Peer review comments recorded online

47
Results 1
  • Which aspects of the peer review did you learn
    from?
  • Giving feedback 10.9
  • Receiving feedback 26.6
  • Giving and receiving feedback 54.7
  • Neither giving or receiving 7.8

48
Results 2
Did you modify your initial submission as a
result of the peer review activity?
Yes, as a result of the peer review given 23.4
Yes, as a result of the peer review received 25.0
Yes as a result of the peer review given AND received 28.1
No 21.9
N/A 1.6
49
Results student comments
  • If yes, please give specific examples of
    modifications (n41)
  • I added a couple of paragraphs and improved
    existing paragraphs, this added two full A4 pages
    to my work
  • I included specific materials as changed the
    formatting of the document so it looked more
    professional
  • I provided more specific numeric values and
    expanded my rationale after seeing someone elses
    PDS and after receiving feedback

50
Results RECEIVING feedback
  • Please give examples of what you learned from
    RECEIVING peer reviews from other students (n54)
  • Parts that I had previously missed were brought
    to eye such as market competition (noticing)
  • Receiving peer reviews gave me insight into what
    others thought of my work and gave me a direction
    to improve (reader response)
  • Where the PDS was confusing to understand (reader
    response)
  • I found out how good mine was (motivational)
  • The person who peer reviewed my PDS gave me
    positive feedback which helped me a lot
    (motivational)
  • Not much, they werent very good

51
Results PROVIDING feedback
  • Please give examples of what you learned from
    PROVIDING peer reviews of others work (n47)
  • When giving advice to people on theirs, it gave
    me greater perception when reviewing my own work
    by listening to my own advice for example
    transfer
  • I had a chance to see other peoples work and
    aspects of their work that I felt were lacking in
    my work, this helped me to improve my work
    transfer
  • I was given a greater understanding of the level
    of the work the course may be demanding
    standards
  • Allowed me to see from an assessors perspective
  • Thinking from a critical point of view critical
    judgement
  • How to look at work critically that isnt your
    own

52
Results How you carried out peer review
  • Could you make any comments about how you
    carried out the peer review? How did you evaluate
    the quality of the work to provide a response to
    the peer review questions? (n37)
  • I compared it to mine and the ideal PDS and said
    how I would improve it
  • Partly by comparing my work to theirs
  • I tried to think about what I wrote and whether
    this PDS was better or worse
  • Looked through the headings first and noted any
    that should have been included then read more in
    detail about each individually and compared it
    with my own PDS

53
Focus groups
  • How did you go about reviewing?
  • I read it through and compared it with what I
    had done to see if they had put something I had
    not done and then I added it in if they hadnt.
    The four questions were useful as they provided a
    framework for the review. If we hadnt had the
    questions it would have been difficult. I did the
    reviews separately and then answered one then the
    other. The first was a better standard than the
    other so I used the ideas from the better one
    to comment on the weaker one. I also read the
    guidelines in class when I did the peer review.
    There were ideas from the good one that I hadnt
    even thought of in mine

54
Focus groups
  • What do you think is best for learning giving or
    receiving feedback?
  • For me it would probably be to give feedback
    because I think seeing what other people have
    done is more helpful than getting other peoples
    comments on what you have already done. By
    looking at other peoples work you can see for
    yourself what you have forgotten or not even
    thought about. When people give feedback on yours
    they generally just talk about what is there.
    They dont say, well I did this on mine and you
    could put that in yours.

55
Results
Would you choose to participate in a peer review
exercise in the future?
  • Yes 76.6
  • No 3.1
  • Maybe 18.8
  • Dont know 1.6

56
Some tentative conclusions
  • Giving and receiving feedback are qualitatively
    different
  • Feedback receipt helps bring deficiencies in own
    work to students attention, can be motivational
    and gives a experience of different audiences
    (readers)
  • Feedback production appears to be better for
    developing judgement critical thinking, using
    criteria and standards, being more objective
    about own work
  • Fortunately receiving and giving usually occur in
    the same domain of assignment production hence
    double duty.
  • More research required
Write a Comment
User Comments (0)
About PowerShow.com