Assessment

1
Assessment FEEDBACK 2010

• HEA Economics Network
• University of Aberdeen Business School
  (Economics)
• WD McCausland and P Smith

2
Presenters
David
McCausland
Peter
Smith
3
Overview

• Assessment and feedback consistently highlighted
  as one facet of student satisfaction where we
  underperform
• NSS increasingly used as a metric for assessing
  quality of teaching provision in league tables
• Understanding of issue gives opportunity to
  develop own (rather than imposed) strategies for
  improvement

4
National Student Survey
5
NSS Survey Questions 2009

• Staff expertise and enthusiasm

Staff are good at explaining things
Staff have made the subject interesting
Staff are enthusiastic about what they are
teaching
The course is intellectually stimulating
6
NSS Survey Questions 2009

• Feedback and assessment

The criteria used in marking have been clear in
advance
Assessment arrangements and marking have been
fair
Feedback on my work has been prompt
I have received detailed comments on my work
Feedback on my work has helped me clarify things
I did not understand
7
NSS Survey Questions 2009

• Advising and timetabling

I have received sufficient advice and support
with my studies
I have been able to contact staff when I needed
to
Good advice was available when I needed to make
study choices
The timetable works efficiently as far as my
activities are concerned
Any changes in the course or teaching have been
communicated effectively
The course is well organised and is running
smoothly
8
NSS Survey Questions 2009

• Personal development

The course has helped me present myself with
confidence
My communication skills have improved
As a result of the course, I feel confident in
tackling unfamiliar problems
Overall, I am satisfied with the quality of the
course
9
Aberdeens NSS performance

• the top 20

1 University of the West of England, Bristol
2 University of Buckingham
3 University of Surrey
4 University of Greenwich
5 University of Kent
6 Coventry University
7 Swansea University
8 University of East Anglia
9 Aberystwyth University
10 University of Exeter
11 University of York
12 University of Birmingham
13 Oxford Brookes University
14 University of Leicester
15 University of Sheffield
16 University of Aberdeen
17 Nottingham Trent University
18 University of Strathclyde
19 Loughborough University
20 University of Ulster

• Overall ranking by rank sum

• Source Latreille (2010)

10
Aberdeens NSS performance Scotland

• 1 University of the West of England, Bristol 2
  University of Buckingham 3 University of Surrey 4
  University of Greenwich 5 University of Kent 6
  Coventry University 7 Swansea University 8
  University of East Anglia 9 Aberystwyth
  University 10 University of Exeter 11 University
  of York 12 University of Birmingham 13 Oxford
  Brookes University 14 University of Leicester 15
  University of Sheffield 16 University of Aberdeen
  17 Nottingham Trent University 18 University of
  Strathclyde 19 Loughborough University 20
  University of Ulster 21 University of Durham 22
  University of St Andrews 23 University of
  Bradford 24 University of Hertfordshire 25
  University of Liverpool 26 University of East
  London 27 University of Portsmouth 28 University
  of Newcastle upon Tyne 29 University of Glasgow
  30 University of Essex 31 University of Bath 32
  University of Sussex 33 University of Warwick 34
  University of Nottingham 34 University of
  Southampton 36 University of Northumbria at
  Newcastle 37 Cardiff University 38 University of
  Hull 39 University of Plymouth 40 Lancaster
  University 41 University College London 42
  University of Dundee 43 Royal Holloway,
  University of London 44 University of Reading 45
  Kingston University 46 London School of Economics
  and Political Science 47 School of Oriental and
  African Studies 48 University of Leeds 49
  University of Edinburgh 50 Goldsmiths College,
  University of London 51 London Metropolitan
  University 51 Queen Mary, University of London 53
  Queen's University Belfast 54 University of
  Bristol 55 Keele University 56 Heriot-Watt
  University 57 University of Manchester 58
  Middlesex University 59 City University, London
  60 Brunel University 61 University of Westminster

11
Aberdeens NSS performance leaders

• 1 University of the West of England, Bristol 2
  University of Buckingham 3 University of Surrey 4
  University of Greenwich 5 University of Kent 6
  Coventry University 7 Swansea University 8
  University of East Anglia 9 Aberystwyth
  University 10 University of Exeter 11 University
  of York 12 University of Birmingham 13 Oxford
  Brookes University 14 University of Leicester 15
  University of Sheffield 16 University of Aberdeen
  17 Nottingham Trent University 18 University of
  Strathclyde 19 Loughborough University 20
  University of Ulster 21 University of Durham 22
  University of St Andrews 23 University of
  Bradford 24 University of Hertfordshire 25
  University of Liverpool 26 University of East
  London 27 University of Portsmouth 28 University
  of Newcastle upon Tyne 29 University of Glasgow
  30 University of Essex 31 University of Bath 32
  University of Sussex 33 University of Warwick 34
  University of Nottingham 34 University of
  Southampton 36 University of Northumbria at
  Newcastle 37 Cardiff University 38 University of
  Hull 39 University of Plymouth 40 Lancaster
  University 41 University College London 42
  University of Dundee 43 Royal Holloway,
  University of London 44 University of Reading 45
  Kingston University 46 London School of Economics
  and Political Science 47 School of Oriental and
  African Studies 48 University of Leeds 49
  University of Edinburgh 50 Goldsmiths College,
  University of London 51 London Metropolitan
  University 51 Queen Mary, University of London 53
  Queen's University Belfast 54 University of
  Bristol 55 Keele University 56 Heriot-Watt
  University 57 University of Manchester 58
  Middlesex University 59 City University, London
  60 Brunel University 61 University of Westminster

12
Aberdeens NSS performance

• the top 20

• Section ranking by section rank sum / section

Section Description Aberdeens Rank
0 Overall 16
1 Staff expertise and enthusiasm 13
2 Feedback and assessment 10
3 Advising 12
4 Timetabling 30
6 Personal development 33

• Source Latreille (2010)

13
Aberdeens NSS performance

• the top 20

1 Staff are good at explaining things.
2 Staff have made the subject interesting.
3 Staff are enthusiastic about what they are teaching.
4 The course is intellectually stimulating.
5 The criteria used in marking have been clear in advance.
6 Assessment arrangements and marking have been fair.
7 Feedback on my work has been prompt.
8 I have received detailed comments on my work.
9 Feedback on my work has helped me clarify things I did not understand.
10 I have received sufficient advice and support with my studies.
11 I have been able to contact staff when I needed to.
12 Good advice was available when I needed to make study choices.
13 The timetable works efficiently as far as my activities are concerned.
14 Changes in the course or teaching have been communicated effectively
15 The course is well organised and is running smoothly.
19 The course has helped me present myself with confidence.
20 My communication skills have improved.
21 As a result of the course, I feel confident in tackling unfamiliar problems.
22 Overall, I am satisfied with the quality of the course.

• ranking by question

Aberdeen Rank / Question 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 19 20 21 22
rank average score 19 14 14 14 14 19 12 10 16 13 17 12 39 34 17 23 38 35 16
rank percentage agree 7 3 4 9 19 11 19 7 15 12 11 10 40 39 17 29 45 23 6

• Source Latreille (2010)

• 4 or 5 on Likert scale

14
Economics Network Survey
15
EN Questions

• 61 respondents
• 69 have A level economics
• 66 have A level mathematics
• 45 have English as first language
• Even mix of years 2/3/4

16
Aberdeens EN survey performance

• 17.o. Feedback on submitted work
• Very useful 27.9
• Useful 29.5
• Of some use 29.5
• Of little use 8.2
• N/A 4.9

• Very few assessments and very little individual
  feedback
• Assessments mainly longer essays, fewer tests
• Prefer more assessments throughout semester
  (quizzes not essays) rather than heavily weighted
  final exams

17
Focus Group
18
EN Focus Group

• Economics Network hosted
• March 1 2010
• 12 students
• Self-selected (response to advert)
• Mainly year 4 (with some 2nd and 3rd years)
• Mainly international

19
Focus Group Questions

• satisfaction with course
• aspects of the course feel happy with
• aspects of the course feel unhappy with or feel
  could be improved
• prior expectations of the course
• extent course met /differed from expectations
• assessment and feedback experienced
• usefulness of assessment and feedback passively
  received or acted upon support needed
• improvement of assessment and feedback procedures
• numeracy and literacy skills prior preparation
  and development placing of EC3001 Maths and
  Stats
• balance between modes of learning

20
Focus Groups Key Themes

• Assessment

I cant understand why there are unequal
proportions between coursework and the exam, e.g.
80 / 20, why not 50 / 50)

Micro 4 was (tests/quizzes) (that) counted (for)
50 of the total mark - and I think I understood
it far better than writing essays
you dont know what exactly is expected on the
exam essay
you only cover a fraction of what youve learnt
with three (exam) questions
presentations in tutorials should count at least
10
I (would prefer) to be forced to study on a
regular basis and have tests more often
they shouldnt mark our (corporate finance
assignments) based on what they think the average
grade should be at the end
21
Focus Groups Key Themes

• Feedback

I cant understand why you cant get any
feedback from the essays written in the exams. I
got three times in a row the same mark, and I
still dont know what Im doing wrong
(each lecturer should) explain how to do the
presentation (as) they (each) have different
requirements
department very helpful - they always found
time, and would explain very patiently
I think the problem is, and most teachers dont
tell you this, is that they have a lot of work -
they do research and stuff. So its a matter of
time constraint. We want more tutorials but
theres no one there to give (them), we want more
feedback ...
You dont get (proper) feedback from your
(coursework) essays
22
Focus Groups Key Themes

• Quantitative skills

The 3rd year maths was just ridiculous. I did
more difficult stuff at high school.
(from England) third year maths was a lot of new
stuff for me continuous assessment instead of
just an exam at the end helped me a lot, because
it made me study constantly
Maybe (maths) could be progressive (during)
first year and second year - rather than
everything in the third year
if they had an introductory maths course in the
first year then I would be fine
Id chosen economics because I thought Im good
in maths when I came here I saw no maths,
(only) literature and writing essays and I
thought this is not what I signed up for
23
Focus Groups Key Themes

• Literary Skills

the last essay I had to write was in primary
school about my favourite book
a lot of people, including Scottish people,
didnt know how to write essays, (so) maybe have
a course on essay writing at the very beginning
to give everyone an equal chance
its economics not English literature, so I
dont know why they expect us to write essays for
the exam. Why cant we just (do) calculations, or
short answers
I dont understand the principle of the essay
(but) they wont change them because they are
very proud of still having essays - in the US
everything is short questions, short answers
24
Focus Groups Key Themes

• Learning methods

the best tutorials were the ones where we had a
set of questions and you had to look at them at
home (and) come prepared

More computer assisted, multiple choice tests
could be more group work (in) geography we had
(a group) project doing research, collecting
data, and you had your own input and were marked
on your own contribution to the project
(enjoyed simulations in) macro second year - how
things apply to the real world
H asked us to read some articles before coming
to the (class, and then we) discussed some
questions to get if we understood the main
concept of the article or not. I found this very
helpful
in Poland I had eight or ten courses per
semester, and here I got just two assignments and
one report
programme is not challenging, or demanding as in
other countries
25
Assessment Feedback Strategy
26
Conclusions

• Your thoughts and feedback are appreciated!