The use of CAA for formative and summative assessment: student views and outcomes.

1
The use of CAA for formative and summative
assessment student views and outcomes.
Sandra Pattinson Animal Production and Science
Group Harper Adams University College Newport Shro
pshire TF10 8NB. e-mail sepattinson_at_harper-adams.
ac.uk
2
1.0 Introduction
1.1 Farm Animal Production
Compulsory core module (Level 1).
Non-agricultural students - high failure rate.
3
1.0 Introduction
1.2 Aim of study
To examine the effects of the introduction of
formative CAA on the results for the module.
The module is taught over two semesters and
results are presented for the first semester only.
4
2.0 Methods
26 students on Farm Animal Production module.
2.1 Odd semester
Subjects taught beef, sheep, poultry.
Methods lectures, tutorials, farm classes.
Optional self-assessment tutorials
using QuestionMark Perception.
5
2.0 Methods
2.2 Self-assessment tests
Each test consisted of 10 multiple-choice
questions selected at random from database.
Size of database Beef production 67
questions Sheep production 58 questions
Feedback provided for each question.
6
2.0 Methods
2.3 Summative assessment
29 sheep production questions 29 beef production
questions 40 poultry production questions 2
general questions
Questions not randomly selected.
N.B. Negative marking used in tutorials and in
summative examination
7
2.0 Methods
2.4 Questionnaire
Presented to students at the start of the
subsequent semester (prior to publication of exam
marks).
Aim - to assess the students views on the use of
the self-assessment tutorials.
8
3.0 Results
3.1 Examination results
2003 - 04 (tutorials available) mean mark 41.5
pass rate 46
2002 - 03 (tutorials not available) mean
mark 34.8 pass rate 25
9
3.0 Results
3.1 Examination results
Use of tutorials Number of Mean
students mark 0 times 10 (38) 32.3 1 -
9 times 9 (35) 37.6 10 times 7
(27) 59.6
10
3.0 Results
3.1 Examination results
Correlations Use of tutorials v. exam
mark Sheep rs 0.59 P lt 0.01 Beef rs 0.69 P
lt 0.01
Mark v. poultry mark Beef r 0.52 P lt
0.01 Sheep r 0.36 P lt 0.1
11
3.0 Results
3.2 Questionnaire
3.2.1 How often did you use the
self-assessment tests? Very often, often,
occasionally, rarely, never.
12
3.2.2 As a method of learning I enjoyed the tests
13
3.2.3 The wording of the questions was unclear
14
3.2.4 The assessment questions were closely
related to the content of the module
15
3.2.5 I used my lecture notes whilst
completing the self-assessment tests
16
3.2.6 The self-assessment tests helped me to
understand the lecture notes
17
3.2.7 The marks helped me to assess how I was
doing on the module
18
3.2.8 The feedback given if you get the question
wrong is useful
19
3.2.9 Doing the tests made it easier to study
for the exams
20
3.0 Results
3.2 Questionnaire
3.2.10 Free response
A) Why did you use/not use the self-assessment
tests
Not used Forgot Used To see how much they
knew To practice for exams
21
3.2.10 Free response
B) What were the three best things about
the self-assessment tests?
Easy to use Detailed answers (feedback) Similar
format to exam
C) Give three suggested improvements.
Remove negative marking Remote access Provide the
correct answer in the feedback More questions and
more feedback
22
3.2.10 Free response
D) Any other comments
They were a good way of learning and
very useful. It was very helpful. Thank you.
23
4.0 Conclusions
Students who sat optional practice tests had
higher marks in subsequent summative tests. This
is in agreement with other authors e.g. Sly
and Rennie (1999), Charman (1999)
Students who scored highest on the beef and sheep
sections of the summative assessment also had the
highest marks in the poultry section.
24
4.0 Conclusions
Questionnaire responses indicate that for most
students the tutorials provided a positive
learning experience.
Some students requested that the correct answer
should be provided if the question was answered
incorrectly Deep v. surface approach to learning
25
5.0 Future direction
5.1 Current semester Pig and dairy production
self-assessment tutorials Revision test at
start of lecture
5.2 Questionnaire
Questions relating to Feedback Test structure
26
Question The feedback should state which
was the correct answer.
27
5.0 Future direction
5.3 Question analysis - to identify problem
areas of the module or poorly written questions
5.4 Remote access