Dr' Yuyuan Zhao

1
A Scheme for Converting MCQ TestScores to
Percentage Marks

• Dr. Yuyuan Zhao
• Department of Engineering
• University of Liverpool

2
Outline

• Multiple Choice Questions (MCQ) Benefits and
  problems
• Conversion algorithm
• Conversion table
• Spreadsheet implementation
• Merits and applicability
• Points for attention
• Summary

3
Why MCQ?

• Easy to set up (?)
• Easy to Mark
• Quick to obtain results
• Suitable for selection processes
• Suitable for large classes
• Suitable for assessing knowledge, aptitude,
  competence and many skills
• Especially suitable for fact based subjects

4
Why is MCQ not widely used?

• Involves guesswork
• Not always suitable for assessment of certain
  skills, e.g.
• describing phenomena ? solving problems
• writing essays ? analysing data
• performing calculations
• Not suitable for big problems
• Lack of databank of questions
• Raw scores not compatible with conventional
  marking scheme

5
A Good MCQ Test

• Assesses most learning outcomes
• Has 4 choices of answer to each question
• Has equally feasible choices to a laymen for each
  question
• Has 20 or more questions
• the probability of obtaining a mark gt40 by pure
  guesswork in a test consisting of 18 four-choice
  questions is lt1.
• Raw scores are converted to marks that are a true
  measure of the students performance.

6
The Conversion Scheme

• The scheme is developed based on probabilistic
  analysis
• For each multiple-choice question,
• A full score is awarded if the choice of answer
  is correct
• A full mark is only awarded if the choice is NOT
  made by guesswork
• A partial mark is awarded if a student knows any
  of the wrong answers (firm answers)
• No mark is awarded for pure guesswork
• The scheme removes the guesswork element
• (YY Zhao, Int. J. Eng. Edu., 21(2005), 1189-1194)

7
Types of 4-Choice Questions
8
Score vs Mark
Each score S corresponds to a mark M. f
fraction of firm answers N number of choices
9
Conversion Table
For a full version of the conversion table see
Y Zhao, Int. J. Eng. Edu., 21(2005), 1189-1194
10
Implementation Using EXCEL

• Enter conversion table in columns A and B
• Enter student names and their raw scores in
  columns C and D
• Enter a formula using VLOOKUP function in column E

11
Merits of the Scheme

• Compatible with standard percentage marking
  scheme
• Independent of class size and historical data
• MCQ can be used in conjunction with other
  assessment methods
• Questions with different weightings acceptable
• Guesswork allowed but not rewarded

12
Reliability of Scheme

• 20 within 2 marks
• 50 within 5 marks
• 70 within 10 marks

The marks of a module assessed entirely by MCQ
tests correlated very well with the average marks
of all the modules in that semester for a class
of 170 students
13
Recommendations

• For summative assessment each question should
  have 4 or 5 choices.
• For formative assessment, 2 or 3-choice questions
  are acceptable.
• The number of questions in a MCQ test should be
  at least 10.
• Scores of individual assessment units should be
  converted and then averaged to give the overall
  mark.

14
Summary

• A scheme has been developed for converting raw
  MCQ scores to percentage marks by removing the
  guesswork element.
• The converted marks are a true measure of the
  students performance.
• The converted marks are compatible with the
  conventional marking scheme.
• The conversion scheme is independent of class
  size and historical data.
• The conversion scheme is easy to implement in a
  spreadsheet programme such as MS Excel.
• The conversion scheme has been proved to be
  reliable.