Guidelines for preparing Multiple Choice Questions MCQs for PBL Tutors

1
Guidelines for preparing Multiple Choice
Questions (MCQs) for PBL Tutors

• Student Assessment Committee
• Undergraduate Program

2
Why write a Question?

• Recognizes a role for tutors
• Recognizes the importance of Assessments
• Focuses tutors on the Learning Objectives
• Emphasizes application of knowledge and the
  Problem Solving Process
• Contribution to the Item Bank is documented for
  inclusion in Teaching Dossier
• Contributors names are submitted for a Prize
  draw

3
How?

• References available
• Case and Swanson, NBME Constructing Questions
  for the Basic and Clinical Sciences
• www.nbme.org publications, item writing manual
• Royal College of Physicians and Surgeons of
  Canada website
• rcpsc.medical.org- publications and
  documents-material for examiners

4
How?

• One-Best Answer Item
• Composed of a stem and generally 5
  distractors, one of which is the correct
  response
• Acute intermittent porphyria is the result of a
  defect in the biosynthetic pathway for
• A. collagen
• B. corticosteroid
• C. fatty acid
• D. glucose
• E. heme

5
How to do it better?

• A better question is one that uses a clinical
  case vignette as the stem
• An otherwise healthy 33 year-old man has mild
  weakness and occasional episodes of steady,
  severe abdominal pain with some cramping and no
  diarrhea. One aunt and a cousin have had similar
  episodes. During an episode, his abdomen is
  distended, and bowel sounds are decreased.
  Neurological examination shows mild weakness in
  the upper arms. These findings suggest a defect
  in the biosynthetic pathway for
• A. collagen
• B. corticosteroid
• C. fatty acid
• D. glucose
• E. heme

6
Good MCQs

• Assess application of knowledge
• Present a homogeneous set of options

7
The Basic Rules for One-Best-Answer Items

• 1. Each item should focus on an important concept
• Common, serious or potentially catastrophic
  clinical problem
• Avoid
• Tricky
• Trivial
• Overly complex

Case and Swanson, 2001
8
The Basic Rules for One-Best-Answer Items

• 2. Each item should assess the application of
  knowledge NOT recall of an isolated fact
• Clinical case vignettes provide a good basis for
  a question
• Stems may be longer
• Options should be short

Case and Swanson, 2001
9
Application of knowledge vs. Isolated Fact

• What area is supplied with blood by the posterior
  inferior cerebellar artery?
• A 63-year old man develops left-sided limb
  ataxia, Horners syndrome, nystagmus, and loss of
  appreciation of facial pain and temperature
  sensations. What artery is most likely to be
  occluded?

Case and Swanson, 2001
10
The Basic Rules for One-Best-Answer Items

• 3. The stem of the item must pose a clear
  question
• It should be possible to arrive at an answer with
  the options covered

Case and Swanson, 2001
11
The Basic Rules for One-Best-Answer Items

• 4. All distractors (incorrect options) should be
  homogeneous
• Correct and incorrect options should fall into
  the same category (all diagnoses, tests,
  treatments, etc)

Case and Swanson, 2001
12
Distractors

• Should be
• Plausible
• Grammatically consistent
• Logically compatible
• Same relative length as the correct answer
• Order the options in logical order (numeric) or
  alphabetical

Case and Swanson, 2001
13
The Basic Rules for One-Best-Answer Items

• 5. Avoid technical item flaws that provide
  special benefit to test-wise examinees or that
  pose irrelevant difficulty

Case and Swanson, 2001
14
ExampleTechnical Item Flaws

• In patients with advance dementia, Alzheimers
  type, the memory defect
• A. can be treated adequately with
  phosphatidylcholine (lecithin)
• B. could be a sequela of early parkinsonism
• C. is never seen in patients with neurofibrillary
  tangles at autopsy
• D. is never severe
• E. possibly involves the cholinergic system

15
Possible changes

• In patients with advance dementia, Alzheimers
  type, the memory defect can be effectively
  treated with
• A. cholinergic agonists
• B. dopaminergic agonists
• C. GABA agonists
• D. nicotinic agonists
• E. serotonergic agonists

16
The Basic Rules for One-Best-Answer Items

• Subject each question you write to the above 5
  tests
• If a question passes all 5, it is probably
  well-written and focused

Case and Swanson, 2001
17
What?

• Each tutor is asked to construct at least two
  questions during the block.
• Each question will be based on one of the weekly
  case objectives

18
Where?

• Please e-mail or provide a paper copy of your
  questions prior noon of the last Friday of the
  block to
• this presenter
• jenas_at_interchange.ubc.ca
• fairholm_at_interchange.ubc.ca
• detches_at_cw.bc.ca
• eling_at_cw.bc.ca
• hnadel_at_cw.bc.ca

19
Final Check?

• Please remember to include
• The learning objective the question is testing
• The case scenario
• The answer to the question
• If you can only write 2 or only parts of the
  question - that is OK. Your questions will be
  reviewed with feedback to you before submission
  to the examination.