Designing Good Tests: Item Analysis is Part of the Equation

1
Designing Good Tests Item Analysis is Part of
the Equation!

• Molly Herman Baker, Ph.D.
• Black Hawk College

2
Planning Tests

• Determine amt of time, location, level of
  difficulty desired, resources available during
  test, and weight of test compared to total of
  assessments
• Based on objectives, determine what is important
  to know, not nice to know
• Based on amt of time, estimate number of
  questions possible (see Test Writing Basics
  handout for estimates)
• Prepare specifications chart

3
Specification Chart Basics

• Specifications chart
• Type and number of items for each content area or
  objective.
• Type and number of items for each type of HOT
  skill (HOTdiscipline-specific Bloom often
  selected, but others may be more relevant for
  your discipline)

4
Sample Specifications Chart 1
5
Sample Specifications Chart 2
6
Writing/Choosing Good Items

• Write good objectives, using verbs that indicate
  desired performance share with students
• Write/select test items based on these verbs
• See handout for samples
• See Test Writing Basics handout
• Discuss strategies for selecting test bank items

7
What about HOTS?

• HOTS (Higher order thinking skills)application,
  analysis, prediction, synthesis, evaluation or
  alternative taxonomy relevant to your discipline
• Important practice HOT in class (critical
  thinking, creative thinking, problem-based
  learning, case-based learning, scenario-based
  lng)
• Important provide guidance, practice feedback
  on preparing for and answering HOT exam questions

8
Item Analysis to Improve Tests

• correct and missed per item
• What does this tell you?
• who answered each MC distractor
• What does this tell you?
• Correlation between answers and overall test
  performance (Discrimination Index/Pt. Biserial)
• What could this tell you?
• Questions with poor item analysis score
• Discuss with students to determine item problems

9
QuickScore II Tips

• Item analysis software
• Use specific Scantron score sheet (101864-ERI-L)
• Options for output
• Save electronic analysis reports in PDF format
• Print reports