Difficulty and Discriminability of Introductory Psychology Test Items

1
Difficulty and Discriminability of Introductory
Psychology Test Items

• Charles Scialfa, Connie Legare,
• Larry Wenger,
• University of Calgary
• Louis Dingley
• Keyano College

2
Acknowledgements

• Drs. Scott Oddie, Jen Thurlow, Cindy Lahar,
  Suzanne Hala, John Mueller, Elzbieta Slawinski,
  Susan Graham and Lenora Brown, who provided data
  for these analyses.
• The Province of Alberta Learning Enhancement
  Envelope.

3
Introduction and Rationale

• Assessing difficulty in the first-course in
  psychology?
• Sanford (1910) - students not prepared in
  physics, physiology, introspection.
• Wolfle (1942) - equal in difficulty to other
  university courses.
• Altman and Hartman (1936) -easy for pre-medical
  students hard for education students and
  engineers.

4
Introduction and Rationale

• Assessing difficulty in the first-course in
  psychology?
• Examination of semantic complexity of texts
  (Whissel, 1997).
• Subjective difficulty estimates from students
  (Laffitte, 1984,1986).
• Ordinally-scaled evaluations of texts (Jackson,
  et al., 1999).

5
Introduction and Rationale

• Assessing difficulty in the first-course in
  psychology?
• Psychometric approach involving item analysis.
• Specifically, examining difficulty and
  discriminability of test items.

6
Introduction and Rationale

• Examining difficulty and discriminability of test
  items.
• Easily done with optically-scored,
  multiple-choice exams
• Only infrequently reported in test banks (Kalat
  Stonebraker, 1996)

7
Present Study

• Purpose
• To examine difficulty and discriminability in a
  large sample of test items.
• To determine if these indices varied as a
  function of topic (e.g., Sensation and
  Perception, Developmental).
• To discuss implications for teaching.

8
Method

• Approximately 4000 items
• Texts used were
• Atkinson, et al. (1996), Baron, et al. (1998),
  Carlson Buskist (1997, Kalat (1996, Morris
  (1996), Sternberg (1995, 1998).
• Text range in estimated difficulty (Jackson, et
  al., 1999) from moderate to high.
• University and college, one- and two-semester.
• Nine different instructors.

9
Difficulty and Discriminability by Topic
10
Difficulty and Discriminability by Topic
11
(No Transcript)
12
(No Transcript)
13
Discussion

• Items generally too easy from purely psychometric
  perspective.
• As a consequence, they fail to discriminate.
• Little difference across topics.
• No substantive relation to textbook difficulty,
  institution, course format.
• Data agree with other similar studies.
• Need to get subjective estimates.