Which General Chemistry Questions Discriminate Between A, B, C, D and F Students

1
Which General Chemistry Questions Discriminate
Between A, B, C, D and F Students ?

• Kimberly D. Schurmeier
• And
• Charles H. Atwood

2
Which General Chemistry Questions Discriminate
Between A, B, C, D and F Students ?

• JExam program
• IRT analysis
• First Exam Fall 2005
• Second Exam Fall 2005
• Conclusions

3
Which General Chemistry Questions Discriminate
the Best Between Students

• JExam
• Computerized exam
• Students given 80 min.
• Three tries per question
• Multiple forms of each question are given
  randomly
• Item Response Theory (IRT)
• Bilog-MG IRT program
• 2-parameter logistic model
• Only first try was analyzed for this
  presentation

4
Item Characteristic Curve

• Ideal curve that highly discriminates at an
  ability of 0.5

5
Item Characteristic Curve

• Ideal Characteristic Curve that Discriminates
  between the A and B students

6
Item Characteristic Curve

• This curve shows that it can not discriminate
  between students

7
JExam item 11192Exam 1 Fall 2005
Question will discriminate between D and F
students
8
JExam item 7083
Will Discriminate between D and F students
9
JExam Item 3739
Discriminates Best Between C and D Students
10
JExam Item 9019
Will Discriminate C and D students
11
JExam Item 11417
Will Discriminate Between C and D students
12
JExam item 7144
Separates B and C students
13
Item 11237
Discriminates best between B and C students
14
JExam Item 11416
Discriminates Best Between A and B Students
15
JExam Item 11203
Item Too Difficult for Students
16
Comparing Items 3739 and 11237

• A C-student will most likely know that boiling
  chlorine is not a chemical property, but they
  will also most likely believe that the molecular
  make-up will alter when a compound boils.
• Is this the disconnect?

17
Comparing Items 7083 and 11203

• When calculating specific heat, the question will
  discriminate between D and F students.
• When asked what has the smallest specific heat,
  it is too difficult even if it is a 50/50 chance.
• The students do not understand specific heat.

18
Item 145Exam 2 Fall 2005
Will Discriminate Between D and F Students
19
Item 11233
Separates C and D Students
20
Item 11448
Discriminates between C and D students
21
Item 9191
Discriminates Between B and C Students
22
Item 9166
Distinguishes Between B and C Students
23
Item 9262
Separates B and C Students
24
Item 11449
Separates A and B Students
25
Information Received from Exam 2 in Fall 2004
26
Information Received from Exam 2 in Fall 2005
27
Where we are headed

• Examining the 2nd and 3rd try to see if the
  students learn any chemistry while taking the
  exam
• Running a logistic 3-parameter model on multiple
  choice items
• Fixing non-equivalent items, to give all students
  an equally difficult exam.
• Writing new questions to help discriminate
  between A and B students
• Helping students get past the disconnect

28
Kudos

• Thanks to
• Jake Martin for IRT output from JExam
• Gary Lautenschlager for help with IRT analysis
• PRISM and NSF for funding