Using Item Response Theory Methods to Shorten an Impulsiveness Scale

1
Using Item Response Theory Methods to Shorten an
Impulsiveness Scale

• Siang Chee Chuah
• Brent Roberts

Society for Industrial and Organizational
Psychology, Orlando, April 11-April 13, 2003
2
In a perfect world .

• Unlimited amount of time with subject
• Administer as many questionnaires as we want
• Fatigue and Interest dont change

3
Shortening Scales

• Limited time with sample
• Competing for space with other variables
• Psychometric qualities of scale

4
Classical Test Theory Method

• Goal Maximize Reliability
• Items with highest item-total correlation
• Factor analysis (single factor)
• Distribution of scale scores (normal)
• Vs Goal Maximize criterion validity

5
IRT method

• Select appropriate IRT model and estimate
  parameters
• Examine fit of IRT model to data
• Selecting subset of items that show good model
  data fit and provide highest precision at the
  desired trait levels
• Test Information Function
• http//io.psych.uiuc.edu/irt

6
Test Information Function
7
Impulsiveness (Self-control)

• Lower-order factor of conscientiousness
• Foolhardy, haphazard, reckless, erratic
• Some disagreement where to place
  impulsiveness Extraversion (Goldberg,
  1999) Neuroticism (Costa McCrae, 1994)

8
Sample Measures

• N 1391 participants in this sample(Roberts et
  al., in review)
• Impulsiveness 20 Items5 Point Likert Scalea
  .9033
• Big5 Markers (Goldberg, 1992)

9
CTT Method

• Factor analysis clearly indicates single dominant
  factor
• Item-total correlation, means, standard deviation
• Item total is normally distributed
• Removed worst 4 items and recalculated item-total
  correlation

10
(No Transcript)
11
IRT Principles

• 2 Parameter Logistic Model
• Dichotomized data
• Unidimensionality
• Test fit of IRT Model
• Item Information Function
• http//io.psych.uiuc.edu/irt

12
20 Items
13
16 Items
14
12 Items
15
8 Items
16
Test Reliability
17
Convergent Discriminate Validity Big 5 Markers
18
Correlations
19
16 Items
20
12 Items
21
8 Items
22
4 Items
23
Learn New Tricks

• Error ? Information Function
• Reliability ? Accuracy