Lecture 3: Reliability and Validity

1
Lecture 3Reliability and Validity

• Reliability
• Validity

2
Question How reliable?

• Measuring something in a consistent and stable
  manner
• A measure is reliable if it consistently gives
  the same answer
• X T E.
• - X Total score (Observed measurement
  score)
• T True score Systemic score Reliable
• E Random error
• Reliability The ratio of T to X
• 0 lt Reliability coefficient lt 1.0

3

• Reliability consists of 3 different dimensions
• - stability the consistency of a result or of a
  measure at different points in time.
• Ex) test-retest
• internal consistency the consistency of
  performance among the items that compose a scale
• Ex) split-half technique, Cronbachs alpha
• equivalency the consistency between two parallel
  forms of a test
• Ex) intercoder reliability

4
Question How valid?

• Measuring what researchers intend to measure
• Measurement Equations (V Valid I Invalid).
• X T E.
• T V I.
• X V I E.

5
Types of Validity

• Judgment-based Face validity (whether it
  measures what it appears to measure -gt very
  subjective, not very informative)
• Criterion-based
• - Predictive validity (measurement instrument
  against some future outcome)
• - Concurrent validity (measurement instrument
  against some present criterion)
• Theory-based Construct validity (whether the
  measurement is logically related to other
  concepts in the framework)

6

• Internal Validity
• Threats to internal validity (i.e., artifacts)
  history, maturation, testing, instrumentation,
  statistical regression, experimental
  morality(text, pp. 27-32)
• External Validity how well the results of a
  study can be generalized across populations,
  settings, and time
• A study that lacks external validity cannot be
  projected to other situations it is valid only
  for the sample tested.

7
Relationship between Reliability and Validity

• One can assume reliability if validity can be
  established.
• If a measurement is unreliable, the measurement
  cannot be valid.
• Reliability is necessary but not sufficient for
  validity.