Validity and Reliability

1
Validity and Reliability

• EAF 410
• July 9, 2001

2
Validity

• Degree to which evidence supports inferences made
• Appropriate
• Meaningful
• Useful

3
Appropriate Inference

• Relevant to the study

4
Meaningful Inference

• Says something about the meaning of the
  information obtained through an instrument

5
Useful Inference

• Helps researchers make decisions related to what
  they studied

6
Do we get useful information about the
topic/variables?
7
Validity depends on

• The amount and type of evidence supporting the
  interpretations

8
Types of Evidence

• Content-related
• Criterion-related
• Construct-related

9
Content-related

• Appropriate?
• Comprehensive?
• Logical?
• Adequate?

10

• Adequacy of the sampling
• representative of content?
• Format of the instrument
• language, directions, etc.
• Who judges the validity of the content?
• Interpretation
• judgement

11
Criterion-related

• Relationship between scores from different
  instruments
• Strong relationship?
• Accurate?

12

• Instrument results compared with independent
  criterion
• a second test or measurement
• Predictive validity
• time between measurements
• Concurrent validity
• instrument and criterion measurements at same time

13

• Validity Coefficient
• r correlation coefficient
• degree of relationship between scores from
  instrument and criterion
• Between -1.00 and 1.00
• .00 no relationship

14
Construct-related

• How well does the theoretical construct account
  for or explain differences?
• Is the construct sufficient?

15

• Three steps
• variable measured clearly defined
• hypotheses (based on theory)
• hypotheses tested logically and empirically
• Collect variety of different types of evidence

16
Reliability

• Consistency of the scores
• among individuals
• among items

17

• Reliable -- valid or not valid
• Not reliable -- not valid
• Validity and reliability dependent on the context
  in which the instrument is used.

18
Measurement Error

• Same test, different scores
• Caused by variety of factors
• motivation
• anxiety
• environment
• etc.

19
Reliability Coefficient

• Reliability estimates
• Relationship between scores at two different
  times, or
• Relationship between two parts of the same
  instrument

20
Test-Retest

• Same test, same group, two times
• Reliability coefficient indicates relationship
  between the two scores

21
Equivalent-Forms

• Two different, equivalent tests, same group
• Reliability coefficient calculated

22
Internal-Consistency

• Single administration of the instrument
• Split-half procedure
• Kuder-Richardson approach
• Alpha coefficient

23
Split-Half Procedure

• Score two halves of test separately
• Degree to which the two halves provide same
  results
• Reliability increases with length of the
  test/instrument

24
Standard Error of Measurement

• Index showing how much a measurement would vary
  under changed circumstances