RELIABILITY and VALIDITY of MEASURES

1
RELIABILITY andVALIDITYofMEASURES
2
Psychometric Properties

• VALIDITY
• extent to which test measures what it is
  purported to measure.
• Is the test valid for the purposes for which I
  want to use it?

• RELIABILITY
• consistency or stability of a test

3
Validity (1)

• CONTENT the degree to which the sample of test
  items represents the content that the test is
  designed to measure.
• FACE do the items appear to deal with relevant
  content? Does the measure seem like a reasonable
  way to gain the information the researchers are
  attempting to obtain? Does it seem well designed?
  
• PREDICTIVE the degree to which the predictions
  made by a test are confirmed by later behavior of
  the test-takers.

4
Validity (2)

• CONCURRENT determined by relating test scores of
  a group of test-takers to a criterion measure
  administered at same time.
• CONCURRENT PREDICTIVE
• CRITERION-RELATED
• CONSTRUCT VALIDITY extent to which a particular
  test can be shown to measure a hypothetical
  construct.

5
Reliability (1)

• the level of internal consistency or stability of
  a test over time.
• Reliability coefficients range from .00 to 1.00
  (.00 indicates no reliability)

6
Reliability (2)

• ALTERNATE FORM give two parallel forms of the
  test (Form A, Form B) to same group of
  test-takers and correlate their scores. (Widely
  used for achievement tests.)
• TEST-RETEST same test given again to same group
  of test-takers after some time interval (eg, 6
  weeks) and T1 and T2 tests are correlated.

7
Reliability (3)

• INTERNAL CONSISTENCY METHODS
• (a) SPLIT-HALF correlates odd-numbered and
  even-numbered items.
• Correction factor Spearman-Brown formula
• (b) RATIONAL EQUIVALENCE analyzes individual
  test items.
• Kuder-Richardson formulas are used
• KR-20 (better method)
• KR-21 (easier to compute!)
• (c) CRONBACHS ALPHA dichotomous items.

8
Relationship Between Reliability and Validity

• A test can be reliable but not valid.
• Validity is more important than reliability.
• To be useful, a measuring instrument (test,
  scale) must be both reasonably reliable and valid.

9
IF THEN

• Unreliable
• Reliable, but not valid
• Unreliable and invalid
• Reliable and valid

• Test validity is undermined.
• Test is not useful.
• Test is NOT useful!
• Test can be used with good results.

10
Scales of Measurement

• Nominal scale
• Ordinal scale
• Interval scale
• Ratio scale

• Categorical data
• RankingS
• Ranking with intervals
• Intervals with zero

11
Examples of Nominal Data

• What is your year in school?
• Freshman
• Sophomore
• Junior
• Senior
• Graduate

• What is your gender?
• Male
• Female
• What is your marital status?
• Single
• Married
• Divorced

12
Nominal scale

• Classifies data according to a category only.
• EX No color has more or less value than any
  other. Numbers assigned to colors have no value.

13
Example of Ordinal Data

• The Billbored Hot 100 CD Sales!
• (1) Frank Sinatra..1,000,000 last week
• (2) Jimmy Buffett.. 900,000 last week
• (3) Yanni... 750,000 last week
• (4) Britney Spears. 150,000 last week
• (5) Wilco 50,000 last week
• (6) Ani DiFranco 25,000 last week

14
Example of Ordinal Data

• Students Test Score Rankings
• Marge Innaverra 100
• Slim Tonun 98
• Natalie Drest 93
• Paige Turner 86
• Walt N. Matilda 85
• Stew Dent 75
• Wade A. Wile 74

15
Ordinal scale

• Classifies data according to rank.
• EX Rate three types of educational innovations
  according to how hot or timely each is.
• Difference between points on scale not equal.

16
Example of Interval Data

• Educational research is my favorite class
• __(4) Strongly agree
• __(3) Agree
• __(2) Disagree
• __(1) Strongly disagree

17
Interval scale

• Assumes the measurements are in equal units.
• Does not have true zero point (it is arbitrary).

18
Examples of Ratio Data

• Height
• Weight
• Income
• (has all the properties of an interval scale, and
  also has a zero point).

19
Ratio scale

• Similar to interval scales, but has a true zero
  point.