Reliability of Assessment

1
Reliability of Assessment

• Chapter 2
• from Popham (2005)
• Beth Christian,Ed.D.

2
Why do we need to know about reliability,
validity, and absence of bias?In other words,
why cant we just get straight into the practical
stuff making tests, rubrics, etc?
3
Quality Assessments

• In order to make the quality assessments
  discussed in the next 7 chapters, we need to
  understand how to determine that our tests are of
  good quality.that is, that they are
• RELIABLE Consistent
• VALID Accurate
• ABSENT of BIAS Fair

4
Reliability

• The extent to which tests are consistent
• In other words, it should not matter whether the
  student takes the test in the morning, afternoon,
  or one day or another, scores should be consistent

5
Three Reliability Measures

• Stability or Test-Retest-(same test twice at
  different timescorrelation between scores and
  time 1 and time 2)
• Alternate Form- (two forms of same
  test)correlation between scores of Test 1 and
  Test 2)
• Internal Consistency-comparison of items on a
  testcompare one half of the test to the other
  half or use (Kuder-Richardson Formula 20 or
  Cronbachs Alpha).

6
Stability Reliability

• Consistent test results over time
• Similar results on different occasions
• No significant events that might alter
  performance on second occasion between testing
  occasions
• Correlation coefficient reflects the degree of
  similarity between two tests

7
Correlation Coefficient r

• Reliabilty coefficients range from 0 to 1.0.
• r0no reliability
• r1perfect reliability
• Ex. r Above .80 usually good
• Ex. r Below .50 usually poor

8
Alternate Form

• The consistency of challenge inherent in two of
  more supposedly equivalent forms of the same
  examination
• Two tests given to same individuals with little
  or no delay b/w testing occasions
• Results given as correlation coefficient

9
Internal Consistency Reliability

• The extent to which the items in an educational
  assessment instrument are functioning a
  consistent fashion
• Can be computed based in single test
  administration
• Ex. If a student answers an item on a test
  correctly, he or she will answer a similar item
  correctly

10
Formulae for computing internal consistency
(Terminology)

• Kuder-Richardson (K-R Formula)used for
  right/wrong answers such as multiple choice
• Cronbach Coefficient Alpha- for items in which
  students are given points such as essay questions
• Dichotomous itemsright/wrong
• Polytomous multiples responses/scores

11
Classification Consistency

• Classification consistency reliability is, as the
  label suggests, a representation of the
  proportion of students who are classified
  identically on two different test forms or two
  different administrations of the same test.

12
Standard Error of Measurement

• The consistency of particular persons
  performance
• The consistency of a persons performance if an
  assessment procedure was administered again,
  again, and again
• The higher the reliability, the smaller the
  standard error of measurement will be
• Reminds teachers about the imprecision of the
  test scores that an individual receives-the
  scores on commercial of classroom tests are not
  all that exact

13
What do classroom teachers really need to know
about reliability?

• Popham (2005) says not all that much.
• Teachers do not need to compute reliability
  coefficients for the own classroom tests.
• Teachers do need to understand the imprecision of
  educational measurement