EPSY 546: LECTURE 1 SUMMARY - PowerPoint PPT Presentation

Loading...

PPT – EPSY 546: LECTURE 1 SUMMARY PowerPoint presentation | free to download - id: 4167cc-M2NkY



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

EPSY 546: LECTURE 1 SUMMARY

Description:

... and item difficulty (item score) IRF: Non-decreasing and parallel (non-intersecting) over Models: Rasch Model, ADISOP RASCH-1PL: ... – PowerPoint PPT presentation

Number of Views:5
Avg rating:3.0/5.0
Slides: 73
Provided by: GeorgeKar
Learn more at: http://www.uic.edu
Category:
Tags: epsy | lecture | summary | model | rasch

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: EPSY 546: LECTURE 1 SUMMARY


1
EPSY 546 LECTURE 1SUMMARY
  • George Karabatsos

2
REVIEW
3
REVIEW
  • Test ( types of tests)

4
REVIEW
  • Test ( types of tests)
  • Item response scoring paradigms

5
REVIEW
  • Test ( types of tests)
  • Item response scoring paradigms
  • Data paradigm of test theory (typical)

6
DATA PARADIGM

7
REVIEW Latent Trait
  • Latent Trait ? ? Re (unidimensional)

8
REVIEW Latent Trait
  • Latent Trait ? ? Re (unidimensional)
  • Real Examples of Latent Traits

9
REVIEW IRF
  • Item Response Function (IRF)

10
REVIEW IRF
  • Item Response Function (IRF)
  • Represents different theories
  • about latent traits.

11
REVIEW IRF
  • Item Response Function (IRF)
  • Dichotomous response
  • Pj(?) PrXj 1
  • PrCorrect Response to item j ?

12
REVIEW IRF
  • Item Response Function (IRF)
  • Polychotomous response
  • Pjk(?) PrXj gt k ?
  • PrExceed category k of item j
    ?

13
REVIEW IRF
  • Item Response Function (IRF)
  • Dichotomous or Polychotomous response
  • Ej(?) Expected Rating for item j ?
  • 0 lt
    Ej(?) lt K

14
IRF Dichotomous items
15
IRF Polychotomous items
16
REVIEW SCALES
  • The unweighted total score Xn stochastically
    orders the latent trait ?
  • (Hyunh, 1994 Grayson, 1988)

17
REVIEW SCALES
  • 4 Scales of Measurement
  • Conjoint Measurement

18
REVIEW
  • Conjoint Measurement
  • Row Independence Axiom

19
REVIEW
  • Conjoint Measurement
  • Row Independence Axiom
  • Property Ordinal Scaling and unidimensionality
    of ? (test score)

20
INDEPENDENCE AXIOM (row)
21
REVIEW
  • Conjoint Measurement
  • Row Independence Axiom
  • Property Ordinal Scaling and unidimensionality
    of ? (test score)
  • IRF Non-decreasing over ?

22
REVIEW
  • Conjoint Measurement
  • Row Independence Axiom
  • Property Ordinal Scaling and unidimensionality
    of ? (test score)
  • IRF Non-decreasing over ?
  • Models MH, 2PL, 3PL, 4PL, True Score, Factor
    Analysis

23
2PL
24
3PL
25
4PL
26
Monotone Homogeneity (MH)
27
REVIEW
  • Conjoint Measurement
  • Column Independence Axiom (adding)

28
REVIEW
  • Conjoint Measurement
  • Column Independence Axiom (adding)
  • Property Ordinal Scaling and unidimensionality
    of both ? (test score) and item difficulty (item
    score)

29
INDEPENDENCE AXIOM (column)
30
REVIEW
  • Conjoint Measurement
  • Column Independence Axiom (adding)
  • Property Ordinal Scaling and unidimensionality
    of both ? (test score) and item difficulty (item
    score)
  • IRF Non-decreasing and non-intersecting over ?

31
REVIEW
  • Conjoint Measurement
  • Column Independence Axiom (adding)
  • Property Ordinal Scaling and unidimensionality
    of both ? (test score) and item difficulty (item
    score)
  • IRF Non-decreasing and non-intersecting over ?
  • Models DM, ISOP

32
DM/ISOP (Scheiblechner 1995)
33
REVIEW
  • Conjoint Measurement
  • Thomsen Condition (adding)

34
REVIEW
  • Conjoint Measurement
  • Thomsen Condition (adding)
  • Property Interval Scaling and unidimensionality
    of both ? (test score) and item difficulty (item
    score)

35
Thomsen condition(e.g.,double cancellation)
36
REVIEW
  • Conjoint Measurement
  • Thomsen Condition (adding)
  • Property Interval Scaling and unidimensionality
    of both ? (test score) and item difficulty (item
    score)
  • IRF Non-decreasing and parallel
    (non-intersecting) over ?

37
REVIEW
  • Conjoint Measurement
  • Thomsen Condition (adding)
  • Property Interval Scaling and unidimensionality
    of both ? (test score) and item difficulty (item
    score)
  • IRF Non-decreasing and parallel
    (non-intersecting) over ?
  • Models Rasch Model, ADISOP

38
RASCH-1PL
39
REVIEW
  • 5 Challenges of Latent Trait Measurement

40
REVIEW
  • 5 Challenges of Latent Trait Measurement
  • Test Theory attempts to address these challenges

41
REVIEW
  • Test Construction (10 Steps)

42
REVIEW
  • Test Construction (10 Steps)
  • Basic Statistics of Test Theory

43
REVIEW
  • Total Test Score (X) variance
  • SumItem Variances
  • SumItem Covariances

44
EPSY 546 LECTURE 2TRUE SCORE TEST THEORY AND
RELIABILITY
  • George Karabatsos

45
TRUE SCORE MODEL
  • Theory Test score is a random variable.
  • Xn Observed Test Score of person n,
  • Tn True Test Score (unknown)
  • en Random Error (unknown)

46
TRUE SCORE MODEL
  • The Observed person test score Xn is a random
  • variable (according to some distribution) with
    mean Tn E(Xn) and variance ?2(Xn) ?2(en).

47
TRUE SCORE MODEL
  • The Observed person test score Xn is a random
  • variable (according to some distribution) with
    mean Tn E(Xn) and variance ?2(Xn) ?2(en).
  • Random Error en Xn Tn
  • is distributed with
  • mean E(en) E(XnTn) 0,
  • and variance ?2(en) ?2(Xn) .

48
TRUE SCORE MODEL
  • True Score
  • Tn true score of person n
  • E (Xn) expected score of person n
  • s Possible score s ? 0,1,,s,,S
  • pns PrPerson n has test score s

49
TRUE SCORE MODEL
  • 3 Assumptions
  • Over the population of examinees, error has a
    mean of 0. Ee 0
  • Over the population of examinees, true scores and
    error scores have 0 correlation.
  • ?T, e 0

50
TRUE SCORE MODEL
  • 3 Assumptions
  • For a set of persons, the correlations of the
    error scores between two testings is zero.
    ?e1, e2 0
  • Two testings when a set of persons take two
    separate tests, or complete two testing occasions
    with the same form.
  • The two sets of person scores are assumed to be
    randomly chosen from two independent
    distributions of possible observed scores.

51
TRUE SCORE ESTIMATION
52
TRUE SCORE ESTIMATION
53
TRUE SCORE ESTIMATION
54
TRUE SCORE ESTIMATION

is test reliability. The proportion of
variance of observed scores that is explained by
the variance of the true scores.
55
TEST RELIABILITY
is the error of
measurement.
56
TEST RELIABILITY
is the standard error of

measurement. (random error)
57
TEST RELIABILITY
is the standard error of

measurement. (random
error) Estimated ((1?)100) confidence
interval around the test score
58
TEST RELIABILITY
  • It is desirable for a test to be Reliable.

59
TEST RELIABILITY
  • Reliability the degree to which the
    respondents test scores are consistent over
    repeated administrations of the same test.

60
TEST RELIABILITY
  • Reliability the degree to which the
    respondents test scores are consistent over
    repeated administrations of the same test.
  • Indicates the precision of a set of test scores
    in the sample.

61
TEST RELIABILITY
  • Reliability the degree to which the
    respondents test scores are consistent over
    repeated administrations of the same test.
  • Indicates the precision of a set of test scores
    in the sample.
  • Random and systematic error can affect the
    reliability of a test.

62
TEST RELIABILITY
  • Reliability the degree to which the
    respondents test scores are consistent over
    repeated administrations of the same test.
  • Test developers have a responsibility to
    demonstrate the reliability of scores obtained
    from their tests.

63
ESTIMATING RELIABILITY
Estimated item variance
Estimated total test score variance
64
ESTIMATING RELIABILITY
Estimated covariance between
items i and j Estimated total
test score variance
65
OTHER FORMS OF RELIABILITY
  • Test-Retest Reliability
  • The correlation between persons test scores
    over two administrations of the same test.

66
OTHER FORMS OF RELIABILITY
  • Split-Half Reliability
  • (using Spearman-Brown correction for test
    length)
  • ?AB Correlation between scores of Test A
  • and Test B

67
TEST VALIDITY
  • VALIDITY A test is valid if it measures what it
    claims to measure.
  • Types Face, Content, Concurrent, Predictive,
    Construct.

68
TEST VALIDITY
  • Face validity When the test items appear to
    measure what the test claims to measure.
  • Content Validity When the content of the test
    items, according to experts, adequately represent
    the latent trait that the test intends to
    measure.

69
TEST VALIDITY
  • Concurrent validity When the test, measuring a
    particular latent trait, correlates highly with
    another test that measures the same trait.
  • Predictive validity When the scores of the test
    predict some meaningful criterion.

70
TEST VALIDITY
  • Construct validity A test has construct
    validity when the results of using the test fit
    hypotheses concerning the nature of the latent
    trait. The higher the fit, the higher the
    construct validity.

71
RELIABILITY VALIDITY
  • Up to a point, reliability and validity increase
    together, but then any further increase in
    reliability (over .96) decreases validity.
  • For e.g., when there is perfect reliability
    (perfect correlations between items), the test
    items are essentially paraphrases of each other.

72
RELIABILITY VALIDITY
If the reliability of the items were increased
to unity, all correlations between items would
also become unity, and a person passing one item
would pass all items and and another failing one
item would fail all the other items. Thus all the
possible scores would be a perfect score of one
or zeroIs the dichotomy of scores the best that
would be expected for items with equal
difficulty? (Tucker, 1946, on the
attenuation paradox)
(see also Loevinger, 1954)
About PowerShow.com