4632001 Tests and Measurements - PowerPoint PPT Presentation

About This Presentation
Title:

4632001 Tests and Measurements

Description:

Weights. Raw regression coefficients. Standardized regression coefficients ... Relatively free from error = reliable. Spearman, Thorndike 1904. Coefficients ... – PowerPoint PPT presentation

Number of Views:53
Avg rating:3.0/5.0
Slides: 30
Provided by: hpcus150
Category:

less

Transcript and Presenter's Notes

Title: 4632001 Tests and Measurements


1
46-320-01Tests and Measurements
  • Intersession 2006

2
More Correlation
  • Spearmans rho two sets of ranks
  • Biserial correlation continuous and artificial
    dichotmous variable
  • Point biserial correlation true dichotmous
    variable

3
Hypothesis Testing Review
  • Independent and Dependent variables
  • In Psychology we test hypotheses
  • Null Hypothesis (H0) a statement of relationship
    between the IV and DV, usually a statement of no
    difference or no relationship we assume there
    is no relationship between IV and DV
  • Alternative/Research Hypothesis (Ha) states a
    relationship, or effect, of the IV on the DV

4
Hypothesis Examples
  • H0 Men and women do not differ in IQ (?men
    ?women)
  • Ha Men and women do differ in IQ (?men ?
    ?women)
  • Any difference in value of the DV between the
    levels of the IV can be explained in 2 ways the
    effect of the IV or sampling error

5
Hypothesis Testing with Correlations
  • Null Hypothesis there is no significant
    relationship between X and Y
  • Alternative Hypothesis there is a significant
    relationship between X and Y (r is significantly
    different from 0)
  • We can use Appendix 3 (p. 641)
  • df N 2
  • robs .832
  • rcrit .195
  • Reject Ho

6
Regression
  • We know the degree to which 2 variables are
    related - correlation
  • How do we predict the score on Y if we know X?
  • Regression line
  • Principle of least squares

7
Equation Explained
  • Y predicted value of Y
  • b regression coefficient slope
  • Describes how much change is expected in Y with
    one unit increase in X
  • a intercept value of Y when X is 0

8
Line of Best Fit
  • Actual (Y) and predicted (Y) scores are almost
    never the same
  • Residual
  • Deviations from Y at a minimum
  • Prediction
  • Interpreting plot

9
More Correlation
  • Standard error of estimate
  • Coefficient of determination
  • Coefficient of alienation
  • Shrinkage
  • Cross validation
  • Correlation does not equal causation!
  • Third variable

10
Multivariate Analysis
  • 3 or more variables
  • Many predictors, one outcome
  • Linear Regression linear combination of
    variables
  • Weights
  • Raw regression coefficients
  • Standardized regression coefficients
  • Predictive power

11
More Multivariate
  • Discriminant Analysis
  • Prediction of nominal category
  • Multiple discriminant analysis
  • Factor Analysis
  • No criterion
  • Interrelation
  • Data reduction
  • Principal components
  • Factor loadings
  • Rotation

12
Reliability
  • Assess sources of error
  • Complex traits
  • Relatively free from error reliable
  • Spearman, Thorndike 1904
  • Coefficients
  • Kuder and Richardson 1934
  • Cronbach 1972 on
  • IRT
  • True Score

13
Reliability
  • Error and True Score
  • X T E
  • Random Error produces a distribution
  • Mean is the estimated true score

14
Reliability
  • True score should not change with repeated
    administrations
  • Standard error of measurement
  • Larger less reliable
  • Use to create confidence intervals

15
Reliability
  • Domain Sampling Model
  • Shorter test estimate, but sample error
  • Reliability Usually expressed as a correlation
  • Reliability Sampling distribution, correlations
    b/w all scores, average correlation

16
Reliability
  • Reliability
  • Percentage of observed variation attributable to
    variation in the true score
  • r .30 70 of variance in scores due to random
    factors

17
Sources of Error
  • Why are observed scores different from true
    scores?
  • Situational factors
  • Unrepresentative qs
  • What else?

18
Test-Retest Reliability
  • Error of repeated administration
  • Correlation b/w 2 times
  • Consider
  • Carryover effects
  • Time interval
  • Changing characteristics

19
Parallel Forms Reliability
  • 2 forms that measure the same thing
  • Correlation between two forms
  • Counterbalanced order
  • Consider time interval
  • Example WRAT-3

20
Internal Consistency
  • Split-Half reliability
  • Divide and correlate (internal consistency)
  • Check method of dividing
  • Why use Spearman-Brown formula?
  • Each test ½ length decreases reliability
  • Cronbachs alpha unequal variances

21
Internal Consistency
  • Intercorrelations among items within same test
  • Extent to which items measure same ability/trait
  • Low? Several characteristics?
  • Use KR20, coefficient alpha
  • Considers all ways of splitting data

22
Difference Scores
  • Same trait reliability 0
  • Use z-score transformations
  • Generally low

23
Observer Differences
  • Estimate reliability of observers
  • Interrater Reliability
  • Percentage Agreement
  • Kappa
  • Corrects for chance agreement
  • 1 (perfect agreement) to 1 (less than chance
    alone)
  • Interpreting
  • gt.75 excellent
  • .40 to .75 fair to good
  • lt .40 poor

24
Interpreting Reliability
  • General rule of thumb
  • Above 0.70 to 0.80 good
  • Higher the stakes, higher the r
  • Use confidence intervals (from standard error of
    estimate)

25
Low Reliability
  • Increase items
  • Spearman-Brown prophecy formula
  • Factor item analysis
  • Omit items that do not load onto one factor
  • Drop items
  • Correct for Attenuation (low correlations)

26
Validity
  • Agreement b/w a test score and what it is
    intended to measure
  • Face validity
  • Looks like its valid
  • Content-validity
  • Representative/fair sample of items
  • Construct underrepresentation
  • Construct-irrelevant variance

27
Criterion-Related Validity
  • How well a test corresponds with a criterion
  • Predictive validity
  • Concurrent validity
  • Validity Coefficient
  • Coefficient of determination

28
Evaluating Validity Coefficients
  • Changes in cause of relationship
  • Meaning of criterion
  • Validity population
  • Sample size
  • Criterion vs predictor
  • Restricted range
  • Validity generalization
  • Differential prediction

29
Construct-Related Validity
  • Define a construct and develop its measure
  • Main type of validity needed
  • Convergent evidence
  • Correlates with other measures of construct
  • Meaning from associated variables
  • Discriminant evidence
  • Low correlations with unrelated constructs
  • Criterion-referenced tests
Write a Comment
User Comments (0)
About PowerShow.com