Use of SEM programs to precisely measure scale reliability - PowerPoint PPT Presentation

About This Presentation
Title:

Use of SEM programs to precisely measure scale reliability

Description:

LM approach. Start from the factor model with no error correlation ... It will become more complicated if error correlations are allowed. 12. New program ... – PowerPoint PPT presentation

Number of Views:24
Avg rating:3.0/5.0
Slides: 19
Provided by: sigmathE
Category:

less

Transcript and Presenter's Notes

Title: Use of SEM programs to precisely measure scale reliability


1
Use of SEM programs to precisely measure scale
reliability
IMPS2001, July 15-19,2001Osaka, Japan
  • Yutaka Kano and Yukari Azuma
  • Osaka University

2
Reliability measure for
  • Reliability with possibly correlated errors
  • a coefficient

3
An example
a 0.69 0.74
0.78? 0.69
0.64 0.60
4
From the example
  • Coefficient alpha can be distorted seriously by
    error correlations
  • e.g. Green-Hershberger (2000), Raykov (2001)
  • In the case, ? has to be used to correctly
    figure out the reliability

5
Problem
  • How can one identify error correlations?
  • The factor model allowing for (fully) correlated
    errors is not identifiable, because it contains
    too many parameters
  • A trivial solution would be It does not work
    because

6
LM approach
  • Start from the factor model with no error
    correlation
  • Perform the LM test for releasing a zero
    covariance between errors using a SEM program
  • EQS can perform it most easily and most accurately

7
Real data analysis
  • A questionnaire on perception on physical
    exercise
  • n653, p15, one-factor model
  • The data were collected by Dr. Oka (Waseda U.)

8
Result_1
  • Best fitted model, with 7 correlated errors
  • ?2250.375(df83) (n653)
  • GFI0.950, CFI0.952, RMSEA0.056

9
Result_2
  • Estimates of reliability
  • a 0.90
  • ? 0.90 by
  • ? 0.87 by LM test
  • Note that

10
Search for variables
  • Even though one factor model is fitted well,
    inclusion of a variable with small true variance
    can reduce reliability
  • There is no convenient way to select variables
    for the composite scale to have maximum
    reliability

11
Mathematically
  • It is complicated
  • It will become more complicated if error
    correlations are allowed

12
New program
  • A new program is being developed which
  • gives a list of reliability estimates for each
    factor
  • gives a list of predicted reliability estimates
    when one variable is removed
  • Error correlations are allowed

13
Flowchart
Well fit?
No
DATA
Free some error covariances to get good fit
Factor analysis
Yes
Decide composite scale items
Print reliability
End
14
Example, continued
  • ? 0.87 with 15 variables

15
Scale developer
16
If V13 is removed, then
17
Results
  • For one-factor model with uncorrelated errors,
    the variable with the smallest factor loading is
    least favorable.
  • If there is a variable whose deletion improves
    reliability, then this is the variable.
  • For one-factor model with correlated errors, the
    variable with the smallest factor loading is not
    always least favorable.
  • While deletion of the variable does not improve
    reliability, there may be other variables to be
    deleted to improve reliability.
  • The example here is the case.

18
Summary
  • Correlated errors invalidate the coefficient
    alpha and traditional one-factor based
    reliability.
  • LM test is useful to find error correlations.
  • Magnitude of factor loadings does not necessarily
    provide accurate information on indicator
    selection when correlated errors exist.
  • The forthcoming Web-based program will help
    reliability analysis.
Write a Comment
User Comments (0)
About PowerShow.com