Testing Model Fit

1
Testing Model Fit

• SOC 681
• James G. Anderson, PhD

2
Assessment of Model Fit

• Examine the parameter estimates
• Examine the standard errors and significance of
  the parameter estimates.
• Examine the squared multiple correlation
  coefficients for the equations
• Examine the fit statistics
• Examine the standardized residuals
• Examine the modification indices

3
Measures of Fit

• Measures of fit are provided for three models
• Default Model this is the model that you
  specified
• Saturated Model This is the most general model
  possible. No constraints are placed on the
  population moments It is guaranteed to fit any
  set of data perfectly.
• Independence Model The observed variables are
  assumed to be uncorrelated with one another.

4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
Overall measures of Fit

• NPAR is the number of parameters being estimated
  (q)
• CMIN is the minimum value of the discrepancy
  function between the sample covariance matrix and
  the estimated covariance matrix.
• DF is the number of degrees of freedom and equals
  the p-q
• pthe number of sample moments
• q the number of parameters estimated

8
Overall measures of Fit

• CMIN is distributed as chi square with dfp-q
• P is the probability of getting as large a
  discrepancy with the present sample
• CMIN/DF is the ratio of the minimum discrepancy
  to degrees of freedom. Values should be close to
  1.0 for correct models.

9
Chi Square ?2

• Best for models with N75 to N100
• For Ngt100, chi square is almost always
  significant since the magnitude is affected by
  the sample size
• Chi square is also affected by the size of
  correlations in the model the larger the
  correlations, the poorer the fit

10
Chi Square to df Ratio ?2/df

• There are no consistent standards for what is
  considered an acceptable model
• Some authors suggest a ratio of 2 to 1
• In general, a lower chi square to df ratio
  indicates a betteer fitting model

11
Transforming Chi Square to Z

• Z ?(2?2) - ?(2df-1)

12
CMIN
13
RMR, GFI

• RMR is the Root Mean Square Residual. It is the
  square root of the average amount that the
  sample variances and covariances differ from
  their estimates. Smaller values are better
• GFI is the Goodness of Fit Index. GFI is between
  0 and 1 where 1 indicates a perfect fit.
  Acceptable values are above 0.90.

14
GFI and AGFI (LISREL measures)

• Values close to .90 reflect a good fit.
• These indices are affected by sample size and can
  be large for poorly specified models.
• These are usually not the best measures to use.

15
RMR, GFI

• AGFI is the Adjusted Goodness of Fit Index. It
  takes into account the degrees of freedom
  available for testing the model. Acceptable
  values are above 0.90.
• PGFI is the Parsimony Goodness of Fit Index. It
  takes into account the degrees of freedom
  available for testing the model. Acceptable
  values are above 0.90.

16
RMR, GFI
17
Comparisons to a Baseline Model

• NFI is the Normed Fit Index. It compares the
  improvement in the minimum discrepancy for the
  specified (default) model to the discrepancy for
  the Independence model. A value of the NFI below
  0.90 indicates that the model can be improved.

18
Baseline Comparisons
19
Bentler-Bonett Index or Normed Fit Index (NFI)

• Define null model in which all correlations are
  zero
• ?2 (Null Model) - ?2 (Proposed Model) ?2
  (Null Model)
• Value between .90 and .95 is acceptable above
  .95 is good
• A disadvantage of this index is that the more
  parameters, the larger the index.

20
Comparisons to a Baseline Model

• RFI is the Relative Fit Index This index takes
  the degrees of freedom for the two models into
  account.
• IFI is the incremental fit index. Values close to
  1.0 indicate a good fit.
• TLI is the Tucker-Lewis Coefficient and also is
  known as the Bentler-Bonett non-normed fit index
  (NNFI). Values close to 1.0 indicate a good fit.
• CFI is the Comparative Fit Index and also the
  Relative Noncentrality Iindex (RNI). Values close
  to 1.0 indicate a good fit.

21
Tucker Lewis Index or Non-normed Fit Index (NNFI)

• Value ?2/df(Null Model) - ?2/df(Proposed
  Model) ?2/df(Null Model)
• If the index is greater than one, it is set to1.
• Values close to .90 reflects a good model fit.
• For a given model, a lower chi-square to df ratio
  (as long as it is not less than one) implies a
  better fit.

22
Comparative Fit Index (CFI)

• If D ?2 - df, then
• D(Null Model) - D(Proposed Model)
• D(Null Model)
• If index gt 1, it is set to 1 if index lt0, it is
  set to 0
• A lower value for D implies a better fit
• If the CFI lt 1, then it is always greater than
  the TLI
• The CFI pays a penalty of one for every parameter
  estimated

23
Parsimony Adjusted Measures

• PRATIO is the Parsimony Ratio. It is the number
  of constraints in the model being evaluated as a
  fraction of the number of constraints in the
  independence model.
• PNFI is the result of applying the PRATIO to the
  NFI.
• PCFI is the result of applying parsimony
  adjustments to the CFI.

24
Parsimony-Adjusted Measures
25
Measures Based on the Population Discrepancy

• NCP is an estimate of the noncentrality parameter
  obtained by fitting a model to the population
  moments rather than to the sample moments.

26
NCP
27
The Minimum Sample Discrepancy Function

• FMIN is the minimum value of the discrepancy.

28
FMIN
29
Root Mean Square Error of Approximation (RMSEA)

• Value ? (?2/df-1)/(N-1)
• If ?2 lt df for the model, RMSEA is set to 0
• Good models have values of lt .05 values of gt .10
  indicate a poor fit.

30
PCLOSE

• PCLOSE is the probability for testing the null
  hypothesis that the population RMSEA is no
  greater than 0.05.

31
RMSEA
32
Information Theoretic Measures

• These indices are composite measures of badness
  of fit and complexity.
• Simple models that fit well receive low scores.
  Complicated poorly fitting models get high
  scores.
• These indices are used for model comparison not
  to evaluate a single model.

33
AIC
34
ECVI
35
Akaike Information Criterion (AIC)

• Value ?2 k(k-1) - 2(df)
• where k number of variables in the model
• A better fit is indicated when AIC is smaller
• Not standardized and not interpreted for a given
  model.
• For two models estimated from the same data, the
  model with the smaller AIC is preferred.

36
Information Theoretic Measures

• BCC is the Browne-Cudeck Criterion
• BIC is Bayes Information Criterion.
• CAIC is the consistent AIC
• ECVI except for a constant scale factor it is the
  same as AIC.
• MECVI except for a scale factor is the same as
  the BCC.

37
Difference in Chi Square

• Value X2diff X2 model 1 -X2 model 2
• DFdiff DF model 1 DFmodel 2

38
Miscellaneous Measures

• HOELTER is the largest sample size for which one
  would acc3ept the hypothesis that a model is
  correct.

39
HOELTER
40
Hoelter Index

• Value (N-1)?2(crit) 1
• ?2
• Where ?2 (crit) is the critical value for the
  chi-square statistic
• The index should only be calculated if chi square
  is statistically significant.

41
Hoelter Index (2)

• If the critical value is unknown, can
  approximate (1.645 ?(2df-1) 2 1
• 2 ?2/ (N-1)
• For both formulas, one rounds down to the nearest
  integer
• The index states the sample size at which the chi
  square would not be significant

42
Hoelter Index (3)

• In other words, how small ones sample size would
  have to be for chi square to no longer be
  significant
• Hoelter Recommends values of at least 200
• Values lt 75 indicate poor fit

43
(No Transcript)