Comments on David Kaplan

1
Comments on David Kaplans Representing a
Research Problem As a Structural Equation Model

• Jelte M. Wicherts
• Psychological Methods Department
• University of Amsterdam

2
Possibilities of SEM in experimental studies

• The use of SEM enables better tests of
  theoretical predictions and assumptions
• Higher degree of falsifiability
• However, in psychology SEM is rarely used in
  experimental studies

3
Experimental effect of exposure therapy on
arachnophobia
Anxiety questionnaire
ExposureTherapy
Frequency ofavoidance
Respirationduring exposure
MANOVA significant multivariate effect for
condition
4
Experimental effect of therapy on arachnophobia
Anxiety questionnaire
Arachnophobia
ExposureTherapy
Frequency ofavoidance
Respirationduring exposure
A test for measurement invarianceacross
conditions using a 1-factor model provides a test
of full mediation of effect by the
arachnophobia-factor
5
Yes SEM has added value

• What is the value added in using SEM?
• Testing mediation (the main value)
• Greater degrees of falisfiability (d.f.)
• Handling of measurement error
• Although not restricted to SEM
• Incredible model flexibility

With SEM much more can be learned from your data
6
But why is there a reluctance to use advanced SEM?

1. Sample size, computer programs
2. Why would one use more strict tests of
   theoretical predictions?
3. It should not be too difficult!
4. Knowledge of models among substantive researchers
5. Theories/predictions are often vague

7
2) Why would one use more strict tests of
theoretical predictions?

• Higher DF means higher falsifiability
• This means more severe empirical test of
  theoretical predictions
• However, the use of restricted SEMs also means
  raising the empirical bar

?
8
3) It should not be too difficult!

• It is often difficult to get papers with a
  modeling perspective published in substantive
  journals
• Typical reviewers reaction this paper has too
  much greek and is more suited for a specialized
  statistical journal like psychometrika
• Also papers with a formal stance often have
  little impact

9
3) It should not be too difficult!

• Impact of empirical papers published in Journal
  of Personality and Social Psychologyin 2000 and
  2001.

Average Impact
Use of Equations? No 19.11 (SD16.1)
Yes 10.83(SD9.10)
Mann-Whitney test Z1.80, plt.05 (one-sided)
10
4) Knowledge of models

• Substantive researchers are often unfamiliar with
  possibilities (and the workings!) of SEM

11
5) Theories are often vague

• In psychology many theories are verbal and not
  very specific (more/less laws).
• It is often very difficult to translate
  theoretical predictions to SEMs.
• E.g., people with high self-esteem benefit more
  from exposure therapy

12
Some solutions

• Sample size, computer programs
• Why would one use more strict tests of
  theoretical predictions?
• It should not be too difficult!
• Knowledge of models among substantive researchers
• Theories/predictions are often vague

-gt Robustness studies -gt Stress advantages the
use of SEM can solve issues! -gt Write tutorials
better education -gt Write clear papers! -gt Join
editorial boards -gt More ( earlier) involvement
in substantive research
13
Good examples of research with SEM will hopefully
lead to an increase in the valid use of SEM