Non-Experimental designs: Correlational

1
Non-Experimental designs Correlational
Quasi-Experiments

• Psych 231 Research Methods in Psychology

2
Announcements

• Lab attendance is critical this week because
  group projects are being administered
• Attendance will be taken.
• Dont forget Quiz 9 (chapters 12 13) due on
  Monday Nov. 5

3
Non-Experimental designs

• Sometimes you just cant perform a fully
  controlled experiment
• Because of the issue of interest
• Limited resources (not enough subjects,
  observations are too costly, etc).
• Surveys
• Correlational
• Quasi-Experiments
• Developmental designs
• Small-N designs
• This does NOT imply that they are bad designs
• Just remember the advantages and disadvantages of
  each

4
Correlational designs

• Looking for a co-occurrence relationship between
  two (or more) variables

• Example 1 Suppose that you notice that the more
  you study for an exam, the better your score
  typically is.
• This suggests that there is a relationship
  between study time and test performance.
• We call this relationship a correlation.
• 3 properties form, direction, strength

• For our example, which variable is explanatory
  (predictor) and which is response (outcome)?
• It depends on your theory of the causal
  relationship between the variables

5
Scatterplot
study time
exam score
X Y
6 6
1 2
5 6
3 4
3 2
6
Scatterplot

• Response Explanatory variables
• For descriptive case, it doesnt matter which
  variable goes where
• Correlational analysis
• For predictive cases, put the response variable
  on the Y axis
• Regression analysis

7
Form
8
Direction
9
Strength
Common statistic computed for correlation
Pearsons r
r -1.0 perfect negative corr.
10
Correlational designs

• Advantages
• Does not require manipulation of variable
• Sometimes the variables of interest cannot be
  manipulated
• Allows for simple observations of variables in
  naturalistic settings (increasing external
  validity)
• Can look at a lot of variables at once

11
Correlational designs

• Disadvantages
• Do not make casual claims
• Third variable problem
• Temporal precedence
• Coincidence (random co-occurence)
• r0.52 correlation between the
• number of republicans in US senate
• and number of sunspots
• From Fun with correlations

• Correlational results are often misinterpreted

12
Misunderstood Correlational designs

• Example 2 Suppose that you notice that kids who
  sit in the front of class typically get higher
  grades.
• This suggests that there is a relationship
  between where you sit in class and grades.

Possibly implied All Children who sit in the
back of the classroom always receive lower
grades than those each and every child who sit
in the front. Incorrect interpretation Sitting
in the back of the classroom causes lower
grades. Better way to say it Researchers X and
Y found that children who sat in the back of the
classroom were more likely to receive lower
grades than those who sat in the front.
Example from Owen Emlen (2006)
13
Non-Experimental designs

• Sometimes you just cant perform a fully
  controlled experiment
• Because of the issue of interest
• Limited resources (not enough subjects,
  observations are too costly, etc).
• Surveys
• Correlational
• Quasi-Experiments
• Developmental designs
• Small-N designs
• This does NOT imply that they are bad designs
• Just remember the advantages and disadvantages of
  each

14
Quasi-experiments

• What are they?
• Almost true experiments, but with an inherent
  confounding variable
• General types
• An event occurs that the experimenter doesnt
  manipulate
• Something not under the experimenters control
• (e.g., flashbulb memories for traumatic events)
• Interested in subject variables
• high vs. low IQ, males vs. females
• Time is used as a variable

15
Quasi-experiments

• Program evaluation
• Research on programs that is implemented to
  achieve some positive effect on a group of
  individuals.
• e.g., does abstinence from sex program work in
  schools
• Steps in program evaluation
• Needs assessment - is there a problem?
• Program theory assessment - does program address
  the needs?
• Process evaluation - does it reach the target
  population? Is it being run correctly?
• Outcome evaluation - are the intended outcomes
  being realized?
• Efficiency assessment- was it worth it? The the
  benefits worth the costs?

16
Quasi-experiments

• Nonequivalent control group designs
• with pretest and posttest (most common)
• (think back to the second control lecture)

• But remember that the results may be
  compromised because of the nonequivalent control
  group (review threats to internal validity)

17
Quasi-experiments

• Advantages
• Allows applied research when experiments not
  possible
• Threats to internal validity can be assessed
  (sometimes)

• Disadvantages
• Threats to internal validity may exist
• Designs are more complex than traditional
  experiments
• Statistical analysis can be difficult
• Most statistical analyses assume randomness

18
Non-Experimental designs

• Sometimes you just cant perform a fully
  controlled experiment
• Because of the issue of interest
• Limited resources (not enough subjects,
  observations are too costly, etc).
• Surveys
• Correlational
• Quasi-Experiments
• Developmental designs
• Small-N designs
• This does NOT imply that they are bad designs
• Just remember the advantages and disadvantages of
  each

19
Developmental designs

• Used to study changes in behavior that occur as a
  function of age changes
• Age typically serves as a quasi-independent
  variable

• Three major types
• Cross-sectional
• Longitudinal
• Cohort-sequential

20
Developmental designs

• Cross-sectional design
• Groups are pre-defined on the basis of a
  pre-existing variable
• Study groups of individuals of different ages at
  the same time
• Use age to assign participants to group
• Age is subject variable treated as a
  between-subjects variable

21
Developmental designs

• Cross-sectional design

• Advantages
• Can gather data about different groups (i.e.,
  ages) at the same time
• Participants are not required to commit for an
  extended period of time

22
Developmental designs

• Cross-sectional design

• Disavantages
• Individuals are not followed over time
• Cohort (or generation) effect individuals of
  different ages may be inherently different due to
  factors in the environment
• Are 5 year old different from 15 year olds just
  because of age, or can factors present in their
  environment contribute to the differences?
• Imagine a 15yr old saying back when I was 5 I
  didnt have a Wii, my own cell phone, or a
  netbook
• Does not reveal development of any particular
  individuals
• Cannot infer causality due to lack of control

23
Developmental designs

• Longitudinal design

• Follow the same individual or group over time
• Age is treated as a within-subjects variable
• Rather than comparing groups, the same
  individuals are compared to themselves at
  different times
• Changes in dependent variable likely to reflect
  changes due to aging process
• Changes in performance are compared on an
  individual basis and overall

24
Longitudinal Designs

• Example
• Wisconsin Longitudinal Study (WLS)
• Began in 1957 and is still on-going (50 years)
• 10,317 men and women who graduated from Wisconsin
  high schools in 1957
• Originally studied plans for college after
  graduation
• Now it can be used as a test of aging and
  maturation

25
Developmental designs

• Longitudinal design

• Advantages
• Can see developmental changes clearly
• Can measure differences within individuals
• Avoid some cohort effects (participants are all
  from same generation, so changes are more likely
  to be due to aging)

26
Developmental designs

• Longitudinal design

• Disadvantages
• Can be very time-consuming
• Can have cross-generational effects
• Conclusions based on members of one generation
  may not apply to other generations
• Numerous threats to internal validity
• Attrition/mortality
• History
• Practice effects
• Improved performance over multiple tests may be
  due to practice taking the test
• Cannot determine causality

27
Developmental designs

• Cohort-sequential design

• Measure groups of participants as they age
• Example measure a group of 5 year olds, then the
  same group 10 years later, as well as another
  group of 5 year olds
• Age is both between and within subjects variable
• Combines elements of cross-sectional and
  longitudinal designs
• Addresses some of the concerns raised by other
  designs
• For example, allows to evaluate the contribution
  of cohort effects

28
Developmental designs

• Cohort-sequential design

Time of measurement
1975
1985
1995
Cohort A
1970s
Cohort B
1980s
Cohort C
1990s
29
Developmental designs

• Cohort-sequential design

• Advantages
• Get more information
• Can track developmental changes to individuals
• Can compare different ages at a single time
• Can measure generation effect
• Less time-consuming than longitudinal (maybe)
• Disadvantages
• Still time-consuming
• Need lots of groups of participants
• Still cannot make causal claims

30
Small N designs

• What are they?
• Historically, these were the typical kind of
  design used until 1920s when there was a shift
  to using larger sample sizes
• Even today, in some sub-areas, using small N
  designs is common place
• (e.g., psychophysics, clinical settings,
  expertise, etc.)

31
Small N designs

• One or a few participants
• Data are typically not analyzed statistically
  rather rely on visual interpretation of the data
• Observations begin in the absence of treatment
  (BASELINE)
• Then treatment is implemented and changes in
  frequency, magnitude, or intensity of behavior
  are recorded

32
Small N designs

• Baseline experiments the basic idea is to show
• when the IV occurs, you get the effect
• when the IV doesnt occur, you dont get the
  effect (reversibility)
• Before introducing treatment (IV), baseline needs
  to be stable
• Measure level and trend

33
Small N designs

• Level how frequent (how intense) is behavior?
• Are all the data points high or low?
• Trend does behavior seem to increase (or
  decrease)
• Are data points flat or on a slope?

34
ABA design

• ABA design (baseline, treatment, baseline)

• The reversibility is necessary, otherwise
• something else may have caused the effect
• other than the IV (e.g., history, maturation,
  etc.)

35
Small N designs

• Advantages
• Focus on individual performance, not fooled by
  group averaging effects
• Focus is on big effects (small effects typically
  cant be seen without using large groups)
• Avoid some ethical problems e.g., with
  non-treatments
• Allows to look at unusual (and rare) types of
  subjects (e.g., case studies of amnesics, experts
  vs. novices)
• Often used to supplement large N studies, with
  more observations on fewer subjects

36
Small N designs

• Disadvantages
• Effects may be small relative to variability of
  situation so NEED more observation
• Some effects are by definition between subjects
• Treatment leads to a lasting change, so you dont
  get reversals
• Difficult to determine how generalizable the
  effects are

37
Small N designs

• Some researchers have argued that Small N designs
  are the best way to go.
• The goal of psychology is to describe behavior of
  an individual
• Looking at data collapsed over groups looks in
  the wrong place
• Need to look at the data at the level of the
  individual