Experiment Basics: Variables

1
Experiment Basics Variables

• Psych 231 Research Methods in Psychology

2
Reminders

• Print out the Class experiment exercise (from the
  Lab web page) and bring it to labs this week
• Group project introduction sections due this week
• Dont forget to take the on-line quizzes
• quiz 5, chapter 4, was due yesterday
• quiz 6, chapter 6, is due Oct 1 (Tuesday)
• Journal Summary 1 is due in labs next week

3
Exam 1

• Results

• Mean 78.3
• Median 78
• Range 59-92

• If you want to go over your exam set up a time to
  see me

4
Many kinds of Variables

• Independent variables (explanatory)
• Dependent variables (response)
• Extraneous variables
• Control variables
• Random variables
• Confound variables

5
Many kinds of Variables

• Independent variables (explanatory)
• Dependent variables (response)
• Extraneous variables
• Control variables
• Random variables
• Confound variables

6
Identifying potential problems

• These are things that you want to try to avoid by
  careful selection of the levels of your IV (may
  be issues for your DV as well).

• Demand characteristics
• Experimenter bias
• Reactivity
• Floor and ceiling effects (range effects)

7
Demand characteristics

• Characteristics of the study that may give away
  the purpose of the experiment
• May influence how the participants behave in the
  study
• Examples
• Experiment title The effects of horror movies on
  mood
• Obvious manipulation Having participants see
  lists of words and pictures and then later
  testing to see if pictures or words are
  remembered better
• Biased or leading questions Dont you think its
  bad to murder unborn children?

8
Experimenter Bias

• Experimenter bias (expectancy effects)
• The experimenter may influence the results
  (intentionally and unintentionally)
• E.g., Clever Hans
• One solution is to keep the experimenter (as well
  as the participants) blind as to what
  conditions are being tested

9

• Knowing that you are being measured
• Just being in an experimental setting, people
  dont always respond the way that they normally
  would.
• Cooperative
• Defensive
• Non-cooperative

Reactivity
10
Range effects

• Floor A value below which a response cannot be
  made
• As a result the effects of your IV (if there are
  indeed any) cant be seen.
• Imagine a task that is so difficult, that none of
  your participants can do it.

• Ceiling When the dependent variable reaches a
  level that cannot be exceeded
• So while there may be an effect of the IV, that
  effect cant be seen because everybody has maxed
  out
• Imagine a task that is so easy, that everybody
  scores a 100
• To avoid floor and ceiling effects you want to
  pick levels of your IV that result in middle
  level performance in your DV

11
Variables

• Independent variables (explanatory)
• Dependent variables (response)
• Extraneous variables
• Control variables
• Random variables
• Confound variables

12
Dependent Variables

• The variables that are measured by the
  experimenter
• They are dependent on the independent variables
  (if there is a relationship between the IV and DV
  as the hypothesis predicts).

• Consider our class experiment

• Conceptual level Memory
• Operational level Free Recall test
• Present list of words, participants make a
  judgment for each word
• 15 sec. of filler (counting backwards by 3s)
• Measure the accuracy of recall

13
Choosing your dependent variable

• How to measure your your construct
• Can the participant provide self-report?
• Introspection specially trained observers of
  their own thought processes, method fell out of
  favor in early 1900s
• Rating scales strongly agree - agree -
  undecided - disagree - strongly disagree
• Is the dependent variable directly observable?
• Choice/decision
• Is the dependent variable indirectly observable?
• Physiological measures (e.g. GSR, heart rate)
• Behavioral measures (e.g. speed, accuracy)

14
Measuring your dependent variables

• Scales of measurement
• Errors in measurement

15
Measuring your dependent variables

• Scales of measurement
• Errors in measurement

16
Measuring your dependent variables

• Scales of measurement - the correspondence
  between the numbers representing the properties
  that were measuring
• The scale that you use will (partially) determine
  what kinds of statistical analyses you can perform

17
Scales of measurement

• Categorical variables (qualitative)
• Nominal scale
• Ordinal scale
• Quantitative variables
• Interval scale
• Ratio scale

18
Scales of measurement

• Nominal Scale Consists of a set of categories
  that have different names.

• Label and categorize observations,
• Do not make any quantitative distinctions between
  observations.
• Example
• Eye color

19
Scales of measurement

• Categorical variables (qualitative)
• Nominal scale
• Ordinal scale
• Quantitative variables
• Interval scale
• Ratio scale

Categories
20
Scales of measurement

• Ordinal Scale Consists of a set of categories
  that are organized in an ordered sequence.

• Rank observations in terms of size or magnitude.
• Example
• T-shirt size

21
Scales of measurement

• Categorical variables
• Nominal scale
• Ordinal scale
• Quantitative variables
• Interval scale
• Ratio scale

Categories
Categories with order
22
Scales of measurement

• Interval Scale Consists of ordered categories
  where all of the categories are intervals of
  exactly the same size.
• Example Fahrenheit temperature scale

• With an interval scale, equal differences between
  numbers on the scale reflect equal differences in
  magnitude.
• However, Ratios of magnitudes are not meaningful.

20º
40º
20º increase
The amount of temperature increase is the same
60º
80º
20º increase
20º
40º
Not Twice as hot
23
Scales of measurement

• Categorical variables
• Nominal scale
• Ordinal scale
• Quantitative variables
• Interval scale
• Ratio scale

Categories
Categories with order
Ordered Categories of same size
24
Scales of measurement

• Ratio scale An interval scale with the
  additional feature of an absolute zero point.

• Ratios of numbers DO reflect ratios of magnitude.
• It is easy to get ratio and interval scales
  confused
• Example Measuring your height with playing cards

25
Scales of measurement
Ratio scale
8 cards high
26
Scales of measurement
Interval scale
5 cards high
27
Scales of measurement
Interval scale
Ratio scale
8 cards high
5 cards high
0 cards high means as tall as the table
0 cards high means no height
28
Scales of measurement

• Categorical variables
• Nominal scale
• Ordinal scale
• Quantitative variables
• Interval scale
• Ratio scale

Categories
Categories with order
Ordered Categories of same size
Ordered Categories of same size with zero point
Best Scale?

• Given a choice, usually prefer highest level of
  measurement possible

29
Measuring your dependent variables

• Scales of measurement
• Errors in measurement

• Reliability Validity

30
Example Measuring intelligence?

• How do we measure the construct?
• How good is our measure?
• How does it compare to other measures of the
  construct?
• Is it a self-consistent measure?

Measuring the true score
31
Errors in measurement

• In search of the true score
• Reliability
• Do you get the same value with multiple
  measurements?
• Validity
• Does your measure really measure the construct?
• Is there bias in our measurement? (systematic
  error)

32
Dartboard analogy
Bulls eye the true score
33
Dartboard analogy
Bulls eye the true score Reliability
consistency Validity measuring what is intended
reliablevalid
reliable invalid
unreliable invalid
34
Reliability

• True score measurement error
• A reliable measure will have a small amount of
  error
• Multiple kinds of reliability

35
Reliability

• Test-restest reliability
• Test the same participants more than once
• Measurement from the same person at two different
  times
• Should be consistent across different
  administrations

Reliable
Unreliable
36
Reliability

• Internal consistency reliability
• Multiple items testing the same construct
• Extent to which scores on the items of a measure
  correlate with each other
• Cronbachs alpha (a)
• Split-half reliability
• Correlation of score on one half of the measure
  with the other half (randomly determined)

37
Reliability

• Inter-rater reliability
• At least 2 raters observe behavior
• Extent to which raters agree in their
  observations
• Are the raters consistent?
• Requires some training in judgment

500
456
38
Validity

• Does your measure really measure what it is
  supposed to measure?
• There are many kinds of validity

39
VALIDITY
CONSTRUCT
INTERNAL
EXTERNAL
CRITERION- ORIENTED
FACE
CONVERGENT
PREDICTIVE
DISCRIMINANT
CONCURRENT
Many kinds of Validity
40
VALIDITY
CONSTRUCT
INTERNAL
EXTERNAL
CRITERION- ORIENTED
FACE
CONVERGENT
PREDICTIVE
DISCRIMINANT
CONCURRENT
Many kinds of Validity
41
Face Validity

• At the surface level, does it look as if the
  measure is testing the construct?

This guy seems smart to me, and he got a high
score on my IQ measure.
42
Construct Validity

• Usually requires multiple studies, a large body
  of evidence that supports the claim that the
  measure really tests the construct

43
Internal Validity

• The precision of the results

• Did the change in the DV result from the changes
  in the IV or does it come from something else?

44
Threats to internal validity

• Experimenter bias reactivity
• History an event happens the experiment
• Maturation participants get older (and other
  changes)
• Selection nonrandom selection may lead to
  biases
• Mortality (attrition) participants drop out or
  cant continue
• Regression to the mean extreme performance is
  often followed by performance closer to the mean
• The SI cover jinx

45
External Validity

• Are experiments real life behavioral
  situations, or does the process of control put
  too much limitation on the way things really
  work?

46
External Validity

• Variable representativeness
• Relevant variables for the behavior studied along
  which the sample may vary
• Subject representativeness
• Characteristics of sample and target population
  along these relevant variables
• Setting representativeness
• Ecological validity - are the properties of the
  research setting similar to those outside the lab

47
Measuring your dependent variables

• Scales of measurement
• Errors in measurement

• Reliability Validity
• Sampling error

48
Sampling

• Errors in measurement
• Sampling error

Everybody that the research is targeted to be
about
The subset of the population that actually
participates in the research
Sample
49
Sampling

• Allows us to quantify the Sampling error

50
Sampling

• Goals of good sampling
• Maximize Representativeness
• To what extent do the characteristics of those in
  the sample reflect those in the population
• Reduce Bias
• A systematic difference between those in the
  sample and those in the population

• Key tool Random selection

51
Sampling Methods

• Probability sampling
• Simple random sampling
• Systematic sampling
• Stratified sampling
• Non-probability sampling
• Convenience sampling
• Quota sampling

52
Simple random sampling

• Every individual has a equal and independent
  chance of being selected from the population

53
Systematic sampling

• Selecting every nth person

54
Cluster sampling

• Step 1 Identify groups (clusters)
• Step 2 randomly select from each group

55
Convenience sampling

• Use the participants who are easy to get

56
Quota sampling

• Step 1 identify the specific subgroups
• Step 2 take from each group until desired number
  of individuals

57
Variables

• Independent variables
• Dependent variables
• Measurement
• Scales of measurement
• Errors in measurement
• Extraneous variables
• Control variables
• Random variables
• Confound variables

58
Extraneous Variables

• Control variables
• Holding things constant - Controls for excessive
  random variability
• Random variables may freely vary, to spread
  variability equally across all experimental
  conditions
• Randomization
• A procedure that assures that each level of an
  extraneous variable has an equal chance of
  occurring in all conditions of observation.
• Confound variables
• Variables that havent been accounted for
  (manipulated, measured, randomized, controlled)
  that can impact changes in the dependent
  variable(s)
• Co-varys with both the dependent AND an
  independent variable

59
Colors and words

• Divide into two groups
• men
• women
• Instructions Read aloud the COLOR that the
  words are presented in. When done raise your
  hand.
• Women first. Men please close your eyes.
• Okay ready?

60
Blue Green Red Purple Yellow Green Purple Blue Red
Yellow Blue Red Green
List 1
61

• Okay, now it is the mens turn.
• Remember the instructions Read aloud the COLOR
  that the words are presented in. When done raise
  your hand.
• Okay ready?

62
Blue Green Red Purple Yellow Green Purple Blue Red
Yellow Blue Red Green
List 2
63
Our results

• So why the difference between the results for men
  versus women?
• Is this support for a theory that proposes
• Women are good color identifiers, men are not
• Why or why not? Lets look at the two lists.

64
List 2Men
List 1Women
Blue Green Red Purple Yellow Green Purple Blue Red
Yellow Blue Red Green
Blue Green Red Purple Yellow Green Purple Blue Red
Yellow Blue Red Green
Matched
Mis-Matched
65

• What resulted in the performance difference?
• Our manipulated independent variable (men vs.
  women)
• The other variable match/mis-match?
• Because the two variables are perfectly
  correlated we cant tell
• This is the problem with confounds

Blue Green Red Purple Yellow Green Purple Blue Red
Yellow Blue Red Green
Blue Green Red Purple Yellow Green Purple Blue Red
Yellow Blue Red Green
66

• What DIDNT result in the performance difference?
• Extraneous variables
• Control
• of words on the list
• The actual words that were printed
• Random
• Age of the men and women in the groups
• These are not confounds, because they dont
  co-vary with the IV

Blue Green Red Purple Yellow Green Purple Blue Red
Yellow Blue Red Green
Blue Green Red Purple Yellow Green Purple Blue Red
Yellow Blue Red Green
67
Debugging your study

• Pilot studies
• A trial run through
• Dont plan to publish these results, just try out
  the methods
• Manipulation checks
• An attempt to directly measure whether the IV
  variable really affects the DV.
• Look for correlations with other measures of the
  desired effects.