Thinking Scientifically in Everyday Life

1
Thinking Scientifically in Everyday Life

• Three Types of Validity

2
Quote

• ... the educated person is not the person who
  can answer the questions, but the person who can
  question the answers.
• --Theodore Schick
  Jr.

3
Quote

• The whole of science is nothing more than the
  refinement of everyday thinking.
• --Albert Einstein

4
Refining Everyday Logic by Questioning

• External Validity
• Internal Validity
• Construct Validity

5
What is external validity?

• The ability to generalize to a broad range of
  people and situations.

6
Why do peoples conclusions often lack external
validity?

• Generalizing based on a sample of 1 because of
  the false consensus effect
• Basing conclusions on small, biased,
  nonrepresentative samples.

7
The False Consensus Effect

• Definition Everyone does what I do
• Example 1 Perot voters thought Perot was going
  to win.
• Example 2 If people help on a task, they think
  most people would help if they dont help, they
  think most people wouldnt help.

8
Dangers of generalizing from small samples

• Depression Overgeneralizing from failures.
• Stereotyping Theyre all like that.
• Foolishness Being convinced by anecdotes, single
  cases, or a single example.

9
Example of a Problem of Biased Samples

• Shere Hites study 7 of women happily married.
• Most national surveys 90 of women happily
  married.
• Example from Myers, 1994.

10
What is internal validity?

• Correctly concluding that a factor causes a
  certain effect.

11
Why do we misidentify causal relationships?

• We may mistakenly see relationships that dont
  exist Illusory Correlation
• We may make mistakes about the direction of the
  causal relationship between the variables.
• People dont always know the causes of their own
  behavior--even though they think they do.

12
Accurately Detecting a Relationship is NOT Easy

• Illusory Correlation Seeing a relationship that
  doesnt exist
• Example 1 The Sports Illustrated Jinx
• Example 2 The hot hand phenomena in
  basketball
• Example 3 Changing answers on multiple-choice
  tests

13

• Teams and individuals who are on the cover of SI
  actually do better after being on the cover.
• Players are actually less likely to make the next
  shot after making one.
• More answers are changed from wrong to right than
  from right to wrong.

14
Determining the Cause of a Relationship is
Difficult

• Effects may really be causes.
• Causes (and effects) may really be side
  effects of other factors.
• There are so many possible causes that it is hard
  to isolate the true cause.

15
Effects may be causes

• Listening to country music and depression?
• Authoritative parents and well-behaved children?

16
Both factors may be side effects of a causal
variable

• Baldness and length of marriage
• Childrens weight and number of questions
  correctly answered on a vocabulary test

17
Isolating the particular cause is very difficult

• Problem with before-after reasoning Person may
  change even without treatment.
• Problem with comparing natural groups Groups
  may differ on a variety of variables.
• Smokers eat differently than non-smokers.
• Heavy drug users differ from non-users even
  before they start using drugs.

18
Quotes about the problems of identifying causes

• I always get better, even without the leeches.
• --Woody Allen
• Dont compare apples with oranges.

19
People dont know why they do things Why ask
why?

• Young children
• Nisbett Wilsons (1975) research

20

• Young children often dont give explanations for
  their behavior even though parents want such
  explanations.
• Nisbett Wilson (1975) documented many examples
  of people not knowing why the did something or
  why they wanted something.

21
What is Construct Validity?

• Definition the degree to which a study actually
  measures and manipulates the variables that the
  researcher claims to be measuring.

22
Why do peoples conclusions often lack construct
validity?

• We can only infer that we have measured or
  manipulated mental constructs.
• These inferences can be wrong.

23
Why are inferences about mental states often
wrong?

• Observers are not cautious about labeling
  behavior.
• Those being observed may behave in a way that
  masks their true mental state.
• Placebo effects

24
Observers are eager to label behavior.

• Anthropomorphism giving animals human
  characteristics.
• Snap judgments

25
Instances of psychiatrists mislabeling behavior

• patient exhibits bizarre writing behavior
• patients had reached inner peace, nirvana,
  heaven on earth

26

• From classic study where researchers checked
  themselves into a mental hospital.
• From reports of early work in the 1950s on major
  tranquilizers that made patients sleep most of
  the time.

27
People may mask their true mental state

• Social desirability bias not responding like one
  really would, but instead responding in a way
  that would make a better impression.

28
Placebo effects

• Definition people will believe they have been
  helped if they expect the treatment to work. They
  are being changed by their expectations of what
  the treatment will do, rather than whats in the
  treatment itself.
• Examples Pills, Doctor visits, Psychotherapy?

29
Summary Questions to Ask
If there is evidence, is it any good?
Construct validity (if making claims about
mental constructs)
External validity (if generalizing)
Internal validity (If making cause- effect claims
)
Assignment
30
Assignment

• Use what you know about the three validities to
  criticize
• Testimonials
• Unprofessional polls