Theory

1
Theory Experimental Design

• Part II

2
FTT Political Science

• Two generations (although overlap time wise)
• First testing equilibrium non-equilibrium
  predictions of social choice theory
• Second testing more applied models, typically
  with greater institutional (political) detail.
• Mirror evolution of formal models in discipline

3
First Generation Tests on Elections Committees
Tests of Spatial Voting Models

• This work is reviewed in
• McKelvey, Richard D. Peter C. Ordeshook, 1990,
  A Decade of Experimental Research on Spatial
  Models of Elections and Comittees, in James M.
  Enelow Melvin Hinich, eds., Advances in the
  Spatial Theory of Voting, Cambridge Cambridge U.
  Press.

4
First Generation Tests on Committee Voting

• Goal of initial work on committees
• see extent disequilibrium really happened in
  lab.
• Not so easy to avoid equilibrium creating
  conditions not suggested by theory.
• For example, if subjects know (or suspect)
  experiment has an end or consider time spent as
  costly, may be motivated to agree on a choice.

5
First Generation Testson Generic Voting Games

• While fundamental largely become province of
  social choice theorists theoretical side
  dominates
• Main emphasis what is necessary get equilibrium,
  considering axioms of social choice theory
  implications.
• Social Choice Welfare (both society journal)
  main avenues, mainly normative

6
Example of Modern First Generation Like
Voting Experiment

• Voting Games and Computational Complexity,
  Glenn W. Harrison Tanga McDaniel, working paper

7
Voting Rule Experiment

• Authors confront Gibbard-Satterthwaite Theorem
  loosely stated only voting rule that is
  strategy proof for all possible preference
  profiles is dictatorial.
• strategy proof when choices people make under
  voting rule are truthful reflect true
  preferences.
• Consider an example 3 choices, x, y, z, 3
  types of voters, A (40 voters), B (20 voters),
  C (40 voters)

8
Example of Strategic Voting
Who will win this election? Not obvious. If
voters A, B, C all voted for their most
preferred choice (truth telling), we use
majority rule, then a tie between x z.
9
Example of Strategic Voting
But is that what we would want as a society?
Normatively? Condorcet argued that most preferred
choice is one who would Defeat or tie all others
in pairwise (binary) contests.
10
Example of Strategic Voting
Who is the Condorcet winner? Binary choice
between x y, A votes for x, B C for y, y
wins. Binary choice between y z, A B for y, C
for z, y wins. y is the Condorcet winner.
11
Example of Strategic Voting
Majority rule is definitely not strategy proof
in fact, normatively we might prefer that
voters strategize positively expect them to
Making Votes Count
12
Goal of Voting Experiment

• Harrison McDaniel contend that some voting
  rules may be more difficult (in a computational
  sense) to manipulate than others.
• They want to test their hypothesis in the lab.

13
Voting Rule Experiment Goal

• H M test a voting rule in laboratory they
  believe is
• Easy to explain to subjects
• Easy to implement
• Difficult for subjects to strategically
  manipulate (because of unspecified cognitive
  limits)

14
Voting Rule Experiment Goal

• Theory testing or fact finding?
• While voting rule is script as in other formal
  theory experimental tests, H M expect something
  not modelling (cognitive limits of subjects) is
  important are fact finding.
• Also, some policy pre-testing
• Nevertheless design of experiment is scripted
  by theory

15
Proposed Voting Rule

• One voting rule that might be good is to always
  select Condorcet winner in fact this has been
  proven to be strategy proof.

16
Condorcet Winner Voting Rule
Can voters manipulate this voting rule getting a
more preferred outcome by misrepresenting their
preferences?
17
Condorcet Winner Voting Rule
Who is the Condorcet winner now?
18
Proposed Voting Rule

• problem Condorcet winner not always exsit.
• Extension of Condorcet rule by Young find
  non-cyclical ranking with most support of voters.
• Turns out solution found by solving linear
  programming problem
• Condorcet Consistent Voting Rule basically
  chooses Condorcet winner if one exists, if not,
  non-cyclical ranking with greatest support.

19
Condorcet Consistent Voting Rule
Who would win? In this example C voters ranking
would be maximal, note similarity w/ dictatorship
in this example -- but rule not same as
dictatorship.
20
Computation Voting Rule

• strategic voting under Condorcet Consistent
  voting rule requires complex computations when
  of alternatives is large
• for n alternatives there are n! non-cyclic
  rankings.
• Thus while not strategy proof in theory, contend
  behaviorally incentive compatible with truth
  telling (non strategic voting).
• Point of Experiment is this true?

21
Inducing Preferences in Voting Experiments

• Usually money
• Tell subjects will pay based on which
  alternative, x,y,z is chosen have subjects
  vote.
• Can measure extent of strategic voting by
  comparing choices to induced preferences.

22
Home Grown Preferences

• Harrison McDaniel use home grown preferences
• Subjects given list of CDs, grouped in
  categories
• Subjects vote over which category for group, then
  each picks a CD from the list of 10 in category
• Difficulty how measure strategic voting?

23
How Measure Strategic Voting?

• Use control treatment subjects choose CDs
  under a random dictator voting rule
• But to be sure instruct subjects in this
  treatment only on advantages of telling truth
  non neutral instructions
• Note importance of random assignment.
• Are home grown preferences desirable here?

24
Heterogeneity of Preferences

• strategic voting harder when preferences of
  voters more heterogeneous.
• Vary heterogeneity
• Simple treatment music categories of Jazz/Easy
  Listening, Classical, RB, Rock, CW.
• Complex treatment music categories of Jazz/Easy
  Listening, Classical, Heavy Metal, Rap, CW.
• Contend most subjects prefer RB or Rock.

25
Table 1 Musical Categories CDsCategory A
Jazz/Easy Listening

• (1) Najee, Share My World
• (2) Kenny G, Breathless
• (3) Art Porter, Undercover
• (4) Russ Freeman the Rippingtons, Sahara
• (5) Tony Bennet, Unplugged
• (6) George Howard, A Home Far Away
• (7) Enigma 2, The Cross of Change
• (8) Billy Joe Walker, Life is Good
• (9) Barry Manilow , Singing in the Big Bands
• (10) Nat King Cole, The Greatest Hits

26
Category B Classical

• (1) John Williams the Boston Pops Orchestra,
  It Dont Mean a Thing if it aint got that
  Swing
• (2) Vivaldi, The 4 Seasons Gil Shattam Orpheus
  Fritz Kreisler.
• (3) Mahler, Symphony 5. The New York
  Philharmonic Leonard Bernstein.
• (4) Yo Yo Ma, The New York Album. Baltimore
  Symphony Orchestra David Zinman.
• (5) Handel, Messiah Atlanta Symphony Orchestra
  Chamber Chorus Robert Shaw.
• (6) Cecilia Bartoli, Mozart Portraits Vienna
  Chamber Orchestra György Fischer.
• (7) Van Clyburn in Moscow. Brahms Rachmaninoff. M
  oscow Philharmonic Orchestra Kiril Konorashin.
• (8) Kiri, Her Greatest Hits Live London
  Symphony Orchestra Steven Barlow.
• (9) Tchaikovsky, Nutcracker London Symphony
  Orchestra S ir Charles Mackerras.
• (10) The Best of Wolfgang Amadeus Mozart. Academy
  of St. Martin-in-the-fields Neville Marriner.

27
Category C Heavy Metal

• (1) Pantera, Far Beyond Driven
• (2) Queensryche, Promised Land
• (3) Magadeth, Youthanasia
• (4) Motley Crüe, Motley Crue (featuring
  Hooligans Holiday)
• (5) Mother Tongue Mother Tongue
• (6) Obituary, World Demise
• (7) Jackyl, Push Comes to Shove
• (8) Alice in Chains, Jar o f Flies
• (9) Alice Cooper, The Last Temptation
• (10) Cinderella, Still Climbing

28
Category D Rap

• (1) Pete Rock and CL Smooth, The Main
  Ingredient
• (2) Lighter Shade of Brown, Layin in the Cut
• (3) Craig Mack, Project Funk Da World
• (4) Ghetto Mafia, Draw the Line
• (5) Common Sense, Resurrection
• (6) Stevie B, Funky Melody
• (7) Salt-n-Pepa, Very Necessary
• (8) j. Little, Puttin it Down
• (9) Celly Gel, Heat 4 Yo Azz
• (10) Hammer, The Funky Headhunter

29
Category E Country Western

• (1) Garth Brooks, In Pieces
• (2) George Ducas, George Ducas
• (3) Shenandoah, In the Vicinity of the Hearth
• (4) Chris Ledoux, Haywire
• (5) Willie Nelson, Healing Hands of Time
• (6) Vince Gill, When Love Finds You
• (7) Pam Tillis, Sweethearts Dance
• (8) Noah Gordon, I Need a Break
• (9) Rodney Crowell, Let the Picture Paint
  Itself
• (10) Ricky Lynn Gregg, Get a Little Closer

30
Information to Subjects

• Varied information gave subjects
• Information Treatment specifics of CC voting
  rule spelled out to subjects examples supplied
• No Information Treatment subjects only told
  that the social ranking chosen would be the one
  which would most likely receive the support of a
  majority of the voters.
• Why vary this?
• Policy Pre-testing . . .

31
Experimental Design
32
Results of Voting Rule Experiment

• Gist found under simple preference profile
  significant difference between rankings in CC
  RD.
• No significant difference between rankings in CC
  RD under tough preference profile.
• Moreover, information matters only in simple
  preference profile experiments.
• Results support argument that when preferences
  are tough to figure out, CC does elicit truth
  telling.

33
First Generation Research Summary

• Voting Rule Experiment example of an experimental
  test of research from social choice theory
• like first generation of FTT in political
  science.
• While experiment interesting results important,
  most political scientists not think FTT as in
  political science today.
• Why?

34
Impact of Disequilibrium Results on Formal Theory
in Political Science

• Economists can frequently start from well
  accepted equilibrium models, then do
  comparative statics by standard techniques.
• Political theory not have well accepted
  equilibrium models to start from.
• Theory must incorporate details of situation.

35
Impact of Disequilibrium Results on Formal Theory
in Political Science

• explicitly model role of
• information,
• repetition
• institutions
• usually accompanied by increasing use of
  non-cooperative game theory,
• incomplete information,
• explicit specification of extensive forms.
• The New Institutionalism

36
Impact of Disequilibrium Results on Formal Theory
in Political Science

• Another trend evolutionary agent based
  models.
• Different view of behavior individuals
  programmed to behave in certain ways only
  change behavior through replacement or imitation
  (e.g. Bendor, Diermeier, Ting)
• Complex processes literature, etc.
• Difficult for laboratory experiments long term
  processes of evolution unlikely in single
  experiment.

37
Experiments on Classic Games Political Science

• Classic games often building blocks in formal
  work in political science.
• Example bargaining games ultimatum dictator
  games add together to get Baron/Ferejohn
  legislative bargaining game
• Example voting turnout like public good/pd
  combined

38
Turnout as Combined Public Good/PD

• Think of a two candidate election w/ two groups
  of supporters (teams or political parties).
• Each group member individually decides whether or
  not to vote, paying individualized cost to
  voting.
• Group with most voters wins group payoff
  distributed equally to all group members whether
  voted or not.
• Team turnout game combination of public good game
  (within a team) a prisoners dilemma game
  (between teams).

39
Turnout Experiment Theory

• Turnout modelled this way by Palfrey Rosenthal,
  1983, A Strategic Calculus of Voting, Public
  Choice in APSR, 1984.
• Showed equilibria exist with positive turnout.
• Model tested experimentally by Schram and
  Sonnemans, International Journal of Game Theory,
  1996.
• Good example of ways formal theory testing works
  with more complex game.

40
Turnout Experiment Design

• Subjects split into 2 groups of 6 each, labelled
  yellow blue.
• Each subject had to decide whether to buy an
  imaginary disc.
• Price of a disc was common knowledge equal for
  everyone.
• of discs bought by group determined payoffs.
• Payoffs equal for everyone within a group.
• Repeated for 20 periods.

41
Turnout Experiment Design

• Two payoff schedules, representing winner take
  all (WIN) proportional representation (PR).
• In WIN, each group member bought most discs
  received payoff of 2.5 Dutch gilders other
  group received zero, ties broken randomly.
• In PR, payoff was proportional to turnout within
  group of discs bought in ones group was
  divided by total number bought multiplied by
  2.22 Dutch gilders.
• Price of disc 1 gilder (WIN), 0.75 gilders (PR)

42
Turnout Experiment Predictions

• Nash equilibria of games in pure strategies (one
  shot)
• In PR, one disc bought by each group.
• In WIN, 6 discs bought by each group.
• (turnout theoretically higher in WIN, why?
  relationship prediction)
• Quasi-symmetric mixed strategy equilibria
• In PR, all subjects buy a disc with probability
  0.098.
• In WIN two equilibria
• All subjects buy with probability 0.051
• All subjects buy with probability 0.949

43
Turnout Experiment Results

• Find comparative static predictions are supported
  (relationship predictions)
• But point predictions not.
• Schram Sonnemans in Journal of Economic
  Psychology, 1996 compare analysis with hypotheses
  derived from a model of turnout that incorporates
  group pressure as explanation of turnout.
• Find some support for group model, although
  experiment not an explicit test of model (since
  designed to test Palfrey/Rosenthal)

44
FTT Experimental Design

• Game directly from mechanics of Palfrey/Rosenthal
  model (script).
• not described to subjects as voting situation
  with candidates as in election (frame).
• Why?
• Advantage
• experiment tests theory of participation
  without baggage subjects may bring about voting
  as an act
• can serve as baseline results to experiments
  where act is described as voting w/ candidates
  etc.
• Disadvantage decreases external validity?
• Is this deception?

45
FTT Experimental Design

• Palfrey/Rosenthal
• general model without specific values for
  experiment specific values must be set for
  payoffs cost of participation.
• Experimenters must solve model for equilibrium
  predictions with specific values.
• Palfrey/Rosenthal model had no PR, only WIN
• experimenters take model, solved it for
  particular values, plus made modifications to
  model.

46
FTT Experimental Design

• Unless working with simple formal theory, usually
  modifications/limitations needed in testing a
  formal theory
• Difficult to truly fit theory as originally
  devised even in laboratory with a lot of control
• Still often easier than w/o naturally occurring
  data . . .

47
FTT Experimental Design

• Probably more difficult to fit theory as script
  for experiment for political science models than
  for economists.
• Why? more institutional detail, more applied,
  thus more bells whistles to either try to put
  in or simplify.

48
Fitting Design to Theory Example of Difficulties

• choice variable often a continuous makes
  solving models easier sometimes necessary for
  solutions.
• two candidate competition over a unidimensional
  policy space, assume candidates can choose any
  point an infinite choice set.
• But suppose wanted to test this if tell subject
  choose number between 1 10 tell subject any
  fraction is acceptable, is subject truly
  thinking continuously?

49
Experiments PTT
50
Theory Testing Review

• Psychological or Social Psychological Theories
• Typically non formal, i.e.
• assumptions underlying theory stated verbally
• hypotheses about variables posited.
• Equilibrium predictions rarely derived.
• Usually decision-theoretic
• Focused often on process of choice internal
  process of mind.

51
Political Psychologists, Social Psychologists
Formal Theory

• Some political psychology or social psychology
  research involves formal theories.
• Prospect theory (Kahneman Tversky)
• Has been subject of political science
  experiments
• Quattrone, George A. Amos Tversky, Contrasting
  Rational and Psychological Analyses of Political
  Choice, American Political Science Review 82, 3,
  Sept. 1988, pages 720-36.

52
Political Psychologists, Social Psychologists
Formal Theory

• Some involve testing hypotheses loosely derived
  from assumptions behind rational choice based
  formal theories.
• Often little of both, testing a non formal theory
  based on research from psychology social
  psychology in contrast to hypothesis about
  rational choice implication for situation.

53
Psychological Social Psychological Theory
Testing Experiments (PTT) in Political Science

• PTT in political science builds on experimental
  research in psychology social psychology
  generally.
• Often application to political science context
  theories arising from psychology social
  psychology.

54
PTT in Political Science

• traditional approach of hypothesis testing
  through random assignment, control (baselines),
  manipulation.
• more of a script (I.e. is placed in a political
  context for external validity reasons) than in
  psych, however still usually less of a script
  than FTT.

55
Example of PPT

• Taber, Charles Milton Lodge, Motivated
  Skepticism in the Evaluation of Political
  Beliefs, 2000
• Results from experiments in psychology
  individuals are prone to accept at face value
  evidence that is congruent with prior beliefs
  denigrate or hyper-critically evaluate evidence
  that is contrary to their priors.
• Taber Lodge evaluate in context of theory of
  political reasoning.

56
Theory of Motivated Reasoning

• Assume all reasoning is motivated.
• Two types of motivation
• Accuracy Goals seek information to be correct
• Partisan Goals reason to defend a prior
  (psychology literature calls these directional
  goals)
• Continuum, pure rationalist to pure partisan

57
Three Mechanisms Assumed Underlie Motivated
Reasoning

• Hot cognition hypothesis social concepts
  evaluated in past affectively charged
  (negatively or positively) confronted with
  concept recall affect feeling.
• On-line processing every time exposed to
  concept, ones summary affective evaluation
  recalled updated.
• How do I feel? Heuristic OL tally stored with
  concept is automatically available for future
  information processing.

58
Experimental Hypotheses

• Hypothesis 1 a prior attitude effect people
  who feel strongly about an issue, even when
  encouraged to be objective, will evaluate
  supportive arguments as stronger more
  compelling than contrary arguments
• Hypothesis 2 a disconfirmation bias people
  will spend more time cognitive resources
  denigrating counter arguing attitudinally
  incongruent than congruent arguments.
• Hypothesis 3 a confirmation bias when free to
  choose information, people will seek out
  confirming over disconfirming information.

59
Experimental Hypotheses

• Contend above three hypotheses imply fourth
• Hypothesis 4 attitude polarization attitudes
  will become more extreme even when confronted
  with a balanced set of pro con arguments.

60
Two Further Experimental Hypotheses

• Hypothesis 5 an attitude strength effect those
  with strongest policy attitudes more likely to
  have biases above.
• Hypothesis 6 a sophistication effect political
  sophisticates, because have greater ammunition to
  counter argue more likely to have biases above.

61
Experimental Design

• Subjects recruited from intro political science
  classes given class credit for participation
• Subjects seated at computers, told participating
  in study of public opinion.
• Completed two tasks
• Note subjects told to put feelings aside
  to be objective etc.

62
Experimental Design First Task

• Asked attitude questions on of issues including
  gun control or affirmative action, randomly
  assigned.
• Given chance to practice on information board
  w/ arguments pro con on same issue arguments
  subjects examined time spent on each is
  measured secretly. is this deception?
• Reevaluated attitudes asked standard
  demographic information measures of political
  sophistication
• Designed to test confirmation bias.

63
Experimental Design Second Task

• Again, asked attitude questions (on gun control
  if assigned affirmative action, vice versa)
• Asked rate strength of 8 arguments, 4 pro 4
  con, presented in random order.
• Followed by attitude battery memory recog. task
• completed thoughts listing task for 2 pro 2 con
  arguments
• Designed to test disconfirmation bias.

64
Experimental Design Misc. Issues

• Arguments gathered from interest groups.
• Edited so similar in sentence length, sentences
  per argument, reading level etc.
• One problem with experimental design where is
  control or baseline experiment?

65
Results Generally Support Hypotheses

• Subjects see arguments congruent with beliefs as
  stronger prior attitude effect
• Effect greater for sophisticated extremists.
• Subjects took more time to read incongruent
  arguments disconfirmation bias
• Effect greater for sophisticated extremes
• When asked thoughts tended to be against
  incongruent arguments
• Some evidence of polarization (although unclear
  how strong)

66
Another Example of PTT

• Huddy, Leonie Nayda Terkildsen, Gender
  Stereotypes and the Perception of Male and Female
  Candidates, American Journal of Political
  Science, 37, 1 (Feb. 1993), 119-147.
• Research questions do voters use gender
  stereotypes in judging political candidates if
  so what type.
• Two types of gender stereotyping investigated
• Gender-trait stereotypes personality traits
  seen as gender linked.
• Belief-trait stereotypes political outlooks
  seen as gender linked.

67
Another Example of PTT

• Trait theory predicts
• candidates with masculine traits more competent
  on military, crime, defense issues regardless of
  gender.
• candidates with feminine traits more competent on
  compassion issues regardless of gender.
• Belief theory predicts inferences about
  ideology leads to competency ratings
• Females Liberals more competent on compassion
  issues
• Males Conservatives better on military, crime,
  defense

68
Asked Subjects about Hypothetical Candidates2 by
2 factorial design
69
Experimental Results

• Used students at Stony Brook
• Find more support for gender traits explaining
  evaluations of candidates, some minor support for
  belief stereotypes.

70
PTT Design v. FTT Design

• Less script for subjects to follow I.e. told
  somewhat vague slightly false information about
  purpose of experiment (some small deception)
• instructions not part of paper, almost always
  appendix to FTT experiments.
• Focus on individual rather than group behavior.
• No repetition.
• Choices subjects made (in responding to
  questions, choosing information to look at
  length of time) not related to payment for
  participation

71
Purpose of Scripts in FTT

• Scripts (instructions) supply descriptions of
  players, action choices, possible payoffs.
• FTT experimenters ask participants to enact
  scripts.
• In Schram Sonnemans experiment, subjects told
  would be purchasing a disk, reward to them if
  their group purchased more disks, etc.

72
Advantages of Scripts

• Scripts increase replicability
• Allow researcher to trace sometimes subtle
  influence of institutional details.
• When experiment is less scripted or not
  believable,
• subject guessing what experiment designed to do
  ad-lib introducing lower control over
  variables in experiment.
• Serious problem if experiment involves complex
  tasks.

73
Disadvantage of Scripts

• If keeping subjects ignorant of purpose is
  important, using a script makes deception more
  explicit.
• If subjects know time spent reading an argument
  is measured etc., behavior may be affected
• By being less precise about experiments purpose,
  some deception is avoided omitting truth
  different from lying about truth.
• Thus in simple decision-making experiments
  detailed script may not be necessary harmful.

74
Do Scripts Make a Difference?

• Hertwig Ortmann, 2001, Experimental Practices
  in Economics A Methodological Challenge for
  Psychologists, Behavioral and Brain Sciences
  24(4),
• Consider two investigations of hindsight bias
  reviewed by Hertwig Ortmann
• Camerer, Loewenstein, Weber (1989) The curse
  of knowledge in economic setting An experimental
  analysis. Journal of Political Economy
  971232-1255.
• Davies, M. F. (1992) Field dependence and
  hindsight bias Cognitive restructuring and the
  generation of reasons. Journal of Research in
  Personality 2658-74.

75
Do Scripts Make a Difference?

• In CLW uninformed group of participants
  guessed future earnings of real companies based
  on information such as previous annual earnings
  per share.
• An informed group of participants (told actual
  earnings) then traded assets that paid dividends
  equal to earnings predicted by uninformed group.
• Participants in both groups provided with precise
  script.
• Those in uninformed group given role (script) of
  market analyst faced with task of predicting
  future dividends of various companies.

76
Do Scripts Make a Difference?

• Those in informed group assigned role of trader
  knew dividend was determined by uninformed
  groups predictions.
• Thus, to price assets optimally (and avoid
  hindsight bias), traders had to predict
  prediction of analysts accurately, to ignore
  their knowledge of actual dividends.
• Eventually, traders traded assets to others in
  actual double-oral auctions, in which buyers
  sellers shouted out bids or offers at which they
  were willing to buy or sell.
• When a bid and offer matched, a trade took place
  (p. 1236).

77
Do Scripts Make a Difference?

• In Davies participants given series of
  assertions asked to rate truth of each,
• then given feedback (i.e., truth values of
  assertions)
• later asked to recall original judgment.
• In contrast to CLR, Davies did not assign
  specific roles to participants or provide them
  with precise script.

78
Do Scripts Make a Difference?

• first stage of study, during which participants
  rated assertions for their truth,
• was described to participants as involving
  evaluation of college students knowledge
  (Davies 1992, p. 61),
• were told that recollection stage concerned
  peoples ability to remember or recreate a
  previous state of knowledge (Davies 1992, p.
  61).

79
Do Scripts Make a Difference?

• CLW compared amount of hindsight bias in
  predictions of participants who enacted role of
  trader (i.e., who actually traded assets in
  double-oral auction)
• to bias in predictions made by another group of
  participants who did not enact role of trader.
• Goal of two groups same
• predict average prediction of uninformed group
  given companies actual earnings.

80
Do Scripts Make a Difference?

• Both groups received incentives for making
  correct predictions.
• CLW reported participants in both conditions
  exhibited some hindsight bias,
• but enactment of trader role reduced bias by
  about half.

81
Another Example Wason Task

• Subjects shown 4 cards displaying symbols such as
  T, J, 4, 8
• are given a conditional rule about cards,
• such as If there is a T on one side of the card
  antecedent P, then there is a 4 on the other
  side of the card consequent Q.
• Participants told each card has letter on one
  side number on other.

82
Another Example Wason Task

• asked which cards would need to turn over to
  discover whether conditional rule is true or
  false.
• Typical result, replicated many times,
• few participants (10) give answer prescribed by
  propositional logic T and 8 (P not-Q).

83
Wason Selection Task

• Most choose either T (P) alone or T and 4 (P
  Q).
• errors seen as reflections of confirmation
  bias, matching bias, availability heuristic.
• However, putting it in a social context,
  increases percentage of logically correct
  answers.

84
Wason Selection Task

• police officer checking whether people conform to
  certain rules
• in the context of a drinking age law (If someone
  is drinking beer P, then they must be over 19
  years of age Q),
• 74 of participants gave the logical P not-Q
  response.

85
Wason Selection Task

• Gigerenzer and Hug (1992) way social context
  affects reasoning depends on perspective
  participants cued.
• For instance, rule If an employee works on
  weekend, then that person gets a day off during
  week depend on whether it is seen from
  perspective of an employer or of an employee.
• Employee role dominant answer was P not-Q
  (75)
• Employer role dominant response was not-P Q (61)

86
Do Scripts Matter?

• Scripts form of framing matter in experiments
  especially when measuring incidents of biases in
  human decision making.
• Subjects in Taber Lodge experiment encouraged
  to be unbiased objective.
• Would it have mattered if they had been told to
  play a role of a political decision maker?

87
Final Word on Scripts

• In FTT, do we tell subjects what to do?
• Try for neutrality how experiment works, role
  to play, but without how to play role.
• That is, not pointing out their supposedly
  optimal strategy, unless it is purpose of
  experiment (as in Harrison McDaniels random
  dictator treatment).
• Is there a difference between framing by giving
  context to a role such prompting a subject
  directly to use certain strategies?

88
Individual v. Group Behavior

• One reason FTT uses scripts strategic behaviors.
• Partly reflects difference in research questions
• Most PTT researchers extremely interested in
  process of mind in making choices, psychological
  mechanics behind choice.

89
Individual v. Group Behavior

• Can see in Taber Lodge experiment, emphasis on
  mechanics -- On Line theory, how reasoning
  works, etc.
• Emphasis on process desire to find what social
  psychologists call mediators or generative
  mechanisms explain how or why manipulated
  treatments have an impact on choice
• Also desire to find moderators or factors that
  affect likelihood of effects demonstrated in
  experiment similar to fact finding in FTT.

90
Individual v. Group Behavior

• Focus of PTT on individual behavior may affect
  results CLWs hindsight experiments may show
  less error than traditional non scripted tests
  because subjects were involved in a game
  situation.
• One argument decision-theoretic experiments
  appropriate for studying voters public opinion
  but not appropriate for understanding behavior of
  elites.
• Ideally, both levels of analysis are useful.

91
Role of Repetition

• Formal theory testers use repetition partly to
  increase data note not all use repetition
  (trust bargaining games often exceptions)
• However, repetition also
• Gives chance to adapt to environment, to accrue
  experience with experimental setting procedure
• Affords opportunity to learn how own choices
  interact with other players.

92
Role of Repetition

• Two types of learning during repetition
• Learning about laboratory environment task
• Learning about possible strategic aspects of
  decision situation. In fact, in some game
  theoretic situations potential of repeated play
  implies different choices.
• Vary rarely do studies by psychologists social
  psychologists have repetition learning
• Hertwig Ortmann estimate only 10 provide any
  feedback for learning to subjects during
  experiment.

93
Role of Repetition

• Hertwig Ortmann show repetition significantly
  decreases biases errors
• found in decision-making experiments, mere
  practice without feedback can improve subjects
  performance.
• For example, research on preference reversals
  shows repetition eliminates such reversals.
• For another example, see Daniel Friedman, Monty
  Halls Three Doors Construction and
  Deconstruction of a Choice Anomaly, American
  Economic Review, 884, 933-946 (Sept. 1998)

94
Argument Against Repetition

• Individuals in real world situations not always
  get opportunity for repetition repetition
  decreases external validity of experiment
• counter argument in real world have sources of
  advice from experienced individuals
• In many trust game experiments others, often no
  repetition.
• Key is research question if believe that
  inexperienced subjects important for question,
  then repetition can distort results.

95
Deception in PTT FTT

• While our example did not contain a lot of
  deception (more lack of information than outright
  deception) deception is much more common in PTT
  than in FTT.
• Hertwig Ortmann report deception in
  experimental articles in top ranked journal in
  social psychology, Journal of Personality and
  Social Psychology (JPSP), averages between 31
  47,
• Hardly any FTT experiments have deception (can
  think of only one recent one).

96
Arguments Against Deception Negative Externality

• Private benefits gt private costs but social costs
  gt social benefits
• Hertwig Ortmann Subjects expectation will
  not be deceived (by experimenter) is a common
  good that would be depleted (contaminated)
  quickly if deception was allowed decision left
  to each experimenter.
• FTT not trust experimenters to make unbiased
  analysis of (private) benefits of deception its
  (public) costs.
• APA requirement to reveal after makes worse.

97
Arguments Against Deception Ethics

• American Psychological Association (APA) ethics
  guidelines (APA 1992, p. 1609) propose to employ
  deception as a last-resort strategy, to be used
  only after careful weighing of benefits costs
• But frequent use of deception in some areas of
  psychology seems to confirm economists fear.
• Second argument against deception ethical
  argument which arises from long history of
  questionable experiments (Tuskegee, etc.)

98
Arguments For Deception Only Way to Measure
Subject Behavior

• HO If subjects aware of true purpose, might
  respond strategically investigator might lose
  experimental control.
• For instance, one might expect participants to
  bend over backwards (Kimmel 1996, p. 68) to
  show how accepting they are of members of other
  races if they know that they are participating in
  a study of racial prejudices.

99
Arguments For Deception Used to Create
Situations Not Natural

• Deception can be used to produce situations of
  special interest that are unlikely to arise
  naturally (e.g., an emergency situation in which
  bystander effects can be studied).

100
Arguments For DeceptionSubjects Like It?

• From HO Smith Richardson (1983 observed that
  subjects in experiments w/ deception reported
  having enjoyed, indeed having benefited from,
  experience more than those in experiments w/o
  deception.
• See also Christensen (1988), Aitkenhead Dordoy
  1985 Sharpe et al. 1992).
• Others have found opposite effect Cook, Bean,
  Calder, Frey, Krovetz Reisman 1970 Epstein,
  Suedfeld Silverstein 1973 Allen 1983 Rubin
  1985 Oliansky 1991 Fisher Fyrberg 1994

101
Empirical Evidence on Deception

• Experimental tests of trust games gives some
  evidence on effect of deception on subject
  behavior.
• Suggest may accept being fooled once, but not
  twice (Dickhaut, Hubbard McCabe 1995).
• Recent results, Krupat Garonzik (1994), suggest
  prior experience with deception affects subjects
  expectations increases their suspicion (see
  also Epley Huff 1998).

102
Empirical Evidence on Deception

• According to Krupat Garonzik (1994), such
  suspicion is likely to introduce "considerable
  random noise" into their responses (p. 219).
• Stang (1976) percentage of suspicious
  participants (in conformity experiments) tracked
  closely increase in deception through 1960s.

103
PTT Summary

• Most PPT work is non formal, tests applications
  of theories experimental results from
  psychology social psychology to political
  science contexts, some deception.
• Methodology differs from most FTT
• Less use of scripts
• Process of individual decision making focused
  rather than choices in strategic situations.
• Very little repetition of tasks
• Financial incentives not tied to decision making