Bayesian Bias

1
Bayesian Bias

• D. Glen Whitman
• Roger Koppl

2
Summary

• The structure of forensic science creates a bias
  toward finding matches
• . . . even if forensic scientists have no
  cognitive biases.
• Its also the structure of the system, not just
  the structure of the brain

3
Why?

• When tests are ambiguous
• You choose how to resolve the ambiguity
• Your choice depends on your preferences and prior
  beliefs
• Your preferences and prior beliefs depend on the
  organization of forensic science
• Organizing forensic science under the police
  skews preferences and prior beliefs in favor of
  matches

4
Key factors favoring a match

• Higher relative likelihood of the test result
  coming from a guilty person
• Higher prior belief in suspects guilt
• Higher benefit of convicting the guilty relative
  to the cost of convicting the innocent

5
Why those factors

• The lab finds a match if the likely gain of doing
  so exceeds the likely loss
• Gain-Lossgt0
• Pros vs. cons of saying match
• Gain probability of guilt X benefit of
  convicting the guilty
• Loss probability of innocence X cost of
  convicting the innocent

6
Gain

• probability of guilt X benefit of convicting
  the guilty
• benefit of convicting the guilty
• probability of guilt depends on
• prior probability of guilt
• test result (e.g. shape of electropherogram)
• Likelihood of the test result if youre guilty
• Likelihood of the test result if youre innocent

7
Gain

• probability of guilt X benefit of convicting
  the guilty
• benefit of convicting the guilty
• probability of guilt depends on
• prior probability of guilt
• test result (e.g. shape of electropherogram)
• Likelihood of the test result if youre guilty
• Likelihood of the test result if youre innocent

8
Who Stole the Strawberries?
9
Evidence
10
The suspects
11
The suspects

• The prior probability of guilt, is low

12
The suspects

• The prior probability of guilt, is high

13
Thus

• probability of guilt depends on
• d prior probability of guilt
• As well as
• r test result (e.g. shape of electropherogram)
• probability the test reads r if youre guilty
• probability the test reads r if youre innocent

14
And now for the gobbledygook
15
Probability of guilt

• probability the test reads r if youre
  guilty
• probability the test reads r if youre
  innocent

16
Gain to declaring a match

• probability of guilt X benefit of convicting
  the guilty

17
Loss from declaring a match

• probability of innocence X cost of convicting
  the innocent

18
Net benefit of declaring a match

• Before rearranging
• After rearranging

19

• Declaring a match is more likely
• The higher relative likelihood of the test result
  coming from a guilty person
• The higher prior belief in suspects guilt
• The greater benefit of convicting the guilty
  relative to the cost of convicting the innocent

20
Organization creates bias 1

• Declaring a match is more likely the higher prior
  belief in suspects guilt
• By submitting the sample, the police convey
  information
• We think this person did it.
• Thus creating a relatively high prior belief in
  the guilt of the sample source
• This effect is strengthened by organizing crime
  labs under the police

21
Organization creates bias 2

• Declaring a match is more likely the greater
  benefit of convicting the guilty relative to the
  cost of convicting the innocent
• Identification with law enforcement may raise
  this ratio
• Performance measures for police and prosecution
• Who may influence labs budget and evaluation

22
Organization creates bias 3

• Declaring a match is more likely the higher
  relative likelihood of the test result coming
  from a guilty person
• For a bogus technique, everything depends on
• prior belief
• gain from convicting the guilty vs loss from
  convicting the innocent

23

• Declaring a match is more likely
• The higher relative likelihood of the test result
  coming from a guilty person
• The higher prior belief in suspects guilt
• The greater benefit of convicting the guilty
  relative to the cost of convicting the innocent

24
For example
25
(No Transcript)
26
Brandon Mayfield

• The higher relative likelihood of the test result
  coming from a guilty person
• The computer generated him as a possible match
• The higher prior belief in suspects guilt
• He converted to Islam, married an Egyptian,
  attends a mosque where radical Muslims worship,
  and defended one of them in a civil action
• The greater benefit of convicting the guilty
  relative to the cost of convicting the innocent
• It is very important to get the terrorists.

27
Implications

• Nature of the crime
• Racial Prejudice and Stereotyping
• Pro-prosecution bias
• Bogus techniques
• Battlefield Forensics and the War on Terror

28
Nature of the crime

• The benefit of convicting the guilty relative to
  the cost of convicting the innocent is higher the
  more heinous or high-profile the crime

29
WARNING! Grisly photo coming up
30
Which photo motivates?
31
(No Transcript)
32
Why does the grisly photo motivate?

• Dror et al. Emotions get the better of us
• Whitman and Koppl Nature of the crime influences
  the benefit of convicting the guilty relative to
  the cost of convicting the innocent

33
Racial prejudice and stereotyping

• Stereotyping influences prior probability of
  guilt
• Racial prejudice influences the benefit of
  convicting the guilty relative to the cost of
  convicting the innocent

34
Pro-prosecution bias

• Raises prior probability of guilt
• Raises benefit of convicting the guilty relative
  to the cost of convicting the innocent

35
Bogus techniques

• Make outcome depend entirely on
• Prior probability of guilt
• benefit of convicting the guilty relative to the
  cost of convicting the innocent

36
Battlefield Forensics and the War on Terror

• Alters the benefit of convicting the guilty
  relative to the cost of convicting the innocent
• May alter prior probability of guilt

37
What is to be done?

• Independence
• Blind testing combined with placebo samples
• Evidence line-ups
• Separation of test and interpretation

38
Independence

• Reduces prior belief in suspects guilt
• Reduces benefit of convicting the guilty relative
  to the cost of convicting the innocent

39
Blind testing combined with placebo samples

• Reduces prior belief in suspects guilt

40
Evidence line-ups

• Reduces prior belief in suspects guilt
• Requires careful design

41
The problem with lineups
42
Separation of test and interpretation

• Providing test results to forensic consultant
  for each side
• Reveals r to trier of fact
• Unpacks Delphic match
• Restores to jury role of trier of fact

43
Closing Remark

• The problem is structural
• Its also the structure of the system, not just
  the structure of the brain
• Structural problems require structural solutions
• We need a different organization of the system

44
END