Stories and statistics: What can the Sally Clark case tell us about the psychology of evidential reasoning? - PowerPoint PPT Presentation


PPT – Stories and statistics: What can the Sally Clark case tell us about the psychology of evidential reasoning? PowerPoint presentation | free to download - id: 69bdbf-ZmFiY


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

Stories and statistics: What can the Sally Clark case tell us about the psychology of evidential reasoning?


Stories and statistics: What can the Sally Clark case tell us about the psychology of evidential reasoning? David Lagnado Division of Psychology and Language Sciences – PowerPoint PPT presentation

Number of Views:20
Avg rating:3.0/5.0
Slides: 61
Provided by: DavidLa186


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Stories and statistics: What can the Sally Clark case tell us about the psychology of evidential reasoning?

Stories and statistics What can the Sally Clark
case tell us about the psychology of evidential
  • David Lagnado
  • Division of Psychology and Language Sciences
  • University College London

Evidential reasoning
  • How do people assess and combine evidence to make
  • Legal, Medical, Financial, Social
  • Cognitive science approach
  • What kinds of representations?
  • What kinds of inference patterns?
  • How do these compare with normative or formal
    methods of evidential reasoning?
  • Bayesian networks now used to model complex
    forensic evidence (Taroni et al, 2006)

Reasoning with legal evidence
  • Legal domain
  • E.g. juror, judge, investigator, media
  • Complex bodies of interrelated evidence
  • Forensic evidence witness testimony alibis
    confessions etc
  • Need to integrate wide variety of evidence to
    reach singular conclusion (e.g. guilt of suspect)

Story model (Pennington Hastie, 1986, 1991,
  • Evidence evaluated through story construction
  • Stories involve human action sequences in which
    relationships of physical causality and
    intentional causality between events are central
  • Jurors use prior causal knowledge and
    expectations about story structure to fill in
    gaps in evidence
  • Active sense-making process to construct an
    account of what happened

Evidential Reasoning
  • Reasoning from evidence
  • Use the evidence to construct most plausible
    account of what happened
  • Generate a causal story based on the evidence
  • Reasoning about evidence
  • Assessing the strength/reliability/validity of
    the evidence
  • How well does the evidence support the putative

think-aloud protocols from jurors in simulated
trials suggest that they predominantly engage in
  • Individual variability in competence at juror
    reasoning (Kuhn et al., 1994) and evidential
    reasoning in general (Kuhn, 1991)

SATISFICING Construct single story using
multiple stories Evaluate against evidence and
Requires ability to reflect on ones own
Group deliberation helps shift ---gt
Stories blessing or curse?
  • Story is concrete and categorical
  • Describes a singular causal process
  • Economy of representation
  • Easy to communicate
  • Clear-cut basis for decision and action
  • Identify key variables to blame
  • Hard to simultaneously compare/evaluate multiple
    stories (cf Wigmore)
  • Danger of neglecting alternative accounts
  • Evidence often gathered/interpreted for a single
    story (confirmation bias)
  • The truth might not make a good story

Binocular rivalry
Switch between two coherent stories (prosecution
vs defence)
Switch between two coherent percepts (green vs
Even when inputs are mixed
Even when evidence is mixed
Sally Clark case
  • Sally Stephen Clark married, both solicitors
  • Son Christopher born in 1996
  • Died suddenly at home aged 11 weeks
  • Sally alone with child noticed he was unwell
    ambulance called, but he could not be
  • Postmortem (Dr Williams)
  • Death from natural causes - lung infection (and
    bruises consistent with resuscitation attempts)
  • Body was cremated

Sally Clark case
  • Harry born in 1997
  • Died suddenly at home aged 8 weeks
  • Stephen at home with Sally but Sally alone with
    child when discovered unwell ambulance called,
    but he could not be resuscitated
  • Postmortem (Dr Williams)
  • Suspicious - death from shaking?
  • Re-examined death of Christopher
  • Concluded it too was unnatural, with evidence of
  • Sally Clark charged with murder of both children

Prosecution case
  • Christopher Harry were smothered
  • Nb change from Dr Williams initial claims of
    shaking for Harry (error in diagnosis of retinal
  • Neither died from SIDS because there were
    unexplained injuries
  • Numerous similarities between the two deaths
  • which would make it an affront to commonsense to
    conclude that either death was natural, and it
    was beyond coincidence for history to so repeat

Prosecution case
  • Similarities
  • Babies died at similar ages
  • Both found unconscious in same room at same
    time shortly after feed
  • Mother alone with child when found unwell
  • Father either away or due to go away
  • (Medical evidence of previous abuse deliberate
  • How unlikely are these given that mother is
    innocent? (beyond coincidence?)

Prosecution case Injuries to Christopher
Between lip and jaw
Small marks on arms and legs
Both fresh older blood
Prosecution case Injuries to Harry
Old fracture dislocation
Spinal bleeding swollen cord
Prosecution case Credibility of witnesses
Sallys testimony in doubt
Harry slumped in bouncy chair?
Police surgeon says impossible for baby of 8
weeks to slump in bouncy chair
Sally Clark states she found Harry slumped in
bouncy chair
Stephens testimony in doubt
Sally Clark smothered Harry
Opportunity/ Motive
Sally Clark alone with Harry
Prosecution case Statistical evidence
  • Professor Sir Roy Meadow (Paediatrics)
  • Report Sudden unexpected deaths in infancy
  • Risk factors age of mother (lt26), smoker in
    household, no wage earner
  • None applied to Clark family
  • Chance of one SIDS in family 1 in 8,543
  • Chance of two SIDS 1/8543 x 1/8543
  • 1/73 million
  • by chance that happening will occur about once
    every hundred years

Defence case
  • Sally Clark did not kill her children
  • They died of natural but unexplained causes
  • Medical evidence amounts only to suspicion
  • Two of prosecution experts said cause of deaths
  • Case hinges upon Dr Williams reliability and

Defence case Injuries to Christopher
Resuscitation attempts
Postmortem effects
NB distinguish event from reports of event
Blood in lungs
Torn frenulum
Report of Torn frenulum
Report of Bruises
Reliability of Dr Williams
Defence case Injuries to Harry
Explain Stephen testimony mistake
Sally Clark smothered Harry
Stephen very unlikely to lie to protect wife if
she killed their children
Opportunity/ Motive
Sally Clark alone with Harry
Stephen Clark reliability
Stephen Clark states he returned home at
Taxi records show Stephen Clark returned home at
Defence case Statistical evidence
Known risk factors
Calculation for two deaths ignores possible
genetic environmental factors
Estimate for probability of one SIDS death
UNKNOWN risk factors
Deaths are not independent (so cannot simply
2 SIDS death significantly greater than 1/73
  • Sally Clark found guilty by 10-2 majority
  • Imprisoned for life

First Appeal Statistical evidence misleading
  • Non independence
  • 1/73 million figure flawed
  • Probabilities are not independent
  • Relevance
  • Probability of two SIDS deaths insufficient
  • needs to be compared against probability that
    mother murders both her children
  • Estimated incidence of this is much lower than of
    two SIDS deaths

it is clearly inadequate to concentrate on a
single cause of death. If we make an assessment
of the probability of two babies in one family
both dying from SIDS, we must equally make a
similar assessment of the probability of two
babies in one family both being murdered (and so
on, for any other causes that may be under
consideration) Dawid (2002)
  • Two alternative causes of the deaths
  • (exclusive but not exhaustive other causes
    possible, also possible that one SIDS, one
    murdered etc)

Prior probability of murder is even lower
Prior probability of SIDS is low
Evidence of 2 deaths
Prior to other/medical evidence, probability of
double SIDS greater than probability of double
Appeal dismissed
  • Court of appeal judgment
  • No need for expert statisticians to give oral
    testimony it was hardly rocket science
  • Defence already pointed out flaws in statistics
  • What matters is that probability of two SIDS
    deaths is very low, not exact figure
  • Statistic might have had larger impact on jury
    than it should have, but case against Sally Clark
    was nevertheless overwhelming
  • "In the context of the trial as a whole, the
    point on statistics was of minimal significance
    and there is no possibility of the jury having
    been misled so as to reach verdicts that they
    might not otherwise have reached."

Second appeal
  • Discovery of new evidence
  • Harry had bacterial infection
  • Known by Dr Williams but not disclosed at trial!
  • (When jury asked about blood tests for Harry,
    Williams said no relevant test results)
  • Plausible cause of Harrys death
  • according to 11 independent experts
  • Also casts doubt on Christophers death due to
    unreliability of Dr Williams

Harrys death
Second appeal
  • Conviction declared unsafe
  • Sally Clark released 2003
  • Postscript
  • Several other similar convictions involving
    Meadow subsequently overturned
  • Meadow struck off medical register 2005
    reinstated on appeal 2006
  • Williams guilty of serious misconduct 2005
  • Sally dies 2007

  • Various repercussions for legal domain
  • Expert witnesses
  • (expert in child health not an expert in
  • Interpretation and presentation of statistical
  • For evidential reasoning
  • Understanding statistical evidence
  • Role of causal networks
  • Reliability of evidence (and experts)
  • Stories and blame

Statistical evidence
  • Well-documented problems when people reason with
    probabilities (Kahneman, 2012)
  • Base rate neglect prosecutor's fallacy
    conjunction errors
  • In contrast people are good at qualitative causal
  • One approach that reconciles these findings
  • People need suitable causal models for
    appropriate probabilistic reasoning (Krynski
    Tenenbaum, 2007 Sloman, 2005 Lagnado, 2011)
  • Classic probability problems facilitated with
    causal models

Medical diagnosis problem (Krynski Tenenbaum,
  • Alternative cause of test made explicit
  • People give better estimates of probability of
  • Improved probabilistic reasoning given suitable
    causal model
  • shown for several classic problems
  • Given test people grossly overestimate
    probability of cancer
  • (Neglect low base rate)
  • Mistaken use of false positive probability
  • Low false ? high probability of cancer

Statistical evidence
  • To avoid errors in Sally Clark case
  • Need suitable (causal) model to understand
  • Need to consider (probability) of alternative
  • Need to combine via Bayes rule

Misleading categories
  • Case framed as murder vs SIDS
  • Exclusive but not exhaustive
  • Tempting to reason not-SIDS -gt murder
  • But other natural explanations possible (eg
    infections etc)
  • Key to represent alternative causes

  • Main focus on flawed assumption of independence
    of SIDS deaths
  • Judges, lawyers, media, etc
  • People understand independence/non-independence
    when framed causally
  • Possible unobserved common causes of SIDS deaths
  • Eg genetic or environmental factors

Understanding/using probability
  • Second error
  • How is probability of SIDS relevant to
    probability that Sally is guilty of murder?
  • Need to use Bayes rule
  • Requires comparison with prior probability of
    child murder
  • Danger of prosecutor's fallacy
  • Assume that 1 in 73 million figure applies to
    probability that Sally Clark is innocent
  • Eg P(2deathsnot guilty) P(not guilty2deaths)

Statistical evidence
  • Probabilistic reasoning improved by explicit
    causal models (Krynski Tenenbaum,2007)
  • Avoid Meadows second error by explicitly
    representing probability of double murder?

Prior of double SIDS is low
Prior for double murder is even lower
Representing alternative cause and its prior
probability should improve probabilistic
Ongoing empirical work on improving Bayesian
reasoning using causal models
Causal networks
  • Key role of causal reasoning borne out by Sally
    Clark case
  • But story model needs to be developed
  • Formal means for representing causal models and
  • Include representation of evidence and
    reliability (and their interrelations)
  • Move closer to theory-evidence co-ordination
  • Even if people dont always do this- they can!

Legal idioms
  • Evidential reasoning in terms of causal building
  • Capture generic inference patterns
  • Reusable and combinable
  • Qualitative causal structure
  • Based on Bayesian networks
  • Akin to schema/scripts

Fenton, Lagnado Neil, 2012
Legal idioms
  • Evidence idiom
  • Evidence depends on Hypothesis
  • Evidence is more likely if hypothesis is true
  • Observed evidence raises the probability of

Smothering causes bruises (probabilistically)
Legal idioms
  • Explaining away
  • Evidence is often rebutted

Evidence for alternative cause of bruises
Legal idioms
  • Distinguish event from report

Legal idioms
  • Evidence Reliability idiom

Impact of evidence on hypothesis is modulated by
its reliability
Legal idioms
  • Reliability of witness reports
  • Separate factors for reliability

From Schum (2001)
Legal idioms
  • Opportunity idiom

Opportunity is often a pre-condition of guilt
Legal idioms
  • Motive idiom

Motive is typically a pre-condition of guilt
Combining idioms alibi evidence
Stephen memory error?
Status of framework
  • Normative
  • Formal model to capture appropriate probabilistic
    inference (and support theory-evidence
  • Descriptive
  • Do peoples inferences conform to the model?
  • Qualitatively? Quantitatively?
  • Empirical studies suggest good fit to qualitative
  • Prescriptive
  • Guide to interpreting complex evidence and
    improving inference (shift towards TEC)

The big picture
  • Combining network fragments into a large-scale

Key factors at trial
Prosecution case
Defence case
Cognitive Economy?
  • How do people do this?
  • Lab-based studies support the claim that they use
    idioms for small-scale problems (Lagnado, 2011
    Lagnado et al., 2012)
  • But how does this scale-up?
  • Story-telling
  • Use of narrative to simplify?
  • Reasoning from but not about evidence

Stories and Blame
  • Stories constructed from causal networks
  • Cohesive narrative to explain events
  • To attribute blame for negative outcomes
  • But focus of stories can compromise proper
    theory-evidence co-ordination

Stories and Blame
  • At trial
  • Prosecution presented one cohesive story Sally
    smothered both babies
  • Explains most of the medical evidence
  • Explains unreliability of Stephen Sally
  • Supported by statistical evidence
  • Defence did not present one single story, but
    numerous disconnected pieces to explain the
    different injuries etc

Possible line of juror reasoning?
  • Jurors reject SIDS due to extreme rarity
  • Neglect low base-rate of smothering because this
    was never raised at trial
  • Accept smothering because
  • it gives simple explanation of injuries
  • (and explains inconsistent testimonies)
  • Assigns blame to someone
  • A plausible story?

Stories and Blame
  • Importance of causal story that assigns blame?
  • At second appeal
  • New story in which Harry died from infection
    and Dr Williams Meadow were blamed
  • Aftermath Media
  • Professor Meadows statistical errors are

Lessons for evidential reasoning
  • Importance of clarity in evidential reasoning
  • For jurors, lawyers, judges, experts, media
  • How can this be improved?
  • Shift from single casual story to theory-evidence
  • Use peoples capacity for causal reasoning to
    support better probabilistic inference?
  • Introduce formal methods eg Bayesian networks etc
    to help model and evaluate evidence?
  • Ongoing research!

Thank you!
  • Collaborators
  • Norman Fenton (QMUL)
  • Martin Neil (QMUL)
  • Evidence project
  • Philip Dawid
  • William Twining

BAYES RULE (odds version)
P(2deathsguilty) 1 P(2deathsguilty)
1/73million (ignoring error of
non-independence) P(guilty) 1/84million
(based on stats for double child murders but
perhaps should just consider guilty at least
one murder)
Nb p/1-p odds P odds/(1odds)
P(guilty2deaths) 0.009