Improving the Quality of Investigative Decision Making

1
Improving the Quality of Investigative Decision
Making

• Craig Bennell, Ph.D.
• Carleton University
• Ottawa, Ontario, Canada
• cbennell_at_connect.carleton.ca

2
Two-Alternative Decisions

• Is this suspect lying?
• Will this negotiation end in failure?
• Are these crimes linked?
• Was this bite mark made by an adult?
• Is this suicide note fake?
• Is this allegation of rape false?
• To improve decision-making we need a valid
  measure of performance...

3
How Should We Measure Performance?
4
Relatively Inaccurate Evidence
5
Relatively Accurate Evidence
6
ROC ANALYSIS
(Bennell, 2005)
7
ROC ANALYSIS
An AUC of .60 indicates that there is a 60
chance that a randomly selected positive case
will be higher on evidence X than a randomly
selected negative case
(Bennell, 2005)
8
The Accuracy of Diagnostic Systems The
Polygraph CQT and GKT

• The Control Question Test
• Relevant (R) and Control (C) questions
• For each R-C pair a score of -3 to 3 is given
• Greater than 5 truthful, Less than -5
  deceitful
• The Guilty Knowledge Test
• Multiple choice questions
• One alternative relates to guilty knowledge (GK)
• Elevated responses to GK questions deceitful
• How accurate are these procedures?

9
Median AUC for GKT .88 Median AUC for CQT .85
Accuracy scores for 52 lab studies examining the
validity of the polygraph
(National Academy of Science, 2003)
10
Identifying Accurate Predictors The Fake
Suicide Note

• What discriminates between real and fake notes?
• Structure variables
• Content variables
• Combinations of the two
• How can we tell if some are better than others?
• How can we determine how much better?

11
(Jones Bennell, 2005)
12
Setting Appropriate ThresholdsLinking Serial
Burglaries

• Many behaviours can be used
• Inter-crime distance is very useful (AUC.80)
• The closer two crimes are to one another the more
  likely it is that they are linked
• When should we decide that crimes are linked?
• What is an appropriate threshold?

13
(Bennell Canter, 2002)
14
Comparing Decision MakersBite Mark
Identification

• It is often unclear as to the source of a bite
  mark
• Non-accidental (adult)
• Accidental (child)
• How well can people perform on this task?
• Is expertise required?

15
Senior forensic dentists (AUC.69) Student
dentists (AUC.69) Junior forensic dentists
(AUC.68) General dentists (AUC.62) Social
workers (AUC.63) Police officers (AUC.62)
(Whittaker et al., 1998)
16
Conclusions

• ROC analysis provides a way of calculating
  accuracy scores that are not dependent on
  arbitrary thresholds
• ROC analysis also provides a way of identifying
  thresholds that meet the demands of specific
  situations
• The result for police investigations will likely
  be improved decision-making accuracy and utility