Integration of predictive and retrospective risk analysis in health care Tjerk van der Schaaf Leiden University Medical Center Eindhoven University of Technology

1
Integration of predictive and retrospective risk
analysis in health care Tjerk van der Schaaf
Leiden University Medical Center Eindhoven
University of Technology
2
overview

• retrospective method PRISMA-medical
• predictive method HFMEA
• 3 examples of possible integration
• direct comparison of predicted vs actualcauses
  (radiotherapy)
• components combined in a Healthcare Safety
  Management System (convergent approach)
• evaluating major interventions (impact of IT on
  medication safety)

3
retrospective risk analysis PRISMA- medical

• (voluntary) incident reporting and analysis
• learning from actual / reported process
  deviations

4
PRISMA-Medical

• Prevention and Recovery Information System for
  Monitoring and Analysis
• Three subsequent steps
• Description by means of causal trees
• Classification according to the Eindhoven
  Classification Model (medical version)
• Determination of countermeasures by means of the
  Classification/Action Matrix

5
Causal tree example
O
T
H
T
O
6
Eindhoven Classification Model
-(medicalversion)
7
Database

• Root causes for failure ?failure profile
• Root causes for recovery ?recovery profile
• Context variables ?black-spot analysis

8
PRISMA failure profile hospital
medication errors
9
Classification/Action Matrix
ECM code Design Technology/work- place Procedures Information and Commu nication Training Motiva tion Escala tion Reflection
T-EX
TD
TC
TM
O-EX
OK
OP
OM
OC
H-EX
HK_ NO
HR_
HS_ NO
10
predictive risk analysis HFMEA / SAFER

• series of group meetings to build a set of
  failure scenarios for a (small) process of care
  what may go wrong why what to do about it
• pro-active appeal

11
Healthcare Failure Mode and Effect Analysis
(HFMEA)

• A systematic approach to identify and prevent
  product and process problems before they occur
• Developed by the "VA National Center for Patient
  Safety"
• (http//www.patientsafety.gov/)

12
Relevance of predictive risk analysis

• Retrospective (incident) analysis takes place
  after incidents did occur ?
  hindsight bias
• Because of underreporting, biases can arise in
  incident databases ?
  identification of "missing risks"
  

13
Definitions

• Failure Mode Different ways that a process or
  subprocess can fail to provide the anticipated
  result (i.e. think of it as what could go wrong)
• Prescribing the wrong dose
• Failure Mode Cause Different reasons as to why a
  process or subprocess would fail to provide the
  anticipated result (i.e. think of it as why it
  would go wrong)
• Miscalculation

14
HFMEA process

• Step 1 Define the topic
• Step 2 Assemble the team
• Step 3 Graphically describe the process
• Step 4 Conduct a hazard analysis
• Step 5 Identify actions and outcome measures

15
examples of integration (1)

• direct comparison of predicted (HFMEA) vs
  reported causes
• user problems with a new radiation therapy
  technology
• both types of failure causes expressed in the
  same PRISMA-medical classification
  (sub-)categories

16
PRISMA vs HFMEA main categories


17
PRISMA vs HFMEA subcategories
Frequency category HFMEA less than yearly yearly monthly weekly
Weight-factor ( translation to 9 months) 0,1 0,89 9 36
18
examples of integration (2)

• combining retrospective and predictive components
  in an overall Healthcare Safety Management System
• convergent approach of two imperfect risk
  identification methodologies
• mutual checks, comparisons, and inputs
• possible

19
examples of integration (2)continued

• are repeatedly predicted problems (failure modes)
  ever being reported?
• can frequently reported problems help to select
  suitable processes for HFMEA and generate
  realistic failure modes?
• can frequently predicted causes steer the
  information gathering after an initial report?
• are proposed interventions for predicted vs
  reported causes similar?
• etc

20
examples of integration (3)

• developing a process-based evaluation methodology
  for major (patient safety) interventions
• predicting and monitoring the impact of IT on
  medication safety

21
Medication safety definitions
Van den Bemt et al., 2000
22
Medication errors causes (1)

• Handwritten prescriptions and drug orders
• Look-alike drug names
• Sound-alike drugs and verbal orders
• Use of abbreviations
• Similar packaging and labelling
• Inadequate training and supervision
• Staff shortages
• Overwork and fatigue

Habraken, 2004
23
Medication errors causes (2)
24
IT possibilities and problems
25
IT possibilities and problems
56
6
4
34
Bates et al., 1995 Bates, 2000
26
IT possibilities and problems
IT application PROS CONS
CPOE Legible prescriptions no handwriting required Possibility of substitution errors
CPOE Data entry only necessary once Failure to warn
CPOE Exchange of data is easy Failure to warn
Computerised decision support Drug information Risk of low vigilance and overtrust
Computerised decision support Patient-specific information and advice Risk of low vigilance and overtrust
Bar coding Ensure five "rights" right drug, right patient, right dose, right route, right time Degraded coordination and communication
Computerised medical record Legible prescriptions no handwriting required Possibility of substitution errors
Computerised medical record Data entry only necessary once Possibility of substitution errors
Computerised medical record Exchange of data is easy Possibility of substitution errors
Habraken and Van der Schaaf, 2006
27
Barriers to the implementation of IT

• Significant costs technical, process redesign,
  and implementation and support
• Cultural obstacles resistance to change
• Privacy and protection of (patient) data
• Lack of data standards
• Lack of (clinical) evaluation

Habraken, 2004
28
Evaluation of effects and impact of IT PRISMA
and HFMEA

• Not only outcomes of care but also the mechanisms
  underlying those outcomes
• Impact of IT on "error recovery "
• Detection
• Diagnosis
• Correction
• of earlier errors / deviations

29
Evaluation of effects and impact of IT PRISMA

• PRISMA can be used to obtain an insight into the
  behavioural mechanisms underlying medication
  errors
• Classification/Action Matrix enables us to
  predict which types of human behaviour will be
  influenced by IT

30
Evaluation of effects and impact of IT PRISMA
ECM code Design/ Technol Procedures Information and Communication Training Motivation Escalation Reflection
T-EX
TD
TC
TM
O-EX
OK
OP
OM
OC
H-EX
HK_ NO
HR_
HS_ NO
31
Evaluation of effects and impact of IT PRISMA

• IT applications would fall in two categories
  "technology" and "information and communication"
• In case of improved technology ? reduction of
  skill based human errors
• In case of information and communication support
  ? reduction of knowledge based errors
• BUT rule based human errors would not be
  influenced by IT

32
Evaluation of effects and impact of IT PRISMA
and HFMEA

• Theoretical predictions could be reinforced by
  predictive risk analysis, such as HFMEA
• Empirical evaluation of actual impact of IT by
  means of intensified incident reporting
• Comparison of causal patterns of incidents that
  occur before, during, and after the IT
  intervention

33
Conclusion (1)

• IT often mentioned as prerequisite for reduction
  of medication errors
• Results regarding effects of IT vary greatly
• Effects of IT on behavioural mechanisms are
  not/hardly taken into account
• PRISMA and HFMEA offer a framework for in-depth
  analysis of impact of IT

34
Conclusion (2)

• Two types of predictions can be made of expected
  effects of IT on error and error recovery
• Theoretical predictions by means of PRISMA
• HFMEA scenario-based predictions
• Intensified incident reporting and analysis would
  enable a fast comparison between predicted and
  actual effects
• On-line corrections of implementation process
  could prevent actual adverse events

35
Thank you for your attention