PCI Risk Model Comparisons - PowerPoint PPT Presentation

Loading...

PPT – PCI Risk Model Comparisons PowerPoint presentation | free to download - id: 7c6af9-N2RiN



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

PCI Risk Model Comparisons

Description:

An alternative model for case level estimation of pre-procedure PCI Mortality Risk Michael Blechner, M.D. Michael Matheny, M.D. Goal Explore alternative models for ... – PowerPoint PPT presentation

Number of Views:49
Avg rating:3.0/5.0
Slides: 39
Provided by: Maboroshi
Learn more at: http://www.matheny.info
Category:
Tags: pci | cabg | comparisons | model | risk

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: PCI Risk Model Comparisons


1
PCI Risk Model Comparisons
  • An alternative model for case level estimation of
    pre-procedure PCI Mortality Risk
  • Michael Blechner, M.D.
  • Michael Matheny, M.D.

2
Goal
  • Explore alternative models for pre-intervention
    risk assessment in patients being considered for
    a percutaneous coronary intervention (PCI)

3
PCI Background
  • A myocardial infarction is typically due to a
    chronic narrowing in one or more of the blood
    vessels supplying the heart combined with an
    acute obstruction at that site
  • Treatment options
  • surgical bypass of the region or
  • PCI in which a catheter is fed through the vessel
    and the temporary inflation of a small balloon
    widens the vessel lumen
  • Both techniques can also be performed on patients
    with evidence of chronic narrowing but who have
    not yet had an MI

4
Pre-intervention Risk Assessment
  • Risk of death in PCI varies widely based on
    co-morbidities
  • Providing case level estimations can greatly aid
    patient and physician decision-making
  • Estimates by physician experts are inaccurate at
    the high and low ends of the probability spectrum

5
History of PCI Risk Assessment
  • PCI is a high volume procedure with significant
    morbidity mortality
  • Early attempts to develop statistical models of
    risk were limited by non-standardized data
  • The American College of Cardiologists (ACC) has
    since mandated that accredited centers maintain
    detailed data on all PCI patients
  • Track outcomes with respect to predictor variables

6
(No Transcript)
7
Current Risk Model Standard Logistical Regression
(LR)
  • Type of generalized non-linear model
  • Used in analysis of a binary outcome
  • Bounded by 0 and 1
  • Produces Coefficients/Odds Ratios and an
    intercept
  • Variable selection
  • From All Available Data
  • Known Risk Factors from Prior Studies
  • Selected Subset of data based on Study Design

8
Summary Logistic Regression
  • Advantages
  • Straightforward
  • Intuitive results in the form of odds ratios
  • Disadvantages
  • Presumes independence between variables
  • Difficulty in applying model to different
    geographies and time periods
  • Missing data points assumed to be negative
    findings

9
(No Transcript)
10
Alternative Risk Model Bayesian Network (BN)
  • Advantages
  • Can incorporate variable co-dependencies
  • Provides a method for the estimation of unknown
    variables, i.e., reasoning under uncertainty
  • Easy to retrain
  • Provides a graphical representation of variable
    relationships
  • Disadvantages
  • Accuracy of network is dependent on nodal
    connections

11
Bayesian Network Methodology
  • Directed acyclic graph (DAG) consisting of
  • Nodes
  • Directed links between nodes
  • Conditional probability tables (CPT)
  • Assumptions of conditional dependence and
    independence based on expert opinion or machine
    learning algorithms
  • Prior and conditional probabilities are developed
    using existing data or expert opinion

12
Study Hypothesis
  • A BN will provide a better case level estimation
    of risk than a model developed using standard LR
    techniques

13
(No Transcript)
14
Source Data
  • Brigham Womens Hospital
  • Interventional Cardiology Database
  • January 1, 2002 October 30, 2004
  • 5383 Cases
  • 2/3 Training Cases (3588)
  • 1/3 Test Cases (1795) beginning October 27, 2003

15
Sample Demographics Overview

Age
0-49 590 10.96
50-59 1167 21.68
60-69 1497 27.81
70-79 1398 25.98
80 652 12.22
Diabetic 1721 31.98
Hypertensive 4083 75.86
Hyperlipidemia 3737 69.44
Prior PCI 1822 33.85
Salvage Procedure 24 0.45
Cardiogenic Shock 98 1.82
Hemodynamic Instability 265 4.92
Death 78 1.45
16
Variable Selection
Age Hyperlipidemia Hx COPD
Gender HTN Hx CVD
Race Diabetes Hx PVD
Cardiogenic Shock Creatinine Thrombolytic
Cardiac arrest Hx CHF BMI
Hemodynamic instability CHF EF
Procedure urgency Prior MI AMI
IABP Prior PCI
Smoker Prior CABG
17
Logistic Regression Model Development
  • Backwards Stepwise Technique
  • Exclusion Threshold P gt 0.10
  • Inclusion Threshold P gt 0.05
  • Variables Evaluated 35
  • Continuous Variables Discretized
  • STATA 8.2 (College Station, Texas)

18
(No Transcript)
19
Methods BN Naïve BN
Netica Release 2.17 (Norsys Software Corp.,
Vancouver, BC, Canada)
20
Methods BN Naïve Hidden BN
21
Methods BN Non-Naïve BN
22
Discrimination
  • A models ability to distinguish between patients
    who die and those who survive
  • Although a Model calculates an outcome
    probability, the classification of a case into
    death vs. survival is based on an arbitrary
    threshold
  • This threshold determines the sensitivity and
    specificity of the prediction
  • ROC curves graph the sensitivity vs.
    1-specificity at different thresholds
  • The discriminatory performance of the model is
    estimated by the area under the ROC curve

23
Calibration
  • Measures how close the models estimates are to
    the true probability
  • The true probability is the probability of
    death for a similar patient population
  • Provides an estimation of case level accuracy
  • Accuracy of the statement The risk of death from
    PCI in patients like you is 1 in 1,000.
  • Hosmer-Lemeshows Goodness-of-Fit Test
  • Ranks population by probability estimate
  • Divides population into equal subsegments
  • Calculates how well the observed and expected
    frequencies match

24
Results BN
25
(No Transcript)
26
Results Logistic Regression Model
OR 95 CI P
Age 60-69 21.79 1.88-252.83 0.014
Age 70-79 57.61 4.76-697.21 0.001
Age gt 80 161.36 13.28-1960.76 lt0.001
Prior PCI 0.29 0.09-0.93 0.037
Cardiogenic Shock 25.03 9.50-65.97 lt0.001
Cardiac Arrest 7.12 1.68-30.13 0.008
Hemodynamic Instability 3.69 1.49-9.17 0.005
Salvage Procedure 201.56 13.54-3001.07 lt0.001
Diabetic 2.43 1.15-5.10 0.019
Hyperlipidemia 0.18 0.08-0.41 lt0.001
27
All Model Test ROC Summary
Training Set Training Set Training Set
Model ROC 95 CI
Logistic Regression 0.94 0.91-0.98
Naïve Bayesian Network 0.93 0.88-0.97
Naïve Hidden Bayesian Network 0.91 0.86-0.96
Non-Naïve Bayesian Network 0.97 0.95-0.98
Test Set Test Set Test Set
Model ROC 95 CI
Logistic Regression 0.86 0.80-0.93
Naïve Bayesian Network 0.89 0.82-0.97
Naïve Hidden Bayesian Network 0.89 0.82-0.96
Non-Naïve Bayesian Network 0.85 0.76-0.93
28
All Models Training ROC Comparison
29
All Models Test ROC Comparison
30
All Models Pair-wise ROC Evaluation
Diff P
Logistic Regression vs Naïve Bayes -0.031 0.373
Logistic Regression vs Naïve Hidden Bayes -0.026 0.277
Logistic Regression vs Non-Naïve Bayes 0.014 0.686
Naïve Bayes vs Naïve Hidden Bayes 0.005 0.877
Naïve Bayes vs Non-Naïve Bayes 0.045 0.277
Naïve Hidden Bayes vs Non-Naïve Bayes 0.040 0.178
31
All Model HL Good-Fit Summary
Training Set Training Set Training Set
Model HL Chi2 Prob gt chi2
Logistic Regression 9.48 0.219
Naïve Bayesian Network 11.94 0.154
Naïve Hidden Bayesian Network 20.40 0.009
Non-Naïve Bayesian Network 24.97 0.002
Test Set Test Set Test Set
Model HL Chi2 Prob gt chi2
Logistic Regression 18.16 0.011
Naïve Bayesian Network 4.91 0.768
Naïve Hidden Bayesian Network 16.3 0.038
Non-Naïve Bayesian Network 18.85 0.016
32
Calibration Plot
33
(No Transcript)
34
Discussion
  • Discrimination
  • All Models had excellent performance
  • None of the models was significantly different in
    performance
  • Calibration
  • Two models achieved calibration on the training
    set Logistic Regression Naïve Bayes
  • The only model to retain calibration on the test
    set was the Naïve Bayes Model

35
Limitations
  • The CPTs for hidden nodes within our BNs were
    built using a machine learning algorithm
  • Data reporting and database quality would be
    expected to improve over time
  • Ambiguity between absent and negative values
    for some database fields
  • Attempts to develop more realistic Bayesian
    Networks were limited by software failures

36
Causal BN
37
Conclusions
  • Calibration is essential for any test where case
    level accuracy is important
  • The only model that retained calibration with the
    test set was the naïve BN
  • This study supports the use of a Naïve Bayesian
    Network for case level estimation in
    pre-procedural PCI risk assessment as an
    alternative to logistic regression

38
The end
About PowerShow.com