PCI Risk Model Comparisons - PowerPoint PPT Presentation


PPT – PCI Risk Model Comparisons PowerPoint presentation | free to download - id: 7c6af9-N2RiN


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

PCI Risk Model Comparisons


An alternative model for case level estimation of pre-procedure PCI Mortality Risk Michael Blechner, M.D. Michael Matheny, M.D. Goal Explore alternative models for ... – PowerPoint PPT presentation

Number of Views:49
Avg rating:3.0/5.0
Slides: 39
Provided by: Maboroshi
Learn more at: http://www.matheny.info
Tags: pci | cabg | comparisons | model | risk


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: PCI Risk Model Comparisons

PCI Risk Model Comparisons
  • An alternative model for case level estimation of
    pre-procedure PCI Mortality Risk
  • Michael Blechner, M.D.
  • Michael Matheny, M.D.

  • Explore alternative models for pre-intervention
    risk assessment in patients being considered for
    a percutaneous coronary intervention (PCI)

PCI Background
  • A myocardial infarction is typically due to a
    chronic narrowing in one or more of the blood
    vessels supplying the heart combined with an
    acute obstruction at that site
  • Treatment options
  • surgical bypass of the region or
  • PCI in which a catheter is fed through the vessel
    and the temporary inflation of a small balloon
    widens the vessel lumen
  • Both techniques can also be performed on patients
    with evidence of chronic narrowing but who have
    not yet had an MI

Pre-intervention Risk Assessment
  • Risk of death in PCI varies widely based on
  • Providing case level estimations can greatly aid
    patient and physician decision-making
  • Estimates by physician experts are inaccurate at
    the high and low ends of the probability spectrum

History of PCI Risk Assessment
  • PCI is a high volume procedure with significant
    morbidity mortality
  • Early attempts to develop statistical models of
    risk were limited by non-standardized data
  • The American College of Cardiologists (ACC) has
    since mandated that accredited centers maintain
    detailed data on all PCI patients
  • Track outcomes with respect to predictor variables

(No Transcript)
Current Risk Model Standard Logistical Regression
  • Type of generalized non-linear model
  • Used in analysis of a binary outcome
  • Bounded by 0 and 1
  • Produces Coefficients/Odds Ratios and an
  • Variable selection
  • From All Available Data
  • Known Risk Factors from Prior Studies
  • Selected Subset of data based on Study Design

Summary Logistic Regression
  • Advantages
  • Straightforward
  • Intuitive results in the form of odds ratios
  • Disadvantages
  • Presumes independence between variables
  • Difficulty in applying model to different
    geographies and time periods
  • Missing data points assumed to be negative

(No Transcript)
Alternative Risk Model Bayesian Network (BN)
  • Advantages
  • Can incorporate variable co-dependencies
  • Provides a method for the estimation of unknown
    variables, i.e., reasoning under uncertainty
  • Easy to retrain
  • Provides a graphical representation of variable
  • Disadvantages
  • Accuracy of network is dependent on nodal

Bayesian Network Methodology
  • Directed acyclic graph (DAG) consisting of
  • Nodes
  • Directed links between nodes
  • Conditional probability tables (CPT)
  • Assumptions of conditional dependence and
    independence based on expert opinion or machine
    learning algorithms
  • Prior and conditional probabilities are developed
    using existing data or expert opinion

Study Hypothesis
  • A BN will provide a better case level estimation
    of risk than a model developed using standard LR

(No Transcript)
Source Data
  • Brigham Womens Hospital
  • Interventional Cardiology Database
  • January 1, 2002 October 30, 2004
  • 5383 Cases
  • 2/3 Training Cases (3588)
  • 1/3 Test Cases (1795) beginning October 27, 2003

Sample Demographics Overview

0-49 590 10.96
50-59 1167 21.68
60-69 1497 27.81
70-79 1398 25.98
80 652 12.22
Diabetic 1721 31.98
Hypertensive 4083 75.86
Hyperlipidemia 3737 69.44
Prior PCI 1822 33.85
Salvage Procedure 24 0.45
Cardiogenic Shock 98 1.82
Hemodynamic Instability 265 4.92
Death 78 1.45
Variable Selection
Age Hyperlipidemia Hx COPD
Gender HTN Hx CVD
Race Diabetes Hx PVD
Cardiogenic Shock Creatinine Thrombolytic
Cardiac arrest Hx CHF BMI
Hemodynamic instability CHF EF
Procedure urgency Prior MI AMI
Smoker Prior CABG
Logistic Regression Model Development
  • Backwards Stepwise Technique
  • Exclusion Threshold P gt 0.10
  • Inclusion Threshold P gt 0.05
  • Variables Evaluated 35
  • Continuous Variables Discretized
  • STATA 8.2 (College Station, Texas)

(No Transcript)
Methods BN Naïve BN
Netica Release 2.17 (Norsys Software Corp.,
Vancouver, BC, Canada)
Methods BN Naïve Hidden BN
Methods BN Non-Naïve BN
  • A models ability to distinguish between patients
    who die and those who survive
  • Although a Model calculates an outcome
    probability, the classification of a case into
    death vs. survival is based on an arbitrary
  • This threshold determines the sensitivity and
    specificity of the prediction
  • ROC curves graph the sensitivity vs.
    1-specificity at different thresholds
  • The discriminatory performance of the model is
    estimated by the area under the ROC curve

  • Measures how close the models estimates are to
    the true probability
  • The true probability is the probability of
    death for a similar patient population
  • Provides an estimation of case level accuracy
  • Accuracy of the statement The risk of death from
    PCI in patients like you is 1 in 1,000.
  • Hosmer-Lemeshows Goodness-of-Fit Test
  • Ranks population by probability estimate
  • Divides population into equal subsegments
  • Calculates how well the observed and expected
    frequencies match

Results BN
(No Transcript)
Results Logistic Regression Model
OR 95 CI P
Age 60-69 21.79 1.88-252.83 0.014
Age 70-79 57.61 4.76-697.21 0.001
Age gt 80 161.36 13.28-1960.76 lt0.001
Prior PCI 0.29 0.09-0.93 0.037
Cardiogenic Shock 25.03 9.50-65.97 lt0.001
Cardiac Arrest 7.12 1.68-30.13 0.008
Hemodynamic Instability 3.69 1.49-9.17 0.005
Salvage Procedure 201.56 13.54-3001.07 lt0.001
Diabetic 2.43 1.15-5.10 0.019
Hyperlipidemia 0.18 0.08-0.41 lt0.001
All Model Test ROC Summary
Training Set Training Set Training Set
Model ROC 95 CI
Logistic Regression 0.94 0.91-0.98
Naïve Bayesian Network 0.93 0.88-0.97
Naïve Hidden Bayesian Network 0.91 0.86-0.96
Non-Naïve Bayesian Network 0.97 0.95-0.98
Test Set Test Set Test Set
Model ROC 95 CI
Logistic Regression 0.86 0.80-0.93
Naïve Bayesian Network 0.89 0.82-0.97
Naïve Hidden Bayesian Network 0.89 0.82-0.96
Non-Naïve Bayesian Network 0.85 0.76-0.93
All Models Training ROC Comparison
All Models Test ROC Comparison
All Models Pair-wise ROC Evaluation
Diff P
Logistic Regression vs Naïve Bayes -0.031 0.373
Logistic Regression vs Naïve Hidden Bayes -0.026 0.277
Logistic Regression vs Non-Naïve Bayes 0.014 0.686
Naïve Bayes vs Naïve Hidden Bayes 0.005 0.877
Naïve Bayes vs Non-Naïve Bayes 0.045 0.277
Naïve Hidden Bayes vs Non-Naïve Bayes 0.040 0.178
All Model HL Good-Fit Summary
Training Set Training Set Training Set
Model HL Chi2 Prob gt chi2
Logistic Regression 9.48 0.219
Naïve Bayesian Network 11.94 0.154
Naïve Hidden Bayesian Network 20.40 0.009
Non-Naïve Bayesian Network 24.97 0.002
Test Set Test Set Test Set
Model HL Chi2 Prob gt chi2
Logistic Regression 18.16 0.011
Naïve Bayesian Network 4.91 0.768
Naïve Hidden Bayesian Network 16.3 0.038
Non-Naïve Bayesian Network 18.85 0.016
Calibration Plot
(No Transcript)
  • Discrimination
  • All Models had excellent performance
  • None of the models was significantly different in
  • Calibration
  • Two models achieved calibration on the training
    set Logistic Regression Naïve Bayes
  • The only model to retain calibration on the test
    set was the Naïve Bayes Model

  • The CPTs for hidden nodes within our BNs were
    built using a machine learning algorithm
  • Data reporting and database quality would be
    expected to improve over time
  • Ambiguity between absent and negative values
    for some database fields
  • Attempts to develop more realistic Bayesian
    Networks were limited by software failures

Causal BN
  • Calibration is essential for any test where case
    level accuracy is important
  • The only model that retained calibration with the
    test set was the naïve BN
  • This study supports the use of a Naïve Bayesian
    Network for case level estimation in
    pre-procedural PCI risk assessment as an
    alternative to logistic regression

The end
About PowerShow.com