Choices and Outcomes: The Effects of Improvement Project Portfolio Choices on Clinical Outcomes

1
Choices and Outcomes The Effects of Improvement
Project Portfolio Choices on Clinical Outcomes

• Anita L. Tucker
• Assistant Professor, U. of Pennsylvania
• Senior Fellow, Leonard Davis Institute of Health
  Economics
• Ingrid M. NembhardDoctoral Candidate, Harvard
  Business SchoolHarvard Graduate School of Arts
  and Sciences
• KLIC 4 INFORMS ConferenceNovember 6, 2006

Financial Support from HBS DOR, Whartons Fishman
Davidson Center In Collaboration with Jeffrey
Horbar, Richard Bohmer and Amy Edmondson
2
Background Health Care in America
The Data
Publicity
1999 Institute of Medicine (IOM) reports 100,000
Americans die annually from preventable medical
errors
1996 Dartmouth Atlas Project shows
inappropriately geographic variations in care

2000 IOM finds quality problems are a systemic
property, requiring process improvement
1998 National Roundtable reports serious and
widespread problems exist in American medicine
. . Problems of underuse, overuse, misuse . .
Quality of care is the problem.
2003 RAND Study reports only 55 of patients
receive the recommended care for their condition

Need for improvement projects in health care
organizations
3
Improvement Projects

• Improvement Projects solve complex
  organizational problems through the work of
  formal teams that use a structured improvement
  method (Christianson et al., 2005 p. 610)
• Healthcare Project Examples
  Target Area
• Increase handwashing
  (Reduce Infections)
• Reduce heelsticks in Neonates
  (Pain management)
• Increase collaboration among MDs and RNs (Staff
  Retention)
• Portfolio of Improvement Projects An
  organizations set of improvement projects that
  are in progress at the same time and draw on the
  same limited set of human, managerial and
  financial resources
  (Cooper, Edgett Kleinschmidt,
  1999 Wheelwright Clark, 1992)

4
Research Question

• How should health care organizations structure
  their portfolios of improvement projects to
  achieve better outcomes?

• Hypotheses about Portfolio Choices
• Number of projects ()
• Concentration within a target area ()
• Level of evidence for portfolio ()
• Novices should start with clinically-oriented
  portfolio ()
• Extent of physician involvement ()

5
Research Setting Vermont Oxford Network

• Collaborative of Neonatal Intensive Care Units
  (NICU)
• 44 NICUs working together for 2 ½ years (Apr 2002
  - Oct 2004)
• Identified 7 target areas for improvement and 93
  improvement projects across those areas
• Met twice a year to learn QI methods (PDSA
  cycles), work on developing best practice
  guidelines and share experiences
• In between meetings, implemented practices,
  conducted site visits to other NICUs and phone
  conferences

Horbar, J. D. et al, 2001. Collaborative quality
improvement for neonatal intensive care.
Pediatrics 107 (1) 14-22.
6
Portfolio Project Options
Target Area Aim of practices Sample practice project(s)
Pain Sedation Decrease mean pain score by 50 during NICU stay 10 Pain management during heelsticks Intubation of medications
Infection Control Decrease hospital-acquired infections by 25-50 over 2 years 7 Promote hand washing education and practices to prevent nosocomial infection
Respiratory Care Decrease chronic lung disease by 10, and decrease oxygen days, ventilator days, steroid use 15 Vitamin A supplementation Ventilation by Tidal Volume Monitoring
OB/NICU Relations Improve maternal newborn caregiver collaboration periviability, delivery response, comfort care 6 Design process to increase collaboration and communication during high-risk delivery
Staff Retention Decrease staff turnover by 50 5 Improve nurse-physician collaboration
Family-Centered Care Enhance ability to co-ordinate and deliver care so the infant and family needs are met 27 Provide resource materials that depict newborn premature infants maturational and postnatal environment.
Discharge Planning Embed discharge planning into all aspect of patient care communication 23 Develop "trigger point" checklist for discharge teaching
93 Potential Projects in the Portfolio
Clinically-oriented
Operationally-oriented
Each NICU indicated which projects they
included in their project portfolio to the
collaborative sponsor.
7
Unique Portfolios of Practices
Excerpt from the practices from Pain Management
Hospital 100 Implemented/Working on Reducing
frequency of tracheal suctioning, standarized
sucrose analgesia, peripheral vascular
procedures, circumcision, post op pain, weaning
from opiods
VERSUS Hospital 102 Reducing frequency of
heelsticks
8
Evidence-base for the portfolio

• Level of evidence for all projects within each
  target area assessed by target area team using
  Muir-Gray (MG) score to rate articles
• 1 strong evidence from at least one
  systematic review of multiple,
• well-designed, randomized, controlled
  trials
• 2 properly designed randomized control trial
  of appropriate size
• 3 well-designed trials without randomization
  
• 4 well-designed non-experimental trials
  
• 5 opinions of respected authorities, based
  on clinical evidence,
• descriptive studies or reports of expert
  committees
• Evidence base for NICU portfolio the average MG
  score of the portfolio

9
Outcome Standardized Mortality Ratio

• SMR level of analysis Babies nested in NICU
• LOGIT model (clustered by NICU) Outcome Death
  (0,1)
• Independent Variables Established risk factors
  (Zupanic et al. 2006)
• Predict probability of death for each baby
• By NiCU, sum up probability of death, actual
  deaths
• Compute ratio
• SMR lt 1 Outcomes BETTER than expected
• SMR 1 Outcomes equal to expected
• SMR gt 1 Outcomes WORSE than expected

10
Means, SD, and correlations (N27)
plt.1 plt.05 plt.01
11
OLS Regression results (H1, H3, H4)
Outcome measure Improvement SMR mortality (2004-2001)
Control Variables
Staff to beds ratio -0.336 (0.145)
Cardiac surgery provider -0.232 (0.240)
History of quality improvement 0.569 (0.208)
Team size -0.020 (0.023)
Teaching Status 0.189 (0.279)
Independent Variables
Number of projects -0.105 (0.044)
Number of projects squared 0.004 (0.001)
Evidence supporting portfolio -0.526 (0.293)
of MDs on QI team -3.429 (1.507)
Constant 4.139 (1.291)
Adj. R-squared 0.37
F 2.62
Sig 0.04
df 9, 16
N 26, (std error) significant at 10
significant at 5 significant at 1
12
R H2 Concentrating the within a target area helps
Outcome measure Improvement SMR length of stay (2004-2001)

Control Variables
Staff to beds ratio -0.049 (0.036)
Cardiac surgery provider 0.118 (0.071)
History of quality improvement 0.082 (0.048)
Team size 0.008 (0.006)
Teaching Status -0.047 (0.077)
Independent Variables
Number of LOS projects -0.013 (0.006)
Constant -0.007 (0.161)
Adj. R-squared 0.23
F 2.22
Sig 0.09
df 6, 19
N 26, (std error) significant at 10
significant at 5 significant at 1
13
R H5 Initial portfolio orientation matters
Outcome measure Improvement SMR mortality (2004-2001)
Control Variables
Staff to beds ratio -0.251 (0.144)
Cardiac surgery provider
History of quality improvement
Team size
Teaching status
Independent Variables
Number of projects -0.213 (0.082)
Number of projects squared 0.005 (0.002)
Evidence supporting portfolio
of MDs on QI team
Clinically oriented portfolio -1.751 (0.697)
Constant 2.951 (1.008)
Adj. R-squared 0.16
F (df) 7.50 (4,6)
Sig 0.02
N 11 (Robust std error) significant at 10
significant at 5 significant at 1
14
Summary and Implications

• An effective improvement project portfolio
• Includes neither too few or too many projects to
  manage the tradeoff between synergy and
  distraction
• Concentrates its efforts within a target area to
  maximize inter-project learning
• Focuses on operationally-oriented projects which
  build performance-improvement capability
• For novices is clinically-oriented where clearer
  benchmarks are available
• Is led by a team with significant physician
  membership.