LearningBased Project Reviews: Report on Year 1 on Phase 2

1
Learning-Based Project ReviewsReport on Year 1
on Phase 2

• Presented to
• CPMR
• Presented by
• Tim Kotnour Catherine Vergopia
• June 26, 2006

2
Agenda

• Summary Progress and Accomplishments
• Summarize progress to date.
• Demonstrate responsiveness to original
  selection/evaluation criteria/award letter.
• The identified NASA/LSP specific challenge
• The refined scope
• The products to date
• Identify successes/accomplishments in such areas
  as
• NASA collaboration/teaming
• Early impacts, spin-offs, related research
• Publications, presentations
• Graduate student participation, degrees awarded.
• Deviation from original plan and steps taken for
  corrective action (if appropriate).
• Plan for Phase II Year 2 Execution
• Expected Outcome and Anticipated Impact Within
  NASA
• Issues

3
Summary of Progress to Date

• We are working an important problem
• NASA governance model
• 7120.5d rewrite
• NASA look at reviews
• KSC engineering reorganization
• We are increasing our understanding
• Refining scope to align to LSP and practicing
  project manager needs
• Deeper knowledge about reviews
• New, emerging questions for future research
• We are developing processes and tools
• See details on 7 product suites
• We are impacting LSP
• Facilitated POP lessons learnedimpact practices
• Leading and integrating results into LSPs
  emerging Flight Project Practices Bookversion
  1.0 due December 2006
• We are impacting NASA

4
We Worked a Post-Mortem Lessons Learned Process
for a NASA Product and a Case Study for a
Research Product.
Youll see the connection to the theoretical
framework later in the discussion
5
Agenda

• Summary Progress and Accomplishments
• Summarize progress to date.
• Demonstrate responsiveness to original
  selection/evaluation criteria/award letter.
• The identified NASA/LSP specific challenge
• The refined scope
• The products to date
• Identify successes/accomplishments in such areas
  as
• NASA collaboration/teaming
• Early impacts, spin-offs, related research
• Publications, presentations
• Graduate student participation, degrees awarded.
• Deviation from original plan and steps taken for
  corrective action (if appropriate).
• Plan for Phase II Year 2 Execution
• Expected Outcome and Anticipated Impact Within
  NASA
• Issues

6
Summary of Accomplishments vs. Award Letter
7
The NASA/LSP Specific Review Learning Challenge

• Where/how do we learn in a project?
• Phase gates
• Routine project management
• Daily activity
• Defined learning activities
• If we assume an average LSP mission life-cycle
  7 years
• 18 formal reviews phase gates (pre 7120.5d)
• 84 potential monthly program reviews
• 315 weekly internal reviews
• How can we support learning in the project
  environment?
• Increase recognition that the basis for learning
  is the daily activity
• Determine how to connect the daily learning
  opportunities results to where learning occurs
• Provides tools to enable this to happen

8
The Research Scope Core Questions.

• Theoretical Questions
• What project reviews lead to enhanced project
  performance?
• What project reviews are performed?
• What project review characteristics are
  associated with good reviews?
• How do we drive learning in project reviews?
• How can we improve project reviews?
• What processes and tools can we use to improve
  project reviews?
• How does the improvement process have an impact?
• Did the improvement process have an impact?
• What processes and tools are available to drive
  good project reviews?
• What are the best practices?
• What questions should we ask? What conversations
  should we have?

• Project Manager Questions
• What reviews do we perform?
• How well are our project reviews?
• How can we improve our project reviews?
• What is our overall maturity for project reviews?
• What specific areas do we need to focus to
  improve our project reviews?

9
Were Producing Seven Groups of Products
10
Core AssumptionThe Right Questions Drive the
Right Reviews.
Questions
Conversation
Information
Decisions

• Actions
• No changes
• Changes to the project
• Changes to project procedures
• Changes to organizational procedure

Data
Nancys model impacts
11
Project Reviews, PDSA the Systems Engineering
Phases

• Types of Reviews
• Initial review to determine if able to proceed
• Routine (e.g., weekly project review)
• End of phase/phase gate
• End of projectpost mortem

12
We Move from One Level of Learningto the Next
Based on Performance.
Gap?
Gap?
Gap?
Level I
Level II
Level III
Level IV

• Organizational procedures

• Status

• Status challenges

• Project procedures

13
Understanding Review Typesand their Learning
Role.
14
Describing the Relationships

• PR Maturity
• Stage 1
• Stage 2
• Stage 3
• Stage 4
• Stage 5

• Learning Level
• Status
• Status Challenges
• Project procedures
• Organizational procedures

• Impacts
• Project
• ? P(project success)
• ? knowledge or risks
• ? Cost, schedule, technical gaps
• ? P(other project success)
• ? Organizational capability

(Survey aligned to above model)
15
Initial PR Maturity Model
16
Process to Improve the Review Process
Define reviews

• Understand current level of learning in reviews
• Observe reviews
• Complete assessments

Understand current maturity level in reviews
Decide where want/need reviews to be

• Set priorities on reviews to improve
• Prioritize importance of reviews
• Prioritize impact/control of changing the review
• Gaps vs. a radar chart

Define actions to change reviews
Understand enablers barriers to reviews
Cross-cutting change management (leadership,
involvement, specific sessions)
17
Tools to Use to Improve the Review Process.
18
Process to Define Reviews.
Precipitator (What caused to have)
Purpose (Desired Outcome)
People (Individuals included in group)
Process (All aspects of the group process)
Problem (Task problem)
Participation (Desired involvement)
Products (Outcomes outputs)
(MSL)
19
Tools for Conducting Project Reviews Framing the
Questions to Ask for Routine Project Reviews.
(Note remember core assumption about the role of
questions)
20
Tools for Conducting Project ReviewsFraming the
Questions to Ask for Post-Mortem Project Reviews.
(Note remember core assumption about the role of
questions)
21
Agenda

• Summary Progress and Accomplishments
• Summarize progress to date.
• Demonstrate responsiveness to original
  selection/evaluation criteria/award letter.
• The identified NASA/LSP specific challenge
• The refined scope
• The products to date
• Identify successes/accomplishments in such areas
  as
• NASA collaboration/teaming
• Early impacts, spin-offs, related research
• Publications, presentations
• Graduate student participation, degrees awarded.
• Deviation from original plan and steps taken for
  corrective action (if appropriate).
• Plan for Phase II Year 2 Execution
• Expected Outcome and Anticipated Impact Within
  NASA
• Issues

22
Summary Successes Accomplishments
23
Agenda

• Summary Progress and Accomplishments
• Summarize progress to date.
• Demonstrate responsiveness to original
  selection/evaluation criteria/award letter.
• The identified NASA/LSP specific challenge
• The refined scope
• The products to date
• Identify successes/accomplishments in such areas
  as
• NASA collaboration/teaming
• Early impacts, spin-offs, related research
• Publications, presentations
• Graduate student participation, degrees awarded.
• Deviation from original plan and steps taken for
  corrective action (if appropriate).
• Plan for Phase II Year 2 Execution
• Expected Outcome and Anticipated Impact Within
  NASA
• Issues

24
Year 2 is Building from the Foundation of Year 1.

• Year 1
• Solidified LSP partnership
• Understand challenge better
• Develop initial processes and tools
• Developed survey
• Conducted applications (POP and GOES-N LL)
• Established contact with GSFC

• Refined
• End Products
• Literature basis
• Characteristics
• Process for improving
• Tools for improving
• Process for conducting
• Tools for conducting
• Direct products to LSP

25
The Data Helps Answer Questions.

• What PR characteristics are associated with good
  reviews?
• K-1, K-2, G-1, G-2
• Did the improvement process have an impact?
• K-1, K-2
• K-3
• What are the best practices?
• K-1, G-1

26
Agenda

• Summary Progress and Accomplishments
• Summarize progress to date.
• Demonstrate responsiveness to original
  selection/evaluation criteria/award letter.
• The identified NASA/LSP specific challenge
• The refined scope
• The products to date
• Identify successes/accomplishments in such areas
  as
• NASA collaboration/teaming
• Early impacts, spin-offs, related research
• Publications, presentations
• Graduate student participation, degrees awarded.
• Deviation from original plan and steps taken for
  corrective action (if appropriate).
• Plan for Phase II Year 2 Execution
• Expected Outcome and Anticipated Impact Within
  NASA
• Issues

27
Expected Impacts to NASA
Enhanced project reviews Enhanced projects
28
Agenda

• Summary Progress and Accomplishments
• Summarize progress to date.
• Demonstrate responsiveness to original
  selection/evaluation criteria/award letter.
• The identified NASA/LSP specific challenge
• The refined scope
• The products to date
• Identify successes/accomplishments in such areas
  as
• NASA collaboration/teaming
• Early impacts, spin-offs, related research
• Publications, presentations
• Graduate student participation, degrees awarded.
• Deviation from original plan and steps taken for
  corrective action (if appropriate).
• Plan for Phase II Year 2 Execution
• Expected Outcome and Anticipated Impact Within
  NASA
• Issues

29
Issues

• Validating survey, processes, and tools with
  advisory board
• Balancing the benefits and challenges of the
  action research approach
• Balancing impacts within LSP and GSFC
• Working the relationship with GSFC
• Integrating this research within the LSP FPPB
  efforts
• Need to integrate after get 1st draft of the book
  completed
• Aligning the work and timing with the needs for
  the dissertation process