Cultural Influence on the Implementation of Lessons Learned in Safety and Risk Management Decisions - PowerPoint PPT Presentation

Loading...

PPT – Cultural Influence on the Implementation of Lessons Learned in Safety and Risk Management Decisions PowerPoint presentation | free to view - id: 96240-ZTk3O



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Cultural Influence on the Implementation of Lessons Learned in Safety and Risk Management Decisions

Description:

... Master's students in Industrial Systems Engineering, undergrad in anthropology and biology ... NSF Engineering Research Center: ... – PowerPoint PPT presentation

Number of Views:111
Avg rating:3.0/5.0
Slides: 39
Provided by: ulrichn
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Cultural Influence on the Implementation of Lessons Learned in Safety and Risk Management Decisions


1
Cultural Influence on the Implementation of
Lessons Learned inSafety and Risk Management
Decisions Isaac Maya, Najmedin Meshkati,
Mansour RahimiDeepak Madabushi, Kevin Pope,
Meredith SchulteCPMR Fellows MeetingJanuary
19-20, 2005
N a t i o n a l S c i e n c e F o u n d a t i o
n E n g i n e e r i n g R e s e a r c h C e n
t e r
2
USC-NASA/CPMR PROJECT MANAGEMENT
  • Project Team consists of PI and 2 Co-PIs with
    complementary skills, 2 Graduate Students, 1 UG
  • Dr. Maya -- 26 years experience in engineering
    RD and executive, program and project management
  • Dr. Meshkati -- experienced engineering
    researcher in reliability, human factors, safety
    culture, risk management of large-scale, complex
    technological systems NASA Faculty Fellow
    2003-04
  • Dr. Rahimi -- experienced engineering researcher
    in system safety, human computer interaction
    (HCI) for advanced technology systems
  • Three Excellent Students -- Two Masters students
    in Industrial Systems Engineering, undergrad in
    anthropology and biology
  • Technical Advisory Board
  • Industrial, academic, and NASA/JPL experience

3
Integrated Media Systems Center
NSF Engineering Research Centera partnership in
pursuit of research and innovation in multimedia
and immersive Internet technologies and their
applications 28 Investigators and 260 students
in partnership with National Science
Foundation University of Southern
California Viterbi School of Engineering
Ranked 6th in US, 135M/yr in CG
funding Annenberg Center for Communication Commer
cial Partners Computer Hardware and Software
Aircraft, Aerospace, Defense
Petroleum, Oil, Gas Telecommunications Entertai
nment Other Government Agencies DARPA, NASA,
JPL, NIMA, ONR, U.S. Army
4
IMSC Program Management Process
  • Application requirements and relevance
  • Collaboration and Integration
  • Research
  • Education and Outreach

5
IMSC Program Management Process for USC-NASA/CPMR
Project
  • NASA/CPMR project requirements and relevance
  • JPL collaboration and integration
  • Project Research
  • Education and Outreach

6
Research Highlights Applicable to NASA/CPMR
Project
IMSC has produced ground breaking results and
fundamental research applicable to NASA/CPMR
efforts
  • immersive audio
  • multichannel and HRTF approaches - holistic DSP
    approach
  • information management and multimedia databases
  • distributed and scalable data analysis and query
  • computer vision
  • computational framework for grouping based on
    tensor voting, tracking for augmented realities
    and SFX
  • graphics animation
  • 3D DSP mesh processing, compression, mesh
    operations, hair modeling and animation

7
Technical Advisory Board for NASA CPMR Project
  • ACADEMIC
  • Dr. Randolph Hall, Professor, Sr. Associate Dean
    of Research, Viterbi School of Engineering, and
    Director of USCs Homeland Security Center (Risk
    and Economic Analysis of Terrorism Events)
  • Dr. Stan Settles, Professor, USC, member of the
    National Academy of Engineering
  • NASA/JPL
  • Dr. Ken Atkins (JPL, Retired), development
    project manager for the fourth mission in NASAs
    cost-capped Discovery program Stardust
  • Dr. Steven Prusha (JPL)
  • Dr. Payman Arabshahi, Senior Research Staff,
    Section 331 (Communication Systems and Research),
    JPL
  • INDUSTRY
  • Mark George, Sr. Manager, RD, Lockheed Martin
    Space Company
  • Eric Honour, President, Honourcode, Inc., past
    President, International Council on System
    Engineering, and Director, Systems Engineering
    Center of Excellence
  • Dr. George Friedman, Retired, Northrop, and
    Founder, past President and Fellow, International
    Council on System Engineering
  • Dr. Milton F. Pope, Systems Engineering
    Directorate, Evolved Expendable Launch Vehicle
    Division, The Aerospace Corporation

8
Collaboration with NASA Personnel and Technical
Advisory Board (TAB)
  • TAB Telecons
  • Dr. Ken Atkins visits at USC and JPL
  • Stephen Prusha, Manager, Strategic Systems
    Technology Program, JPL, information exchanges
  • Dudley Killam, Operations Manager, Office of
    Safety and Mission Success, JPL, visits
  • Carol Dumain, JPL, David Oberhettinger, NG, and
    JPL Lessons Learned Committee
  • Al Gallo, Goddard information exchange
  • Jeanne Holm, Chief Knowledge Architect, and
    Manson Yew, JPL visits
  • Manson Yew and staff visit at JPL
  • Mark George and LM personnel

9
Methodology and Sources
  • Direct discussions with external collaborators
    (NASA/JPL and industry)
  • Extensive literature review and analysis
  • In-depth review of NASA Lessons Learned
    Information System (LLIS)
  • Analysis of LLIS data entries
  • Human Factors studies

10
Summary Findings
  • Mishap investigation boards have identified
  • Technical lessons learned
  • Need for NASA culture change to reap benefits
  • Our preliminary LLIS data analysis shows
  • Culture change to implement LL processes has been
    evolutionary and Center-dependent
  • LLIS use is not measurably tracked
  • Found evidence that system-level management
    change can overcome LL culture change barriers
  • Currently, several processes and protocols in use
  • No NASA-wide, standard, system-level LL template
  • JPL has successfully implemented comprehensive LL
    process addressing technical and culture change
    challenges

11
Summary Findings
  • New NASA Engineering Network (NEN) is in final
    stages of beta testing before release
  • Modernization of LLIS with current IT
  • Addresses IT aspects of LL
  • Limited address of system-level issues
  • Human factors and systems perspectives and
    advanced information management technologies
    offer significant potential for
  • Addressing culture change barriers
  • Enhancing usability of NEN
  • Increasing effectiveness of overall LL process

12
PRESENTATION OUTLINE
  • Introduction
  • Objective and Focus
  • Literature Review Findings
  • Cultural Issues Findings
  • Lessons Learned Systems and NASA LLIS Findings
  • NASA and JPL Management and System Processes for
    Lessons Learned
  • NASA Engineering Network (NEN)
  • Human Factors Studies
  • Education and Outreach
  • Summary Findings
  • Conclusions and Recommendations

13
INTRODUCTION
  • General impression that NASA Lessons Learned from
    past mishaps not being applied effectively in
    subsequent programs/projects
  • Contributing factor could be that NASA culture
    features lead to this result
  • Examine NASAs Lessons Learned Information System
    (LLIS) for data
  • Compare to Lockheed Martin culture, lessons
    learned system and data

14
Objective and Focus of Research
  • Research influence of culture on use of LL system
  • Focus on three of the most frequently cited
    recurring causal factors for project failures
  • Poor team communication
  • Inadequate management review process in all
    life-cycle phases of the project
  • Inadequate use of systems engineering (project
    management) principles for LL in complex/large
    scale projects
  • Look for mishap, LL analysis and implementation
  • Consult experts and examine LLIS for data

15
Selected Literature Review Findings
  • Report to the Subcommittee on Space and
    Aeronautics, Committee on Science, House of
    Representatives (GAO-02-195) named Better
    Mechanisms needed for sharing Lessons learned,
    January 2002.
  • http//www.gao.gov/new.items/d02195.pdf
  • No assurance Lessons Learned being applied
  • Unfamiliarity with lessons learned across
    centers/programs
  • Cultural barriers inhibit sharing, capturing and
    submitting lessons
  • Lack of support from agency leaders
  • Success in industry comes from commitment to
    knowledge sharing

16
Selected Literature Review Findings (cont.)
  • Earlier survey of NASAs Project/Program Managers
    on issues of lessons learned and the process
    (GAO-01-1015R) Survey of NASAs Lessons Learned
    Process, September 2001.
  • http//www.gao.gov/new.items/d011015r.pdf
  • Topics included collection and sharing of
    lessons, strengths and weaknesses of lessons
    learned processes and procedures, comments on
    LLIS, etc.

Source GAO-01-1015R, Survey of NASAs Lessons
Learned Process, September 2001.
17
Selected Literature Review Findings (cont.)
  • Presentation at CPMR Workshop December 9-10, 2003
    by the Best Practices and Lessons Learned Working
    Group Report Panel
  • Examined integration of best business practices
    and Lessons Learned
  • Discusses road blocks and challenges in achieving
    such an effort
  • Aha, D.W., Weber, R. (2000). Intelligent
    Lessons Learned Systems Papers from the AAAI
    Workshop, Menlo Park, CA AAAI Press Weber, R.,
    Aha, D.W., Becerra-Fernandez, I. (2001).
    Intelligent lessons learned systems.
    International Journal of Expert Systems Research
    Applications, Vol. 20, No. 1,
    17-34. http//www.kmlab.fiu.edu/Papers/Intelligent
    20Lessons20Learned20Systems.pdf
  • Discussed implementation issues of lessons
    learned in any general organization.
  • Comparisons can be made on aspects of current
    NASA system
  • Could be used as a benchmark in the second phase.
  • Characterizations
  • Content, Role, Orientation, Duration,
    Organization Type, Architecture, Attributes and
    Formats, Confidentiality, Size

18
Selected Literature Review Findings (cont.)
  • NASA Culture Survey, Assessment and Plan for
    Organizational Culture Change at NASA, March 15,
    2004.
  • Three key themes emerge from the survey data
  • Overall NASA has strong work-group level teamwork
    and communications.
  • Overall NASA has improvement opportunities in
    upward communications about safety and in
    employee perceptions about the extent to which
    the organization cares about employees.
  • Overall there is little variation among NASA
    locations, among offices within NASA locations,
    or between programs.
  • Executive Summary of Survey
  • NASA personnel are committed to safety, but NASA
    culture not fully supportive of safety Open
    communication not the norm, people not fully
    comfortable raising safety concerns to management
  • People do not feel respected by organization
    Technical commitment does not translate into
    organizational commitment
  • Excellence is treasured for technical but not for
    management, support, communications
  • Integrity understood, but mixed signals from
    management about raising issues.

19
Initial (General) Application to Current Effort
  • Communication Between Centers
  • General inability to effectively collect and
    disseminate lessons learned
  • PMs lack of awareness of lessons being learned in
    other centers
  • Lessons learned mainly through group meetings,
    project reviews
  • Project Review and Management Processes
  • Importance and priority of collecting, sharing
    and use of the lessons as a key component of NASA
    management goals and objectives not clear
  • Incentives, resources and appreciation associated
    with adding to/referring to lessons learned not
    present
  • Perception that discussion of problems not
    welcome, or problems too easily dismissed,
    reduces LL contributions
  • Implementation of Systems Engineering Concepts
  • Redundant/exclusive/varied systems for sharing
    lessons within NASA despite existence of LLIS
  • Different methods/processes/systems exist for
    review and approval of lessons learned for LLIS
  • LL systems engineering design lacks incentives
    and feedback

20
Findings from Literature Review and Collaborative
Discussions
  • Cultural
  • Notion that lessons not apt, relevant or useful,
    lack of time for LL (input/use)
  • Communication of LL among Centers and HQ is
    incomplete, leading to inconsistent sharing,
    learning and application of lessons
  • Tools and System Processes
  • Need better LL IT tools and project management
    review processes and check points
  • Organizational
  • Management-driven initiative and systems-based
    process are needed for effectively collecting and
    sharing lessons
  • Value and importance of LL to organizational
    goals needs to be emphasized and put in practice
    at all centers

21
Overview Of Lessons Learned Systems
  • Many organizations use LL systems
  • Elaborate list of LLS worldwide at
    http//home.earthlink.net/dwaha/research/lessons.
    html
  • Many examples
  • FAA
  • NASA Aviation Safety Report System
  • Licensee Event Report (NRC)
  • DoD (Army, Navy, etc.), DoE
  • Contractors (LM, NG, etc.)

22
NASA LLIS Research Plan
  • Familiarize ourselves with current LLIS database
    and historical evolution
  • Early development in Code Q, limited capabilities
  • Assignment to HQ/Chief Engineer, expanded use
  • Goddard maintenance, full deployment (current)
  • Knowledge management (future)
  • Evaluate the timeline of lessons based on impact
    of mishaps, incident review boards and GAO
    reports
  • Graphically review data for each Center to
    determine trends, and study variations in the
    approval process

23
Some Results
24
Some Encouraging Results
Progress in Reducing Approval Time
25
A Lessons Learned Success Story at JPL From Mars
Climate Orbiter and Mars Polar Lander
Mishaps/Failures in 1999 to Mars Explorer Rovers
Successes in 2003
  • Major Noteworthy Actions/Outcomes
  • Creation of the Mars Climate Orbiter (MCO) Mishap
    Investigation Board (MIB)
  • MCO Phase I Report Released November 10, 1999
    (identified root causes and factors contributing
    to MCO failure)
  • Report on Project Management in NASA by the MIB
    March 13, 2000 (Recommendations for a culture
    shift by focusing on four categories of themes,
    which, in addition to the Mars Program at JPL,
    were also applicable to other programs
    throughout NASA (p. 36).
  • People, Process, Execution, and Technology

26
A Lessons Learned Success Story at JPL From Mars
Climate Orbiter and Mars Polar Lander
Mishaps/Failures in 1999 to Mars Explorer Rovers
Successes in 2003 (cont.)
  • Major Noteworthy Actions/Outcomes (cont.)
  • Culture change based on lessons learned via
    development of systematic new project and process
    management tools and practices
  • Clarification of major responsibilities of
    project managers concerning project quality,
    schedule and budget and understanding their
    interactions and trade-offs

27
A Lessons Learned Success Story at JPL From Mars
Climate Orbiter and Mars Polar Lander
Mishaps/Failures in 1999 to Mars Explorer Rovers
Successes in 2003 (cont.)
  • Major Noteworthy Actions/Outcomes (cont.)
  • Codifying and standardizing practices, and
    development of procedures
  • Corrective Action Notices
  • Design, Verification/Validation and Operations
    Principles for Flight Systems (D-17868)
  • Development of the Flight Project Life Cycle
    concept and its enforcement of its multiple
    reviews
  • Concept Review, Preliminary Mission System
    Review Project/System Preliminary Design Review
    Project/System Critical Design Review, Assembly,
    Test, and Launch Operations Readiness Review
    Mission Readiness Review Post Launch Assessment
    Review and Critical Events Readiness Review
  • Flight Project Practices (Revision 5, 2/27/03)

28
A Lessons Learned Success Story at JPL From Mars
Climate Orbiter and Mars Polar Lander
Mishaps/Failures in 1999 to Mars Explorer Rovers
Successes in 2003 (cont.)
  • Major Noteworthy Actions/Outcomes (cont.)
  • Over 100 Return-to-Flight criteria and
    lessons-learned developed as a result of review
    boards were implemented in Rover program

29
JPL Lessons Learned Committee Manages Process and
Input to LLIS
30
JSC and Ames LL Process Flow Charts
31
NASA MANAGEMENT AND SYSTEMS PROCESSES FOR LESSONS
LEARNED
32
NASA ENGINEERING NETWORK PROTOTYPE STUDY
Learning Process Occurs Behind All
Components Embed lessons into tools and
communities
  • New capability
  • New portal
  • Push vs. pull
  • Improved search
  • Metadata/mining
  • Collaborative tool
  • Query experts

Interagency/Aerospace Lessons Learned
Center Lessons Learned
NASA Lessons Learned
Collaborative Tools (NX, Jabber, WebEx)
Exploration Systems Project Environment
Metasearch
Training APPL SOLAR NET
Policies and Procedures
Feedback
Advanced Engineering Tools
Feedback
Responsibility Areas Chief EningeerGreen
Agency ResourcesBlue
Document and Data Repositories
33
HUMAN FACTORS (HF) IN LESSONS LEARNED SYSTEM
DESIGN
  • HF-based design of HCI systems such as LLIS/NEN
    can significantly improve user interaction and
    efficiency
  • Our preliminary usability testing results of
    HF-based redesign of LLIS interface indicate
    tasks such as
  • Conducting searches
  • Browsing search results
  • Refining a search
  • Can
  • Be carried out in significantly less time
  • Reduce potential for confusion

34
HUMAN FACTORS (HF) IN LESSONS LEARNED SYSTEM
DESIGN (cont.)
  • HF methods can apply to future NEN design
    iterations, and can yield significant benefits
  • Interface design
  • Navigation efficiency
  • Searching effectiveness
  • Usability testing-based design of NEN
  • Cognitive compatibility of NEN user with system
    design

35
EDUCATION PUBLIC OUTREACH
  • Lessons learned and results of this effort
    integrated in
  • Graduate-level PM class (ISE 515, taught by Dr.
    Maya)
  • Two courses in engineering work design and human
    factors engineering (ISE 370 and ISE 570, taught
    by Dr. Meshkati and Dr. Rahimi)
  • Educational lessons learned, case studies and
    classroom experience will be distributed via
    APPL, PMI, NASA, INCOSE and other industry and
    university channels
  • Short courses integrating LL for PMs in NASA and
    aerospace industry, such as those offered by the
    USC Aviation Safety (and Security) Program, will
    be developed

36
Summary Findings
  • Mishap investigation boards have identified
  • Technical lessons learned
  • Need for NASA culture change to reap benefits
  • Our preliminary LLIS data analysis shows
  • Culture change to implement LL processes has been
    evolutionary and Center-dependent
  • LLIS use is not measurably tracked
  • Found evidence that system-level management
    change can overcome LL culture change barriers
  • Currently, several processes and protocols in use
  • No NASA-wide, standard, system-level LL template
  • JPL has successfully implemented comprehensive LL
    process addressing technical and culture change
    challenges

37
Summary Findings
  • New NASA knowledge management system, NEN, is in
    final stages of beta testing before release
  • Modernization of LLIS with current IT
  • Addresses IT aspects of LL
  • Limited address of system-level issues
  • Human factors perspectives and advanced
    information management technologies offer
    significant potential for
  • Addressing culture change barriers
  • Enhancing usability of NEN
  • Increasing effectiveness of overall LL process

38
Conclusions and Recommendations
  • NASA LL culture change is possible and underway
  • Executive level systems-based LL guidance and
    support critical for NASA-wide implementation
  • Systems-based incentives, check points,
    consistency
  • Successful JPL process can be adapted to address
    needs of other Centers
  • NEN will provide much needed upgrade of LLIS
  • Can effect culture change as part of
    systems-based approach
  • HF-based HCI design can significantly enhance
    usability, acceptance effectiveness
  • Advanced IT features can address major challenges
About PowerShow.com