NASA Navy Cooperation

1
NASA / Navy Cooperation Process Based Mission
Assurance Knowledge Management System (PBMA-KMS
) functional support to program and project
managers
Henry Hartt and Don Vecellio ARES Corporation NA
SA Office of Safety Mission Assurance
April 5, 2005
2

• RATIONALE
• FOR
• NASA / NAVY COLLABORATION
• Notable similarities between human space flight
  and nuclear submarine programs.
  
• Both spacecraft and submarines operate in extreme
  environments
  
• Both require integration of complex systems and
  subsystems
  
• Both must maintain the highest levels of safety
  and reliability to perform their missions.
  
• Navy has continued to operate safely and
  effectively in resource-constrained and declining
  production environments.
  
• As NASA explores application of nuclear
  propulsion and power for space exploration,
  lessons learned from the Navys nuclear safety
  program could be beneficial.
• Given current management challenges the Agency
  might benefit from in-depth examination of the
  engineering management, safety, and mission
  assurance practices employed by the Navy
  submarine force.

3
Initial Areas of Emphasis for NNBE Investigations

• Assurance Requirements
• SUBSAFE, Deep Submergence, Nuclear Reactors,
  Space Shuttle Program
  
• Assurance Planning and Analysis
• Life-Cycle Risk Management Requirements,
  Approaches, Tools (e.g., FMEA, Criticality
  Analysis, PRA, Hazard Analysis, etc.) for
  Design, Manufacturing, and Operations
• Assurance Processes
• Management, Organizational approach (reporting
  relationships / requirements flow-down),
  Resource Loading, Engineering, Training
  
• Control Processes
• Work Control, work instructions, configuration
  management, component/work documentation /
  pedigree (NAVY SUBSAFE Re-entry Control (REC)
  Process), Non-conformance disposition, Work
  review, Surveillance / Inspection, Change
  Control, Design Change Control, Configuration
  management
• Verification Processes
• Audits Functional (SUBSAFE Periodic ), NASA
  Process Verification, NASA NEQA Audit,
  Certification, Ships Leave the Shipyard (hull
  or vehicle) SUBSAFE Pre-Fast Cruise, Audit,
  Parts see REC, Operational Readiness,
  SUBSAFE Unrestricted Operations Maintenance
  Requirements Certification (URO-MRC), Space
  Shuttle Certification of Flight Readiness Process

4
Navy Organizations Visited by NASA

• NAVSEA (Naval Sea Systems Command) HQ / WNY
• NAVSEA 07 (SUBSAFE Program)
• NAVSEA 08 (Naval Reactors)
• NAVSEA 05 (Ship Design Integration and
  Engineering)
  
• SUPSHIP (Supervisor of Shipbuilding Conversion
  and Repair)
  
• Portsmouth Naval Shipyard
• SUBMEPP (Submarine Maintenance Engineering,
  Planning and Procurement)
  
• NAVSEALOGCEN (NAVSEA Logistics Center)
• SHAPEC (Ship Availability Planning and
  Engineering Center )
  
• Pearl Harbor Naval Shipyard Integrated
  Maintenance Facility
  
• SUBSAFE Functional Audit
• General Dynamics Electric Boat Division
• Bath Iron Works

5
Navy Strengths

• Rigor of SUBSAFE Safety Assurance Process
• NAVSEA 08 religious assignment of lifecycle
  assurance responsibility
  
• Emphasis on Minority Opinion in the Decision
  Process
  
• Emphasis on Recurrent Training
• NAVSEA Warrants Independent Technical
  Authority
  
• PDREP/RYG processes of maintaining a record of
  contractor/supplier Quality Assurance (QA)
  performance

6
NASA/NAVY Memoranda of Agreement
NASA / NAVSEALOGCENDET Portsmouth
NASA / NAVSEA 07
Supplier QA Information Exchange

• Audit Participation

NASA / NAVSEA 05

• Engineering Investigations and Analyses

7
Navy Approaches Infused into NASA Processes

• NAVSEA SUBSAFE and Nuclear Reactor training led
  to Safety Critical Decision Making (CSDM)
  training initiative implemented by Office of the
  Chief Engineer (OCE)
• SUBSAFE Audits Model adapted to NASA Programmatic
  Audit and Review Process
  
• Ongoing collaboration in Human Factors and
  Software development IVV
  
• Pyramidal (three point) decision process
  (Technical, Program, Safety) adapted by NASA
  
• ITA Technical Warrant Holders process established
  by OCE
  
• Establishment of NESC
• Creation of OSMA Review and Assessment Division

8
Summary Reports Published

• December 20, 2002 -- Report 1 Navy Submarine
  Program Safety Assurance
  
• July 15, 2003 -- Report 2 Naval Nuclear
  Submarine Safety Assurance
  
• October 22, 2004 -- Ongoing NNBE Activities
  Software Subgroup Report I
  
• http//pbma.nasa.gov/program/nnbe.htm

9
Safety Cultural Emphasis

• "The only way to operate a nuclear power plant
  and indeed a nuclear industry -- the only way to
  ensure safe operation, generation after
  generation, as we have -- is to establish a
  system that ingrains in each person a total
  commitment to safety a pervasive, enduring
  devotion to a culture of safety and environmental
  stewardship."
• ADM F.L. BOWMAN

10
RESPONSIBILITY IS A UNIQUE CONCEPT"

• It can only reside and inhere in a single
  individual.
  
• You may share it with others, but your portion is
  not diminished.
  
• You may delegate it, but it is still with you.
• You may disclaim it, but you cannot divest
  yourself of it.
  
• Even if you do not recognize it or admit its
  presence, you cannot escape it.
  
• If responsibility is rightfully yours, no
  evasion, or ignorance, or passing the blame can
  shift the burden to someone else.
  
• Unless you can point your finger at the man who
  is responsible when something goes wrong, then
  you have never had anyone really responsible.
  
• ADM H.G. RICKOVER

11
Process Based Mission Assurance Knowledge
Management System (PBMA-KMS ) functional support
to program and project managers
12
Background

• PBMA-KMS deployed in March of 2001 is the first
  fully operational NASA-wide multi-functional
  Knowledge Management System
  
• Developed and implemented under the sponsorship
  of the Office of Safety and Mission Assurance
  
• Merged existing NASA SMA program/project
  life-cycle knowledge architecture with
  state-of-the-art KM concepts presented in GWU/KM
  graduate coursework
• PBMA Knowledge Architecture reflects integration
  of SMA functions (work processes) into the
  systems engineering program/project life-cycle
  
• Maintaining ongoing dialogue with KM community at
  GWU, GMU, Washington KM Roundtable, KM-Pro /
  Universal KM Framework Workshop
  
• PBMA-KMS serves program/project managers and
  safety and mission assurance professionals within
  a traditional life-cycle work breakdown context
  
• Widely accessible / user friendly / content
  rich
  
• In place policies requirements, best
  practices, lessons learned, tacit knowledge
  capture (video nuggets), collaborative tools,
  etc.

13
PBMA-KMS Timeline
14
PBMA-KMS Work Groups/Communities of Practice

• Over 7200 work group members
• More than 340 individual communities of practice
• 38 Communities of Practice supporting the Space
  Shuttle Program return-to-flight efforts.
  
• 145 other NASA programs and projects also
  supported
  

Volume More people are coming to PBMA, they are
using the site for longer, and they are coming
back again for more information. - 190,000 hits
per month (an increase of over 20 from the
beginning of 2004). Return Visits numbers of d
aily visitors and returning visitors (key
indicators of value) have increased more than 25
during the last 12 months. Length of Visit most
telling indication of value is the length of
visit. Visitors are spending 45 more time at
PBMA then they were at the beginning of 2004.
15
PBMA - Knowledge Architecture
Each cell contains video-nuggets, text, links
16
KM Functional Utilities
The PBMA-KMS employs a core set of KM
functionalities that have the potential to serve
all users. These are "no-brainer" functional
requirements (i.e., the gas, water, electricity
and cable TV of knowledge management)
applicable/available to every unique knowledge
management system and architecture.
Individual Business Units with Unique
Knowledge Architectures
Business Unit-Specific Knowledge Architectures
Safety and Mission Assurance
Deep Space Programs
Financial Mgmt
Document Repository
Advanced Search / Discovery (across public
domains)
Secure Communities of Practice (Work Groups)
Secure Web Meetings
Knowledge Registry (expert finder)
Agency/Enterprise-wide KM Utilities
17
Selected PBMA Functionality

• Framework Mode
• Knowledge Architecture
• Video Nuggets (a verbal corporate/tacit knowledge
  archive)
  
• Standard Security Work Groups
• Web-based collaborative environment (CE) tool for
  public domain information
  
• Enhanced Security Work Groups
• Secure Web-based CE Tool for sensitive
  information
  
• Knowledge Registry
• Web-based repository for SMA/engineering/technical
  expertise
  
• Secure Web Meeting
• Secure, real-time white-board tool for sharing
  information and conducting meetings remotely via
  the Internet (secure uplinks/downlinks and
  servers)

18
(No Transcript)
19
Standard Security Work Groupsand Communities of
Practice

• COTS Web-hosted Product
• Industry Best Practice
• Password protected / NPR 2810.1 compliant
• No ITAR / EAR or other Sensitive Data
• Document Sharing and Management, Calendars,
  Action Tracking, Announcements, Polls, Contacts,
  Links, Threaded Discussions, etc.

20
Enhanced Security Work Groupsand Communities of
Practice

• Developed in Partnership with GRC-CIO/IT
  Security
  
• One-factor strong authentication
• 128-bit SSL encryption
• Designed to Support Sensitive Information
  Management and Exchange
  
• ITAR / Export Controlled Information
• Source Evaluation Boards and Competition
  Sensitive Information
  
• Mishap Investigation Information

Provides a means of establishing a secure,
accessible site for team collaboration when
Administratively Controlled Information (ACI) is
involved

21
PBMA Knowledge Registry

• Locate/Contact Experts
• Identifies where Agency SMA, Engineering, and
  Technology expertise and knowledge resides
  
• Voluntary Registration
• Linked to NASA Competency Management System
  (CMS)
  
• Allows users to locate specific subject matter
  experts in a fast, convenient mechanism
  
• Searchable by selected (e.g., .mil, .nasa,
  .gov) domains
  
• Can assist in Resource Management Planning
  Activities
  
• Currently supporting NASA / NAVSEA expert
  collaboration

22
Secure Web Meeting

• Reduced travel
• Schedule on your own, no dial-in numbers
• Easy access plug-and-play appliance with no
  recurring costs
  
• Real time collaboration
• Remote white-board collaboration in a secure
  environment
  
• Protected Transient Events 128-bit encryption
• Implemented on a NASA Server behind NASAs
  firewall
  
• Operates with standard Web browsers
• Does not require similar software on attendees
  PCs
  
• Easy client setup install or download it
  on-the-fly on first use
  
• Host a "shared desktop" session over the
  Internet
  
• Capable of holding 10 meetings with 50 people in
  each simultaneously
  
• Share any document, briefing, spreadsheet, etc.
  between multiple users
  
• Pass control of the meeting among attendees
• Make changes in real-time
• Built-in meeting scheduling and user invitation

23
BACKUP CHARTS
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
NR Key Organizational Observations (NASA/Navy
Benchmarking Exchange Interim Report 2 )

• Total programmatic and safety responsibility for
  all aspects of design, fabrication, training,
  test, installation, operation, and maintenance of
  all U.S. Navy nuclear propulsion activities.
• Flat organization with quick and assured access
  to the Director about 40 direct reports from
  within HQ, the field offices, and prime
  contractors. Communications between headquarters
  and prime contractors and shipyard personnel
  occurs frequently at many levels, and a cognizant
  engineer at a prime or shipyard may talk directly
  with the cognizant headquarters engineer, as
  necessary.
• The Naval Nuclear Propulsion Program (NNPP) is a
  very stable program based on long-term
  relationships with three prime contractors and a
  relatively small number of critical suppliers and
  vendors.
• NR embeds safety and quality process within its
  organization i.e., the desired state of an
  organization completely mainstreams safety and
  quality assurance .
• Reliance on highly qualified, highly trained
  people who personally accountable and responsible
  for safety.
  
• Recurrent training a major element of safety
  culture. NR incorporates extensive outside
  experience (Challenger, Chernobyl, Three Mile
  Island, Army SL-1 reactor) into a safety
  training regimen that has become a major
  component of the NR safety record 128,000,000
  miles of safe travel using nuclear propulsion.
• NR promotes the airing of differing opinions.
  Even with an absence of differing opinions,
  management is responsible to ensure critical
  examination of an issue.

28
NR Safety Observations(NASA/Navy Benchmarking
Exchange Interim Report 2 )

• NR has an institutionally embedded closed-loop
  process that begins with a technical requirements
  base built on lessons learned from more than
  5,400 reactor years of experience, which in turn
  represents the foundation for the next-generation
  propulsion plant design specifications.
• There is no single (stand-alone) document that
  prescribes NR design safety criteria or
  standards. Safety requirements are embedded in a
  uniform set of technical requirements.
• NR has a rigorous change control process that
  enforces review and concurrence of each
  recommended change by all stakeholders. Managing
  change is frequently discussed at senior levels.

29
NR Implementation Observations(NASA/Navy
Benchmarking Exchange Interim Report 2 )

• Each independent lab general manager is required
  to be technically competent and is directly
  responsible for the safety of the reactors and
  facilities under his/her cognizance.
• The NR Director exercises (by law) direct
  supervision over the laboratories.
  
• Review by Quality Assurance or Safety does not
  diminish responsibility of line organization for
  program/product safety.
  
• There is no separate systems engineering group or
  a job category of systems engineer within NR.
  While no single individual serves as system
  safety engineer or integrator, there is an
  individual (Reactor Safety and Analysis Director)
  responsible for maintaining an overall design
  safety perspective.
• Responsibility for safety of an action remains
  with the authoring engineer and his Section
  Heads. The Reactor Safety and Analysis Section
  reviews, consults and concurs in decisions on
  product nuclear safety aspects, but
  responsibility for product safety remains with
  the cognizant engineer and engineering
  organization.
• The Reactor Safety and Analysis Section has an
  independent and equal voice in design and
  operational decisions.
  
• Evolutionary application of more than 50 years
  Lessons Learned to each program reduces
  operational risk and uncertainty.
  
• Freedom to Dissent is a primary element within
  NR.
  
• Systemic emphasis on recruiting, training, and
  retaining the very best people for their entire
  careers.
  
• Critical self-evaluation of problems with strong
  Headquarters oversight isolates and controls the
  small problems before they escalate into large
  problems.
• Closed loop corrective action is mandatory.
  Problems must be identified, analyzed, and
  resolved and their resolutions proven
  successful.
• Cause analysis is performed via a formal
  fact-gathering critique, supplemented by expert
  assessment of root cause/corrective actions.
  
• Heavy emphasis placed on reactor design
  ergonomics through the use of methods, such as
  interactive visualization techniques,
  walk-throughs, and discussions with operators.
  Operational human factors are emphasized but
  change for the sake of change is not permitted.

30
NR Compliance Verification Observations(NASA/Navy
Benchmarking Exchange Interim Report 2 )

• Emphasis on Silver Bullet Thinking is Dangerous
  -- "there is no silver bullet tool or technique.
  All elements of quality assurance and compliance
  assurance must be rigorously implemented to
  ensure delivery and operation of safe, reliable,
  and high quality systems.
• Audit teams include the requirement owner
  (technical authority) for a particular area.
  Owner participates in the audit process to
  acquire first-hand understanding of how technical
  requirements are (or are not) being implemented.
• NR field offices act as day-to-day audit and
  inspection groups. Responses to their findings
  are required, and they must approve final actions
  in response to major comments.
• Functional audits of shipyards supplemented by
  field office assessments and comparative
  evaluations of the sites own self-assessments.
  
• Qualification and biennial re-qualification of
  all nuclear operators by written examination and
  oral board examination assures currency of
  skills. In addition, the NPEB administers an
  annual examination to the entire engineering
  department of a ship and reports results to the
  ships CO, the command authority for that ship,
  and NR Headquarters.
• DCMA is used, but is given technical direction by
  NR directly rather than by DCMA HQ.
  
• NR has Process Sponsor Program where engineering
  activity retains technical responsibility for its
  components but consults with process experts
  (sponsors) within their identified areas of
  responsibility, as necessary.

31
NR Key Certification Observations (NASA/Navy
Benchmarking Exchange Interim Report 2 )

• NR performs incremental audits (similar to
  SUBSAFE) prior to key events to evaluate critical
  processes and to correct any problems with work
  accomplishment or critical documentation.
• A seven-phase test program begins with visual
  check of installation and progresses through
  higher levels of detail to actual operation of
  the reactor and delivery of power to assure
  readiness of the reactor plant for sea trials.
• A Joint Test Group (JTG), composed of
  representatives from the construction shipyard,
  NRRO, Ships Force, and the cognizant laboratory,
  reviews and approves the administration and
  performance of test documents and acceptance of
  test results.

32
Software Assurance Opportunities for
NASA(NASA/Navy Benchmarking Exchange Interim
Report 3 )

• Reappraise Shuttle software using CMM or CMMI
• NASA may want to consider reappraising Shuttle on
  both the contractor and civil servant sides using
  CMM or CMMI to verify that their exemplary rigor
  has not diminished. This is especially important
  since the Shuttle is still certified at CMM Level
  5 despite not having been appraised in 8 years,
  during which time it has changed contractors
  twice.
• Strengthen the levels of defense for assuring
  software safety
  
• NASA may want to consider strengthening its
  levels of defense for assuring software safety
  and quality. Specifically, this opportunity
  includes establishing and implementing better
  contractor requirements (Level 1), bolstering the
  Agencys Software Assurance (SA) resource pool
  (Level 2), and ensuring that IVV is called upon
  only in critical situations (Level 3).

33
Software Assurance Opportunities for
NASA(NASA/Navy Benchmarking Exchange Interim
Report 3 )

• 3. Strengthen Agency CMM/CMMI related
  requirements for mission critical software.
  
• When updating NPD 2820, NASA Software Polices,
  NASA may want to consider not only keeping, but
  potentially strengthening, the CMM/CMMI related
  requirements for organizations developing or
  maintaining mission critical software. Some of
  this work has already been initiated in NASA SWE
  NPR 7150.2 (Software Engineering Requirements),
  which was in the administrative review cycle at
  the time of this report.
• 4. Institute Agency-wide software inspection
  efforts.
  
• The use of rigorous formal software inspections,
  developed based on industry best practices such
  as those prescribed by Fagan and Gilb, has
  provided positive lessons learned for NASA. NASA
  is considering instituting an Agency-wide effort
  to re-infuse these, or similar inspection
  processes, into all software intensive projects.

34
GLOSSARY

• ACI Administratively Controlled Information
• CAIB Columbia Accident Investigation Board
• CE Collaborative Environment
• CIO/IT Chief Information Officer/Information
  Technology
  
• CMS Competency Management system
• CoP Community of Practice
• COTS Commercial Off The Shelf
• DCMA Defense Contract Management
  Administration
  
• EAR Export Administration Regulations
• FMEA Failure Modes and Effects Analysis
• GRC Glenn Research Center
• MRC Maintenance Requirement Card
• GWU George Washington University
• GMU George Mason University
• ICV Independent
• ITA Independent Technical Authority
• ITAR International Traffic in Arms Regulations
• JTG Joint Test Group
• KM Knowledge Management

NPEB Naval Nuclear Propulsion Examining Board
NR NAVSEA 08 Naval Reactors NRRO Naval Reactors
Representative Office OCE Objective Quality Evid
ence PDREP Product Data Reporting and Evaluation
Program PBMA-KMS Process Based Mission Assurance
Knowledge Management System
PM Program Manager PRA Probabilistic Risk Asses
sment REC Re-entry Control RYG Red/Yellow/Green
contractor evaluation process
SA Software Assurance SHAPEC Ship Availability P
lanning and Engineering Center
SUBMEPP Submarine Maintenance Engineering,
Planning and Procurement SUBSAFE Submarine Safet
y Program SUPSHIP Supervisor of Shipbuilding, Co
nversion and Repair SWE Software Engineering URO
Unrestricted Operations WNY Washington Navy Ya
rd