Identifying Organizational Risk Factors in Space Support Environments: The Use of Surveys

1
Identifying Organizational Risk Factors in Space
Support Environments The Use of Surveys

• Bonny Parke, Ph. D.¹,
• Judith Orasanu, Ph. D.², and Yuri Tada, Ph. D.¹
• ¹San Jose State University Foundation/NASA Ames
• ²NASA Ames

2
Human and Organizational Risk Management Ames
Research Center, NASA

• Funded by Engineering for Complex Systems, a NASA
  Program

3
Agenda

• Introduction
• Development of surveys
• Risk factors and possible survey items
• In different work contexts
• 1. Organization-wide
• 2. Team
• 3. Individual
• In aerospace risk areas
• As seen by respondents
• Conclusion

4
Organizational Factors Can Affect System Risk

• Aviation accidents
• Lack of training
• Time pressure
• Over-scheduling ( resulting fatigue)
• Faulty procedures

5
Organizational Factors Can Affect System Risk

• Spacecraft accidents
• Lack of communication between NASA a non
  co-located contractor (Mars Climate Orbiter)
• Inadequate documentation practices
  (TitanIV/Milstar)
• Lack of communication channels for minority
  opinions (Challenger)

6
Columbia Accident Investigation Board (CAIB)
Report

• Cited need for
• . . .organizations committed to effective
  communication to seek avenues through which
  unidentified concerns and dissenting insights can
  be raised, so that weak signals are not lost in
  background noise. . . These avenues must
  mitigate the fear of retribution, and management
  and technical staff must pay attention. (p. 192)
• Anonymous surveys are one of these avenues

7
Agenda

• Introduction
• Development of surveys
• Risk factors and possible survey items
• In different work contexts
• 1. Organization-wide
• 2. Team
• 3. Individual
• In aerospace risk areas
• As seen by respondents
• Conclusion

8
Anchor in Organizational Risk Literature

• Major research approaches in organizational risk
  literature
• High-Reliability Organizations (HROs)
• Roberts, Weick
• Generative Organizations
• Westrum
• Learning Organizations
• Reason

9
Example of One Approach (Westrum)3 Types of
Organizations

• Pathological
• Info hidden
• New ideas crushed
• Mistakes covered up
• People reluctant to bring up problems

• Bureaucratic
• Info ignored
• New ideas ignored
• Mistakes dealt with fairly
• People who identify problems not appreciated

• Generative
• Info sought
• New ideas
• sought
• Mistakes
• examined
• People are
• rewarded for
• bringing up
• problems

10
Focus on Specific Risk Areas in Domain

• Space support environment
• Teamwork
• Stability of personnel
• Co-location
• Meeting quality
• Documentation and databases
• Software
• Schedule and fatigue issues

11
Collaborate with Future RespondentsThose Who
Want to Be Heard

• Become familiar with domain
• Interview future respondents
• Change survey after interviews until it becomes a
  truly collaborative product

12
Collaborate with Those Who Make DecisionsThose
Who Want to Know

• Become familiar with relevant issues
• What upcoming decisions do they have to make?
• What do they want to know?

13
Type of Survey

• Paper pencil or electronic?
• Electronic
• Email surveys
• Online surveys
• Ratings or free text responses?
• Ratings with free text space next to them
• Allows for elaboration of ratings

14
Agenda

• Introduction
• Development of surveys
• Risk factors and possible survey items
• In different work contexts
• 1. Organization-wide
• 2. Team
• 3. Individual
• In aerospace risk areas
• As seen by respondents
• Conclusion

15
1. Organization-wide

• Decision factors
• How often do the following factors play a role in
  upper management's decisions?
• How often do the following factors play a role in
  your decisions and recommendations?

16
Examples of Decision Factors

• Crew safety
• Vehicle safety
• Cost
• Schedule
• Science output
• Contract negotiations

• Public opinion support
• International cooperation
• Influence from other govt. bodies
• Interpersonal conflict

17
Example Table for Decision Factors
When upper management makes decisions about the
mission, how often do you think the following
factors play a role?
18
Advantages to Including Decision Factors

• Can compare factors in respondents' decisions
  with their perceptions of management decisions
• Can examine the extent some factors override
  others
• Can assess how factors vary over time

19
Organization-wide Characteristics from Risk
Literature

• Management's responsiveness to problems
• Relevant information gets to decision makers
• Crisis teams formed promptly to deal with crises
• Those who spot problems respected
• Mistakes investigated to identify causes, not
  cast blame
• "Lessons learned" databases

20
Example Table for Some Organization-wide
Characteristics
21
Organization-wide Characteristics Especially
Relevant to Space Domains

• Accessibility of written history, decisions and
  rationales on current spacecraft
• Need in order to assess risks in current
  decisions
• Verbal rationales degenerate
• "We've always done it this way."
• "We've never had a problem with it."

22
If Working with International Partners or
Contracting Organizations

• Assess quality of information flow between groups

23
2. Team Context

• Collaboration
• Within teams
• Between teams
• Other team members' training
• In own subsystem
• In overall spacecraft system
• Degree to which team members respect those who
  spot problems
• Morale and turnover
• Co-location
• Reduces risk

24
Team Meetings

• Major risk is failure to communicate
• Cooke, Helmrich, Parke, Janis, Prince
• Low rates of member participation
• Unilateral decision making
• Members not feeling free to disagree
• Member inputs not considered
• Ineffective meetings

25
3. Individual Context

• Working conditions
• Job satisfaction
• Schedule-related fatigue issues
• Adequacy of own training
• Rewarded for good work

26
Back-up

• Knowledge only one deep?
• "Are there those who can back you up if you get
  sick?" "If yes, how many?"
• "Which areas need more back-up?"

27
Workload

• How close to deadlines are problems being worked?
• Before deadlines, at deadlines, past deadlines?
• How many days past?
• How many hours/week overtime?

28
Relationship with Supervisors

• Related to information flow to next level
• Supervisors accessible?
• Supervisors willing to listen?
• Trust supervisors?

29
Agenda

• Introduction
• Development of surveys
• Risk factors and possible survey items
• In different work contexts
• 1. Organization-wide
• 2. Team
• 3. Individual
• In aerospace risk areas
• As seen by respondents
• Conclusion

30
Documentation Databases

• List specific documents databases
• Provide rating scales
• Accuracy?
• Accessibility?
• Provide room for descriptive comments

31
Software Issues

• Have contributed to many spacecraft accidents
  (Leveson, 2003)
• Need to assess
• Adequacy of development testing
• Robustness
• Speed with which problems are fixed

32
Agenda

• Introduction
• Development of surveys
• Risk factors and possible survey items
• In different work contexts
• 1. Organization-wide
• 2. Team
• 3. Individual
• In aerospace risk areas
• As seen by respondents
• Conclusion

33
Respondents' Viewpoint

• Ask for responses to
• "What do you perceive as the 3 most serious
  organizational vulnerabilities?"
• "What are their consequences?"
• "Do you have any suggestions?"
• Ask for ratings on
• "How serious do you think they are?"
• "How much do they affect you personally?"

34
Advantages to Descriptive Responses

• Enable other issues to "percolate up"
• Enable assessment of relative importance of
  organizational vulnerabilities on survey

35
Conclusion

• Achieving mission goals requires
• Technical excellence
• Organizational excellence
• Just as you need instruments to monitor the
  health of the space craft
• You need instruments to monitor the health of the
  supporting organization
• Transfer information from those who want to be
  heard to those who want to hear