The HEATACT Preliminary Safety Case: A case study in the use of Goal Structuring Notation

1
The HEAT/ACTPreliminary Safety CaseA case
study in the use ofGoal Structuring Notation

• Paul Chinneck
• Safety Airworthiness Department, Westland
  Helicopters
• chinnecp_at_whl.co.uk
• David Pumfrey, John McDermid
• Department of Computer Science, University of
  York
• david.pumfreyjohn.mcdermid_at_cs.york.ac.uk

2
PSC requirements

• Clear need for a thorough and convincing safety
  case
• Phased approach
• First step Preliminary Safety Case
• complete argument, showing all important claims
• early ideas of eventual evidence
• in place in time to influence design
• review with customer and authorities to ensure
  acceptability
• Initially, only 2 months to construct PSC

3
Timeline
4
Goal Structuring Notation

• Purpose of a Goal Structure
• To show how goals are broken down into
• sub-goals, and eventually supported by evidence
• (solutions) whilst making clear the
• strategies adopted, the rationale for the
• approach (assumptions, justifications)
• and the context in which goals are
  stated

A/J
5
A Simple Goal Structure
6
Developing the Argument (1)
Top goal Trials aircraft is acceptably safe to
fly with HEAT/ACT fitted
7
Developing the Argument (2)

• definition of the top few levels of GSN structure
  relatively easy
• lots of guidance available
• became increasingly difficult to extend the
  argument into lower levels of sub-goals
• difficulty in reaching existing evidence
• unclear how to resolve obviously related goals in
  different branches of argument structure
• lack of inspiration

8
Progress through patterns

• problems tackled through use of patterns, and
  reuse / adaptation from existing safety cases
• Tim Kellys safety case pattern catalogue
• Eurocontrol RVSM safety case (published on web)
• Kenny Grahams MSc project
• patterns for major revisions to in-service
  systems
• very rapid progress with this approach
• real safety case material more useful than
  lifeless patterns?

9
Progressive Development
10
Managing Complexity

• explosion in size of structure
• where argument is enumerated over hazards
• solution hazards grouped by function
• hazards in group are strongly related
• common evidence
• GSN supported by matrix
• functions ? hazards

11
Supplier Contributions (1)
12
Supplier Contributions (2)

• pulling all a contributors evidence under a
  single goal led to distortion of structure
• better to construct an ideal argument structure
• without considering who would be responsible for
  sourcing each item of evidence
• use PSC structure to show suppliers what they
  will be required to contribute
• how their contributions fit into the big
  picture
• in practice, this resulted in sub-system safety
  cases with standard structures

13
The Completeness Problem
14
Challenges of Integration

• analysing impact of new equipment on legacy
  platform and systems is complicated
• differing safety management standards in use
• target aircraft certified to BCAR
• each system has independent Hazard Assessment,
  Detailed Analysis and Safety Assessment Report
  documents
• HEAT system using Def Stan 00-56
• different document suite.
• systems sorted into physically changed /
  indirectly affected / unaffected, and re-analysed
• further analysis to identify where previously low
  severity failure conditions (e.g. total loss of
  AC power) are now catastrophic
• another example of the completeness argument
  issue

15
The Self-Referential Safety Case

• clear that PSC was being used as an important
  part of the safety process
• phases of the safety case included as explicit
  evidence documents within the argument structure
• initially attached simply as evidence satisfying
  goal of safety case developed and maintained
• generic pattern for SMS items

• pattern applies to safety case itself, as well as
  external documents

16
Purging of Assumptions

• review of assumptions (initial release and
  subsystems) to check that all were valid and
  acceptable
• External assumptions relate to world outside
  safety case
• define or limit the scope of the safety case
• e.g. assumed that test aircraft acceptably safe
  before modification
• genuine assumptions can remain provided
  significance understood
• Internal assumptions relate to items within the
  scope of the safety case / safety process
• predict results of incomplete activity
• e.g. assuming that designers will select a
  solution incorporating redundancy
• really place holders unacceptable in the
  completed safety case and must be converted into
  goals

17
Acceptance and Accessibility

• PSC must be accessible (and acceptable) to all
  disciplines involved in the project
• engineering and management
• range of reviews throughout project
• informal review
• PDR 26 pages of GSN reviewed in 30 minutes
• Project Safety Meetings
• GSN benefits
• quick to review the entire Safety Case
• easy to see what had changed or been added
• GSN diagrams can be made into a projection slide
• document complemented for clarity and
  comprehensibility
• by technical and non-technical staff

18
Notable Successes

• structure of the PSC document (GSN fragments
  introducing the argument in small sections)
  proved very successful
• structured approach helped in reducing re-work
• many revisions were merely expansions of earlier
  work
• time saving through development and re-use of
  patterns
• simple to keep the "big picture" in mind
• confidence that areas of the safety argument
  would not be overlooked
• no question has ever been raised for which the
  questioner could not be directed to some part of
  the PSC
• simplified discussion of parts of the document
• route map to show where fragment fitted in

19
Looking to the Future

• HEAT/ACT is expected to have a long lifecycle
• consider changes in context (physical,
  operational, legal)
• Defence Standard 00-56 completely revised
• key standard used in project!
• assessment of impact of changes
• 00-56 Issue 3 intended to reflect MoD policy of
  as civil as possible, and only as military as
  necessary
• sets goals, rather than prescribing processes
• good practice established under Issue 2 should
  meet the requirements of Issue 3
• argument presented in the PSC should be readily
  converted to a DS 00-56 Issue 3 form
• need to explain principles rather than referring
  to prescriptive clauses contained in Issue 2
• should be a good test of the wording of Issue 3!

20
Conclusions

• GSN really works!
• (and not just for toy examples)
• real confidence that PSC will provide solid
  foundation for subsequent phases
• use of patterns
• fast development of argument
• robust argument (already tested)
• some issues
• inherent problems in building satisfactory
  argument, highlighted by use of GSN