CONSTANCE HEITMEYER

1

TOOLS FOR BUILDING AND EVALUATING HIGH ASSURANCE
SOFTWARE SYSTEMS AND COMPONENTS
CONSTANCE HEITMEYER CENTER FOR HIGH ASSURANCE
COMPUTER SYSTEMS NAVAL RESEARCH
LABORATORY WASHINGTON, DC McMaster
University March 17, 2003
2
OUTLINE

• Background
• History of SCR
• SCR Tools
• Applying the tools to high assurance systems
• Navys A-7 Flight Program
• Rockwells Flight Guidance System
• Navys Weapon Control Panel
• NASAs Flight Protection Engine
• Navy Family of Cryptographic Devices
• CD I - case study
• CD II - formal specification/verification of real
  device
• Specification-based test case generation
• Role of invariants in formal analysis

3
HISTORY OF SCR APPROACH

• 1978 Heninger,Parnas publish A-7/SCR
  requirements document
• Tabular notation
• Events and conditions
• Mode classes and terms
• 1980s-early 1990s SCR applied to a wide range of
  systems
• Telephone networks (ATT Bell Labs)
• Submarine communications (NRL)
• Control software for nuclear plants (Ontario
  Hydro)
• Avionics software (Grumman)
• Early 1990s Development of Four Variable Model
  and CoRE
• Parnas introduce and apply Four Variable Model
• Softw. Productivity Consortium develops CoRE
  (based on SCR)
• Lockheed applies CoRE and SCR tables to C-130J
  flight program
• 1992-present NRL develops formal SCR model and
  tools

SCR Æ Software Cost Reduction
4
SCR REQUIREMENTS MODEL SOME TERMINOLOGY AND
NOTATION

• A system is represented as a state machine
• A condition is a predicate on a state
• A conditioned event is denoted _at_T(c) WHEN d,
  where c and d are conditions. It is defined by
• _at_T(c) WHEN d c ? c ? d,
• where the unprimed c denotes c in the old
  state and the primed c denotes c in the new state
• Changes in monitored variables (called monitored
  events) provide the input alphabet
• Each table (condition or event) defines the value
  of a dependent variable (mode class, term,
  controlled variable) as a function
• The next-state function of the state machine is
  the composition of the functions defined by the
  tables

5
SCR GOAL MAKE FORMAL METHODS PRACTICAL

• Usable, scalable tabular notation
• Integrated set of robust, powerful software
  tools
• light-weight tools whose use does not
• require math. sophistication/thm proving
• heavy-duty tools (e.g., theorem prover)

6
SCR TOOLS FOR DEVELOPING SOFTWARE REQUIREMENTS
SCR TOOLSET
system spec
DEPENDENCY GRAPH BROWSER
SPECIFICATION EDITOR


modes
terms

• most mature tools
• installed at 100 orgns in industry, govt., and
  academia

conditions
SIMULATOR
CONSISTENCY CHECKER
cont vars
events
mon vars
MODEL CHECKER

• Consistency and completeness
• Is the spec well-formed?
• Validation
• Is this the right spec?
• I.e., does the spec capture the intended
  behavior?
• Verification
• Is the spec right?
• I.e., does the spec satisfy critical properties
  (e.g., safety, security)?

New ANALYSIS TOOLS
THEOREM PROVER
Heitmeyer et al., Proc. CAV 98.
7

TOOLS FOR TESTING CODE SYNTHESIS ARE BEING
DEVELOPED
SCR TOOLSET
system spec
DEPENDENCY GRAPH BROWSER
SPECIFICATION EDITOR

• most mature tools
• installed at 100 orgns in industry, govt., and
  academia

modes
terms
CONSISTENCY CHECKER
conditions
cont vars
SIMULATOR
events
mon vars
MODEL CHECKER
New ANALYSIS TOOLS
THEOREM PROVER (TAME)

• TAME is an interface to PVS designed to prove
  properties of state machine models

8
APPLYING CONSISTENCY CHECKING TO THE
A-7REQUIREMENTS DOCUMENT
9
CONSISTENCY CHECKING THE A-7 REQ. DOCUMENT
RESULTS1

• Checked all 36 condition tables, a total of 98
  rows
• Results 17 rows in 11 tables violated the
  Coverage Property (i.e., identified missing
  cases)
• Checked all 3 mode transition tables, a total of
  700 rows (4319 logical expressions)
• Results 57 violations of the Disjointness
  Property were detected (identified
  non-determinism)

Consistency checking finds MANY errors that
human inspections miss and usually does so in a
very short time (seconds to minutes)
1Heitmeyer et al., ACM TOSEM, 1996.
10
EXAMPLE DETECTION OF A DISJOINTNESS ERROR
For each error detected, the consistency checker
displays 1. the table containing the error
with erroneous entry highlighted 2. a state
pair demonstrating the error (counterexample)
11
APPLYING THE SCR TOOLS TO ROCKWELLS FLIGHT
GUIDANCE SYSTEM
12
ROCKWELL-COLLINS AVIATIONFLIGHT GUIDANCE SYSTEM

• Experimental application of SCR tools by Rockwell
  
• Despite extensive reviews by Rockwell engineers,
  the tools found many errors in the spec
• 28 errors detected, many of them significant
• one third each constructing the specification,
  applying the completeness and consistency checks,
  and simulating the system behavior based on the
  specification
• Example Disjointness error leading to two
  possible flight modes
• Example Missing cases (Lateral Armed
  Annunciation field undefined in certain cases)

...preliminary execution of the specification
and completeness and consistency checking with
the SCR tools has found several errors in a
specification that represented our best effort
at producing a correct specification manually.
Steve Miller Rockwell-Collins Aviation
13
APPLYING THE SIMULATOR AND MODEL CHECKING TO A
WEAPONS CONTROL PANEL
system spec
DEPENDENCY GRAPH BROWSER
SPECIFICATION EDITOR


modes
terms
CONSISTENCY CHECKER
conditions
cont vars
SIMULATOR
events
mon vars
MODEL CHECKER
14
ANALYZING A CONTRACTOR REQ. SPEC OF A WEAPONS
CONTROL PANEL

• WCP OVERVIEW
• WCP used to prepare launch weapons
• Sizable, complex program (15KLOC)
• Monitored quantities
• switches and dials
• numeric quantities (read by sensors)
• Controlled quantities
• lights
• doors and valves (set by actuators)
• PRODUCING THE SCR SPEC
• Used scanner and OCR to read in contractor spec
  of the WCP (250 vars)
• Used text editor to convert to SCR spec

• USER-FRIENDLY SIMULATION
• Scanned in diagrams of operator interface
• Used interface builder to develop realistic
• simulator front-end
• Operators unfamiliar with SCR can run
• scenarios to validate requirements spec

1Heitmeyer et al., IEEE TSE, 1998.
15
ANALYZING THE WCP SPECIFICATION FOR SAFETY
PROPERTIES
_at_T(cVENT_SOLENOID) ? kMinTRANS_OK lt TRANS_A ?
TRANS_A lt kMaxTRANS_OK ? kMinTRANS_OK lt
TRANS_B ? TRANS_B lt kMaxTRANS_OK
16
MODEL CHECKING THEWCP SPECIFICATION (1)
PROBLEM TOO MANY VARIABLES SOLUTION REMOVE
VARIABLES IRRELEVANT TO
THE VALIDITY OF THE
PROPERTY
17
MODEL CHECKING THEWCP SPECIFICATION (2)
PROBLEM Some variables are real-valued SOLUTION
Apply data abstraction -- i.e., replace
each real-valued variable with a variable with
a small, discrete value set
Size of type set of tSEL_TRANS goes from
infinite to 3
18
USING SIMULATION TO VALIDATE VIOLATION OF A
SAFETY PROPERTY
Simulator notification of violation in spec
Spin notification of violation in abstract model
18.0
18.0
Input sequence (scenario) that produces violation
18.0
18.0
Corresponding system history (each input and its
results)
19
APPLYING SCR TO WCP REQUIRED EFFORT
PERSON- WEEKS Translate
contractor SRS into SCR 0.8 Use
light-weight tools to detect errors
0.2 Correct errors
0.3 Abstraction/Detection of safety violation
0.7 Develop customized simulator front-end
3.0 5
TASK
TOTAL

• This small effort is quite surprising given that
• the contractor-produced SRS was large and
  complex
• the contractor had no prior knowledge of SCR

20
APPLYING THE SCR TOOLS, INCLUDING THE TEST CASE
GENERATOR, TO NASASFAULT PROTECTION ENGINE (FPE)
21
TEST CASE GENERATION FOR NASAS FPE

• PROBLEM
• NASA is using slightly different implementations
  of the FPE in various spacecraft
• NASA needs high reliance in the correctness of
  each version of the FPE code
• Our task
• To develop a formal spec of the FPE beh.
• From the spec, to constuct a set of test cases
  satisfying some coverage criteria
• The tests will be used to check the FPE code

22
EXPERIENCE TO DATE

• Our goal is to produce
• A good requirements spec of the required FPE
  behavior
• Describes precisely and clearly normal and
  abnormal behavior
• Avoids both overspecification and
  underspecification
• Describes likely ways that the FPE component will
  change
• A set of critical properties that the FPE code
  must satisfy
• Proof that the spec satisfies the properties and
  validation that the spec captures the intended
  behavior
• A set of test cases satisfying some coverage
  criterion
• Progress to Date
• An SCR spec that is well-formed and relatively
  understandable
• A simulator for use in validating the spec
• A small set of test cases constructed using our
  algorithm

NASA personnel learned to read and understand the
SCR specs very quickly
23
TECHNICAL AND OTHER ISSUES

• SCR LANGUAGE
• FPE algorithm involves many complex constructs
  that stretch our model
• e.g., feedback loops, queues, simult. input
  events, priorities, etc.
• Problem How to specify these in a way that
  satisfies our criteria
• PROPERTIES/LIKELY CHANGES
• How to determine what these are
• Little of this is captured in the current
  documentation
• TEST CASE GENERATION
• How to deal with the input data at a more
  abstract level
• How to reduce the length of the test cases

24
GUIDELINES FOR WRITING AREADABLE (FORMAL) REQ.
SPEC

• Begin with a spec language that practitioners can
  easily learn and that is designed for describing
  requirements
• Choose short variable names that the applications
  expert will understand
• Make it easy to distinguish input and output
  variables
• To the extent feasible, eliminate redundancy from
  the spec
• Reduces the length of the spec, so there is less
  to validate
• Reduces the opportunity for errors
• Some planned redundancy is ok -- e.g., critical
  system properties
• Given a choice between implementation bias and
  readability, opt for readability
• Minimize the use of artifacts in the spec
• Select variable types very carefully (e.g., do
  not use a boolean if you only need a toggle
  switch)
• Identify key parts of the spec and focus on
  making those readable
• If some aspect of the requirements makes the spec
  hard to understand, postpone introducing that
  aspect
• Organize the spec for ease of understanding and
  ease of change
• Provide comments that explain the rationale for
  the requirements

25
APPLYING THE SCR TOOLS TO CD I, A MEMBER OF A
FAMILY OF CRYPTO SYSTEMS
26
CD FAMILY OF CRYPTOGRAPHIC DEVICES
encrypt
decrypt
P E I P
P E I P
To From Subj ISR Assets
comm. system
CD SERVICES

• Load (and zeroize) crypto algorithms and keys
• Configure channel (i.e., write alg and key into
  channel space)
• Encrypt and decrypt data using a crypto algorithm
  and a key
• Take emergency action when, e.g., device is
  tampered with
• Provide the above services for m channels

Each family member is implemented in handware and
software
27
TAME -- A SPECIALIZED PVS INTERFACE

• Objective
• Reduce human effort needed to verify properties
  with a theorem prover

Specialized Top Layer
Reasoning in the Timed Automata Model
Induction
Templates
Simulation
R.-T. System Modeled as Timed Automata

• Design Goals
• Easy to create specs
• Natural formulation of properties
• Natural proof steps that match in size/kind
  steps used in hand proofs
• Proofs similar to hand proofs

Timed Automaton Theory Logic
PVS
TAME
Proof System
Type Theory
User-Defined Strategies
Higher-Order Logic
Timed Automata Modeling Environment

• Why build upon PVS?
• Avoid reinventing existing, well-known techniques
• Use PVS logic as a flexible means of further
  proof support for automata models
• State properties in the expressive but natural
  logic of PVS

28
STEPS IN HAND PROOFS VS. STEPS IN PVS PROOFS
HUMAN-STYLE
PVS
TAME Goal Provide natural proof steps
29
VERIFYING THE CD I SPEC (1)
SECURITY PROPERTIES

• When the zeroize switch is activated, the keys
  are zeroized
• No key can be stored before an algorithm in the
  assoc. location is activated
• If undervoltage occurs in backup power while
  primary power is un-available, CD enters alarm or
  off mode
• If backup power is overvoltage, then CD is in
  initialization, standby, alarm, or off mode
• When an overvoltage occurs in primary power, then
  CD is in standby, alarm or off mode, or goes into
  initialization
• When an undervoltage occurs in primary power,
  then CD is in standby, alarm, or off mode, or
  goes into initialization mode
• If CD is tampered with, the keys are zeroized

30
VERIFYING THE CD I SPEC (2)
SECURITY PROPERTIES
AUTOMATICALLY GENERATED INVARIANTS

• When the zeroize switch is activated, the keys
  are zeroized
• No key can be stored before an algorithm in the
  assoc. location is activated
• If undervoltage occurs in backup power while
  primary power is unavailable, CD enters alarm or
  off mode
• If backup power is overvoltage, then CD is in
  initialization, standby, alarm, or off mode
• When an overvoltage occurs in primary power, then
  CD is in standby, alarm or off mode, or goes into
  initialization
• When an undervoltage occurs in primary power,
  then CD is in standby, alarm, or off mode, or
  goes into initialization mode
• If CD is tampered with, the keys are zeroized

• In Initialization mode, primary power is not
  unavailable
• In Configuration mode, the system is healthy,
  backup power is not overvoltage, and primary
  power is not unavailable
• In Idle mode, the system is healthy, backup power
  is not overvoltage, and power power is not
  unavailable
• In Traffic Processing mode, the system is
  healthy, backup power is not overvoltage, and
  primary power is not unavailable
• In Off mode, KeyBank1Key10...

Jeffords, Heitmeyer, 1998, 2001. Kirby, Archer,
Heitmeyer, 1999. .
31
APPLYING THE SCR TOOLS TO CD II OF CRYPTO SYSTEMS
32
NEW EFFORT VERIFYING THE SECURITY OF CD II

• Unlike CD I, CD II
• both receives and transmits
• data
• processes data at many
• different security levels

• CD I has been certified by National Security
  Agency (NSA) and is currently installed aboard US
  Navy platforms
• To certify CD II as a multi-level secure device,
  NSA requires an EAL7-like Common Criteria
  evaluation of the CD II separation kernel
• Our plan Use the SCR/TAME tools to support the
  evaluation

33
WHAT SECURITY POLICY MUST CD II SATISFY?
CD II
SHARED
CHANNEL i
CHANNEL j
Channel
process for data encrypt/ decrypt on channel j
process for data encrypt/ decrypt on channel i
process for storing shared algorithms and keys
dedd memory for channel i
dedd memory for channel j
shared memory for algs/keys
USER COMMANDS
SECURITY POLICY ENFORCE SEPARATION
Data never flows from channel i to channel j and
vice versa
34
WHAT SECURITY POLICY MUST CD II SATISFY?
CD II
Channel
SHARED
CHANNEL i
CHANNEL j
process for data encrypt/ decrypt on channel j
process for data encrypt/ decrypt on channel i
process for storing shared algorithms and keys
dedd memory for channel i
dedd memory for channel j
shared memory for algs/keys
SECURITY POLICY ENFORCE SEPARATION
Data never flows from channel i to channel j and
vice versa
35
HOW TO OBTAIN ASSURANCE THAT CD II ENFORCES
SEPARATION?
DEDICATED channel i
DEDICATED channel j
SHARED
SHARED COMMANDS
CHANNEL COMMANDS
The function of the Separation Kernel is to
prevent illegal data flows.
36
OBTAINING A HIGH ASSURANCE SEPARATION KERNEL

• Develop a formal SECURITY POLICY MODEL to
  describe the CD II notion of separation (Use
  style of 1)
• Produce ABSTRACT SPEC--a formal spec of the
  behavior of the CD II separation kernel
• Prove that the ABSTRACT SPEC satis-fies the
  SECURITY POLICY MODEL
• Produce CONCRETE SPEC-- a formal spec of the CD
  II implementation of the separation kernel
• Prove that the CONCRETE SPEC refines the ABSTRACT
  SPEC.
• Show that the CD II code (i.e., the
  implementation) satisfies the CONCRETE SPEC

SECURITY POLICY MODEL
ABSTRACT SPEC spec of security-relevant behavior
CONCRETE SPEC spec of security-relevant code
security-relevant CODE
1 Landwehr, Heitmeyer, McLean, ACM TOCS, 1984.
37
WHY IS THIS A HARD PROBLEM?

• Should we use a top-down or bottom-up approach?
• How is the separation kernels environment
  represented?
• What is the security perimeter? The answer is not
  obvious
• Exactly what kernel does is not obvious
• Mediates access to storage (i.e., grant/revoke R,
  W, and X privileges)
• includes mediating access to temporary and
  working storage
• Handles interrupts
• Initiates processes in response to interrupts
• Copies sensitive data from one storage area to
  another
• When switching from one subtask to another,
  sanitizes storage areas
• We have largely mechanized proofs of invariant
  properties
• However, automated support of refinement requires
  extensions to our mechanized theorem proving
  support

The devil is in the details...
38
AUTOMATED TEST CASE GENERATION FROM REQUIREMENTS
SPECIFICATIONS
39
AUTOMATIC TEST CASE GENERATION

• Our technique
• Generates one or more test cases for each
  decision point in the spec
• Does so in a manner that covers all possible
  system executions described by the spec
• Uses req. spec. as an oracle to generate the
  system response associated with an input sequence
• Uses the counterexample capability of a model
  checker to construct the test cases

System Req. Spec
OUR FOCUS
Test Case

• Our approach to software testing
• specification-based
• black-box does the software satisfy
• the requirements specification?

Gargantini, Heitmeyer, ESEC/FSE, 1999.
40
SOME BASICS

• Test Case A sequence of inputs, each input
  paired
• with a set of outputs (also called Test
  Sequences)
• Test Suite A collection of test cases
• Test Predicate A predicate used to construct
  test cases
• (also called a Test Purpose)
• Goals of Test Case Generation
• The number of test sequences in the test suite
  should be as small
• as possible
• The test suite should cover all errors that
  any implementation
• may contain

NOTE To simplify the discussion, we refer to
inputs and outputs (rather than to monitored and
controlled variables).
41
EXAMPLE THE SAFETY INJECTION SYSTEM (SIS)

• Based on a control system in a real nuclear power
  plant
• System is required to turn on SafetyInjection
  when water pressure (WaterPres) falls below a
  threshold Low10.
• Operator can set a Block button to inhibit safety
  injection and a Reset button to reset the system
  after blockage
• We assume WaterPres may never increase by more
  than 3 from one state to the next and its
  initial value is 2.

reqd relation
assumption

Low Permit
Pressure
Mode Class
Term Overridden
Constants
High
TooLow
ControlledVariables
Monitored Variables
Permitted
WaterPres
Safety Injection
Safety Injection System
Block
Environment
Environment
Reset
Courtois,Parnas, Documentation for
Safety-Critical Software, Proc., ICSE93,
Baltimore.
42
CONSTRUCTING TEST PREDICATES FROM SYSTEM
PROPERTIES

• Suppose the SCR spec of SIS satisfies the
  following safety property
• To operate safely, all executions of the SIS
  must satisfy this
• safety property
• This property is an example of a test predicate
• Each test predicate may be used to generate
  one or more test cases.

_at_T(WaterPres lt Low) WHEN Block On ? Reset
Off ? SafetyInjection Off
43
LIMITATIONS OF THIS APPROACH

• A set of system properties may not be available
• The set of system properties is incomplete
  --i.e., all possible system behaviors are not
  usually covered

44
CONSTRUCTING TEST PREDICATESFROM A STATE MACHINE
SPEC
Event Table Defining the Mode Class Pressure
45
GENERATING MORE TEST PREDICATES
if Pressure TooLow if _at_T(WaterPres ?
Low) ? Pressure Permitted C1 (else) ?
Pressure Pressure C2 fi
Pressure Permitted if _at_T(WaterPres ?
Permit) ? Pressure High C3
_at_T(WaterPres lt Low) ? Pressure TooLow
C4 (else) ? Pressure Pressure
C5 fi Pressure High if
_at_T(WaterPres lt Permit) ? Pressure Permitted
C6 (else) ? Pressure Pressure
C7 fi fi
Alternate Representation of the Function Defining
Pressure With One else Clause per Mode
46
CONSTRUCTING TRAP PROPERTIES FROM SYSTEM
PROPERTIES
Suppose the SCR spec of SIS satisfies the
following safety property
Test Predicate P
_at_T(WaterPres lt Low) WHEN Block On ? Reset
Off ? SafetyInjection Off

• How to generate a test sequence from predicate P
• If Spec satisfies P and P (R ? S) , use Spec
  as the state machine model
• Then, use a model checker, say SMV, to check
  Spec for the property R
• This generates a counterexample, a system
  execution allowed by Spec that
• satisfies R (and S as well, since R ? S)
• This counterexample defines a test case that
  can be used to test the
• correctness of the software

47
CONSTRUCTING TRAP PROPERTIES FROM SYSTEM
PROPERTIES
Test Predicate P
_at_T(WaterPres lt Low) WHEN Block On ? Reset
Off ? SafetyInjection Off
P (R ? S) , where R _at_T(WaterPres lt Low)
WHEN Block On ? Reset Off S
SafetyInjection Off
We use TRAP PROPERTIES to force the model checker
to construct a counterexample -- i.e., a desired
test case
48
GENERATING A TEST SEQUENCE FROM A SYSTEM PROPERTY
Test predicate
_at_T(WaterPres lt Low) WHEN Block On ? Reset
Off ? SafetyInjection Off
49
GENERATING A TEST SEQUENCE FROM AN OPERATIONAL
SPEC
Pressure TooLow ? _at_T(WaterPres ? Low) ?
Pressure Permitted
Trap Property
assert(CasePressure)! 1

• The above trap property causes SPIN to construct
  a counterexample
• This counterexample is the basis for the
  following test case

Test Sequence Constructed from SPIN Counterexample
50
ACHIEVING BRANCH COVERAGE(ALSO CALLED DECISION
COVERAGE)
Our technique, called branch coverage, uses
the following rules to ensure coverage 1. In each
event table, each event is tested at least once.
2. In each condition table, each condition not
equivalent to false is tested at least once. 3.
In each event table, in each mode, each no-change
case is tested at least once. To produce more
test cases, rule 3 can be replaced with a
stronger rule 3. In each event table, in each
mode, a change in each monitored variable that
does not change the value of the variable the
table defines is tested at least once.
51
AUTOMATIC GEN. OF TEST CASESEXPERIMENTAL RESULTS

• SPIN vs. SMV
• SPIN produces very long counterexamples, while
  SMV produces the shortest possible
  counterexamples
• But, SMV produces more test sequences than SPIN
• Also, SMV does poorly on specs with wide
  numerical ranges and on large specs
• Infeasible Test Cases
• Some predicates define an impossible state
  transition
• By analyzing a test predicate, we can determine
  whether a test case is infeasible
• Apply assumptions, e.g., a transition from
  TooLow to High is ruled out because WaterPres can
  only change by 3
• Apply state invariants, e.g., IgnOn becomes false
  when CC mode doesnt change

52
APPLYING STATE INVARIANTS
53
USE OF STATE INVARIANTSIN SOFTWARE DEVELOPMENT

• Validation of requirements specification
• show invariants to domain experts to determine
  whether specification correctly captures the
  required behavior
• Verification of requirements specification
• use invariants as auxiliary lemmas in proving
  that the specification satisfies properties of
  interest
• use invariants and Incremental Compositional
  Proof Rule to prove properties more efficiently
  (Jeffords and Heitmeyer)
• Generation of counterexamples in formal analysis
  of spec
• use invariants to rule out candidate
  counterexamples (e.g., unreachable state pairs)
• Specification-based generation of test cases
• use invariants to rule out infeasible test cases
  (e.g., a state invariant rules out a given state
  transition)
• Optimizing code generated from the specification
• use invariants to detect parts of the
  specification that would lead to dead code (e.g.,
  unreachable code) or redundant code

54
APPLYING STATE INVARIANTSIN VERIFICATION
55
MORE EFFICIENT VERIFICATION USING COMPOSITION AND
INVARIANTS

• Incremental Compositional Proof Rule Makes
  formal analysis more efficient.
• a. Divide system ? into subsystems ?1 and ?2 .
  ?1 is based on one or more dependent variables
  from which invariants P constructed . ?2 based
  on remaining vars.
• b. Introduce invariants P of ?1 as additional
  constraints in ?2 .
• c. If we can prove properties of ?2 under the
  constraints P,
• then the original system ? satisfies the
  properties

56
APPLYING STATE INVARIANTSIN OPTIMIZATION
57
SYNTHESIZING SOURCE CODEFROM SPECIFICATIONS

• APPROACH
• Apply a sequence of
• transformations to
• the tabular spec
• Prove that each
• transformation is
• a refinement of the
• preceding spec
• This process produces
• a provably correct,
• efficient program

SCR Tabular Spec
Original spec produced by a user
equivalence
Use a grammar and a set of semantic rules to
translate the tabular spec to an equivalent
intermediate representation
Intermediate Spec
prove refinement
To produce efficient code, apply a set of
optimization techniques to the spec
Optimized Spec
prove refinement
Transform the optimized spec into source code
(e.g., C or Java)
Source Code
58
RELATED WORK

• Optimization
• R. Paige et al., "Finite differencing of
  computable expressions," TOPLAS, 1982.
• Refinement
• D. Smith, "KIDS A semiautomatic program
  development system, IEEE TSE, 1990.
• ...
• On-the-Fly Translation Validation
• A. Pnueli, Siegel, and Singerman, Translation
  validation, Proc. 4th Intern. Conf. on Tools and
  Analysis of systems (TACAS 98), LNCS 1384,
  Springer.
• G. Necula, "Translation validation for an
  optimizing compiler," Proc. PLDI, 2000.

59
ELIMINATE PARTS OF THE SPECTHAT ARE UNREACHABLE
true if _at_T(Blockon) WHEN Resetoff AND
Pressure in TooLow, Permitted AND
Overriddenfalse false if _at_T(PressureHigh)OR
_at_T(Reseton WHEN Pressure in TooLow,
Permitted AND Overriddentrue OR
_at_F(PressureHigh) WHEN Pressure High AND
Overriddentrue
Overridden
GENERATED STATE INVARIANT
Overridden true ? Pressure ? High
Unreachable parts of the spec will lead to dead
code
60
ELIMINATE PARTS OF THE SPECTHAT ARE UNREACHABLE
true if _at_T(Blockon) WHEN Resetoff AND
Pressure in TooLow, Permitted AND
Overriddenfalse false if _at_T(PressureHigh)OR
_at_T(Reseton WHEN Pressure in TooLow,
Permitted AND Overriddentrue
Overridden
GENERATED STATE INVARIANT
Overridden true ? Pressure ? High
Unreachable parts of the spec will lead to dead
code
61
IDENTIFY PARTS OF SPECTHAT ARE UNNEEDED
GENERATED STATE INVARIANTS
M Inactive ? IgnOn M Override ? IgnOn ?
EngRunning
These parts of the spec would lead to unnecessary
code
62
TWO APPROACHES TO USING FORMAL METHODS
63
USE OF FORMAL TECHNIQUESBY PRACTICING ENGINEERS

• TWO APPROACHES
• Practitioners will depend on experts to apply the
  techniques
• This currently is the predominant approach
• Practitioners themselves will apply the formal
  techniques and their supporting tools
• Movement in this direction is urgently needed
• Better education of practitioners required

But, before practitioners can apply formal
techniques and their support tools directly,
major obstacles must be overcome
64
MATHEMATICS VS.ENGINEERING
MATHEMATICALLY WELL-FOUNDED SOFTWARE ENGINEERING
DISCIPLINE
MATHEMATICAL RESOURCES (E.G., THEORIES)
Logics (predicate, 1st order,
higher order, etc.) Automata models Theories
underlying decision procedures Theories of
composition, refinement, etc. ...
Humans use this discipline, with support from
tools, to produce high-quality software
Problem How to achieve this?
65
WHAT RESEARCHERS CAN DO TO MAKE FORMAL TECHNIQUES
EASIER TO USE

• Notation that is easy to learn and understand
  (e.g., tabular) with an explicit formal semantics
• Model checking using automatic abstraction
• Integration of decision procedures with the
  formal techniques
• Pushbutton analysis
• e.g., automatic consistency checking
• Understandable feedback when an error is detected
• Automatically generated invariants

• Theorem provers customized for a particular class
  of systems
• e.g., TAME allows a user to specify and reason
  about automata models
• Theorem provers that are more natural and easy to
  use
• use strategies so that a step in the mechanized
  proof corresponds closely to a step in a hand
  proof
• generate natural language explanations

We need more good violins and less reliance on
highly talented violinists
Moshe Vardi CAV 98
66
MAKING FORMAL TECHNIQUES MORE ATTRACTIVE TO
SOFTWARE DEVELOPERS

• Integrate the formal techniques with a simulator
• simulation helps convince customer that you know
  about his domain
• simulation can be used
• to validate the specification, especially if one
  adds a fancy graphical interface
• to demonstrate/validate counterexamples produced
  by a model checker
• Integrate the formal techniques with software
  testing
• More generally, integrate the formal techniques
  into the software developers process

WEAPONS CONTROL PANEL
CRYPTO DEVICE
FLIGHT PROGRAM
67
TRANSFERRING FORMAL TECHNIQUESAND THEIR SUPPORT
TOOLS TO PRACTICE

• To some extent, transferring formal techniques
  and supporting technology is a non-technical
  problem
• Best hope is
• education and training
• finding champions, i.e., people in industry who
  are willing to try something new AND have a
  problem that the techniques and tools solve

no. of users
technology becomes an industry standard
basic research
increasing use of technology in indl pilot
projects
time
use of technology by risk-tolerant indl users
68
WIDER USE OF FORMAL TECHNIQUES BY INDUSTRY
INCENTIVES

• More demands (e.g., by governments) for
  convincing evidence that a high assurance
  software product satisfies its requirements
• Examples In the U.S., a number of products are
  being evaluated against the Internat. Common
  Criteria (e.g., firewalls, smartcards, etc.)
• As of July, 2002, the U.S. Navy is requiring all
  of its security products to satisfy the Common
  Criteria
• our group is developing a formal spec and
  verification of the separation kernel of a crypto
  system without an ICC Level 7 eval., NSA will
  not certify the device
• in the U.S., federal agencies, such as the Food
  and Drug Admn. and the Nuclear Regulatory Comm.,
  also want more assurance of the correctness of
  safety-critical systems (e.g., medical devices,
  control systems for nuclear plants)
• Because software testing is enormously expensive,
  there is growing interest in industry in
  automatic generation of test cases
• e.g., three sites of Lockheed Martin are using
  the SCR tools in conjunction with a tool for
  automatic test case generation
• our group also has a contract with NASA to apply
  automatic test case generation to their Fault
  Protection Engine
• producing the test cases requires a formal spec!

69
CONCLUSIONS

• UTILITY OF TOOLS (AND FORMAL TECHNIQUES) IN
  DEVELOPING/EVALUATING SOFTWARE SYSTEMS
• Find errors people miss
• Help in validating the spec
• Support formal verification
• Reduce the time/effort required to construct test
  cases and run them
• Provide more confidence in testing by
  constructing a carefully constructed suite of
  test cases
• Based on selected coverage criteria and a
  high-quality spec
• Support automatic construction of provably
  correct, efficient code

Libertates developers to do the hard mental work
required to develop high assurance systems
70
QUESTIONS?
71
THE END