Formal Methods and Testing: Possible Attributes for Success

1
Formal Methods and TestingPossible Attributes
for Success

• A. J. Cowling

Department of Computer Science University of
Sheffield
2
Rationale

• The Number of Different Methods
• Many formal and semi-formal methods exist
• New ones are still being created
• Their Usefulness
• Determining which are useful requires empirical
  work
• The number of methods would imply a lot of work
• Therefore priorities need to be set
• Technical Features
• Some methods appear to be more successful than
  others
• Possibly because of their technical features
• These would be more appropriate candidates for
  study

3
Role of Models
4
Testing and Models

• Basic Testing Methods
• Base the generation of test cases on one of these
  models
• Black-box testing uses the specification model
• White-box testing uses the implementation model
• Hybrid Testing Methods
• Combine the approaches eg
• Black-box methods to generate the test sets
• White-box methods to measure their coverage
• May provide more effective testing than
  individual basic methods
• At least, according to some papers
• State-based Testing
• Uses state-machine models for specification and
  implementation
• Extended models (eg the X-machine) allow powerful
  results
• absence of faults up to some bounds,
• under some assumptions, complete absence of faults

5
Key Formal Methods

• Model Checking
• Requires state-based specification models
• Shows whether required properties hold for the
  models
• Can handle very large systems (1020 states)
• Machine Model Verification
• Uses state-based specification and implementation
  models (eg B)
• Can verify that implementation is consistent with
  specification
• Refinement
• Typically uses relational models (eg Z, VDM)
• Refinement steps produce correct-by-construction
  implementations
• Discontinuities in the models need to be
  accommodated
• Retrenchment has been proposed for this

6
Attributes for Success

• Role of Models
• Successful approaches appear to all be
  model-based
• State machine models are particularly successful
• Extended state-machine models even more so
• Differences between Models
• The different stages require different models for
  one system
• Any form of V V must accommodate these
  differences
• ie must represent design transformations
• currently an interest within model-driven
  architecture
• It appears that successful methods
• explicitly handle multiple models, and
• explicitly represent the differences between them.