Bayesian Graphical Models for Software Testing David A Wooff, Michael Goldstein, Frank P.A. Coolen

1
Bayesian Graphical Models for Software
TestingDavid A Wooff, Michael Goldstein, Frank
P.A. Coolen

• Presented By
• Scott Young

2
Applying Detailed Metrics to Testing

• Can provide insight for performing risk analysis
• Can provide concrete values to inform the
  customer or management, concerning software
  quality
• May result in more efficient testing procedures

3
Meaningful Metrics

• Simple metrics concerning testing can inform
  concerning the success of the testing process
• They also can provide insight for when the test
  process is reaching completion
• More complex testing analysis needs to be done in
  order to locate points of inefficiency in
  testing, as well as providing more fine-grained
  knowledge about test results

4
What is a Bayesian Graphical Model?

• Also commonly called a Bayesian Belief Network
• Is a directed graph, with nodes signifying
  indeterminate factors
• Bayesian models are most commonly heard of today
  in relation to email spam filtering
• They are used to calculate probability based on
  pre-defined knowledge and relationships between
  components

5
Software Actions (SAs)

• A Software Action is an individual, fine grained
  component of the software project which
  accomplishes a single task.
• An example of a software action in a system would
  be the processing of a credit card number.

6
Specifying Nodes

• A node should be a collection of operations with
  the same prior probability of failure, as well as
  the same change in probability of failure given a
  test covering that set of operations.

7
Factors For Defining Probabilities

• Level of code complexity
• Reliability comparison with existing code which
  has been evaluated
• Maturity of codebase
• Typical reliability of authors code
• Similarities to existing code

8
Updating The Model

• As testing continues, the model must be update in
  stepwise fashion to follow changes to components
  as they occur.
• The probability of an individual node can be
  updated according to multiple criteria (which are
  necessarily assumptions) about remaining defects.

9
What The Results Provide

• Tests should be arranged according to the
  software action(s) which they provide coverage
  for. Tests discovered to be redundant may be
  safely removed
• Results demonstrate the perceived probability (or
  strength of belief/confidence) that there are no
  more existing faults within each SA

10
What Does This Mean For VV?

• Software producers can demand a level of
  confidence for components from their testing
  according to the role of the software and
  potential financial impact of defects in specific
  components.

11
Drawbacks

• Informed knowledge is required in order to build
  a reliable model
• This informed knowledge still consists of
  assumptions of relationships (though an
  assumption within an order of magnitude can still
  provide useful results)
• The amount of additional work to formally track
  every SA may be prohibitive

12
Resources

• David A Wooff, Michael Goldstein, Frank P.A.
  Coolen, Bayesian Graphical Models for Software
  Testing. IEEE Transactions on Software
  Engineering, May 2002.
• Murray Cumming, Bayesian Belief Networks.
  http//www.murrayc.com/learning/AI/bbn.shtml Date
  unknown.
• Kevin Murphy, A brief introduction to Bayes
  Rule. http//www.ai.mit.edu/murphyk/Bayes/bayes
  rule.html, Jan 2004.