Test Case Prioritization Based on Boundary Value Coverage

1
Prioritizing Test Cases for Regression Testing
Based on Boundary Value CoverageTao Xie and
David NotkinComputer Science Engineering,
University of Washington
What test cases are most likely to expose faults ?
Introduction
Standard characterizations of test cases include
Problem If resources are constrained, how can
we still perform regression testing effectively?
Structural coverage exercising structural
entities (e.g., statements, edges) that encode
faults is essential to exposing those faults ?
try to choose test cases that achieve high
coverage of those entities. Data Coverage
exercising entities with special values (e.g.,
boundary values) increases the ability to expose
many faults ? try to choose special values that
achieve high coverage of those entities.
Solutions Test Case Selection selects a subset
of test cases that exercise those changed
entities Test Case Prioritization orders the
test cases with the intent of executing the test
cases most likely to expose faults earlier
What special values are dangerous?
Boundary values tend to be dangerous.
Extensively discussed in practitioner and
researcher's testing handbooks The
Art of Software Testing by G. J. Meyers, 1979
Software Testing Techniques by B.
Beizer, 1990 and research
literatures .
Related WorkBoundary Value Coverage
Related Work Test Case Prioritization
Most existing applications of boundary value
coverage are in test case generation.
Most existing test case prioritization techniques
are based on structural coverage.
Supportive Evidence
Challenging Problem
is based on
Usually the specification or an assertion-like
annotation is required to infer boundary value.
Few programs are equipped with them. Lack of
tool support to collect boundary value coverage
information without a priori specification.
Previous empirical studies showed data coverage
techniques, e.g. boundary value coverage,
outperformed structural coverage ones in testing
effectiveness Complementing structural coverage
techniques with data coverage ones are suggested
in practice.
Our Solution Test Case Prioritization Based on
Boundary Value Coverage witho
ut a priori specification
Boundary predicates (used to infer boundary
values) are extracted from invariants dynamically
detected by Daikon, an invariant detection tool.
The boundary value coverage for a test case
execution is collected based on the data trace of
that execution and the boundary predicates. Then
test case prioritization can incorporate this
boundary value coverage information to improve
performance.
Our initial experiments show promise for boundary
value coverage in particular, boundary value
coverage dominated structural coverage for these
small programs.
This work was supported in part by the National
Science Foundation under grant ITR 0086003.