Systematic Testing of protocol Robustness by Evaluation of Synthesized Scenarios STRESS

1
Systematic Testing of protocol Robustness by
Evaluation of Synthesized Scenarios (STRESS)
USC
UNIVERSITY OF SOUTHERN CALIFORNIA

• STRESS Project Goals
• Provide methodologies for systematic design and
  evaluation of protocol robustness and performance
• Develop automatic test generation algorithms for
  test scenario synthesis topology, host events
  and fault generation, to be integrated with
  simulation
• Apply test generation at various stages of the
  protocol design for multicast routing, reliable
  multicast and multiparty applications
• Facilitate network diagnosis by providing
  behavioral profiles obtained through sensitivity
  analyses

Problem Dimensions

• STRESS Tools
• Automated and semi-automated tools for scenario
  generation
• Scenario Generator structured, random-based
• Simulation-based STRESS
• Fault-independent test generation
• Fault-oriented test generation
• Key issues
• Modeling using extended global FSM
• Techniques for forward and backward search with
  reduced complexity
• Methods for worst case test generation e.g. for
  timer mechanisms
• Studies of ripple effects and sensitivity
  analysis

Solution Spectrum
Fault-oriented Topology Synthesis Example
Equivalence to Reduce Search Complexity Example

• Loss of a Join message

E1,NR2,NR3
NR1,E2,NR3
NR1,NR2,E3
S1
J2
J3
...
...
...
...
F1,NR2,NR3
E1,R2,NR3
E1,NR2,R3
J3
J2
J3
...
...
S1
F1,NR2,R3
F1,R2,NR3
E1,R2,R3
J3
S1
Equivalent Subspace
...
...
Pruned Space
F1,R2,R3

• Ongoing work
• Quantitative comparison and complexity analysis
  of the search algorithms
• Application to other multicast protocols e.g.
  IGMP, SRM
• Integration of timing and delay semantics
• End-to-end protocol performance studies
• Ripple effect studies

• Initial Results
• Applied STRESS to PIM-SM and PIM-DM
• Detected several design/robustness errors loops,
  black-holes, extra overhead
• Achieved reduction from O(4n) to O(n4) for
  forward search for the case study protocol
• Synthesized worst case topology for
  request/response suppression timer mechanism

• Contributors (Co-PIs)
• USC Deborah Estrin, Sandeep Gupta, Ahmed Helmy,
  Ying Liu
• http//netweb.usc.edu/stress