An Automatabased Approach to Testing Properties in Event Traces

1
An Automata-based Approach to Testing Properties
in Event Traces

• H. Hallal, S. Boroday, A. Ulrich, A. Petrenko

Sophia Antipolis, France, May 2003
2
Outline

• Motivation
• Event traces
• Problem
• Our approach
• Implementation
• Case study
• Conclusions and extensions

3
Motivation

• Analysis of distributed systems is complex and
  costly
• Asynchrony
• Lack of global timing
• Absence of reference specification
• A practical solution is to instrument the system
  to generate traces of events that can be
  visualized and analyzed further
• This solution can be used to debug the system
• During development
• After deployment

4
Visualization Vs Analysis Tools

• Visualization tools facilitate the manual
  inspection of collected traces

• Analysis tools automate the verification of
  properties in the traces

Analysis Tools

• elaborate ad-hoc algorithms
• more efficiency
• more efforts
• reuse an existing model checker
• more expressiveness
• less efforts

5
Trace Analysis Problem

• Given
• A distributed system under test (SUT)
• Some properties
• Verify whether the SUT satisfies the properties
• Solution
• Monitor the SUT and collect an execution trace
• Model the collected trace
• Use an existing model checker to verify the
  properties

6
Trace

• Distributed processes generate local traces
• Local events state update, parameter change
• Communication events message exchange, RMI, RPC
• Local traces are sequential
• Communication
• Asynchronous send and receive events
• Synchronous rendezvous events
• Point-to-point communication
• Each message has a send and a receive in the
  trace
• Each rendezvous involves at least two parties

7
Event Traces

• Event ordering induced by local orders ?i and
  point-to-point communication
• A trace is a partially ordered set E of all
  events
• Causality relation on events ?
• If a ?i b then a ? b 
• for every message m, send(m) ? receive(m)
• ? is transitive If a ? b and b ? c then a ? c
• Event trace
• a tuple of local traces with an irreflexive
  causality relation on all events

8
Lattice of Ideals

• Encodes all the possible linearizations of E

• Offers an efficient way to check properties

9
Problem

• Given
• An event trace of a distributed system
• A set of properties
• How to build the lattice of ideals to verify
  the properties?
• Monolithic approach
• build the lattice explicitly
• use a model checker
• Modular approach
• model the event trace as a system of
  communicating automata
• build the composition of automata
• prove it is isomorphic to the lattice

10
Our Approach

• We use finite automata to model
• Local traces of processes
• states are ideals
• transitions are events
• Message delays

• We build the composition of all automata
• We prove
• composition of automata ? lattice of ideals
• Use the composition automaton to verify the
  properties
• use an existing model checker
• avoid full state space search

11
Implementation

• We use SDL and ObjectGEODE (OG)
• We model the SUT as an SDL system
• Local traces designated processes
• Local events SDL TASK
• Communication signal exchange
• How to treat the message delay automata?
• Individual processes
• Individual queues
• SDL SAVE
• Properties are specified in GOAL of OG

12
Workflow of the Approach

• Front-End tool to ObjectGEODE
• System specification
• Pattern specification
• Library of property patterns
• Parameterized GOAL observers
• State-based, event-based, mixed

13
Pattern Library

• Property patterns already exist
• Repository of common properties
• Mappings to main formalisms used in finite state
  verification
• LTL, CTL, INCA, QRE,
• Library of GOAL observers
• Address finiteness of traces
• Encode common patterns
• Class order vs. occurrence
• Name response, universality, ...
• Scope global, before, after, ...
• Parameterized GOAL specification
• parameters are predicates on states, events, or
  both

14
Pattern Template

• Name and Intent
• Response
• Cause-effect relationship
• Class
• Order
• Scope
• Global the entire execution
• Example
• resource granted after request

S responds to P in the execution
15
TRAYSIS

• Input XML logfile
• Output SDL model
• Features
• Logfile conformance check
• Synchronous/asynchronous
• Statistics on the model
• processes, channels, variables, signals,...
• Model customization
• scalability
• Access to OG

16
Property Manager

• Supports property specification
• Easy access to library
• Customize observers

17
Case Study

• An implementation of the Sliding Window Protocol
• Extension to the PROFIBUS protocol stack
• Supports communication in distributed power
  control system
• Properties of interest
• Maximum window size is respected
• Total number of unacknowledged messages less than
  limit
• Total number of messages in transit less than
  limit
• Execution traces are collected using protocol
  analyzers
• We used out tool set to automatically analyze the
  system
• We have analyzed large traces (15k 20k events)

18
Conclusions and Future Work

• Formal definition of event traces
• A framework to model mixed communication modes
  (GALS)
• Automata-based approach to analyze event traces
• A component based implementation of the approach
• A case study the SWP
• Target more general logfiles
• Enhancement of the tool set

19

• Merci beaucoup!