272: Software Engineering Fall 2008 - PowerPoint PPT Presentation

Loading...

PPT – 272: Software Engineering Fall 2008 PowerPoint presentation | free to download - id: 169881-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

272: Software Engineering Fall 2008

Description:

272: Software Engineering. Fall 2008. Instructor: Tevfik Bultan ... Omega. Library. CUDD. Package. MONA. Composite Symbolic Library. Presburger. Arithmetic ... – PowerPoint PPT presentation

Number of Views:51
Avg rating:3.0/5.0
Slides: 104
Provided by: ValuedSony
Learn more at: http://www.cs.ucsb.edu
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: 272: Software Engineering Fall 2008


1
272 Software Engineering Fall 2008
  • Instructor Tevfik Bultan
  • Lectures 13 Modularity, Interfaces and
    Verification

2
Joint Work with My Students
  • Action Language Verifier (ALV)
  • Tuba Yavuz-Kahveci, University of Florida,
    Gainesville
  • Web Service Analysis Tool (WSAT)
  • Xiang Fu, Hofstra University
  • (co-advised with Jianwen Su)
  • Design for verification
  • Aysu Betin-Can, Middle East Technical University
  • Interface Grammars
  • Graham Hughes, PhD candidate
  • http//www.cs.ucsb.edu/bultan/vlab/publications.h
    tml

3
Outline
  • Motivation
  • PART 1 Verification of Synchronization Policies
    in Concurrent Programs via Interfaces
  • PART 2 Verification of Conversations among Web
    Services via Interfaces
  • PART 3 Modular Verification with Interface
    Grammars
  • Conclusions

4
Model Checking Software
  • Model checking
  • An automated software verification technique
  • Exhaustive exploration of the state space of a
    program to find bugs
  • Systematically explore all possible behaviors of
    a program
  • look for violations of the properties of interest
  • assertion violations, deadlock
  • Software model checkers Verisoft, Java
    PathFinder (JPF), SLAM, BLAST, CBMC

5
Two Challenges in Model Checking
  • State space explosion
  • Exponential increase in the state space with
    increasing number of variables and threads
  • State space includes everything threads,
    variables, control stack, heap
  • Environment generation
  • Finding models for parts of software that are
  • either not available for analysis, or
  • are outside the scope of the model checker

6
Modular Verification
  • Modularity is key to scalability of any
    verification technique
  • Moreover, it can help in isolating the behavior
    you wish to focus on, removing the parts that are
    beyond the scope of your verification technique
  • Modularity is also a key concept for successful
    software design
  • The question is finding effective ways of
    exploiting the modularity in software during
    verification

7
Interfaces for Modularity
  • How do we do modular verification?
  • Divide the software to a set of modules
  • Check each module in isolation
  • How do we isolate a module during
    verification/testing?
  • Provide stubs representing other modules
    (environment)
  • How do we get the stubs representing other
    modules?
  • Write interfaces
  • Interfaces specify the behavior of a module from
    the viewpoint of other modules
  • Generate stubs from the interfaces

8
Interfaces and Modularity Basic Idea
  • Write interface specifications for the modules
  • Automatically generate stubs from the interface
    specifications
  • Automatically generated stubs provide the
    environment during modular verification

9
Three Applications
  • I will talk about three different instantiations
    of this basic idea
  • Verification of synchronization policies in
    concurrent programs using finite state interfaces
  • Verification of conversations among web services
    using finite state interfaces
  • Verification of sequential interactions using
    interface grammars

10
PART 1 Concurrency Controller Pattern for
Synchronization
11
An Infinite State Model Checker
Action Language Specification CTL property
Action Language Verifier (ALV)
Action Language Parser
Composite Symbolic Library
Model Checker
Omega Library
CUDD Package
MONA
Counter-example
Verified
BDD Manipulator
Automata Manipulator
Not sure
Presburger Arithmetic Manipulator
12
What Can One Do with ALV?
  • Check if a specification satisfies a CTL property
  • For example, check a Read-Write lock
    implementation

integer nr boolean busy initial !busy and
nr0 r_enter !busy nr nr1 r_exit nr
nr-1 w_enter !busy nr0 busy
true w_exit busy false
13
Read-Write Lock in Action Language
S Cartesian product of variable domains
defines the set of states
  • module main()
  • integer nr
  • boolean busy
  • restrict nrgt0
  • initial nr0 and !busy
  • module ReaderWriter()
  • enumerated state idle, reading, writing
  • initial stateidle
  • r_enter stateidle and !busy and nrnr1
    and statereading
  • r_exit statereading and nrnr-1 and
    stateidle
  • w_enter stateidle and !busy and nr0 busy
    and statewriting
  • w_exit statewriting and !busy and
    stateidle

I Predicates defining the initial states
R Atomic actions of a single process
R Transition relation of a process, defined as
asynchronous composition of its atomic actions
R Transition relation of main, defined as
asynchronous composition of finite but arbitrary
number of reader-writer modules
14
Arbitrary Number of Threads?
  • How do we check arbitrary number of threads?
  • Counting abstraction
  • Create an integer variable for each thread state
  • Each variable counts the number of threads in a
    particular state
  • Generate updates and guards for these variables
    based on the specification
  • Counting abstraction is automated

15
Parameterized Read-Write Lock
module main() integer nr boolean busy
parameterized integer numReaderWriter
restrict nrgt0 and numReaderWritergt1
initial nr0 and !busy module
ReaderWriter() integer idle, reading,
writing initial idlenumReaderWriter
r_enter idlegt0 and !busy and nrnr1
and idleidle-1 and readingreading1
r_exit readinggt0 and nrnr-1 and
readingreading-1 and
idleidle1 w_enter idlegt0 and !busy and
nr0 and busy and idleidle-1 and
writingwriting1 w_exit writinggt0 and
!busy and writingwriting-1 and
idleidle1 ReaderWriter r_enter
r_exit w_enter w_exit endmodule main
ReaderWriter() spec invariant(busy gt nr0)
spec invariant(busy gt eventually(!busy))
endmodule
16
Read-Write Lock Verification with ALV
Integers Booleans Cons. Time (secs.) Ver. Time (secs.) Memory (Mbytes)
RW-4 1 5 0.04 0.01 6.6
RW-8 1 9 0.08 0.01 7
RW-16 1 17 0.19 0.02 8
RW-32 1 33 0.53 0.03 10.8
RW-64 1 65 1.71 0.06 20.6
RW-P 7 1 0.05 0.01 9.1
17
Read-Write Lock in Java
  • class ReadWriteLock    private Object lockObj   
    private int totalReadLocksGiven   private boolean
     writeLockIssued   private int threadsWaitingForW
    riteLock   public ReadWriteLock()      lockObj 
     new Object()     writeLockIssued  false     
      public void getReadLock()      synchronized (lo
    ckObj)        while ((writeLockIssued)  (thread
    sWaitingForWriteLock ! 0))          try        
        lockObj.wait()          catch (InterruptedEx
    ception e)                        totalReadLock
    sGiven           public void getWriteLock() 
        synchronized (lockObj)        threadsWaiting
    ForWriteLock       while ((totalReadLocksGiven
     ! 0)  (writeLockIssued))          try       
         lockObj.wait()          catch (InterruptedE
    xception e)            //                      
     threadsWaitingForWriteLock--       writeLockIssu
    ed  true           public void done()      sy
    nchronized (lockObj)        //check for errors  
         if ((totalReadLocksGiven  0)  (!writeLock
    Issued))          System.out.println(" Error Inv
    alid call to release the lock")         return  
                if (writeLockIssued)         writeLoc
    kIssued  false       else         totalReadLocks
    Given--       lockObj.notifyAll()        

How do we translate this to Action Language?
Action Language Verifier
Verification of Synchronization in Java Programs
18
A Design for Verification Approach
  • Our design for verification approach is based on
    the following principles
  • Use of design patterns that facilitate automated
    verification
  • Use of stateful, behavioral interfaces which
    isolate the behavior and enable modular
    verification
  • An assume-guarantee style modular verification
    strategy that separates verification of the
    behavior from the verification of the conformance
    to the interface specifications
  • A general model checking technique for interface
    verification
  • Domain specific and specialized verification
    techniques for behavior verification

19
Concurrency Controller Pattern
ThreadA
Shared
ThreadB
Helper classes
int
used at runtime
used during interface verification
used both times
20
Concurrency Controller Pattern
  • Avoids usage of error-prone Java synchronization
    primitives
  • synchronize, wait, notify
  • Separates controller behavior from the threads
    that use the controller
  • Supports a modular verification approach that
    exploits this modularity for scalable verification

21
Reader-Writer Controller
This helper class is provided. No need to rewrite
it!
  • class Action
  • protected final Object owner
  • private boolean GuardedExecute()
  • boolean resultfalse
  • for(int i0 iltgcV.size() i) try
  • if(((GuardedCommand)gcV.get(i)).guard())
  • ((GuardedCommand)gcV.get(i)).update()
  • resulttrue break
  • catch(Exception e)
  • return result
  • public void blocking()
  • synchronized(owner)
  • while(!GuardedExecute())
  • tryowner.wait()
  • catch (Exception e)
  • owner.notifyAll()
  • public boolean nonblocking()
  • class RWController implements RWInterface
  • int nR boolean busy
  • final Action act_r_enter, act_r_exit
  • final Action act_w_enter, act_w_exit
  • RWController()
  • ...
  • gcs new Vector()
  • gcs.add(new GuardedCommand()
  • public boolean guard()
  • return (nR 0 !busy)
  • public void update()busy true
  • )
  • act_w_enter new Action(this,gcs)
  • public void w_enter()
  • act_w_enter.blocking()
  • public boolean w_exit()
  • return act_w_exit.nonblocking()
  • public void r_enter()

22
Controller Interfaces
  • A controller interface defines the acceptable
    call sequences for the threads that use the
    controller
  • Interfaces are specified using finite state
    machines
  • public class RWStateMachine implements
    RWInterface
  • StateTable stateTable
  • final static int idle0,reading1,writing2
  • public RWStateMachine() ...
    stateTable.insert("w_enter",idle,writing)
  • public void w_enter()
  • stateTable.transition("w_enter")
  • ...

reading
r_enter
r_exit
idle
w_exit
writing
w_enter
23
Modular Design / Modular Verification
Thread Modular Interface Verification
Concurrent Program
Thread 1
Thread 2
Thread n
Thread 1
Thread 2
Thread n
Interface Machine
Interface Machine
Interface Machine
Interface
Controller
Modular Behavior Verification
Shared Data
Controller Behavior
24
Behavior Verification
  • Analyzing properties (specified in CTL) of the
    synchronization policy encapsulated with a
    concurrency controller and its interface
  • Verify the controller properties assuming that
    the user threads adhere to the controller
    interface
  • Behavior verification with Action Language
    Verifier
  • We wrote a translator which translates controller
    classes to Action Language
  • Using counting abstraction we can check
    concurrency controller classes for arbitrary
    number of threads

25
Interface Verification
  • A thread is correct with respect to an interface
    if all the call sequences generated by the thread
    can also be generated by the interface machine
  • Checks if all the threads invoke controller
    methods in the order specified in the interfaces
  • Checks if the threads access shared data only at
    the correct interface states

26
Interface Verification
  • Interface verification with Java PathFinder
  • Verify Java implementations of threads
  • Correctness criteria are specified as assertions
  • Look for assertion violations
  • Assertions are in the StateMachine and SharedStub
  • Performance improvement with thread Isolation
  • thread modular verification

27
Thread Isolation Part 1
  • Interaction among threads
  • Threads can interact with each other in only two
    ways
  • invoking controller actions
  • Invoking shared data methods
  • To isolate the threads
  • Replace concurrency controllers with controller
    interface state machines
  • Replace shared data with shared stubs

28
Thread Isolation Part 2
  • Interaction among a thread and its environment
  • Modeling threads calls to its environment with
    stubs
  • File I/O, updating GUI components, socket
    operations, RMI call to another program
  • Replace with pre-written or generated stubs
  • Modeling the environments influence on threads
    with drivers
  • Thread initialization, RMI events, GUI events
  • Enclose with drivers that generate all possible
    events that influence controller access

29
Verification Framework
Behavior Verification
Action Language Verifier
Controller Behavior Machine
Controller Classes
Counting Abstraction
Concurrent Program
Controller Interface Machine
Interface Verification
Java Path Finder
Thread
Thread Isolation
Thread
Thread Class
Thread Classes
30
A Case Study TSAFE
  • Tactical Separation Assisted Flight Environment
    (TSAFE) functionality
  • Display aircraft position
  • Display aircraft planned route
  • Display aircraft future projected route
    trajectory
  • Show conformance problems between planned and
    projected route

31
TSAFE Architecture
ltltTCP/IPgtgt
User
Server
Radar feed
Client
EventThread
ltltRMIgtgt
21,057 lines of code with 87 classes
32
Reengineering TSAFE
  • Found all the synchronization statements in the
    code
  • (synchronize, wait, notify, notifyAll)
  • Identified 6 shared objects protected by these
    synchronization statements
  • Used 2 instances of a reader-writer controller
    and 3 instances of a mutex controller for
    synchronization
  • In the reengineered TSAFE code the
    synchronization statements appear only in the
    Action helper class provided by the concurrency
    controller pattern

33
Behavior Verification Performance
Controller Time(sec) Memory (MB) P-Time (sec) P-Memory (MB)
RW 0.17 1.03
Mutex 0.01 0.23
Barrier 0.01 0.64
BB-RW 0.13 6.76
BB-Mutex 0.63 1.99
8.10 12.05
0.98 0.03
0.01 0.50
0.63 10.80
2.05 6.47
P denotes parameterized verification for
arbitrary number of threads
34
Interface Verification Performance
Thread Time (sec) Memory (MB)
TServer-Main 67.72 17.08
TServer-RMI 91.79 20.31
TServer-Event 6.57 10.95
TServer-Feed 123.12 83.49
TClient-Main 2.00 2.32
TClient-RMI 17.06 40.96
TClient-Event 663.21 33.09
35
Effectiveness in Finding Faults
  • Created 40 faulty versions of TSAFE by fault
    seeding
  • Each version had at most one interface fault and
    at most one behavior fault
  • 14 behavior and 26 interface faults
  • Among 14 behavior faults ALV identified 12 of
    them
  • 2 uncaught faults were spurious
  • Among 26 interface faults JPF identified 21 of
    them
  • 2 of the uncaught faults were spurious
  • 3 of the uncaught faults were real faults that
    were not caught by JPF

36
Falsification Performance
Thread Time (sec) Memory (MB)
TServer-RMI 29.43 24.74
TServer-Event 6.88 9.56
TServer-Feed 18.51 94.72
TClient-RMI 10.12 42.64
TClient-Event 15.63 12.20
Concurrency Controller Time (sec) Memory (MB)
RW-8 0.34 3.26
RW-16 1.61 10.04
RW-P 1.51 5.03
Mutex-8 0.02 0.19
Mutex-16 0.04 0.54
Mutex-p 0.12 0.70
37
Conclusions
  • ALV performance
  • Cost of parameterized verification was somewhere
    between concrete instances with 8 and 16 threads
  • Falsification performance was better than
    verification
  • Completeness of the controller properties
  • Effectiveness of behavior verification by ALV
    critically depends on the completeness of the
    specified properties
  • Concrete vs. parameterized behavior verification
  • When no faults are found, the result obtained
    with parameterized verification is stronger
  • However for falsification we observed that
    concrete instances were as effective as
    parameterized instances

38
Conclusions
  • JPF performance
  • Typically falsification performance is better
    than verification performance
  • In some cases faults caused execution of new code
    causing the falsification performance to be worse
    than verification performance
  • Thread isolation
  • Automatic environment generation for threads
    result in too much non-determinism and JPF runs
    out of memory
  • Dependency analysis was crucial for mitigating
    this
  • Deep faults were difficult to catch using JPF
  • Three uncaught faults were created to test this

39
Conclusions
  • Unknown shared objects
  • The presented approach does not handle this
    problem
  • Using escape analysis may help
  • We could not find a scalable and precise escape
    analysis tool
  • Environment generation
  • This is the crucial problem in scalability of the
    interface verification
  • Using a design for verification approach for
    environment generation may help

40
PART 2 Peer Controller Pattern for Web Services
41
Web Services
  • Loosely coupled, interaction through standardized
    interfaces
  • Standardized data transmission via XML
  • Asynchronous messaging
  • Platform independent (.NET, J2EE)

WSCDL
Interaction
WSBPEL
Composition
Service
WSDL
Implementation Platforms
Microsoft .Net, Sun J2EE
SOAP
Message
XML Schema
Type
XML
Data
Web Service Standards
42
Web Service Conversations
  • A composite web service consists of
  • a finite set of peers
  • and a finite set of message classes
  • The messages among the peers are exchanged using
    reliable and asynchronous messaging
  • FIFO and unbounded message queues
  • A conversation is a sequence of messages
    generated by the peers during an execution

43
Properties of Conversations
  • The notion of conversation enables us to reason
    about temporal properties of the composite web
    services
  • LTL framework extends naturally to conversations
  • LTL temporal operators
  • X (neXt), U (Until), G (Globally), F (Future)
  • Atomic properties
  • Predicates on message classes (or contents)
  • Example G ( payment ? F receipt )
  • Model checking problem Given an LTL property,
    does the conversation set satisfy the property?


44
Top-Down vs. Bottom-Up
confirm
query
Agent
Customer
Hotel
reserve
suggest
suggest
?
reserve
Conversation Protocol
G(query? F(confirm))
confirm
query
suggest
query
Agent
Customer
Hotel
!query
?query
Input Queue
!query
?reserve
?suggest
?query
!suggest
!confirm
!reserve
?suggest
?confirm
!suggest
...
?
Conversation
G(query? F(confirm))
45
Realizability Analysis
  • Conversation protocol specifies the global
    communication behavior
  • How do we implement the peers?
  • How do we obtain the contracts that peers have to
    obey from the global contract specified by the
    conversation protocol?
  • Project the global protocol to each peer
  • By dropping unrelated messages for each peer
  • A conversation protocol is realizable if the
    following are equal
  • Conversations specified by the conversation
    protocol
  • Conversations generated by the projected services

46
Synchronizability Analysis
  • A composite web service is synchronizable, if its
    conversation set does not change
  • when asynchronous communication is replaced with
    synchronous communication
  • If a composite web service is synchronizable we
    can check its properties about its conversations
    using synchronous communication semantics
  • For finite state peers this is a finite state
    model checking problem

47
Web Service Analysis Tool (WSAT)
Verification Languages
Web Services
Front End
Analysis
Back End
Intermediate Representation
GFSA to Promela (synchronous communication)
success
BPEL to GFSA
SynchronizabilityAnalysis
Guarded automata
BPEL
fail
(bottom-up)
GFSA to Promela (bounded queue)
Promela
skip
GFSA parser
Conversation Protocol
Guarded automaton
GFSA to Promela (single process, no
communication)
success
Realizability Analysis
fail
(top-down)
48
Checking Service Implementations
  • There are some problems
  • People write web service implementations using
    programming languages such as Java, C, etc.
  • Synchronizability analysis works on state machine
    models
  • How do we generate the state machines from the
    Java code?

Synchronizability Analysis
Checking Service Implementations
49
Design for Verification Approach
  • Use the same principles
  • Use of design patterns that facilitate automated
    verification
  • Use of stateful, behavioral interfaces which
    isolate the behavior and enable modular
    verification
  • An assume-guarantee style modular verification
    strategy that separates verification of the
    behavior from the verification of the conformance
    to the interface specifications
  • A general model checking technique for interface
    verification
  • Domain specific and specialized verification
    techniques for behavior verification

50
Peer Controller Pattern
used at runtime
Red Bordered classes are the ones the user has to
implement
used during interface verification
used both times
51
Peer Controller Pattern
  • Eases the development of web services
  • Uses Java API for XML messaging (JAXM)
  • Asynchronous communication among peers
  • Supported by a modular verification technique
  • Behavior Verification Checks properties of
    conversations of a web service composed of
    multiple peers
  • assuming that peers behave according to their
    interfaces
  • Interface Verification Checks if each peer
    behaves according to its interface

52
Modular Design / Modular Verification
Peer Modular Interface Verification
Composite Service
Peer 1
Peer 2
Peer n
Peer 1
Peer 2
Peer n
Interface Machine
Interface Machine
Interface Machine
interface
interface
interface
Modular Conversation Verification
Conversation Behavior
53
Verification Framework
Thread
Promela
WSAT
Thread
Peer State Machines
Promela Translation
Synchronizability Analysis
Conversation Verification
Composite Service
Spin
Peer State Machine
Interface Verification
Java Path Finder
Peer Code
54
Behavior Verification
  • Uses WSAT for synchronizability analysis
  • Uses Spin model checker for conversation
    verification
  • Automated translation to Promela using WSAT
  • Spin is a finite state model checker
  • We have to bound the channel sizes, session
    numbers, message types
  • Synchronizability analysis
  • Enables us to verify web services efficiently by
    replacing communication channels with channels of
    size 0 (i.e., synchronous communication)
  • The verification results hold for unbounded
    channels

55
Interface Verification
  • If the call sequence to the Communicator class is
    accepted by the state machine specifying the
    interface, then the peer implementation conforms
    to the behavior in the contract
  • Uses JPF model checker

56
Interface Verification
  • Isolated check of individual peer implementations
  • CommunicationController is replaced with
    CommunicatorInterface
  • Drivers simulating other peers are automatically
    generated
  • State Space reduction
  • Usage of stubs
  • Each session is independent
  • just need to check each peer for one session

57
Examples
  • We used this approach to implement several simple
    web services
  • Travel agency
  • Loan approver
  • Product ordering
  • Performance of both interface and behavior
    verification were reasonable

58
Interface Verification
Interface Verification with JPF for Loan Approver
Threads T (sec) M (MB)
Customer 8.86 3.84
Loan Approver 9.65 4.7
Risk Assesor 8.15 3.64
59
Behavior Verification
  • Sample Property Whenever a request with a small
    amount is sent, eventually an approval message
    accepting the loan request will be sent.
  • Loan Approval system has 154 reachable states
  • because queue lengths never exceeds 1
  • Behavior verification used lt1 sec and 1.49 MB
  • SPIN requires restricted domains
  • Have to bound the channel sizes ? bounded message
    queues
  • In general there is no guarantee these results
    will hold for other queue sizes
  • Using synchronizability analysis we use queues of
    size 0 and still guarantee that the verification
    results hold for unbounded queues!

60
Conclusions
  • We were able to use our design for verification
    approach based on design patterns and behavioral
    interfaces in different domains
  • Use of domain specific behavior verification
    techniques has been very effective
  • Interface verification is challenging
  • Model checking research resulted in various
    verification techniques and tools which can be
    customized for specific classes of software
    systems
  • Automated verification techniques can scale to
    realistic software systems using design for
    verification approach

61
Conclusions
  • Once the behavior is isolated (using concurrency
    controller or peer controller patterns) behavior
    verification was quite efficient
  • Interface verification was very hard
  • It is necessary to find effective behavioral
    interface specification and verification
    techniques

62
PART 3 Interface Grammars
63
Interface Grammars
Interface Compiler
Interface Grammar
Component B
Interface Grammar
Component B Stub
Component A
Model Checker
Component A
64
An Example
  • An interface grammar for transactions
  • Specifies the appropriate ordering for method
    calls to a transaction manager
  • Method calls are the terminal symbols of the
    interface grammar

Start ? Base Base ? begin Tail Base
e Tail ? commit rollback
65
An Example
  • Consider the call sequence
  • begin rollback begin commit
  • Here is a derivation
  • Start
  • Base
  • begin Tail Base
  • begin rollback Base
  • begin rollback begin Tail Base
  • begin rollback begin commit Base
  • begin rollback begin commit

Start ? Base Base ? begin Tail Base
e Tail ? commit rollback
66
Another Example
  • This interface can also be specified as a Finite
    State Machine (FSM)
  • However, the following grammar, which specifies
    nested transactions, cannot be specified as a FSM

begin
commit rollback
Start ? Base Base ? begin Base Tail Base
e Tail ? commit rollback
67
Yet Another Example
  • Lets add another method called setrollbackonly
    which forces all the pending transactions to
    finish with rollback instead of commit
  • We achieve this by extending the interface
    grammars with semantic predicates and semantic
    actions

Start ? rfalse l0 Base Base ? begin
ll1 Base Tail ll-1 if l0 then
rfalse Base setrollbackonly rtrue
Base e Tail ? rfalse commit rollback
68
Interface Grammar Translation
  • Our interface compiler translates interface
    grammars to executable code
  • the generated code is the stub for the component
  • The generated code is a parser that
  • parses the incoming method calls
  • while making sure that the incoming method calls
    conform to the interface grammar specification

69
Verification with Interface Grammars
Interface Compiler
Interface Grammar
Top-down parser
Program
parser stack
method invocation (lookahead)
Component Stub
parse table
semantic predicates and semantic actions
Model Checker
70
A Case Study
  • We wrote an interface grammar for the EJB 3.0
    Persistence API
  • This is an API specification for mapping Java
    object graphs to a relational database
  • Hibernate is an implementation of this API
  • Hibernate distribution contains several example
    test clients that are designed to fail and test
    exceptional behavior by violating the interface
    specification

71
A Case Study, Continued
  • We used these simple clients to check the
    fidelity of the stub generated from our interface
    specification
  • We used the JPF software model checker
  • None of these examples can run under JPF directly
  • Time taken to develop the interface was dominated
    by the need to understand EJB Persistence first
  • about a couple of hours

72
Experiments Falsification
Client 1
Client 2
Client 3
Client 4
sec. sec. sec. sec. obj. iter.
1.9 1.8 1.8 2.1 1 1
1.9 1.8 1.8 2.1 1 5
1.9 1.8 1.8 2.1 5 1
1.9 1.8 1.8 2.1 5 5
73
Experiments Verification
Client 1
Client 2
Client 3
Client 4
sec. sec. sec. sec. obj. iter.
3.1 1.9 1.8 2.5 1 1
26.9 11.1 9.9 17.1 1 5
10.1 5.9 4.2 11.8 5 1
126.3 63.9 43.5 138.5 5 5
74
A Case Study, Continued
  • For these simple clients, interface violations
    can be detected by JPF in a couple of seconds
    using the EJB stub generated by our interface
    compiler
  • Falsification time does not increase with the
    number of operations executed or the number of
    objects created by the clients
  • When we fix the errors, JPF is able to verify the
    absence of interface violations
  • Verification time increases with the number of
    operations executed or the number of objects
    created by the clients

75
Interface Grammars Uni/Bidirectional
  • Interface grammars can be
  • Unidirectional No callbacks
  • Bidirectional Need to handle Callbacks

Caller
Callee
Interface
Comp A
Comp B
Interface
76
Interface Grammars Client/Server
  • Interface grammars can be used for
  • Client verification Generate a stub for the
    server
  • Server verification Generate a driver for the
    server

Client
Stub
Interface Compiler
Interface
Server
Driver
77
Semantic Elements in JML
  • To handle both client and server side
    verification
  • We need to generate stubs and drivers from the
    same specification
  • Semantic predicate in one direction becomes a
    semantic action in the other direction and visa
    versa
  • We focused on a subset of Java Modeling Language
  • A restricted subset that is reversable
  • Semantic predicates and actions are written in
    this subset
  • Interface compiler automatically generates code
    from them both for client and server side
    verification

78
Interface Grammars and Data
  • A crucial part of the interface specification is
    specifying the allowable values for the method
    arguments and generating allowable return values
  • Approach 1 These can be specified in the
    semantic actions and semantic predicates of the
    interface grammars
  • Approach 2 Can we specify the constraints about
    the arguments and return values using the grammar
    rules?
  • Yes, grammar productions can be used to specify
    the structure of most recursive data structures.

79
Checking Arguments
  • A crucial part of the interface specification is
    specifying the allowable values for the method
    arguments and generating allowable return values
  • In what I discussed so far all these are done in
    the semantic actions and semantic predicates
  • The question is, can we specify the constraints
    about the arguments and return values using the
    grammar rules
  • Recursive data structures are especially good
    candidates for this!

80
Shape Types
  • Shape types Fradet, Metayer, POPL 97 provide a
    formalism for specifying recursive data
    structures
  • It is a specification formalism based on graph
    grammars
  • Shape types can be used to specify the
    connections among the heap allocated objects
  • Objects become the parameters of the nonterminals
    and the constraints on the connections among the
    objects are specified on the right-hand-sides of
    the grammar rules (similar to semantic predicates)

81
Shape Type for Doubly Linked List
Doubly ? p x, prev x null, L x L x ? next x y,
prev y x, L y L x ? next x null
p
next
next
next
prev
next
1
2
3
4
prev
prev
prev
  • Doubly ? p 1, prev 1 null, L 1
  • next 1 2, prev 2 1, L 2
  • next 2 3, prev 3 2, L 3
  • next 3 4, prev 4 3, L 4
  • next 4 null

82
Shape Type for Binary Tree
Bintree ? p x, B x B x ? left x y, right x z, B
y, B z B x ? left x null, right x null
p
1
right
left
2
3
right
right
left
left
5
4
left
right
right
left
83
Extension to Interface Grammars
  • In order to support shape types we extend the
    interface grammars as follows
  • We allow nonterminals with parameters
  • This extension is sufficient since the
    constraints about the connections among the
    objects can be stated using semantics predicates
    and semantic actions

84
Interface Grammars Shape Types
Doubly ? p x, prev x null, L x L x ? next x y,
prev y x, L y L x ? next x null
Doublyx ? ensure x \new(Node)
x.getPrev() null Lx Lx ? Node
y ensure y \new(Node) x.getNext() y
y.getPrev() x Ly
ensure x.getNext() null
85
Interface Grammars Shape Types
Bintree ? p x, B x B x ? left x y, right x z, B
y, B z B x ? left x null, right x null
Bintreex ? ensure x \new(Node) Bx
Bx ? Node y, z ensure y
\new(Node) z \new(Node) x.getLeft()
y x.getRight() z By Bz
ensure x.getLeft() null x.getRight()
null
86
Objection Generation vs. Validation
  • The use of shape types in interface grammars has
    two purposes
  • For the objects that are passed as method
    arguments we need to check that their shape is
    allowed by the shape type
  • We call this object validation
  • For the objects that are returned by the
    component we need to generate an object that is
    allowed by the shape type
  • We call this object generation

87
Object Generation vs. Validation
  • Object generation and validation tasks are
    broadly symmetric
  • The set of nonterminals and productions used for
    object generation and validation are the same and
    are dictated by the shape type specification
  • In object generation semantic actions are used to
    set the fields of objects to appropriate values
    dictated by the shape type specification
  • In object validation these are constraints that
    are checked using semantic predicates specified
    as guards
  • Given the semantic elements specified in JML, our
    interface compile generates code both for object
    generation and validation

88
Object Generation vs. Validation
  • There is a minor problem with object validation
  • In shape type specifications, the assumption is
    that there is no aliasing among the objects
    unless it is explicitly specified
  • This assumption is easy to enforce during object
    generation since every new statement creates a
    new object that has nothing else pointing to it
  • In order to enforce the same constraint during
    object validation we need to make sure that there
    is no unspecified aliasing
  • This can be enforced by using a hash-set for
    storing and propagating all the observed objects

89
Modular Verification of Web Services
  • We applied our modular verification approach
    based on interface grammars to both client and
    server side verification of Web services

90
Interface Grammars and Web Services
  • Our approach
  • A WSDL-to-interface grammar translator
    automatically generates grammar productions that
    generate and/or validate XML arguments and return
    values
  • User adds control flow constraints by modifying
    the grammar
  • Interface compiler automatically generates a stub
    for client side verification and a driver for
    server-side verification

91
Interface Grammars for Web Services
92
Another Case Study AWS-ECS
  • We tested the Amazon E-Commerce Web Service
    (AWS-ECS) using our approach
  • AWS-ECS WSDL specification lists 40 operations
  • that provide differing ways of searching Amazons
    product database
  • We focus on the core operations
  • ItemSearch, CartCreate, CartAdd, CartModify,
    CartGet, CartClear

93
Client-side Verification
  • For client verification we used a demonstration
    client provided by Amazon
  • This client does not check any constraints such
    as
  • You should not try to insert an item to a
    shopping cart before creating a shopping cart
  • When such requests are sent to AWS-ECS they would
    return an error message
  • Using our approach we can easily check if the
    client allows such erroneous requests
  • Falsification time changes with the type of
    faults we are looking for (data or control
    errors), changes from 10 to 60 seconds

94
AWS-ECS Server Verification
  • Our interface compiler automatically generates a
    driver that sends sequences of requests to
    AWS-ECS server and checks that the return values
    conform to the interface specification
  • The driver is a sentence generator
  • It generates sequences of SOAP requests based on
    the interface specification
  • We used two algorithms for sentence generation
  • A random sentence generation algorithm
  • Purdoms algorithm A directed sentence
    generation algorithm that guarantees production
    coverage

95
Directed Sentence Generation
  • Number of sentences generated 5
  • Average derivation length 24
  • Average number of SOAP requests/responses 3.8
  • Verification time 20 seconds

96
Random Sentence Algorithm
  • Number of sentences generated 100
  • Average derivation length 17.5
  • Average number of SOAP requests/responses 3.2

97
Server-side verification
  • We found two errors during server side
    verification
  • Errors were discovered within 10 seconds
  • These errors indicate a mismatch between the
    interface specification and the server
    implementation
  • It may mean that we misunderstood the description
    of the Web service interface
  • It may also mean that there is an error in the
    service implementation

98
Conclusions
  • Modular verification is a necessity
  • Interfaces are crucial for modular verification
  • Interface grammars provide a new specification
    mechanism for interfaces
  • Interface grammars can be used for automated stub
    and driver generation leading to modular
    verification

99
Conclusions
  • Behavioral interfaces can be useful both
  • From software design perspective by enabling
    better modular designs
  • From verification perspective by enabling more
    efficient modular verification
  • The challenge is to find behavioral interface
    specification mechanisms that serve both of these
    goals

100
Related Work Modular Verification
  • Clarke, Long, McMillan, Compositional Model
    Checking
  • Henzinger, Qadeer, Rajamani, Assume Guarantee
    Reasoning in Model Checking
  • Flanagan, Qadeer, Thread-Modular Model Checking
  • Krishnamurthi, Fisler, Modular Verification of
    Feature Oriented Programs

101
Related Work Design for Verification
  • Meyer, Design by Contract
  • Flanagan, Leino, et al. ESC Java
  • Mehlitz, Penix, Design for Verification Using
    Design Patterns
  • Sarna-Starosta, Stirewalt, Dillon, Design for
    Verification for Synchronization

102
Related Work Interfaces
  • L. de Alfaro and T. A. Henzinger. Interface
    automata.
  • O. Tkachuk, M. B. Dwyer, and C. Pasareanu.
    Automated environment generation for software
    model checking.
  • T. Ball and S. K. Rajamani. SLAM interface
    specification language.
  • G. T. Leavens et al. JML

103
Related Grammar-based Testing
  • A. G. Duncan, J. S. Hurchinson Using attributed
    grammars to test designs and implementations
  • P. M. Maurer Generating test data with enhanced
    context free grammars
  • P. M. Maurer The design and implementation of a
    grammar-based data generator
  • E. G. Sirer and B. N. Bershad Using production
    grammars in software testing
About PowerShow.com