Practical Approach to Specification and Testing of Distributed Network Applications

1
Practical Approach to Specification and Testing
of Distributed Network Applications

• Victor Kuliamin kuliamin_at_ispras.ru
• Nickolay Pakoulin
• Alexander Petrenko
• ISP RAS, Moscow

2
Outline

• Introduction
• Event Contracts Specification
• Application 1 Standards Clean-up
• Application 2 Conformance Testing

3
Internals and Externals of Availability

• Two approaches to ensure service availability
• InternalBased on guarantees of quality of
  components themselves
• Rigorously defines obligations of participants
• Tries to enforces obligation fulfillment
• FAILED
• Systems are too complex to comprehend or formally
  describe as a whole
• Formal methods work only in completely described
  cases
• Classic formal methods need high-educated and
  experienced staff
• ExternalBased on external infrastructure
  providing availability while components can fail
  or go down
• Uses additional infrastructure and mechanisms of
  service delivery
• Provides additional means of control
• Imposes external restrictions on components
  operation
• Enforces a set of standards

4
Factors of Success

• Standards enforcement
• Rigorous definitions of what is required need
  for strict requirements specifications
• Guarantees of interoperability consistency and
  unambiguity
• Conformance testing and certification
• Constraints on specification technique used
• Support of iterative and component-wise
  development
• Lightweight techniques able to get results
• Based on incomplete descriptions
• Without special requirements to staff
• Support of full-scale testingAutomated test
  construction is preferable

5
How to Describe Requirements?

• The description should be
• Sufficiently expressive
• As clear as possible
• Scalable to rather complex systems
• preferably, component-wise
• Suitable to distributed systems
• include several sides
• How do people describe mutual obligations in
  complex cases including several parties?
• By means of contracts!

6
Outline

• Introduction
• Event Contracts Specification
• Application 1 Standards Clean-up
• Application 2 Conformance Testing

7
Contract Specifications

• Pre- and postconditions (Hoare, 1969)
• means for reasoning about program behavior
• augment code elements to enforce rigorous
  development
• Design by Contract (Meyer, 1992)
• software is considered as a set of components
  interacting trough their interfaces
• pre- and postconditions are defined for
  interface operations
• constraints on data integrity are stated in
  invariants
• together they form software contract between
  a component and its environment

8
Design by Contract Pro and Contra

• Advantages
• Component-wise consideration of software
• support for reuse, incremental and parallel
  development
• Possibility to use for different aspects and on
  different abstraction levels
• Drawback
• Insufficient for distributed systems
• does not consider concurrency and
  asynchronous interaction
• does not consider callbacks

PU 2
PU 2
PL 1
PU 1
PU 1
9
Interaction in Distributed Systems
10
Event Contracts
Obligations of the environment
Obligations of the system
Precondition
says in what states such an event is possible
Pre-state
Input event
Pre-state
Output event
System
System
Environment
Environment
Post-state
Post-state
Postcondition
says what post-states can follow such an event in
such a pre-state
Obligations of the system
Obligations of the system
11
Concurrency

12
Implementation

• Software Contracts
• Pre- and postconditions of events, invariants
• Possibility to specify constraints in form of
  predicates on the results, not the algorithm
• Component-wise consideration of software
• Asynchronous events and callbacks included
• Specifications in extensions of widely-used
  programming languages (C, Java, C)
• Simplifications where possible
• Joint description of call and return if
  intermediate states do not matter

13
Example

• public specification class Barrier
• int awaitedThreads 0
• int waitingThreads 0
• invariant CountersAreNonnegative() return
  awaitedThreads gt 0 waitingThreads gt 0
• public specification void Init(int n)
• post
• if(n lt 0 waitingThreads gt 0)
  branch NoChanges
• return awaitedThreads pre
  awaitedThreads waitingThreads pre
  waitingThreads
• else
  branch NewHeightSet
• return awaitedThreads n
  waitingThreads 0
• public specification void Wait()
• post
• if(awaitedThreads lt 1)
  branch Immediate
• return awaitedThreads 0
  waitingThreads pre waitingThreads
• else deferred
  branch Waiting

14
The Proposed Approach
Testing goals
Formalization
Test Suites
Contract Specifications
Standards
Conformance testing Certification
Interoperability testing Early debugging Requireme
nts traceability
Inconsistencies, ambiguities, interoperability
flaws
Software
15
Outline

• Introduction
• Event Contracts Specification
• Application 1 Standards Clean-up
• Application 2 Conformance Testing

16
Case Studies I

• IPv6 2001
• Parts considered
• Sending datagrams / receiving packets
• Neighbor discovery
• Multicast Listener Discovery
• UPD over IPv6
• Results
• Minor defects found in RFC 2460
• Conformance test suite developed (further)
• IPMP-2 2004
• Results
• Several contradictions between standard parts
  found
• Interoperability flaws detected in Mutual
  Authentication protocol
• 2 accepted submissions on elaboration of the
  standard

17
Outline

• Introduction
• Event Contracts Specification
• Application 1 Standards Clean-up
• Application 2 Conformance Testing

18
Testing Fundamentals

• How to test?
• We act upon the system under test
• We watch its reaction
• We check whether that reaction is what should be
• We repeat this until all the reasonable
  situations are exhausted

19
Testing Goals

• post
• if ( f(a, b) g(a) )
• else if( h(a, c) !g(b) )
• else

20
The Testing Scheme
Testing Model
Behavior Model
System under Test
Coverage Model
On-the-fly Test Sequence Generation
Single Input Checking
21
UniTesK Test Construction Tools

• C / Visual Studio 6.0, gcc 2002
• Java / NetBeans 2002
• C / NetBeans MS Visual Studio
  2003specifications in Java extension
• Specialized tool for compiler testing 2003and
  complex data generation
• C / Visual Studio .NET 7.1 2003
• Java / Eclipse 2005

22
Tool Demonstration
23
Case Studies

• ISP RAS Nortel Networks 1994-1997functional
  test suite development for Switch Operating
  System kernel
• IPv6 implementations 2001-2003
• Microsoft Research
• Mobile IPv6 (in Windows CE 4.1)
• Oktet
• Intel compiler optimization units 2001-2003
• IPSec 2004-
• Pilot projects
• Enterprise application development
  framework 2003
• Components of TinyOS 2003
• Web-based banking client management system
  (Luxoft) 2004
• Components of billing system (Vympelkom) 2005
• http//www.unitesk.com

24
References

• V. Kuliamin, A. Petrenko, I. Bourdonov, and A.
  Kossatchev. UniTesK Test Suite Architecture.
  Proc. of FME 2002. LNCS 2391, pp. 77-88,
  Springer-Verlag, 2002.
• V. Kuliamin, A. Petrenko, N. Pakoulin, I.
  Bourdonov, and A. Kossatchev. Integration of
  Functional and Timed Testing of Real-time and
  Concurrent Systems. Proc. of PSI 2003. LNCS 2890,
  pp. 450-461, Springer-Verlag, 2003.
• V. Kuliamin, A. Petrenko. Applying Model Based
  Testing in Different Contexts. Proceedings of
  seminar on Perspectives of Model Based Testing,
  Dagstuhl, Germany, September 2004.
• A. Kossatchev, A. Petrenko, S. Zelenov, S.
  Zelenova. Using Model-Based Approach for
  Automated Testing of Optimizing Compilers. Proc.
  Intl. Workshop on Program Undestanding,
  Gorno-Altaisk, 2003.
• V. Kuliamin, A. Petrenko, A. Kossatchev, and I.
  Burdonov. The UniTesK Approach to Designing Test
  Suites. Programming and Computer Software, Vol.
  29, No. 6 , 2003, pp. 310-322. (Translation from
  Russian)
• S. Zelenov, S. Zelenova, A. Kossatchev, A.
  Petrenko. Test Generation for Compilers and Other
  Formal Text Processors. Programming and Computer
  Software, Vol. 29, No. 2 , 2003, pp. 104-111.
  (Translation from Russian)
• V. Kuliamin, N. Pakoulin, A. Petrenko. Extended
  Design-by-Contract Approach to Specification and
  Conformance Testing of Distributed Software.
  Proc. of 9-th World Multi-Conference on
  Systemics, Cybernetics, and Informatics, Model
  Based Testing Session, July 2005, to be published.

25
Contacts

• RedVerst group web site http//www.ispras.ru/grou
  ps/rv/rv.html
• UniTesK projects web site http//www.unitesk.com
• Group leaderAlexander Petrenkopetrenko_at_ispras.ru
  

26
Thank You!