Verification and Validation Process Definition and Tool Development Within Euclid RTP 11'13

1
Verification and Validation - Process Definition
and Tool Development Within Euclid RTP 11.13
Andreas Nyberg SaabTech, Sweden Richard
Harris TTSL, UK Wim Huiskamp TNO-FEL, The
Netherlands João Pereira INESC-ID, Portugal

• 03E-SIW-063, 2003 Euro-SIW

2
Overview

• Background Euclid/SEDEP
• V V process
• V V toolset design
• V V toolset implementation/example
• Conclusions

3
SEDEP Steps and Products
4
Overview of Feedback loops
5
Verification and Validation Definitions

• The following Two definitions are used in the
  Euclid 11.13
• Programme for Verification and Validation
• - A product is considered Verified if it meets
  its System Requirements. The Process or Means
  leading to such a State is called Verification
• - A product is considered Validated if it meets
  its User Requirements. The Process or Means
  leading to such a State is called Validation

6
Observations

• The closer the mapping from User Requirements to
  System Requirements, the less work is needed to
  go from a verified System to a validated one.
• If there is a One-to-One map between the User
  Requirements and the System Requirements, the
  Validation of the System will be trivial once
  verified

7
The Euclid 11.13 Verification Validation Process

• Conformance Testing A single Federate
  communicates using correct HLA syntax
• Application Testing A single Federate
  communicates using correct Federate Semantics
• Integration Testing Two or more Federates
  Communicates correctly within a Federation
• Functional Testing All Federates in a
  Federation interact as required by the Federation
• Scenario Testing All Federates in Federation
  interact as required by specific exercises

8
Verification Validation Test Suites
9
Tools, General

• Tools needed to support following
• FUT Stimulation
• HLA traffic analysis
• Test Session Management
• COTS and Standards used as much as possible
• Euclid tools where no suitable COTS could be
  found
• Tools to interact with Repository where
  applicable
• Only Tool prototypes implemented by Euclid
• However Requirements Analysis and Design done for
  Full version Tools as well.

10
Federate Test Suite (FTS) Overview
11
Tools selected for Euclid
Tool Role Source

• Test Navigation Tool Test session
  manager Euclid
• (TNT) Directs operator through
• the series of tests
• Test Federate (TF) Stimulator DMSO
• Stimulates the FUT with RTI
• service calls
• Federation Verification Analyser RTI
  communication DMSO
• Tool (FVT) Analyses objects, interactions,
• RTI services
• Verification Analysis Analyser FVT
  output Euclid
• Tool (VAT) Consolidates FVT results
• OMDT Analyser SOMs FOM DMSO
• FEPW Editor Analyser FEPW DMSO
• Optional Tools Additional Monitoring Various
• eg Stealth, 2D Map, Logger

12
Test Navigation Tool (TNT)

• Test Session Manager
• Assists Operator in composing, managing and
  tracking the tests
• Loads requirements for allowing tests to be
  created and suitable mapped for each requirement
• Allows for the reuse of previously created tests
• Leads the operator through the tests, step by
  step
• Allows for both automatic and manual testing

13
TNT Screenshot
14
FVT and TF Overview

• Federation Verification Tool (FVT)
• Analyses the traffic recorded during the testing
• FVT is itself a Federate
• Provides two output files, Summary File and
  Discrepancy File
• Test Federate (TF)
• Stimulates the FUT. By pre-defined Test Scripts
  or by individual RTI services specified by
  Operator
• TF is itself a Federate
• Can prepare and store Test Scripts

15
FVT Screenshot
16
Verification Analysis Tool (VAT)

• Simplifies the collection of Test Results
• Operates after the test execution itself.
• Reads and Analyses the two FVT files
• Determines whether the FUT has passed the tests
• Graphically displays report to the Operator
• Writes result to an XML file

17
VAT Screenshot
18
FTS Architecture
19
Conclusions Lessons Learned

• VV has received alot of attention by Euclid
  11.13
• VV is an iterative process and an integral part
  of the SEDEP
• Focus on reducing work load for Developers and
  Test Engineers
• Tools Suite contains both COTS and in-house
  Components
• Final Demonstration, November 2003