An Automated Test Strategy Based on UML Diagrams

1
An Automated Test Strategy Based on UML Diagrams

• F. Basanieri , A. Bertolino , E. Marchetti ,
  A. Ribolini , G. Lombardi , G. Nucera
   PISATEL LAB, IEI-CNR, Pisa, Italy 
  ERICSSON LAB ITALY

2
Agenda

• Cow_Suite tool
• New features
• Traceability
• Cow_Suite vs RUP
• Guidelines
• Discussion and future work

3
Objectives

• Use for test planning the same UML diagrams
  developed (e.g., by Rational Rose) for design and
  analysis
• Tool-supported generation and maintenance of an
  updated and strategic test plan, as the design
  evolves.

4
Cow_Suite approach

• The processing of the Cow_Suite approach starts
  by analysing the project description, and
  proceeds in parallel with the design and analysis
  refinement steps.
• The reference documentation is derived by
  collecting together and automatically organizing
  the diagrams developed during the design phase
  without additional manual effort of formalization
• the internal description of the project
  model, given by Rational Rose in a .mdl file

5
Cow_Suite Scheme
Test Procedures list
6
Tree Derivation

• Use Cases (UCs) and Sequence Diagrams (SDs) of
  the .mdl project design realized with Rational
  Rose are automatically organized by Cow_Suite
  tool in a sort of hierarchical (pseudo-)tree

The case study is Course Registration System from
RUP documentation
7
Tree Derivation

• The users can get a complete view of the status
  of the functionalities specification and a
  detailed documentation
• UCs and their realization
• SDs associated to each specification level
• The links between Use Case and Design Model
• Reused nodes
• Not linked model elements

8
Use Case View vs Logical View

• The Main Tree can be analyzed from two different
  points of view which correspond to two different
  typologies of test.
• Using the information of the Use Case View
• System test
• Subsystem test
• Using the information of the Logical View
• Subsystem test
• Component test

9
Use Case View

• describes the set of UCs (and scenarios) that
  represents the significant functionalities and
  the interaction of the system with the external
  world.

• The Main Tree derived represents a detailed
  documentation of how features/system requirement
  are realized in the Use Case View.
• The path from a high level Use Case to a leaf
  (UCs or SDs) describes the refinement of the
  associated system feature
• The Test Cases derived from the SDs of the Use
  Case View are specifically designed for
• System integration Test
• Subsystem integration Test

10
Logical View

• Describes how the system functionalities are
  provided in term of static structure and dynamic
  collaborations of the objects.

• Cow_Suite tool analyses the Logical View
  information about
• Use Cases realization
• Sequence diagrams, classes and their organization
  in packages and subsystems
• The tool automatically derives
• The UC traceability by analyzing the package Use
  Case Realizations.
• The Design Tree derived by analyzing the Design
  Model packages

11
Use Case Diagram Traceabilities

• The Use Case Realizations package links the
  information of the Use Case View and the Logical
  View .
• The Use Case Diagram Traceabilities traces the
  evolution of the Use Cases in the Design Model

• Each Traceability Use Case
• Represents the evolution of a UC of the Use Case
  View into the Design Model
• Details the sub-flow of a UC with Sequence
  (Collaboration) Diagrams
• Specifies the Classes and the relationship that
  participate in the realization of the UC (Class
  Diagrams)
• Optionally holds the package Design Link

12
Use Case Realization analysis

• The Main Tree derived from the Use Case View is
  automatically extended with the information of
  the Use Case Realizations Package
• The path from a high level Use Case, through its
  Use Case Realization, to a leaf documents and
  traces the implementation of a system feature
• Considering the subtree rooted in a Traceability
  Use Case, its SDs
• contain low level information with respect to
  those of the Use Case View
• describe in a detailed manner the collaboration
  of the involved objects
• The Test Cases derived from these SDs are
  specifically designed for
• Subsystem integration Test
• Component integration Test

13
Design Link Package

• Design Link package for every use case
  realization it collects the package/s that
  represent the implementation of the relative UC.

It is an explicit link between the Main Tree and
the Design tree
14
Design Tree
Design Tree is derived directly from the Design
Model and represents the organization of the
packages in a hierarchy.

• Analyzing this tree it is possible to
• Observe the dependence relationship between
  packages
• Know which are the reused packages and where the
  reuse happens
• Connect to the main tree the low level SDs not
  explicitly linked in the .mdl file

By selecting the SDs of the Design Tree and
applying to them the Cow_Suite test derivation
strategies, a list of low level Test Cases can be
derived
15
Deduce the Critical Profile

• The effort devoted to the test of different
  functionalities must be planned and scheduled
  considering their relative importance
  (risk-based testing, reliability guided testing,
  etc).
• The importance, represented by number (weight)
  belonging to 0,1 interval, can be assigned in
  two different manner
• Interacting with Cow_Suite Tool Following level
  by level the tree structure of the UCs and SDs,
  the user annotates the corresponding node in the
  tree with its weights
• Directly in the .mdl file (UCs only) During the
  development of the Use Case Model
  architects/designers assign a value, belonging to
  1,9 interval, to each Use Case.
• The value is memorized in the use case
  documentation tab with the following structure
  WGH value
• (For SDs or when no value is expressed for UCs
  the default value is 5)

16
Weighted tree derivation

• The tool
• reads the assigned values the .mdl file (if they
  exist)
• sums the values of the node at the same level in
  the tree
• normalizes the weights among the nodes at the
  same level so that their sum is 1.

The user can always modify the assigned weights
17
Integration Stage selection

• The Cow_Suite user selects the integration stage
  at which the test is conducted.
• The nodes that are considered by Cow_Suite tool
  for the subsequent evaluations are all and only
  those belonging to the selected integration stage
  plus all the tree leaves existing at higher
  levels.

18
Cow_Test Strategy

• Two different situations are considered in test
  cases planning
• 1 Cowtest_ing with fixed number of tests
• A certain budget is available, which is
  translated in terms of number of tests, NT, to
  execute.
• In this case the tool select the most adequate
  selection of NT tests from among the (many)
  various test cases that could be potentially
  conceived.

19
Cow_Test Strategy

• 2 Cowtest_ing with fixed functional coverage
• A certain percentage of functionalities must be
  covered.
• In this case by the tool is possible to define in
  advance which are the functionalities to be
  covered and the minimum number of tests to
  execute.

Functional Coverage 90
20
Select the Test Strategy
21
Sequence Diagrams Selection

• For each fixed integration stage, a weighted
  subtree is derived according to the chosen test
  criterion. Only the relative SDs are considered
  for Test Case generation.

22
Test Case Derivation

• The tool uses the UIT method, to derive the Test
  Cases.
• The method is mainly based on the SDs
  (particularly objects, messages and parameters)

23
Sequence Diagram analysis

• Test Units horizontal axis analysis.
• Interactions Categories and Settings Categories
  vertical axis analysis.
• MessageSequences set of Messages (in temporal
  order), involved in a SD, used by objects to
  define and elaborate specific functionalities.

24
Test Case Derivation

• For each traceable Message Sequence a Test Case
  is generated.
• It contains the list of all its Settings and
  Interactions Categories and their values.
• The final weight of every SDs is used to
  automatically derive the number of Test
  Procedures associated to each Test Case.

25
Test Procedures Specification and derivation

• Test Specification
• For each identified category, we consider all its
  possible values and constraints (choices).

A Test Procedure is automatically generated from
a Test Specification, for each possible
combination of choices of every category involved
in a Messages Sequence.
26
Cow_Suite vs RUP

• Guidelines for using Cow_Suite inside RUP
• Best Practices for Cow_Suite application

• RUP is based on an iterative process
  characterized by four phases divided in
  workflows.
• A workflow is a set of activities that produces a
  observable result (artifacts). The involved
  workflows are
• Project Management
• Requirement
• Analysis and Design
• Test

27
Inception Phase

• During the inception phase
• The scope of the project and the external
  entities with which the system will interact
  (actors) are defined.
• The use cases describing the most significant
  interaction are also identified

• Workflow Requirement
• Analyse the Problem
• Understand Stakeholder Needs
• Define the System
• Manage the scope of the System
• Refine the System Definition
• Manage Changing Requirements
• Classify Requirements

• Workflow Analysis and Design
• Perform Architectural Synthesis
• Evaluate Use Case
• Derive Cow_tree
• Workflow Test
• Test Plan
• Testing Cost Estimation

28
Elaboration Phase

• During the inception phase
• The analysis of the problem domain and the basic
  architecture are established
• The project plan is developed and the possible
  risk elements are evaluated
• A prototype is built, and the use case model is
  almost complete

• Workflow Project Management
• Plan for Next Iteration
• Manage Iteration
• Monitor ad Control Project
• Analyse Cow_tree
• Reevaluate Project Scope and Risk
• Workflow Requirement
• Analyse the Problem
• Understand Stakeholder Needs
• Define the System
• Manage the scope of the System
• Refine the System Definition
• Manage Changing Requirements
• Classify Requirements
• Workflow Analysis and Design
• Define a Candidate Architecture

• Process UCs and SDs
• Derive Cow_tree
• Analyse Behaviour
• Implement Traceability
• Evaluate Traceability
• Design component/design for Real-Time
• Cow design tree
• Design database
• Workflow Test
• Testing Cost Estimation
• Plan test
• Derive UIT Test
• Test Traceability
• Design test
• Implement test
• Execute Test in Integration Test Stage
• Execute Test in System Test Stage
• Evaluate Test

29
Construction Phase

• Includes the development of the system remaining
  components and features and the test of the
  entire system.

• Workflow Project Management
• Plan for Next Iteration
• Manage Iteration
• Monitor ad Control Project
• Analyse Cow_tree
• Reevaluate Project Scope and Risk
• Workflow Requirement
• Manage Changing Requirements
• Classify Requirements
• Workflow Analysis and Design
• Define a Candidate Architecture
• Process UCs and SDs
• Derive Cow_tree

• Analyse Behaviour
• Implement Traceability
• Evaluate Traceability
• Design component/design for Real-Time
• Cow design tree
• Design database
• Workflow Test
• Testing Cost Estimation
• Plan test
• Derive UIT Test
• Test Traceability
• Design test
• Implement test
• Execute Test in Integration Test Stage
• Execute Test in System Test Stage
• Evaluate Test

30
Inception Phase Workflow Requirement Classify
Requirements

• The requirements must be classified and grouped
  depending on
• criticality
• system importance
• difficulty in the development
• Optional a number in 1,9 interval
  representing this importance value should be
  assigned to each requirement/requirement set.

31
Inception Phase Workflow Analysis and Design
Evaluate Use Case

• Basing on the requirement classification an
  importance value ( 1,9 interval) must be
  assigned to each developed UC.
• The importance is an absolute value
  characterizing the UC
• The value is memorized in the use case
  documentation tab with the following structure
• WGH value

32
Inception Phase Workflow Analysis and Design
Derive Cow_Tree

• Apply the Cow_Suite and derive the main tree.
• Possible design weaknesses and incompleteness are
  displayed.
• The user profile is derived
• The most important UCs, i.e. system
  functionalities, are highlighted

33
Inception Phase Workflow Test Testing Cost
Estimation

• Applying Cow_Suite test strategies it is possible
  to
• Distribute the Test Cases among the system
  functionalities
• Know how many Test Cases are necessary to cover
  the developed system functionalities
• Derive the completion time or cost (in terms of
  man hours/required budget) of the Test Phase by
  assigning a value to each typology of Test Case

34
Elaboration Phase Workflow Project Management
Analyse Cow_tree

• Apply the Cow_Suite and derive the main tree
  that
• Evidences the degree of detail reached in the
  requirement development.
• Highlights the design weaknesses in the Use Case
  Model/Design Model developed.
• The main tree integrates the information derived
  from the Use Case View with those of the Logical
  view and can be used for the completion of the Sw
  Architecture Document.
• Marks the most important functionalities of the
  developed system
• The final weight associated to each node
  represent the importance of the implemented
  functionalities for the system.
• The correctness of the values assigned must be
  checked and eventually modified

35
Elaboration Phase Workflow Requirement
Classifying Requirements

• Software architects and Program managers
• highlight the most critical UCs
• select the UCs that will be refined
• They assign or modify the importance value
  associated to the UCs during the Inception Phase.
  

36
Elaboration Phase Workflow Analysis and Design
Process UCs and SDs

• Basing on the requirement classification an
  importance value ( 1,9 interval) must be
  assigned to each developed UC.
• The value is memorized in the use case
  documentation tab with the following structure
• WGH value
• Optional Insert the default values in the SDs.
  They represent the values of the Interaction
  Categories of the UIT method.
• The values are memorized in the message/class
  method documentation tab with the following
  structure
• DEF (value1, value2)

37
Elaboration Phase Workflow Analysis and Design
Derive Cow_tree

• Apply the Cow_Suite and derive the Main Tree.
• An extension/modification of the main tree
  corresponds to every refinement of the Use
  Case/Design Model.
• The evolution of the system requirement (Use case
  View), into diagrams of the Logical View is
  highlighted
• The missing links between the two views and the
  diagram not properly connected are shown and
  evidenced.

38
Elaboration Phase Workflow Analysis and Design
Implement Traceability

• Associate to every Traceability Use Case a value
  representing the criticality in the realization
  of the associate UC, and annotate in the UC
  documentation tab this number.
• Link to every Traceability Use Case the SDs
  representing the most important scenarios and the
  interaction between the involved classes. In this
  manner every requirement will be automatically
  associated to the proper set of Test Cases
• Optional create a class diagram, called the
  Design Link, with the list of the packages in the
  Design Model that represent the implementation of
  the UC

39
Elaboration Phase Workflow Analysis and Design
Evaluate Traceability

• Apply the Cow_Suite and derive the Main tree and
  the Design Tree
• The Design Link evidences the evolution of the
  UCs in the Design model
• The Design Tree represents the degree of detail
  reached in the requirements (UCs) implementation
• The objects or packages involved in the
  requirement implementation are organized in a
  hierarchical tree using packages dependencies.
• The weights associated to the Traceability Use
  Cases are analyzed and possibly modified

40
Elaboration Phase Workflow Analysis and Design
Cow Design Tree

• The Design Tree
• shows the hierarchy between the different system
  components, the subcomponents, the reused
  components and the associated SDs
• Represents a detailed documentation which
  associates to each requirement the relative
  system components.
• following the path in the main Tree from an high
  UC, throw Use Case realization, to the Design
  Link node. Then automatically jump to the proper
  position in the Design tree and cover it down
  till the leaves.

41
Elaboration Phase Workflow Test Testing Cost
Estimation

• Considering the main tree
• Evaluate the correctness of the assigned weights
• Possibly modify the values
• Applying Cow_Suite test strategies it is possible
  to
• Distribute the Test Cases among the system
  functionalities
• Know how many test are necessary to cover the
  developed system functionalities
• Derive the completion time or cost (in terms of
  man hours/required budget) of the Test Phase by
  assigning a value to each typology of Test Case

42
Elaboration Phase Workflow Test Derive UIT
Test

• Apply the UIT method.
• The Test Cases derived from the main tree can be
  used for system or subsystem or component test
  depending on the integration level chosen.
• The Test Cases derived from the Design tree can
  be used for component/unit test.
• The Test Procedures, as derived by the tool, are
  not directly executable (? Test Scripts). The
  derived Test Procedures have to be executed and
  logged by a test driver.
• The degree of detail of the Test Set depends
  directly on, the design detail level and
  granularity of the inserted values (choices of
  the UIT categories).

43
Elaboration Phase Workflow Test Test
Traceability

• Considering the main tree and the Design tree
• the SDs on the path from an high level UC to a
  leaf (of the main tree or Design tree) produces
  Test Cases specialized for the testing and the
  verification of the requirement (system
  functionalities) associated to the UC.
• The degree of detail of the derived Test Cases
  depend on hierarchical position of the SDs in the
  tree (the higher is the level, the lower is the
  degree of detail)

44
Discussion and Future Work
Systematic and automated support for test
planning and test case generation, based on the
same UML diagrams developed for design and
analysis
OK
Tool integrated with the design tool (e.g.
Rational Rose)
OK
Test case derivation (UIT) to be validated on ERI
case studies
To be continued
FUTURE OF COW_SUITE ?!?
45
Questions?