Component Testing

1
Component Testing

• Ednaldo Dilorenzo de Souza Filho
• ednaldo.filho_at_cesar.org.br

2
Summary

• Introduction
• Fundaments of Testing
• Software Reuse
• Component Testing Motivation
• Test-Driven Development
• Third-Party Testing
• Regression Testing based on Version Changes of
  Components
• Testability Analysis for Software Components
• White on Black

3
Introduction

• Testing is an important process to support
  assurance
• As software becomes more pervasive and is used
  more often to perform critical tasks, it will be
  required to be of higher quality
• Because testing requires the execution of the
  software, it is often called dynamic analysis
  Harrold
• Testing consists in compare the outputs from the
  executions with expected results to identify
  those test cases on which the software failed
• Testing cannot show the absence of faults, if can
  show only their presence.

4
Fundaments of Testing

• Testing goals
• Find faults in a software program
• A good test case have a high probability to find
  errors
• A good test process should find errors
• Testing fundamentals
• Testing should be connected with client
  requirements
• Testing should be planned before testing
  execution
• Isolation of suspected components
• Testing should start in individual components
• Complete testing is impossible
• Testing shouldnt be executed for developers

5
Software Reuse

• Increased reliability Reuse is supposed to have
  a positive effect on the quality of the software
  entity reused
• Reduce Process Risk Software reuse can reduce
  risks inherent in software projects
• Effective use of specialists Software reuse can
  also contribute to an effective use of
  specialists
• Standards compliance Improve its standards
  compliance, which in turn can have other
  benefits
• Accelerated development Software reuse can
  accelerate development

Beydeda
6
Component Testing Motivation

• The Ariane 5 Lesson Weyuker
• In June 1996, during the maiden voyage of the
  Ariane 5 launch vehicle, the launcher veered off
  course and exploded less than one minute after
  take-off
• The explosion resulted from insufficiently tested
  software reused from Ariane 4 launcher
• Context-Dependent Testing of a Component
  Beydeda
• Insufficient Documentation of a Component
• Component Users Dependence on the Component
  Provider

7
Component Testing Motivation

• Problems with systems built from reusable
  components
• Performance problems
• Fitting selected components together
• Absence of low-level understanding
• Developing a component for a particular project
  with no expectation of reuse, testing proceed as
  usual
• Significant additional testing should be done for
  changing priority
• Modify or not the source code?
• Facilitating Reusable Component Testing
• Associate a person or a team for maintaining each
  component
• Add software specification
• Add test suite
• Add pointers between the specification and parts
  of the test suite

8
Test-Driven Development Janzen

• Requires writing automated tests prior to
  developing functional code
• Used in agile methods
• To promote testing to an analysis and design step
  the practice of refactoring must be introduced
• TDD in industry
• 18 percent to 50 percent more external test cases
  than code produced by corresponding control
  groups
• Less time debugging code
• TDD in academia
• Early detection of defects
• Improvement in software quality and programmer
  productivity

9
Third-Party Testing and the Quality of Software
Components Councill

• 10 contribuitors devoted 70 person-hour to
  developing a definition of component
• Ariane Project
• Certification business introduced a Maturity
  Model focused on large organizations
• 99 percent of all US businesses are small
  businesses
• Producers provide the developing and testing
  procedures, purchasers or their agents conduct
  second-party testing and independent testing
  organizations perform third-party testing

10
Third-Party Testing and Stirrings of the New
Software Engineering Councill2

• Licensed software engineers must manage
• Intense requirements elicitation
• Comprehensive requirements specifications that
  address all states end endeavor to assure
  fail-safe and fault-tolerant operations
• Designs that can be tested on paper
• The anticipation of third-party testing
• Regulatory software component certification
• Software engineer licensing
• Mandated standards for third-party testing of
  software components

11
Regression Test Selection Based on Version
Changes of Components Sajeev

• Regression test strategy test components tested
  before to assure new components didnt affected
  it
• Most regression testing techniques are based on
  white box strategies by analyzing code
• Use of UML (Unified Modeling Language) and its
  associated with OCL (Object Constraint Language)
  to formally specify the test selection process

12
Regression Test Selection Based on Version
Changes of Components Sajeev

• Regression Test Selection
• Let P be a program, and P be a new version of
  program created by replacing some of the
  components in P with new versions
• Let T be a set of test cases that is already run
  on P
• The aim of regression testing is to show that P
  satisfies T
• Base case
• T T
• Optimal case
• T ? T

13
Regression Test Selection Based on Version
Changes of Components Sajeev
14
Regression Test Selection Based on Version
Changes of Components Sajeev

• Test Selection Strategy
• Step 1 Select all test sequences which include a
  direct call to a modified method
• Step 2 Select all test sequences which include a
  call to any method which in turn uses, directly
  or indirectly, a modified method of a revised
  component
• Union of two steps bring the test sequences for
  regression testing

15
Testability Analysis For Software Components
Nguyen

• Testability is a quality factor, which can be
  predicted as soon as the system is specified, to
  indicate the ease (or difficulty) for testing the
  system
• A component is said to be domain-observable, if
  distinct outputs are produced from distinct
  inputs
• A component is said to be a domain-controllable,
  if its output domain is covered from its input
  domain
• The testability of a system is based on
  controllability and observability of the
  components in the system
• Controllability is defined as the ease to
  generate the inputs of a component from the
  inputs of the system
• Observability is defined as the ease to propagate
  the outputs of a component to the final outputs
  of the system

16
Testability Analysis For Software Components
Nguyen

• Testability Model

17
Testability Analysis For Software Components
Nguyen

• Testability measures
• Information channel
• Coefficient of information loss for a module
• Module capacity
• Edge capacity
• Information transfer
• Testability measures

18
Testability Analysis For Software Components
Nguyen

• Static Single Assignment Form

19
Testability Analysis For Software Components
Nguyen

• Case Study

20
Testability Analysis For Software Components
Nguyen

• Case Study

Observability and Controlability values
21
White on Black A White-Box-Oriented Approach for
Selecting Black-Box-Generated Test Cases Chen

• It would be ideal to test a program with its
  entire input domain.
• A more practical work is to construct a test
  suite for testing.
• A test suite constructed should be as
  comprehensive as possible, and it should be as
  small as possible.
• Approach for reduce the test case based on
  specification (Black box) using white box
  information.
• In the black box approach, the category-partition
  method (CPM) and the classification-tree method
  are particularly useful.

22
White on Black A White-Box-Oriented Approach for
Selecting Black-Box-Generated Test Cases Chen

• Background
• In partition testing approach input domain of a
  program is divided into subsets, called
  subdomains
• The tester identifies relevant aspects of the
  specification for testing called classification
• The subdomains associated with the classification
  are called classes
• Test cases are formed and every incompatible
  classes identified said to be illegitimate
• Only legitimate classes are considered reducing
  the number of test cases in the test suite

23
White on Black A White-Box-Oriented Approach for
Selecting Black-Box-Generated Test Cases Chen

• Background

24
White on Black A White-Box-Oriented Approach for
Selecting Black-Box-Generated Test Cases Chen

• Information of the paths executed by the test
  cases are used for helping test case reduction
• When two test cases are identical differing by
  only one class they are called a matching pair
• Classes differing from a matching pair are called
  differentiating classes, or a differentiating
  class pair

Matching Pair
Differentiating Classes
25
White on Black A White-Box-Oriented Approach for
Selecting Black-Box-Generated Test Cases Chen

• Based on the white box analysis we can verify
  which test cases can be omitted

Here tc4 or tc5 can be ommited
26
White on Black A White-Box-Oriented Approach for
Selecting Black-Box-Generated Test Cases Chen
27
References

• Beydeda Sami Beydeda and Volker Gruhn, Testing
  Commercial-of-the-Shelf Components and Systems,
  Springer, 2005.
• Councill William T. Councill, Third-Part
  Testing and the Quality of Software Components,
  IEEE Software, Quality Time, July 1999.
• Councill2 William T. Councill, Third-Part
  Testing and Stirrings of the New Software
  Engineering, IEEE Software, Quality Time, July
  1999.
• Chen T.Y. Chen, P.L. Poon, S.F. Tang and Y.T.
  Yu, White on Black A White-Box-Oriented Approach
  for Selecting Black-Box-Generated Test Cases,
  IEEE, 2000.
• Janzen David Janzen and Hossein Saiedian,
  Test-Driven Development Concepts, Taxonomy, and
  Future Direction, IEEE Computer Society, Cover
  Feature, Semtember 2005
• Sajeev A.S.M. Sajeev and Bugi Wibowo,
  Regression Test Selection Based on Version
  Changes of Components, IEEE, Proceedings of the
  Tenth Asia-Pacific Software Engineering
  Conference, 2003

28
References

• Nguyen T.B. Nguyen, M. Delaunay and C. Robach,
  Testability Analysis For Software Components,
  IEEE, Proceedings of the International Conference
  on Software Maintnance, 2002.
• Harrold M. J. Harrold, Testing a Roadmap,
  College of Computing, Georgia Institute
  Technology, 2000.