Integravimo testavimas

1
Integravimo testavimas

• Testavimui išbaigtos sistemos ar posistemes,
  sudarytos iš integruotu komponentu.
• Integravimo testavimas turetu buti juodos dežes
  su testais, gautais iš specifikaciju.
• Pagrindinis sunkumas yra klaidu lokalizavimas.
• Palaipsninis integravimo testavimas mažina šia
  problema.

2
Palaipsninis integravimo testavimas
3
Integravimo testavimo metodai

• smulkinantis (Top-down ) testavimas
• pradeti nuo sistemos aukšciausio lygio ir
  integruoti nuo viršaus žemyn, kur reikia,
  pakeiciant individualius komponentus kamšciais
  (stubs) - tušciomis proceduromis
• stambinantis (Bottom-up) testavimas
• integruoti individualius komponentus i lygius
  iki tol, kol sukuriama visiškai sukomplektuota
  sistema.
• Praktiškai integravimas itraukia šiu strategiju
  kombinacijas.

4
Smulkinantis testavimas





5
Stambinantis testavimas






6
Testavimo metodai

• Architekturinis atestavimas smulkinantis
  integravimo testavimas yra geresnis atrandant
  klaidas sistemos architekturoje.
• Sistemos demonstravimas smulkinantis integravimo
  testavimas leidžia ribota demonstravima pradineje
  kurimo stadijoje.
• Testo realizavimas dažniausiai yra lengviau
  realizuoti stambinanti integravimo testavima.
• Testo stebejimas problematiškas yra abiem
  metodais. Kad stebeti testus, gali buti
  reikalaujamas papildomas kodas.

7
Sasajos testavimas

• Naudojamas, kai moduliai ar posistemes jungiami,
  kad sukurti dideles sistemas.
• Tikslas yra aptikti klaidas del sasajos defektu
  arba del neteisingu prielaidu apie sasaja.
• Ypatingai svarbus objektiškai-orientuotam
  kurimui, kai objektai yra apibrežiami ju paciu
  sasaju.

8
Sasajos testavimas
9
Sasaju tipai

• Parametru sasajos
• duomenys perduodami iš vienos proceduros i kita.
• Bendrai naudojamos atminties sasajos
• atminties bloku bendrai naudojamasi.
• Procedurines sasajos
• posisteme inkapsuliuoja proceduru rinkini, kuris
  kvieciamas kitos posistemes.
• Žinuciu perdavimo sasajos
• posistemes reikalauja paslaugu iš kitu posistemiu.

10
Sasajos klaidos

• Klaidingai naudojama sasaja
• pakviestas komponentas iškviecia kita komponenta
  ir daro klaidas jo naudojamoje sasajoje, pvz.
  parametrai neteisinga tvarka.
• Sasajos neteisingas interpretavimas
• kvieciantis komponentas remiasi prielaida apie
  kiekvieno komponento elgesi, kuris yra
  neteisingas.
• Sinchronizacijos klaidos
• pakviestas ir kvieciantysis komponentai dirba
  skirtingais greiciais ir nebegaliojanti
  informacija vis dar buna prieinama.

11
Nuorodos sasajos testavimui

• Suprojektuokite testus taip, kad parametrai
  iškviestai procedurai turetu ribines reikšmes.
• Visada testuokite nuorodos parametrus su nuline
  rodykle.
• Sukurkite testus, kurie priverstu komponenta
  suklysti.
• Naudokite stresini testavima (stress) žinuciu
  perdavimo sistemoje.
• Bendrai naudojamos atminties sistemose keiskite
  tvarka, kuria komponentai yra aktyvuojami.

12
Stresinis testavimas

• Testuoja sistema virš jos maksimaliai
  suprojektuoto kruvio. Sistemos spaudimas dažnai
  lemia defektu pasirodyma.
• Sistema spaudžiama, kad klaidingai veiktu. Ji
  neturetu katastrofiškai suklysti. Stresinis
  testavimas tikrina nepriimtina duomenu ar
  paslaugu praradima.
• Ypatingai tinka paskirstytoms sistemoms, kurios
  gali eksponuoti žymu nuosmuki, kai tinklas
  perkrautas.

13
Drivers Stubs

• Driver An empty function that calls functio
  contains just enough code to set up parameters
  and globals prior calling.
• Stub An empty function that has the same
  interface as a module, and does minimal
  manipulation.

14
(No Transcript)
15
(No Transcript)
16
Any other Stubs?

• Test (validate) the input data
• Print message I am here with parameters
• Get return values from interactive input
• Return a standard answer regardless of input
• Burn up clock cycles
• Read data from file and return
• Use a table of input/output map to return values

17

• Principle
• Individual program units may work in isolation
• But may not work correctly when integrated
• Localization of defects
• Defect may be manifested not at source, but in a
  different program unit

18
(No Transcript)
19
Integration Testing
20
(No Transcript)
21
(No Transcript)
22
Example 1 Three Layer Call Hierarchy
23
Big-Bang Approach(fine for small software
systems)
24
Bottom-up Testing Strategy

• The subsystem in the lowest layer of the call
  hierarchy are tested individually
• Then the next subsystems are tested that call the
  previously tested subsystems
• This is done repeatedly until all subsystems are
  included in the testing
• Because we have no components ready to call the
  lowest-level programs, we need to write special
  code (called driver) to aid integration.
• A component driver is a routine that calls a
  particular component and passes a test case to
  it.

25
Bottom-up Integration

26
Top-down Testing Strategy

• Test the top layer or the controlling subsystem
  first
• Then combine all the subsystems that are called
  by the tested subsystems and test the resulting
  collection of subsystems
• Do this until all subsystems are incorporated
  into the test
• A component being tested may call another that is
  not yet tested, so we write a stub
• -- Stub a special-purpose program to simulate
  the activity of missing component. The stub
  answers the calling sequence and passes back
  output data that lets the testing process
  continue.

27
Top-down Integration Testing
28
Its still not Perfect

• Test cases focus on the functionality of the
  system (functional requirements)
• Stubs must allow all possible conditions to be
  tested Its difficult to write them!!
• Possibly a very large number of stubs may be
  required, especially if the lowest level of the
  system contains many methods.
• Any solution? Modified top-down testing
  strategy
• Test each layer of the system decomposition
  individually before merging the layers
• Disadvantage of modified top-down testing Both,
  stubs and drivers are needed

29
Sandwich Strategy

• Combines top-down strategy with bottom-up
  strategy
• The system three layers
• A target layer in the middle
• A layer above the target
• A layer below the target
• Testing converges at the target layer
• How do you select the target layer if there are
  more than 3 layers?
• Heuristic Try to minimize the number of stubs
  and drivers

30
Sandwich Strategy
31
Can it be Further Improved ?

• Top and Bottom Layer Tests can be done in
  parallel
• Does not test the individual subsystems
  thoroughly before integration
• Solution? Modified Sandwich Strategy

32
Modified Sandwich Testing Strategy

• Test in parallel
• Middle layer with drivers and stubs
• Top layer with stubs
• Bottom layer with drivers
• Test in parallel
• Top layer accessing middle layer (top layer
  replaces drivers)
• Bottom accessed by middle layer (bottom layer
  replaces stubs)

33
(No Transcript)
34
Bottom-Up Integration
35
Top Down Integration
36
Sandwich Testing
37
Examples
38
(No Transcript)
39
Example 2 Incremental Testing
40
(No Transcript)
41
Example 3 Incremental Testing
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
Example 4 Buttom-Up Testing
46
(No Transcript)
47
Example 5 Buttom-Up Testing
48
(No Transcript)
49
(No Transcript)
50
Symptoms Causes
symptom and cause may be
geographically separated

symptom may disappear when
another problem is fixed

cause may be due to a
combination of non-errors

cause may be due to a system
or compiler error

cause may be due to
symptom
assumptions that everyone
cause
believes

symptom may be intermittent
51
Debugging Techniques
brute force / testing

backtracking

induction

deduction
52
System Testing

• Functional Testing (white-box)
• Structure Testing (black-box)
• Performance Testing
• Stress, Volume, Configuration, Backward
  compatibility, Security, Timing, Environmental,
  Reliability, Maintainability, Recovery, Human
  factors.
• Acceptance Testing by the client,
• Alpha test
• Sponsor uses the software at the developers
  site.
• Software used in a controlled setting, with the
  developer always ready to fix bugs.
• Beta test
• Conducted at sponsors site (developer is not
  present)
• Software gets a realistic workout in target
  environ-ment
• Potential customer might get discouraged

53
System Testing

• Installation Testing
• Impact of requirements on system testing
• The more explicit the requirements, the easier
  they are to test.
• Quality of use cases determines the ease of
  functional testing
• Quality of subsytem decomposition determines the
  ease of structure testing
• Quality of nonfunctional requirements and
  constraints determines the ease of performance
  tests

54
Example 6 Acceptance Testing
55
Song Relevancy Client Run

• User starts client GUI
• Client prompts for users Napster login
  password
• User types in a song name or keywords to search
  for
• Search options are Exact Search or Relative
  Search Level of Relevancy
• Query is sent to our Search Engine Server
• Server consults our MusicMap
• Server sends a list of relative songs back to
  client
• List shows up to user in GUI
• User selects a song he/she wants to download
• A list of servers is shown from which the user
  can download the song from
• User selects server file is downloaded to
  users computer

56
(No Transcript)
57
UML-based Integration Testing
Jean Hartmann, Claudio Imoberdorf Siemens
Corporate Research Princeton NJ 08540 Tel 1
609 734 3361 Fax [phone] Email
jhartmann_at_scr.siemens.com
58
Objectives

• Growing Importance of Domain-specific Frameworks
• Modeling of Software Components and their
  Interfaces
• Using a standardized design notation gt UML
• The model must express the component interactions
• Using the Model for Test Generation and Execution
• Focusing on unit and integration testing
  (black-box)
• Automating the test generation and execution
  steps
• Providing test execution support for
  middleware-based components

59
Modeling Components

• Use UML Statecharts to model each component
  (object)
• Statecharts should reflect the normal and
  erroneous behavior of the component
• To facilitate component interaction, we extend
  UML with a new transition labeling scheme
• For integration testing, users define the
  collection of components to be tested (subsystem
  definition)
• Assumptions
• Point-to-point communication semantics between
  components rather than a shared event model
• Communications are synchronous (blocking)
• Implementation is deterministic

60
Steps Taken During Model Composition

• 1. Normalizing the UML-based Models
• Importing the Rational Rose repository
• Noting the subsystem definition, if any
• Resolving transitions with multiple events
• 2. Composing the Global Behavioral Model
• Applying an incremental composition and reduction
  algorithm
• Determining a composition order (based on
  subsystem definition)
• Scalability
• Algorithm complexity is better than exponential

61
Example A Communication Protocol

• Focus on integration testing
• Generate test cases to validate component
  interaction
• Consider subsystem A
• External interfaces
• tuser
• timer
• txport
• Internal Interfaces
• timing

62
Matching Send and Receive Events Between
Components
63
Normalizing the UML-based Models...
64
Composing a Global Behavioral Model...
65
Test Case Generation
For integration testing all transitions between
components are covered
66
Test Case Execution

• Executable test driver is created from the
  generated test cases
• We have specific support for reactive components
• This is achieved through an event pattern
  matching mechanism
• Supports automated regression testing during unit
  and integration testing

67
Implementation TnT TDE/UML TECS
Interface Test Language (ITL)
COM Interface TSL Test Design
Rational Rose Test
Generation Test Execution Modeling
Tool Tool (TDE/UML) Environment (TECS)
68
Applying TnT in the Software Lifecycle
Development Process
Inception
Elaboration
Transition
Construction
Use Cases
Detailed Design
High-level Design
Artifacts
Unit Test
Integration Test
System Test
69
Future Research and Development

• Modeling
• Support additional UML model elements (e.g.
  nested states)
• Examine the issues related to asynchronous
  communication between components
• Modeling real-time aspects (e.g. timing
  constraints)
• Develop a philosophy for an optimal composition
  order
• Test Generation
• Exploit the use of data variations within test
  generation for integration testing purposes
• Test Execution
• Focus on a distributed test execution environment

70
Conclusions

• We provide a UML-based test generation and
  execution environment using Statecharts
• We focus on unit and integration testing for
  components
• We support the testing of middleware-based
  components
• We are starting to apply it within Siemens
• We are continuing to refine and improve TnT