Test this group using stubs to take the place of .. - PowerPoint PPT Presentation

Loading...

PPT – Test this group using stubs to take the place of .. PowerPoint presentation | free to download - id: 10d8e-NjcwZ



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Test this group using stubs to take the place of ..

Description:

Test this group using stubs to take the place of ... Software Compatibility Test. Focus is compatibility with other products/systems in the environment and/or ... – PowerPoint PPT presentation

Number of Views:86
Avg rating:3.0/5.0
Slides: 65
Provided by: mcat
Learn more at: http://www.cise.ufl.edu
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Test this group using stubs to take the place of ..


1
Integration and Higher Level Testing
Software Testing and Verification Lecture 11
  • Prepared by
  • Stephen M. Thebaut, Ph.D.
  • University of Florida

2
Context
  • Higher-level testing begins with the integration
    of (already unit-tested) modules to form
    higher-level program entities (e.g., components).
  • The primary objective of integration testing is
    to discover interface errors among the elements
    being integrated. (Somewhat in keeping with the
    spirit of smoke testing.)

3
Context (contd)
  • Once the elements have been successfully
    integrated (i.e., once they are able to function
    together), the functional and non-functional
    characteristics of the higher-level element can
    be tested thoroughly (via component, product, or
    system testing).

4
Integration Testing
  • Integration testing is carried out when
    integrating (i.e., combining)
  • Units or modules to form a component
  • Components to form a product
  • Products to form a system
  • The strategy employed can significantly affect
    the time and effort required to yield a working,
    higher-level element.

5
Integration Testing (contd)
  • Note that integration testing is sometimes
    defined as the level of testing between unit and
    system. We use a more general model of the
    testing process.

6
Levels of Testing
A More General View
System Test
A Popular View
Integration Test
System Test
Product Test
Integration Test
Integration Test
Unit Test
Component Test
Integration Test
Unit Test
7
Integration Testing Strategies
  • The first (and usually the easiest...) issue to
    address is the choice between instantaneous and
    incremental integration testing.
  • The former is sometimes referred to as the big
    bang approach. (Guess why!) Locating subtle
    errors can be very difficult after the bang.
  • Integration is a process, not an event.

8
Integration Testing Strategies (contd)
  • Incremental integration testing results in some
    additional overhead, but can significantly reduce
    error localization and correction time. (What is
    the overhead?)
  • The optimum incremental approach is inherently
    dependent on the individual project and the pros
    and cons of the various alternatives.

9
As an aside
  • Heres a slide from an on-line lecture that I
    came
  • across recently

Principles of Integration
  • Proper policy and plans
  • Advocacy
  • Manpower training
  • Realistic tasks
  • Coordination with other
  • sectors
  • Proper support
  • Access to drugs

Integration of Substance Abuse Disorders in
National Rural Health Mission (NRHM), Centre
for Community Medicine, All India Institute
of Medical Sciences, New Delhi
10
Incremental Strategies
  • Suppose we are integrating a group of modules to
    form a component, the control structure of which
    will form a calling hierarchy as shown.

11
Incremental Strategies (contd)
  • In what order should the modules be integrated?
  • From the top (root) module toward the bottom?
  • From bottom (leaf) modules toward the top?
  • By function?
  • Critical or high-risk modules first?
  • By availability?

12
Incremental Strategies (contd)
  • How many should be combined at a time?
  • What scaffolding (i.e., drivers and stubs to
    exercise the modules and oracles to
    interpret/inspect test results) will be required?
  • Is scaffolding ever required outside the context
    of integration testing?

13
Top-Down Strategy
  • Start with the root and one or more called
    modules.
  • Test this group using stubs to take the place of
    missing called modules, and one driver (if
    necessary) to call the root module.
  • Add one or more other called modules, replacing
    and providing new stubs as necessary.
  • (contd)

14
Top-Down Strategy (contd)
  • Continue the process until all elements have been
    integrated and tested.

15
Top-Down Strategy (contd)
A
B
C
D
C
D
E
F
G
H
I
G
H
I
J
K
L
16
Top-Down Strategy (contd)
driver
A
B
stub
stub
stub
stub
17
Top-Down Strategy (contd)
driver
A
B
stub
C
C
stub
stub
stub
stub
18
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
stub
stub
stub
stub
stub
stub
19
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
stub
stub
stub
stub
stub
20
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
stub
stub
stub
stub
stub
21
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
stub
stub
stub
stub
22
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
H
H
stub
stub
stub
stub
23
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
H
H
I
I
stub
stub
stub
24
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
H
H
I
I
J
stub
stub
25
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
H
H
I
I
J
K
stub
26
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
H
H
I
I
J
K
L
27
Top-Down Strategy (contd)
  • Potential Advantages
  • Allows early verification of high-level behavior.
  • One driver (at most) is required.
  • Modules can be added one at a time with each step
    if desired.
  • Supports both breadth first and depth
    first approaches.

28
Top-Down Strategy (contd)
  • Potential Disadvantages
  • Delays verification of low-level behavior.
  • Stubs are required for missing elements.
  • Test case inputs may be difficult to formulate.
  • Test case outputs may be difficult to interpret.
    (Oracles may be needed to interpret/inspect test
    results.)

29
Bottom-Up Strategy
  • Start at the bottom of the hierarchy with two or
    more sibling leaf modules, or an only-child leaf
    module with its parent.
  • Test this group using a driver. (No stubs are
    required.)
  • Add one or more additional siblings, replacing
    drivers with modules only when all modules they
    call have been integrated.
  • (contd)

30
Bottom-Up Strategy (contd)
  • Continue the process until all elements of the
    subtree have been integrated and tested.
  • Repeat the steps above for the remaining subtrees
    in the hierarchy (or handle in parallel).
  • Incrementally combine the sub-trees and then add
    the root module.

31
Bottom-Up Strategy (contd)
A
B
C
D
C
D
E
F
G
H
I
G
H
I
J
K
L
32
Bottom-Up Strategy (contd)
driver
F
J
33
Bottom-Up Strategy (contd)
driver
F
E
J
34
Bottom-Up Strategy (contd)
driver
B
F
E
J
35
Bottom-Up Strategy (contd)
driver
B
F
E
driver
J
K
L
36
Bottom-Up Strategy (contd)
driver
B
driver
F
E
H
H
J
K
L
37
Bottom-Up Strategy (contd)
driver
B
driver
F
E
H
H
I
I
J
K
L
38
Bottom-Up Strategy (contd)
driver
driver
B
D
D
F
E
H
H
I
I
J
K
L
39
Bottom-Up Strategy (contd)
driver
driver
B
D
D
driver
F
E
H
H
I
I
G
G
J
K
L
40
Bottom-Up Strategy (contd)
driver
driver
driver
B
D
D
C
C
F
E
H
H
I
I
G
G
J
K
L
41
Bottom-Up Strategy (contd)
driver
driver
D
D
C
C
B
H
H
I
I
G
G
E
F
K
L
J
42
Bottom-Up Strategy (contd)
driver
driver
B
C
C
D
D
F
E
G
G
H
H
I
I
J
K
K
L
43
Bottom-Up Strategy (contd)
driver
B
C
C
D
D
E
F
G
G
H
H
I
I
J
K
L
44
Bottom-Up Strategy (contd)
driver
B
C
C
D
D
E
F
G
G
H
H
I
I
J
K
L
45
Bottom-Up Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
H
H
I
I
J
K
L
46
Bottom-Up Strategy (contd)
  • Potential Advantages
  • Allows early verification of low-level behavior.
  • No stubs are required.
  • Easier to formulate input data for some subtrees.
  • Easier to interpret output data for others.

47
Bottom-Up Strategy (contd)
  • Potential Disadvantages
  • Delays verification of high-level behavior.
  • Drivers are required for missing elements.
  • As subtrees are combined, a large number of
    elements may be integrated at one time.

48
Hybrid Incremental Integration Approaches
  • Risk Driven
  • Start by integrating the most critical or
    complex modules together with modules they call
    or are called by.
  • Schedule Driven
  • To the extent possible, integrate modules as
    they become available.
  • (contd)

49
Hybrid Incremental Integration Approaches (contd)
  • Function or Thread Driven
  • Integrate the modules associated with a key
    function (thread) continue the process by
    selecting another function, etc.

50
How about Object-Oriented Systems?
  • Just as a calling hierarchy allows design of an
    integration strategy for imperative software,
    use/include relations serve this purpose for
    object-oriented software.
  • Since there may be no single root class,
    testing usually proceeds cluster by cluster in a
    bottom-up fashion, starting with leaf classes
    that depend on no others.
  • We will come back to this in Lecture 12.

51
Higher-Level Testing
  • Higher-level tests focus on the core
    functionality specified for higher level
    elements, and on certain emergent properties that
    become more observable as testing progresses
    toward the system level.
  • The black-box testing strategies already
    considered (e.g., partition and combinatorial
    approaches) apply to functional testing at any
    level.

52
Higher-Level Testing (contd)
  • Higher-level testing typically includes
  • A brief overview of each follows.
  • Usability test
  • Installability test
  • Serviceability test
  • Performance test
  • Stress test
  • Security test
  • Software compatibility test
  • Device and configuration test
  • Recovery test
  • Reliability test

53
Usability Test
  • Focus is on factors which influence the ease with
    which potential end-users are able to utilize the
    system to accomplish their goals.
  • Specialized and sophisticated HCI experts
    conduct experiments in simulated work
    environments.
  • Protocol analysis is utilized to identify
    usability bottlenecks.
  • Application-specific metrics related to
    under-standability, learnability, and operability
    may be employed.

54
Installability Test
  • Focus is functional and non-functional
    requirements related to the installation of the
    product/system.
  • Coverage includes
  • Media correctness and fidelity
  • Relevant documentation (including examples)
  • Installation processes and supporting system
    functions.
  • (contd)

55
Installability Test (contd)
  • Functions, procedures, documentation, etc.,
    associated with product/system decommissioning
    must also be tested.

56
Serviceability Test
  • Focus is requirements related to upgrading or
    fixing problems after installation.
  • Coverage includes
  • Change procedures (adaptive, perfective, and
    corrective service scenarios)
  • Supporting documentation
  • System diagnostic tools

57
Performance Test
  • Focus is requirements related to throughput,
    response time, memory utilization, input/ output
    rates, etc.
  • Also very specialized in some organizations
    sophisticated test-beds and instrumentation are
    the norm.
  • Statistical testing based on an operational
    profile is often employed.
  • Requirements must be stated in quantifiable terms.

58
Stress Test
  • Focus is system behavior at or near overload
    conditions (i.e., pushing the system to
    failure).
  • Often undertaken with performance testing.
  • In general, products are required to exhibit
    graceful failures and non-abrupt perfor-mance
    degradation.

59
Security Test
  • Focus is vulnerability of resources to
    unauthorized access or manipulation.
  • Issues include
  • Physical security of computer resources, media,
    etc.,
  • Login and password procedures/ policies,
  • Levels of authorization for data or procedural
    access, etc.

60
Software Compatibility Test
  • Focus is compatibility with other
    products/systems in the environment and/or with
    interoperability standards.
  • May also concern source- or object-code
    compatibility with different operating
    environment versions.
  • AKA compatibility/conversion testing when
    conversion procedures or processes are involved.

61
Device and Configuration Test
  • Focus is configurability for, and/or
    compatibility with, all supported hardware
    configurations.
  • Particularly taxing for client/server-based
    applications
  • Tests are usually limited to combinations of
    representative devices for each supported
    protocol.

62
Recovery Test
  • Focus is ability to recover from exceptional
    conditions associated with hardware, software, or
    people.
  • This can involve
  • detecting exceptional conditions,
  • switch-overs to standby systems,
  • recovery of data,
  • maintaining audit trails, etc.
  • May also involve external procedures such as
    storing backup tapes, etc.

63
Reliability Test
  • Requirements may be expressed as
  • the probability of no failure in a specified time
    interval, or as
  • the expected mean time to failure.
  • Appropriate interpretations for failure and time
    are critical.
  • Utilizes Statistical testing based on an
    operational profile.

64
Integration and Higher Level Testing
Software Testing and Verification Lecture 11
  • Prepared by
  • Stephen M. Thebaut, Ph.D.
  • University of Florida
About PowerShow.com