Integration and Higher Level Testing - PowerPoint PPT Presentation

Loading...

PPT – Integration and Higher Level Testing PowerPoint presentation | free to download - id: 64e054-Yjc4Y



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Integration and Higher Level Testing

Description:

Integration and Higher Level Testing Software Testing and Verification Lecture 11 Prepared by Stephen M. Thebaut, Ph.D. University of Florida Recovery Test Focus is ... – PowerPoint PPT presentation

Number of Views:77
Avg rating:3.0/5.0
Slides: 137
Provided by: mcat
Learn more at: http://www.cise.ufl.edu
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Integration and Higher Level Testing


1
Integration and Higher Level Testing
Software Testing and Verification Lecture 11
  • Prepared by
  • Stephen M. Thebaut, Ph.D.
  • University of Florida

2
Context
  • Higher-level testing begins with the integration
    of (already unit-tested) modules to form
    higher-level program entities (e.g., components).
  • The primary objective of integration testing is
    to discover interface and blatant higher-level
    design errors associated with the elements being
    integrated. (Somewhat in keeping with the spirit
    of smoke testing.)

3
Context
  • Higher-level testing begins with the integration
    of (already unit-tested) modules to form
    higher-level program entities (e.g., components).
  • The primary objective of integration testing is
    to discover interface and blatant higher-level
    design errors associated with the elements being
    integrated. (Somewhat in keeping with the spirit
    of smoke testing.)

4
Context (contd)
  • Once the elements have been successfully
    integrated (i.e., once they are able to function
    together), the functional and non-functional
    characteristics of the higher-level element can
    be tested thoroughly (via component, product, or
    system testing).

5
Integration Testing
  • Integration testing is carried out when
    integrating (i.e., combining)
  • Units or modules to form a component
  • Components to form a product
  • Products to form a system
  • The strategy employed can significantly affect
    the time and effort required to yield a working,
    higher-level element.

6
Integration Testing
  • Integration testing is carried out when
    integrating (i.e., combining)
  • Units or modules to form a component
  • Components to form a product
  • Products to form a system
  • The strategy employed can significantly affect
    the time and effort required to yield a working,
    higher-level element.

7
Integration Testing (contd)
  • Note that integration testing is sometimes
    defined as the level of testing between unit and
    system. We use a more general model of the
    testing process.

8
Levels of Testing
A More General View
System Test
A Popular View
Integration Test
System Test
Product Test
Integration Test
Integration Test
Unit Test
Component Test
Integration Test
Unit Test
9
Integration Testing Strategies
  • The first (and usually the easiest...) issue to
    address is the choice between instantaneous and
    incremental integration testing.
  • The former is sometimes referred to as the big
    bang approach. (Guess why!) Locating subtle
    errors can be very difficult after the bang.
  • Integration is a process, not an event.

10
Integration Testing Strategies
  • The first (and usually the easiest...) issue to
    address is the choice between instantaneous and
    incremental integration testing.
  • The former is sometimes referred to as the big
    bang approach. (Guess why!) Locating subtle
    errors can be very difficult after the bang.
  • Integration is a process, not an event.

11
Integration Testing Strategies
  • The first (and usually the easiest...) issue to
    address is the choice between instantaneous and
    incremental integration testing.
  • The former is sometimes referred to as the big
    bang approach. (Guess why!) Locating subtle
    errors can be very difficult after the bang.
  • Integration is a process, not an event.

12
Integration Testing Strategies (contd)
  • Incremental integration testing results in some
    additional overhead, but can significantly reduce
    error localization and correction time. (What is
    the overhead?)
  • The optimum incremental approach is inherently
    dependent on the individual project and the pros
    and cons of the various alternatives available.

13
Integration Testing Strategies (contd)
  • Incremental integration testing results in some
    additional overhead, but can significantly reduce
    error localization and correction time. (What is
    the overhead?)
  • The optimum incremental approach is inherently
    dependent on the individual project and the pros
    and cons of the various alternatives available.

14
As an aside
  • Heres a slide from an on-line lecture that I
    came
  • across recently

Principles of Integration
  • Proper policy and plans
  • Advocacy
  • Manpower training
  • Realistic tasks
  • Coordination with other
  • sectors
  • Proper support
  • Access to drugs

Integration of Substance Abuse Disorders in
National Rural Health Mission (NRHM), Centre
for Community Medicine, All India Institute
of Medical Sciences, New Delhi
15
Incremental Strategies
  • Suppose we are integrating a group of modules to
    form a component, the control structure of which
    will form a calling hierarchy as shown.

16
Incremental Strategies (contd)
  • In what order should the modules be integrated?
  • From the top (root) module toward the bottom?
  • From bottom (leaf) modules toward the top?
  • By function?
  • Critical or high-risk modules first?
  • By availability?

17
Incremental Strategies (contd)
  • In what order should the modules be integrated?
  • From the top (root) module toward the bottom?
  • From bottom (leaf) modules toward the top?
  • By function?
  • Critical or high-risk modules first?
  • By availability?

18
Incremental Strategies (contd)
  • In what order should the modules be integrated?
  • From the top (root) module toward the bottom?
  • From bottom (leaf) modules toward the top?
  • By function?
  • Critical or high-risk modules first?
  • By availability?

19
Incremental Strategies (contd)
  • In what order should the modules be integrated?
  • From the top (root) module toward the bottom?
  • From bottom (leaf) modules toward the top?
  • By function?
  • Critical or high-risk modules first?
  • By availability?

20
Incremental Strategies (contd)
  • In what order should the modules be integrated?
  • From the top (root) module toward the bottom?
  • From bottom (leaf) modules toward the top?
  • By function?
  • Critical or high-risk modules first?
  • By availability?

21
Incremental Strategies (contd)
  • In what order should the modules be integrated?
  • From the top (root) module toward the bottom?
  • From bottom (leaf) modules toward the top?
  • By function?
  • Critical or high-risk modules first?
  • By availability?

22
Incremental Strategies (contd)
  • How many should be combined at a time?
  • What scaffolding (i.e., drivers and stubs to
    exercise the modules, and oracles to
    interpret/inspect test results) will be required?
  • Is scaffolding ever required outside the context
    of integration testing?

23
Incremental Strategies (contd)
  • How many should be combined at a time?
  • What scaffolding (i.e., drivers and stubs to
    exercise the modules, and oracles to
    interpret/inspect test results) will be required?
  • Is scaffolding ever required outside the context
    of integration testing?

24
Incremental Strategies (contd)
  • How many should be combined at a time?
  • What scaffolding (i.e., drivers and stubs to
    exercise the modules, and oracles to
    interpret/inspect test results) will be required?
  • Is scaffolding ever required outside the context
    of integration testing?

25
Top-Down Strategy
  • Start with the root and one or more called
    modules.
  • Test this group using stubs to take the place of
    missing called modules, and one driver (if
    necessary) to call the root module.
  • Add one or more other called modules, replacing
    and providing new stubs as necessary.
  • (contd)

26
Top-Down Strategy
  • Start with the root and one or more called
    modules.
  • Test this group using stubs to take the place of
    missing called modules, and one driver (if
    necessary) to call the root module.
  • Add one or more other called modules, replacing
    and providing new stubs as necessary.
  • (contd)

27
Top-Down Strategy
  • Start with the root and one or more called
    modules.
  • Test this group using stubs to take the place of
    missing called modules, and one driver (if
    necessary) to call the root module.
  • Add one or more other called modules, replacing
    and providing new stubs as necessary.
  • (contd)

28
Top-Down Strategy (contd)
  1. Continue the process until all elements have been
    integrated and tested.

29
Top-Down Strategy (contd)
A
B
C
D
C
D
E
F
G
H
I
G
H
I
J
K
L
30
Top-Down Strategy (contd)
driver
A
B
stub
stub
stub
stub
31
Top-Down Strategy (contd)
driver
A
B
stub
C
C
stub
stub
stub
stub
32
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
stub
stub
stub
stub
stub
stub
33
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
stub
stub
stub
stub
stub
34
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
stub
stub
stub
stub
stub
35
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
stub
stub
stub
stub
36
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
H
H
stub
stub
stub
stub
37
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
H
H
I
I
stub
stub
stub
38
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
H
H
I
I
J
stub
stub
39
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
H
H
I
I
J
K
stub
40
Top-Down Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
H
H
I
I
J
K
L
41
Top-Down Strategy (contd)
  • Potential Advantages
  • Allows early verification of high-level behavior.
  • One driver (at most) is required.
  • Modules can be added one at a time with each step
    if desired.
  • Supports both breadth first and depth
    first approaches.

42
Top-Down Strategy (contd)
  • Potential Advantages
  • Allows early verification of high-level behavior.
  • One driver (at most) is required.
  • Modules can be added one at a time with each step
    if desired.
  • Supports both breadth first and depth
    first approaches.

43
Top-Down Strategy (contd)
  • Potential Advantages
  • Allows early verification of high-level behavior.
  • One driver (at most) is required.
  • Modules can be added one at a time with each step
    if desired.
  • Supports both breadth first and depth
    first approaches.

44
Top-Down Strategy (contd)
  • Potential Advantages
  • Allows early verification of high-level behavior.
  • One driver (at most) is required.
  • Modules can be added one at a time with each step
    if desired.
  • Supports both breadth first and depth
    first approaches.

45
Top-Down Strategy (contd)
  • Potential Advantages
  • Allows early verification of high-level behavior.
  • One driver (at most) is required.
  • Modules can be added one at a time with each step
    if desired.
  • Supports both breadth first and depth
    first approaches.

46
Top-Down Strategy (contd)
  • Potential Disadvantages
  • Delays verification of low-level behavior.
  • Stubs are required for missing elements.
  • Test case inputs may be difficult to formulate.
  • Test case outputs may be difficult to interpret.
    (Oracles may be needed to interpret/inspect test
    results.)

47
Top-Down Strategy (contd)
  • Potential Disadvantages
  • Delays verification of low-level behavior.
  • Stubs are required for missing elements.
  • Test case inputs may be difficult to formulate.
  • Test case outputs may be difficult to interpret.
    (Oracles may be needed to interpret/inspect test
    results.)

48
Top-Down Strategy (contd)
  • Potential Disadvantages
  • Delays verification of low-level behavior.
  • Stubs are required for missing elements.
  • Test case inputs may be difficult to formulate.
  • Test case outputs may be difficult to interpret.
    (Oracles may be needed to interpret/inspect test
    results.)

49
Top-Down Strategy (contd)
  • Potential Disadvantages
  • Delays verification of low-level behavior.
  • Stubs are required for missing elements.
  • Test case inputs may be difficult to formulate.
  • Test case outputs may be difficult to interpret.
    (Oracles may be needed to interpret/inspect test
    results.)

50
Top-Down Strategy (contd)
  • Potential Disadvantages
  • Delays verification of low-level behavior.
  • Stubs are required for missing elements.
  • Test case inputs may be difficult to formulate.
  • Test case outputs may be difficult to interpret.
    (Oracles may be needed to interpret/inspect test
    results.)

51
Bottom-Up Strategy
  • Start at the bottom of the hierarchy with two or
    more sibling leaf modules, or an only-child leaf
    module with its parent.
  • Test this group using a driver. (No stubs are
    required.)
  • Add one or more additional siblings, replacing
    drivers with modules only when all modules they
    call have been integrated.
  • (contd)

52
Bottom-Up Strategy
  • Start at the bottom of the hierarchy with two or
    more sibling leaf modules, or an only-child leaf
    module with its parent.
  • Test this group using a driver. (No stubs are
    required.)
  • Add one or more additional siblings, replacing
    drivers with modules only when all modules they
    call have been integrated.
  • (contd)

53
Bottom-Up Strategy
  • Start at the bottom of the hierarchy with two or
    more sibling leaf modules, or an only-child leaf
    module with its parent.
  • Test this group using a driver. (No stubs are
    required.)
  • Add one or more additional siblings, replacing
    drivers with modules only when all modules they
    call have been integrated.
  • (contd)

54
Bottom-Up Strategy (contd)
  • Continue the process until all elements of the
    subtree have been integrated and tested.
  • Repeat the steps above for the remaining subtrees
    in the hierarchy (or handle in parallel).
  • Incrementally combine the sub-trees and then add
    the root module.

55
Bottom-Up Strategy (contd)
  • Continue the process until all elements of the
    subtree have been integrated and tested.
  • Repeat the steps above for the remaining subtrees
    in the hierarchy (or handle in parallel).
  • Incrementally combine the sub-trees and then add
    the root module.

56
Bottom-Up Strategy (contd)
  • Continue the process until all elements of the
    subtree have been integrated and tested.
  • Repeat the steps above for the remaining subtrees
    in the hierarchy (or handle in parallel).
  • Incrementally combine the sub-trees and then add
    the root module.

57
Bottom-Up Strategy (contd)
A
B
C
D
C
D
E
F
G
H
I
G
H
I
J
K
L
58
Bottom-Up Strategy (contd)
driver
F
J
59
Bottom-Up Strategy (contd)
driver
F
E
J
60
Bottom-Up Strategy (contd)
driver
B
F
E
J
61
Bottom-Up Strategy (contd)
driver
B
F
E
driver
J
K
L
62
Bottom-Up Strategy (contd)
driver
B
driver
F
E
H
H
J
K
L
63
Bottom-Up Strategy (contd)
driver
B
driver
F
E
H
H
I
I
J
K
L
64
Bottom-Up Strategy (contd)
driver
driver
B
D
D
F
E
H
H
I
I
J
K
L
65
Bottom-Up Strategy (contd)
driver
driver
B
D
D
driver
F
E
H
H
I
I
G
G
J
K
L
66
Bottom-Up Strategy (contd)
driver
driver
driver
B
D
D
C
C
F
E
H
H
I
I
G
G
J
K
L
67
Bottom-Up Strategy (contd)
driver
driver
D
D
C
C
B
H
H
I
I
G
G
E
F
K
L
J
68
Bottom-Up Strategy (contd)
driver
driver
B
C
C
D
D
F
E
G
G
H
H
I
I
J
K
K
L
69
Bottom-Up Strategy (contd)
driver
B
C
C
D
D
E
F
G
G
H
H
I
I
J
K
L
70
Bottom-Up Strategy (contd)
driver
B
C
C
D
D
E
F
G
G
H
H
I
I
J
K
L
71
Bottom-Up Strategy (contd)
driver
A
B
C
C
D
D
E
F
G
G
H
H
I
I
J
K
L
72
Bottom-Up Strategy (contd)
  • Potential Advantages
  • Allows early verification of low-level behavior.
  • No stubs are required.
  • Easier to formulate input data for some subtrees.
  • Easier to interpret output data for others.

73
Bottom-Up Strategy (contd)
  • Potential Advantages
  • Allows early verification of low-level behavior.
  • No stubs are required.
  • Easier to formulate input data for some subtrees.
  • Easier to interpret output data for others.

74
Bottom-Up Strategy (contd)
  • Potential Advantages
  • Allows early verification of low-level behavior.
  • No stubs are required.
  • Easier to formulate input data for some subtrees.
  • Easier to interpret output data for others.

75
Bottom-Up Strategy (contd)
  • Potential Advantages
  • Allows early verification of low-level behavior.
  • No stubs are required.
  • Easier to formulate input data for some subtrees.
  • Easier to interpret output data for others.

76
Bottom-Up Strategy (contd)
  • Potential Advantages
  • Allows early verification of low-level behavior.
  • No stubs are required.
  • Easier to formulate input data for some subtrees.
  • Easier to interpret output data for others.

77
Bottom-Up Strategy (contd)
  • Potential Disadvantages
  • Delays verification of high-level behavior.
  • Drivers are required for missing elements.
  • As subtrees are combined, a large number of
    elements may be integrated at one time.

78
Bottom-Up Strategy (contd)
  • Potential Disadvantages
  • Delays verification of high-level behavior.
  • Drivers are required for missing elements.
  • As subtrees are combined, a large number of
    elements may be integrated at one time.

79
Bottom-Up Strategy (contd)
  • Potential Disadvantages
  • Delays verification of high-level behavior.
  • Drivers are required for missing elements.
  • As subtrees are combined, a large number of
    elements may be integrated at one time.

80
Bottom-Up Strategy (contd)
  • Potential Disadvantages
  • Delays verification of high-level behavior.
  • Drivers are required for missing elements.
  • As subtrees are combined, a large number of
    elements may be integrated at one time.

81
Hybrid Incremental Integration Approaches
  • Risk Driven
  • Start by integrating the most critical or
    complex modules together with modules they call
    or are called by.
  • Schedule Driven
  • To the extent possible, integrate modules as
    they become available.
  • (contd)

82
Hybrid Incremental Integration Approaches
  • Risk Driven
  • Start by integrating the most critical or
    complex modules together with modules they call
    or are called by.
  • Schedule Driven
  • To the extent possible, integrate modules as
    they become available.
  • (contd)

83
Hybrid Incremental Integration Approaches
  • Risk Driven
  • Start by integrating the most critical or
    complex modules together with modules they call
    or are called by.
  • Schedule Driven
  • To the extent possible, integrate modules as
    they become available.
  • (contd)

84
Hybrid Incremental Integration Approaches (contd)
  • Function or Thread Driven
  • Integrate the modules associated with a key
    function (thread) continue the process by
    selecting another function, etc.

85
Hybrid Incremental Integration Approaches (contd)
  • Function or Thread Driven
  • Integrate the modules associated with a key
    function (thread) continue the process by
    selecting another function, etc.

86
How about Object-Oriented Systems?
  • Just as a calling hierarchy allows design of an
    integration strategy for imperative software,
    use/include relations serve this purpose for
    object-oriented software.
  • Since there may be no single root class,
    testing usually proceeds cluster by cluster in a
    bottom-up fashion, starting with leaf classes
    that depend on no others.
  • We will come back to this in Lecture 12.

87
How about Object-Oriented Systems?
  • Just as a calling hierarchy allows design of an
    integration strategy for imperative software,
    use/include relations serve this purpose for
    object-oriented software.
  • Since there may be no single root class,
    testing usually proceeds cluster by cluster in a
    bottom-up fashion, starting with leaf classes
    that depend on no others.
  • We will come back to this in Lecture 12.

88
How about Object-Oriented Systems?
  • Just as a calling hierarchy allows design of an
    integration strategy for imperative software,
    use/include relations serve this purpose for
    object-oriented software.
  • Since there may be no single root class,
    testing usually proceeds cluster by cluster in a
    bottom-up fashion, starting with leaf classes
    that depend on no others.
  • We will come back to this in Lecture 12.

89
Higher-Level Testing
  • Higher-level tests focus on the core
    functionality specified for higher level
    elements, and on certain emergent properties that
    become more observable as testing progresses
    toward the system level.
  • The black-box testing strategies already
    considered (e.g., partition and combinatorial
    approaches) apply to functional testing at any
    level.

90
Higher-Level Testing
  • Higher-level tests focus on the core
    functionality specified for higher level
    elements, and on certain emergent properties that
    become more observable as testing progresses
    toward the system level.
  • The black-box testing strategies already
    considered (e.g., partition and combinatorial
    approaches) apply to functional testing at any
    level.

91
Higher-Level Testing (contd)
  • Higher-level testing typically includes
  • A brief overview of each follows.
  • Usability test
  • Installability test
  • Serviceability test
  • Performance test
  • Stress test
  • Security test
  • Software compatibility test
  • Device and configuration test
  • Recovery test
  • Reliability test

92
Higher-Level Testing (contd)
  • Higher-level testing typically includes
  • A brief overview of each follows.
  • Usability test
  • Installability test
  • Serviceability test
  • Performance test
  • Stress test
  • Security test
  • Software compatibility test
  • Device and configuration test
  • Recovery test
  • Reliability test

93
Usability Test
  • Focus is on factors which influence the ease with
    which potential end-users are able to utilize the
    system to accomplish their goals.
  • Specialized and sophisticated HCI experts
    conduct experiments in simulated work
    environments.
  • Protocol analysis is utilized to identify
    usability bottlenecks.
  • Application-specific metrics related to
    under-standability, learnability, and operability
    may be employed.

94
Usability Test
  • Focus is on factors which influence the ease with
    which potential end-users are able to utilize the
    system to accomplish their goals.
  • Specialized and sophisticated HCI experts
    conduct experiments in simulated work
    environments.
  • Protocol analysis is utilized to identify
    usability bottlenecks.
  • Application-specific metrics related to
    under-standability, learnability, and operability
    may be employed.

95
Usability Test
  • Focus is on factors which influence the ease with
    which potential end-users are able to utilize the
    system to accomplish their goals.
  • Specialized and sophisticated HCI experts
    conduct experiments in simulated work
    environments.
  • Protocol analysis is utilized to identify
    usability bottlenecks.
  • Application-specific metrics related to
    under-standability, learnability, and operability
    may be employed.

96
Usability Test
  • Focus is on factors which influence the ease with
    which potential end-users are able to utilize the
    system to accomplish their goals.
  • Specialized and sophisticated HCI experts
    conduct experiments in simulated work
    environments.
  • Protocol analysis is utilized to identify
    usability bottlenecks.
  • Application-specific metrics related to
    under-standability, learnability, and operability
    may be employed.

97
Installability Test
  • Focus is functional and non-functional
    requirements related to the installation of the
    product/system.
  • Coverage includes
  • Media correctness and fidelity,
  • Relevant documentation (including examples), and
  • Installation processes and supporting system
    functions.
  • (contd)

98
Installability Test
  • Focus is functional and non-functional
    requirements related to the installation of the
    product/system.
  • Coverage includes
  • Media correctness and fidelity,
  • Relevant documentation (including examples), and
  • Installation processes and supporting system
    functions.
  • (contd)

99
Installability Test
  • Focus is functional and non-functional
    requirements related to the installation of the
    product/system.
  • Coverage includes
  • Media correctness and fidelity,
  • Relevant documentation (including examples), and
  • Installation processes and supporting system
    functions.
  • (contd)

100
Installability Test
  • Focus is functional and non-functional
    requirements related to the installation of the
    product/system.
  • Coverage includes
  • Media correctness and fidelity,
  • Relevant documentation (including examples), and
  • Installation processes and supporting system
    functions.
  • (contd)

101
Installability Test
  • Focus is functional and non-functional
    requirements related to the installation of the
    product/system.
  • Coverage includes
  • Media correctness and fidelity,
  • Relevant documentation (including examples), and
  • Installation processes and supporting system
    functions.
  • (contd)

102
Installability Test (contd)
  • Functions, procedures, documentation, etc.,
    associated with product/system decommissioning
    must also be tested.

103
Serviceability Test
  • Focus is requirements related to upgrading or
    fixing problems after installation.
  • Coverage includes
  • Change procedures (adaptive, perfective, and
    corrective service scenarios),
  • Supporting documentation, and
  • System diagnostic tools.

104
Serviceability Test
  • Focus is requirements related to upgrading or
    fixing problems after installation.
  • Coverage includes
  • Change procedures (adaptive, perfective, and
    corrective service scenarios),
  • Supporting documentation, and
  • System diagnostic tools.

105
Serviceability Test
  • Focus is requirements related to upgrading or
    fixing problems after installation.
  • Coverage includes
  • Change procedures (adaptive, perfective, and
    corrective service scenarios),
  • Supporting documentation, and
  • System diagnostic tools.

106
Serviceability Test
  • Focus is requirements related to upgrading or
    fixing problems after installation.
  • Coverage includes
  • Change procedures (adaptive, perfective, and
    corrective service scenarios),
  • Supporting documentation, and
  • System diagnostic tools.

107
Serviceability Test
  • Focus is requirements related to upgrading or
    fixing problems after installation.
  • Coverage includes
  • Change procedures (adaptive, perfective, and
    corrective service scenarios),
  • Supporting documentation, and
  • System diagnostic tools.

108
Performance Test
  • Focus is requirements related to throughput,
    response time, memory utilization, input/ output
    rates, etc.
  • Also very specialized in some organizations
    sophisticated test-beds and instrumentation are
    the norm.
  • Statistical testing based on an operational
    profile is often employed.
  • Requirements must be stated in quantifiable terms.

109
Performance Test
  • Focus is requirements related to throughput,
    response time, memory utilization, input/ output
    rates, etc.
  • Also very specialized in some organizations
    sophisticated test-beds and instrumentation are
    the norm.
  • Statistical testing based on an operational
    profile is often employed.
  • Requirements must be stated in quantifiable terms.

110
Performance Test
  • Focus is requirements related to throughput,
    response time, memory utilization, input/ output
    rates, etc.
  • Also very specialized in some organizations
    sophisticated test-beds and instrumentation are
    the norm.
  • Statistical testing based on an operational
    profile is often employed.
  • Requirements must be stated in quantifiable terms.

111
Performance Test
  • Focus is requirements related to throughput,
    response time, memory utilization, input/ output
    rates, etc.
  • Also very specialized in some organizations
    sophisticated test-beds and instrumentation are
    the norm.
  • Statistical testing based on an operational
    profile is often employed.
  • Requirements must be stated in quantifiable terms.

112
Stress Test
  • Focus is system behavior at or near overload
    conditions (i.e., pushing the system to
    failure).
  • Often undertaken with performance testing.
  • In general, products are required to exhibit
    graceful failures and non-abrupt perfor-mance
    degradation.

113
Stress Test
  • Focus is system behavior at or near overload
    conditions (i.e., pushing the system to
    failure).
  • Often undertaken with performance testing.
  • In general, products are required to exhibit
    graceful failures and non-abrupt perfor-mance
    degradation.

114
Stress Test
  • Focus is system behavior at or near overload
    conditions (i.e., pushing the system to
    failure).
  • Often undertaken with performance testing.
  • In general, products are required to exhibit
    graceful failures and non-abrupt perfor-mance
    degradation.

115
Security Test
  • Focus is vulnerability of resources to
    unauthorized access or manipulation.
  • Issues include
  • Physical security of computer resources, media,
    etc.,
  • Login and password procedures/policies,
  • Levels of authorization for data or procedural
    access, etc.

116
Security Test
  • Focus is vulnerability of resources to
    unauthorized access or manipulation.
  • Issues include
  • Physical security of computer resources, media,
    etc.,
  • Login and password procedures/policies,
  • Levels of authorization for data or procedural
    access, etc.

117
Security Test
  • Focus is vulnerability of resources to
    unauthorized access or manipulation.
  • Issues include
  • Physical security of computer resources, media,
    etc.,
  • Login and password procedures/policies,
  • Levels of authorization for data or procedural
    access, etc.

118
Security Test
  • Focus is vulnerability of resources to
    unauthorized access or manipulation.
  • Issues include
  • Physical security of computer resources, media,
    etc.,
  • Login and password procedures/policies,
  • Levels of authorization for data or procedural
    access, etc.

119
Security Test
  • Focus is vulnerability of resources to
    unauthorized access or manipulation.
  • Issues include
  • Physical security of computer resources, media,
    etc.,
  • Login and password procedures/policies,
  • Levels of authorization for data or procedural
    access, etc.

120
Software Compatibility Test
  • Focus is compatibility with other
    products/systems in the environment and/or with
    interoperability standards.
  • May also concern source- or object-code
    compatibility with different operating
    environment versions.
  • AKA compatibility/conversion testing when
    conversion procedures or processes are involved.

121
Software Compatibility Test
  • Focus is compatibility with other
    products/systems in the environment and/or with
    interoperability standards.
  • May also concern source- or object-code
    compatibility with different operating
    environment versions.
  • AKA compatibility/conversion testing when
    conversion procedures or processes are involved.

122
Software Compatibility Test
  • Focus is compatibility with other
    products/systems in the environment and/or with
    interoperability standards.
  • May also concern source- or object-code
    compatibility with different operating
    environment versions.
  • AKA compatibility/conversion testing when
    conversion procedures or processes are involved.

123
Device and Configuration Test
  • Focus is configurability for, and/or
    compatibility with, all supported hardware
    configurations.
  • Particularly taxing for client/server-based
    applications
  • Tests are usually limited to combinations of
    representative devices for each supported
    protocol.

124
Device and Configuration Test
  • Focus is configurability for, and/or
    compatibility with, all supported hardware
    configurations.
  • Particularly taxing for client/server-based
    applications
  • Tests are usually limited to combinations of
    representative devices for each supported
    protocol.

125
Device and Configuration Test
  • Focus is configurability for, and/or
    compatibility with, all supported hardware
    configurations.
  • Particularly taxing for client/server-based
    applications
  • Tests are usually limited to combinations of
    representative devices for each supported
    protocol.

126
Recovery Test
  • Focus is ability to recover from exceptional
    conditions associated with hardware, software, or
    people.
  • This can involve
  • detecting exceptional conditions,
  • switch-overs to standby systems,
  • recovery of data,
  • maintaining audit trails, etc.
  • May also involve external procedures such as
    storing backup tapes, etc.

127
Recovery Test
  • Focus is ability to recover from exceptional
    conditions associated with hardware, software, or
    people.
  • This can involve
  • detecting exceptional conditions,
  • switch-overs to standby systems,
  • recovery of data,
  • maintaining audit trails, etc.
  • May also involve external procedures such as
    storing backup tapes, etc.

128
Recovery Test
  • Focus is ability to recover from exceptional
    conditions associated with hardware, software, or
    people.
  • This can involve
  • detecting exceptional conditions,
  • switch-overs to standby systems,
  • recovery of data,
  • maintaining audit trails, etc.
  • May also involve external procedures such as
    storing backup tapes, etc.

129
Recovery Test
  • Focus is ability to recover from exceptional
    conditions associated with hardware, software, or
    people.
  • This can involve
  • detecting exceptional conditions,
  • switch-overs to standby systems,
  • recovery of data,
  • maintaining audit trails, etc.
  • May also involve external procedures such as
    storing backup tapes, etc.

130
Recovery Test
  • Focus is ability to recover from exceptional
    conditions associated with hardware, software, or
    people.
  • This can involve
  • detecting exceptional conditions,
  • switch-overs to standby systems,
  • recovery of data,
  • maintaining audit trails, etc.
  • May also involve external procedures such as
    storing backup tapes, etc.

131
Recovery Test
  • Focus is ability to recover from exceptional
    conditions associated with hardware, software, or
    people.
  • This can involve
  • detecting exceptional conditions,
  • switch-overs to standby systems,
  • recovery of data,
  • maintaining audit trails, etc.
  • May also involve external procedures such as
    storing backup tapes, etc.

132
Recovery Test
  • Focus is ability to recover from exceptional
    conditions associated with hardware, software, or
    people.
  • This can involve
  • detecting exceptional conditions,
  • switch-overs to standby systems,
  • recovery of data,
  • maintaining audit trails, etc.
  • May also involve external procedures such as
    storing backup tapes, etc.

133
Reliability Test
  • Requirements may be expressed as
  • the probability of no failure in a specified time
    interval, or as
  • the expected mean time to failure.
  • Appropriate interpretations for failure and time
    are critical.
  • Utilizes Statistical testing based on an
    operational profile.

134
Reliability Test
  • Requirements may be expressed as
  • the probability of no failure in a specified time
    interval, or as
  • the expected mean time to failure.
  • Appropriate interpretations for failure and time
    are critical.
  • Utilizes Statistical testing based on an
    operational profile.

135
Reliability Test
  • Requirements may be expressed as
  • the probability of no failure in a specified time
    interval, or as
  • the expected mean time to failure.
  • Appropriate interpretations for failure and time
    are critical.
  • Utilizes Statistical testing based on an
    operational profile.

136
Integration and Higher Level Testing
Software Testing and Verification Lecture 11
  • Prepared by
  • Stephen M. Thebaut, Ph.D.
  • University of Florida
About PowerShow.com