White-Box Testing Techniques I - PowerPoint PPT Presentation

Loading...

PPT – White-Box Testing Techniques I PowerPoint presentation | free to view - id: 6eeffa-Mzk5M



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

White-Box Testing Techniques I

Description:

White-Box Testing Techniques I Software Testing and Verification Lecture 7 Prepared by Stephen M. Thebaut, Ph.D. University of Florida – PowerPoint PPT presentation

Number of Views:66
Avg rating:3.0/5.0
Slides: 105
Provided by: mcat150
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: White-Box Testing Techniques I


1
White-Box Testing Techniques I
Software Testing and Verification Lecture 7
  • Prepared by
  • Stephen M. Thebaut, Ph.D.
  • University of Florida

2
Definition of White-Box Testing
  • Testing based on analysis of internal logic
    (design, code, etc.). (But expected results still
    come from requirements.)
  • Also know as structural testing.
  • White-box testing concerns techniques for
    designing tests it is not a level of testing.
  • White-box testing techniques apply primarily to
    lower levels of testing (e.g., unit and
    component).

3
Definition of White-Box Testing
  • Testing based on analysis of internal logic
    (design, code, etc.). (But expected results still
    come from requirements.)
  • Also know as structural testing.
  • White-box testing concerns techniques for
    designing tests it is not a level of testing.
  • White-box testing techniques apply primarily to
    lower levels of testing (e.g., unit and
    component).

4
Definition of White-Box Testing
  • Testing based on analysis of internal logic
    (design, code, etc.). (But expected results still
    come from requirements.)
  • Also know as structural testing.
  • White-box testing concerns techniques for
    designing tests it is not a level of testing.
  • White-box testing techniques apply primarily to
    lower levels of testing (e.g., unit and
    component).

5
Definition of White-Box Testing
  • Testing based on analysis of internal logic
    (design, code, etc.). (But expected results still
    come from requirements.)
  • Also know as structural testing.
  • White-box testing concerns techniques for
    designing tests it is not a level of testing.
  • White-box testing techniques apply primarily to
    lower levels of testing (e.g., unit and
    component).

6
White-Box Testing Topics
  • Logic coverage (lecture I)
  • Dataflow coverage (lecture II)
  • Path conditions and symbolic execution (lecture
    III)
  • Other white-box testing strategies (e.g.,
    fault-based testing) (lecture IV)

7
Types of Logic Coverage
  • Statement each statement executed at least once
  • Branch each branch traversed (and every entry
    point taken) at least once
  • Condition each condition True at least once and
    False at least once
  • Branch/Condition both Branch and Condition
    coverage achieved

8
Types of Logic Coverage
  • Statement each statement executed at least once
  • Branch each branch traversed (and every entry
    point taken) at least once
  • Condition each condition True at least once and
    False at least once
  • Branch/Condition both Branch and Condition
    coverage achieved

9
Types of Logic Coverage
  • Statement each statement executed at least once
  • Branch each branch traversed (and every entry
    point taken) at least once
  • Condition each condition True at least once and
    False at least once
  • Branch/Condition both Branch and Condition
    coverage achieved

10
Types of Logic Coverage
  • Statement each statement executed at least once
  • Branch each branch traversed (and every entry
    point taken) at least once
  • Condition each condition True at least once and
    False at least once
  • Branch/Condition both Branch and Condition
    coverage achieved

(contd)
11
Types of Logic Coverage (contd)
  • Compound Condition all combinations of
    condition values at every branch statement
    covered (and every entry point taken)
  • Path all program paths traversed at least once

12
Types of Logic Coverage (contd)
  • Compound Condition all combinations of
    condition values at every branch statement
    covered (and every entry point taken)
  • Path all program paths traversed at least once

13
Pseudocode and Control Flow Graphs
  • input(Y)
  • if (Ylt0) then
  • Y -Y
  • end_if
  • while (Ygt0) do
  • input(X)
  • Y Y-1
  • end_while

nodes
edges
14
Statement Coverage
  • Statement Coverage requires that each statement
    will have been executed at least once.
  • Simplest form of logic coverage.
  • Also known as Node Coverage.
  • What is the minimum number of test cases required
    to achieve statement coverage for the program
    segment given below?

15
Statement Coverage
  • Statement Coverage requires that each statement
    will have been executed at least once.
  • Simplest form of logic coverage.
  • Also known as Node Coverage.
  • What is the minimum number of test cases required
    to achieve statement coverage for the program
    segment given below?

16
Statement Coverage
  • Statement Coverage requires that each statement
    will have been executed at least once.
  • Simplest form of logic coverage.
  • Also known as Node Coverage.
  • What is the minimum number of test cases required
    to achieve statement coverage for the program
    segment given below?

17
Statement Coverage
  • Statement Coverage requires that each statement
    will have been executed at least once.
  • Simplest form of logic coverage.
  • Also known as Node Coverage.
  • What is the minimum number of test cases required
    to achieve statement coverage for the program
    segment given below?

18
How many test cases required for Statement
Coverage?
  • input(Y)
  • if (Ylt0) then
  • Y -Y
  • end_if
  • while (Ygt0) do
  • input(X)
  • Y Y-1
  • end_while

19
How many test cases required for Statement
Coverage?
  • input(Y)
  • if (Ylt0) then
  • Y -Y
  • end_if
  • while (Ygt0) do
  • input(X)
  • Y Y-1
  • end_while

Claim only 1 test case is required.
20
How many test cases required for Statement
Coverage?
  • input(Y)
  • if (Ylt0) then
  • Y -Y
  • end_if
  • while (Ygt0) do
  • input(X)
  • Y Y-1
  • end_while

Claim only 1 test case is required.
21
How many test cases required for Statement
Coverage?
  • input(Y)
  • if (Ylt0) then
  • Y -Y
  • end_if
  • while (Ygt0) do
  • input(X)
  • Y Y-1
  • end_while

Claim only 1 test case is required.
22
How many test cases required for Statement
Coverage?
  • input(Y)
  • if (Ylt0) then
  • Y -Y
  • end_if
  • while (Ygt0) do
  • input(X)
  • Y Y-1
  • end_while

Claim only 1 test case is required.
23
How many test cases required for Statement
Coverage?
  • input(Y)
  • if (Ylt0) then
  • Y -Y
  • end_if
  • while (Ygt0) do
  • input(X)
  • Y Y-1
  • end_while

Claim only 1 test case is required.
24
How many test cases required for Statement
Coverage?
  • input(Y)
  • if (Ylt0) then
  • Y -Y
  • end_if
  • while (Ygt0) do
  • input(X)
  • Y Y-1
  • end_while

Claim only 1 test case is required.
25
How many test cases required for Statement
Coverage?
  • input(Y)
  • if (Ylt0) then
  • Y -Y
  • end_if
  • while (Ygt0) do
  • input(X)
  • Y Y-1
  • end_while

Claim only 1 test case is required.
26
Branch Coverage
  • Branch Coverage requires that each branch will
    have been traversed, and that every program entry
    point will have been taken, at least once.
  • Also known as Edge Coverage.

27
Branch Coverage (contd)
  • Why and that every program entry point will
    have been taken, at least once?

28
Branch Coverage (contd)
  • Why and that every program entry point will
    have been taken, at least once?

29
Branch Coverage (contd)
  • Why and that every program entry point will
    have been taken, at least once?

30
Branch Coverage (contd)
  • What is the relationship between Statement and
    Branch Coverage?

31
Branch Coverage (contd)
  • What is the relationship between Statement and
    Branch Coverage?
  • Possible relationships
  • None.
  • Statement Coverage subsumes Branch Coverage
    (statement gt branch).
  • Branch Coverage subsumes Statement Coverage
    (branch gt statement).
  • Both (2) and (3) (i.e., they are equivalent)

32
Branch Coverage (contd)
  • What is the relationship between Statement and
    Branch Coverage?
  • Possible relationships
  • None.
  • Statement Coverage subsumes Branch Coverage
    (statement gt branch).
  • Branch Coverage subsumes Statement Coverage
    (branch gt statement).
  • Both (2) and (3) (i.e., they are equivalent)

33
Branch Coverage (contd)
  • What is the relationship between Statement and
    Branch Coverage?
  • Possible relationships
  • None.
  • Statement Coverage subsumes Branch Coverage
    (statement gt branch).
  • Branch Coverage subsumes Statement Coverage
    (branch gt statement).
  • Both (2) and (3) (i.e., they are equivalent)

34
Branch Coverage (contd)
  • What is the relationship between Statement and
    Branch Coverage?
  • Possible relationships
  • None.
  • Statement Coverage subsumes Branch Coverage
    (statement gt branch).
  • Branch Coverage subsumes Statement Coverage
    (branch gt statement).
  • Both (2) and (3) (i.e., they are equivalent)

35
Branch Coverage (contd)
  • What is the relationship between Statement and
    Branch Coverage?
  • Possible relationships
  • None.
  • Statement Coverage subsumes Branch Coverage
    (statement gt branch).
  • Branch Coverage subsumes Statement Coverage
    (branch gt statement).
  • Both (2) and (3) (i.e., they are equivalent)

36
Does statement gt branch ???
Min. number of cases required for Statement
Coverage? Min. number of cases required for
Branch Coverage?
37
Does statement gt branch ???
Min. number of cases required for Statement
Coverage? Min. number of cases required for
Branch Coverage?
1
38
Does statement gt branch ???
Min. number of cases required for Statement
Coverage? Min. number of cases required for
Branch Coverage?
1
2
39
Does statement gt branch ???
Min. number of cases required for Statement
Coverage? Min. number of cases required for
Branch Coverage?
1
2
  • Therefore, Statement Coverage does NOT subsume
    Branch Coverage.

40
Does branch gt statement ???
  • Normally, YES

41
Does branch gt statement ???
  • Normally, YES in the absence of DEAD CODE.

DEAD CODE is not reachable via any executable
program path.
42
Does branch gt statement ???
  • If a program has "dead (i.e., unreachable) code",
    then "statement coverage" is unachievable. (We
    would need to modify the program in order to
    bring the dead code back to life.)
  • Bottom line we will always assume the nominal
    case of no dead code" unless explicitly stated
    otherwise. Under this assumption, Branch
    Coverage does indeed subsume Statement Coverage.

43
Condition Coverage
  • A branch predicate may have more than one
    condition.

input(X,Y) if (Ylt0) or (X0) then Y
-Y end_if while (Ygt0) and (not EOF)
do input(X) Y Y-1 end_while
44
Condition Coverage (contd)
  • Condition Coverage requires that each condition
    will have been True at least once and False at
    least once.
  • What is the relationship between Branch and
    Condition Coverage?

45
Condition Coverage (contd)
  • Condition Coverage requires that each condition
    will have been True at least once and False at
    least once.
  • What is the relationship between Branch and
    Condition Coverage?

46
Condition Coverage (contd)
  • if A or B then
  • s1
  • else
  • s2
  • end_if_then_else

A B Branch
test 1 T F ?
test 2 F F ?
47
Condition Coverage (contd)
  • if A or B then
  • s1
  • else
  • s2
  • end_if_then_else

A B Branch
test 1 T F true
test 2 F F ?
48
Condition Coverage (contd)
  • if A or B then
  • s1
  • else
  • s2
  • end_if_then_else

A B Branch
test 1 T F true
test 2 F F false
49
Condition Coverage (contd)
  • if A or B then
  • s1
  • else
  • s2
  • end_if_then_else

A B Branch
test 1 T F true
test 2 F F false
?
?
?
50
Condition Coverage (contd)
  • if A or B then
  • s1
  • else
  • s2
  • end_if_then_else

A B Branch
test 1 T F true
test 2 F F false
?
?
?
? Branch Coverage gt Condition Coverage
51
Condition Coverage (contd)
  • if A or B then
  • s1
  • else
  • s2
  • end_if_then_else

A B Branch
test 3 T F ?
test 4 F T ?
52
Condition Coverage (contd)
  • if A or B then
  • s1
  • else
  • s2
  • end_if_then_else

A B Branch
test 3 T F true
test 4 F T ?
53
Condition Coverage (contd)
  • if A or B then
  • s1
  • else
  • s2
  • end_if_then_else

A B Branch
test 3 T F true
test 4 F T true
54
Condition Coverage (contd)
  • if A or B then
  • s1
  • else
  • s2
  • end_if_then_else

A B Branch
test 3 T F true
test 4 F T true
?
?
?
55
Condition Coverage (contd)
  • if A or B then
  • s1
  • else
  • s2
  • end_if_then_else

A B Branch
test 3 T F true
test 4 F T true
?
?
?
? Condition Coverage gt Branch Coverage
56
Branch/Condition Coverage
  • Branch/Condition Coverage requires that both
    Branch AND Condition Coverage will have been
    achieved.
  • Therefore, Branch/Condition Coverage subsumes
    both Branch Coverage and Condition Coverage.

57
Branch/Condition Coverage
  • Branch/Condition Coverage requires that both
    Branch AND Condition Coverage will have been
    achieved.
  • Therefore, Branch/Condition Coverage subsumes
    both Branch Coverage and Condition Coverage.

58
Compound Condition Coverage
  • What if the compiler generates code that masks
    the evaluation of conditions?
  • That is, suppose
  • if (A) or (y/x5) then...
  • is compiled in such a way that if A is true,
    y/x5 will not be evaluated.

59
Compound Condition Coverage (contd)
  • Compound Condition Coverage requires that all
    combinations of condition values at every branch
    statement will have been covered, and that every
    entry point will have been taken, at least once.
  • Also know as Multiple Condition Coverage.
  • Subsumes Branch/Condition Coverage, regardless of
    the order in which conditions are evaluated.

60
Compound Condition Coverage (contd)
  • Compound Condition Coverage requires that all
    combinations of condition values at every branch
    statement will have been covered, and that every
    entry point will have been taken, at least once.
  • Also know as Multiple Condition Coverage.
  • Subsumes Branch/Condition Coverage, regardless of
    the order in which conditions are evaluated.

61
Compound Condition Coverage (contd)
  • Compound Condition Coverage requires that all
    combinations of condition values at every branch
    statement will have been covered, and that every
    entry point will have been taken, at least once.
  • Also know as Multiple Condition Coverage.
  • Subsumes Branch/Condition Coverage, regardless of
    the order in which conditions are evaluated.

62
Compound Condition Coverage (contd)
Combinations of condition values TT, TF, FT, FF
input(X,Y) if (Ylt0) or (X0) then Y
-Y end_if
63
Compound Condition Coverage (contd)
  • In general, how many different combinations of
    condition values must be considered when a branch
    predicate has N conditions?

64
Compound Condition Coverage (contd)
  • In general, how many different combinations of
    condition values must be considered when a branch
    predicate has N conditions?
  • 2

N
65
Path Coverage
  • Path Coverage requires that all program paths
    will have been traversed at least once.
  • Often described as the strongest form of logic
    coverage. (Is it stronger than Compound
    Condition Coverage?)
  • Path Coverage is usually impossible when loops
    are present. (How many test cases would be
    required to cover all paths in the example below?)

66
Path Coverage
  • Path Coverage requires that all program paths
    will have been traversed at least once.
  • Often described as the strongest form of logic
    coverage. (Is it stronger than Compound
    Condition Coverage?)
  • Path Coverage is usually impossible when loops
    are present. (How many test cases would be
    required to cover all paths in the example below?)

67
Path Coverage
  • Path Coverage requires that all program paths
    will have been traversed at least once.
  • Often described as the strongest form of logic
    coverage. (Is it stronger than Compound
    Condition Coverage?)
  • Path Coverage is usually impossible when loops
    are present. (How many test cases would be
    required to cover all paths in the example below?)

68
Path Coverage (contd)
  • for I 1 to 30 do
  • input(X,Y)
  • if (Ylt0) then
  • if (Xlt0) then
  • Y -X
  • else
  • Y-Y
  • end_if_else
  • else
  • Y XY
  • end_if_else
  • end_for_do

repeat 29 times
69
Path Coverage (contd)
3 paths
3 X 3 9 paths
3 paths
70
Path Coverage (contd)
repeat 29 times
3 X 3 XX 3 3 paths
30
71
Path Coverage (contd)
  • Various strategies have been developed for
    identifying useful subsets of paths for testing
    when Path Coverage is impractical
  • Loop Coverage,
  • Basis Paths Coverage, and
  • Dataflow Coverage (Lecture 8).

72
Loop Coverage
  • Loop Coverage requires that the body of loops be
    executed 0, 1, 2, t, max, and max1 times, where
    possible.

73
Loop Coverage
  • Loop Coverage requires that the body of loops be
    executed 0, 1, 2, t, max, and max1 times, where
    possible.
  • Rationale
  • 0 Is some action taken in the body that must
    also be taken when the body is not executed?
  • 1 Check lower bound on number of times body may
    be executed.

74
Loop Coverage
  • Loop Coverage requires that the body of loops be
    executed 0, 1, 2, t, max, and max1 times, where
    possible.
  • Rationale
  • 0 Is some action taken in the body that must
    also be taken when the body is not executed?
  • 1 Check lower bound on number of times body may
    be executed.

75
Loop Coverage
  • Loop Coverage requires that the body of loops be
    executed 0, 1, 2, t, max, and max1 times, where
    possible.
  • Rationale
  • 0 Is some action taken in the body that must
    also be taken when the body is not executed?
  • 1 Check lower bound on number of times body may
    be executed.

76
Loop Coverage (contd)
  • Rationale (contd)
  • 2 Check loop re-initialization.
  • t Check typical number of iterations.
  • max Check upper (valid) bound on number of
    times body may be executed.
  • max1 If the maximum can be exceeded, what
    behavior results?

77
Loop Coverage (contd)
  • Rationale (contd)
  • 2 Check loop re-initialization.
  • t Check typical number of iterations.
  • max Check upper (valid) bound on number of
    times body may be executed.
  • max1 If the maximum can be exceeded, what
    behavior results?

78
Loop Coverage (contd)
  • Rationale (contd)
  • 2 Check loop re-initialization.
  • t Check typical number of iterations.
  • max Check upper (valid) bound on number of
    times body may be executed.
  • max1 If the maximum can be exceeded, what
    behavior results?

79
Loop Coverage (contd)
  • Rationale (contd)
  • 2 Check loop re-initialization.
  • t Check typical number of iterations.
  • max Check upper (valid) bound on number of
    times body may be executed.
  • max1 If the maximum can be exceeded, what
    behavior results?

80
Basis Paths Coverage
  • A coverage criterion associated with McCabes
    Structured Testing.
  • Based on idea of identifying a spanning (i.e.,
    basis) set of paths for a programs path space.
  • The number, C, of such paths is equal to the
    number of (2-way) branch statements in the
    program 1. (This is also the number of enclosed
    regions in the program graph 1.)

81
Basis Paths Coverage
  • A coverage criterion associated with McCabes
    Structured Testing.
  • Based on idea of identifying a spanning (i.e.,
    basis) set of paths for a programs path space.
  • The number, C, of such paths is equal to the
    number of (2-way) branch statements in the
    program 1. (This is also the number of enclosed
    regions in the program graph 1.)

82
Spanning vectors in 3n-space
A
X-coordinate
A (x, y, z) x (1, 0, 0) y (0, 1, 0) z
(0, 0, 1)
83
Basis Paths Coverage
  • A coverage criterion associated with McCabes
    Structured Testing.
  • Based on idea of identifying a spanning (i.e.,
    basis) set of paths for a programs path space.
  • The number, C, of such paths is equal to the
    number of (2-way) branch statements in the
    program 1. (This is also the number of enclosed
    regions in the program graph 1.)

84
Basis Paths Coverage (contd)
  • C is what McCabe calls the Cyclomatic Complexity
    of a program.
  • Any C distinct, simple program paths that provide
    branch coverage also form a basis set of paths.
    (In a simple program path, while loop bodies are
    executed at most once and repeat-until loop
    bodies are executed at most twice.)

85
Basis Paths Coverage (contd)
  • C is what McCabe calls the Cyclomatic Complexity
    of a program.
  • Any C distinct, simple program paths that provide
    branch coverage also form a basis set of paths.
    (In a simple program path, while loop bodies are
    executed at most once and repeat-until loop
    bodies are executed at most twice.)

86
Example 1
  • if a then s1
  • else if b then s2
  • else if c then s3
  • else s4
  • end_if_then_else
  • end_if_then_else
  • end_if_then_else

Paths ___ Basis Paths ___ Cases for branch
coverage ___
87
Example 1
  • if a then s1
  • else if b then s2
  • else if c then s3
  • else s4
  • end_if_then_else
  • end_if_then_else
  • end_if_then_else

4
Paths ___ Basis Paths ___ Cases for branch
coverage ___
88
Example 1
  • if a then s1
  • else if b then s2
  • else if c then s3
  • else s4
  • end_if_then_else
  • end_if_then_else
  • end_if_then_else

1
2
3
4
Paths ___ Basis Paths ___ Cases for branch
coverage ___
89
Example 1
  • if a then s1
  • else if b then s2
  • else if c then s3
  • else s4
  • end_if_then_else
  • end_if_then_else
  • end_if_then_else

1
2
3
4
4
Paths ___ Basis Paths ___ Cases for branch
coverage ___
90
Example 1
  • if a then s1
  • else if b then s2
  • else if c then s3
  • else s4
  • end_if_then_else
  • end_if_then_else
  • end_if_then_else

1
2
3
4
4
4
Paths ___ Basis Paths ___ Cases for branch
coverage ___
91
Example 2
  • if a then
  • s1
  • end_if_then
  • if b then
  • s2
  • end_if_then
  • if c then
  • s3
  • end_if_then

Paths ___ Basis Paths ___ Cases for branch
coverage ___
92
Example 2
  • if a then
  • s1
  • end_if_then
  • if b then
  • s2
  • end_if_then
  • if c then
  • s3
  • end_if_then

8
Paths ___ Basis Paths ___ Cases for branch
coverage ___
93
Example 2
  • if a then
  • s1
  • end_if_then
  • if b then
  • s2
  • end_if_then
  • if c then
  • s3
  • end_if_then

1
2
3
8
Paths ___ Basis Paths ___ Cases for branch
coverage ___
94
Example 2
  • if a then
  • s1
  • end_if_then
  • if b then
  • s2
  • end_if_then
  • if c then
  • s3
  • end_if_then

1
2
3
8
4
Paths ___ Basis Paths ___ Cases for branch
coverage ___
95
Example 2
  • if a then
  • s1
  • end_if_then
  • if b then
  • s2
  • end_if_then
  • if c then
  • s3
  • end_if_then

1
2
3
8
4
2
Paths ___ Basis Paths ___ Cases for branch
coverage ___
96
Example 3
while a do if b then s1 else s2 end_if_then_els
e end_while
Paths ___ Basis Paths ___ Cases for branch
coverage ___
97
Example 3
while a do if b then s1 else s2 end_if_then_els
e end_while
?
Paths ___ Basis Paths ___ Cases for branch
coverage ___
98
Example 3
while a do if b then s1 else s2 end_if_then_els
e end_while
1
2
?
Paths ___ Basis Paths ___ Cases for branch
coverage ___
99
Example 3
while a do if b then s1 else s2 end_if_then_els
e end_while
1
2
?
3
Paths ___ Basis Paths ___ Cases for branch
coverage ___
100
Example 3
while a do if b then s1 else s2 end_if_then_els
e end_while
1
2
?
3
1
Paths ___ Basis Paths ___ Cases for branch
coverage ___
101
In General
Number of test cases required for branch coverage
Number of program Paths
Number of Basis Paths
?
?
Basis Paths Coverage
Path Coverage
Branch Coverage
gt
gt
102
Exercise
  • Prove that Path and Compound Condition Coverage
    are independent.
  • (Hint consider the proof that Branch and
    Condition Coverage are independent.)

103
Coming Up Next
  • In the next lecture we consider a family of path
    selection criteria based on the idea that program
    paths along which variables are defined and then
    used should be covered.
  • The strategy is popularly known as Dataflow
    Coverage.

104
White-Box Testing Techniques I
Software Testing and Verification Lecture 7
  • Prepared by
  • Stephen M. Thebaut, Ph.D.
  • University of Florida
About PowerShow.com