1 / 54

SE 4367Functional Testing

Learning Objectives

- What is functional testing?

- How to perform functional testing?

- How to generate test inputs?

- What are equivalence partitioning, boundary value

testing, domain testing, state testing, and

decision table testing?

What is Functional Testing?

- When test inputs are generated using functional

specifications, we say that we are doing

functional testing.

- Functional testing tests how well a program meets

the functionality requirements.

The Methodology

- The derivation of test inputs is based on

functional specifications.

- Clues are obtained from the specifications.

- Clues lead to test requirements.

- Test requirements lead to test specifications.

- Test specifications are then used to actually

execute the program under test.

Test Methodology

Specifications

Program output is correct

Equivalence Partitioning

- The input domain is usually too large for

exhaustive testing.

- It is therefore partitioned into a finite number

of sub-domains for the selection of test inputs.

- Each sub-domain is known as an equivalence class

and serves as a source of at least one test

input.

Equivalence Partitioning

Four test inputs, one selected from each

sub-domain.

Too many test inputs.

How to partition?

- Inputs to a program provide clues to partitioning.

- Example 1
- Suppose that program P takes an integer X as

input X. - For XT1 and for X0 task T2.

How to partition? (contd.)

- The input domain is prohibitively large because X

can assume a large number of values.

- However, we expect P to behave the same way for

all X

- Similarly, we expect P to perform the same way

for all values of X0.

- We therefore partition the input domain of P into

two sub-domains.

Two sub-domains

All test inputs in the Xconsidered equivalent. The assumption is that if

one test input in this sub-domain reveals an

error in the program, so will the others. This

is true of the test inputs in the X0 sub-domain

also.

Non-overlapping Partitions

- In the previous example, the two equivalence

classes are non-overlapping. In other words the

two sub-domains are disjoint.

- When the sub-domains are disjoint, it is

sufficient to pick one test input from each

equivalence class to test the program.

Non-overlapping Partitions

- An equivalence class is considered covered when

at least one test has been selected from it.

- In partition testing our goal is to cover all

equivalence classes.

Overlapping Partitions

- Example 2
- Suppose that program P takes three integers X, Y

and Z. It is known that - X
- ZY

Overlapping partitions

Overlapping Partition-Test Selection

- In this example, we could select 4 test cases as
- X4, Y7, Z1 satisfies X
- X4, Y2, Z1 satisfies XY
- X1, Y7, Z9 satisfies ZY
- X1, Y7, Z2 satisfies Z

- Thus, we have one test case from each equivalence

class.

Overlapping Partitions-Test Selection

- However, we may also select only 2 test inputs

and satisfy all four equivalence classes - X4, Y7, Z1 satisfies X
- X4, Y2, Z3 satisfies XY and ZY

- Thus, we have reduced the number of test cases

from 4 to 2 while covering each equivalence class.

Partitioning using non-numeric data

- In the previous two examples the inputs were

integers. One can derive equivalence classes for

other types of data also.

- Example 3
- Suppose that program P takes one character X and

one string Y as inputs. P performs task T1 for

all lower case characters and T2 for upper case

characters. Also, it performs task T3 for the

null string and T4 for all other strings.

Partitioning using non-numeric data

lc Lower case character UC Upper case

character null null string.

Non-numeric Data

- Once again we have overlapping partitions.

- We can select only 2 test inputs to cover all

four equivalence classes. These are - X lower case, Y null string
- X upper case, Y not a null string

Guidelines for equivalence partitioning

- Input condition specifies a range create one for

the valid case and two for the invalid cases.

- e.g. for a
- a
- Xb (the invalid cases)

Guidelines (contd.)

- Input condition specifies a value create one for

the valid value and two for incorrect values

(below and above the valid value). This may not

be possible for certain data types, e.g. for

boolean.

- Input condition specifies a member of a set

create one for the valid value and one for the

invalid (not in the set) value.

Sufficiency of Partitions

- In the previous examples we derived equivalence

classes based on the conditions satisfied by the

input data.

- Then we selected just enough tests to cover each

partition.

- Think of the advantages and disadvantages of this

approach!

Boundary Value Analysis (BVA)

- Another way to generate test cases is to look for

boundary values.

- Suppose a program takes an integer X as input.

- In the absence of any information, we assume that

X0 is a boundary. Inputs to the program might

lie on the boundary or on either side of the

boundary.

BVA (contd.)

- This leads to 3 test inputs
- X0, X-20, and X14.

Note that the values -20 and 14 are on either

side of the boundary and are chosen arbitrarily.

- Notice that using BVA we get 3 equivalence

classes. One of these three classes contains only

one value (X0), the other two are large!

BVA (contd.)

- Now suppose that a program takes two integers X

and Y and that x1

BVA (contd.)

- In this case the four sides of the rectangle

represent the boundary.

- The heuristic for test selection in this case is
- Select one test at each corner (1, 2, 3, 4).
- Select one test just outside of each of the four

sides of the boundary (5, 6, 7, 8)

BVA (contd.)

- Select one test just inside of each of the four

sides of the boundary (10, 11, 12, 13).

- Select one test case inside of the bounded region

(9).

- Select one test case outside of the bounded

region (14).

- How many equivalence classes do we get?

BVA (contd.)

- In the previous examples we considered only

numeric data.

- BVA can be done on any type of data.

- For example, suppose that a program takes a

string S and an integer X as inputs. The

constraints on inputs are - length(S)

- Can you derive the test cases using BVA?

BVA Applied to Output Variables

- Just as we applied BVA to input data, we can

apply it to output data.

- Doing so gives us equivalence classes for the

output domain.

- We then try to find test inputs that will cover

each output equivalence class.

Finite State Machines (FSMs)

- A state machine is an abstract representation of

actions taken by a program or anything else that

functions!

- It is specified as a quintuple
- A a finite input alphabet
- Q a finite set of states
- q0 initial state which is a member of Q.

FSMs (contd.)

- T state transitions which is a mapping
- Q x A-- Q
- F A finite set of final states, F is a subset of

Q.

- Example Here is a finite state machine that

recognizes integers ending with a carriage return

character.

- A0,1,2,3,4,5,6,7,8,9, CR
- Qq0,q1,q2
- q0 initial state

FSMs (contd.)

- T ((q0,d),q1),(q1,d),q1), (q1,CR),q2)
- F q2

- A state diagram is an easier to understand

specification of a state machine. For the above

machine, the state diagram appears on the next

slide.

State diagram

d denotes a digit

State Diagram-Actions

x/y x is an element of the alphabet and y is an

action.

d/iid10j jj1

d/ i d j 1

CR/output i

i is initialized to d and j to 1 when the machine

moves from state q0 to q1. i is incremented by

10jd and j by 1 when the machine moves from q1

to q1. The current value of i is output when a CR

is encountered.

Can you describe what this machine computes?

State Machine Languages

- Each state machine recognizes a language.

- The language recognized by a state machine is the

set S of all strings such that - when any string s in S is input to the state

machine the machine goes through a sequence of

transitions and ends up in the final state after

having scanned all elements of s.

State diagram-Errors

d/iid10j jj1

d/ i d j 1

CR/output I

CR/output error

reset

q4

q4 has been added to the set of states. It

represents an error state. Notice that reset is a

new member added to the alphabet.

State Diagram-Code

- A state diagram can be transformed into a program

using case analysis. Let us look at a C program

fragment that embodies logic represented by the

previous state diagram.

- There is one function for each action.

- digit is assumed to be provided by the lexical

analyzer.

Program for integer state machine

/ state is global, with values q0, q1, q2. i is

also global./

void event_digit()

- case q0
- idigit j 1 / perform action. /
- stateq1 / set next state. /
- break / event digit is done. /
- case q1
- ii10jdigit j / Add the next digit. /
- stateq1
- break
- /complete the program. /

switch (state)

Checking State Diagrams

- Unreachable state One that cannot be reached

from q0 using any sequence of transitions.

- Dead state One that cannot be left once it is

reached.

Test Requirements

- Every state must be reached at least once, Obtain

100 state coverage.

- Every transition must be exercised at least once.

Obtain 100 transition coverage.

Example Test Requirements

- For the integer state machine
- state machine transitions
- event digit in state q0
- event CR in state q0
- event digit in state q1
- event CR in state q1
- event reset in state q4

More testing of state machines?

- Yes, it is possible!

- When we learn about path coverage we will discuss

how more test requirements can be derived from a

state diagram.

Test Specifications

- As before, test specifications are derived from

test requirements.

- In the absence of dead states, all states and

transitions might be reachable by one test.

- It is advisable not to test the entire machine

with one test case.

Decision Tables

- Requirements of certain programs are specified by

decision tables.

- A decision table is useful when specifying

complex decision logic

Decision Tables

- A decision table has two parts
- condition part
- action part

- The two together specify under what condition

will an action be performed.

Decision Table-Nomenclature

- C denotes a condition
- A denotes an action
- Y denotes true
- Ndenotes false
- X denotes action to be taken.
- Blank in condition denotes dont care
- Blank in action denotes do not take the action

Bank Example

- Consider a bank software responsible for debiting

from an account. The relevant conditions and

actions are - C1 The account number is correct
- C2 The signature matches
- C3 There is enough money in the account
- A1 Give money
- A2 Give statement indicating insufficient funds
- A3 Call vigilance to check for fraud!

Decision tables

Example (contd.)

- A1 is to be performed when C1, C2, and C3 are

true.

- A2 is to be performed when C1 and C2 are true and

C3 is false.

- A3 is to be performed when C1 is true and C2 is

false.

Default Rules

- Are all possible combinations of conditions

covered?

- No! Which ones are not covered?

- We need a default action for the uncovered

combinations. A default action could be an error

report or a reset.

Example-Test Requirements

- Each column is a rule and corresponds to at

least one test requirement.

- If there are n columns then there are at least n

test requirements.

- What is the maximum number of test requirements?

Example-Test Specifications

- For each test requirement find a set of input

values of variables such that the selected rule

is satisfied.

- When this test is input to the program the output

must correspond to the action specified in the

decision table.

- Should the testing depend on the order in which

the conditions are evaluated?

Summary-continued

Number of test cases

high

low

Sophistication

Boundary Value

Equivalence Class

Decision Table

Summary-continued

Effort to Identify Test Cases

high

low

Sophistication

Boundary Value

Equivalence Class

Decision Table