Testing

1
Testing

• Philip Windridge

2
Contents

• Overview
• Rationale and outline
• Test Plan
• Static Testing
• Program inspection
• Dynamic Testing
• Black Box
• White Box
• Conclusion

3
Overview
4
Introduction

• Why test?
• Functional reasons
• Financial reasons
• Marketing reasons
• How to go about testing?
• What kinds of test?
• How much testing?

5
Testing in General

• Verification and validation fit for purpose
• Static testing and dynamic testing
• Testing can reveal presence of errors
• Testing cannot prove absence of errors!
• Need to balance between effort put in and returns
  for that effort

6
Test Planning

• The Test Plan

7
Test Plan

• Sets out standards to be adhered to
• Describes tests, items for testing, resources/
  constraints
• Major components of the plan
• Testing process
• Requirements traceability
• Tested items
• Testing schedule
• Test recording procedure
• Hardware and software requirements
• Constraints

8
Testing Process

• Unit testing individual components
  (independent)
• Module testing dependent components
  (independent)
• Sub-system testing module collections
  (interface errors)
• System testing integrated sub-systems (validate
  functional/non-functional requirements)
• Acceptance testing (beta testing)

9
V-Model
Requirements specification
System specification
System design
Detailed design
Sub-system integration test plan
System integration test plan
Acceptance test plan
Module and unit code and test
Acceptance test
System integration test
Sub-system integration test
Service
from Sommerville, 1995. Software Engineering,
5th edn, Addison-Wesley p. 451
10
Regression Testing

• Defect testing vs. debugging
• Regression testing applied having fixed some
  code
• Expensive
• Unnecessary?
• Mitigated by reducing/removing dependencies in
  the code

11
Static Testing
12
Progam/Document Inspection

• Review to find defects/issues
• Team of at least four people carry out roles such
  as
• Author, reader, inspector, moderator

13
Inspection Process

• Planning
• Overview (500 source code statements per hour)
• Individual preparation (125 per hour)
• Inspection meeting (90-125 per hour 2 hours
  max)
• Rework
• Follow-up

14
Requires

• Precise specification
• Team familiar with the organisations standards
• Up-to-date version being inspected
• Checklist
• Experience

15
Checklist sample

• Data faults
• Initialise variables before use?
• Are all variables used?
• Upper bound array size
• Control faults
• Conditional statements correct?
• Exhaustive case statements?

16
Program Inspections

• No concern with interactions between errors
• Possible to inspect partial systems
• Can include standards compliance,
  maintainability, etc.

17
Dynamic Testing
18
Example

• Type an integer between 1 and 5 (inclusive) to
  get the string equivalent (1One, 2Two, etc.)

if UserNumber is between 1 and 5 NumberArray()
of String One, Two, Three, Four,
Five output NumberArray(UserNumber)
19
Test Cases

• Set of inputs, execution conditions, expected
  outputs
• Defect test cases
• Cause the system to perform incorrectly

20
Black Box Testing

• Also known as Functional Testing
• References the program specification
• Focus on the behaviour of the system
• Studies the inputs and outputs that these lead to
• Looking to break the program
• Identify inputs with high probability of causing
  anomalies
• Relies on heuristics to determine these

21
Black Box Testing

• Equivalence Partitioning
• Categorisation of possible test cases into
  partitions common characteristics
• Identified from program specification and general
  testing guidelines (common sense)
• Each test case belongs to one partition

22
Example

• Type an integer between 1 and 5 (inclusive) to
  get the string equivalent (1One, 2Two,
  etc.)

Input Condition Output
Integer between 1 and 5 (inclusive) String equivalent
Integer outside 1 and 5 Nothing
Non-integer between 1 and 5 (inclusive) Nothing
23
Black Box Testing

• Boundary Value Analysis
• Values at extremes of input domains cause most
  problems
• Test cases generated from these boundary values

24
White Box Testing

• Also known as Structural Testing
• Requires knowledge of system structure
• Can be used to identify further equivalence
  partitions

25
White Box Testing

• Path Testing
• Test every execution path
• Produce flow graph
• nodes represent decisions
• arcs/edges represent control flow
• An independent path traverses 1 new edge
• Design Test Case for each independent path
• Complexity quickly leads to uncertain coverage

26
White Box Testing

• Path Testing
• Cyclomatic Complexity (CC)
• CC (G) (Number(edges) Number(nodes)) 2

Loop - While
If Then - Else
Case - Of
(diagrams from Sommerville, 1995. Software
Engineering, 5th edn, Addison- Wesley p. 474)
27
Example
if UserNumber is between 1 and 5 NumberArray()
of String One, Two, Three, Four,
Five output NumberArray(UserNumber)
UserNumber between 1 and 5
1
2
3
Two Test Cases (one for each path)
4
CC (4 4) 2 2
28
Dynamic Testing

• Test emergent properties of integrated systems
• Test system outputs meet customer requirements

29
Conclusion

• Overview
• Cant test everything need a plan
• Static Testing
• Program inspection
• Dynamic Testing
• Black Box
• White Box

30
Further Reading

• General Information on Testing http//hissa.ncsl.n
  ist.gov/HHRFdata/Artifacts/ITLdoc/235/sttoc.htm
• Sommerville, 1995. Software Engineering, 5th
  edn, Addison Wesley (Chapters 22-24)
• Program Inspection http//www.hpl.hp.com/hpjournal
  /94dec/dec94a8.pdf
• Cyclomatic Complexity
• McCabe 1976
• http//www.literateprogramming.com/mccabe.pdf