Testing

1
Testing

• Team exercise
• Have each team member contribute answers
• Do you test your code? If no, why not? If yes
• When?
• How?
• How often?
• What is your teams test plan? You have one,
  right? -)

2
Some TLAs(Three Letter Acronyms)

• SQA
• Software Quality Assurance
• many different activities (including good
  communication, good design, etc)
• VV
• Verification and Validation
• Part of SQA
• Verification are we building the product right?
• Validation are we building the right product?
• Fault detection
• Reviews Walkthroughs and Inspections
• Testing
• Debugging
• what you do once youve established that there is
  a bug

3
Types of testing

• Unit testing
• Integration testing
• Functional testing
• Acceptance testing
• Pilot testing
• Performance testing
• Installation testing
• Usability testing

4
Who does the testing?

• Developers
• Testers
• Customers

5
Unit tests

• Written by developers
• Tests individual components
• In TDD (another TLA) unit tests are written
  before code
• Automated test frameworks
• JUnit
• CPPUnit
• etc.

6
Integration testing

• Testing of components when brought together
• Different strategies
• big bang test all components together at once
• bottom-up test layer by layer, from bottom up
• top-down test layer by layer, from top (UI) down
• Unintegrated components are simulated by stubs
  during integration testing

7
Functional testing

• A whole-system test
• Also called requirements testing
• Uncovers differences between use-case model and
  observed system behavior

8
Acceptance testing

• Benchmark testing
• testing against a specific set of test cases
• Competitor testing
• evaluation wrt competitor product
• Shadow testing
• evaluation wrt legacy system

9
Pilot testing

• Field testing
• alpha test system is tested by users in
  development environment
• beta test system is tested by users in the
  deployment environment

10
Performance testing

• Stress testing simultaneous requests
• Volume large amounts of data
• Security testing is it secure?
• Timing tests are timing constraints violated?
• Recover tests can system recover from errors?

11
Installation testing

• Testing of system in deployment environment
  (after beta test)

12
Usability testing

• Is user interface acceptable to users?

13
Random testing

• Series of studies done by B. Miller and
  colleagues
• First study Unix utilities (50-85 utilities,
  depending on OS version, 6 versions)
• Second study Unix utilities and X-Windows
  applications
• Third study Windows NT applications

14
Results

• How many utilities/applications crashed or hung
  on random input?

15
Results

• How many utilities/applications crashed or hung
  on random input?
• first study 25-33
• second study
• utilities 40
• applications 25

16
Window NT study

• Similar study for Windows NT, using
  software/utilities from different vendors
  (Microsoft, Adobe, Command Software Systems,
  Qualcomm, Aladdin, Ghostgum, Free Software
  Foundation, Sun, Netscape, Jasc, Van Dyke, MIT
  Kerberos Group, Nullsoft, Ipswitch, Metrowerks)
• 23 applications

17
Results

• How many utilities/applications crashed or hung
  on random input?

18
Results

• How many utilities/applications crashed or hung
  on random input?
• 21 of applications crashed (valid keyboard/mouse
  events)
• Additional 24 hung (valid keyboard/mouse events)
• 100 crashed or hung with random input streams

19
Blackbox vs. whitebox testing

• Blackbox testing considers only input-output
  behavior of a component.
• Blackbox testing tests functionality.
• Whitebox testing considers the internal structure
  of components.
• Whitebox testing tests structural and dynamic
  aspects of the component
• Bruegge Dutoit, Object-Oriented Software
  Engieering, p. 448

20
Equivalence testing

• Blackbox technique
• Partition inputs into equivalence classes.
• A representative from each equivalence class is
  used for each test.
• Assumption all members of equivalence class will
  behave the same.
• Bruegge Dutoit, Object-Oriented Software
  Engieering, p. 453

21
Example

• Calendar
• Month equivalence classes
• months with 30 days
• months with 31 days
• months with 28 or 29 days
• Invalid inputs monthlt1, monthgt12
• Year equivalence classes
• non-leap years
• leap years
• Invalid inputs year lt 0
• If method depends on both month and year, we have
  236 equivalence classes.
• Bruegge Dutoit, Object-Oriented Software
  Engieering, p. 454

22
Example

• Calendar
• Month equivalence classes
• months with 30 days
• months with 31 days
• months with 28 or 29 days
• Invalid inputs monthlt1, monthgt12
• Year equivalence classes
• non-leap years
• leap years
• Invalid inputs year lt 0
• If method depends on both month and year, we have
  236 equivalence classes (My question why not
  4312?)
• Bruegge Dutoit, Object-Oriented Software
  Engieering, p. 454

23
Boundary testing

• Test boundaries of equivalence classes
• In example, February yields many boundary cases.
  Leap years are years divisible by 4, unless
  divisible by 100, unless divisible by 400.
• leap years divisible by 400 (e.g. Feb 2, 2000)
• non-leap years divisible by 100 (e.g. Feb 2,
  1900)
• Bruegge Dutoit, Object-Oriented Software
  Engieering, p. 454

24
Path testing

• Whitebox testing
• Ensure that all paths through the code are
  exercised
• Start with a flow graph (a graph whose nodes are
  executable blocks of code and whose edges
  represent flow of control)
• Test cases must cover each edge in graph
• Minimum number of test cases required is the
  number of independent paths in graph, which is
  edges - nodes 2
• Can work off of activity diagram
• Bruegge Dutoit, Object-Oriented Software
  Engieering, p. 456

25
State-based testing

• Whitebox testing
• Ensures that state transitions of system work as
  expected
• Works off of statechart diagram
• Bruegge Dutoit, Object-Oriented Software
  Engieering, p. 459

26
Unit Testing Example

• In this exercise we will develop a simple class
  (RationalNumber) in a TDD fashion.

27
Properties

• A rational number is of the form n/d, where n is
  an integer, and d is an integer except zero.
• Rational numbers can be added, multiplied, tested
  for equality.
• Each rational number has an additive inverse (its
  negative) and a multiplicative inverse (its
  reciprocal).

28
Rational class

• Will provide public constructor
• public Rational(int numerator, int denominator)
• Will provide public methods
• public Rational add(Rational r)
• public Rational negative()
• public Rational multiply(Rational r)
• public Rational reciprocal()
• public boolean equals(Object obj)

29
Exercise structure

• Break into 6 groups
• Each group identifies tests to perform on their
  operation (construction, addition, negative,
  multiplication, reciprocal, equals)
• I will write the tests, then we will write the
  code