Testing and Debugging

1
Chapter 8

• Testing and Debugging

2
Chapter Goals

• To learn how to carry out unit tests
• To understand the principles of test case
  selection and evaluation
• To learn how to use logging and assertions
• To become familiar with the debugger
• To learn strategies for effective debugging

3
Unit Test

• Test classes in isolation, outside the program in
  which they are used
• Test one class at a time
• Supply test inputs through test harness
• Test harness program that feeds test inputs to
  a class

4
Root Approximator

• Example program to illustrate testing square
  root approximator
• Algorithm known to the ancient Greeks (Heron)
• Task to compute the square root of a
• Given a guess x (ok to start with 1) Actual
  square root lies between x and a/x

5

• Take midpoint (x a/x) / 2 as a better guess
  Method converges rapidly. Square root of 100
• Guess 1 50.5 Guess 2 26.24009900990099
  Guess 3 15.025530119986813 Guess 4
  10.840434673026925 Guess 5 10.032578510960604
  Guess 6 10.000052895642693 Guess 7
  10.000000000139897 Guess 8 10.0

6
RootApproximator.java

• /
• Computes approximations to the square root of a
  number, using Heron's algorithm
• /
• public class RootApproximator
• private double a // number whose
  //square root is computed
• private double xnew // current guess
• private double xold // old guess

7

• /
• Constructs a root approximator for a given
  number
• _at_param aNumber the number from which to extract
  the square root
• (Precondition aNumber gt 0)
• /
• public RootApproximator(double aNumber)
• a aNumber
• xold 1
• xnew a

8

• /
• Compute a better guess from the current guess.
• _at_return the next guess
• /
• public double nextGuess()
• xold xnew
• if (xold ! 0)
• xnew (xold a / xold) / 2
• return xnew

9

• /
• Compute the root by repeatedly improving the
  currentguess until two successive guesses are
  approximately equal. _at_return the computed value
  for the square root
• /
• public double getRoot()
• while(!Numeric.approxEqual(xnew, xold))
• nextGuess()
• return xnew
• // RootApproximator

10
RootApproximatorTest.java

• import javax.swing.JOptionPane
• /
• This program prints ten approximations
• for a square root.
• /
• public class RootApproximatorTest
• public static void main(String args)
• String input JOptionPane.showInputDialog(
  "Enter a number")
• double x Double.parseDouble(input)
• RootApproximator r new RootApproximator(x)

11

• final int MAX_TRIES 10
• for (int tries 1
• tries lt MAX_TRIES tries)
• double y r.nextGuess()
• System.out.println("Guess "
• tries " " y)
• System.exit(0)
• // RootApproximatorTest

12
Unit Test with BlueJ
13
RootApproximatorTest2.java(multiple user inputs)
14

• import javax.swing.JOptionPane
• /
• This program computes square roots of
  user-supplied inputs.
• /
• public class RootApproximatorTest2
• public static void main(String args)
• boolean done false
• while (!done)
• String input JOptionPane.showInputDialog( "En
  ter a number, Cancel to quit")
• if (input null)
• done true
• else

15

• double x Double.parseDouble(input)
• RootApproximator r new RootApproximator(x)
  
• double y r.getRoot()
• System.out.println("square root of " x "
  " y)
• // else
• // while
• System.exit(0)
• // main
• // RootApproximatorTest2

16
Read Inputs from File

• Prepare a file with test inputs, such as
• 10020410.250.01
• Use input redirection
• java RootApproximatorTest3 lt test.in
• Output
• square root of 100.0 10.0
• square root of 20.0 4.47213595499958

17
RootApproximatorTest3.java(can redirect input
from file)

• import java.io.BufferedReader
• import java.io.InputStreamReader
• import java.io.IOException
• /
• This program computes square rootsof inputs
  supplied through System.in
• /

18

• public class RootApproximatorTest3
• public static void main( String args)
• throws IOException
• BufferedReader console new
  BufferedReader( new InputStreamReader(
• System.in))
• boolean done false

19

• while (!done)
• String input console.readLine()
• if (input null) done true
• else
• double x Double.parseDouble(input)
• RootApproximator r new
  RootApproximator(x)
• double y r.getRoot()
• System.out.println("square root of x
  " " y)

20
Sources of Test Data

• Provided by humans
• Read from file - RootApproximatorTest3
• Computer-generated sequence (perhaps from a for
  loop) - RootApproximatorTest4
• Random sequence - RootApproximatorTest5

21
RootApproximatorTest4.java

• /
• This program computes square roots
• of numbers supplied by a loop.
• /
• public class RootApproximatorTest4

22

• public static void main(String args)
• final double MIN 1
• final double MAX 10
• final double INCREMENT 0.5
• for (double x MIN x lt MAX x x
  INCREMENT)
• RootApproximator r new RootApproximator(x
  )
• double y r.getRoot()
• System.out.println("square root of " x
  " " y)
• // for
• // main
• // RootApproximatorTest4

23
RootApproximatorTest5.java(loop generating
random inputs)

• import java.util.Random
• /
• This program computes square
• roots of random inputs.
• /
• public class RootApproximatorTest5

24

• public static void main(String args)
• final double SAMPLES 100
• Random generator new Random()
• for (int i 1 i lt SAMPLES i)
• // generate random test value
• double x 1.0E6 generator.nextDouble()
• RootApproximator r new RootApproximator(x)
  
• double y r.getRoot()
• System.out.println("square root of "
• x " " y)
• / for
• // main
• // RootApproximatorTest5

25
Test Cases

• Positive test case expect positive outcomeE.g
  square root of 100
• Negative test case expect negative outcomeE.g
  square root of -2
• Boundary test case at boundary of
  domainFrequent cause for errorsE.g square root
  of 0

26
Test Case EvaluationHow do you know when the
result is correct?

• Manual (visual inspection)RootApproximatorTest3
• Check property of resultE.g. square root squared
  original valueRootApproximatorTest6
• Check if oracle your computed answerE.g.
  square root of x x1/2RootApproximatorTest7

27
RootApproximatorTest6.java

• import java.util.Random
• /
• This program verifies the computation of
• square root values by checking a
• mathematical property of square roots.
• /

28

• public class RootApproximatorTest6
• public static void main(String
• args)
• final double SAMPLES 100
• int passcount 0
• int failcount 0
• Random generator new Random()

29

• for (int i 1 i lt SAMPLES i)
• // generate random test value
• double x 1.0E6 generator.nextDouble()
• RootApproximator r new RootApproximator(x)
• // call method being tested
• double y r.getRoot()
• System.out.println("square root of " x "
  " y)

30

• // check that test value fulfills
• // square property
• if (Numeric.approxEqual(y y, x))
• System.out.println("Test passed.")
• passcount
• else
• System.out.println("Test failed.")
• failcount

31

• // output a report of
• // number passed and number
• // failed
• System.out.println("Pass "
• passcount)
• System.out.println("Fail " failcount)

32
RootApproximatorTest7.java

• import java.util.Random
• /
• This program verifies the
• computation of square root values
• by using an oracle.
• /
• public class RootApproximatorTest7

33

• public static void main(String
• args)
• final double SAMPLES 100
• int passcount 0
• int failcount 0
• Random generator new Random()

34

• for (int i 1 i lt SAMPLES i)
• // generate random test value
• double x 1.0E6
• generator.nextDouble()
• RootApproximator r new RootApproximator(x)
  
• double y r.getRoot()
• System.out.println("square root of " x
  " " y)
• double oracleValue Math.pow(x,0.5)

35

• // check that test value approximately
• // equals oracle value
• if (Numeric.approxEqual(y,oracleValue))
• System.out.println("Test passed.")
• passcount
• else
• System.out.println("Test failed.")
• failcount

36

• System.out.println("Pass " passcount)
• System.out.println("Fail " failcount)

37
Regression Testing

• Save test cases
• Automate testing
• java Program lt test1.in gt test1.outjava Program
  lt test2.in gt test2.outjava Program lt test3.in gt
  test3.out
• Repeat test whenever program changes
• Test suite collection of test cases
• Cycling bug that is fixed but reappears in
  later versions
• Regression testing testing against past
  failures(rerunning all tests when a change or
  fix is made)

38
Test Coverage

• Black-box testing test functionality without
  regard to the methods internal structure
• White-box testing take internal structure into
  account when designing tests
• Test coverage the lines of code that are
  actually executed during test cases
• Easy to overlook error branches during
  testingMake sure to execute each branch in at
  least one test case

39
Program Trace

• Output statements in your program for diagnostic
  purposes
• if (status SINGLE)
• System.out.println(
• "status is SINGLE") ... ...
• Stack trace tells you the contents of the call
  stack
• Throwable t new Throwable()
• t.printStackTrace(System.out)

40

• Looks like exception report
• java.lang.Throwable at TaxReturn.getTax(TaxRet
  urn.java26) at TaxReturnTest.main(TaxReturnTe
  st.java30)
• Drawback of trace messages Need to remove them
  when testing is complete, stick them back in when
  another error is found

41
Logging

• Get global logger objectLogger logger
  Logger.getLogger("global")
• Log a messagelogger.info("status is SINGLE")
• By default, logged messages are printed. Turn
  them off withlogger.setLevel(Level.OFF)

42
Assertions

• Assertion assumption that you believe to be
  true
• assert y gt 0root Math.sqrt(y)
• If assertion fails, program is terminated
• Can be used to assert pre/postconditions
• Must compile/run with special flags
• javac -source 1.4 MyProg.javajava
  -enableassertions MyProg

43
The Debugger

• Debugger program to run your program, interrupt
  it, and inspect variables
• Three key commands
• Set Breakpoint
• Single Step
• Inspect Variable
• Two variations of Single Step
• Step Over don't enter method call
• Step Inside enter method call

44
The Debugger Stopping at a Breakpoint
45
Inspecting Variables
46
Sample Debugging Session

• Word class counts syllables in a word
• Each group of adjacent vowels (aeiouy) is a
  syllable
• However, an e at the end of a word doesn't count
• If algorithm gives count of 0, increment to 1
• Constructor removes non-letters at beginning and
  end
• Buggy output
• Syllables in hello 1Syllables in regal
  1Syllables in real 1

47
File Word.java

• public class Word
• private String text
• /
• Constructs a word by removing leading and
  trailing non-
• letter characters, such as punctuation marks.
• _at_param s the input string
• /

48

• public Word(String s)
• int i 0
• while (i lt s.length() !Character.isLetter(s.ch
  arAt(i)))
• i
• int j s.length() - 1
• while (j gt i !Character.isLetter(s.charAt(j)))
  
• j--
• text s.substring(i, j 1)

49

• /
• Returns the text of the word, after removal of
  the
• leading and trailing non-letter characters.
• _at_return the text of the word
• /
• public String getText()
• return text
• /
• Counts the syllables in the word.
• _at_return the syllable count
• /

50

• public int countSyllables()
• int count 0
• int end text.length() - 1
• if (end lt 0) return 0 // the empty string has
  no syllables
• // an e at the end of the word doesn't count as
  a vowel

51

• char ch Character.toLowerCase(text.charAt(end))
  
• if (ch 'e') end--
• boolean insideVowelGroup false
• for (int i 0 i lt end i)
• ch Character.toLowerCase(text.charAt(i))
• if ("aeiouy".indexOf(ch) gt 0)

52

• // ch is a vowel
• if (!insideVowelGroup)
• // start of new vowel group
• count
• insideVowelGroup true
• else

53

• insideVowelGroup false
• // every word has at least one syllable
• if (count 0)
• count 1
• return count

54
WordTest.java

• import java.util.StringTokenizer
• import javax.swing.JOptionPane
• public class WordTest
• public static void main(String args)

55

• String input JOptionPane.showInputDialog( "E
  nter a sentence")
• StringTokenizer tokenizer new
  StringTokenizer(input)
• while (tokenizer.hasMoreTokens())
• String token tokenizer.nextToken()
• Word w new Word(token)

56

• int syllables w.countSyllables()
• System.out.println("Syllables in "
  w.getText() " " syllables)
• System.exit(0)

57
Final Letter Test is Not Correct

• Set breakpoint in line 35 (first line of
  countSyllables)
• Start program, supply input hello
• Method checks if final letter is 'e'
• Run to line 41
• Inspect variable ch
• Should contain final letter but contains 'l'

58
Debugging the countSyllables Method
59
The Current Values of the Local and Instance
Variables
60
More Problems Found

• end is set to 3, not 4
• text contains "hell", not "hello"
• No wonder countSyllables returns 1
• Culprit is elsewhere
• Can't go back in time
• Restart and set breakpoint in Word constructor

61
Debugging the Word Constructor

• Supply "hello" input again
• Break past the end of second loop in constructor
• Inspect i and j
• They are 0 and 4--makes sense since the input
  consists of letters
• Why is text set to "hell"?
• Off-by-one error Second parameter of substring
  is the first position not to include
• text substring(i, j)should betext
  substring(i, j 1)

62
Debugging the Word Constructor
63
Another Error

• Fix the error
• Recompile
• Test again
• Syllables in hello 2Syllables in regal
  1Syllables in real 1
• Oh no, it's still not right
• Start debugger
• Erase all old breakpoints
• Supply input "regal"

64
Debugging countSyllables (again)

• Break in the beginning of countSyllables
• Single-step through loop
• First iteration ('r') skips test for vowel
• Second iteration ('e') passes test, increments
  count
• Third iteration ('g') skips test
• Fourth iteration ('a') passes test, but doesn't
  increment count

65

• insideVowelGroup was never reset to false
• Fix and retest All test cases pass
• Is the program now bug-free? The debugger can't
  answer that.

66
The First Bug
67
Therac-25 Facility