Testing and Debugging (Depura

1
Testing and Debugging(Depuração em Haskell)

• Complementa as seções anteriores
• Original autorizado por John Hughes
• http//www.cs.chalmers.se/rjmh/
• Adaptado por Claudio Cesar de Sá

2
Whats the Difference?
Testing means trying out a program which is
believed to work, in an effort to gain confidence
that it is correct. If no errors are revealed by
thorough testing, then, probably, relatively few
errors remain. Debugging means observing a
program which is known not to work, in an effort
to localise the error. When a bug is found and
fixed by debugging, testing can be resumed to see
if the program now works. This lecture describes
recently developed tools to help with each
activity.
3
Debugging
Heres a program with a bug
Mediangt median 8,4,6,10,2,7,3,5,9,1 2 Mediangt
isort 8,4,6,10,2,7,3,5,9,1 1,8,4,6,10,2,7,3,5,9

4
Debugging Tools
The Manual Approach We choose cases to try, and
manually explore the behaviour of the program, by
calling functions with various (hopefully
revealing) arguments, and inspecting their
outputs. The Automated Approach We connect a
debugger to the program, which lets us observe
internal values, giving us more information to
help us diagnose the bug.
5
The Haskell Object Observation Debugger
Provides a function which collects observations
of its second argument, tagged with the String,
and returns the argument unchanged. Think of it
as like connecting an oscilloscope to the
program the program's behaviour is unchanged,
but we see more. (You need to import the library
which defines observe in order to use it add at
the start of your program).
6
Garantindo que Observe.lhs foi carregado ...
Listasgt l listas_haskell.hs Reading
file "listas_haskell.hs" Reading file
"/usr/share/hugs/lib/exts/Observe.lhs" Reading
file "listas_haskell.hs"
Hugs session for /usr/share/hugs/lib/Prelude.hs
/usr/share/hugs/lib/exts/Observe.lhs listas_haskel
l.hs Listasgt
7
What Do Observations Look Like?
Mediangt sum observe "nn" (nn) n lt-
1..4 30 gtgtgtgtgtgtgt Observations ltltltltltlt nn 1
4 9 16
8
Observing a List
Mediangt sum (observe "squares" nn n lt-
1..4) 30 gtgtgtgtgtgtgt Observations
ltltltltltlt squares (1 4 9 16 )
9
Observing a Pipeline
Mediangt (sum . observe "squares" . map (\x-gtxx))
1..4 30 gtgtgtgtgtgtgt Observations ltltltltltlt squares
(1 4 9 16 )
10
Observing Counting Occurrences
countOccurrences map (\ws -gt (head ws,
length ws)) . observe "after groupby" .
groupBy () . observe "after sort" . sort
. observe "after words . words
11
Observing Counting Occurrences
Maingt countOccurrences "hello clouds hello
sky" ("clouds",1),("hello",2),("sky",1) gtgtgtgtgtgtgt
Observations ltltltltltlt after groupby (("clouds"
) ("hello" "hello" ) ("sky" )
) after sort ("clouds" "hello" "hello"
"sky" ) after words ("hello" "clouds"
"hello" "sky" )
12
Observing Consumers
An observation tells us not only what value
flowed past the observer -- it also tells us how
that value was used! Maingt take 3 (observe "xs"
1..10) 1,2,3 gtgtgtgtgtgtgt Observations
ltltltltltlt xs (1 2 3 _)
13
Observing Length
Maingt length (observe "xs" (words "hello
clouds")) 2 gtgtgtgtgtgtgt Observations ltltltltltlt xs (_
_ )
14
Observing Functions
We can even observe functions themselves! Maingt
observe "sum" sum 1..5 15 gtgtgtgtgtgtgt Observations
ltltltltltlt sum \ (1 2 3 4 5 ) -gt
15
15
Observing foldr
Recall that foldr () 0 1..4 1 (2 (3
(4 0))) Lets check this, by observing the
addition function. Maingt foldr (observe "" ())
0 1..4 10 gtgtgtgtgtgtgt Observations ltltltltltlt \
4 0 -gt 4 , \ 3 4 -gt 7 , \ 2 7 -gt 9 , \ 1
9 -gt 10
16
Observing foldl
We can do the same thing to observe foldl, which
behaves as foldl () 0 1..4 (((0 1) 2)
3) 4 Maingt foldl (observe "" ()) 0
1..4 10 gtgtgtgtgtgtgt Observations ltltltltltlt \ 0
1 -gt 1 , \ 1 2 -gt 3 , \ 3 3 -gt 6 , \ 6 4
-gt 10
17
How Many Elements Does takeWhile Check?
takeWhile isAlpha ''hello clouds hello sky''
''hello'' takeWhile isAlpha selects the
alphabetic characters from the front of the
list. How many times does takeWhile call isAlpha?
18
How Many Elements Does takeWhile Check?
Maingt takeWhile (observe "isAlpha" isAlpha)
"hello clouds hello sky" "hello" gtgtgtgtgtgtgt
Observations ltltltltltlt isAlpha \ ' ' -gt False
, \ 'o' -gt True , \ 'l' -gt True , \ 'l'
-gt True , \ 'e' -gt True , \ 'h' -gt True
19
Observing Recursion
Maingt observe "fac" fac 6 720 gtgtgtgtgtgtgt
Observations ltltltltltlt fac \ 6 -gt 720
20
Observing Recursion
Maingt fac 6 720 gtgtgtgtgtgtgt Observations
ltltltltltlt fac \ 6 -gt 720 , \ 5 -gt 120 , \
4 -gt 24 , \ 3 -gt 6 , \ 2 -gt 2 , \ 1 -gt
1 , \ 0 -gt 1
21
Debugging median
median xs observe "isort xs" (isort xs) !!
(length xs div 2) Maingt median
4,2,3,5,1 2 gtgtgtgtgtgtgt Observations ltltltltltlt isort
xs (1 4 2 3 5 )
22
Debugging isort
isort Ord a gt a -gt a isort foldr
(observe "insert" insert) Maingt median
4,2,3,5,1 2 gtgtgtgtgtgtgt Observations
ltltltltltlt insert \ 1 -gt 1 , \ 5 (1
) -gt 1 5 , \ 3 (1 5 ) -gt 1 3
5 , \ 2 (1 3 5 ) -gt 1 2 3
5 , \ 4 (1 2 3 5 ) -gt 1 4 2
3 5
23
Debugging insert
insert x x insert x (yys) xlty
observe "xlty" (xyys) xgty observe "xgty"
(yxys) Maingt median 4,2,3,5,1 2 gtgtgtgtgtgtgt
Observations ltltltltltlt xgty (1 5 ) (1 3
5 ) (1 2 3 5 ) (1 4 2 3
5 )
24
The Bug!
I forgot the recursive call insert x
x insert x (yys) xlty xyys xgty
yinsert x ys Maingt median 4,2,3,5,1 3
25
Summary

• The observe function provides us with a wealth
  of information about how programs are evaluated,
  with only small changes to the programs
  themselves
• That information can help us understand how
  programs work (foldr, foldrl, takeWhile etc.)
• It can also help us see where bugs are.

26
Testing
Testing means trying out a program which is
believed to work, in an effort to gain confidence
that it is correct. Testing accounts for more
than half the development effort on a large
project (Ive heard all from 50-80). Fixing a
bug in one place often causes a failure somewhere
else -- so the entire system must be retested
after each change. At Ericsson, this can take
three months!
27
''Hacking'' vs Systematic Testing
''Hacking'' Systematic testing

• Try some examples until the software seems to
  work.
• Record test cases, so that tests can be repeated
  after a modification (regression testing).
• Document what has been tested.
• Establish criteria for when a test is successful
  -- requires a specification.
• Automate testing as far as possible, so you can
  test extensively and often.

28
QuickCheck A Tool for Testing Haskell Programs

• Based on formulating properties, which
• can be tested repeatedly and automatically
• document what has been tested
• define what is a successful outcome
• are a good starting point for proofs of
  correctness
• Properties are tested by selecting test cases at
  random!

29
Random Testing?
Is random testing sensible? Surely carefully
chosen test cases are more effective?

• QuickCheck can generate 100 random test cases in
  less time than it takes you to think of one!
• Random testing finds common (i.e. important!)
  errors effectively.

30
A Simple QuickCheck Property
prop_Sort Int -gt Bool prop_Sort xs ordered
(sort xs)
Maingt quickCheck prop_Sort OK, passed 100 tests.
31
Some QuickCheck Details
import QuickCheck prop_Sort Int -gt
Bool prop_Sort xs ordered (sort xs)
Maingt quickCheck prop_Sort OK, passed 100 tests.
32
A Property of insert
prop_Insert Int -gt Int -gt Bool prop_Insert x
xs ordered (insert x xs) Maingt quickCheck
prop_Insert Falsifiable, after 4
tests -2 5,-2,-5
33
A Corrected Property of insert
prop_Insert Int -gt Int -gt Property prop_Inser
t x xs ordered xs gt ordered (insert x
xs) Maingt quickCheck prop_Insert OK, passed
100 tests.
34
Using QuickCheck to Develop Fast Queue Operations

• What were going to do
• Explain what a queue is, and give slow
  implementations of the queue operations, to act
  as a specification.
• Explain the idea behind the fast implementation.
• Formulate properties that say the fast
  implementation is correct.
• Test them with QuickCheck.

35
What is a Queue?

• Examples
• Files to print
• Processes to run
• Tasks to perform

36
What is a Queue?

• A queue contains a sequence of values. We can add
  elements at the back, and remove elements from
  the front.
• Well implement the following operations
• empty Queue a -- an empty queue
• isEmpty Queue a -gt Bool -- tests if a queue
  is empty
• add a -gt Queue a -gt Queue a -- adds an element
  at the back
• front Queue a -gt a -- the element at the
  front
• remove Queue a -gt Queue a -- removes an
  element from the front

37
The Specification Slow but Simple
type Queue a a empty isEmpty q
qempty add x q qx front (xq) x remove
(xq) q
38
The Idea Store the Front and Back Separately
Old
New
39
The Fast Implementation
type Queue a (a,a) flipQ (,b) (reverse
b,) flipQ (xf,b) (xf,b) emptyQ
(,) isEmptyQ q qemptyQ addQ x (f,b)
(f,xb) removeQ (xf,b) flipQ (f,b) frontQ
(xf,b) x
40
Relating the Two Implementations
What list does a double-ended queue represent?

• What does it mean to be correct?
• retrieve emptyQ empty
• isEmptyQ q isEmpty (retrieve q)
• retrieve (addQ x q) add x (retrieve q)
• retrieve (removeQ q) remove (retrieve q) and
  so on.

41
Using Retrieve Guarantees Consistent Results
Example frontQ (removeQ (addQ 1 (addQ 2
emptyQ))) front (retrieve (removeQ (addQ 1
(addQ 2 emptyQ)))) front (remove (retrieve
(addQ 1 (addQ 2 emptyQ)))) front (remove
(add 1 (retrieve (addQ 2 emptyQ)))) front
(remove (add 1 (add 2 (retrieve emptyQ))))
front (remove (add 1 (add 2 empty)))
42
QuickChecking Properties
prop_Remove Queue Int -gt Bool prop_Remove q
retrieve (removeQ q) remove (retrieve
q) Maingt quickCheck prop_Remove 4 Program
error removeQ (,sized_v1740
(instArbitrary_v1
43
Correcting the Property
prop_Remove Queue Int -gt Property prop_Remove
q not (isEmptyQ q) gt retrieve
(removeQ q) remove (retrieve q) Maingt
quickCheck prop_Remove 0 Program error removeQ
(,Arbitrary_arbitrary instArbitrary
44
Making Assumptions Explicit
We assumed that the front of a queue will never
be empty if the back contains elements! Lets
make that explicit goodQ Queue a -gt
Bool goodQ (,) True goodQ (xf,b)
True goodQ (,xb) False prop_Remove q
not (isEmptyQ q) goodQ q gt retrieve
(removeQ q) remove (retrieve q)
45
How Do We Know Only Good Queues Arise?
Queues are built by add and remove
New properties prop_AddGood x q goodQ q gt
goodQ (addQ x q) prop_RemoveGood q not
(isEmptyQ q) goodQ q gt goodQ (removeQ q)
46
Whoops!
Maingt quickCheck prop_AddGood Falsifiable, after
0 tests 2 (,)
See the bug?
47
Whoops!
Maingt quickCheck prop_AddGood Falsifiable, after
0 tests 2 (,)
48
Looking Back

• Formulating properties let us define precisely
  how the fast queue operations should behave.
• Using QuickCheck found a bug, and revealed hidden
  assumptions which are now explicitly stated.
• The property definitions remain in the program,
  documenting exactly what testing found to hold,
  and providing a ready made test-bed for any
  future versions of the Queue library.
• We were forced to reason much more carefully
  about the programs correctness, and can have
  much greater confidence that it really works.

49
Summary

• Testing is a major part of any serious software
  development.
• Testing should be systematic, documented, and
  repeatable.
• Automated tools can help a lot.
• QuickCheck is a state-of-the-art testing tool
  for Haskell.

50
The remaining slides discuss an important
subtlety when using QuickCheck
51
Testing the Buggy insert
prop_Insert Int -gt Int -gt Property prop_Inser
t x xs ordered xs gt ordered (insert x
xs) Maingt quickCheck prop_Insert Falsifiable,
after 51 tests 5 -3,4
52
Observing Test Data
prop_Insert Int -gt Int -gt Property prop_Inser
t x xs ordered xs gt collect (length
xs) (ordered (insert x xs)) Maingt quickCheck
prop_Insert OK, passed 100 tests. 43 0. 37
1. 11 2. 8 3. 1 4.
53
A Better Property
prop_Insert Int -gt Property prop_Insert x
forAll orderedList (\xs -gt collect (length
xs) (ordered (insert x xs))) Maingt quickCheck
prop_Insert2 OK, passed 100 tests. 22 2. 15
0. 14 1. 8 6. 8 5.
8 4. 8 3. 4 8. 3 9. 3 11.
2 12. 2 10. 1 7. 1 30. 1 13.
54
What is forAll?
A higher order function! forAll (Show a,
Testable b) gt Gen a -gt (a -gt b) -gt Property
forAll orderedList (\xs -gt collect (length xs)
(ordered (insert x xs)))
55
What is orderedList?
A test data generator orderedList Gen Int
Some primitive generators arbitrary Arbitrary
a gt Gen a oneof Gen a -gt Gen a frequency
(Int,Gen a) -gt Gen a
56
Defining orderedList
orderedList Gen Int orderedList do n
lt- arbitrary listFrom n where
listFrom n frequency
(1,return ), (4,do m
lt- arbitrary ns lt-
listFrom (nabs m) return (nns))