Unit Testing

1
Unit Testing

• Quickly Designing and Writing Things That Work

By Jim Moore, Symantec Software, ETS
2
Which Is the Culture Where You Work?

• I coded up the feature you wanted. I think it
  works, but Im not sure.Just send it over to
  QA and start working on the next thing. Well
  deal with any issues they find later.
  or
• I coded up the feature you wanted, but I still
  need to add a few more tests to make sure its
  solid.Great, when do you think youll be
  done?
• The first relies on a testing and maintenance
  cycle to handle issues later. The second assumes
  that its as solid as possible before you stop
  working on it.
• Bugs found while the developer is still in the
  mindset of that feature are much faster to fix
  than ones found later.
• The developer is always better off continuing to
  develop from a solid foundation. (ie, Its best
  not to develop against buggy code.)

3
The Problem
Every programmer knows they should write tests
for their code. Few do. The universal response to
Why not? is Im in too much of a hurry. This
quickly becomes a vicious cycle the more
pressure you feel, the fewer tests you write. The
fewer tests you write, the less productive you
are and the less stable your code becomes. The
less productive and accurate you are, the more
pressure you feel.

• JUnit Test Infected Programmers Love Writing
  Tests
• http//junit.sourceforge.net/doc/testinfected/test
  ing.htm

4
Unit Testing Is About Going Faster
...Unit Testing is about delivering working
software quickly. That means you need to be able
to write the code quickly, get it to work (avoid
and fix defects) and to be able to change and
maintain the code in future. The fastest way to
deliver working software is to write correct
code, straight off, in one sitting, to a known
set of requirements. Most programmers cant write
code thats perfect, and most customers change
their minds about the features they want (this is
a good thing were embracing change). Your
response as a programmer to the reality of
imperfect code is to write unit tests to help
check your work.

• Mike Mason
• http//mikemason.ca/2004/01/29032UnitTestingIsAbo
  utGoingFaster

5
When To Write Tests

• Martin Fowler makes this easy for you. He says,
  Whenever you are tempted to type something into
  a print statement or a debugger expression, write
  it as a test instead....
• Here are a couple of the times that you will
  receive a reasonable return on your testing
  investment
• During Development - When you need to add new
  functionality to the system, write the tests
  first. Then you will be done developing when the
  test runs.
• During Debugging - When someone discovers a
  defect in your code, first write a test that will
  succeed if the code is working. Then debug until
  the test succeeds.

• JUnit Test Infected Programmers Love Writing
  Tests
• http//junit.sourceforge.net/doc/testinfected/test
  ing.htm

6
What Is A Test?

• At the high-level, a test is a requirement. As
  every trained project manager knows, a project
  requirement must be clear and testable.
  Automated tests are just automatic requirement
  checks.
• There are tools, like FITnesse and Mercurys
  Quality Center, that handle this from the high
  level (functional and acceptance tests). This
  presentation will focus on the lower level unit
  tests.
• Probably the easiest way to think of what a good
  unit test is, is that its something that
  verifies a piece of functionalitys contract.
• Pre-conditions Those things that must be true
  before you run the code. (Example The objects
  buffer has been initialized.)
• Post-conditions Those things that must be true
  after you run the code. (Example This method
  will never return null.)
• Invariants Those things that must always be
  true. (Example The list size will always be
  greater than or equal to 0.)

7
What Makes a Good Unit Test? Part 1

• It sufficiently tests the contract. It doesnt
  need to be complete, just sufficient.
• If its complex code, then have lots of tests.
• If its a simple accessor, then you dont need a
  test.
• Most things are in between
• It runs quickly. Part of the point is to make it
  so that youre able to run them regularly, so
  speed plays into that.
• Its much easier to fix a bug you introduced five
  minutes ago than one you did five weeks ago
• cont

8
What Makes a Good Unit Test? Part 2

• Tests are independent.
• If you have dependencies between tests, or have
  to do tons of setup, then its a pain to write
  tests and to run them.
• Loosely coupled functionality enables independent
  tests.
• Write loosely coupled, highly cohesive code!
• Run the tests regularly. (Ideally, use something
  like CruiseControl to run the complete test suite
  run automatically after you check code in.)
• One of the seriously cool benefits of having the
  automated tests is that it gives you automatic
  regression testing.
• You can be fearless when you need to make
  changes, because you dont have to worry that you
  mightve broken something.

9
Writing Good Tests

• If you do all the stuff that you know youre
  supposed to anyway (loose coupling, high
  cohesion, etc.), writing tests is really easy.
  -)
• Seriously Loosely coupled and highly cohesive
  software is much easier to test than tightly
  coupled systems, or ones where its hard to tell
  what something is really for (low cohesion).
• Theres a lot of Patterns for making that easier,
  like Inversion of Control, Strategies, etc.
• Writing the test before you write the code
  guarantees that your code is testable.
• With very few exceptions, code that is hard to
  test is a HUGE flag that the design is bad (ie,
  tightly coupled, low cohesion).

10
Example of How To Think of Tests

• The required functionality for a particular
  service (getVersionsForProductId) is the ability
  to get a list of all versions for a given product
  id. If theres no product for that id, then an
  exception is thrown. If theres no versions for
  a valid product id, then an empty list is
  returned.

11
The Tests That Are Needed

• Starting from the easiest to the hardest
• If theres no product for that id, then an
  exception is thrown (pre-condition)
• If theres no versions for a valid product id,
  then an empty list is returned (post-condition)
• If there are versions for a product id, then a
  non-empty list of all the versions is returned
  (post-condition)

12
Implementing the Tests

• def testInvalidProduct
• begin
• vers getVersionsForProductId(-99)
• assert(false, "Should have thrown an
  exception, but got "vers.to_s)
• rescue
• an exception should've been thrown
• end
• end
• def testNoVersionsForProduct
• 1234 is a legit product id, but we know it
  doesn't have any versions
• vers getVersionsForProductId(1234)
• assert(vers.length 0, "Got versions back
  "vers.to_s)
• end
• def testListOfVersionsForProduct
• 2345 is a legit product id, and we know it
  has versions
• vers getVersionsForProductId(2345)
• assert(vers.length 0, "Didn't get any
  versions back")

13
Tools To Help Make Testing Easy

• The de-facto framework that everybody uses is
  JUnit. Theres clones in pretty much every
  language, like NUnit for C. Heres a sample
• using System
• using NUnit.Framework
• using System.Text.RegularExpressions
• namespace Notepad
• TestFixture
• public class TestRegex Assertion
• Test
• public void SimplePattern()
• Regex r new Regex("")
• Match m r.Match("contains here")
• Assert(m.Success)
• m r.Match("contains no para")
• Assert(!m.Success)
• For a good list of all the various languages, go
  to http//www.xprogramming.com/software.htm

14
What Is JUnit?

• JUnit is an open source Java testing framework
  used to write and run repeatable tests. It is an
  instance of the xUnit architecture for unit
  testing frameworks.
• JUnit features include
• Assertions for testing expected results
• Test fixtures for sharing common test data
• Test suites for easily organizing and running
  tests
• Graphical and textual test runners
• JUnit was originally written by Erich Gamma and
  Kent Beck

- FAQ on junit.org
15
Simple Example

• public class TestUser
• extends junit.framework.TestCase
• public void testSetName()
• User user new User()
• assertEquals(, user.getName())
• user.setName(Jim)
• assertEquals(Jim, user.getName())

16
Notes On The Simple Example

• All JUnit tests implement the Test interface,
  typically by extending the TestCase class.
• The testing harness can automatically determine
  what to run by looking for any method that is
  public, returns void, and starts with test.
• Test conditions are asserted. On the previous
  screen, for example, we were asserting the
  equality of two values.
• If it makes it through the test without any
  assertions failing, the test is considered to
  have passed.

17
Types of Assertions (part 1)

• assertTrue / assertFalse
• assertTrue(list.size() 10)
• assertTrue(list.size() 10 list.size(),
  list.size() 10)
• assertEquals
• assertEquals(10, list.size())
• assertEquals(Expected 10, Actual
  list.size(), 10, list.size())
• assertTrue/False is more flexible, but if it
  fails the message isnt very useful unless you
  provide your own message.
• assertEquals automatically shows the expected and
  actual values so you dont have to write your own
  message.

18
Types of Assertions (part 2)

• assertNull / assertNotNull
• Convenience for assertTrue(obj null)
• assertSame / assertNotSame
• Convenience for assertTrue(obj1 obj2)
• fail(msg)
• Convenience for assertTrue(msg, false)
• Used to indicate that a point in the code should
  not have been reached (eg, an exception should
  have been thrown)

19
Example Using fail

• public void testBadLoad()
• try
• User.load(-1234) // bad id
• fail(Should have thrown a PersistenceExceptio
  n)
• catch (PersistenceException exp)
• // should have happened

20
Errors Failures

• At the end of testing, JUnit reports three
  statistics (with an implied 4th)
• Tests run the number of tests run
• Failures the number of assertXXX statements that
  failed
• Errors the number of uncaught Exceptions
• Passed (Tests run (FailuresErrors))

21
Throwing Exceptions

• The test methods must be public, have no return
  value or arguments, and begin with test.
  (Theres actually a way to get around this
  restriction, but its not worth it) It can
  throw anything, so its common to have all the
  tests throw Exception so you dont have to worry
  about dealing with try/catch in your test code.
• If an exception is thrown, its counted as an
  error (as opposed to a failure)

22
Running JUnit

• There are three basic ways to run a test
• Within your IDE
• As a GUI using junit.swingui.TestRunner
• As text using either junit.textui.TestRunner or
  Ant

23
Running JUnit in IDEA
24
Running JUnit As a GUI
25
Running JUnit as Text

• java junit.textui.TestRunner test.ast.titan.busi
  ness.customer.TestCustomerDao..Time 4.031OK
  (2 tests)
• ant test -Dtestclasstest.ast.titan.business.cus
  tomer.TestCustomerDaoRunning test.ast.titan.busin
  ess.customer.TestCustomerDaoTests run 2,
  Failures 0, Errors 0, Time elapsed 3.515 sec
• Using Ant you can also easily have it do reports
  using the junitreport task

26
JUnitReport Task Output
27
Other Options for Running Tests

• In IDEA you can tell it to run tests for an
  entire package, a particular class, or a
  particular test method.
• In Ant you can use the batchtest subtask of the
  junit task to run a specific batch of tests.
  (The junitreport task it usually used to
  process the results of a batchtest run.)
• Both IDEA and Ant have lots and lots of options
  for customizing what tasks are run and how.

28
Putting Together A TestSuite

• public static Test suite()
• TestSuite suite new TestSuite()
• suite.addTest(new TestCustomerDao("testLoad"))
• suite.addTest(new TestCustomerDao("testCommit"))
  
• return suite
• This way you can specify exactly what tests are
  run and in what order. If suite() is in your
  test class, it wont use automatic detection of
  tests.
• Theres an alternate idiom for TestSuite where
  you can pass it a Class instead of a TestCase,
  which uses automatic detection of tests in a
  TestCase, but allows you to string together
  TestCases.
• This can be handy if you have a few test classes
  that should be run together (like for integration
  testing).

29
Why TestSuite Is Generally Evil

• Its duplication youve already declared the
  test, let it do the work of finding it and
  running it
• The order of your tests shouldnt matter if they
  do its a flashing red light with air-raid sirens
  that the tests are either poorly done, or that
  what they are testing is poorly designed
• If you want to temporarily disable a test, its
  easiest to just prepend XX to the name (eg,
  XXtestLoad), which both causes it to not be run
  automatically and makes it obvious when you look
  at the test that its not being run
• Its recommended practice to separate the tests
  that you run all the time (ie, standard unit
  tests) and those that are too expensive to run
  all the time (eg, integration tests) by either
  package or directory structure. That way it
  reduces maintenance by not having to maintain
  TestSuites, and makes it clear which tests are
  for and how often they are run.

30
Running Tests Automatically

• Whereas
• Running the complete test suite regularly makes
  sure that the minimum amount of time has past
  between a test breaking and discovering that fact
• Running a complete test suite can take a long
  time, thereby being impractical to run regularly
  manually
• People are forgetful, and even when it would be
  quick and easy to run the tests they will forget
• Therefore be it resolved
• A tool such as CruiseControl (http//cruisecontrol
  .sf.net/) should be used to automate the
  execution of regular builds and running of test
  suites
• When a developer checks in code that breaks the
  test(s), notification should be sent so he/she
  can fix it quickly
• If the offending developer does not immediately
  start fixing the problem, a wedgy should
  administered once an hour, upon the hour, by all
  the other developers who are being spammed
  because the tests are broken

31
Test Granularity

• The same principles that apply to coding in
  general still apply to tests
• Each test should be highly focused, testing just
  one piece of functionality (high cohesion)
• Each test should be independent of the others,
  without side-effects (low coupling)
• Example As you add tests to testLoad it may be
  best to get rid of it and split it apart into
  testLoadValid, testLoadInvalid,
  testLoadDatabaseError, etc.

32
Setting Up and Tearing Down

• Before each test method, the protected void
  setUp() method is called
• After each test method, the protected void
  tearDown() method is called
• This is extremely useful for doing common
  configuration that every test in the TestCase
  needs (eg, opening and closing a socket
  connection)
• Note Because its run for each test, its not
  appropriate for expensive resource setup (eg,
  connecting to a database). For that use either
  the TestSetup decorator class, or a static
  variable and initialize it in the class static
  block. However, make sure that using that
  resource doesnt cause side-effects between tests!

33
Other Tools For Running Tests

• http//www.junit.org/news/extension/index.htm
  links to hundreds of extensions for various needs
• Some of the ones I like
• NoUnit Shows a report of what methods are
  actually tested (by reading your byte-code)
• JUnitPerf Lets you do everything from
  performance testing (making sure a test returns
  in a specific amount of time) to load testing
  (running a test in X threads Y times), which can
  also be very handy for finding intermittent
  problems (eg, race-conditions)
• JFCUnit Provides some nice tools to make it
  almost reasonable to test Swing code
• HttpUnit Provides some nice tools to make it
  almost reasonable to test web code
• StrutsTestCase Provides a mock of Struts that
  can be run outside a servlet engine for testing
• Cactus The best way to test J2EE, its
  ridiculously complicated, but thats because
  testing code in J2EE containers is very complex
  (an alternative is to use MockEJB) its best to
  avoid having to do this altogether by using
  Dependency Injection!

34
Naming Conventions

• You can name your TestCases anything you like,
  but recommended practice is prepending Test
  before the name of the class youre testing. (eg,
  TestCustomerDao)
• Test names should reflect what it is they are
  testing, obviously, but what does that mean?
• testXXX, where XXX is the method being tested
• This is nice because it makes it obvious what
  method is being tested, and works well with tools
• testXXX, where XXX is not a method name
• This is needed when testing several methods
  together, or when it takes multiple tests to test
  a method (like the testLoad example being
  broken apart earlier)

35
Showing Test Names

• Tools like TestDox (http//agiledox.sf.net/) can
  be used to create pretty names from TestCases
• Example
• public class TestFoo extends TestCase
• public void testIsASingleton()
• public void testAReallyLongNameIsAGoodThing()
  
• Generates
• Foo
• - Is a singleton
• - A really long name is a good thing
• There are plugins for IDEs (like IDEA) that will
  do the same thing when they display tests

36
Testing Protected/Private Code

• In general, you should only be testing public
  methods
• In Java, access privileges are a suggestion
• For protected code, you can override the class in
  your test.
• class A
• protected int m() return 2
• class B extends A
• _at_Override public int m() return super.m()
• For private and protected code, you can use
  reflection
• class A
• private int m() return 2
• class MyTestCase extends TestCase
• public void testM() throws Exception
• A a new A()
• Method method a.getClass().getMethod("m",
  null)
• method.setAccessible(true)
• Integer result (Integer)method.invoke(a,
  null)

37
Common Code

• Theres nothing that says you have to extend
  directly from the TestCase class. If you have a
  number of TestCases that requires common setup
  and utility methods, create an abstract class
  that extends from TestCase that your classes
  extend from.

38
What Is Test Driven Development?
We start by writing some client code as though
the code we want to develop already existed and
had been written purely to make our life as easy
as it could possibly be. This is a tremendously
liberating thing to do by writing a model client
for our code, in the form of a test, we can
define programmatically the most suitable API for
our needs.
- Dan NorthJava Developers Journal, Nov 2003
39
Some Principles of TDD

• Main Ideas
• When youre done, youre done
• Do as little work as possible
• Corollaries
• By having tests that demonstrate that things
  work, you know when youre done
• Assume that the code you write should have APIs
  that exist to make your life easier
• By having tests, you can refactor things to make
  them even simpler and be assured of the safety of
  doing so

40
Doing As Little Work As Possible
Baby steps Baby steps
- Bill Murray, What About Bob?
41
Sample Work List

• Need way to load users
• Username for user 9999 is jmoore
• Firstname for user 9999 is Jim
• Username for user 9998 is smoore
• What if user doesnt exist?
• Was there a problem loading?

42
Loading Users

• The first item is that I need a way to load
  users
• Whats the simplest way that I can ask a user to
  load?
• Well, first thing that comes to mind is
  user.load()
• Lets write a test to see how well doing that
  would work

43
Writing the Test

• public void testLoad() throws Exception
• User user createUser()
• user.load()
• Since user.load() hasnt been written yet, this
  wont even compile

44
Compiling

• Whats the fastest way to get the test to at
  least compile?
• Add this to the User class
• public void load()
• Woo-hoo! It compiles!

45
Work List

• Need way to load users
• Username for user 9999 is jmoore
• Firstname for user 9999 is Jim
• Username for user 9998 is smoore
• What if user doesnt exist?
• Was there a problem loading?

46
Writing the Test

• public void testLoad() throws Exception
• User user createUser()
• user.setModelID(9999L)
• assertTrue(!"jmoore".equals(user.getUserName()))
  
• user.load()
• assertEquals("jmoore", user.getUserName())

47
Setting Username

• The test fails.
• Whats the fastest way to get a working test?
• public void load()
• setUserName("jmoore")
• The test passes! Were done! Seriously. Just
  walk away

48
Work List

• Need way to load users
• Username for user 9999 is jmoore
• Firstname for user 9999 is Jim
• Username for user 9998 is smoore
• What if user doesnt exist?
• Was there a problem loading?

49
First Name

• Lets addassertEquals("Jim", user.getFirstName())
  to the test case
• Bummer, it failed.
• public void load() setUserName("jmoore")
  setFirstName("Jim")
• Yay! It passes now!

50
Work List

• Need way to load users
• Username for user 9999 is jmoore
• Firstname for user 9999 is Jim
• Username for user 9998 is smoore
• What if user doesnt exist?
• Was there a problem loading?

51
Other User Test

• public void testLoadSean() throws Exception
• User user createUser()
• user.setModelID(9998L)
• assertTrue(!"smoore".equals(user.getUserName()))
  
• user.load()
• assertEquals("smoore", user.getUserName())
• When I run it, it fails.

52
Other User Load

• public void load() if (getModelId() 9999L)
  setUserName("jmoore")
  setFirstName("Jim") else if (getModelId()
  9998L) setUserName("smoore")
  setFirstName("Sean")
• Yay! All tests pass now!

53
Work List

• Need way to load users
• Username for user 9999 is jmoore
• Firstname for user 9999 is Jim
• Username for user 9998 is smoore
• What if user doesnt exist?
• Was there a problem loading?

54
Invalid User Test

• public void testLoadInvalidUser() throws
  Exception
• User user createUser()
• user.setModelID(-1234L) // known bad id
• try
• user.load() fail("Should have thrown an
  exception")
• catch (IllegalStateException exp) // should
  have happened
• When I run it, it fails.

55
Invalid User Load

• public void load() throws IllegalStateException
  if (getModelId() 9999L)
  setUserName("jmoore") setFirstName("Jim")
  else if (getModelId() 9998L)
  setUserName("smoore") setFirstName("Sean")
  else throw new IllegalStateException(
  "There was a problem loading user
  "getModelId())
• Yay! All tests pass now!

56
Work List

• Need way to load users
• Username for user 9999 is jmoore
• Firstname for user 9999 is Jim
• Username for user 9998 is smoore
• What if user doesnt exist?
• Was there a problem loading?

57
Problem Loading

• Well say that IllegalStateException should also
  be used if theres a problem loading
• What could go wrong while loading?
• Well, the database or network could be down
• Shoot, we need to make sure that the values come
  from the database. Better add that to the list

58
Work List

• Need way to load users
• Username for user 9999 is jmoore
• Firstname for user 9999 is Jim
• Username for user 9998 is smoore
• What if user doesnt exist?
• Was there a problem loading?
• Load values from the database

59
Load from Database

• public void load() throws IllegalStateException
  
• try
• // lots and lots of code for getting the
• // values from the database...
• catch (SQLException e)
• IllegalStateException exp new
  IllegalStateException(
• "There was a problem loading user
  "getModelId())
• exp.initCause(e)
• throw exp
• finally
• // close all the open resources
• Ouch, that was a lot of work between runs of
  tests
• Fortunately, all the tests we had before still
  pass. Yay!

60
Work List

• Need way to load users
• Username for user 9999 is jmoore
• Firstname for user 9999 is Jim
• Username for user 9998 is smoore
• What if user doesnt exist?
• Was there a problem loading?
• Load values from the database

61
Work List
Finished!

• Need way to load users
• Username for user 9999 is jmoore
• Firstname for user 9999 is Jim
• Username for user 9998 is smoore
• What if user doesnt exist?
• Was there a problem loading?
• Load values from the database

62
Some Principles of TDD (Review)

• Main Ideas
• When youre done, youre done
• Do as little work as possible
• Corollaries
• By having tests that demonstrate that things
  work, you know when youre done
• Assume that the code you write should have APIs
  that exist to make your life easier
• By having tests, you can refactor things to make
  them even simpler and be assured of the safety of
  doing so

63
Results

• When youre done, youve got
• code that you know works
• an API that is easy to use (since it was designed
  by actually being used)
• highly cohesive, loosely coupled components
  (writing tests for anything else is too hard, and
  since the point is to do things the easiest way
  possible, this happens naturally)
• more time, since youre not over-designing (if it
  isnt needed to make a test pass, why write it?)
  and not under-designing (you still have to make
  sure all the tests pass)
• cont

64
Results (cont.)

• When youre done, youve got
• examples on how the code is meant to be used
  developers like to see code examples for an API.
  You also know the documentation is accurate!
• documentation on what kinds of problems you were
  trying to solve when you wrote the code (which
  also nicely solves the What the heck was I
  thinking when I wrote this?? problem)
• a suite of regression tests, making maintenance
  much easier since you can make changes without
  fear
• freedom to try things, since changes are made in
  small increments and there are tests to make sure
  nothing breaks

65
Patterns in Testing 1

• Child Test If a test case is getting too big,
  break it into smaller, more easily maintained
  tests
• Mock Objects If something is too expensive or
  unreliable to test with (for example, a database
  or other network resource), then mocking it out
  can improve speed and reliability. To guard
  against relying too much on the mock, make sure
  it can easily be switched out with the real thing.

66
Patterns In Testing 2

• Log String If you need to make sure a sequence
  is called in a particular order, append the fact
  that a call was made to a common string, then
  compare the string to whats expected.
• Crash Test Dummy If you want to test that your
  code reacts appropriately to a condition that you
  cant easily replicate (eg, database crash,
  network failure, etc.), artificially create the
  error. For example, test the screen that loads a
  user by testing with a version of User that
  throws IllegalStateException every time its
  called.

67
What About Things That Are Hard To Test?

• Some things that are traditionally hard to test
  are
• GUIs
• Databases and other external services
• Things that need to run in containers (eg,
  J2EE)
• Using a combination of Separation of Concerns and
  Mock Objects, its not hard to do this.
• When it really is hard, its almost always an
  indication that your object is doing too much
  (ie, low cohesion).

68
Resources 1

• The FAQ is excellent, and really does answer
  pretty much any question about JUnit itself that
  youre likely to come across http//junit.sourcef
  orge.net/doc/faq/faq.htm
• There are a lot of great resources (extensions,
  articles, use cases, etc.) referenced from the
  http//junit.org site.
• A great book about JUnit, a number of its
  extensions, Ant, and more is Java Extreme
  Programming Cookbook from OReilly.

69
Resources 2

• http//www.junit.org/news/article/index.htmLots
  and lots of articles on how to test
• http//junit.sourceforge.net/doc/testinfected/test
  ing.htmJUnit Test Infected Programmers Love
  Writing Tests
• Lots and lots and lots of discussions, blogs, etc.

70

QUESTIONS
ANSWERS