Dave Holscher

1
Dave Holscher International Truck and Engine Joe
Batt Solid Design
2
Agenda

• Background on International projects using
  Continuous Integration
• Description of Continuous Integration
• Description of tools with Windows specific
  highlights
• Description of team communication tools
• Gotchas and Tips
• Questions, deeper dives as needed

3
Successful Projects - Hyperion

• Hyperion
• Tool for engineering to create software
  configurations for body controller
• Wipers, remote module control, plow-lights, etc.
• Used in order processing to generate or update
  10000 configurations per day
• Introduced Ant, JUnit, source control
• Developers run tests before submitting

4
Successful Projects - DLB

• Diamond Logic Builder (DLB)
• Embedded Hyperion
• Thick application for engineers and mechanics to
  configure, program, test, and simulate variety of
  truck controllers
• Introduced CVSNT, JFCUnit, CruiseControl
• Long running tests forced a need automated
  integration

5
Development Team

• Small team (5 full time max)
• No separate QA team
• Bugs hitting the field fall back on the
  development team
• Near daily customer interaction
• Dedicated to producing high quality software

6
Development Process

• Track bugs/enhancement/requirements
• Organize next round of changes into a deliverable
• Code the changes
• Submit after developer has tested
• Changes automatically picked up and full test is
  run
• Release to business for acceptance test

7
Development Process

• Release production quality release if acceptable
• Frequent releases to internal engineering, less
  frequent releases to external customers.
• Start over with next round regardless if release
  is accepted

8
Continuous Integration

• Definitions
• See http//www.martinfowler.com/articles/continuou
  sIntegration.html
• http//c2.com/cgi/wiki?ContinuousIntegration
• http//www.xprogramming.com/xpmag/whatisxp.htmcon
  tinuous

9
Continuous Integration

• What it means to us
• Software version control
• Automated builds, testing and deployment
• Automated build kick-off
• Feedback
• Repeatability

10
Tools

• Common, well established tools
• Provide broad base of support
• Continually improved by the community
• Why Windows?

11
Tools - CVSNT

• Source control
• Supports NT authentication (client and server)
• Supports pserver and ext (ssh) protocols
• Download and run installer from
  http//www.cvsnt.com/
• Check http//www.cvsnt.org/wiki/InstallationTips
  for install help

12
Tools - CVSNT
13
Tools - CVSNT
14
Tools - CVSNT

• Do completeinstall tosupportViewCVS

15
Tools - CVSNT

• Be sure to set all compatibility options on to
  work with Eclipse

16
Tools - CVSNT

• Set up repository
• No need to set up users

17
Tools CVS Integration

• Need excellent CVS integration with IDE
• Eclipse
• Tortoise CVS
• Good for CVS client outside the IDE
• See http//www.tortoisecvs.org/
• Integrates into Windows shell, works great with
  CVSNT

18
Tools Ant

• Automation using Ant
• De-facto standard Java build tool
• Download from http//ant.apache.org/
• Unzip, e.g. C\dev\ant
• Set ANT_HOME as system variable
• Add ANT_HOME\bin to system PATH
• Put libraries to additional task support in
  ANT_HOME\lib

19
Tools - Ant

• All builds should share the same Ant, put it CVS
• Use Ant script to automate everything

20
Tools - JUnit

• Using JUnit
• Download from http//www.junit.org/
• Drop jars into ANT_HOME\lib
• Many ways to use, lots of resources
• Database for testing
• Scripting, using CVS

21
Tools - JFCUnit

• Using JFCUnit
• Rich support for writing JUnit tests for Swing
  apps
• Download from http//jfcunit.sourceforge.net/
• Drop jar into ANT_HOME\lib

22
Tools JFCUnit

• Example1
• public void testMenuBar()
• FrameFinder f new FrameFinder(getName())
• JFrame frame (JFrame) f.find()
• JMenu file (JMenu) new JMenuItemFinder("File
  ").find(frame, 0)
• PathData path new PathData(new String
  "File", "Measure", "H")
• JMenuBar bar (JMenuBar) path.getRoot(file)
• getHelper().enterClickAndLeave(new
  JMenuMouseEventData(this, bar,
• path.getIndexes(bar), 1, 0, false, 0))
• flushAWT()
• assertTrue(somethinghappened)

23
Tools - JFCUnit

• Dealing with timing issues
• How long to wait after performing an action to
  check the result?
• Finders have support for waiting
• Sometimes whole test needs started over
• Create Watch class

24
Tools - JFCUnit

• abstract public class Watch
• private int time
• /
• A chunk of code to execute over and over
  until the Watch runs out
• of time, or the code executes without
  throwing an exception.
• /
• abstract public void run() throws Exception
• /
• _at_param time The amount of wall clock time in
  milliseconds to retry the ltcodegtrun()lt/codegt.
• /
• public Watch(int time)
• this.time time

25
Tools - JFCUnit

• / do the testing /
• public void start() throws Exception
• long end System.currentTimeMillis() time
• Throwable error null
• while (System.currentTimeMillis() lt end)
• try
• run()
• return
• catch (Throwable t)
• error t
• Thread.sleep(100)
• if (error instanceof Exception)
• throw (Exception)error
• else
• throw (Error)error

26
Tools - JFCUnit

• public void testDiagnosticMode() throws
  Exception
• SmartToolBar toolbar (SmartToolBar)new
  ItecComponentFinder(SmartToolBar.class).find(0)
• AbstractButton b (AbstractButton) new
  NamedComponentFinder(SmartToggleButton.class,
  "Activate Com Link").find(
• toolbar, 0)
• assertTrue("Communications link not enabled",
  b.isEnabled()) //NON-NLS-1
• diagModeOn()
• b (AbstractButton)new NamedComponentFinder(A
  bstractButton.class, "Diagnose").find(toolbar,
  0)
• new Watch(20000) public void run() throws
  Exception
• assertTrue("Not diagnostics mode",
  ((SeleneWindow)Selene.current().getWindow()).isDia
  gnostics()) //NON-NLS-1
• .start()

27
Tools - CruiseControl

• http//cruisecontrol.sourceforge.net/
• Takes time to set up, but only done once
• Run as a service
• Uses Ant to build and run tests
• XML result logs interpreted by J2EE web
  application for reporting
• Email notifications

28
Tools - CruiseControl

• Download and unpack
• Set up Ant build wrapper
• ltproject name"cc-build" default"build"
  basedir"."gt
• ltproperty name"iris-platform.dir"
  location"../iris-platform" /gt
• ltproperty name"cvsroot" value"pservernavista
  r\train20_at_cvs.ntc.navistar.com/ve" /gt
• lttarget name"build"gt
• ltdelete dir"iris-platform.dir/build" /gt
• ltcvs cvsRoot"cvsroot" dest"iris-platfor
  m.dir" command"-q update" /gt
• ltant antfile"build.xml" dir"iris-platform.
  dir" gt
• lttarget name"clean" /gt
• lttarget name"compile" /gt
• lttarget name"test" /gt
• lt/antgt
• lt/targetgt
• lt/projectgt

29
Tools CruiseControl

• Create CruiseControl Configuration
• ltcruisecontrolgt
• ltproject name"iris-platform"
  buildafterfailed"false"gt
• ltbootstrappersgt
• ltcurrentbuildstatusbootstrapper
• file"../logs/iris-platform/curren
  tbuildstatus.txt" /gt
• lt/bootstrappersgt
• ltmodificationset quietperiod"60"gt
• ltcvs localworkingcopy"../ant" /gt
• ltcvs localworkingcopy"../iris-platfor
  m" /gt
• ltfilesystem folder"config.xml" /gt
• ltfilesystem folder"compile" /gt
• lt/modificationsetgt

30
Tools CruiseControl

• Create CruiseControl Configuration
• ltschedule interval"60"gt
• ltant buildfile"cc-build.xml"
  antscript"../ant/bin/ant.bat"
• antWorkingDir"."
• target"build"
• useLogger"false"
• /gt
• lt/schedulegt
• ltlog encoding"UTF-8" dir"../logs/iris-pl
  atform"gt
• ltmerge dir"../iris-platform/build/test"
  /gt
• lt/loggt

31
Tools CruiseControl

• Create CruiseControl Configuration
• ltpublishersgt
• ltcurrentbuildstatuspublisher
• file"../logs/iris-platform/curren
  tbuildstatus.txt" /gt
• lthtmlemail mailhost"smtphub.navistar.
  com"
• returnaddress"NTEC-System_at_Nav-Int
  ernational.com"
• buildresultsurl"http//ntcpc-0029
  53.ntc.navistar.com/cruisecontrol/buildresults/iri
  s-platform"
• reportsuccess"always"
  spamwhilebroken"no"
• defaultsuffix"_at_nav-international.
  com"
• logDir"c\iris\logs\iris-platform
  "
• css"c\usr\cruisecontrol-2.2.1\re
  porting\jsp\webcontent\css\cruisecontrol.css"
• xslDir"c\usr\cruisecontrol-2.2.1
  \reporting\jsp\webcontent\xsl"gt
• ltalways address"chris.oman" /gt
• ltalways address"david.holscher"
  /gt
• lt/htmlemailgt
• lt/publishersgt

32
Tools - CruiseControl

• Setting up a Service
• Create a local tester account with administrative
  access
• Download and unpack JavaService from
  http//forge.objectweb.org/project/showfiles.php?g
  roup_id137
• Rename JavaService.exe to CCService.exe
• Place CCService.exe in CruiseControl bin
• Copy cruisecontrol.bat to ccservice.bat

33
Tools - CruiseControl

• Change
• set EXEC"JAVA_HOME\bin\java" -cp
  "CRUISE_PATH" -Djavax.management.builder.initial
  mx4j.server.MX4JMBeanServerBuilder CruiseControl
  
• To
• set EXECCruiseControlService.exe
• -install "Cruise Control" JAVA_HOME\jre\bin\
  client\jvm.dll
• -Djava.class.pathCRUISE_PATH
• -start CruiseControl
• -params
• -out CCDIR\service_out.log
• -err CCDIR\service_err.log
• -current CCDIR
• Run cruisecontrol.bat to check your configuration
  and save the project state to disk

34
Tools - CruiseControl

• Run ccbat.bat to install the service
• Use Windows Service Manager to set the service to
  run under the tester account and set it to
  Automatic
• Also set Allow Service to Interact with Desktop
• Start the service

35
Acceptance Testing Deployment

• Automating deployment is next step after
  automating builds
• Development, Test, Production deployment targets
  in Ant
• Automated deployment documents the process that
  is often not documented.

36
Acceptance Testing Deployment

• Tag releases when they go functional users for
  testing
• Using the build number to identify tag
• Run automated deployment against production
  against tested tag
• Try to deploy from the same directory where test
  version was built

37
Team Communication

• Tools will not replace communication but can keep
  everyone aware
• CruiseControl mail notifications for watching
  builds

38
Team Communication - ViewCVS

• ViewCVS for making it easy for everyone to see
• Install Apache for Win32
• Install ActivePython from http//www.activestate.c
  om/Products/ActivePython/ (include Windows
  extensions)
• Download and unpack ViewCVS from
  http//russ.hn.org/viewcvs/

39
Team Communication - ViewCVS

• Run python viewcvs-install from the unpack
  directory
• Edit cvs_roots and default_root in viewcvs.conf
• Copy web\cgi\viewcvs.cgi from ViewCVS install
  directory to cgi-bin in Apache directory
• Goto http//myviewcvsserver/cgi-bin/viewcvs.cgi

40
Team Communication CVS Mailer

• CVS mail notifications via CVSMailer for watching
  changes
• See http//web.telia.com/u86216121/cvsmailer/CVSM
  ailer.html

41
Team Communication CVS Mailer
42
Team Communication CVS Mailer
43
Team Communication CVS Mailer
44
Team Communication CVS Mailer
45
Team Communication CVS Mailer
46
Team Communication CVS Mailer
47
Team Communication CVS Mailer
48
Team Communication CVS Mailer

• CVSROOT/users
• The "users" file contains a mapping of CVS
  usernames against their email
• addresses so that CVS can resolve the usernamse
  to valid email for notifications
• Format for each line (one line per user)
• ltcvsusernamegtltemail addressgt
• Note that case is important for some tools,
  Cruise Control seems to want to look up all names
  in lowercase but CVSNT will use the id as
  returned by Windows
• john"John Doe ltjohnd_at_somedomain.comgt"
• u00dmh6david.holscher_at_nav-international.com
• U00DMH6david.holscher_at_nav-international.com
• u00wds3william.sears_at_nav-international.com
• U00WDS3william.sears_at_nav-international.com

49
Team Communication CVS Mailer

• CVSROOT/loginfousers
• The "loginfousers" file contains a list of CVS
  users who have subscribed to email
• on any commit in the repository.
• The name listed here must match the login
  user id that is recognized by CVS.
• Case sensitivit is ON here so you must be
  careful!
• A as first character on a line makes it a
  comment
• johnd
• YYYLTG1

50
Team Communication CVS Mailer

• CVSROOT/checkoutlist
• The "checkoutlist" file is used to support
  additional version controlled
• administrative files in CVSROOT/CVSROOT, such
  as template files.
• The first entry on a line is a filename which
  will be checked out from
• the corresponding RCS file in the
  CVSROOT/CVSROOT directory.
• The remainder of the line is an error message
  to use if the file cannot
• be checked out.
• File format
• ltwhitespacegtltfilenamegtltwhitespacegtlterror
  messagegtltend-of-linegt
• comment lines begin with ''
• users Unable to check out 'users' file in
  CVSROOT.
• loginfousers Unable to check out 'loginfousers'
  file in CVSROOT.

51
Team Communication CVS Mailer

• CVSROOT/taginfo
• The "taginfo" file is used to control pre-tag
  checks.
• The filter on the right is invoked with the
  following arguments
• 1 -- tagname
• 2 -- operation "add" for tag, "mov" for tag
  -F, and "del" for tag -d
• 3 -- repository
• Use this format for in-line specification of
  recipients
• DEFAULT C/Programs/CVSMailer/CVSMailer.exe
  -tCVSPID USER CVSROOT -rjohn,jane,charlie
• Or this for sending the email to all users
  specified in loginfousers
• DEFAULT C/Programs/CVSMailer/CVSMailer.exe
  -tCVSPID USER CVSROOT
• DEFAULT C/ProgramFiles/CVSMailer/CVSMailer.exe
  -tCVSPID USER CVSROOT

52
Team Communication CVS Mailer

• CVSROOT/loginfo
• (hyperiondlbemuutilvendorares)
  C/ProgramFiles/CVSMailer/CVSMailer.exe -lCVSPID
  USER CVSROOT -ru00dmh6,u00wds3,YYYB751,U00J994,u
  00k859 sVv
• CVSROOT C/ProgramFiles/CVSMailer/CVSMailer.exe
  -lCVSPID USER CVSROOT -ru00dmh6,YYYB751,u00wds3
  sVv
• (atdg451f) C/ProgramFiles/CVSMailer/CVSMailer.e
  xe -lCVSPID USER CVSROOT -ru00dmh6 sVv
• CcOoNnCcEeRrNnSsEeRrVvIiC
  cEe C/ProgramFiles/CVSMailer/CVSMailer.exe
  -lCVSPID USER CVSROOT -rYYYMAGI sVv
• CVSTest C/ProgramFiles/CVSMailer/CVSMailer.exe
  -lCVSPID USER CVSROOT -rYYYB751 sVv
• DEFAULT C/ProgramFiles/CVSMailer/CVSMailer.exe
  -lCVSPID USER CVSROOT sVv

53
Gotchas

• JFCUnit
• Doesn't work headless, need dedicated system.
• Timing issues
• Individual tests that re-run themselves.
• More threads, more issues.
• Some code changes can simplify testing.

54
Gotchas

• CVSNT
• Uses a lot of CPU cycles, put it on a dedicated
  server if possible
• Unix/Linux CVS and SSH more mature
• Beware of upgrading to latest release

55
Tips

• Know your version control
• Dont be afraid to branch
• Use tags
• A good CVS client can save a lot of time
• Tortoise CVS is great for novices
• Eclipse is great for Java or C
• Others?

56
Tips

• Don't check in build artifacts that can be
  regenerated
• Do check in any third party libraries
• We even check in Eclipse so everyone's
  configuration matches

57
Tips

• Automate everything
• Formalizing the process is worth the effort
• Documents the process
• Something will go wrong with your automated
  deployment
• Nothing is perfect, have an expert on call

58
Tips

• Get a bug tracking system
• Track everything
• Issue, bug, clarification, etc.
• Encourage everyone to read and participate in
  updating.
• Know what you are releasing.

59
Recommended Reading

• http//www.pragmaticprogrammer.com/starter_kit/ind
  ex.html
• Pragmatic Project Automation How to Build,
  Deploy, and Monitor Java Applications - Mike Clark

60
Conclusion

• Continuous Integration helped us create higher
  quality software with shorter time to market
• Tools are the key to making the process work
• All the tools to make Continuous Integration in
  Windows environment are out there