Model-based Testing Introduction and Hands-on session Stefan Ekman Dublin, 2011-01-27

1
Model-based TestingIntroduction and Hands-on
sessionStefan EkmanDublin, 2011-01-27
2
Model Based Testing

• What is Model-based Testing?

3
What is MBT?

• Wikipedia
• Model-based testing is software testing in which
  test
• cases are derived in whole or in part from a
  model that
• describes some (usually functional) aspects of
  the system
• under test (SUT).

4
Advantages

• Model-based testing is easy to understand from
  both the business and developer communities
• Model-based testing separates (business-) logic
  from testing code
• Model-based testing increases the test coverage
  with the same effort as Classic test automation
• Model-based testing is the fastest way to get use
  of automated test

5
Advantages

• Model-based testing enables us to switch testing
  tool if needed or support multiple platforms
  using the same model
• Model-based testing focuses on requirement
  coverage, not how many test cases you executed
  last week etc.
• Model-based testing proved to positively affect
  the maintenance problem, the nemesis of all test
  automation.

6
The Model

• Modelling your tests

7
Tools / Environment
.GraphMLModel with yEd
GraphWalkerMBT engine
Hudson Build system
Selenium Java or SOAP
QTP Via SOAP
8
(No Transcript)
9
Code behind

• public void e_Nav_GoogleImages()
• log.info("Edge e_Nav_GoogleImages")
• selenium.click(linkImages")
• selenium.waitForPageToLoad(30000)

10
Modelling

• What should the model represent?
• I suggest that you start with the model
  representing the end usage/user functionality of
  the system. That will usually resemble your
  traditional functional test case approach. There
  are several other approaches for you to explore
  later in you new MBT career

11
Modelling

• To what level do we model?

12
Modelling

• Who creates the model?
• Start out yourself to begin with
• Present the model show it to SMEs and
  developers for input
• If the system is non-existing, you probably would
  consider user stories, product backlog items,
  use-cases or the equivalent requirement's
  information.

13
Modelling

• During this process you will collect the first
  benefits of working with MBT!

14
GraphWalker

• GraphWalker is a tool for generating offline and
  online test sequences from Finite State
  Machines and Extended Finite State Machines.
• http//www.graphwalker.org/
• Lightweight MBT No UML
• No exit/stop points
• Online
• Event-driven
• GraphWalker is an Open Source MBT engine created
  by Kristian Karl and are stable and used in large
  IT testing environment.

15
GraphWalker - Stop Criteria's

• REACHED_EDGE
• REACHED_VERTEX
• EDGE_COVERAGE
• VERTEX_COVERAGE
• TEST_LENGTH
• TEST_DURATION
• REACHED_REQUIREMENT
• REQUIEMENT_COVERAGE
• NEVER

16
GraphWalker XML configuration

• lt?xml version"1.0" encoding"UTF-8"?gt
• lt!DOCTYPE MBTINIT SYSTEM "mbt_setup.dtd" gt
• ltMBTINIT EXECUTOR"javacom.paddypower.mbt.BetPlac
  ementRegression" EXTENDED"true" GUItrue"gt
• ltMODEL PATH"model/BetPlacementRegression.graphm
  l"/gt
• ltCLASS PATH"Java\PPautomation\bin"/gt
• ltCLASS PATH".\servlet-2.3.jar"/gt
• ltCLASS PATH"selenium\selenium-java-client-drive
  r.jar"/gt
• ltCLASS PATH"selenium\selenium-server-2.0a6.jar"
  /gt
• ltSCRIPTgt
• PortNumber 5555
• DBconnectionString "jdbcinformix-sqli//
  10.10.10.10/dbinformixservertestdb1useruserpa
  ssword12345"
• MAX_SELECTIONS 10
• lt/SCRIPTgt
• ltGENERATOR TYPE"RANDOM"gt
• ltCONDITION TYPE"REACHED_VERTEX"
  VALUE"v_Done"/gt
• lt/GENERATORgt
• lt/MBTINITgt

17
GraphWalker - The Walks

• RANDOM
• A_STARWill try to generate the shortest possible
  test
• sequence through a model with given stop
• condition. The algorithm only works well on
• smaller models.
• SHORTEST_NON_OPTIMIZED

18
GraphWalker - Edge

• An edge is a transition. Could be a click on a
  button, a timeout, a server API call.
• An edge can have a label, but it is not
  mandatory.
• The label of an edge will map against a
  method, function or sub routine during test
  execution.
• As a rule of thumb, the best practice is to start
  the name with 'e_'. The reason for this, is that
  it is easier to recognize the function in the
  test execution tool, as an edge. For
  examplee_StartApplicatione_EnterBasicView
• The same name of an edge, can be re-used in the
  model.

19
GraphWalker - Vertex

• A vertex is some kind of state that is possible
  to verify using some kind of oracle.
• A vertex must always have a label.
• The label of a vertex is mapped against a method,
  function or sub routine during test execution.
• As a rule of thumb, the best practice is to start
  the name with 'v_'. The reason for this, is that
  it is easier to recognize the function in the
  test execution tool, as a vertex. For
  examplev_ApplicationStartedv_BasicView
• The same name of a vertex, can be re-used in the
  model.

20
GraphWalker - Labels

• Label cond. expr. / Statement1Statement2
• White spaces not allowed.
• Follow the naming convention for the implementing
  programming
• language.
• Example
• e_StartApplication
• Example
• e_EnterBasicView

21
GraphWalker - Guards

• Label cond.expr. / Statement1Statement2
• Conditional expression that returns a Boolean
  value.
• The guard indicates if the edge is accessible in
  the current state of the
• machine. The language is either Java or
  JavaScript.
• Example
• e_Evaluate xgt0 ylt10
• Example
• e_Checkout shoppingBasket.contains("Cerials")

22
GraphWalker - Statements

• Label cond.expr. / Statement1Statement2
• Any applicable Java or JavaScript statements
• Example
• e_Calculate / x5y
• Example
• e_AddCerials / shoppingBasket.add("Cerials")

23
GraphWalker Keyword Start

• Used by Vertices only.
• Each graph must have a start vertex, and that
  vertex holds this
• keyword. There is only one vertex in a graph,
  holding this keyword.
• The Start vertex can only have one out-edge.
• If the graph is the main-graph, that edge must
  have a
• label.
• If the graph is a sub-graph, that edge cannot
  have a label.

24
GraphWalker - REQTAG
v_BookInfomationREQTAGUC01 2.2.3

• Used by Vertices
• Used by GraphWalker for keeping track of which
• requirements are fulfilled during testing.
• In the example above, if v_BookInfomation is
  passed, it
• would mean the requirement with tag UC01 2.2.3
  has
• passed.

25
GraphWalker - Weight
e_NavAweight0.25
v_BookInfomationREQTAGUC01 2.2.3
e_NavB

• Used by Edges only.
• Used by MBT during random walks during test
  sequence generation. It holds a real value
  between 0 and 1, and represents the probability
  that a specific edge should be chosen. A value of
  0.05, would mean a 5 chance of that edge to be
  selected during a run.
• Note It only works with the RANDOM generator

26
GraphWalker Keyword Blocked

• Used by both Vertices and Edges.
• A vertex or an edge with the key word BLOCKED,
  will be
• excluded from the model when walked.

v_BookInfomationBLOCKED
27
Metrics

• Standard test metrics does not apply so well.
• Test cases are not meaningful when running online
  tests.
• MBT tends to drive the requirement work.

28
GraphWalker QTP SOAP

• WebService("SoapServicesService").SetTOProperty
  "WSDL", "http//myComputerName9090/mbt-services?W
  SDL"If ( not WebService("SoapServicesService").Re
  load()  )Then    Reporter.ReportEvent micFail,
  "MBT failure", "MBT encountered an error. See the
  MBT log files for information."    ExitTestEnd
  IfDo until not WebService("SoapServicesService")
  .HasNextStep()    action WebService("SoapServic
  esService").GetNextStep()    If len(action) gt 0
  Then        If not Eval( action )
  Then            Reporter.ReportEvent micFail,
  "Script failure", "The script encountered an
  error when trying to run function "
  action            ExitTest        End if   
  End IfLoopReporter.ReportEvent micDone, "MBT
  Statistics", WebService("SoapServicesService").Get
  Statistics()

29
5 min break..

• I am not in the office at the moment. Please send
  any work to be translated.

30
Hands on session

1. Setup the environment
2. Create a model
3. Generate code from model
4. Automate with selenium
5. Create XML configuration file for GraphWalker
6. Run the model!

31
Files

• 1. Create directory MBT i.e C\MBT
• 2. Create directory Models under in the MBT
  folder
• 3. Copy GraphWalker and Selenium jar files to the
  directoryselenium-java-client-driver.jarselenium
  -server.jargraphwalker-2.5.1-SNAPSHOT-standalone.
  jar

32
Model

• Visit www.brickor.com/MBT.htm with Firefox
• Record with Selenium IDE by clicking on all
  elements and as well right clicking the result
• Switch off recording.
• Options gt Format gt JUnit4 (Java)

33
Model

• Start yEd to create the model

34
Generate the Java Code

• /
• This method implements the EDGE_VERTEX
  'LABEL'
• /
• public void LABEL()
• log.info( "EDGE_VERTEX LABEL" )
• throw new RuntimeException( "The EDGE_VERTEX
  LABEL is not implemented yet!" )
• java -jar graphwalker-2.5.1-SNAPSHOT-standalone.ja
  r source -f Models/mbttest.graphml -t
  java.template.txt gt javacode.txt

35
Eclipse Create the Java project

• Create new Java Project MbtTest
• Create new Java Package ie.mbt.test
• Create new Class MBTtest
• Copy paste the contents of the generated Java
  code from the model into the created class.

36
The java code

• Include references to Selenium and GraphWalker in
  your project. Right click on your project and
  choose Properties. Under Java Build Path choose
  Libraries and then click Add External JARs..
  Choose the two jar files
• graphwalker-2.5.1-SNAPSHOT-standalone.jar
• selenium-java-client-driver.jar
• private HttpCommandProcessor proc new
  HttpCommandProcessor("localhost",4444, "chrome",
  http//www.brickor.com/MBT.htm)
• private Selenium selenium new
  DefaultSelenium(proc)
• private Logger log Util.setupLogger(MBTtest.clas
  s)
• // import org.apache.log4j.Logger
• // import org.graphwalker.Util

37
Writing the Java Code

• In e_init we start the selenium object.
• selenium.start()
• selenium.open("http//www.brickor.com/MBT.htm")
• Delete the Throw exception!
• Update all e_r11, e_r12 etc
• selenium.click("Radio1_1")
• row1 Integer.parseInt(selenium.getText(r11"))

38
Writing the Java Code

• Create the utility GetResult
• public boolean GetResult()
• int rowTotal row1 row2 row3
• int calcTotal Integer.parseInt(selenium.getText(
  "totalDiv"))
• if (rowTotalcalcTotal)
• return true
• else
• return false
• IMPORTANT!
• Compile you class after any changes, CTRL B

39
GraphWalker XML

• lt?xml version"1.0" encoding"UTF-8"?gt
• lt!DOCTYPE MBTINIT SYSTEM "mbt_setup.dtd" gt
• ltMBTINIT EXECUTOR"javaie.mbt.test.MBTtest"
  EXTENDED"true"gt
• ltMODEL PATH"models/MBTTest.graphml"/gt
• ltCLASS PATH"MbtTest\bin"/gt
• ltCLASS PATH"selenium-java-client-driver.jar"/gt
• ltCLASS PATH"selenium-server.jar"/gt
• ltGENERATOR TYPE"RANDOM"gt
• ltCONDITION TYPE"TEST_LENGTH" VALUE"1000"/gt
• lt/GENERATORgt
• lt/MBTINITgt

40
Selenium Server

• java -jar selenium-server.jar
• Will start with default settings on port 4444

41
Start the scripts..

• Command line to start GraphWalker with the
  correct settings
• java -jar graphwalker-2.5.1-SNAPSHOT-standalone.ja
  r gui xml -f MBTTest.xml

42
Questions?

• More info at
• www.graphwalker.org
• mbt.tigris.org
• http//en.wikipedia.org/wiki/Model-based_testing
• Email stefan.ekman_at_gmail.com