Coverage Criteria for Testing of Object Interactions in Sequence Diagrams

1
Coverage Criteria for Testing of Object
Interactions in Sequence Diagrams

• Atanas (Nasko) Rountev
• Scott Kagan
• Jason Sawin
• Ohio State University

2
UML Sequence Diagrams

• Shows messages exchanged among objects
• Contains control information relevant to message
  passing

3
Sequence Diagrams and Testing

• Object interaction testing
• Coverage criteria for sequence diagrams
• all paths, all branches, etc.
• Most existing work utilizes the all paths
  criterion

4
Sequence Diagrams and Testing

• Most existing work focuses on design time
  diagrams
• Diagrams can also be reverse engineered from code
• more paths due to less abstraction
• Do existing sequence diagram based testing
  approaches generalize to reverse engineered
  diagrams?
• should the all paths criterion be considered?

5
Our Work

• Investigated 4 different coverage criteria for
  reverse engineered sequence diagrams
• defined each criterion in terms of an RCFG tree
• developed a method for comparing criteria
• performed a comparison of the criteria on real
  world Java code

6
RCFG Tree

• Restricted Control Flow Graph (RCFG)
• similar to a traditional CFG
• for message passing only
• RCFG tree represents a sequence diagram
• set of all start to end paths for both structures
  is the same
• Root RCFG is top level method
• Edges connecting RCFGs represent method
  invocations
• RCFG n is a child of RCFG m means method m may
  invoke method n

7
m1
aA
dD
cC
bB
start
m1
m6
m2
m2
c1 m3
start
m4
m6
m2
c2 m5
end
end
c3 m6
m2
c1 m3
m2
m2
m4
start
m3
start
m3
c2 m5
m5
m4
m5
m4
m4
end
end
start
end
8
RCFG Tree Details

• Straightforward to construct when
  reverse-engineering sequence diagrams
• Can be traversed at run time to collect coverage
  statistics

9
Coverage Criteria Considered
m1
m6

• All start to end paths
• All RCFG paths
• All RCFG branches
• All unique branches

start
start
m2
m2
m6
end
m2
m2
end
start
m3
start
m3
m5
m4
m5
m4
end
end
10
Criterion Comparison Technique

• Each start to end path corresponds to one test
  case
• For each criterion, calculate minimum number of
  start to end paths required to achieve complete
  coverage
• indirect estimate of testing effort

11
Minimum Number of Paths
m1
m6

• All start to end paths 20
• All RCFG paths 5
• All RCFG branches 3
• All unique
• branches 2

start
start
m2
m2
m6
end
m2
m2
end
start
m3
start
m3
m5
m4
m5
m4
end
end
12
Calculating Minimum Number of Paths

• All start to end paths
• calculate the number of paths from the start node
  of an RCFG to the end node of the RCFG
• must consider paths in children
• start with the leaves, propagate up to the root
• All RCFG paths, all RCFG branches, all unique
  branches
• integer linear programming

13
Valid Path Property

• If node n represents a method call and edge (m,
  n) is part of a path p,
• p must include an inter-method edge (n, start)
• p must include an intra-method edge (n, r)

m
n
start
r
q
14
Valid Path Property

• If node n represents a method call and edge (n,
  start) is part of a path p,
• p must include an intra-method edge (start, q)

m
n
start
r
q
15
RCFG Branches Property

• Consider some set of paths S that satisfies the
  RCFG branch property
• every RCFG edge appears in at least one path

m
n
start
r
q
16
Integer Linear Programming Formulation

• ILP equations encode the valid path property and
  the RCFG branches property
• variables are edge frequencies
• constraints
• each frequency is at least 1
• frequency sums are equal
• (red, green, blue)

m
n
start
r
q
17
Proof Obligation

• Solution to the integer linear programming
  problem is a frequency count for each edge
• Need to prove
• There exists a set of paths that meets the
  coverage criterion and has the same edge
  frequencies
• No smaller set of paths satisfies the criterion

18
Remaining Criteria

• Integer linear programming formulation
• valid paths property
• a criterion specific property
• All unique branches
• the sum of the edges frequencies for equivalent
  edges is at least one
• All RCFG paths
• each edge is covered at least as many times as
  its local edge frequency

19
Experimental Study

• Implemented as part of the RED tool (Reverse
  Engineering of sequence Diagrams)
• Implementation used lp_solve linear programming
  tool
• Input code
• code from the Java standard libraries
• 1104 RCFG trees
• average 36 RCFGs per tree

20
(No Transcript)
21
Future Work

• Implement dynamic analyses for obtaining coverage
  statistics
• Measure coverage achieved by real world test
  suites