Querying and Monitoring Business Processes Tova Milo TelAviv University

1
Querying and Monitoring Business ProcessesTova
MiloTel-Aviv University

2
Thanks Catriel Beeri, Anat Eyal, Simon
Kamenkovich, Daniel Deutch, Alon Pilberg, Tal
Sterenzy, Eran Balan,
3
Outline

• Introduction and motivation
• Overview of BP-QL by example (Querying)
• - Formal model
• - Compact representation of query answers
• - Implementation
• Overview of BP-Mon by Example (Monitoring)
• - Formal model
• - Optimization
• - Implementation
• Summary

4
The Standard Example
Credit Service
Consolidate Results
PO Service
Client
Inventory Service
5
Web Services Meet Business Processes
Local to company A
Web Service 1
Web Service 4
Web Service 2
Web Service 5
Web Service 3
Web Service n
At company B
On the Web
Company A business process
6
Recent History of Business Process Standards
BPML (Intallio et al)
BPSS (ebXML)
WS-Choreography (W3C)
WSCI (Sun et al)
2000/05
2001/03
2001/05
2001/06
2002/03
2002/06
2003/01
2003/04
2002/08
XLang (Microsoft)
WSFL (IBM)
BPEL4WS 1.0 (IBM, Microsoft)
BPEL4WS 1.1(OASIS)
WSCL (HP)
BPEL
7
BPEL in a nutshell

• Business process -- control and data flow
• Actions
• atomic
• compound fork, join, while, procedure/process
  call,
• Process spec. represented in XML
• Commercial products
• design tools with conceptual model
• Application generators compile to executable code
• Application servers execute the code

8
Points to note

• 1) Not just a modeling language
• applications are automatically generated from
  the
• (relatively declarative) specs.
• 2) There will be plenty of such specs around
• 3) Specs are data, a lot of interest in querying
  them
• What kind of credit services are used
  (in)directly?
• How can I buy a plane ticket ?
• Can one get a price quote without giving
  first credit card info?
• 4) Can also use queries for monitoring
• Notify me if the above nevertheless
  happens
• 5) Querying audit trails (logs)
• ?
  motivation for developing a query facility

9
Can a query language be based on BPEL?

• NO!
• The BPEL XML representation is machine oriented,
  unfit for human consumption
• It decomposes the system diagram into a
  (XML-based) relational representation of
  nodes and edges
• ? loss of understanding queries require lots
  of joins
• Need Similarity of system and query models

10
Why not XQuery?(on XML representation of
conceptual model)

• Additional requirements from query language
• - (Nested) graphs rather than trees
• - Queries about control data flow paths
• - Need to query at different level of granularity
• Zoom-in/zoom-out
• - P2P architecture Specs are distributed
• ? We use a home-brewed QL

11
Outline

• Introduction and motivation
• Overview of BPQL by example (Querying)
• - Formal model
• - Compact representation of query answers
• - Implementation
• Overview of BP-Mon by Example (Monitoring)
• - Formal model
• - Optimization
• - Implementation
• Summary

12
A system specification

• Property, data activity nodes
• Control and data flow edges
• A process is a di-graph of atomic compound
  activities
• Compound activities can be zoomed-in (a-la
  statecharts)

13
Travel Agency Process
14
Zoom In
15
BPQL Queries

• Use process patterns (like tree patterns for
  xml)
• Single/double-headed edges (compare to / and //
  in XPath)
• edges
• paths of arbitrary length
• Single/double-bounded activities
• w/o zoom-in
• unbounded zoom-in
• Node label variables
• Mark requested nodes/edges by

16
Query1 provided operations?
17
Query2 used credit card services?
local
18
Query3 search without login?
19
Query4 data flow
Data elements affected by searchRequest and
affecting returnTripResults
20
Outline

• Introduction and motivation
• Overview of BPQL by example (Querying)
• - Formal model
• - Compact representation of query answers
• - Implementation
• Overview of BP-Mon by Example (Monitoring)
• - Formal model
• - Optimization
• - Implementation
• Summary

21
Formal Model

• System
• Process graphs implementation function
• Graph refinement (graph rewriting)
• Query
• A system where some nodes edges are marked
  transitive

22
Queries and their semantics

• Query A system where some nodes edges are
  marked transitive
• An embedding a mapping
• from query graphs
• to refinements of process graphs
• Satisfying conditions
• (nodes) preserves nodes types and labels
• (edges) edge are mapped to edges
• transitive edge mapped to
  a paths
• (imp) preserves implementation
  relationships
• A result image of query graph under an
  embedding
• Answer all results

23
Structural vs. Behavioral Semantics

• Sub-graph homomorphism vs. bisimulation

24
Outline

• Introduction and motivation
• Overview of BPQL by example (Querying)
• - Formal model
• - Compact representation of query answers
• - Implementation
• Overview of BP-Mon by Example (Monitoring)
• - Formal model
• - Optimization
• - Implementation
• Summary

25
Large or Infinite answers!

• Many fork/joins ? exponential number of paths

Cycles in process graph Cycles (recursion) in
zoom-in relationships ? Infinite of
refinements
26
Recursive zoom-in
27
A query with an infinite answer
28
Finite representation
29
Before presenting our algo
lets talk about related work

• Model
• BPQL (S)Recursive State Machines
• lt (VR) CF
  Graph Grammars Equaltional Sets
• Query language
• Behavioral semantics expressible in CTL
  and mu-calculus
• In general Can be described as a Regular Set
  
• Possible algos
• CTL on Recursive State Machine (double
  exponential in Q)
• Intersection of Regular sets and Equaltional Sets
  (non elementary)

30
Query Evaluation Algorithm

• Systems and queries are essentially
• (Vertex Replacement) Context
  Free Graph Grammars
• Bad news ?
• These are not closed in general under
  intersection
• Good news ?
• Our systems and queries are sufficiently simple
  Psize
  representation (as a grammar) can be computed in
  Ptime (data complexity!)
• Complexity S2 2Q MATCH(Q,S)
• NP Complete in Q (even when MATCH is in PTime)

31
Extensions

• OK extensions
• label predicates ,
• Regular path expressions (on node labels),
• Negation,
• Joins on node labels
• Not OK extensions
• Joins on path variables
• The result is not a VR context free graph grammar
• Emptiness is undecidable
• NP-hard (data complexity) even without cycles
  and recursion

32
Outline

• Introduction and motivation
• Overview of BPQL by example (Querying)
• - Formal model
• - Compact representation of query answers
• - Implementation
• Overview of BP-Mon by Example (Monitoring)
• - Formal model
• - Optimization
• - Implementation
• Summary

33
First Attempt

• Translation to XQuery queries over the BPEL XML
• Problem too many joins, because of the
  relational representation in BPEL

nodes
process
variables
descrription

var1
links
confirmTrip
searchTrip
reserveTrip
cancelTrip
start
end


link

source
link
description
target
description
source
source
description
target
source



L1
L5
L2
L2
L4
L1
L1
L3
edges
34
Solution use Active XML (AXML)
Travel agency
serviceProvider
capability
behavior
zoomin
start
Embedded calls to web services
searchTrip
zoomin
reserveTrip
start
confirmTrip
cancelTrip
getTripRequest
xmlsc
end
fork
getActivity(end)
searchFlights
searchCars
searchRooms
join
getActivity(join)
getActivity(join)
zoomin
end
getOperation(searchRooms,join)
zoomout
getActivity(reserveTrip)
35
Architecture
BPQL
specs
BPQL Process Query Editor
Result Viewer
BPEL to AXML
results
queries
BPQL to XQuery
Post Processor
BPEL peer
AXML peer
SOAP
query
BPEL to AXML
Query processor
Evaluator
BPQL peer
read update
SOAP wrapper
read update
consults
SOAP
AXML
service descriptions
BQueL peer
AXML
SOAP
AXML document store
36
Results
37
Distribution Effect
38
Outline

• Introduction and motivation
• Overview of BPQL by example (Querying)
• - Formal model
• - Compact representation of query answers
• - Implementation
• Overview of BP-Mon by Example (Monitoring)
• - Formal model
• - Optimization
• - Implementation
• Summary

39
Monitoring BPs with Queries

• Imagine you run an auction service
• Guarantee fair play notify on too many cancels
• Maintain SLA monitor response time
• Promotions prizes for the x10,000 transaction
• Illegal access notify on buyers attempt to
  confirm bids without registering first
• And more
• A new opportunity for monitoring BPs

40
Audit Trail
ltactionDatagt ltheadergt ltprocessNamegt
auctionHouse lt/processNamegt ltinstanceIdgt
517 ltinstanceIdgt ltsensor targetnotify_winn
er/gt lttimestampgt 2006-05-31T113246.51000
00 lt/timestampgt lt/headergt
ltactivityDatagt ltactivityTypegtinvoke
lt/activityTypegt ltevalPointgt completion
lt/evalPointgt
41
Requirements

• Query Language
• Visual and intuitive
• Execution patterns
• Describe sequential parallel execution
• Flexible granularity
• Easy deployment

42
Why not use existing XML streaming tools?

• XML tools assume the stream comes in document
  order.
• This is not the case here!
• DAGs vs. trees
• Zooming in/out to compound activities
• Distribution

43
Contribution

• Query language
• Visual and intuitive with tight analogy to the
  spec.
• Flexible description of execution patterns
• (sequential parallel executions, and/or,
  repetitions).
• Reporting facility
• (notifications, reports, invocation of
  corrective actions).
• Implementation
• Easy deployment, portability
• Compiles a BP-Mon query into BPEL process
• Query evaluation and optimization
• Defines the notion of first match of a query
• Prunes redundant monitoring

44
Query Example (1)

• Unfair play (too many cancellations)

Report/ Report
45
Query Example (2)

• Monitor response time of datastore

• Sliding window
• Time based
• Report Every 1 hrs Range 2 hrs
• Instance based
• Every 100 entries Range 200

• Output
• Xquery like
• Group by, having

46
Query Example (3)

• Prizes to the X10000 deal

47
Query Example (4)

• Illegal bidding (mix of static and run-time
  analysis)

48
Outline

• Introduction and motivation
• Overview of BPQL by example (Querying)
• - Formal model
• - Compact representation of query answers
• - Implementation
• Overview of BP-Mon by Example (Monitoring)
• - Formal model
• - Optimization
• - Implementation
• Summary

49
Execution trace as DAG

• Nodes
• Activation
• Completion
• Timestamps
• Edges
• Flow
• Zoom-in, Zoom-out

a
c
50
Execution Pattern

• Execution trace with or and rep (and
  transitivity)
• Defines a set of concrete patterns that can be
• obtained by

• Or choosing some internal trace
• Rep replacing by some copies

51
Semantics

• Embedding of a concrete pattern into a trace
• a mapping of nodes and edges,
• satisfying the natural constraints
• Greedy embedding
• the (concrete) embedding that ends first ?
• also
• each node is matched to the earliest
  possible event

52
The Algorithm

• Execution pattern gt (lazy) DFA
• with some
  parallelism
• Incrementally extends a greedy embedding to
• one of a
  larger prefix
• On failure, backtracks retries
• Issues
• Backtracking time and memory

53
Optimization

• Inferring Irrelevancy inconsistency using BP-QL

54
Outline

• Introduction and motivation
• Overview of BPQL by example (Querying)
• - Formal model
• - Compact representation of query answers
• - Implementation
• Overview of BP-Mon by Example (Monitoring)
• - Formal model
• - Optimization
• - Implementation
• Summary

55
Architecture
56
Visual Interface
57
Report
58
Experiment Results
Queries overhead
Varying of queried activities
Impact of optimization
Varying of queries per process
59
Outline

• Introduction and motivation
• Overview of BPQL by example (Querying)
• - Formal model
• - Compact representation of query answers
• - Implementation
• Overview of BP-Mon by Example (Monitoring)
• - Formal model
• - Optimization
• - Implementation
• Summary

60
Summary

• Simple and intuitive formulation of queries
  monitoring tasks
• similar to how processes are specified
• two navigation axis
• flexible granularity
• path retrieval
• Operates in distributed environment
• System and queries modeled as graph grammars
• ? allows compact representation of
  large/infinite answers
• Ignores semantics of some BPEL constructs and
  data values
• reasonable tradeoff of expressivity and
  complexity
• Monitoring tasks are modeled as regular (graph)
  expressions
• Easy deployment by translation to BPEL
  processes

61
Ongoing and future work

• Incomplete information
• Querying/mining logs
• Process integration
• Applications to software verification
  monitoring
• Capturing more BPEL semantics
• Optimization

62
???? ( Thanks)