Using Publish/Subscribe and Events to Enhance BPM Solutions

1
Using Publish/Subscribe and Events to Enhance
BPM Solutions
MIDDLEWARE SYSTEMS
RESEARCH GROUP
Hans-Arno Jacobsen Bell University Laboratory
Chair Middleware Systems Research
Group University of Toronto
http//www.padres.msrg.utoronto.ca
2
Agenda

• Motivating examples
• Business process management (BPM)
• Common denominator
• Enabler and approach
• Content-based publish/subscribe
• Content-based routing
• The PADRES ESB for event-based BPM
• Effective BPM publish/subscribe events
• Business processes execution, monitoring and
  control
• Summary and take-away

3
Business Process Example
Loan Application Processing
Store inDB

Reject
lt 0.3
Creditcheck 2
lt 0.5
Checkscore
Checkscore 2
Creditcheck
Approve
gt 0.7

Send toofficer
else

else
4
Large-scale Business Processes
Vendor
Dispatch B
Packaging
Goods selection
Goods delivery
Pick-up goods
Out-stock B
FedEx
Delivery
Pick up
Sale prediction
Sign Contract
Sale
Fill order
Determinate plan
Process
Check order
CCC administrate
Check stock
Fill out-stock bill
Manufactory
Confirm features
Design
Fill dispatch bill
Control
Determinate plan
Prototype
Out
Take
Raw materials
Execute plan
Out-stock B
Warehouse
Material
Pay
Check
Credit card
Assign
Audit
Process control
Make plan
Target price
Signature
Raw
Check dealer
Check credit
Finance
Confirm
Approval
Approval
Monitoring
Feature selection
Print receipt
Validate
Monitor
Statistic
Marketing
Requirement collection
Feedback
Affirm order
Chart
Strategy
Design
Marketing
Order
Manufactory
Payment
5
What is the Common Denominator?

• BPM applications, such as business processes, are
  driven by asynchronous state transitions.
• Something happens, an appropriate reaction is
  expected and required.
• Asynchronous state transitions represent events.
• A process is triggered, a request submitted,
• BPM applications require event management and
  processing capabilities to run effectively.

6
What Event Processing Support is Required ?

• De-coupling and loose coupling
• Fine-grained event filtering
• In-network event processing
• Composite event detection
• Event correlation

7
What Abstractions Enable Effective Event
Processing?

• It is our opinion that the afore-mentioned
  requirements can best be addressed by
• The content-based publish/subscribe paradigm
• Realized by content-based message routing
• Events represent state transitions in the
  environment.
• Conveyed as publications to the pub/sub system
• Event filtering and correlation is based on
• Subscriptions managed by the pub/sub system

8
Many Applications are Event-based Benefit from
Content-based Pub/Sub
Workflows, business processes and job scheduling
Supply chain and logistics
Job A done
In flight
Trigger
Delivered
Fault
Order
Event-Based
Callback
Razor SKU
Light
Invoke Loan
Temperature
Transform
RFID and sensor networks
Service oriented architectures
9
Publish/Subscribe 101

• Not all publish/subscribe is equal
• Publish/Subscribe models and evolution
• Channel-based
• Ex. OMG CORBA Event Service,
• Topic-based
• Ex. WS Notifications,
• Type-based
• Ex. OMG Data Dissemination Service (partially),
  
• Content-based
• Ex. The PADRES ESB (see below),
• State-based
• a.k.a., Subject Spaces (active research)

10
The Content-based Pub/Sub Model

• Language and data model
• Boolean functions over predicates
• Subscriptions are conjunctions of predicates
• Publications are sets of attribute-value pairs
• Matching semantic
• A subscription matches if all its predicates
  match
• Approximate semantic (e.g., close to, cheap,
  sunny)
• Semantic and similarity-based matching

Example Tree-structured data Graph-structured data Un-structured data Regular languages Relational model
Subscription XPath RDF Query Keywords Regular expressions SQL
Publication XML RSS feeds Text, documents Sentences over some alphabet DBs, i.e., tables
11
Content-based Message Routing
1. Advertise
3. Publish
2. Subscribe
Event-Based
Flexible
Content Routing
Decoupled
Responsive
Declarative
12
Benefits of Content-based Publish/Subscribe

• Simplifies IT development and maintenance by
  decoupling enterprise components
• Supports sophisticated interactions among
  components using expressive subscription
  languages going beyond the limits of topics
• Allows fine-grained queries and event management
• Achieves scalability with in-network filtering
  and processing

13
The PADRES ESB for Event-based BPM

• First generation of students, when I looked away
  ?
• Peng Alex David aRno Eli Serge
• PADRES is Publish/subscribe Applied to
  Distributed
• Resource Scheduling
• PAdres is Distributed REsource Scheduling
• http//www.padres.msrg.utoronto.ca

Supported and sponsored by
14
The PADRES ESB Architecture
14
2015-7-2
15
The PADRES ESB

• Is a pub/sub message broker
• Store-and-forward message queuing
• Content-based publish/subscribe interface
• Content-based message routing
• Builds a federation of brokers as overlay
• Offers a slim client library for application
  development
• Soon available under an open source license model

16
PADRES Pub/Sub Message Broker
temperature gt 37 dest2
temperature gt 40 dest3
temperature 38
temperature 42
temperature 36
17
Innovative PADRES Features
HistoricAccess
Management
CompositeEvents
Security
Robustness
LoadBalancing
18
Event Correlation and In-network Processing
Monitoring patients who needs critical attention
heart rate 150 bpm
P
medication DrugX heart rate gt 120
bpm blood pressure lt 70 mmHg
P
B
S
B
age 70
Monitoring elder patients who are losing blood
pressure
P
S
B
B
B
heart rate lt 30 temperature lt 33 blood
pressure lt 50
temperature 38C
P
S
Monitoring patients having a fever but not having
a cold
B
temperature 40C
heart rate gt 140 bpm temperature gt 39
illness ! cold
B
P
P
illness diabetes
blood pressure 60 mmHg
19
Illustration of In-network Processing
20
Modeling Business Processes

• Dependency in processes and more complex process
  patterns requires event correlation
• Event correlation enabled by the detection of
  composite events
• Composite events are expressed via composite
  subscriptions
• Composite subscription consists of atomic
  subscriptions
• Subscription language features for BPM modeling
• Operators (AND, OR) and variables (x)
• Example D executes, if B and C
• have completed (D depends on
• B and C)

B
C
D
21
Business Process Execution

• Transformation of process into pub/sub language
  and data model
• Deployment of the transformed process
• Triggering and executing the process instances
• Monitoring the process execution

trigger multiple instances concurrently
trigger
Exception compensation
A
B
C
D
22
Monitoring Client

• Monitor is a Pub/Sub client
• Visualizes topology of broker overlay network
• Visualizes message routing
• Monitor/control process execution
• Process level
• Activity level
• Supports dynamic process modifications

23
PADRES on PlanetLab
24
The PADRES ESB in RD

• Develop flexible BPM concept
• With CA Labs, Sun Microsystems, OCE NSERC
• Develop SLA-based autonomic BPM concept
• With IBM and NSERC
• Evaluate and experiment with the use of the
  PADRES ESB for application integration
• With Bell Canada and BUL

Acknowledgements
25
Benefits of Content-based Publish/Subscribe for
BPM

• Naturally enables centralized and distributed
  business process coordination
• Coordination can span administrative domains and
  physically distributed resources
• Supports process orchestration and choreography
• Monitoring control is integral part of paradigm
• Agile on the fly process adaptation and
  versioning
• Correlation of application events with low-level
  infrastructure events

26
The PADRES ESB Architecture
26
2015-7-2
27
Summary

• Many applications are inherently event-driven.
• Effective BPM requires capable event processing
  abstractions.
• Content-based publish/subscribe is a powerful
  event processing abstraction and paradigm.
• PADRES is based on the pub/sub paradigm.
• PADRES is an ESB targeted at event-based BPM.
• PADRES enables real-time business analytics and
  business activity monitoring.

28
References
The DEBS Conference http//www.debs.org July 2008
in Rome, Italy

• The PADRES ESB project home
• http//padres.msrg.utoronto.ca
• An eQoSystem for declarative distributed
  applications with SLAs
• http//research.msrg.utoronto.ca/Eqosystem/
• The Micro-ToPSS event processing middleware for
  sensor networks
• http//microToPSS.msrg.utoronto.ca/
• Mobile-ToPSS publish/subscribe for mobile and
  location-based applications
• http//research.msrg.utoronto.ca/Mobile/
• ToPSS - the Toronto Publish/Subscribe System
  Family
• http//www.ToPSS.biz (coming soon ? )
• Quantifying events in software to increase
  modularity customization in C-based systems and
  software-based product lines
• http//www.AspeCtC.net (ACC - the AspeCt-oriented
  C compiler)
• The Middleware Systems Research Group
• http//www.msrg.utoronto.ca
• My web site
• http//www.eecg.toronto.edu/jacobsen

_at_ the University of Toronto
29
Acknowledgements

• Graduate students, visitors, and PDFs currently
• working on PADRES.
• Alex Cheung
• Guoli Li
• Jian Li
• Vinod Muthusamy
• Alex Wun
• Songlin Hu
• Reza Sherafat
• Partner from CA directly involved in project
• Serge Mankovskii

30
Questions?
31
Additional slides
32
The Toronto Publish/Subscribe System Family
(ToPSS)
A-ToPSS (approximate)
ToPSS (matching)
CS-ToPSS (composite subs)

• Matching algorithms
• Language expressiveness vs. efficient matching
• Routing protocols
• Network architectures scalability
• Higher level abstractions
• Process execution
• Monitoring

S-ToPSS (semantic)
L-ToPSS (location-based)
Rb-ToPSS (rule-based)
X-ToPSS (XML matching)
persistent-ToPSS (subject spaces)
M-ToPSS (mobile)
P2P-ToPSS (peer-to-peer)
LB-ToPSS (load balancing)
Ad hoc-ToPSS (ad hoc networking)
Federated-ToPSS (federation of ToPSS brokers)
FT-ToPSS (fault tolerance)
Historic-ToPSS (historic data)
BPEL-ToPSS (BPEL execution)
JS-ToPSS (job scheduling)
33
Historic Query Processing
34
Historic Query with Event Correlation
Monitoring patients who still have fever after
taking Advil
heart rate 150 bpm
P
Medical records database
medication NoNameDrugX
medication NoNameDrugX blood pressure gt
100 heart rate gt 130
blood pressure gt 100
B
heart rate gt 130
S
B
medication NoNameDrugX
Monitoring patients who still have high blood
pressure after taking NoNameDrugX
P
S
B
B
B
temperature 38C
temperature gt 42 medication Advil
P
S
Monitoring patients having a fever but not having
a cold
B
temperature 40C
heart rate gt 140 bpm temperature gt 39
illness ! cold
B
P
P
illness diabetes
blood pressure 60 mmHg
35
Experimental results
36
Evaluations

• Experimental Setup
• Two Intel Xeon 3GHz PCs with 2GB RAM
• Two Intel Xeon 1.7GHz PCs with 512MB RAM
• A broker network with 12 brokers and 10 agents
• Processes with 10 parallel branches
• Metrics
• Average process execution time
• Average system throughput

37
Experimental Setup
38
Highly Parallel Process
39
Varying Request Rate

• Improvement over centralized scenario
• Also for lower request rates (networking
  negligible)

40
Varying Request Rate

• Improvement also in terms of throughput (130
  increase in max. th.put)

41
Varying Service Delay

• Longer external service delays, lead to more
  concurrent instances
• being handled, require more resources,
  therefore longer delay

42
Varying Message Size
43
Effect of Parallelism