BPM in Cloud Architectures: Business Process Management with SLAs and Events - PowerPoint PPT Presentation

Loading...

PPT – BPM in Cloud Architectures: Business Process Management with SLAs and Events PowerPoint presentation | free to view - id: 696899-MzgxM



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

BPM in Cloud Architectures: Business Process Management with SLAs and Events

Description:

Title: PADRES A Content-based Pub/Sub System Author: Li Guoli Last modified by: jacobsen Created Date: 8/12/2004 5:46:01 PM Document presentation format – PowerPoint PPT presentation

Number of Views:36
Avg rating:3.0/5.0
Slides: 93
Provided by: LiGu3
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: BPM in Cloud Architectures: Business Process Management with SLAs and Events


1
BPM in Cloud Architectures Business Process
Management with SLAs and Events
BPM 2010, Hoboken, NJ
Hans-Arno Jacobsen Bell University Laboratory
Chair University of Toronto
http//www.padres.msrg.utoronto.ca
An eQoSystem for declarative distributed
applications with SLAs
http//eqosystem.msrg.org/
2
Business Process Example
Loan Application Processing
service RTT lt 100ms uptime gt 99.99
Store in DB

service time lt 2s
Reject
lt 0.3
gidc001
Credit check 2
lt 0.5
  • Pidc1

Pidc001
Pidc3301
c001 end
Pidc401
gidc001
Check score
Check score 2
Credit check
Approve
service cost lt 0.02
gt 0.7

Send to officer
else

else
3
Large-scale Business Processes
Vendor
Dispatch B
Packaging
Goods selection
Goods delivery
Pick-up goods
  • Case Study (Chinese Electronics Manufacturer)
  • Department-level processes with 26 to 47
    activities
  • Global processes that compose departmental ones
  • Thousands of concurrent instances
  • Hundreds of collaborating partners
  • Geographically distributed
  • Administrative boundaries

Out-stock B
FedEx
Delivery
Pick up
Sale prediction
Sign Contract
Sale
Fill order
Determinate plan
Process
Check order
CCC administrate
Check stock
Fill out-stock bill
Manufactory
Confirm features
Design
Fill dispatch bill
Control
Determinate plan
Prototype
Out
Take
Raw materials
Execute plan
Out-stock B
Warehouse
Material
Pay
Check
Credit card
Assign
Audit
Process control
Make plan
Target price
Signature
Raw
Check dealer
Check credit
Finance
Confirm
Approval
Approval
Monitoring
Feature selection
Print receipt
Validate
Monitor
Statistic
Marketing
Requirement collection
Feedback
Affirm order
Chart
Strategy
Design
Marketing
Order
Manufactory
Payment
4
What Support is Required ?
  • De-coupling and loose coupling
  • Fine-grained event filtering
  • In-network event processing
  • Composite event detection
  • Event correlation

5
Conflicting Goals
End user Performance
Administrator Cost, utilization
Developer Flexibility, simplicity
BPM 2010, Hoboken, NJ
6
Agenda
  • Enabler
  • The PADRES approach
  • Event-driven BPM with PADRES
  • SLA-driven BPM with PADRES

7
What Abstractions Enable BPM?
8
What Abstractions Do Not Work?
Cum grani salis
  • Databases
  • Great for managing historic data
  • But what about future data (e.g., events)
  • Data streams
  • Great for managing structured streams of tuples
  • But what about un-structured, multi-typed,
    sporadic, un-ordered events from many sources
  • Rule-based expert systems
  • Great for inference and reasoning
  • But what about managing large numbers of
    fined-grained filters in distributed environments

9
What Abstractions Enable BPM?
  • It is our opinion that the afore-mentioned
    requirements can best be addressed by
  • The content-based publish/subscribe paradigm
  • Realized by content-based message routing
  • Events represent state transitions in the
    environment.
  • Conveyed as publications to the pub/sub system
  • Event filtering and correlation is based on
  • Subscriptions managed by the pub/sub system

10
Content-based Publish/Subscribe
Content-based Routing
3. Publish
Publisher
Publisher
Publications
Service credit check
1. Advertise
class Loan,
status approved,
Service RTT lt 2s
amount gt 500K

10
Uptime gt 99.99
Subscriptions
Service RTT lt 150ms
2. Subscribe
Subscriber
Subscriber
11
Application Modeling
  • Advertisements (schema or types)
  • classLoan, action, customerID,
    amountlt100K, regionEast
  • Publications events (data)
  • class, Loan, action, request,
  • customerID, 876594, amount, 50K
  • Subscriptions (query)
  • classLoan, amountgt500K, region
  • Application semantics is expressed via
  • advertisements (data sources), publications
  • (data sources), and subscriptions (data sinks)

S
A
P
12
Benefits of Publish/Subscribe
  • Simplifies IT development and maintenance by
    decoupling enterprise components
  • Supports sophisticated interactions among
    components using expressive subscription
    languages
  • Supports fine-grained subscriptions for event
    processing
  • Achieves scalability with in-network filtering
    and processing

13
Many Applications are Event-based Benefit from
Publish/Subscribe
Workflows, business processes and job scheduling
Supply chain and logistics
Job A done
In flight
Triggered
Delivered
Fault!
Ordered
Event-Based
Callback
RFID Razor SKU
Light
Invoke Loan
Temperature
Transform
RFID and sensor networks
Service oriented architectures
14
Our Approach
SLA model
An SLA model can communicate goals to cloud
infrastructure providers.
Process monitoring
Distributed execution
Service selection
An event-based system is an enabling technology
for a number of the listed features.
Content-based routing messaging middleware
14
15
Our PADRES ESB for Event-driven BPM
Enterprise Services Bus (Events Services Bus)
  • First generation of students, when I looked away
    ?
  • Peng Alex David aRno Eli Serge
  • PADRES is Publish/subscribe Applied to
    Distributed
  • Resource Scheduling
  • PAdres is Distributed REsource Scheduling
  • http//www.padres.msrg.utoronto.ca

Web start open source padres.msrg.org Implement
ed in Java
Acknowledgements (2004-present)
15
16
Our PADRES ESB Stack Vision
17
PADRES ESB is an Overlay of P/S Brokers
S
P
service time lt 3s dest2
service time lt 2s dest3
service time 2.5s
service time 3s
service time 1s
S
P
publisher subscriber
Try out and download at http//www.padres.msrg.or
g
S
18
Innovative PADRES Features
Historic Access
CSRG TR 2009 ACM DEBS2007
Management
Composite Events
ACM Middleware2007
ACM Middleware2004 IEEE ICDCS2005
IEEE ICDCS2009 ACM Middleware2008
ACM DEBS2007
Security
Robustness
IEEE ICDCS2010 ACM Middleware2006
Load Balancing
19
Event-driven BPM with PADRES
But not as complex
20
Modeling Business Processes
  • Dependency in processes and more complex process
    patterns require event correlation
  • Event correlation is enabled by the detection of
    composite events
  • Composite events are expressed via composite
    subscriptions
  • Composite subscription consists of atomic
    subscriptions
  • Subscription language features for BPM modeling
  • E.g., AND, OR, and variables (x)
  • Example D executes, if B and C
  • have completed (D depends on
  • B and C).

21
Composite Subscription Examples
  • Expresses a structural property of a process
  • class Activity Status, cmd.
    Archived,Process ID X AND
  • class Activity Status, cmd. Signed
    Off,Process ID X
  • Expresses a performance property of a process
  • cmd. Credit check request, Process ID X
    AND
  • status Approved, Process ID X


Loan Process
Process
Archive
Approve
gt 50K
Check score
Credit check

Singed Off
Reject
22
Business Process Management
  • Transformation of process into pub/sub language
  • Deployment of transformed process
  • Execution of process by triggering instances
  • Monitor process instance execution
  • Manage, i.e., control, version,

trigger multiple instances concurrently
trigger
Exception compensation
A
B
C
D
23
Business Process Execution
END
WS Gateway Agent
WS client
PADRES ESB
6
Invoke
1
3
4
Web Service
2
5
Receive
Assign
Web Service
Reply
Wait
http/soap
pub/sub
24
BPEL Transformation Example
Sub1 (flow agent) class ACTIVITY_STATUS,
process Process5, activityName
activity1, IID isPresent any, status
SUCCESS
Sub4 (actvity6 agent) ( class
ACTIVITY_STATUS, process Process5,
activityName activity5, IID X,
status SUCCESS ) AND ( class
LINK_STATUS, process Process5,
activityName activity2, IID X,
status isPresent any )
Sub1 class,eq,ACTIVITY_STATUS, process,eq,Pro
cess5, activityName,eq,activity1, IID,isPre
sent,any, status,eq,SUCCESS
Pub3 class,LINK_STATUS, process,Process5,
activityName,actitiy2, IID,g001, status,P
OSITIVE
Sub5 class,eq,ACTIVITY_STATUS, process,eq,Pro
cess5, activityName,eq,activity4, IID,isPre
sent,any, status,eq,SUCCESS class,eq,ACTI
VITY_STATUS, process,eq,Process5, activityNa
me,eq,activity7, IID,isPresent,any, status,e
q,SUCCESS
Process 5
activity1
Sub4 class,eq,ACTIVITY_STATUS, process,eq,Pro
cess5, activityName,eq,activity2, IID,eq,X
, status,eq,SUCCESS class,eq,LINK_STATUS
, process,eq,Process5, activityName,eq,activ
ity2, IID,eq,X, status,isPresent,any
Pub1 class, ACTIVITY_STATUS, process,Process5
, activityName,flow1, IID,g001, status,
STARTED
flow1
activity2
activity5
activity3
activity6
Sub2 class,eq,ACTIVITY_STATUS, process,eq,Pro
cess5, activityName,eq,flow1, IID,isPresent
,any, status,eq,STARTED
Pub5 class, ACTIVITY_STATUS, process,Process5
, activityName,actitiy7, IID,g001, stat
us,SUCCESS
activity4
activity7
activity8
Pub2 class, ACTIVITY_STATUS, process,Process5
, activityName,actitiy2, IID,g001, stat
us,SUCCESS
Pub4 class, ACTIVITY_STATUS, process,Process5
, activityName,actitiy6, IID,g001, stat
us,SUCCESS
Cf. our ACM Trans Web2010 for full BPEL mapping
25
Evaluation Changing Request Rate
26
SLA-driven BPM
More organized!
An eQoSystem for declarative distributed
applications with SLAs
http//eqosystem.msrg.org/
27
  • Currently, business goals must be manually
    considered at every stage of the business process
    development cycle

Only trusted partners
service time lt 3s
Find flight
Y
Far?
Validate request
Get destination
Find train
N
cost lt 0.02
28
Service Level Agreements (SLAs)
SLAs are contracts between service consumers and
providers that specify the expected behavior of
each party and the penalties of violating the
contract. SLAs specify business goals
declaratively.
Layer SLA Metric Example
Business Process Cost Cost of search service lt 0.02 per use
Business Process Fidelity, quality, utility Map resolution gt 300x300
Business Process Trust, reputation Only use trusted payment services
Deployment Operations Service time Execution time lt 3s
Deployment Operations Throughput Throughput gt 100/min
Deployment Operations Availability Uptime gt 99.9
Deployment Operations Load balance Server utilization delta lt 10
Network Latency Service RTT lt 100ms
Network Bandwidth Max bandwidth lt 3Mbps
Network Jitter Delay jitter lt 10ms
29
Runtime Uses of SLAs
Process
Dynamic service discovery Discover services with
capabilities that satisfy goals. Monitoring Only
monitor the business events related to
goals. Feed back measurements to support runtime
adaptations. Distributed execution Fine-grained
allocation of process to available
resources. Move portions of process to strategic
locations. ESB adaptation Reconfigure the ESB
topology to satisfy goals.
ESB broker topology
C

A,B
service time lt 2s
1a
M
service time lt 1s
1b
D
2
Web service
Execution engine
ESB node (PADRES broker)
M
Monitor
30
Distributed Execution Example
  • Execution of one process
  • instance incurs 5 events
  • 3 from P to T
  • 2 from T to S
  • In practice many concurrent
  • instances (a rate)

Process
Activity
P
T
S
Execution engine hosting activity P
ESB broker topology
T
2
5
8
P

1
4
7

S

3
6
9



Execution engine
Broker
31
SLA Cost Function Example
  • SLA is minimize traffic
  • SLA metric is message hops
  • Minimize message hops
  • f() msg_hops_rate msg_rate
    engine_distance
  • Measure the msg_hops_rate between activities
  • Measure the distance between execution engines
  • Strive to find better and better deployments
    subject to execution constraints

T
2
5
P
1
4
32
SLA Management Stack
Execution Engine
Candidate Engine Discovery (ACM DEBS2009)
Activity Profiler
Engine Profiler
Activity Manager
Latency
Bandwidth
Engine resource
Activities (ACM TWEB2010)
Instance states
Atomic Redeployer (IEEE ICDCS2009)
Input, output queues
PADRES messaging layer
33
Cost Model Components
The cost of a process is based on metrics
1. Distribution cost Cdist f(Cd3, Cd1,
Cd2) Cd1 Message rate Cd2 Message size Cd3
Latency 2. Engine cost Ceng f(Ce1,
Ce2,Ce3) Ce1 Load (number of instances) Ce2
Resources (CPU, memory, etc.) Ce3 Activity
complexity
3. Service cost Cserv f(Cs1, Cs2, Cs3) Cs1
Latency of external service Cs2 Execution time of
external service Cs3 Marshalling/unmarshalling C
ost(activity) f(wiCi) Cost(process)
?cost(activity)
These metrics can be weighted to achieve
different objectives
Optimize time wd1 wd3 we3 wserv 1, other
wi 0
Optimize network overhead wd2 wd3 1, other wi
0
Various optimization criteria can be specified
Threshold criteria ?wiCi gt x E.g., Report SLA
violations within 3 s.
Minimized criteria min( ?wiCi ) E.g.,
Minimize distribution overhead
34
Activity Profiler
Example
  • Profiles execution of local activities
  • Maintains profiles for various metric types
  • Message hops, disk I/O, energy usage, etc.

Profile for message hops metric Profile for message hops metric Profile for message hops metric
Activity Pred/Succ Msg rate
T P 40/min
T S 10/min
Process
Activity
P
T
S
ESB broker topology
Profile for execution time metric Profile for execution time metric
Activity Exec time
T 0.3 s
T
2
5
8
P

1
4
7

S

Profile for memory usage metric Profile for memory usage metric
Activity Mem
T 30 MB
3
6
9



Execution engine
Broker
35
Engine Profiler (e.g., distance)
  • Discover paths
  • e2?e5 , e5?e7
  • Probe paths
  • e5?e4 (for candidate C)
  • Compute paths
  • e2?e4 , e4?e7
  • Cases
  • e4 in e2?e5
  • e4 in e5?e7
  • otherwise
  • Computes and caches information about candidate
    engines
  • Cf. DEBS2009 for our resource discovery
    algorithms to identify candidates

Process
P
T
S
Discovered paths Discovered paths Discovered paths Discovered paths
Source Target Distance Disc _at_
2 5 3 148pm
5 7 2 149pm
T
2
5
8
P

1
4
7

S
Probed paths Probed paths Probed paths Probed paths
Source Target Distance Probe _at_
5 4 1 134pm
C
3
6
9



Computed paths Computed paths Computed paths Computed paths
Source Target Distance Comp _at_
2 4 2 202pm
4 7 1 204pm
Execution engine
Broker
36
Redeployment Manager
  • Estimator Computes an estimate of the metric
    cost ck(ai,ej) of hosting an activity ai at
    engine ej
  • Cost model Computes an estimate of the cost
    c(ai,ej) of hosting activity ai on engine ej
  • Check deployment Determines what to do with an
    activity ai
  • Determine best engine e
  • Compute benefit c(ai) c(ai,e)
  • Compute resident time at current engine
  • If resident long enough
  • If benefit is large enough move ai to e
  • Otherwise, apply pressure to other activities

37
Atomic Redeployment
  • Traditional pub/sub client movement protocols are
    expensive and do not offer transactional
    properties
  • Transactional movement
  • Formalized movement properties similar to ACID
    properties
  • Efficient and guaranteed routing reconfiguration
  • For example, guarantee that no messages are lost,
    if an activity is re-deployed
  • See IEEE ICDCS2009

38
Evaluation and design issues
39
Process Hotspot Illustration
Process
p 90
q 10
B
G
D
E
F
A
I
C
H
Red activities are pinned to brokers
ESB broker topology
SLA Minimize traffic
2
5
8



AB
GI
Metric Message hops
1
4
7
D
F

E
3
6
9



C
H
Execution engine
Broker
40
Process Hotspot Results
  • Traffic with redeployment is 47 of the static
    case
  • Post-redeployment traffic is 10
  • Redeployment triggered in about 30 sec

10 of static
40
41
Varying Hotspot Illustration
Process
p
q
B
G
D
E
F
A
I
C
H
ESB broker topology
2
5
8



AB
GI
Red activities are pinned to brokers
1
4
7
D
F

E
3
6
9



C
H
Execution engine
Broker
42
Varying Hotspot Results
  • Dynamic redeployment suffers temporarily after
    process hotspot moves
  • Traffic with redeployment is 42 of the static
    case

43
Larger Process - Results
  • Larger process with several parallel branches
  • Traffic with redeployment is 48 of the static
    case
  • 14 in steady state

44
Summary on SLA-driven BPM
  • Distributed execution engine has qualitative and
    quantitative advantages
  • Redeployment algorithm can optimize SLA in many
    cases
  • Challenges and design issues
  • Deployment stuck in local optima
  • Coordination among re-deployment decisions
  • SLA granularity
  • Interference of processes

45
Summary on SLA-driven BPM
  • Distributed execution engine has qualitative and
    quantitative advantages
  • Redeployment algorithm can optimize SLA in many
    cases
  • Challenge 1 Local optima
  • Techniques to selectively expand knowledge can
    work
  • Widen candidate radius
  • Redeploy sets of activities
  • Challenge 2 Coordination
  • Independent decisions can destabilize system
  • Potential problem has not manifest in evaluations
    so far
  • Challenge 3 SLA granularity (more an engineering
    issue)
  • Cant specify SLA on portions of a process
  • Cant specify SLA on particular instances of a
    process (e.g., VIP user)

46
Benefits of Content-based Publish/Subscribe for
BPM
  • Naturally enables centralized and distributed
    business process coordination
  • Coordination can span administrative domains and
    physically distributed resources
  • Supports process orchestration and choreography
  • Monitoring control is integral part of paradigm
  • Agile on the fly process adaptation and
    versioning
  • Correlation of application events with low-level
    infrastructure events

47
Summary The PADRES Stack
Download PADRES _at_ http//padres.msrg.org
48
Outlook
  • Enabled by the PADRES Events Services Bus
  • Ad hoc and dynamic business processes
  • Business performance management
  • Business impact assessment
  • Business entity management

49
Conclusions
  • Many applications are inherently event-driven.
  • Effective BPM requires capable event processing
    abstractions.
  • Content-based publish/subscribe is a powerful
    event processing abstraction and paradigm.
  • PADRES is based on the pub/sub paradigm.
  • PADRES is an ESB targeted at event-based BPM.
  • PADRES enables real-time business analytics and
    business activity monitoring.

50
Acknowledgements Current PADRES Team
  • Chen Chen
  • Alex Cheung
  • Alton Chiu
  • Amer Farroukh
  • Patrick Lee
  • Guoli Li
  • Bala Maniymaran
  • Vinod Muthusamy
  • Reza Sherafat
  • Naweed Tajuddin
  • Chunyang Ye
  • Young Yoon

Countless alumni (see our web site) http//padres.
msrg.org
51
Questions Discussion?
52
Selected Literature about PADRES
  • All our papers are available from
  • http//msrg.org/tags/padres hyperlinked below
  • A Distributed Service Oriented Architecture for
    Business Process Execution. ACM Trans. on the
    Web, 2010.
  • Publisher Placement Algorithms in Content-based
    Publish/Subscribe. IEEE ICDCS, 2010.
  • Transactional Mobility in Distributed
    Content-Based Publish/Subscribe Systems. IEEE
    ICDCS, 2009.
  • Distributed Automatic Service Composition in
    Large-Scale Systems. ACM DEBS, 2008.
  • Adaptive Content-based Routing in General Overlay
    Topologies. ACM Middleware, 2008.
  • Historic Data Access in Publish/Subscribe. ACM
    DEBS, 2007.
  • Papers on managing SLAs with PADRES
  • http//msrg.org/tags/sla

53
References
DEBS Conference http//www.debs.org July 2011 _at_
IBM in New York, US
  • The PADRES ESB project home
  • http//padres.msrg.utoronto.ca
  • An eQoSystem for declarative distributed
    applications with SLAs
  • http//research.msrg.utoronto.ca/Eqosystem/
  • The Micro-ToPSS event processing middleware for
    sensor networks (RFID)
  • http//microToPSS.msrg.utoronto.ca/
  • Mobile-ToPSS publish/subscribe for mobile and
    location-based applications
  • http//research.msrg.utoronto.ca/Mobile/
  • ToPSS - the Toronto Publish/Subscribe System
    Family
  • http//www.ToPSS.biz (coming soon ? )
  • Quantifying events in software to increase
    modularity customization in C-based systems and
    software-based product lines
  • http//www.AspeCtC.net (ACC - the AspeCt-oriented
    C compiler)
  • The Middleware Systems Research Group
  • http//www.msrg.utoronto.ca
  • My web site
  • http//www.eecg.toronto.edu/jacobsen

_at_ the University of Toronto
54
(No Transcript)
55
Additional slides
56
Local Optima Illustration
Process
p 90
q
B
G
D
E
F
A
I
C
H
ESB broker topology
DE
ABC
2
5
8

1
4
7
F


3
6
9
GHI


Red activities are pinned to brokers
Execution engine
Broker
57
Local Optima Problem
  • Handcrafted static deployment shows effect of
    being stuck in a locally optimal deployment
  • Finding non-local optima increases complexity
  • Becomes centralized at the limit

p 90
q
D
E
F
ooo
ooo
DE
ESB broker topology
ABC
2
5
8

1
4
7
F


3
6
9

GHI

58
Local Optima Initial Conditions
  • If D moves to 7 first
  • Average 34.4 messages per instance
  • If E moves to 8 first
  • Average 36.3 messages per instance

58
59
Local Optima Solution Illustration
Process
p
q
B
G
D
E
F
A
I
C
H
ESB broker topology
DE
ABC
2
5
8

1
4
7
F


3
6
9
GHI


Red activities are pinned to brokers
Execution engine
Broker
60
Local Optima Set Optimization Solution
ESB Fragment
  • Static
  • Activities D and E remain at engine 8
  • Average 18.6 messages per instance
  • Dynamic
  • Activities D and E move together to engines 7, 4,
    1
  • Average 9.1 messages per instance (over entire
    run)

7
4
1
60
61
Local Optima Algorithm Extensions
  • Compute the cost of deploying contiguous local
    activities ai at candidate engines ej
  • Compute the cost of deploying contiguous local
    activities ai at candidate engines ej
  • Compute the cost of deploying contiguous local
    activities ai at candidate engines ej

Measurements C(ai) Cost at local
engine E(P(ai)) Location of predecessors E(S(ai))
Location of successors
Cost Model
Given F(ai) Cost function P(ai)
Predecessors S(ai) Successors
Complexity Memory O( aid ej ) Computation
O( aid ej Navg(ai) )
Compute C(ai1,ej) C(aid,ej) for ai1, ,
aid, ej Estimated cost of deploying d contiguous
activities at candidate engines
62
Convergence Illustration
Process
p
q
B
G
D
E
F
A
I
C
H
ESB broker topology
2
5
8


F
Red activities are pinned to brokers
1
4
7



3
6
9



Execution engine
Broker
63
Convergence Results
  • Increasing the candidate space to 2-hop
    neighbours stabilizes in half the time
  • Tradeoff convergence speed with complexity
  • Larger candidate space can also help escape local
    minima

63
64
Sensitivity Problem
  • Initially p 0.9, q 0.1
  • Dynamic algorithm adapts properly
  • Between instances 75-80 there is a short blip
    where q 0.999
  • While short, this is significant and causes
    activity E to move to engine 7

65
Sensitivity Solution
  • By examining the distribution of messages we can
    discard outliers
  • In this case, the algorithm does not react to the
    temporary spike
  • There are fewer movements
  • The steady state cost is lower

66
Common Denominator
  • BPM applications, such as business processes, are
    driven by asynchronous state transitions.
  • Something happens, an appropriate reaction is
    expected and required.
  • Asynchronous state transitions represent events.
  • A process is triggered, a request submitted,
  • BPM applications require event management and
    processing capabilities to run effectively.

67
Example
  • Atomic subscription for Credit Check agent
  • class Activity Status, cmd. Trigger,
    Process ID X
  • Composite subscriptions
  • class Activity Status, cmd.
    Archived,Process ID X AND
  • class Activity Status, cmd. Signed
    Off,Process ID X
  • cmd. Credit check request, Process ID X
    AND
  • status Approved, Process ID X

Structural property of process

Performance property of process
Loan Process
Archive
Process
Approve
gt 50K
Check score
Credit check

Send to officer
Reject
68
Application Modeling
  • Advertisements (schema or types)
  • A1 class,,reading, shipID,,, level,lt,10
  • A2 class,,manifest, shipID,,, firm,,,
    content,,
  • A3 class,,audit, firm,,, trust,gt,0
  • Publications events (data)
  • class, reading,shipID,ACME123,level,
    4 (induced from A1)
  • class, manifest,shipID,ACME123,firm,ACME (in
    duced from A2)
  • class, reading,shipID,ACME123,level, 12
    (not induced from A1)
  • Subscriptions (query)
  • class,,reading, level,gt,3
  • class,,audit, firm,,,trust, gt, 3

69
Business Process Deployment
sub/advs Job D
sub/advs Job C
sub/advs Job B
sub/advs Receive
Deployer
sub/advs Invoke
PADRES Broker Overlay
6
Invoke
1
3
4
sub/advs Receive
2
5
receive
sub/advs Reply
sub/advs Assign
Reply
Assign
Activity Agent
Process Deployer
70
What Abstractions Do Not Work?
Cum grano salis
  • Databases
  • Great for managing historic data
  • But what about future data (e.g., events)
  • Data streams
  • Great for managing structured streams of tuples
  • But what about un-structured, multi-typed,
    sporadic events from many sources
  • Rule-based expert systems
  • Great for inference and reasoning
  • But what about managing large numbers of
    fined-grained filters in distributed environments

71
The Toronto Publish/Subscribe System Family
(ToPSS)
A-ToPSS (approximate)
ToPSS (matching)
CS-ToPSS (composite subs)
  • Matching algorithms
  • Language expressiveness vs. efficient matching
  • Routing protocols
  • Network architectures scalability
  • Higher level abstractions
  • Process execution
  • Monitoring

S-ToPSS (semantic)
L-ToPSS (location-based)
Rb-ToPSS (rule-based)
X-ToPSS (XML matching)
persistent-ToPSS (subject spaces)
M-ToPSS (mobile)
P2P-ToPSS (peer-to-peer)
LB-ToPSS (load balancing)
Ad hoc-ToPSS (ad hoc networking)
Federated-ToPSS (federation of ToPSS brokers)
FT-ToPSS (fault tolerance)
Historic-ToPSS (historic data)
BPEL-ToPSS (BPEL execution)
JS-ToPSS (job scheduling)
72
Historic Query Processing
73
Historic Query with Event Correlation
Monitoring patients who still have fever after
taking Advil
heart rate 150 bpm
P
Medical records database
medication NoNameDrugX
medication NoNameDrugX blood pressure gt
100 heart rate gt 130
blood pressure gt 100
B
heart rate gt 130
S
B
medication NoNameDrugX
Monitoring patients who still have high blood
pressure after taking NoNameDrugX
P
S
B
B
B
temperature 38C
temperature gt 42 medication Advil
P
S
Monitoring patients having a fever but not having
a cold
B
temperature 40C
heart rate gt 140 bpm temperature gt 39
illness ! cold
B
P
P
illness diabetes
blood pressure 60 mmHg
74
Experimental results
75
Publish/Subscribe
76
Publish/Subscribe 101
  • Not all publish/subscribe is equal
  • Publish/Subscribe models and evolution
  • Channel-based
  • Ex. OMG CORBA Event Service,
  • Topic-based
  • Ex. WS Notifications,
  • Type-based
  • Ex. OMG Data Dissemination Service (partially),
  • Content-based
  • Ex. The PADRES ESB (see below),
  • State-based
  • a.k.a., Subject Spaces (active research)

77
The Content-based Pub/Sub Model
  • Language and data model
  • Boolean functions over predicates
  • Subscriptions are conjunctions of predicates
  • Publications are sets of attribute-value pairs
  • Matching semantic
  • A subscription matches if all its predicates
    match
  • Approximate semantic (e.g., close to, cheap,
    sunny)
  • Semantic and similarity-based matching

Example Tree-structured data Graph-structured data Un-structured data Regular languages Relational model
Subscription XPath RDF Query Keywords Regular expressions SQL
Publication XML RSS feeds Text, documents Sentences over some alphabet DBs, i.e., tables
78
In-network processing
79
Event Correlation and In-network Processing
Monitoring patients who needs critical attention
heart rate 150 bpm
P
medication DrugX heart rate gt 120
bpm blood pressure lt 70 mmHg
P
B
S
B
age 70
Monitoring elder patients who are losing blood
pressure
P
S
B
B
B
heart rate lt 30 temperature lt 33 blood
pressure lt 50
temperature 38C
P
S
Monitoring patients having a fever but not having
a cold
B
temperature 40C
heart rate gt 140 bpm temperature gt 39
illness ! cold
B
P
P
illness diabetes
blood pressure 60 mmHg
80
Illustration of In-network Processing
81
Monitoring
82
Monitoring Client
  • Monitor is a Pub/Sub client
  • Visualizes topology of broker overlay network
  • Visualizes message routing
  • Monitor/control process execution
  • Process level
  • Activity level
  • Supports dynamic process modifications

83
PADRES on PlanetLab
84
(No Transcript)
85
Cost Model
  • Routing cost of CS
  • RC(CS)
  • Subscription cardinality
  • P(S) The number of matching publications per
    unit of time.
  • P(S)
  • P(CS) P(Sl) P(Sr) if op OR

Matching Engine

Routing Table
subscription
dest
temperature gt 37 B1
temperature gt 40 B2
86
Dynamic CS Routing (DCSR)
Adv 3
Broker 5 and 8 are the join point brokers
Adv 2
2
1
7
S3
CS
3
4
8
S3
S2
Adv 1
S2
S1
5
6
9
S1
CSS1 AND S2 AND S3
CS
87
(No Transcript)
88
Estimator Example distance
  • Discover paths
  • BP?BT , BT?BS
  • Probe paths
  • BT?BC for each candidate C
  • Compute paths
  • BP?BC , BC?BS
  • Cases
  • C in BP?BT
  • C in BT?BS
  • otherwise
  • Efficiently compute and cache information about
    candidate engines

Process
P
T
S
Topology
2
5
8
P
T

1
4
7

S
C
Source Target Distance Probe _at_
2 5 3 148pm
5 7 2 149pm
5 4 1 134pm
3
6
9



Execution engine
Broker
89
Engine Resource Monitor
  • Profile execution of local tasks

Process
P
T
S
Topology
2
5
8
P
T

Task Pred/Succ Msg rate Exec time Mem
T P 40/min 0.3s 30MB
T S 10/min
1
4
7

S
C
3
6
9



Execution engine
Broker
90
Redeployment Manager
  • Compute the cost of deploying local agents Ai at
    candidate engines Ej

Measurements C(Ai) Cost at local
engine E(P(Ai)) Location of predecessors E(S(Ai))
Location of successors
Cost Model
Given F(Ai) Cost function P(Ai)
Predecessors S(Ai) Successors
Complexity Memory O( Ai Ej ) Computation O(
Ai Ej Navg(Ai) )
Compute C(Ai,Ej) for every Ai,Ej Estimated cost
of deploying agent Ai at candidate engine Ej
91
Cost Model
The cost of a process is based on three cost
components
1. Distribution cost Cdist Cd3 (Cd1
Cd2) Cd1 Message rate Cd2 Message size Cd3
Latency 2. Engine cost Ceng f(Ce1,
Ce2,Ce3) Ce1 Load (number of instances) Ce2
Resources (CPU, memory, etc.) Ce3 Task complexity
3. Service cost Cserv Cs1 Cs2 Cs3 Cs1
Latency of external service Cs2 Execution time of
external service Cs3 Marshalling/unmarshalling C
ost(agent) ?wiCi Cost(process) ?cost(agent)
These costs can be weighted to achieve different
objectives
Optimize time wd1 wd3 we3 Cserv 1, other
wi 0
Optimize network overhead wd2 wd3 1, other wi
0
Various optimization criteria can be specified
Threshold criteria ?wiCi gt x E.g., Report SLA
violations within 3 s.
Minimized criteria min( ?wiCi ) E.g.,
Minimize distribution overhead
92
(No Transcript)
93
Transformation Example
  • class,,reading, level,gt,3
  • AND
  • class,,audit, firm,,, trust, gt, 3
  • class,,reading, shipID,,X, level,gt,3
  • AND
  • class,,manifest, shipID,,X, firm,,Y,
    content,!,fertilizer
  • AND
  • class,,audit, firm,,Y, trust,gt,7

composite subscription
atomic subscription
94
(No Transcript)
95
The PADRES ESB for Event-driven BPM
Enterprise Services Bus (Events Services Bus)
  • First generation of students, when I looked away
    ?
  • Peng Alex David aRno Eli Serge
  • PADRES is Publish/subscribe Applied to
    Distributed
  • Resource Scheduling
  • PAdres is Distributed REsource Scheduling
  • http//www.padres.msrg.utoronto.ca

Web start open source padres.msrg.org
Supported and sponsored by
96
The PADRES ESB in RD
  • Develop flexible BPM concept
  • With CA Labs, Sun Microsystems, OCE NSERC
  • Develop SLA-based autonomic BPM concept
  • With IBM and NSERC
  • Evaluate and experiment with the use of the
    PADRES ESB for application integration
  • With Bell Canada and BUL

Acknowledgements
About PowerShow.com