A Unified Approach to Routing, Covering and Merging in PubSub Systems Based on Modified Binary Decis

1
A Unified Approach to Routing, Covering and
Merging in Pub/Sub Systems Based on Modified
Binary Decision Diagrams

• Guoli Li Shuang Hou Hans-Arno Jacobsen
• Middleware Systems Research Group
• University of Toronto

2
Agenda

• PADRES Project
• MBD-based Router Architecture
• Covering-based Routing
• Merging-based Routing
• Evaluations

3
PADRES Project

• Goal development of a distributed,
  content-based, pub/sub routing fabric to support
  distributed applications

Distributed Overlay Broker Network


PADRES
4
PADRES Architecture

• Based on advertisement/subscription/publication
  message routing paradigm, similar to SIENA,
  Gryphon and REBECA

PADRES
5
Problem
6
BackgroundCovering Relation

• Publications matching S2 must match S1
• S1 covers S2 iff
• Covering-based routing

S1
1
Publisher
3
4
5
S1 1
S1 3
S2
2
S2 2
S2 3
If S1 covers S2
7
BackgroundMerging Relation

• Publications matching S1 overlap with
  publications matching S2
• S is a merger of S1 and S2, iff

S1
1
Publisher
3
4
5
S1 3
S1 1
S2
2
S2 2
S2 3
If S1 and S2 can be merged to S
8
BackgroundPerfect vs. Imperfect Merging

• Perfect Merging
• S is a perfect merger of S1 and S2, iff
• Imperfect Merging
• S is an imperfect merger of S1 and S2, iff

9
BackgroundBDD-based Subscriptions

• BDDs are binary decision diagrams Bryant,1986
• BDDs are abstract representations of Boolean
  functions based on a predefined variable ordering
• For each path in a BDD, the sequence of variables
  along the path is strictly increasing wrt. lt
• A node v represents a function
• A subscription is a Boolean function, when each
  predicate is assigned a Boolean variable

1
0
1
0
0
1
1
0
XltYltZ
10
MBD-based Subscriptions

• MBDs are modified binary decision diagrams
  Jacobi,1993
• An MBD has multiple roots and represents multiple
  Boolean functions
• An MBD represents a set of subscriptions
• MBD(O1, O2,,On), where Oi is a subscription
• In an MBD, common predicates are shared
• An MBD evaluates shared predicates only once

11
MBD-based Subscriptions
Subscription 1 (X Y Z) (price150)
(time06/08/2005) (destinationColumbus)
Subscription 2 (X Y Z) (pricegt150)
(time06/08/2005) (destinationColumbus)
MBD(O1, O2)
O1
O2
0
X
X
X
X
1
Y
Y
Y
Z
Z
Z
1
0
1
0
1
0
S2
S1
12
Variable Ordering

• A reasonable variable ordering leads to an MBD
  with less vertices
• The variable ordering affects the number of
  subscriptions organized in the same MBD
• Attribute ordering
• Popular attributes are placed in a lower position
  in an MBD
• Related attributes in the application semantics
  are placed close to each other in the ordering
• Variable ordering within an attribute
• No significant improvements over random orders

13
Global Predicate Index

• Indexed by
• Data type
• Attribute
• Operator
• Value
• Each predicate is assigned a Boolean variable
• Benefits
• Storage
• Evaluation
• Evaluate both publication and subscription

10


X price 150
150
price
Integer
gt
lt

Y time 06/08/2005
Date

Toronto



destination
Z destinationColumbus
Columbus
substring
String


Publication (price,200)(destination, Columbus)
14
Publication Routing

• Publication routing against subscriptions
• Each attribute-value pair traverses the index to
  evaluate the predicates
• All Boolean variables have value assignments
• Evaluate MBDs wrt. the truth assignment
• For MBD(O1,O2,,On), if Oi1 a subscription
  represented by Oi is matched by the publication

15
MBD-based Covering

• Covering relations encoding LHS RHS
• No covering relations 0 0
• Is-covered-by 0 1
• Is-covering 1 0
• Equal 1 1

LHS
RHS
(price150)
A new Subscription (pricegt100)
(time06/08/2005) (destinationColumbus)
X 1 0
X
X
(time06/08/2005)
Y
Y
Y 1 1
(destinationColumbus)
Z
Z
Z 1 1
1 0
1
0
1
0
16
MBD-based Covering Algorithm

• Subscription routing against subscriptions
• Each predicate traverses the index to evaluate
  the covering relation at predicate level
  resulting invl, vr
• Evaluate LHS/RHS MBDs wrt. the truth assignment
• If a RHS of MBD(O1,O2,,On) has Oi1, the new
  subscription is covered by Oi.
• Not insert the new subscription
• If a LHS of MBD(O1,O2,,On) has Oi1, the new
  subscription covers Oi.
• Evaluate the LHS of MBDs only
• Insert the new subscription and remove all Oi,
  where Oi1
• Otherwise, insert the new subscription.

17
Optimization for Covering

• Optimization cache the predicate covering
  relations
• e.g., if x1(pricegt150) is true, then
  x2(pricegt100) must also be true
• Cache the variable dependencies for each
  attribute
• Speed up the covering evaluation at the predicate
  level, especially when the index grows large

18
MBD-based Merging

• Subscriptions in the same MBD have a better
  chance to be merged
• The merger may cause new covering relations

O1 (price150)
O2 (pricegt150)
O (pricegt150)
X
X
X1
Y
Y
Z
Z
1
0
1
0
19
Imperfect Merging

• Imperfect Degree
• Imperfect Merging Threshold (IMT)
• Advertisement-based Optimization

where Aj is an advertisement available at the
broker
20
Evaluations

• Environment
• Intel Xeon 3GHz, 2GB RAM
• JDK 1.4.2 using JavaBDD package
• Workload description
• Predicates based on 20 pre-ordered attributes
  with random operators
• Values are uniformly distributed in the value
  range
• Data sets with different number of distinct
  predicates
• Metrics
• Routing table size
• Routing delay per publication
• False positives

21
Routing Table Size
Publication Routing delay
22
Routing Table Size with IMT
23
False Positives
24
Related Work

• A. Campailla, S. Chaki, E. Clarke, S. Jha, H.
  Veith. Efficient filtering in publish-subscribe
  systems using binary decision diagrams. ICSE'01.
• Focuses on centralized publication matching
• We extend to an MBD-based approach
• We unify routing, covering and merging with the
  same data structure
• REBECA G. Mühl. Generic constraints for
  content-based publish/ subscribe systems.
  CoopIS'01.
• Establishes merging-based routing
• We further explore imperfect merging
• SIENA A. Carzaniga, D. S. Rosenblum, A. L. Wolf.
  Design and Evaluation of a Wide-Area Event
  Notification Service. ACM Transactions on
  Computer Systems, 2001
• Establishes advertisement-based distributed
  pub/sub paradigm
• We apply the knowledge of advertisements for the
  optimization of imperfect merging

25
Conclusions

• Novel data structures and algorithms that unify
  publication routing, subscription covering and
  merging
• MBD-based approach is flexible to support
  extended subscription languages
• Covering and merging lead to a compact routing
  table and reduce the network traffic
• Efficient publication routing and subscription
  processing

26
Thank You!
27
Routing Table Size
Publication Routing Time
28
Routing Table Size
29
Variable Ordering
30
(No Transcript)
31
(No Transcript)