ZStream: A Cost-based Query Processor for Adaptively Detecting Composite Events

1
ZStream A Cost-based Query Processor for
Adaptively Detecting Composite Events

• Presented by Rabia Nuray-Turan and Xiaozhi Yu

2
What is ZStream

• It is a composite event processing system(CEP).
• It uses tree based query plans for both the
  logical and physical representation of query
  patterns.
• It is able to unify the evaluation of sequence,
  conjunction, disjunction, negation, and Kleene
  closure as variants of the join operator.
• It is able to capture the actual runtime behavior
  of a plan, and chooses the optimal plan.

3
Language Specification

• Example Stock market monitoring query
• Pattern T1T2T3
• WHERE T1.name T3.name
• AND T2.name Google
• AND T1.price gt (1x)T2.price
• AND T3.price lt (1-y) T2.price
• WITHIN 10 secs
• RETURN T1, T2, T3
• Event Operators
• Sequence------AB
  Disjunction----AB
• Negation------!A
• Conjunction ----AB

4
Tree-Based Plans
5
Operator Evaluation

• Sequence Operator
• Has two operands first and second event class in
  the sequential pattern
• Right buffer is the main iteration to make sure
  the output is sorted in end time order
• Any event in right/left buffer with start time
  less than EAT is discarded
• Only valid events in right buffer are compared
  against the events in left buffer
• Negation Operator
• Must be combined with other operators
• Pushed down in the tree to avoid unnecessary
  intermediate results
• Finds time intervals when non-negation events can
  produce only valid results
• Searches for negation events that can negate
  non-negation event instance

6
Operator Evaluation

• Conjunction
• Similar to sequence operator, but do not
  distinguish between the right and left buffers
• Works like a sort-merge join
• It maintains two cursors to both input buffers
• Finds the earlier not-yet-matched event in one
  buffer and merges with the earlier events in the
  other buffer satisfying the constraints
• Disjunction
• Input is directly added to the output buffer if
  it satisfies both time and value constraints
• Merging is done according to their end time
• Kleene Closure
• Trinary operator, with start and end closure
• Kleene closure looks for matches in the midle
  buffer
• If the count is not specified, maximum number of
  middle buffer events are found
• If not, a window of specified size is applied

7
Optimal Query Plan

• Rule Based Transformations
• Smaller number of operators
• If the number of operators same, plan with the
  lower cost operators will be selected
• Hashing for equality Predicates
• For sequence AB where A.f B.f, built hashtable
  on A
• For conjunction built hash tables for A and B
• Reduces the search cost for equality predicates
• Optimal Reordering
• Dynamic Programming based
• Finds the optimal subtrees to find the optimal
  physical plan
• Plan Adaptation
• Recompute the physical query plan on the fly
  based on the collected statistics.
• Change the plan without any damage to the
  expected output.

8
Performance Evaluation
Query 7 Negate Pattern IBM!SUNORACLE Pattern
IBM!SUNORACLE WITHIN 200 Units