Operator Placement for InNetwork Stream Query Processing

1
Operator Placement for In-Network Stream Query
Processing

• U. Srivastava, K. Mungala, and J. Widom,
• PODS 2005
• ICS280 class presentation by Iosif Lazaridis
• (Winter 2005)

2
Problem Motivation
3
Previous Solutions

• Push all data to the server queries are
  processed there
• Does not utilize in-network resources
• Push simple filters to the leaf nodes
• e.g., select all values gt3
• Perform aggregation in intermediate nodes
• But what about expensive operations?
• e.g., filters over image data, or operations
  involving remote lookups

4
Basic System Model
5
Basic theorem Rank

• Placing filters in order of increasing rank is
  optimal
• rank(F) cost(F) / (1-selectivity(F))
• Intuition
• Evaluate cheap filters early
• Evaluate very strict filters early

6
Problem Statement

• n filters and m levels of hierarchy
• Hence mn possible filter placements
• Problem choose optimal plan from mn different
  choices
• A greedy and an optimal algorithm

7
Greedy Algorithm

• Let c(P,i) be the cost of plan P incurred at node
  i
• i.e., the cost of applying the filter and
  transmitting the results to i1
• Greedy Algorithm minimize c(P,1) by choosing a
  set of filters F1 from total set F then minimize
  c(P,2) by choosing F2 from F-F1 etc.
• Choose all filters with rank less than l1

8
Example
1
2
l115
Three filters F1, F2, F3

• Then, evaluate F3, F2 in node 1
• Cost 10.530.50.6157
• Better than e.g., (cost15) or F3, F2 , F1
  (cost 10.530.50.60.8159.1)

9
Why Greedy is not optimal

• Previous plan F3, F2 then F1 has total cost
  70.50.689.4
• Consider plan F3, F2 , F1 then (total
  cost9.1)

10
Optimal Algorithm

• Model a link as a filter with selectivity ?i and
  cost li
• Each node has an incoming and an outgoing
  link
• Evaluate all filters with rank between the ranks
  of incoming and outgoing transmissionfilters
• If the rank of the incoming link is greater than
  of the outgoing link
• Optimally short-circuit node dont evaluate
  any filters on the node

11
Processing Joins
Filters F1,2 temperature0.5pressuregt120

• Two input streams R, S with rates r1, r2
• Output stream consists of tuples (r,s) with r in
  R and s in S
• Join cost ar1br2cr1r2
• Order filters that apply on r and s separately
• Order filters that apply to (r,s)
• Example temperaturegt10 and temperaturelt20 and
  pressuregt100 and temperature0.5pressuregt120

Join
Rate r1
Rate r2
Filters F1 Filters
F2 temperaturegt10 pressuregt100 temperature
lt20
12
Conclusions

• Systematic way to push filters into the network,
  taking into account their relative cost and the
  capabilities of nodes
• Perhaps does not take into account practical
  issues such as broadcast communication or faults
• Interesting to see practical values for ?, c, s
  in a real deployment.