Continuous Query Language: From CQL to CAPE Algebra Plans

1
Continuous Query Language From CQL to CAPE
Algebra Plans

• Lee Chu
• Che Wai Kwan
• MQP 2004/2005

2
Continuous Query Processing

• Emerging Applications
• Traffic management
• Network monitoring
• Require
• Online processing of data streams
• But
• Traditional databases handle persistent data

3
Databases Systems VS Data Stream System

• Database System
• One time query
• Random access

• Data Stream System
• Continuous queries
• Sequential access

4
CAPE Constraint-exploiting Adaptive Processing
Engine

• An on-going project at WPI

5
CAPEs limitation

• Desire
• High-level query language, such as SQL
• Instead
• Enter queries as low-level execution plan
• Problems
• Tedious to enter
• Error prone

6
Algebra Plan VS SQL

• ltqueryplangt
• ltoperator root true id 1 className
  gt
• ltclassVariablesgt
• ltvariable namegroup_pos value0/gt
• ltvariable namefunction valuenull/gt
• ltvariable namefunction_pos value0/gt
• ltvariable namefunction valuecount/gt
• ltvariable namefunction_pos value0/gt
• ltvariable namepropagate valuefalse/gt
• ltvariable namedebug valuetrue/gt
• lt/classVariablesgt
• ltpropertiesgt
• lt/propertiesgt
• ltparentsgt lt/parentsgt
• ltchildrengt
• ltchild id 2/gt
• lt/childrengt
• ltstreamsgt lt/streamsgt
• lt/operatorgt

ID 1 Group By
ID 2

• select S.A from R, S, Q
• where R.A S.A

7
Objective

• Define and implement a high-level query language
  for CAPE

8
Methodology

• Study existing Continuous Processing Language
  proposals
• Identify one, adopt and adapt if appropriate
• Implement it for CAPE

9
Requirements on Language

• SQL-alike
• Data Streams
• Windows on streams

10
Continuous Processing Languages

• UDA UCLA
• TelegraphCQ Berkeley
• STREAM-CQL Stanford

11
STREAM-CQL

• Well defined semantics
• Open source available
• Query example
• query rstream (select S.A from R, Q, Srange 1
  minute where R.A S.A)

12
Our Query Plan Generator Big Picture
13
Step 1 STREAM Parser
Yacc and Lex
14
Step 2 STREAM Plan Generator
STREAM Plan Generator
Modified Plan
CAPE Plan Rewriter
Parse Tree
15
STREAM Plan Generator Default Query Plan

• query rstream
• (select S.A from R,
• S range 1 minute, Q,
• where R.A S.A)

16
STREAM Plan GeneratorCleaned Query Plan

• query rstream
• (select S.A from R,
• S range 1 minute, Q,
• where R.A S.A)

17
Step 3 CAPE Plan Rewriter
CAPE Plan Rewriter
Optimized Tree
Cleaned Tree
18
ThetaJoin Rule
RStream ID 7
19
WindowPushUp Rule
RStream ID 7
20
Step 4 CAPE XML Plan Writer
XML Plan
Optimized Tree
CAPE XML Plan Writer
21
Evaluation Methodology

• Query test bed
• Test individual operators
• Test complex query plans
• Evaluation
• Manual inspection of generated XML plan
• Test XML file on CAPE

22
Evaluation of Individual Operators

• Regular Project
• Function Project
• Select
• Stream Source
• Range Window
• Partition
• Distinct

23

• CQL
• Rstream (Select A
• from S
• where A 5)

24

• CQL
• rstream (select A B from S)

25
Conclusion

• Identified query language for CAPE
• Designed a loosely coupled translation frameworks
  from CQL to CAPE
• Rewrite algebra tree
• Generate CAPE XML plans
• Evaluation of generated query plans

26
Future Works

• Implement Relations
• Which will maximize CAPEs capability
• Research on the window size
• Support different time range variation
• Implement a Graphical User Interface
• Drag and Drop feature to input CQL

27
Acknowledgements

• Prof. Rundensteiner
• Yali Zhu
• Luping Ding

28
Question or Comments?