Control-based Quality Adaptation in Data Stream Management Systems (DSMS)

1
Control-based Quality Adaptation in Data Stream
Management Systems (DSMS)

• Yicheng Tu, Song Liu, Sunil Prabhakar, and Bin
  Yao
• Department of Computer Sciences, Purdue
  University, USA
• School of Mechanical Engineering, Purdue
  University, USA

2
Data Stream Management

• Continuous data, discarded after being processed
• Continuous query
• Data-active query-passive model
• Applications
• Financial analysis
• Mobile services
• Sensor networks
• Network monitoring
• More

3
DSMS architecture

• Network of query operators (O1 O3)
• Each operator has its own queue (q1 q4)
• Scheduler decides which operator to execute
• Query results (Q1, Q2) pushed to clients
• Example systems
• Aurora/Borealis
• STREAM

4
Quality-of-Service (QoS) in DSM

• Data processing is QoS-critical in DSMS
• Tuple delay is the major concern results
  generated from old data are useless!
• Highly dynamic environment ? hard to maintain QoS
• Bursty data input
• Unpredictable unit processing cost
• Overloading during spikes ? degraded (delay) QoS
• Solution adjust data loss ? load shedding

5
Load Shedding

• Eliminating excessive load by dropping data items
  ? less QoS violations
• Key questions
• When?
• How much?
• Where?
• Which tuples?
• Basic algorithm (Tatbul et al., 2003)
  periodically

6
Whats wrong?

• Fail to capture current system status (queue
  length) and output (delay)
• Delay positively related to queue length
• Examples 1. Unbounded increase of delay
• Example 2. Unnecessary data loss

7
Our approach

• View load shedding as a feedback control problem
• Feedback Control manipulation of system behavior
  by adjusting system input based on system output
• Cruise control of automobiles, room temperature
  control, etc.
• The feedback control loop
• Plant
• Monitor
• Controller
• Actuator
• How it works
• Error measured output desirable output
• Focal point controller, which maps error to
  control signal

8
Why Feedback Control ?
9
The feedback control loop

• Plant current DSMS
• Input load admitted
• Output delay QoS
• Reference output specified by DBA
• Actuator
• adaptor load shedder
• admission controller
• Monitor new
• Controller new
• System dynamics disturbances
• Discrete control control period T

10
Controller design

• Performed all three steps in a real DSMS the
  Borealis system
• We set output to average delay
• System identification gives a first-order model
  structure
• Control function
• Controller analysis gives the following set of
  parameters

11
Experiments
12
Experimental results
13
Experimental results
14
Experimental results
15
Experimental results
16
Summary

• Load shedding is an important quality adaptation
  method
• Current solutions focusing on steady-state
  performance do not work well under bursty inputs
  
• We propose an approach to guide load shedding in
  a highly dynamic environment based on feedback
  control theory
• Initial experimental results show promising
  potential of our approach