Incremental Update to Aggregated Information in a Data Warehouse Miranda chan, Hong Va Leong, Antoni

1
Incremental Update to Aggregated Information in a
Data WarehouseMiranda chan, Hong Va Leong,
Antonio Si

• DCS891C Research Seminar III
• Idea Paper
• Presented By
• Gauri Ghare
• DPS 2007

2
A Database

• Purpose To store data Run MIS reports
• Normalized data
• Tables, Queries,
• Views
• Procedures
• Functions
• Built-in
• Simple
• Aggregate
• Custom

3
A Data Warehouse

• Purpose OLAP, Decision Support
• Data
• De-Normalized form
• Source from different databases
• Layered Approach
• Facts and Dimensions
• Tables?Views with Union of rows from different
  tables
• Queries ? Materialized Views
• Procedures
• Built-in
• Custom 1. Package 2. Triggers
• Functions
• Built-in 1. Simple 2. Aggregate
• Custom Combination of Built-In-Functions

4
Aggregate Functions

• Commonly Used
• Sum, Count, Min, Max, Average
• Not So Commonly Used
• Ceiling, Floor, Sine, Cosine,Tangent etc.
• Handling an Aggregate Function call
• Database Strategy
• Impact
• Architecturally
• Database Performance

5
Why Data Refresh is a Problem

• Stored the Query Results
• Materialized Views
• Change at the source
• Changes in the Aggregate Function Results
• Change in the underlying data
• Multiple Data sources
• Exponential data
• Multiple resources

6
Suggested Data Refresh Strategy

• Incremental Refresh of Aggregate Result
  Values12
• Caching Changed Values1
• Creating update Log1
• Queue updated / Changed Information1 2
• Creating Segment trees and B-trees2

7
Research Plan

• Research and Study refresh strategies of
  different data warehouse enabled databases
• Create a prototype
• Using suggested algorithms benchmark effect on
  database performance
• Develop new algorithms and compare the effect on
  database performance

8
References

• Incremental update to Aggregated Information for
  Data Warehouse over Internet Miranda chan,
  Department of Computing, Hong Kong Polytechnic
  Univ. Hong Kong Hong Va Leong, Department of
  Computing, Hong Kong Polytechnic Univ. Hong Kong
  Antonio Si Oracle Corporation Redwood Shores, CA
  94065
• Incremental Computation and Maintenance of
  Temporal Aggregates By Jun Yang Computer Science
  Department, Duke University, Jennifer Wildom
  Computer Science Department, Stanford Univeristy
• Performing Partial Aggregates Oracle 9i/OLAP
  Developers Guide to the OLAP DML Release 2
  (9.2) online documentation from Oracle.com
• Incremental Maintenance for Non-Distributive
  Aggregate Functions Themis Palpanas (U of
  Toronto), Richard Sidle, Bobbie Cochrane, Hamid
  Pirahesh( work done at IBM Almaden Research
  Center)
• Incremental updates in OLAP cubes MSDN