An Introduction to Data Warehousing Concept and Technology - PowerPoint PPT Presentation

Loading...

PPT – An Introduction to Data Warehousing Concept and Technology PowerPoint presentation | free to download - id: 4b74dd-MWI3Y



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

An Introduction to Data Warehousing Concept and Technology

Description:

Data Warehousing Concept and Technology Mort Anvari Data Warehousing Concept Data Access Technology Enterprise Real-Time Knowledge Architecture for Data Warehousing ... – PowerPoint PPT presentation

Number of Views:199
Avg rating:3.0/5.0
Slides: 76
Provided by: MortA1
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: An Introduction to Data Warehousing Concept and Technology


1
An Introduction to Data Warehousing Concept and
Technology
  • Mort Anvari

2
Data Warehousing Concept Data Access
Technology Enterprise Real-Time Knowledge
Architecture for Data Warehousing Data
Collection and Delivery
Topics
3
Benson Parkers Square Wheel
Technology Environment
Business Environment
Business Planning
Business Operations
4
Benson Parkers Square Wheel
Technology Environment
Business Environment
Business Planning
Technology Planning
Business Operations
Technology Operations
5
Benson Parkers Square Wheel
Technology Environment
Business Environment
Impact
Business Planning
Technology Planning
Organization
Opportunity
Alignment
Business Operations
Technology Operations
6
Benson Parkers Square Wheel
Technology Environment
Business Environment
Impact
Business Planning
Technology Planning
Information Technology has to do more than just
align itself with the business, it has to
help the business have the maximum impact in the
marketplace.
Organization
Opportunity
Alignment
Business Operations
Technology Operations
7
Data Access and Delivery System
8
Technology Evolution
  • New classes of computers
  • New classes of communications
  • New classes of technology (image, sound, video,
    multimedia)
  • New classes of software
  • Much more complex technical environment
  • Cooperative Processing/Client-Server
  • Distributed Data Bases
  • LANs, WANs, etc.
  • Obsolescence Problem
  • Multiple Legacy Systems

9
IT Impact on Business
Enterprise Network Computing and Client/Server
Technology are changing the way organizations
look at all of their information systems
HP
Compaq
Obsolescence IT Wastes
DEC
IBM
Data Jail
10
The Existing Enterprise
  • Support Existing Products
  • Support Existing Customers
  • Support Existing Organization
  • Support Existing Workforce
  • Support Existing Technology

11
Controlling the (Global) Real-time Organization
  • RTO 24 x 7 x E

(Where E means every major market)
12
Information and the Enterprise
  • Organizational needs for data
  • Organizational needs for information
  • Organizational needs for knowledge

13
Information and the Enterprise
  • An Insurance IS Executive estimated that his
    organization could only access something like 1
    of all the data on their data base
  • A Bell Labs report has indicated that the amount
    of data doubles every 5 years, ... and they can
    only use about 5 of it!
  • Data Warehousing is Data Delivery System

14
Needs for Data
  • Data Values (Measurements)
  • Data to operate
  • Data to control
  • Data to plan

15
Needs for Information
  • Information Content Structure (Relationships)
  • Structure of the Real-world
  • Relating data to the business
  • Cross functional processes
  • Relating data to the real world
  • External DB
  • External Data Feeds (DB, Reuters, etc.)
  • Text, Image, Voice, Video, etc.
  • Statistical Studies

16
Needs for Knowledge
  • Knowledge Goals Actions Learning
  • Learning more about our business
  • Learning more about our market
  • Learning more about the business environment
  • Knowledge is the area in which Data Warehousing
    and Data Mining are potentially critical
    technologies

17
Data, Information and Knowledge
  • Data Centers
  • Information Centers
  • Knowledge Centers
  • Data Bases
  • Information Bases
  • Knowledge Bases

18
Old Data Never Dies
60s
70s
80s
90s
Batch On-line Minis PCs Networking Enterprise
Computing (Peer to Peer, Network to Network)
  • Note that none of the early computing styles have
    ever gone away!!!

19
Operational vs. Informational Systems
Information Access Today
20
Operational vs. Informational Systems
Operational Systems
Ord. Entry
Mafg.
Information Access Today
21
Operational vs. Informational Systems
Operational Systems
Informational Systems
Information Access Today
22
Operational vs. Informational Systems
Operational Systems
Informational Systems
Estimating Analysis
Marketing Systems
Product Planning
Information Access Today
23
Operational vs. Informational Systems
Operational Systems
Information Delivery System
Informational Systems
Information Access Today
24
Operational vs. Informational Systems
Operational Systems
Information Delivery System
Data Warehousing is fundamentally an issue of
Enterprise Data Architecture
Informational Systems
Information Access Today
25
Operational vs. Informational Systems
Operational Systems
Information Delivery System
Informational Systems
26
Operational vs. Informational Systems
Operational Systems
Information Delivery System
Data Warehouse
Informational Systems
27
Operational vs. Informational Systems
Operational Systems
Information Delivery System
Data Warehouse
Data Marts
Informational Systems
28
Operational vs. Informational Systems
External Data
Operational Systems
Information Delivery System
Data Warehouse
Informational Systems
Data Garages
29
Operational vs. Informational Systems
30
End User Evolution
  • Data Base Management Systems users
  • Ad Hoc Reports users
  • Todays Customer Demands Automated Real-Time
    Response.
  • End User Systems
  • Decision Support Systems
  • Executive Information Systems
  • Information Centers

31
Ways to Organize Data
  • Tables Flexible, Simple
  • Hierarchies Speed, Natural Reporting
  • Networks Multiple Directions, Complex
    Structure
  • Lists Updating Complex Structure
  • Matrices / Array Manipulate Multiple
    Dimensions
  • Inverted Files Unplanned queries, text
    retrieval
  • Objects Complex structures, hide structure
  • Multidimensional Data Bases (Data Warehousing)

32
End User Computing Evolution
33
Data Warehousing
  • Data Warehouse can be thought of as an automated
    version of the Information Center that was widely
    popular in the mid-1980s or even ultimately as
    the automation of Information Resource
    Management. And while technologies such as
    client-server have begun to put enormous
    computing and graphics power in the hands of
    individuals, however, these technologies have
    not, in general, provided the link to the
    operational data that end users need to make
    critical business decisions.

34
Data Warehouse Requirements
  • Support for Universal Access to Multi-platform
    Data Bases
  • Support for Multiple User Types
  • Separation of Operational and Informational
    Concerns
  • Support for Networked Data
  • Support for Directories, Repositories and
    Information Models,
  • Support for Advanced End User Interfaces

35
Access to Heterogeneous Data
HP
Compaq
DEC
IBM
36
Multiple User Types (Knowledge workers)
  • Top Executives
  • Managers
  • Analysts
  • Planners
  • Product Developers
  • Consultants
  • Lawyers
  • etc.

37
Separation of Operational and Informational
Concerns
  • Operational Systems
  • Response Time
  • Reliability
  • Security
  • Recoverability
  • Informational Systems
  • Flexibility, Performance, Ease of Navigation
  • Large numbers of different views
  • Manage Huge Amounts of Data (VLDBs)
  • Need to drill down/drill thru into data
  • Need to draw on data from many sources

38
Support for Networked Data
  • All the data that is required to support
    informational needs is often not on the same
    operational data base. The need for Labor
    Negotiations, for example, may come from a
    variety of operational data bases, such as
    Manufacturing, Personnel, and Accounting.
  • Distributed Systems

39
Support for Advanced End User Interfaces
40
Dimensions of Data Warehousing
Performance
Security
Connection to the Operational Data
Ease of Use
Flexibility
Distributed Data
Quality
Scalability
41
Enterprise Knowledge Architecture for Data
Warehousing
42
Operational vs. Informational Systems
Operational Systems
Information Delivery System
Informational Systems
43
Operational vs. Informational Systems
44
Enterprise Network Computer Architecture
Data Mart
45
Freeing the Data in Jail
46
The Information Access Layer
47
The Legacy Data Layer
48
The External Data Layer
49
The Data Access Layer
50
The Data Access Layer
Data Access Filter
51
The Data Access Layer
SQL Queries
52
The Data Access Layer
SQL Queries SQL Answers
53
Application Messaging
54
The Meta-Data Repository Layer
55
The Process Management Layer
56
The Core Data Warehouse
57
Data Staging and Quality
58
Data Mart (Post-process/Indexing)
Post- Proc. Indexing
59
Goals of Warehouse
  • 1. Performance (Canned queries, MD Analysis, Ad
    hoc, Impact on Operational System)
  • 2. Flexibility (MD Flex, Ad hoc, Change data
    structure)
  • 3. Scalability (No. of Users, Volume of Data)
  • 4. Ease of Use (Location, Formulation,
    Navigation, Manipulation)
  • 5. Data Quality (Consistent, Correct, Timely,
    Integrated)
  • 6. Connection to the Detail Business Transactions

60
Virtual Warehouse
61
Virtual Warehouse
62
Virtual Warehouse
A Virtual Data Warehouse approach is often
chosen when there are infrequent demands for
data and management wants to determine if/how
users will use operational data.
63
Virtual Warehouse
One of the weaknesses of a Virtual Data
Warehouse approach is that user queries are made
against operational DBs. One way to minimize
this problem is to build a Query Monitor to
check the performance characteristics of a query
before executing it.
64
Distributed Data Warehouse
65
Distributed Data Warehouse
A Distributed Data Warehouse is similar in most
respects to a Central Data Warehouse, except that
the data is distributed to separate mini-Data
Warehouses (Data Marts ) on local or specialized
servers
66
Information Access Tools
  • Desktop DBs
  • Spreadsheets
  • 4GL/Desktop Query Tools
  • Decision Support Systems (DSS)
  • Multi-dimensional DBs (MDDs)
  • OLAP (On-line Analytical Processing
  • Executive Information Systems (EIS)
  • Data Visualization Tools
  • Data Mining Tools
  • Business Modeling and Simulation Tools

67
Data Warehousing Tools and Technology
  • Desktop Data Bases
  • Structured for Database Manipulation
  • Provides facility for selecting, and loading
    of Desktop DBs from Informational DBs
  • Provides ability to Create Highly
    Personalized Informational Systems
  • Examples
  • Access
  • Paradox
  • dBase/FoxPro/Clipper

68
Enterprise Network Computer Architecture
  • Spreadsheets
  • Structured to get any subset of Information
  • Ability to Interface with standard
    Spreadsheet tools (
  • Examples
  • Excel
  • 1-2-3
  • Quatro Pro

69
Enterprise Network Computer Architecture
  • Ad Hoc Query Systems
  • Tailored for Flexible Reporting
  • Ability to do Sophisticated Analysis
    Functions
  • Aimed a a variety of users from casual to
    the power user
  • Examples
  • Focus for Windows (IBI)
  • SAS
  • Business Objects
  • GQL (Anadyne)
  • Esperant (Software AG)
  • Forrest Trees (Platinum)
  • Visualizer (IBM)
  • Impromptu (Cognos)
  • Beacon (Prodea)

70
Enterprise Network Computer Architecture
  • Multi-dimensional Databases (MDDB)
  • OLAP (On-line analytical processing)
  • Highly Structured Data
  • Tailored for Financial Modeling
  • Tailored for Power Users
  • Ability to do Sophisticated Financial
    What-if Analysis
  • Ability to drill-down from high-level to
    Detail Data
  • Examples
  • Acumate (Kenan Tech.)
  • Beacon (Prodea)
  • CrossTarget (Dimensional Insight)
  • eSSbase (Arbor)
  • Oracle Express (Oracle)

71
Enterprise Network Computer Architecture
  • Executive Information Systems (EIS)
  • Highly Structured Data
  • Tailored for Non-technical Users
  • Ability to slice and dice data
  • Ability to drill-down
  • Examples
  • Commander OLAP Server
  • Pilot (Lightship)
  • VB
  • Powerbuilder

72
Enterprise Network Computer Architecture
  • Data Visualization
  • Automatic Categorization
  • Visualization of Multi-dimensional data
  • Automatic Analysis and/or Indexing
  • Examples
  • WinViz (IBI)
  • dbExpress (Computer Concepts)
  • Data Explorer (IBM)
  • ARC Info/ARC View
  • Strategic Mapping

73
Enterprise Network Computer Architecture
  • Data Mining
  • High Speed Analysis of Detail Data
  • Constructs Business Patterns
  • Provides Statistical Support
  • Examples
  • IBM beta-test
  • Information Harvester
  • IDIS
  • d.b.Express
  • DataMind

74
Enterprise Network Computer Architecture
  • Business Modeling and Simulation
  • Business Feedback Model
  • Direct Manipulation
  • Business Gaming
  • Management/Operations Training
  • Examples
  • SimRefinery
  • SimTelephone
  • iThink
  • Microworlds

75
3. Meta-data Repository Layer
  • Data Dictionary/
  • Repository
  • Meta-data Modeling
  • Meta-data Updating
  • Meta-data
  • Examples
  • o Platinum
  • o Rochade
  • o MSP
  • o Data Atlas (IBM)
  • o MS/TI

76
3. Process (Systems) Management
  • Process Management
  • Scheduling
  • Execution
  • Subscription
  • Examples
  • o Data Harvester
  • o Data Hub
  • o Detect and Alert
  • (Comshare)

77
3. Post-processing/Indexing Layer
  • Post-processing/
  • Indexing
  • Examples
  • Sybase IQ Accelerator
  • OMNIdex
  • Oracle 7.3
  • eSSbase
  • IRI Express
About PowerShow.com