Design and Implementation of a Distributed Data Exploration and Processing System

1
Design and Implementation of a Distributed Data
Exploration and Processing System

• by Marin Bezic
• Presented as a partial fulfillment of the
  requirements of the degree of
• MASTER OF SCIENCEin Computer Science

2
Presentation Outline

• Motivation
• Design Overview
• Enabling Technologies
• Implementation
• Applications
• Conclusions and Recommendations

3
Motivation
Marin when talking about the Environmental
Informatics show a picture or two of a grid or a
movie that shows how data turns into information

• Center for Air Pollution Impact and Trend
  Analysis (CAPITA)
• Creating information from raw monitoring data
• Data management and rendering software libraries
• Applications data servers, data operators, data
  viewers
• Problems
• Connectivity between applications
• Data abstraction

4
Motivation

• Developments in the computer technology
• The Programmers Playground
• OLE and ActiveX
• Goals
• Create distributed applications based on the
  Programmers Playground
• Provide object based abstraction for the
  environmental data

5
Distributed Data Exploration and Analysis System

• Uses
• Data quality control
• Data exploration through overlays, comparisons,
  pattern analysis
• Data manipulation through filtering,
  aggregation, fusion
• Data presentation through storybooks, animations

6
Distributed Data Exploration and Analysis System

• Pattern 1 Distributed Data Refinery

7
Distributed Data Exploration and Analysis System

• Pattern 2 Distributed Data Browser

8
Enabling Technologies

• The Programmers Playground
• CAPITAs data management libraries
• ActiveX/OLE

9
The Programmers Playground

• Tool for creating distributed applications
• Developed by the DPE Group
• Features
• I/O Abstraction
• Connection Management
• Application Management

10
CAPITAs Data Management

• Software (DLL) for management and rendering of
  spatial data
• Developed at CAPITA
• Layers Types
• Table
• Grid
• Image
• Geographic
• Windows NT, 16 bit

11
CAPITAs Data Management
12
OLE Automation and ActiveX

• OLE Automation
• A way of exposing an applications internal
  functionality
• Programmable applications
• ActiveX Controls
• encapsulate both functionality and the user
  interface
• events
• lightweight, suitable for transfer over the
  Internet

13
Implementation

• Layer Data Object
• exposes properties through the Programmers
  Playground
• ActiveX/OLE programmable interface
• encapsulates data management functionality

Layer Data Object
Data Management
PG Interface
ActiveX/OLE Interface
14
Implementation

• Layer Data Object features
• events
• self traversal
• may be script driven
• facilitates the creation of new data operators
• sends and receives data objects
• enables data exchange between distributed
  application modules

15
Implementation

• Modifications to CAPITA applications

Contourer
MapEditor
Table layer
Image layer
Grid layer
16
Applications

• Ozone Rose Analysis System
• a distributed data exploration application
• Data Animation Pipeline Creating Animations from
  Monitoring Data
• a distributed data processing application

17
Ozone Rose Analysis System
18
Demo
19
Conclusions

• Practical distributed applications can be built
  with modest programming experience
• Layer Data Object facilitates creation of both
  individual data analysis components and
  distributed applications
• Successful symbiosis continues
• CAPITA and DPE