ADaM System Architecture - PowerPoint PPT Presentation

Loading...

PPT – ADaM System Architecture PowerPoint presentation | free to download - id: 142314-NTVlN



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

ADaM System Architecture

Description:

General Mining Steps. Select data files to be mined ' ... Write a 'Mining Plan' consisting of sequence of input filter and operations ... – PowerPoint PPT presentation

Number of Views:33
Avg rating:3.0/5.0
Slides: 24
Provided by: ipam
Learn more at: http://www.ipam.ucla.edu
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: ADaM System Architecture


1
ADaM System Architecture
  • Rahul Ramachandran, Sara Graves and
  • Ken Keiser
  • Mathematical Challenges in Scientific Data Mining
  • IPAM January 14-18, 2002
  • Information Technology and Systems Center
  • University of Alabama in Huntsville
  • rramachandran_at_itsc.uah.edu

2
Talk Overview
  • Mining System Requirements
  • ADaM System Architecture
  • ADaM Plan Builder
  • Research directions

3
Mining System Requirements When,Where and Who
  • WHERE
  • User Workstation
  • Data Archive Center
  • Data Mining Center
  • WHEN
  • Real Time
  • On-Ingest
  • On-Demand
  • Repeatedly
  • WHO
  • Casual Users
  • Domain Experts
  • Mining Experts

Data Mining
4
Algorithm Development and Mining (ADaM) System
  • ADaM system developed under NASA research grant
  • The system provides knowledge discovery, feature
    detection and content-based searching for data
    values, as well as for metadata.
  • It contains over 120 different operations to be
    performed on the input data stream.
  • Operations vary from specialized atmospheric
    science data-set specific algorithms to different
    digital image processing techniques, processing
    modules for automatic pattern recognition,
    machine perception, neural networks and genetic
    algorithms.

5
ADaM Features
  • Handles science data set variability
  • Multiple resolution/multiple scales
  • Variability of formats
  • Granularity of data
  • Includes spatial/temporal dimensions
  • Allows addition of new algorithms
  • Allow scientists to select and sequence different
    operations

6
ADaM Engine Architecture
Preprocessed Data
Patterns/ Models
Results
Data
Translated Data
Processing

Preprocessing
Analysis
Selection and Sampling Subsetting
Subsampling Select by Value Coincidence
Search Grid Manipulation Grid Creation
Bin Aggregate Bin Select Grid Aggregate
Grid Select Find Holes Image Processing
Cropping Inversion Thresholding Others...
Clustering K Means Isodata
Maximum Pattern Recognition Bayes Classifier
Min. Dist. Classifier Image Analysis
Boundary Detection Cooccurrence Matrix
Dilation and Erosion Histogram Operations
Polygon Circumscript Spatial Filtering
Texture Operations Genetic Algorithms Neural
Networks Others...
7
ADaM Mining Environment
Data Mining Server
Mining Results
Event/ Relationship Search System
8
ADaM Architecture
9
ADaM Miner Engine
  • Manages the processing of data through a series
    of specified operations
  • Loads input, processing and output modules
    dynamically as needed at execution time
  • Allows for the addition of newly developed
    modules without the need to rebuild the engine
  • Interprets a mining plan script that provides the
    details about specified operations and the order
    that they should be executed

10
ADaM Miner Database
  • Used to store information that includes the
    names, locations and related metadata for input
    data sets available on the server
  • Includes information about users, jobs, mining
    results, and other related information
  • Simple relational database

11
ADaM Daemon and Scheduler
  • Scheduler
  • Examines the list of jobs to be executed on the
    server and determines which job or jobs to
    execute at any given time
  • Queues the requests and executes them
    sequentially.
  • Daemon
  • Handles all network communications with the
    mining system
  • Is configured to listen on a specific port for
    any socket communications

12
ADaM Input/Operation Filters
  • Input/Output Filters are data readers and writers
  • Operations are the algorithms
  • Each of the operations and (input/output)
    filters is implemented as a shared library
  • New modules may be added to the system without
    recompiling or relinking.
  • All operations/filters either produce or operate
    on a data collection, which provides a common
    format for representing scientific data.

13
General Mining Steps
  • Select data files to be mined
  • Check-In the data files into the Miner Database
  • Write a Mining Plan consisting of sequence of
    input filter and operations
  • Execute the Mining Plan using the engine
  • Check and save results
  • Iterate

14
What is Check-In?
  • Process of encoding information such as the
    names, locations and related metadata for input
    data sets available on the server
  • Create complex data hierarchy in the database

15
ADaM Plan Builder Check-In
  • Two Modes of Operation
  • General which only requires
  • minimal information
  • Advanced requires more
  • detailed information and
  • Allows user to set up
  • structured database

Path to the data file
Data file name
Input Filter associated with the Data file
Load an XML file containing existing Check-In
specifications
16
ADaM Plan Builder Layout
Operation Menu contains the list of operations
one can select
Input Menu contains the list of Input Filters one
can select
  • Plan Menu allows one to
  • Select a new plan
  • Load existing plan
  • Check-In data

17
ADaM Plan Builder Layout
Panel where Mining Plan can be viewed either as
text or a tree
18
ADaM Plan Builder Layout
Description about the Operation/Input Filter can
be viewed in this panel
19
ADaM Plan Builder Layout
All the parameters needed for the Operation are
described here
20
ADaM Plan Builder Layout
Sample values for Operations parameters are
shown in this panel
21
ADaM Plan Builder Layout
Allows user to select the operation and add it to
the Mining Plan
Go Mine the data using the Mining Plan
22
Research Directions
  • Generic Data Reader for ADaM
  • ESML Earth Science Markup Language
  • Programmers Guide for ADaM
  • Distributed Mining
  • Grid Mining
  • Successful implementation and testing of the ADaM
    system on the NASA Information Power Grid
  • Mining Onboard the Space Craft
  • The EnVironmEnt for On-Board Processing (EVE)
    system

23
ADaM Information
  • Web site
  • datamining.itsc.uah.edu
  • ADaM Lite beta version download
  • Contact rramachandran_at_itsc.uah.edu
About PowerShow.com