A%20Data%20Management%20Life-Cycle

1
A Data Management Life-Cycle
By David Ferderer Project Chief Chris
Skinner Contractor Greg Gunther Contractor d
ferdere_at_usgs.gov
2
Presentation Outline

• USGS Landscape
• Life-Cycle Model and Strategy
• Component Descriptions (Skinner)
• Demonstration (Gunther)
• Conclusions and Future Directions

3
USGS Landscape - Energy Program

• What We Do
• Provides Science-Based Energy Assessments
• Organization Issues
• Regional Centers and Competitive Funding Process
• Multiple Project Areas, Applications, Data Types,
  and Platforms
• Information Issues
• Technology and Data Explosion
• Access, Delivery, and Archive Requirements
• Diverse Client and Product Needs
• Policy and Mandates

4
USGS Landscape - Central Energy Team

• 125 Full and Part-Time Employees
• Independent Thinkers and Researchers
• Multiple Application Platforms
• UNIX (ArcInfo 8, ArcView 3x, SDE 3, ORACLE 8,
  EarthVision, Seismic, PETROMOD)
• PC/NT (ArcInfo 8, ArcView 3x, Geographix)
• Centralized and Distributed Data Storage
• 100mb Fast Ethernet Network

5
Central Energy Team Information Shift
Information Services
Data Management
Project Life-Cycle
GIS
Integration
6
Life-Cycle Model and Strategy

• Life-Cycle Model (Conceptual)
• A Series of Processes and Utilities that Manage
  the Flow of Data to Information, Products, and
  Knowledge
• Life-Cycle Implementation Strategy (Actual)
• Processes are Translated into the Find, Get, Use,
  Deliver, and Maintain Strategy
• Strategy Defines Tasks, Components, and
  Deliverables

7
Implementation Strategy

• DM Finds Internal and External Data Resources
• DM Gets the Data Organized, Documented, and
  Accessible to Team Projects
• Projects Use the Data and Other Resources in
  Research
• DM Assists Projects in Delivering Products to
  Public
• DM Maintains the System and Upgrades Components

8
Strategy Components and Utilities (Internal USGS)
Find External Data and Information
Get Data Organized
Find Internal Data and Information (Archive
and Reuse)
Data Life-Cycle
Maintain (Upgrades and Documentation)
Use Data and Other Resources In Research Projects
Deliver Data and Knowledge to Projects and the
Public
9
Team Data Library

• Centralized Storage
• Team Data Resources (primarily spatial)
• Theme and Sub-Theme Organization
• Standardized
• Naming Conventions
• Directory Structure
• Storage Formats (e00, shape, SDE)
• Common Data Projection (geographic)
• Metadata
• Browse Graphics

10
Team Archive Library

• Offline Storage of Team Data Resources
• Contains
• Publications
• USGS Digital Data Products (DLG, DEM, DOQ)
• Team Archives
• Standardized File Names and Directory Structure

11
Inventory Database

• MS Access Database Tracking Teams Data Holdings
• Contains
• 60 Information Fields (10 Required) in 21 Tables
• 28 Fields Corresponding to FGDC Metadata Elements
• Inventoried 4600 Datasets and 680 Archives (gt 500
  GB)

12
Inventory Database

• Features
• Tracks Multiple Types of Data (Spatial, Text,
  Graphic and Tabular)
• Separately Tracks Archives, Publications, and
  Individual datasets
• Automatic Loading and Editing Scripts
• Serves as the Engine to DART

13
DART

• Data Access, Retrieval, and Tracking System
• Easy Access to Team Data Resources via Web
  Browsers
• Customized Search and Browse of Archives,
  Publications, and Datasets
• Direct Data and Metadata Download to Users
  Desktop
• Object-Oriented Application
• Java Server Pages on ServeletExec 3.1
• Stay Tuned for the Demonstration!

14
Metadata Utilities

• Web-Based Metadata Entry and Creation System
• Users Generate, Modify, and Save Compliant
  Metadata Output to the Desktop
• Provides a Simplified and Comprehensive Online
  Help System
• Contains
• Links to Other Metadata Tools and Resources
• Library of Metadata

15
Other Data Management Products

• Data Processing and Automation Utilities
• Portal to How-To, AMLs, and FAQ Documents
  Residing in the Team and On the WWW
• Project and Workspace Design Recommendations
• Templates Promote Efficient Work-Flow, Data
  Organization, Archives, and Rapid Publication
• CD-ROM Templates and Hypermedia Distribution

16
Maintenance

• DM Provides Continual Maintenance and Upgrades of
  System Components
• Develop Publications and Documentation
• User Manuals
• Formal Component Documentation
• Templates, Guidelines, and Policies
• Fact Sheets and Bulletins

17
Demonstration
Greg Gunther
18
System Summary

• Easy Access to Datasets
• Generate Metadata Quickly and Easily
• Find External Data with Over 1000 WWW Links
• Simplify Data Processing Tasks
• Organizes Projects with Workspace Templates
• Streamlines CDROM Publications
• Provides One-Stop Shopping For Shared Internal
  Resources

19
Future Directions

• Increase Inventory Effort
• Integrate GeoDatabase Model (ArcGIS) for
  Proprietary Datasets
• Formalize Metadata Extension to FDGC Standard
• Streamline Product Delivery - Implement IMS
• Publish Documented Tools and Utilities
• Implement Enterprise Architecture and Planning

20
Future Architecture

• Enterprise Planning

Getting Started
Planning Initiatives
Current Systems
BusinessProcesses
Where We Are Today

Data Architecture
IS/IT Architecture
GIS Application Architecture
Where We Want To Be
Implementation and Migration Plans
Plan To The Future
Modified from Spewak Model
21
Conclusions What We Have Learned

• Data Management
• Its ESSENTIAL for Survival But Needs to be
  Promoted
• Distributed Projects REQUIRE Data Centralization
• Projects RARELY Account for Data Management
  Planning and Costs
• Data Stewardship MUST Begin at the Onset of
  Projects
• The Terms EASY and USEFUL - Lead to
  Implementation
• Component Model Must be FLEXIBLE to Adapt to
  Technology Trends

22
The End And The Beginning Of a New Cycle