North Carolina Geospatial Data Archiving Project/NDIIPP: Collection and preservation of at-risk digital geospatial data Partners: NCSU Libraries Project Lead: Steve Morris NC Center for Geographic Information

1
North Carolina Geospatial Data Archiving
Project/NDIIPP Collection and preservation of
at-risk digital geospatial data PartnersNCSU
Libraries Project Lead Steve MorrisNC Center
for Geographic Information Analysis Project
Lead Zsolt Nagy
State Archives Meeting
February 24, 2006
2
Project Context

• Partnership between NCSU Libraries and NCCGIA
  under NDIIPP
• Focus on state and local geospatial content in
  North Carolina (state demonstration)
• Tied to NC OneMap initiative, which provides for
  seamless access to data, metadata, and inventory
  information
• Objective engage existing state/federal
  geospatial data infrastructures in preservation

3
Targeted Content

• Resource Types
• GIS vector (point/line/polygon) data
• Digital orthophotography
• Digital maps
• Tabular data (e.g. assessment data)
• Content Producers
• Mostly state, local, regional agencies
• Some university, not-for-profit, commercial
• Selected local federal projects

4
Local Government GIS Landscape

• Data resources are highly distributed and subject
  to frequent update
• More detailed, current, accurate than
  federal/state data resources
• North Carolina local agency GIS environment
• 100 counties, 95 with GIS
• 85 counties with high resolution orthophotography
• Growing number of municipal systems
• Value hundreds of millions of dollars investment

5
Geospatial data types Vector data
6
Geospatial data types Aerial imagery
7
Geospatial data types Aerial imagery
8
Geospatial data types Aerial imagery
9
Geospatial data types Tabular data (w/vector)
10
Time series vector data Parcel Boundary Changes
2001-2004, North Raleigh, NC
11
Time series Ortho imagery Vicinity of
Raleigh-Durham International Airport 1993-2002
12
Todays geospatial data as tomorrows cultural
heritage
13
Risks to Digital Geospatial Data

• Producer focus on current data
• Time-versioned content generally not archives
• Future support of data formats in question
• Vast range of data formats in use--complex
• Shift to streaming data for access
• Archives have been a by-product of providing
  access
• Preservation metadata requirements
• Descriptive, administrative, technical, DRM
• Geodatabases
• Complex functionality

14
Earlier NCSU Acquisition Efforts

• NCSU University Extension project 2000-2001
• Target County/city data in eastern NC
• Digital rescue not digital preservation
• Project learning outcomes
• Confirmed concerns about long term access
• Need for efficient inventory/acquisition
• Wide range in rights/licensing
• Need to work within statewide infrastructure
• Acquired experience unanticipated collaboration

15
Work plan in a Nutshell

• Work from existing data inventories
• NC OneMap Data Sharing Agreements as the
  blanket, individual agreements as the quilt
• Partnership work with existing geospatial data
  infrastructures (state and federal)
• Technical approach
• METS with FGDC, PREMIS?, GeoDRM?
• Dspace now re-ingest to different environment
• Web services consumption for archival development

16
NCGDAP Philosophy of Engagement
Provide feedback to producer organizations/ inform
state geospatial infrastructure
Take the data as in the manner In which it can
be obtained
Wrangle and archive data
Note the Project in North Carolina Geospatial
Data Archiving Project the process, the
learning experience, and the engagement with
geospatial data infrastructures are more
important than the archive
What is the long term solution?
17
Expected Project Outcomes

• Demonstration archive
• Outreach activity planting seeds
• International, national, state, local, commercial
• Learning experience, informing
• Spatial data infrastructure
• Commercial vendors (data/software/consulting)
• Repository software communities
• Metadata practice (both GIS preservation)
• Rights management developments
• Data and interoperability standards

18
Big Challenges

• Format migration paths
• Management of data versions over time
• Preservation metadata
• Harnessing geospatial web services
• Preserving cartographic representation
• Keeping content repository-agnostic
• Preserving geodatabases
• More

19
Preserving Cartographic Representation
Counterpart to the map is not just the dataset
but also models, symbolization, classification,
annotation, etc.
20
Project Status

• Storage system and backup deployed
• DSpace deployed
• FGDC Metadata workflow finalized
• Ingest workflow near finalization
• Content migration workflow plan near finalization
• Regional site visits planned for coming months
• Wide range of outreach/collaboration FGDC, ESRI,
  EDINA (JISC), USGS, OGC, TRB, etc.
• Pilot project, georegistering digital archival
  geologic maps

21
Questions?
Contact Steve Morris Head, Digital Library
Initiatives NCSU Libraries Steven_Morris_at_ncsu.edu
Web site http//www.lib.ncsu.edu/ncgdap/
22
Risks to Digital Geospatial Data
.shp
.mif
.gml
.e00
.dwg
.dgn
.bsb
.bil
.sid
23
Metadata Availability
24
NC OneMap Initial Data Layers Produced by Cities
and Counties
25
Content Identification and Selection

• Work from NC OneMap Data Inventory
• Combine with inventory information from various
  state agencies and from previous NCSU efforts
• Develop methodology for selecting from among
  early, middle, and late stage products
• Develop criteria for time series development
• Investigate use of emerging Open Geospatial
  Consortium technologies in data identification

26
Content Acquisition

• Work from NC OneMap Data Sharing Agreements as a
  starting point (the blanket)
• Secure individual agreements (the quilt)
• Investigate use of OGC technologies in capture
• Explore use of METS as a metadata wrapper
• Ingest FGDC metadata Xwalk to MODS? PREMIS?
• Maybe METS DRM short term GeoDRM long term
• Consider links to services version management
• Get the geospatial community to tackle the
  content packaging problem (maybe MPEG 21?)

27
Partnership Building

• Work within context of the NC OneMap initiative
• State, local, federal partnership
• State expression of the National Map
• Defined characteristic Historic and temporal
  data will be maintained and available
• Advisory Committee drawn from the NC Geographic
  Information Coordinating Council subcommittees
• Seek external partners
• National States Geographic Information Council
• FGDC Historical Data Committee
• more

28
Content Retention and Transfer

• Ingest into Dspace
• Explore how geospatial content interacts with
  existing digital repository software environments
• Investigate re-ingest into a second platform
• Challenge keep the collection repository-agnostic
  
• Start to define format migration paths
• Special problem geodatabases
• Purse long term solution
• Roles of data producing agencies, state agencies
  NC OneMap NCSU

29
Rights Issues

• Various interpretations of public records law
• 53.9 of local NC agencies charge for data
• 43.7 of local NC agencies restrict
  redistribution
• Desire for downstream control of data
• Disclaimer clickthrough liability concerns
• Filtered locations/individuals post 9/11 issues
• Restrictions on redistribution commercial resale
• Web services area in Wild West stage
• Both content and technical agreements
• GeoDRM initiative in the works

30
Big Challenges

• Management of data versions over time
• How to get current object/metadata/DRM?
• Relation of the canonical metadata package to the
  ingest (and export) metadata package
• Tailor to repository environment or make the
  acquaintance when needed?
• Format migration paths (geodatabases, etc.)
• Preserving Cartographic Representation
• The counterpart to the map is not just the
  datasetalso models, symbology, interpretation,
  etc.

31
Geospatial data types Satellite imagery