The Open Archives Initiative: Perspectives on Metadata Harvesting OAI Provider

1
The Open Archives Initiative Perspectives on
Metadata HarvestingOAI Provider Harvesting
Servicesat the University of Illinois

• Timothy W. ColeMathematics LibrarianChair,
  Library Information Technology Committee
• University of Illinois at Urbana-Champaignt-cole3
  _at_uiuc.edu
• 25 June 2001

2
Illinois DL Projects Generating Metadata

• Sci-Tech Article Testbed (CNRI D-Lib Test Suite)
• Digital Imaging Multimedia Technology
• Teaching with Digital Content (IMLS)
• Historical Map Aerial Photo Digitization
  Projects
• University Archives Special Collections
• Manuscript Archive Finding Aids
• Emblem Book Digitization Project
• Kolb-Proust Archive

3
Metadata Schemas Formats

• Metadata Schemas Currently Used
• Dublin Core (DC), Qualified Dublin Core (DCQ)
• Federal Geographic Data Committee (FGDC)
• Encoded Archival Description (EAD)
• Text Encoding Initiative (TEI)
• Local custom schemas (e.g., Emblem Book Project)
• Metadata Storage Formats
• HTML, XML, RDF-XML
• Relational Databases (MS Access, SQL)
• XPat (OpenText) Document Database

4
To Implement OAI Provider Service

• Map (Transform) Metadata
• OAI Aimed at Sharing Item-Level Metadata
• OAI Requires Mapping to LCD Schema (DC)
• OAI Supports Other Schemas, But XSD Required
• Create the Service (HTTP-Based)
• Easy, Lightweight Protocol
• Does Require XML Scripting / CGI Programming
• Designed for Bulk Interchange of Metadata

5
Illinois OAI Provider Implementation

• Service Architectures
• Windows NT, IIS 4, Access / SQL Server, VBScript
• Strategy 1 Administrative Descriptive Data
  in DB
• Strategy 2 Administrative Data in
  DB Descriptive Metadata in XML Use XSLT When
  Constructing OAI Response
• Sample Turnkey Implementations Available From
  www.openarchives.org Site

6
Illustrations

• Aerial Photos Database Schema
• In DCQ-RDF
• As OAI_DC Record
• Journal Article Metadata in XML (DCQ / RDF)
• As OAI_DC Record
• EAD Finding Aid in XML
• As DCQ-RDF
• As OAI_DC Record
• Collection-Level vs. Item-Level Metadata

7
OAI Provider Service Issues Tools

• Implementation Maintenance Issues
• Records Mgmt. (Sets, Deleted Records)
• Synchronicity (Integration w/ Existing Services)
• Flow Control (Mgmt. Of Service Load)
• Resumption Token Issues
• Access Control (Restrictions on Metadata)
• Useful Tools
• OAI Repository Explorer
• W3Cs XSV Validating Parser
• OAI Registry

8
Plans for Harvesting Service

• Implement OAI Spider Indexing Systems
• Harvest Metadata -- 2 Collection Approaches
• MichiganFreely Available, Online Scholarly Full
  Content
• IllinoisCultural Heritage Materials, Whether
  Print or Digital, Whether Freely Available or
  Restricted Access
• Investigate Post-Harvesting Issues

9
Harvesting Service Research Issues

• Interface Design Value Added Services
• Impact of Cross-Schema Mapping Practices
• Options for Exploiting Non-DC Metadata
• Impact of Variations in Completeness
  Granularity of Metadata Harvested
• Techniques to Ameliorate Those Impacts
• Roles of Providers Harvesters / Sustainability
• 1-Day Workshop in Urbana Planned For Interested
  OAI Providers (September 2001)