Building the Archives of the Future:

1
Building the Archives of the Future The National
Archives Records Administrations Electronic
Records Archives Initiative
Kenneth Thibodeau, Ph.D. Director Electronic
Records Archives Program National Archives and
Records Administration
2
National Archives Records Administration(NARA)

• An independent agency in the Executive Branch of
  the U.S. Government
• Provides direction and assistance for lifecycle
  management of records of all federal agencies
• Preserves historically valuable federal records
  in the National Archives
• Provides courtesy storage for records of the
  Congress and the Supreme Court in the National
  Archives
• Preserves Presidential records in the
  Presidential Libraries
• Operates Federal Records Centers for inactive
  records which remain under the legal control of
  their creators
• Publishes the Federal Register

3
The Challenge NARA Faces in Preserving Electronic
Records

• Preserve any type of electronic record,
• Created using any type of application,
• On any computing platform
• From any entity in the Federal Government,
• and any donor
• Provide discovery and delivery to anyone with an
  interest and legal right of access
• Now and for the life of the Republic

4
Facets of the Challenge of Preserving Electronic
Records

• Obsolescence constantly changing technology
• Variety different types of records
• Complexity records in complex formats and with
  demanding behaviours
• Volume enormous numbers of records
• Scope the entire federal government, plus
• Time Frame from x years to forever
• Authenticity assuring the record remains as
  reliable as when first created
• Accessibility the public has a right of access
  to the records of its government

5
The Challenge Obsolescence

• Data Types (aka format)
• Access to electronic records entails processing
  stored bits
• Data types depend on specific applications
  software
• Applications depend on operating systems
• Operating systems make assumptions about hardware
• Software, operating systems and hardware become
  obsolete
• Storage media
• Finite lifespan
• Rapid succession of newer media and related
  hardware
• Continuing improvements in cost/performance

6
The Challenge Variety
Any type of electronic record Some examples

• Office Automation Files
• Word processing documents
• Spreadsheets
• Presentations
• E-mail,
• with attachments
• Scanned paper documents

• Digital Photography
• Satellite Imagery
• Digital audio files
• HDTV
• Web pages
• Databases
• Geographic Information Systems

7
The Challenge Complexity

• Documents with embedded or linked objects
• Databases with hundreds of tables, forms,
  reports, views
• XML files with Schemas, Style Sheets, Forms,
  namespaces, extensions, et al.
• Interactive Web pages
• Virtual Reality Models
• Product data needed for manufacture

8
Complexity Operational Context
9
The Challenge Volume

• Clinton Administration 32,000,000 email
  messages
• State Department 25,000,000 electronic
  diplomatic messages, 1972 - 2000
• Department of Defense 1,000,000,000 images of
  digital Official Military Personnel Files
• Census Bureau 600 to 800,000,000 million image
  files (2000 census)

10
Projected Storage VolumesPermanent Electronic
Records
11
The Challenge Authenticity

• Requirement A record must remain as reliable as
  it was when first created
• Identified
• Uncorrupted
• Premises
• Most digital formats will become obsolete to the
  extent that they become either completely
  inaccessible or effectively inaccessible using
  the technologies customers want to use.
• Preserving accessibility will require
  transforming the digital encoding of obsolete
  electronic records.

12
The Challenge Accessibility

• Requirement Any individual has a right of access
  to federal records must remain as reliable as it
  was when first created
• Subject to restrictions based in law
• Premises
• Individuals will want to use the best currently
  available technologies for access to electronic
  records.
• Records in obsolete digital formats will not be
  readily available in current technology.

13
NARAs Strategy

• Attack the critical preservation problem
• Define the requirements in terms of the lifecycle
  management of records
• Align with overall direction of Information
  Technology in the U.S. Government
• Find solutions in commercially viable, mainstream
  technologies being developed to support
  e-commerce, e-government and the next generation
  national information infrastructure

14
The ERA System Will Support

• Workflow and collaboration tools for lifecycle
  management of all types of records, temporary and
  permanent
• within NARA
• between NARA and other agencies
• Ingestion, preservation, management, and access
  to electronic records
• In the National Archives,
• Federal Records Centers, and
• Presidential Libraries

15
Strategic Alignment
Strategies to assure long-term preservation of
digital records constitute another particularly
pressing issue for research....
16
ERA Development Plan
2004
2011
2005
2006
2010
2009
2008
2007
Jul
Oct
Jan
Apr
Jul
Oct
Jan
Apr
Jul
Oct
Jan
Apr
Jul
Oct
Jan
Apr
Jul
Oct
Jan
Apr
Jul
Oct
Jan
Apr
Jul
Oct
Jan
Apr
Jul
Oct
Jan
FOC
Increment 5 Preservation and Capacity
Increment 4 Preservation and Capacity
Increment 3 Preservation and Capacity
Increment 2 Secure Facility/ Instances and
Redaction
Increment 1 Fully Instantiated Archives
IOC
SDR
Analysis and Design
17
System Design Drivers

• Obsolescence Improved Technology Time Frame ?
• Evolvability
• Growing Volumes Special Needs ?
• Scalability, up and down
• New Data Types Increasing Complexity ?
• Extensibility
• Authenticity Accessibility ?
• Persistent Preservation

18
How the Lockheed Martin Architecture and Design
Satisfies NARAs Needs
19
Evolvability, Scalability, ExtensibilitySystem
Architecture
20
Evolvability Storage Design
Business Applications
Mediation Layer
Storage Manager

• Storage Manager
• Product independence
• Heterogeneous Environment
• Storage Equipment
• Storage Media

Equipment and Media
21
Authenticity AccessibilityPreservation

• Preservation Framework
• Define what needs to be preserved
• Specifically for preserving records
• Identify tools which meet those needs
• Apply tools which transform the digital encoding
  into a one suitable for preservation

22
Preservation Planning
I must have pagination preserved. Id really like
color, if possible.
Preservation Plan
Record
???
Legal Contract
Defined Objectives
ERA Matches System Capabilitiesto Archival
Objectives to Maximize Best-Fit
23
Preservation Processing
If it were only simple
Adapted Data File
Record
Data File
Memorandum
Word Document
Adobe PDF-A
Prototype Tackles Complex Records and Complex
Data Types
24
Challenges of Persistent Archives
25
Making the Connections

• Making sense of the 0s and 1s is dependent on a
  web of connections

Binary Sequence
Record Context
Data Type Template
Record Type Template
Documentation
Encoding
Processing
Archival Context
Data Format
The Persistence of an Archives is Only as Strong
as its Weakest Link
26
Lockheed Martin PrototypeViews
27
General Users View Initial View into Search
28
Records Manager Workbench
29
Disposition AgreementDescribe the Records
Covered by the Agreement
30
Disposition AgreementDefine Retention Periods
and Dispositions
31
Disposition AgreementAttaching Sample Records to
Assist Appraisal
32
For Additional Information
www.archives.gov/era