Report on Preservation of ETDs: The LOCKSS Prototype

1
Report on Preservation of ETDsThe LOCKSS
Prototype

• The work of Kamini Santhanagopalan Virginia Tech
  Graduate Student in Computer Science
• Reported at the 9th International Symposium on
  ETDs, Quebec City

Presented By Gail McMillan, Director Digital
Library and Archives Virginia Tech
2
Agenda

• Goals
• What is LOCKSS?
• Participating Universities
• International ETD Preservation
• Analysis and Results
• Conclusion

3
Digital Preservation

• Goal Information should be
• Readable
• Usable in the future
• Preservation NOT just backup
• Existing preservation techniques
• Floppy, CD and hard disk drives
• Central and distributed database servers

4
Technical Infrastructure Goals

• Build on successful LOCKSS open-source model
• Create dark archive for locally produced digital
  content
• Use off-the-shelf hardware
• Use open-source software
• Easy replication
• Demonstrate LOCKSS scalability

5
LOCKSS

• Lots of Copies Keep Stuff Safe
• Peer-to-peer digital preservation system
• Open source software
• Turns an inexpensive desktop computer into a
  digital preservation appliance
• Easy, inexpensive way to
• Collect
• Store
• Preserve
• Provide access to the contents--or, not.

6
Functions of LOCKSS (1)

• Collect
• Via a web crawler
• Appropriate crawl rules are specified
• Preserve and Audit
• Every institution preserves
• Its own contents
• Contents of partner universities
• Contents are polled to determine authenticity and
  reinstate bad files

7
Functions of LOCKSS (2)

• Provide access
• By running web proxies
• Open or restricted access
• Dark Archives for partners ETDs
• Levels of access controlled at originating
  institutions
• Administration
• Via a web user interface
• Controlling access to cached contents and other
  functions

8
LOCKSS Preservation

• Contents of each university (nodes M1 through M5)
  preserved at every other university
• Multiple, dispersed copies
• Not a backup-- nothing is overwritten
• All versions retained

9
ASERL-LOCKSS-ETD Initiative

• Florida State University
• Georgia Institute of Technology
• University of Kentucky
• University of Tennessee
• Vanderbilt University
• Virginia Polytechnic Institute and State
  University
• http//www.aserl.org/

10
Preservation using LOCKSS

• Prerequisites
• Minimum hardware configuration
• LOCKSS software installed on all participating
  partners systems
• Permissions for the LOCKSS system to collect,
  preserve, periodically validate, repair ETDs

11
Example Hardware Configuration

• Enterprise (3TB)
• Dell PowerEdge Server 1850 LOCKSS - 3500
• Dell PowerEdge Server 1850 Firewall - 2500
• Dell/EMC AX100 SAN (3TB) - 10,000
• RedHat Enterprise AS 2_at_50 100
• UPS - 700
• Server Rack - 1200
• Grand Total - 16,800.00
• w/ Rack - 18,000.00

• Desktop (200Gb)
• Intel Based Desktop LOCKSS (200Gb) - 500
• Intel Based Desktop Firewall - 350
• CentOS Linux - 0
• UPS - 50
• Grand Total - 900.00

12
Participating Universities

• International universities
• Pontifícia Universidade Católica do Rio de
  Janeiro, Brazil
• Humboldt-Universität, Germany
• University of Cape Town, South Africa
• US universities
• Florida State University
• Georgia Tech
• Virginia Tech

13
International ETDs Preservation (1)

• For international universities
• KS wrote plug-ins to collect contents (ETDs) from
  the 3 universities
• For US universities
• Verified and reused OAI plug-ins for the 3
  universities

14
International ETD Preservation (2)

• Example ETD collection
• University of Cape Town ETD collection
• Manifest (i.e., permissions) page
  http//pubs.cs.uct.ac.za/lockss/manifest.html
• Screen shots of UCT plug-in and the crawl results
  of contents follow

15
University of Cape Town Plug-in (1)
16

• UCT plug-in
• Crawl Results with
• Level (depth) 4
• Fetch delay 6 seconds

17
Harvested International ETD Collections
18
Harvested American ETD Collection source
http//lockss-etd.lib.vt.edu8081/DaemonStatus
19
Tutorial on how to write plug-ins

• KS developed mini-tutorial http//scholar.lib.vt.e
  du/lockss/introduction.htm
• 10 screens
• This tutorial can be
• Generalized for ETD plug-ins
• Extended to write OAI plug-ins

20
Conclusion and Future Work

• International ETDs can be harvested and preserved
  using LOCKSS and OAI-PMH
• It requires cooperation and collaboration from
  participating universities
• Future Work
• An online portal open for the public to view
  certain details
• Brazil expressed interest in formalizing ETD
  preservation for the NDLTD using LOCKSS

21
Acknowledgements

• Special thanks to LOCKSS (Stanford University)
• Thomas Robertson
• Seth Morabito
• Thanks to all participating universities
• Florida State
• Georgia Tech
• Humboldt-Universität, Germany
• Pontifícia Universidade Católica do Rio de
  Janeiro, Brazil
• University of Cape Town, South Africa
• Virginia Tech

22
Send Questions/Comments to ksanthan_at_vt.edu