What's All This About Digital Preservation: Challenges, Barriers, Realities

1
'What's All This About Digital Preservation
Challenges, Barriers, Realities'

• Joint UK- Ireland Digitisation Workshop
• Frances Boyle
• 20th May 2008

2
Overview

• Terminology and Definitions
• Why bother the rationale, the drivers
• Challenges
• The DPC
• Snapshot of DP activity
• Key Messages

3

• Digital documents last
• forever or five years,
• which ever comes first
• Jeff Rothenberg 1997

Bytes dont survive well in regimes of benign
neglect
Clifford Lynch

4
Terminology

• Digital Preservation
• all of the actions required to maintain access
  to digital materials beyond the limits of media
  failure or technological change.
• Preservation Management of Digital Materials A
  Handbook, 2001. p.10
• Digital Preservation
• combines policies, strategies and actions that
  ensure access to information in digital formats
  over time.
• ALCTS PARS blog 2007

5
Digital Preservation - Aims

• Key aims for digital preservation
• Keeping digital assets for as long as they are
  needed
• Making sure they can still be accessed,
  regardless of inevitable changes in hardware and
  software
• The assets remain an accurate rendering of
  authenticated content over time

6
DP - Key Concepts

• Accessibility i.e. Having information as long as
  you need it.
• Material on the web can disappear (link rot)
• Anarchic development of the web individuals can
  add/remove material at random
• Material stored offline is vulnerable to media
  instability and technological obsolescence
• Material not securely backed up is vulnerable to
  sudden catastrophic loss

7
DP - Key Concepts

• Authenticity i.e. How can you be sure the digital
  information you are citing is authentic?
• Changes can be made with ease and are difficult
  to detect changes could have been innocently or
  maliciously made
• Material may need to be verified for evidential
  reasons
• Scholars need to be confident that they are
  citing the correct version

8
DP - Key Concepts

• Longevity
• Material created for one purpose may have other
  uses
• How do we know what material is worth preserving
  and who should do it?
• Organisations wishing to preserve material often
  have no control over it
• Opportunities to reuse, repurpose

9
Why Bother? - Rationale

• 988 exabytes of digital information created in
  one year
• Need to maximise investment so
• recycle
• reuse
• repurpose
• Growth of mass digitisation projects
• Reliance on digital resources growing
• Digital lives - digital footprints
• Dont want to wait for a catastrophe
• We are creating technical solutions

10
Why Bother? - Drivers

• External drivers from all sectors
• Government initiatives
• Modernising Government
• Public good reuse of publicly funded data
• Public service transparency, accountability
• Research agenda
• Data deluge
• Funding body mandates
• RIN Data Forum
• Legal compliance
• UK FoI, DPA
• Legal claims/evidence
• Financial requirements
• IPR etc

11
Challenges - Organisational

• Technological, organisational, cultural, legal
  overlap
• Need integrated, joined up and committed
  strategies
• Need to begin to embed good practice
• Holistic life cycle approach
• Digital materials need to be actively managed
  over time
• Preservation management needs to start as early
  as possible in the lifecycle, preferably from
  creation
• Many more people need to be involved creators
  are particularly important
• Roles and responsibilities are less clearly
  defined
• A better understanding of costs

12
Challenges - Technical

• Media degradation
• Multitude of formats
• Technology obsolesce e.g. software/hardware
• Machine dependency
• Scale

13
Challenges - Preservation Strategies - 1

• A number of strategies adopted
• Migration of information
• Periodic transfer of digital assets
• Different degrees of migration
• Normalisation - at ingest migrate to more
  stable format
• Emulation
• Preserve original bit streams
• Hardware/software to allow programmes to run on
  different platforms which ape the behaviour of
  original system

14
Challenges - Preservation Strategies - 2

• Technology preservation
• Keep kit plus asset - sustainable?
• Digital archaeology - forensic rescue
• Retrieval of inaccessible assets
• They are not mutually exclusive all part of
  preservation planning
• Need to consider a range of factors to determine
  approach e.g. data type, costs, timeframes, staff
  resource, risks etc.
• OAIS - management framework. It provides
• Common vocabulary
• Defines model for objects metadata
• Defines functional model for activities

15
Presservation Pyramid - Caplan
Preservation Pyramid Caplan
Authentication

Format strategies
Documentation
Capture Selection
16
Creation file format metadata
Preservation active storage look and
feel digital rights
Appraisal retention schedules significant
properties
Access discovery delivery
Digital Life-cycle
17
Challenges - Costs - 1

• espida project - Glasgow University
• Developing a business-focused model
• Digital materials are assets
• Help to understand value of your assets
• Help to manage them properly in accordance with
  their value
• Roles, responsibilities, costs, benefits and
  risks
• Value
• Needs to be contextualised for the
  organisation/collection

18
Challenges - Costs - 2

• LIFE LIFE2 (BL UCL)
• Develop a life-cycle methodology
• Enable costs to be identified for digital
  materials
• LIFE2 refining model testing on new collections
  
• Preservation is one of the six stages
• Apply this model to any digital collection
• L AqITMTAcTSTP

19
Challenges - Costs - 3

• Archaeology Data Service cost model
• C A I D R
• where
• C (Cost of preservation)
• A (Management and Administration)
• I (Ingest costs)
• D (Dissemination costs)
• R (Refreshment costs)
• Recent JISC study Keeping Research Data Safe

20
Digital Preservation Where we are

• No overall national policy in the UK
• In development in other countries
• Major UK players
• National Archives e.g. TNA, NAS
• National Libraries e.g. NLS, BL
• Funding Bodies e.g. JISC, RCUK
• UK bodies e.g. DPC, DCC, UKWAC
• International activity
• Pan European initiatives FP6 FP7
• Funding is often short-term

21
Digital Preservation Coalition - Who

• Membership organisation
• Cross-sectoral
• Formed in 2001, and became a registered company
  in 2002
• Formed in the belief that there needs to be
  concerted co-ordinated effort to progress the DP
  agenda in the UK
• 30 members e.g. JISC, BL, PRONI, MLA, RLUK,
  RCUK, UKDA, BBC etc

22
DPC Aim

• Overall aim of the Coalition is to secure the
  preservation of digital resources in the UK and
  to work with others internationally to secure our
  global digital memory and knowledge base
• Aims to develop a UK digital preservation agenda
  within an international context
• Key message - raise inform public
  practitioner awareness, an authoritative voice

23
Member Benefits
24
DPC Impact 1

• Mind the Gap report
• Challenge of quantifying the problem
• Challenge of articulating the danger
• Between hysteria (Digital Dark Ages) and
• Complacency (the problem is exaggerated)
• More facts and figures on which to base
  sustainable forward plans
• Information which will engage funders and
  decision makers
• Recommendations and Needs

25
DPC Impact 2

• Mind the Gap recommendations
• Growth in awareness needed
• Awareness into action
• New discipline in an evolving environment
• Government policy
• Legal and regulatory frameworks
• Collaborative standards and methodologies
• MtG follow up project
• Recent ITT - 20/05/08
• Recommendations ? Prioritised Action Plan
• Report by October 2008

26
DPC Impact 2
27
DPC Impact Digital Preservation Award

• A biannual event recognises achievement in the
  digital preservation field
• Sponsored by the DPC, it is aimed at encouraging
  creative and practical approaches to the
  preservation of born-digital materials
• The 2007 winner was a team from the TNA who
  developed a tool to identify and examine file
  formats

28
DPC Impact

• Training
• DPTP JISC funded led by ULCC other DPC
  members
• Core modules in association with Cornell
• Responding to member needs survey
• Other European countries setting up similar
  bodies
• Netherlands Coalition for Digital Preservation
  (NCDD)
• Undertaking a national survey government funded
• Germany nestor
• Denmark looking to harness government funding

29
DPC Impact

• Publications
• Preservation Management of Digital Materials
  Jones Beagrie
• TWR - User-friendly reports the DPC on emerging
  preservation standards, technical formats, and
  tool developments
• Reports available on the DPC website include
• Preservation metadata
• JPEG2000 A Practical Preservation Format?
• Alerting quarterly digest on dp activities

30
UNESCO Guidelines

• 5.2.3 Responsibility
• Everyone does not have to do everything
  everything does not have to be done all at once
• it is usually better for non-comprehensive and
  non-reliable action to be taken than for no
  action at all. Small steps are usually better
  than no steps
• 5.2.10 Management
• Waiting for comprehensive, reliable solutions to
  appear before taking responsible action will
  probably means material is lost
• 9.3.2
• the often-repeated claim that digital
  preservation involves very long-term commitment
  may well act as a barrier to preservation by
  discouraging agencies that are well placed to
  take short-term action when it is needed.

31
Key messages

• We cant afford not to find workable solutions to
  preserving digital heritage
• Who can afford to lose their existing and future
  knowledge base in the knowledge driven economy
• It will become increasingly cost-effective as we
  harness practical experience, collaboration,
  standards and services, and research
• Retrospective actions are much more difficult and
  far more costly
• Dont lose sight of the endgame digital
  preservation is a business need
• Develop a sustainable, scaleable strategy
• Look at best practice globally
• Not all challenges need to be overcome before
  taking action

32
Contact

• www.dpconline.org
• fb_at_dpconline.org