Media madness I: PARADISEC video trial 2006

1
Media madness I PARADISEC video trial 2006

• Linda Barwick
• Sydney Conservatorium of Music
• University of Sydney
• DELAMAN IV, London 2 November 2006

2
PARADISEC

• Collaborative digital research resource set up by
  University of Sydney, University of Melbourne
  Australian National University, 2003. (UNE
  joined 2004)

75 funding from Australian Research Council
LIEF Scheme (3 successful applications)
3
Why video?

• From 1980s, portable battery-powered video
  cameras increasingly used by fieldworkers
• Primary data for ethnographic studies of gesture,
  dance, performance, signed language, social
  interaction
• Highly valued by cultural heritage communities

4
Threats to video

• Obsolescence of consumer video formats (VHS,
  super-8, Hi-8 etc)
• Media deterioration much more advanced that audio
  media (estimate 15 years)
• Kept in private research collections rather than
  deposited in archives (hence less than ideal
  storage conditions, lack of metadata)

5
Deterrents to digitisation

• Lack of internationally agreed archival formats
  and standards for digital video, hence researcher
  fear of making wrong choice
• Difficulties of storage and management once
  digitised - huge filesizes difficult to manage on
  desktop computer
• Lack of user-friendly and robust tools to allow
  researchers to analyse video data and collaborate
• Ethical and rights obligations - fears of
  compromising security and anonymity of
  consultants by outsourcing

6
Preservation and Digital Sustainability

• the only way to guarantee sustainability of
  digital audiovisual media is secure mass storage
  with well-structured metadata and an ongoing
  digital migration strategy, using authenticated
  streaming for access.
• Kevin Bradley, APSR Sustainability Issues
  Discussion Paper, National Library of Australia,
  2005.
• International Association of Sound and
  Audiovisual Archives (IASA) Guidelines on the
  Production and Preservation of Digital Audio
  Objects (IASA-TC04). Aarhus, Denmark
  International Association of Sound and
  Audiovisual Archives (IASA), 2004.

7
Purposes of trial

• Feasibility study for video ingestion
• select appropriate standards
• test impact on existing systems
• design archival workflows
• model costs for future funding applications
• Preserve significant collections
• Provide access copies for researchers and
  communities
• Provide test data for EthnoER online annotation
  projects

8
Equipment and data

• Outsourced ingestion to professional video-maker
  (Paul Cockram) using Final Cut Pro 4 on Mac
  computer, ingesting video data from professional
  decks over firewire 400 cable.
• Video formats tested VHS, Hi-8, miniDV
• Video files saved on 500GB LaCie firewire drive
  for transport

9
Formats and standards adopted for trial

• Master copy .dv standard (maximum quality coming
  across firewire into Final Cut Pro) (saved inside
  Quicktime .mov wrapper) - approx 15GB per hour
• 720x576 pixels, 25 fps, PAL
• Titles basic metadata plus test signal at
  beginning of file, end file title at end of
  file.
• No chapter markers unless depositor provides list
  of timecodes for this purpose

10
Access copies

• Also sent to archive because of time taken to
  process (will review later)
• DVD copy (MPEG 2) - compress item to approx 4GB.
  Save as muxed .mpg file or disk image .img
• Chapter markers inserted every 5 minutes
• Web copy (.mov or .mp4)
• frame size 320 x 240 (480 x 360)
• 30 frames per second
• keyframe every 300 frames
• data rate 40 - 50 KB/sec (320 - 400 kbps) or 100
  - 120 KB/sec (800 - 960 kbps) for larger frame
  sizes.

11
Issues (1)

• Damage to videos (dropped frames) may cause stall
  in ingestion, need to set FCP to tolerate
• Some VHS tapes (1980s) had too many tracking
  errors to digitise
• Unstable picture (both hi8 and VHS) leads to
  higher filesize in mpeg-2 (image stabiliser
  needed?)
• Communication issues with contractor led to wrong
  filenames being used in titles, hence requiring
  reprocessing
• Compression to DVD and web copies very slow (8
  hours to process a 3-hour file) , ties up machine
  - better done in-house by batch processing

12
Issues (2)

• Data deluge at time of HD shuttle
• 500GB needing to be copied onto system - took a
  day
• swamped storage (ill-prepared)
• server partitions 200GB - not large enough, hence
  loss of time in copying across partitions
• Backup scripts at Sydney and APAC became out of
  sync, leading to data not being written to tape
  in Sydney, or being removed from server before
  APAC could mirror
• timeconsuming diagnosis and repair, but systems
  now redesigned

13
Transcoding for web delivery

• EthnoER project working with CSIROs Annodex
  platform to deliver streamed video
• Uses Ogg Theora - best transcoded from
  full-resolution version (currently done by CSIRO)
• Online delivery of annotations via EOPAS (EthnoER
  online annotation standard) - developed with
  University of Queensland Vannotea team building
  on Michel Jacobsons ITE work (LACITO)
• More details at our December conference!

14
Costing and feasibility

• Final report on data trial in preparation - we
  will post on our website and blog
• Costing only of production (staff time,
  consumables) - model at this stage does not take
  into account long-term storage and networking
  costs, equipment deterioriation
• Funding application for equipment in 2007
  unsuccessful - we will regroup

15
Video data in PARADISEC collection (30/10/06)

• 0.5 of collection files account for 19 of the
  data volume

16
Further information
Thanks to Participating Universities Australian
Research Council Australian Partnership for
Advanced Computing Australian Partnership for
Sustainable Repositories GrangeNet
http//paradisec.org.au http//blogs.usyd.edu.au/e
lac http//ethnoer.unimelb.edu.au http//conferenc
es.arts.usyd.edu.au/index.php?cf11