Synchronised content: How to add value to audio visual material using structured transcripts

1
Synchronised content How to add value to audio
visual material using structured transcripts

• Ted van der Togt
• Utrecht, Jan 19th 2009

2
DAISY What accessible audio means

• Combination of (structured) text audio
• Navigation structure Table of Contents, High
  level structure (Chapters etc.)Fine grained
  structure (browse by sentence)Bookmarks
• Searchability

3
Role of Dedicon

• Production (conversion) of DAISY books (1.6M CDs
  /yr)
• Development of standards (DAISY consortium)
• Development of tools (playback and recording)
• Assistance of other libraries worldwide
• New educational products (with publishers)

4
Accessible Video

• Captions helpful for
• Deaf and hearing impaired
• Use in noisy (or silence) environments
• Understanding by non native speakers
• Search

5
Captioning of courseware

• I was absolutely thunderstruck by the value that
  our closed captioning service offers, and how it
  was the right thing to do for our business, no
  matter what it took.
• Since then, weve been in a major push to add
  closed captioning to the majority of our courses.
  Its going to take us several months, but were
  well on our way and Im happy to report weve
  already completed captioning on 13 courses in the
  past 4 weeks!
• Closed captioning, in addition to making our
  training accessible to those with hearing
  impairments, also enables us to provide everyone
  with enhanced search functionality. Our Find
  text in captions feature is a great way to
  locate information about a specific tool or
  topicwithin a particular course or across the
  entire library.
• - Lynda Weinman - www.lynda.com

6

• Example of keynote speech Picnic 2007,
  searchable by captions (demo...)

7
Synchronized transcripts

• Full text view (instead of caption by caption)
• Chapterization (manual)
• Speaker identification
• Topic identification
• Full text search

8
New York Times

• Transcript
• Sections
• Analyzer (word count, speaking time)
• demo...

9
MIT lecture browser

• Chapters
• Transcript scrolling with audio
• Search Transcript
• demo...

10
Things to solve

• Standards (W3C timed text, SMIL, DAISY for
  video?)
• Easy and affordable production of transcripts
• Flexible representation from single source
  (device specific mobile, PC, TV, or based on
  personal profile no video, captions on braille
  bar, additional audio description, etc.)

11
Transcript generation (raw)

• Human
• Professional subtitling (EUR 500/hr)
• Cheap service (Castingwords.com from EUR 36/hr)
• Machine
• Quality depending on jargon and vocabulary
• SAIL Labs media mining indexer

12
Transcript alignment
13
Transcript review and correction
14
CaptionWeb

• Do it yourself transcription of video (web
  portal)
• 1. Audio analysis (detection of language, jargon
  etc.)
• 2a. First pass with ASR
• 2b. Manual process (pre-segmented, speech
  follower)
• 3. Review Correction
• 4. Export to different formats

15
Potential solutions

• Further standardization on formats (W3C, DAISY
  consortium, EBU, etc. )
• Automatic alignment of existing transcripts with
  video (Telecats, Automatic Sync Technologies)
• Developments in speech recognition to produce raw
  first transcript (for human review correction)
• Transcription / subtitling collaboration portals
  like Captionweb.

16
Thank you

• Sources
• http//www.daisy.org
• http//herodotus.dedicon.nl/demo/dolr.html?proj/o
  ld/Repelsteeltje
• http//www.nytimes.com/interactive/2007/11/28/us/p
  olitics/20071128_DEBATE_GRAPHIC.htmlvideo
• http//web.sls.csail.mit.edu/lectures/
• More information tvdtogt_at_dedicon.nl or
  ted_at_vandertogt.nl