Vid2RSS Proposed CLEF Task

1
Vid2RSSProposed CLEF Task
2
Vid2RSS in a nutshell

• Task Automatically generate topic-based
  RSS-feeds from video
• Data Dual language TV programs
• Research challenge Exploit speech recognition
  transcripts containing non-redundant spoken
  information in two languages

2
3
Examples of topic-based RSS-feeds for video

• RSS-feeds on 3 topics displayed in Netvibes

3
4
What is an RSS-feed?

• RSS-feeds also called channels they deliver
  content intended for a specific audience
• Widely used XML-based standard for distributing
  Internet content
• Used to deliver news and other dynamic
  information sources (blogs, podcasts, vodcasts,
  )
• Encodes simple metadata (e.g., title,
  description, link) that allow users to decide to
  view/read the source
• RSS-feed lists items syndicated by the same
  source or aggregated by topic/theme
• Originally text only, but increasingly containing
  images and video keyframes

4
5
Basic task

• Given
• Video of a collection of episodes from a dual
  language television program
• Speech recognition transcripts in both languages
• Metadata such as title, description, keyframe and
  link
• Produce
• Categorization assigning the episodes to
  subject/topic category
• Two feeds for each category, one in each language
• Evaluate by
• Comparing to ground-truth subject/topic
  categories
• Visualizing the feeds in an RSS-reader

5
6
Data

• Suggested data (for starters) Dutch language
  program Noorderlicht http//noorderlicht.vpro.nl/
• Dutch documentary show about scientific topics
  using spoken Dutch and a large portion of spoken
  English (interviews with international
  scientists)
• Ca. 100 episodes included in TRECVID 2007 data
  collection
• Provided by Beeld and Geluid (Sound and Vision)
• Transcribed by the University of Twente
• Spoken content was not fully exploited by TRECVID
• Other dual language programs can be added to
  Vid2RSS if they can be found

6
7
Output of basic task

• RSS-feeds with a ltchannelgt element that is parent
  to a set of ltitemgt elements
• Each ltitemgt element represents an episode in the
  basic task (example on next slide)
• The automatically generated RSS-feeds output by
  the basic task should be as close as possible to
  the hand-generated RSS-feed used on the
  Noorderlicht site, cf. http//noorderlicht.vpro.nl
  /themasites/rss/weblog.jsp?rssnr22236939
• By aiming to produce metadata of the same type
  that is currently already used to deliver the
  content will ensure that progress achieved in the
  Vid2RSS task will have good chance of direct
  take-up

7
8
Sample RSS-item

• ltitemgt
• lttitlegtStemmen zit tussen de orenlt/titlegt
• ltlinkgthttp//noorderlicht.vpro.nl/noorderlog/
  bericht/36627903/lt/linkgt
• ltdescriptiongtWat we stemmen houden we graag
  voor ons. Maar dat zou in de toekomst wel eens
  moeilijker kunnen worden. Want stemvoorkeuren
  blijken vast te liggen in ons brein, ontdekken
  Amerikaanse psychologen.lt/descriptiongt
• ltenclosuregthttp//images.vpro.nl/img.db?3662830
  6 s(400)lt/enclosuregt
• ltpubDategtMon, 10 Sep 2007 174200
  0200lt/pubDategt
• lt/itemgt

8
9
Vid2RSS basic task summary

• Vid2RSS is a new problem how to best exploit
  dual language multi-modal content
• Basic task of Vid2RSS lays a basis for tasks
  which present increasingly challenging areas for
  scientific research (see next 4 sides)
• If BG and UTwente support the task, data is
  already available
• Evaluation will be supported by the availability
  of human generated metadata for the video
  episodes
• By generating a widely used metadata standard,
  task output can be directly used to deliver
  content

9
10
Advanced Task ISummarization/Keywords

• Given
• Video of a collection of episodes from a dual
  language television program
• Speech recognition transcripts in both languages
• Produce
• Automatically generated sets of keywords or
  summaries that can be used to fill the lttitlegt
  and ltdescriptiongt elements
• Two feeds for each category, one in each language
• Evaluate by
• Exploiting existing metadata
• Visualizing the feeds in an RSS-reader

10
11
Advanced Task IIKeyframe selection

• Given
• Video of a collection of episodes from a dual
  language television program
• Speech recognition transcripts in both languages
• Human generated metadata
• Produce
• Automatic selection of keyframe to represent each
  episode and to fill the ltenclosuregt element
• Two feeds for each category, one in each language
• Evaluate by
• Human annotators
• Visualizing the feeds in an RSS-reader

11
12
Advanced Task IIIExploiting metadata

• Given
• Video of a collection of episodes from a dual
  language television program
• Speech recognition transcripts in both languages
• Human generated metadata
• Produce
• Produce sets of keywords or summaries that can be
  used to fill the lttitlegt and ltdescriptiongt
  elements
• Select keyframe
• Two feeds for each category, one in each language
• Evaluate by
• Exploiting existing metadata
• Visualizing the feeds in an RSS-reader

12
13
Advanced Task IVSemantic segmentation

• Given
• Collection of episodes from dual language
  television program
• Speech recognition transcripts in both languages
• Human generated metadata
• Produce
• Division of episode into segments each relating
  to a sub topic
• Summarization and keyword generation on a segment
  level
• Keyframe selection on a segment level
• Two feeds for each episode, one in each language
• Evaluate by
• Human annotators
• Help from existing human generated metdata
• Visualizing the feeds in an RSS-reader

13
14
Vid2RSS

• One basic task
• Mixing languages, content, media, metadata
• Many possible advanced tasks
• Mixing focused IR, summarization, and ...
• See http//ilps.science.uva.nl/Vid2RSS/

14