Video on the Semantic Web Experiences with Media Streams - PowerPoint PPT Presentation

Loading...

PPT – Video on the Semantic Web Experiences with Media Streams PowerPoint presentation | free to download - id: 522dfb-NGMyM



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Video on the Semantic Web Experiences with Media Streams

Description:

Video on the Semantic Web ... Three generations Web content explain title Requirements for 2nd & 3rd generation multimedia why multimedia is different from ... – PowerPoint PPT presentation

Number of Views:29
Avg rating:3.0/5.0
Slides: 30
Provided by: Jac8
Learn more at: http://joostgeurts.com
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Video on the Semantic Web Experiences with Media Streams


1
Video on the Semantic WebExperiences with Media
Streams
  • CWI Amsterdam
  • Joost Geurts
  • Jacco van Ossenbruggen
  • Lynda Hardman
  • UC Berkeley SIMS
  • Marc Davis

2
Talk Overview
  • Video Generation
  • Annotating Video/Media Streams
  • Porting Media Streams
  • Video generation using semantic web technology
  • Conclusion

3
Why video?
  • Video is powerful information source but
    complicated on syntax and semantics done by
    mostly domain experts.
  • Search/Retrieve is difficult for humans near to
    impossible for machines.
  • Media metadata is not DC labels, real need for
    formal semantics
  • Media support is lacking/under developed on the
    semantic web

4
Why video generation?
  • Practical experience with semantic web technology
    from a media perspective
  • Media typically is knowledge intensive
  • Ambitious application to trigger problems

5
What can we generate?
  • Cinema is like a language which has structure
    which can be manipulated
  • Establishing shot gives the impression an event
    is happening at a specific location
  • Reaction shot gives the impression an actor is
    reacting to an event.
  • Example

estab1.smil
estab2.smil
6
Continuity editing in cinema
  • Continuity editing rules ensures consistency
    between shots.
  • Mise en scene
  • Indoors - Outdoors
  • Clothing/costume
  • Weather conditions
  • Cinematography
  • Black/white Color
  • 180 degree rule (walking direction)
  • Framing
  • Jump cut

7
Continuity editing
  • Works best with generic objects/characters
  • Not so well with recognizable objects/characters
  • Theoretic principles in cinema can be formalized
    and used to automatically generate video
    sequences.

8
Trip report scenario
  • Student visits Berkeley informs colleagues about
    his trip.
  • Scenario uses establishing and reaction shots.
  • scenario_iswc2004.smil

9
Scenario wrap up
10
Generating an establishing shot
  • User
  • defines location
  • defines character
  • Continuity rules
  • angle shot 1 gt angle shot 2
  • Both shots outdoors
  • Annotation requirements on
  • Mise en scene (describing scene)
  • Cinematography (material, lens foci)

11
Annotating Video in Media Streams (Davis 1995)
  • Retrieve/combine existing material for reuse
  • Content driven annotations
  • Multiple descriptive dimensions
  • Icon based
  • 7000 icons in ontology 10000 annotations
  • subClass, partOf, looksLike, occursWith

12
Media Streams Timeline
13
Media Streams Icon Space
14
Porting Media Streams to the Semantic Web (syntax)
  • Media Streams application
  • 5 10 year old Lisp using old libraries
  • Programmers moved away
  • Source code only partly available
  • Ontology and annotations are in binary format
  • Not efficiently scalable
  • Reverse engineering
  • Important working legacy application
  • Lessons learned
  • Knowledge of the application is required to
    understand the ontology and the annotations.
  • Focus on syntax

15
Porting Media Streams to semweb (semantics)
  • Ontology
  • subClass relationship mapped to rdfssubClassOf
  • partOf, looksLike, occursWith relations mapped to
    generic rdfproperty (future work OWL)
  • Modularity for streams in ontology

XML ltcidi id"4497"gt ltnamegttelephone
booklt/namegt lthasSuperClassesgt ltcidi
id"4499" name"book" /gt lt/hasSuperClassesgt
ltoccursWithgt ltcidi id"4455" name"telephone"
/gt lt/occursWithgt lt/cidigt
RDF ltrdfsClass rdfID"CIDI_4497"gt ltrdfslabel
xmllang"en"gttelephone booklt/rdfslabelgt
lt!--subclassOf book--gt ltrdfssubClassOf
rdfresource"CIDI_4499"/gt lt!--cidi4455
telephone--gt ltoccursWith rdfresource"CIDI_445
5"/gt lt/rdfsClassgt
16
Human cognition vs. machine cognition.
  • Media Streams concepts were represented by icons
    with a textual label.
  • In RDF representation there is only the label
  • Icons can denote complex actions not
    representable by a single word
  • Need for WorldNet to retrieve synonyms (hack!)
  • Just RDF is not enough

17
Porting Media Streams to semweb (semantics)
18
Porting Media Streams to semweb (semantics)
  • Lessons learned
  • Annotations are not instances of the ontology but
    refer to it
  • Structure embodies semantics
  • Need for annotation template which describes
    structure

19
Video generation
  • Need for detailed annotations result in detailed
    queries
  • Detailed queries give fewer results
  • Need to be able to relax query
  • Specification (ask pants, retrieve jeans)
  • Generalization (ask jeans, retrieve pants)
  • System searches for shots which comply with
    continuity constraints
  • Query or Rule?

20
Conclusion for porting knowledge to semweb
  • Thoroughly understand the original application
    domain before porting knowledge to the semantic
    web
  • First focus on porting knowledge to an accessible
    format such as XML postponing modeling issues.
  • Annotations are not necessarily instances of an
    ontology but can refer to it, in which case a
    annotation template defines the structure of the
    annotation.

21
Conclusion for applications on the semantic web
  • Need for both, precise queries and, queries which
    allows for relaxing.
  • Distinction between queries and rules is small.
  • Combining proprietary heterogeneous knowledge
    sources on the semantic web leads to
    inconsistencies which have to be dealt with.

22
Take home message
  • Legacy sources are because of their longtime
    existence valuable resources worthwhile porting
    to the Semantic Web. Best practices guidelines
    are needed to facilitate this.
  • Ambitious applications test and give requirements
    on technology

23
Video on the Semantic WebExperiences with Media
Steams
24
Scratch
lt!-- Character (joost)--gt ltoccurence
id"character1"gt ltstartframegt1lt/startframegt
ltendframegt139lt/endframegt ltcompound
id"joost"gt ltnamegt ltcidi
id"3990" name"BODY"/gt lt/namegt
ltvaluegt ltcompound id"joost-body"gt
ltslotgt ltnamegtltcidi id"6532"
name"APPARENT-BODY"/gtlt/namegt
ltvaluegt ltcompound
id"joost-apparent-body"gt ltslotgt
ltnamegtltcidi id"6553"
name"SEX/AGE"/gtlt/namegt
ltvaluegtltcidi id"3868" name"adult
male"/gtlt/valuegt lt/slotgt
ltslotgt ltnamegtltcidi
id"6557" name"SKIN-COLOR"/gtlt/namegt
ltvaluegtltcidi id"6570" name"dark peach
skin"/gtlt/valuegt lt/compoundgt lt/compoundgt
lt/occurence
  • Video generation requires detailed, multi
    dimension descriptive annotations and is
    therefore a well suited test case for semantic
    web technology read some knowledge intensive
    applications, like video generation are dependent
    on shared knowledge sources provided by the
    semanticweb. The semantic web should support
    these. read Video generation is a real
    application giving insights in practical problems
    with the semantic web read technology should
    subordinate applications read ambitious
    applications show deficiencies in technology

25
Content structure vs. Documents structure (a)
  • Different ways of annotating
  • Structure based (scene, shot, frame)
  • Context important
  • Content based ( Character walks from left to
    right)
  • Context not important

26
Semantics after Syntax (b)
27
Ontology and Annotation (c)
28
Query requirements (d)
29
Rules (e)
About PowerShow.com