MPEG7 Multimedia Content Description Interface Presented by: Moustafa A. Hammad

1
MPEG-7 Multimedia Content Description
InterfacePresented by Moustafa A. Hammad
2
Introduction

• More and more digital audio - visual information
  exists and increasing.
• Increasing Internet popularity .
• More audio-visual information processing systems
  emerged.

How fast and easy can desirable information be
made available?
MPEG-1 Standard for storage and
retrieval MPEG-2 The digital television
standard MPEG-4 Multimedia production,
distribution and content access -
developed a Syntactic Description Language.
Fine - where is the semantic?
3
Introduction (Cont.)

• MPEG-7 Multimedia content description
  interface.
• Represents information about the contents, but
  not the content itself.
• Satisfies both database and signal processing
  communities.
• Goal audio-visual material as searchable as
  text.
• What is the standard? (To be finalized mid 2001)

4
Topics of Discussion

• Scope of the standard
• Terminology
• Interaction between MPEG-7 and Applications.
• Requirements
• Applications
• Case study A Proposal for MPEG-7 Description
  Definition.

5
Scope of the Standard

• MPEG-7 processing chain
• feature extraction (analysis)
• the description itself
• the search engines (application)
• What is in the standard?

6
MPEG-7 Terminology

• Data,
• Feature,
• Descriptor (D),
• Descriptor value,
• Description Scheme (DS),
• Description,
• Coded Description and
• Description Definition Language (DDL)

7
Interaction between MPEG-7 and Applications
8
MPEG-7 Requirements

• Descriptors(Cross modularity, Direct data
  manipulation, Data adaptation, Language of text
  based descriptions, Linking, Prioritization of
  related information, Unique identification)
• Description Schemes( Description Scheme
  relationships, Prioritization of descriptors,
  Hierarchy of descriptors, Scalability of
  descriptors, Description of temporal range, Data
  adaptation)
• DDL (Compositional capabilities, Unique
  identification, Primitive/composite data types,
  Multiple media types, Relationships between
  description and data, Grammar, Intellectual
  Property Management and Protection (IPMP), Real
  time support ..Etc.)
• Descriptors requirement
• General (Types of features N-dimensional
  spatio-temporal structure, Objectives,
  subjective, Production, composition, Concepts,
  Referencing analogue data,..Etc.)

9
MPEG-7 Requirements (Cont.)

• Functional ( Retrieval effectiveness,
  similarity-base retrieval .Etc.)
• Coding (Description efficient representation,
  Description extractionEtc.)
• Visual specific (Types of features (color,
  texture, sketch), Visual data formatsEtc.)
• Audio specific (Types of features (Frequency
  contour, Harmony), Auditory data formatsEtc.)
• Text specific (Text retrieval, consistency of
  text description tools)
• (Types of features (Frequency contour, Harmony),
  Auditory data formatsEtc.)
• System requirement (multiplexing, Temporal
  synchronization, File format, IPMPEtc.)
• Ref
• MPEG Requirement Group, MPEG-7 Requirement,
  Doc, ISO/MPEG N2859, MPEG Vancouver Meeting, July
  1999.

10
MPEG-7 Applications

• Pull applications
• video retrieval (storage and retrieval of video
  database. Sound effects library, historical
  speech databaseEtc.)
• Push applications
• video selection and filtering( Personalized
  television services, information access
  facilities for people with special needs..Etc.)
• Specialized professional and control
  applications.
• (Remote sensing applications, Surveillance
  applications..Etc.)

11
A proposal for an MPEG-7 Description Definition
language (DDL)

• Reference J. Hunter (DSTC)
• A schema is based on different schemas Resource
  Description Framework (RDF) Schema, XML Document
  Type Descriptors (DTD), Document Content
  Description (DCD), A Schema for Object-Oriented
  XML (SOX).
• Satisfies The DDL requirements.
• Consists of classes, properties and relations
  between classes.
• Uses of Dublin Core (DC) attributes. (Name,
  Identifier, Version, Registration Authority,
  Language, Definition, Obligation, Datatype,
  Maximum Occurrence, Comment)

12
The Description Scheme
MM Document
13
Features of the proposed MPEG-7 DDL

• Namespace Declarations
• ltx
• xmlnsdchttp//purl.org/metadata/dublin_core
  gt
• lt!-- the dc" prefix is bound to
  http//purl.org/metadata/dublin_core for the "x"
  element and contents --gt
• ltdcTitlegt CNN News lt/ dcTitlegt
• lt/xgt
• The Class Type declarations and Class Hierarchies
• ltclass idMM_Documentgt
• ltproperty type dc_attribs/gt
• lt/classgt
• ltclass idVideo_Documentgt
• ltsubclassof typeMM_Document /gt
• ltproperty type duration/gt
• lt/classgt

14
Features .. (Cont.)

• Property type declaration
• ltpropertyType idframeNum datatypeint/gt,
  ltpropertyType idsecs datatypefloat/gt
• ltpropertyType idtimestampgt
• ltAltgt
• ltproperty type frameNumgt
• ltproperty type secsgt
• lt/Altgt
• lt/propertTypegt
• The relationship type declaration
• ltrealtionType idcontains directionuni
  inversecontained_bygt
• ltdomain typeMM_Document/gt
• ltrange typeMM_Document
  occurszerormore orderSeq/gt
• ltconstraint typeboolean value
  ((range1.startgtdomain.start)rangen.end lt
  domain.end ) )/gt
• lt/relationTypegt
• ltclass idscenegt
• ltsubclassof typeMM_Document /gt
• ltproperty type dc_attribs/gt
• ltrelation typecontains rangeobject/gt

15
Features .. (Cont.)

• Order and Occurs (Seq, Bag, Alt, Par)
• Data typing user defined datatypes
• Attribute Definitions ltattributeType idsrc
  datatypeuri/gt
• Synchronization and temporal specification
• ltseqgt
• ltaudio_track srcaudio1/gt
• ltaudio_track begin5s srcaudio2/gt
• lt/seqgt
• Spatial specification
• Both rectangle and polygon representation, HTML
  syntax and semantics.

Audio1
5s
Audio2
16
Example MPEG-7 description
. ltMM_Document src http//./test.mp
ggt lt!-- other propertiesgt ltcontainsgt
ltPargt ltSeq idvidoe_sequencesgt
ltsequence idseq1
srchttp//. /gt lt!-- other
sequencesgt lt/Seqgt ltSeq
idaudio_tracksgt ltAudio
idspeech srctest.ra/gt lt!-- other
audiosgt lt/Seqgt lt/Pargt
lt/containsgt ltsequence idseq1
srchttp//../gt ltcontainsgt
ltSeqgt ltScene idscene1
srchttp//. /gt lt!-- other scenesgt
lt/Seqgt lt/containsgt
.. lt!-- declaration od shots, frames and
objects/gt .... ltObject
idcar srchttp//../test.jpgcargt
.. ltDC.Description.textgtA red car
which has been severely damaged by the
exposition. lt/DC.Description.textgt
... lt/objectgt
17
Conclusion

• The proposed DDL provides most of the DDL
  requirement.
• There are some remarks
• Lack of provision to push applications
  (filtering and selection, real time support)
• No representation for subjective and concept
  features.
• Simple representation and support for spatial
  features.