Challenges in Multimedia Information Retrieval & Filtering - PowerPoint PPT Presentation

Loading...

PPT – Challenges in Multimedia Information Retrieval & Filtering PowerPoint presentation | free to download - id: 3d5302-NWUzY



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Challenges in Multimedia Information Retrieval & Filtering

Description:

Challenges in Multimedia Information Retrieval & Filtering xyxue_at_fudan.edu.cn – PowerPoint PPT presentation

Number of Views:22
Avg rating:3.0/5.0
Slides: 29
Provided by: dragonsta
Learn more at: http://dragonstar.ict.ac.cn
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Challenges in Multimedia Information Retrieval & Filtering


1
Challenges in Multimedia Information Retrieval
Filtering
  • ???
  • xyxue_at_fudan.edu.cn
  • ?????????????
  • ??????????????

2
Outline
  • Potential Applications, Query Examples
    Achievements
  • Basic Concepts Architectures
  • Key Techniques Problems

3
Many Potential Applications
  • Broadcast media selection (e.g. radio channel, TV
    channel)
  • Cultural services (e.g. history museums, art
    galleries)
  • Digital libraries (e.g. image catalogue, musical
    dictionary, bio-medical imaging catalogues, film,
    video and radio archives)
  • Journalism (e.g. searching speeches of a certain
    politician using his name, his voice or his face)
  • Multimedia directory services (e.g. yellow pages,
    Tourist information)

4
Video Query Examples(TREC)
  • a specific person
  • I want all the information you have on Ronald
    Reagan
  • a specific thing
  • I'm interested in any material on Hoover Dam. I'm
    looking for a picture of the OGO satellite

5
Informedia CMU
  • Establishment of large video libraries as a
    searchable information resource
  • Full content information retrieval in both spoken
    language and video/image domains
  • Integration of speech, image and natural language
    understanding for library creation and
    exploration
  • Fully automated transcriptions generated entirely
    speech recognition or with closed captions
  • Information summaries at varying detail, both
    visually and textually

6
CueVideo IBM
  • Developing fully automatic means for indexing,
    hyper-linking and preparation of media material
    for effective searching and browsing by users
  • Combines several automated indexing,searching and
    browsing tools
  • Video analysis and summarization
  • Use of speech recognition for spoken media
    retrieval

7
Outline
  • Potential Applications, Query Examples
    Achievements
  • Basic Concepts Architectures
  • Key Techniques Problems

8
An Instance of IR System
9
Information Retrieval
  • Information Retrieval (IR) Deals with
  • Representation (or Modeling)
  • Storage
  • Organization
  • Access
  • of / to Information Items

10
ArchitectureIR
offline

Multi
-
Modal User Interface

Representation
Modeling

,
Relevance feedback
Description (MPEG
-7/XML)


Multimedia

ng

Query Processi
Database

Organizing

Index Structure

Searching

Ranking

11
Information Filtering
  • Generally, the goal of an Information Filtering
    (IF) system is to sort through large volumes of
    dynamically generated information and present to
    the user those which are likely to satisfy his or
    her information requirement

12
ArchitectureIF
13
Applications using MPEG7 
14
ComparisonIR IF
  • Information Retrieval
  • User Information Needs or Query Varying
  • Database or Collection Static
  • Information Filtering
  • User Information Needs or Profile Static
  • Incoming Data Varying
  • Common to both
  • how to represent information
  • how to select relevant information

15
Outline
  • Practical Applications, Query Examples
    Achievements
  • Basic Concepts Architectures
  • Key Techniques Problems

16
Digital TV Program Filtering Searching System
17
Representation extract low level features
  • Text Features
  • stop word elimination, stemming, index term
    selection, thesauri, word cut
  • Image and Video Features
  • color, texture, shape, motion,
  • Audio (Speech,Music) Features
  • zero-crossing ratio, short time energy
  • Spectral, Spectral Flux, Spectral Centroid, LPC,
    MFCC
  • Pitch,Rhythm,Timbre,
  • Requirements - Good Representation, Fast,
    Automatic, Robust

18
Representation get high level features
  • Structured Video Analysis
  • Video Scene Shot Key frame
  • Summaries at varying detail, both visually and
    textually
  • Audio Visual Object Recognition
  • Face,Character,Car,
  • Word Spotting,Speech Recognition,Speaker,
  • Problem - Low Precision, Infant, Inevitable
    Incompleteness in the Representation,

19
(No Transcript)
20
Retrieval Model
  • Boolean Model
  • Vector Model
  • Probabilistic Model
  • Fuzzy Set Model
  • Neural Network

21
Storage Organization
  • Storage
  • Standardized Descriptors - MPEG-7
  • Management of XML Documents
  • Index Structures For Fast Query
  • Inverted File for Text
  • Index Structure for XML Documents
  • Index Structures for High Dimensional Vector
    (Visual Features) - Dimensionality Curse

22
Curse of Dimensionality
An Intuitive Explanation Assume n-dimensional
points distributed in super-cubic. Selectivity
can be computed When n increasing, P(n)
will go down to zero exponentially. In order to
find relevant points, searching window should be
enlarged!
23
Multi-Modal Interface - 1
  • Input Information Needs
  • Key Word,
  • Example Image, Example Face, Example Video Clip,
  • Speech, Humming,
  • Relevance feedback
  • How to submit users query easily and friendly to
    IR system?
  • How can IR system understand users query
    intention?
  • People are unable to specify that which they
    don't know
  • There is inevitable uncertainty in the
    representation or understanding of information
    problems

24
Multi-Modal Interface - 2
  • Output Query Results
  • Enable user to browse full content in hierarchy
    or web
  • Visualization is important for presentation

25
How to Compute Relevance?
  • Relevance is a dynamic and idiosyncratic
    relationship between person and information
    object
  • Information objects mean many different things to
    different people (or the same person at different
    times)
  • There is inherent uncertainty in the relevance
    relationship

26
ComparisonIR DR
27
Conclusion
  • Many types of data without strict structure in
    huge multimedia database
  • Almost all algorithms of intelligent information
    processing and recognition (audio visual) are
    necessary for better representation
  • Seeking good retrieval model may be key to reduce
    gap between person and computer
  • Uncertain chaotic task unable to be formulated

28
Q/A? Thank You!
About PowerShow.com