Contentbased Video Retrieval - PowerPoint PPT Presentation

1 / 48
About This Presentation
Title:

Contentbased Video Retrieval

Description:

Content-based Video Retrieval. Willem Jonker. Philips Research / Twente University ... Shots: (1) Playing; (2) Close up; (3) Audience; and other. Video ... – PowerPoint PPT presentation

Number of Views:640
Avg rating:3.0/5.0
Slides: 49
Provided by: jon146
Category:

less

Transcript and Presenter's Notes

Title: Contentbased Video Retrieval


1
Content-based Video Retrieval
  • Willem Jonker
  • Philips Research / Twente University
  • IFIP 2.6 meeting, Lausanne, May 15 17, 2002

2
Contents
  • Image Retrieval Current Practice
  • Content Based Video Retrieval
  • Retrieving events from Tennis Video
  • Retrieving events from Formula-1 Video

3
Video Content Management
4
Image Retrieval Current Practice
5
Image Retrieval Current Practice
6
Image Retrieval Current Practice
7
Precision Recall
8
Multi-level Modeling
9
Features
background
composition
color
texture
face recognition
motion
text rec. OCR
shape
closed caption
berg-etappe, etappe nr 17
speach rec.
10
Query _at_ feature level
11
QBIC
12
Amore
13
Video Retrieval
14
Video Retrieval Modeling
15
Video Retrieval Modeling
16
Video Retrieval Modeling
17
Video Retrieval Modeling
18
Video Retrieval Query
19
Video Retrieval Query
20
Video Retrieval Query
21
Video Retrieval Query
22
The Semantic Gap
User
Concepts
Semantic gap
Raw Video Data
Data
23
Tennis Goal
Video
Shot segmentation and classification
Shots (1) Playing (2) Close up
(3) Audience and other
24
Tennis HMM History
  • 50s - statisticians tried to characterize
    processes with incomplete observations.
  • 60s - Baum et al. published HMM theory in a
    series of papers.
  • 70s - HMMs were used at IBM and CMU for speech
    recognition.
  • 90s - widespread applications (Gesture
    recognition, human actions)

25
Tennis HMM example
26
Tennis HMM example
27
Tennis Pre-processing
initial segmentation (robust M-estimator)
Step 1 Step 2 Step 3
locate player
fit 3D model
final segmentation
28
Tennis Features
Shape Pie features Skeleton features
29
Tennis HMM
Feature extraction
Time
Time
30
Tennis HMM
. .
time
states
31
Tennis Experiments
  • Goals
  • Determine the right feature set
  • Investigate person independence
  • Two series of experiments
  • Same player in the training and evaluation
  • Training with one player, evaluation of strokes
    performed by various players
  • Six events Forehand, Backhand, Service, Smash,
    Forehand volley, and Backhand volley

32
Tennis Experiment 1 results
Recognition results ()
  • HMM parameters
  • Codebook size of 24 symbols
  • HMMs with 8 states
  • Comparison to related work
  • Improvement of 15
  • TV videos

33
Tennis System Screen Dump
34
Tennis Some Performance Figures
Dual Pentium II, 350 MHz Preprocessing 6.5 min.
per stroke of 45 frames initial segmentation
95sec fitting 3D models 150 sec final
segmentation 105 sec feature extraction 40
sec Training the model 20 sequences of 45
frames 50 iterations Baum-Welch 4 states 8
symbol codebook 12 sec 48 states 80 symbols 20
min. Inference between 1 and 30 sec depending
on model complexity in our case for 8 states
model with 20 symbols 2 sec
35
Formula-1 Goal
36
Formula-1 Bayesian Networks
A
B
C
D
E
P(X,Y) P(XY) P(Y) P(XY)P(Y)
P(YX)P(X) P(X1,X2,,Xn) P i1..n P(XiPa(Xi))
37
Formula-1 Audio BN
Kword Key Words PRate Pause Rate AV Average
Values DR Dynamic Range MV Maximum Values STE
Short Time Energy MFCC Mel-Frequency Cepstral
Coefficients
38
Formula-1 Audio DBNs
39
Formula-1 Effect of Audio DBNs

40
Formula-1 Example Videos
41
Formula-1 Highlights on Audio
42
Formula-1 Combined Audio-Video
43
Formula-1 Experimental Results
44
Formula-1 System Screen Dump
45
Formula-1 Some Performance Figures
46
Bridging the Gap
User
Concepts
Domain knowledge
Domain features
Features
Raw Video Data
Data
47
DMW projecthttp//wwwhome.cs.utwente.nl/dmw/
48
Some Papers
  • M. Petkovic, W. Jonker, Content-Based Video
    Retrieval by Integrating Spatio-Temporal and
    Stochastic Recognition of Events, In Proc. IEEE
    International Workshop on Detection and
    Recognition of Events in Video, Vancouver,
    Canada, July 2001.
  • M.Petkovic, W. Jonker, Z. Zivkovic, "Recognizing
    Strokes in Tennis Videos Using Hidden Markov
    Models", International Conference on
    Visualization, Imaging and Image Processing,
    Marbella, Spain, September 3-5, 2001.
  • H. E. Blok, M. Windhouwer, R. Zwol, M. Petkovic,
    P. M. G. Apers, W. Jonker, M. Kersten, "Flexible
    and Scalable Digital Library Search", 27th
    International Conference on Very Large Databases,
    Roma, Italy, September 11-14, 2001.
  • V. Mihajlovic, M. Petkovic, W. Jonker,
    Content-Based Retrieval of TV Formula 1
    Programs, 2nd International Workshop on
    Multimedia Data Document Engineering (MDDE'02),
    March 24 2002, Prague
Write a Comment
User Comments (0)
About PowerShow.com