Segmentation and Event Detection in Soccer Audio

1
Segmentation and Event Detection in Soccer Audio

• Lexing Xie, Prof. Dan EllisEE6820, Spring 2001
  April 24th, 2001

2
The problem

• Event detection in sports video
• In this project the audio part
• Our approach
• Segmentation Event Detection
• Incorporate domain knowledge

3
Outline

• Related work
• Observations on soccer audio
• Segmentation
• Features
• Decision scheme
• Result

• Event detection
• Scope
• Feature metric
• Result
• Generalization
• Next step

4
Related Work

• Audio segmentation
• Speech-silence discrimination Rabiner78
• Speech / music / mixture segmentation
  Saunders96 Scheirer97 Williams99
• Sports audio analysis
• Classify excited speech Rui2000
• Keyword/event template matching Chang96
  Rui2000

5
Observations 1

• Sound Types
• Foreground speech
• Noisy vocal sound with visible phoneme structure
• Background noise
• Ambient crowd, whistles, cheers, etc.
• Acoustics Fahy2001
• Sound intensity in open space
• Sound attenuation in air
• Production conditions
• Frequency response of microphone
• Automatic Gain Control

6
Observations 2

• Large variety across games
• Commentator verbosity
• Audience excitability
• ? not labeling and training
• In different languages ? not ASR
• Not template-matching training
• Assumptions on temporal characteristics
• Short-term dynamics ?
• Long-term variety ?

-- Seg. Det.
unit 0.03sec 0.51
context 15 gt100
7
Segmentation Algorithm

• Commentary vs. Crowd segmentation

8
Segmentation Result
commentary
commentary
commentary
crowd
crowd
Sound length Ground truth Hits Misses False Alarms
100 sec 50 46 4 2
9
Detection 1

• Detecting audio events in crowd noise
• Examples crowd cheering, whistle,
• Subjective definition

10
Detection 2

• Compute Mahalanobis distances Duda 73
• Feature element normalization and decorrelation
• Pick up distinctive segments
• Largest distance to all other segments (typically
  top 510)
• Clustering detecting outliers
• Merge adjacent segments

11
Detection Results

• The game River Plate vs. Los Andes

• Assumptions
• The majority are Unimportant
• We do have Important parts!
• Cluster analysis helps

12
Generalization

• Segmentation tasks
• Other Sports (baseball, tennis, etc.)
• Film sound track (Sense and Sensibility)
• Detection of sparse audio events
• Surveillance video

Music
Speech
Speech
Silence
Silence
13
Next step

• More experiments
• Improve decision scheme
• Improve GMM in segmentation
• Use cluster analysis in detection
• New features
• Wish list
• Classification of speech segments
• Other interesting noise patterns
• Investigate sound mixtures

14
Summary

• Segmentation
• Use energy features
• Best result precision 95, recall 92
• Event detection
• Use feature distance
• Interesting segments retrieved
• More work to follow

15
Thanks!
16
(No Transcript)