Real-time and Retrospective Analysis of Video Streams and Still Image Collections using MPEG-7 - PowerPoint PPT Presentation

1 / 24
About This Presentation
Title:

Real-time and Retrospective Analysis of Video Streams and Still Image Collections using MPEG-7

Description:

– PowerPoint PPT presentation

Number of Views:48
Avg rating:3.0/5.0
Slides: 25
Provided by: ResearchM53
Category:

less

Transcript and Presenter's Notes

Title: Real-time and Retrospective Analysis of Video Streams and Still Image Collections using MPEG-7


1
Real-time and Retrospective Analysis of Video
Streams and Still Image Collections using MPEG-7
  • Ganesh Gopalan,
  • College of Oceanic and Atmospheric Sciences,
    Oregon State University

2
Introduction
  • HD video streams have potential to improve
    understanding of deep sea eco-systems
  • However, volume and complexity associated with
    the HD streams and formats can be overwhelming
  • Our approach Use industry standards to transform
    video into a data type vs. treating it as viewing
    material

3
MPEG-7 Overview
  • Multimedia content description interface
  • Consists of low-level descriptors and high-level
    description schemes
  • Low-level descriptors provide statistical
    information about the pixel values in content
  • Description Schemes are used to represent
    semantic information

4
Low Level Descriptors
  • Structures that describe content in terms of the
    distribution of edges, colors, textures, shapes
    and motion
  • Descriptors extracted using MPEG-7 Experimental
    Model (XM) software
  • The input is a still image or a frame from video
  • The output is an XML description of the
    statistical information

5
Examples of Low Level Descriptors
  • Edge Histogram
  • Homogeneous Texture
  • Color Layout
  • Color Structure
  • Motion Activity
  • Descriptors are rotation and scaling invariant

6
Descriptor Extraction and Search
  • Phase 1 descriptor XML for collection of
    frames/still images is generated and cached
  • Phase 2 difference between query image
    descriptor from those values cached in phase one
    is computed
  • The cache can be augmented with the descriptors
    from a new video or still image collection

7
Description Schemes
  • Description Schemes attempt to model the reality
    behind the content
  • Low level descriptors can be used to tag objects
    of interest the tags are then used to construct
    a high level description
  • A search can then be performed against the higher
    level description schemes

8
High Definition Video Search Engine
  • Applied MPEG-7 to the development of an HD search
    engine
  • Extracted descriptors for approximately 10,000
    frames from 2.5 hours of high definition content
  • Content provided by the University of Washington
    from Visions 05 Cruise
  • Also applied to search for eddies in satellite
    image collections super-cells in radar images

9
Application Architecture
  • .NET Windows Forms front end with an embedded
    Windows Media Player
  • SQL Server back-end
  • Common Language Run-time Integration for
    development of stored procedures to manage MPEG-7
    XML
  • Procedures can be written in .NET languages
    rather than SQL

10
(No Transcript)
11
Creating a CLR Stored Procedure
  • CREATE FUNCTION FindUsingVisualDescriptor
  • (
  • _at_uid int,
  • _at_token uniqueidentifier,
  • _at_queryImage varbinary(MAX),
  • _at_descriptorName nvarchar(256)
  • )
  • RETURNS nvarchar(MAX)
  • AS EXTERNAL NAME MPEG7Document.StoredProcedures.Fi
    ndUsingVisualDescriptor
  • GO

12
Creating an HTTP Endpoint
  • CREATE ENDPOINT MPEG7
  • STATE Started
  • AS HTTP
  • (
  • SITE XXX.XXX.XX.XXX',
  • PATH '/MPEG7Endpoint',
  • AUTHENTICATION (BASIC),
  • PORTS (SSL),
  • SSL_PORT 444
  • )
  • FOR SOAP
  • (WEBMETHOD 'FindUsingVisualDescriptor'
  • (NAME 'looking.dbo.FindUsingVisualDe
    scriptor',
  • FORMAT ALL_RESULTS), )

13
User Interface
  • UI allows conversion of video into frames using
    ffmpeg
  • Descriptors of choice are then generated for all
    frames
  • Descriptors are persisted to the server

14
Retrospective Search
  • A query image initiates the search
  • The descriptor value for the given image is
    compared with those cached from the video frames
    or still images
  • The top 100 frames that are closest to the query
    image are returned

15
Retrospective Search Example
16
Real-time Event Detection
  • In this case, we have a set of known images that
    have objects of interest
  • Descriptors of frames from a real-time stream are
    compared on a continuous basis with those in the
    event library
  • When the difference in descriptor values is below
    a threshold, an event has been detected

17
Example of an Event
18
Reference Event
19
(No Transcript)
20
Use of Multi-Core Systems
  • The descriptor extraction process can be made
    faster by taking advantage of multiple processors
    or cores
  • The total number of frames can be divided up
    amongst the available processors
  • Threads extract the descriptors concurrently to
    generate chunks of XML
  • The threads then signal each other to combine the
    chunks into a single file with the descriptor XML

21
Challenges
  • Shadows and other lighting issues can create
    false positives
  • May be necessary to use multiple descriptors for
    classification
  • Processing high definition video at 30fps is
    computationally intensive
  • Scaling to a large number of images such as on
    the web presents a challenge

22
Conclusion
  • MPEG-7 supports a rich framework for
    content-based searches through its low level
    descriptors
  • Detected content can be tagged effectively using
    the high level description schemes that can be
    used to locate, search through and distribute
    content

23
Future Directions
  • Need to explore ways to speed up descriptor
    extraction using GPUs or hybrid GPGPUs.
  • Explore Cloud Services to implement video
    services transcoding video on the fly for
    different devices, descriptor extraction using
    HPC clusters, streaming services
  • Explore the Surface Computer as a UI

24
Acknowledgements
  • We are thankful to Professor John Delaney from
    the University of Washington for providing the HD
    footage
  • We are also thankful to the NSF funded LOOKING
    team for supporting this effort
Write a Comment
User Comments (0)
About PowerShow.com