Embedded Streaming Media with GStreamer and BeagleBoard

1
Embedded Streaming Media with GStreamerand
BeagleBoard

• ESC-228
• Presented by Santiago Nunez
• santiago.nunez (at) ridgerun.com

2
Agenda

• Introduction to BeagleBoard multimedia
  architecture features.
• GStreamer concepts
• OMAP GStreamer hands on exercises
• DMAI and GStreamer
• Questions

3
Introduction

• Primary objective for this class is to introduce
  audience to GStreamer on OMAP3 with hands on
  exercises
• This doesnt class won't cover extensively the
  APIs available and assumes basic knowledge on C
  programming, Linux and computer graphics.

4
BeagleBoard and OMAP3architecture

• Provides architecture with several multimedia
  features
• Cortex A8 with Neon
• C64x DSP
• Video accelerators
• How to utilize the hardware features with the
  software stack?

5
BeagleBoard and OMAP3architecture

• Provides architecture with several multimedia
  features
• Cortex A8 with Neon
• C64x DSP
• Video accelerators
• How to utilize the hardware features with the
  software stack?

6
BeagleBoard and OMAP3architecture

• Provides architecture with several multimedia
  features
• Cortex A8 with Neon
• C64x DSP
• Video accelerators
• How to utilize the hardware features with the
  software stack?

7
GStreamer

• Streaming media framework audio and video
• Close to 200 plug-ins available
• Higher level than just input / filters / output
• Networking, audio/video mixed streams, auto data
  handling
• Various options utilizing hardware accelerators

8
GStreamer Overview

• Elements
• Sources, filters, sinks
• Bins and Pipelines
• Containers, pipeline is the overall bin
• Pads
• Element source / sink connection points
• Caps
• Capabilities organized by stream type with a set
  of properties

9
GStreamer Overview

• Elements
• Sources, filters, sinks
• Bins and Pipelines
• Containers, pipeline is the overall bin
• Pads
• Element source / sink connection points
• Caps
• Capabilities organized by stream type with a set
  of properties

• Plugin
• Collection of elements

10
Simple MP3 Player

• Create dynamically using gst-launch
• Source element reads from a file
• Filter element converts MP3 to PWM
• Sink element passes to ALSA output

gst-launch filesrc locationa.mp3 ! mad ! alsasink
11
Hands On Exercise 0

• Start up Windowing environment with two terminals
• cd gst
• . s

Period character to source the file named 's'
12
Hands On Exercise 0

• Play audio file
• a2
• Actual command
• gst-launch filesrc locationbbb.flac ! flacdec
  !alsasink
• You can see the contents of the script
• cat a2
• There are lots of audio scripts to try
• a1, a2, a3

13
Simple Audio PlayerSource Code

• Create pipeline, source, filter, sink
• Set element properties
• Build into pipeline
• Connect src and sink pads
• Setup pipeline event handler
• End of stream
• Set pipeline state to play
• Run

14
Simple PA System

• Create dynamically using gst-launch
• Source element ALSA audio in
• No filters
• Sink element passes to ALSA output

• Try it
• r

gst-launch alsasrc num-buffers100 ! alsasink
15
Keeping Plug-ins Organized

• Each known plug-in is added to registry
• Most aspects of plug-in are tracked in the
  registry
• Registry support run-in pipeline creation and
  dynamic filter selection
• Use gst-inspect to list plug-ins

gst-inspect lessgst-inspect filesrc
16
Hands On Exercise 1

• Using gst-inspect, list
• All plug-ins
• All video plug-ins
• Element properties for filesrc plug-in

17
Hands On Exercise 2

• GStreamer video pipelines
• v1 script shown below
• gst-launch videotestsrc ! ffmpegcolorspace !
  fbdevsink
• Other video scripts
• v1, v2, v3, v4
• Idea is the same
• source data, filter data, send data to sink

18
PerformanceData Passing

• Stream held in buffers with data, timestamp,
  other info
• When possible, buffer memory allocated by sink
  pad
• Use hardware when data copy is necessary

19
PerformanceData Transformation

• Cortex A8 compiler optimization
• NEON
• Single Instruction Multiple Data
• C64
• Video accelerator
• DMA and other data movers

20
Davinci Multimedia Application Interface
21
DMAI and GStreamer

• Davinci Multimedia Application Interface
• Exposes OMAP/Davinci hardware using high level of
  abstraction
• Stream audio / video
• Graphics display
• Hardware optimized frame/data copy

22
OMAP3 Codec Engine

• Isolates users for audio/video codecs from those
  implementing the codecs
• Codec can run in several places without the
  calling application being aware
• Cortex A8, NEON, C64, hardware accelerator
• Uses DSPLink and DSPBios conventions to support
  DSP based algorithms dynamically

23
Video Phone

• Vision video phone application can create audio
  and video GStreamer pipelines for capture /
  rendering
• DMAI, Codec engine, DSPLink, C64 algorithms
  perform heavy lifting
• OMAP3 hardware capabilities utilized
• Video phone application focused on call
  management, not streaming data manipulation

24
GStreamer in3 Layers