Windows Media Format SDK

1
Windows Media Format SDK

• Rejean Lau M.Eng

2
Introduction

• Windows Media Format SDK is a component of the
  Microsoft Windows Media Software Development Kit.
  Other components include the Windows Media
  Services SDK, Windows Media Encoder SDK, Windows
  Media Rights Manager SDK, Windows Media Device
  Manager SDK, and Windows Media Player SDK.
• The Windows Media Format SDK enables developers
  to create applications that play, write, edit,
  encrypt , and deliver Advanced Systems Format
  (ASF) files and network streams, including ASF
  files and streams that contain audio and video
  content encoded with the Windows Media Audio and
  Windows Media Video codecs.
• ASF files that contain Windows Media-based
  content have the .wma and .wmv extensions.

3
Supported Codecs in Windows Media Format 9 SDK
4
Key Features of Windows Media Format SDK

• Support for industry leading Microsoft codecs.
• Support for writing ASF files.
• Support for reading ASF files.
• Support for delivering ASF streams over a
  networks.
• Support for editing metadata in ASF files.
• Improved support for editing applications
  including fast access to decompressed content,
  frame-based indexing and seeking, and general
  improvements in the accuracy of seeking.
• Support for reading and editing metadata in MP3
  files.
• Support for Digital Rights Management protection.

5
Advanced Systems Format

• Advanced Systems Format (ASF) is the container
  format for Windows Media Audio and Windows Media
  Video.
• The extension wma or wmv is used to specify an
  ASF file that contains content encoded with the
  Windows Media Audio and/or Windows Media Video
  codecs.
• The Windows Media Format SDK can be used to
  create and read Windows Media files, as well as
  ASF files that contain other types of compressed
  or uncompressed data.

6
Advanced Systems Format

• Goals of the ASF format
• Support efficient playback from media servers,
  HTTP servers, and local storage devices.
• Support scalable media types such as audio and
  video
• Permit a single multimedia composition to be
  presented over a wide range of bandwidths.
• Allow authoring control over media stream
  relationships, especially in constrained-bandwidth
  scenarios.
• Independence of any particular multimedia
  composition system, computer operating system, or
  data communications protocol.

7
Advanced Systems Format

• An ASF file can contain multiple independent or
  dependent streams, including multiple audio
  streams for multichannel audio, or multiple bit
  rate video streams suitable for transmission over
  different bandwidths.
• Streams can be in any compressed or uncompressed
  format
• In addition to audio and video media types, an
  ASF file can contain text streams, web pages and
  script commands, and any arbitrary data type.
• ASF supports live and on-demand mutlimedia
  content for example, recording or playback H.32X
  or MBONE conferences.

8
Advanced Systems Format
ASF file contains three objects
9
Profiles

• A profile is a collection of data that describes
  the configuration of an ASF file.
• At a minimum, a profile must contain
  configuration settings for a single stream
• The stream information in a profile contains the
  bit rate, buffer window, and media properties for
  the stream.
• The stream information for audio and video
  describes exactly how the media is configured in
  the file, including which codec (if any) will be
  used to compress the data.
• A profile also contains information about the
  various ASF file features that will be used in
  files created with it including mutual exclusion,
  stream prioritization, bandwidth sharing and Data
  Unit Extensions.

10
Mutual Exclusion

• Mutual exclusion is a feature of the Windows
  Media Format SDK that enables you to specify a
  number of mutually exclusive streams that all
  equate to the same output.
• Every ASF file contains one or more streams, each
  containing digital media data.
• There are situations where you do not want every
  stream delivered to the client.
• Ex. You create a video file with five audio
  streams, one for each of five languages, you want
  only one of them delivered at a time.
• Mutual exclusion is defined in the profile used
  to create a file.
• Windows Media Format SDK recognizes four types of
  mutual exclusion (fourth is custom)

11
Stream Prioritization

• When you create an ASF file, you can specify a
  priority order for its constituent streams.
• If you stream a prioritized file and the
  available bandwidth is not enough to deliver all
  of the streams, the reader will drop streams in
  reverse priority order.
• Stream prioritization is configured with a stream
  prioritization object and added to the profile.

12
Bandwidth Sharing

• You can specify streams in a file that, when
  taken together, use less bandwidth than the sum
  of their states bit rates combined.
• By specifying bandwidth sharing in the profile,
  you clarify to reading applications the available
  bandwidth needed to stream the file
• Bandwidth sharing is configured with a bandwidth
  sharing object and is added to a profile before
  beginning to write a file.

13
Data Unit Extensions

• Used to supplement data in samples
• A data unit extension is a name/value pair that
  is attached to the sample in the data section of
  the file.
• Several standard types are used to provide
  additional data for file names (in script and web
  streams), SMPTE time code data, non-square pixel
  aspect ratio, duration and types of interlacing.

14
Profile Data
Profile data takes 3 different forms 1. data
contained within a profile object in an
application 2. An XML file on disk 3. Data in the
header of an ASF file

15
Media Samples

• A media sample, or sample, is a block of digital
  media data.
• A sample is the basic unit that is manipulated by
  the reading and writing objects of the Windows
  Media Format SDK.
• The actual contents of an individual sample are
  dictated by the media type associated with the
  sample.
• Ex. For video, each sample represents a single
  frame.
• For audio, the amount of data in an
  individual
• sample is set in the profile used to
  create the ASF file.

16
Media Samples

• Sample can contain uncompressed data, or they can
  contain compressed data, in which case they are
  called stream samples.
• When creating an ASF file, you pass samples to
  the writer. The writer coordinates compression of
  the samples with the appropriate codec and
  arranges the compressed data in the data section
  of the ASF file.
• On playback, the reader reads the compressed
  data, decompresses it, and provides the
  reconstructed uncompressed data as output
  samples.
• All samples used by the Windows Media Format SDK
  are encapsulated in a buffer object whose memory
  is allocated automatically by the SDK run-time
  components.

17
Inputs, Streams and Outputs

• An input is any digital media data stream (such
  as audio or video) that your application delivers
  to the writer object from a source by using
  appropriate APIs.
• If a specified input format is not supported
  natively by the codec, the writer object will
  instantiate either an audio or video helper
  object that is capable of converting a wide
  variety of formats into formats the codec can
  accept.
• For audio inputs, the helper object will adjust
  the bit depth, sample rate, and number of
  channels as necessary.
• For video inputs, the video helper object will
  perform color-space conversions and
  rectangle-size adjustments.
• In some cases, compressed audio and video data
  can be passed in an input stream.
• An input may be of some other media type besides
  audio and video, such as text, script commands,
  still images, or arbitrary file data.

18
Inputs, Streams and Outputs

• An output refers to data that the reader object
  passes to an application for rendering.
• An output equates to a single stream at the time
  of playback.
• If using mutual exclusion, all of the mutually
  exclusive streams share a single output.
• Typically, output data is in the form of
  uncompressed audio or video data, although it can
  contain any type of data.

19
Inputs, Streams and Outputs

• The term stream refers to data in an ASF file,
  as opposed to (1) the input source data before it
  is processed by the writer object and (2) the
  output data after it is decompressed by the
  reader object.
• An ASF stream contains data that comes from a
  single input on the writer object, although more
  than one stream can be created from the same
  input.
• A stream has the same format and compression
  settings from beginning to end.
• The media format and compression settings that
  the writer object will apply to each stream are
  specified in the profile.
• Examples
• A simple ASF file has two streams, one for audio
  and one
• for video.
• A more complex file might have two audio streams
  and several video streams.
• The audio streams might have the same compression
  settings but contain different content, such as a
  narration in different languages.
• The video streams might contain the same content,
  but have different compression settings.

20
Inputs, Streams and Outputs

• The most basic relationship (between input,
  stream and output), is a profile without any
  mutual exclusion.
• Each input is processed by the writer and
  inserted in the ASF file as a single stream.
• On playback, the reader reads the stream and
  delivers uncompressed samples as a single output.

21
Inputs, Streams and Outputs

• A more complex relationship occurs when multiple
  bit rate mutual exclusion is used.
• A single input is processed by the writer and
  encoded at several bit rates.
• Each encoding of the data is inserted in the ASF
  file as a separate stream.
• On playback, the reader determines which stream
  to decompress based upon the available bandwidth.
• The reader then reads the selected stream and
  delivers uncompressed samples as a single output.

22
Inputs, Streams and Outputs

• The third type of relationship can occur when a
  language-based or custom mutual exclusion is
  used.
• Example is including soundtracks in multiple
  languages.
• Multiple inputs are processed by the reader and
  each is inserted into the ASF file as an
  individual stream.
• On playback, your application manually selects
  which stream to decompress based upon logic you
  provide.
• The reader then reads the selected stream and
  delivers uncompressed samples as a single output.

23
Bit Rate

• Bit rate refers to the amount of data per second
  that is delivered from an ASF file.
• Bit rate measurements are in bits per second
  (bps) or kilobits per second (Kbps).
• Bit rate is often confused with bandwidth, which
  is a measurement of the data transfer capacity of
  a network. Bandwidth is also measured in bps and
  Kbps.
• When you are streaming data across a network or
  the Internet, the bit rate is of vital important
  to the end-user experience.
• If the bandwidth available to the network is less
  than the bit rate of the ASF file, the playback
  of the file will be interrupted usually
  insufficient bandwidth will result in either
  samples being skipped, or a pause in playback
  while more data is buffered.

24
Bit Rate

• Every ASF file is assigned a bit rate value at
  the time of creation, based upon the type and
  number of streams that are included in the
  profile used.
• Individual streams have their own bit rates.
• Bit rates can be constant or variable
• Constant bit rate means the original data is
  compressed in such a way as to maintain a
  constant flow of data at approximately the same
  rate
• Variable bit rate means the original data is
  compressed in such a way as to maintain the same
  quality throughout, even though this may mean
  uneven data flow.

25
Bit Rate

• Different bit rate types (constant or variable)
  can be applied to different streams within the
  same file
• You can encode the same content to several
  different streams, each with a different bit rate
  then you can configure the streams so that they
  are mutually exclusive.
• This enables you to create a single file that can
  be streamed to users with different bandwidths.
  This feature is called multiple bit rate, or MBR.

26
Constant Bit Rate (CBR) Encoding

• Default method of encoding with the Windows Media
  Format SDK
• With CBR encoding, the bit rate and size of the
  encoded stream are known prior to encoding.
• Ex. Encoding a 3 minute song at 32,000 bps, you
  know that the file size will be about 704 kB and
  the bandwidth required to stream content is
  32,000 bps.
• The bit rate over time always remains close to
  the average or target bit rate, and the amount of
  variation can be specified.
• Disadvantage of CBR encoding is that the quality
  of the encoded content will not be constant.
• Ex. If encoding a movie using CBR, the scenes
  that are static and easy to encode efficiently
  will be of higher quality than the action scenes
• In general, variations in the quality of a CBR
  file are more pronounced at lower bit rates,
  therefore you should set the BW as high as
  possible.

27
Variable Bit Rate (VBR) Encoding

• Whereas CBR encoding strives to maintain the bit
  rate of the encoded media, VBR strives to achieve
  the best possible quality of the encoded media.
• 3 Types of VBR encoding quality-based,
  unconstrained, and constrained.

28
Quality Based VBR encoding

• Enables you to specify a level of quality for a
  digital media stream instead of a bit rate.
• Codec then encodes the content so that all
  samples are of comparable quality
• Main advantage is that quality is consistent
  within a file, and from one file to the next
• Ex. Different songs encoded are of same quality
• Inappropriate for circumstances where memory or
  bandwidth are restricted, such as portable media
  players, or low-bandwidth internet connections
• In general, it is well suited for local playback
  or high bandwidth network connections.

29
Unconstrained VBR Encoding

• Uses two-pass encoding
• Specify a bit rate for the stream, as you would
  with CBR encoding.
• However, the codec uses the value only as the
  average bit rate for the stream and encodes so
  that the quality is as high as possible.
• The actual bit rate at any point in the encoded
  stream can vary greatly from the average value
• Codec computes the size of the required buffer
  window based on the requirements of the encoded
  samples
• Advantage is that the compressed stream has the
  highest possible quality while staying within a
  predictable average bandwidth.

30
Two-Pass Encoding

• Codec processes all of the sample for the stream
  twice.
• On the first pass, the codec gathers information
  about the content of the stream.
• On the second pass, the codec uses the
  information gathered on the first pass to
  optimize the encoding process for the stream.
• In CBR encoding mode, files that are encoded in
  two passes are more efficient than files encoded
  in a single pass.
• Quality based VBR is one pass, while bitrate
  based VBR is two-pass.
• Cannot be used on live streams

31
Constrained VBR Encoding

• Identical to unconstrained VBR encoding, except
  that you specify a maximum bit rate and a maximum
  buffer window in the profile.
• Codec then optimizes quality within set limits
• If maximum values are set high enough,
  constrained VBR encoding will produce the same
  encoded stream as unconstrained VBR encoding.