Title: Internet Media Delivery Systems: Current Design, Open Issues, Research Problems.
1Internet Media Delivery Systems Current Design,
Open Issues, Research Problems.
- G. S. Greenbaum, Ph.D
- RealNetworks Codec Group
- Stanford University
- Feb 13, 2001
2(No Transcript)
3Outline
- Introduction to Internet-based Media Distribution
Systems - Components
- A Real Example of a Media System RealSystem iQ?
- RealProducer 8.5
- RealSystem Sever 8 and Proxy 8 with Neuralcast
Technologies - RealPlayer 8
- Open Problems
- source/channel coding
- server storage/complexity tradeoffs
- network traffic optimization problems
4Introduction Ambitions of a Media Delivery
System
- The Overall Goal is to Maximize the End-User
Subjective Experience to Streaming Multimedia - A Successful Internet Media Delivery System Must
Provide - Tools for the easy re-purposing of new and extant
content distribution over digital networks - Robust, Distributed, Scalable, Secure, and
Flexible Architecture for content distribution - Ubiquitous client distribution supporting diverse
data types
5Introduction Components
Capture Encoding Serving Internet
distribution Playback
IP network
Real Player
Real Server
Real Producer
SDK
SDK
SDK
Plug-ins
Plug-ins
Plug-ins
Real Proxy
Plug-ins
6Introduction Components
- Content Creation Digitization and Encoding
- Most general purpose content starts life in
analog form which needs conversion to digital
form - Rich multimedia data types audio, video, image,
VRML, etc. - Lossy and lossless modes of compression
- Download vs Broadcast (streamed) modes
- Encoding tools typically have little knowledge of
the distribution network or computational
resources of the client playback device - Metadata for efficient search and retrieval
- Rights of content provider must be insured
7Introduction Components
- Transmission and Distribution
- Live and pre-recorded Broadcast and VOD modes of
operation - Redundant, Intelligent, and Reliable architecture
needed - Redundant Zero points of failure at each
critical node of the distribution chain - Intelligent Load balancing (dynamic capacity
allocation) for efficient distribution - Reliable Stream distributions that traverse
public and private network segments fortified
against loss and congested - Architecture usually contains splitters/repeaters
and caches for efficient usage of network/gateway
capacity working in both a push and pull mode
8Introduction Components
- Client-side Player residing on different
appliances and diverse networks - DSL, Cable Modem, Intranets/firewalls, POTS,
ISDN, etc. - PCs, cellular devices, PDAs, set-top boxes,
digital VCRs, etc.
9RealSystem iQ RealProducer 8.5
- State of the Art Narrowband and Broadband Audio
and Video Codecs RealVideo 8 and RealAudio 8 - Pre-processing
- Noise removal
- Resampling
- Inverse-Telecine
- Deinterlacing
- Rate control
- Two-pass, VBR
- Channel adaptation
- SureStream, forward unequal ECC, data interleaving
10Broadcast
SLTA library
File System
File Format
Producer
Live Text
Live Pix
Remote Broadcast Library
Internet/Intranet
11RealSystem iQ Server 8
- Adaptive serving
- Bandwidth detection
- Stream switching
- Thinning
- multicast
- Designed for distributed
- delivery networks in mind
- proxy plug-ins
- edge-server plug-ins
- distributed licensing
- Features
- Ad Insertion, Authentication extensions, remote
administration, firewall proxy, etc. (see
www.realnetworks.com for more details)
12RealSystem iQ Server 8
Broadcast Applications
Admin, monitoring Applications
Broadcast Plug-in
File Format Plug-in
File System Plug-in
Generic Plug-ins (logging, monitoring, etc.)
Server Core
Packet Sink Plug-in
Allowance Plug-in
Internet/Intranet
13Server Clustering
Originate Deliver from Any Where An
architecture free from origins and edges
14Distributed Architectures
15RealSystem iQ Server 8 Neuralcast Technologies
- NeuralCast Communications Protocol
- The ability for a network of RealSystem Servers
to become self-aware by exchanging information
and making decisions. Current abilities include
capacity sharing and capacity fail over in the
event of a network or equipment issue. - NeuralCast Live Distribution
- The intelligent and reliable delivery of
broadcasts through a network or RealSystem
Servers. Current abilities include
multi-protocol transmission between servers,
error correcting methods for streams, terrestrial
and satellite multicast support. - NeuralCast Live Redundancy
- From encoder to server or from server to server,
the ability to send redundant streams, providing
a fail-over feed in the event of a network or
equipment outage.
16RealSystem iQ Server 8 Neuralcast Technologies
- NeuralCast Live Distribution
- Push and Pull models
- Re-Transmission requests made at receiver not
originating transmitter - FEC configurable
- Multiple Transport Support allows the same
broadcast to be transmitted via conventional
unicast UPD or TCP across non-multicast enabled
segments
17RealSystem iQ Server 8 Neuralcast Technologies
- NeuralCast Live Redundancy
- Stateless protocol decouples live broadcast
packet contents from the transmitting network - Live broadcast streams that are sent redundantly
over different networks may be re-converged at
the Server receiver
Encoder/ SLTA
18RealSystem iQ Live Webcast Deployment
Server Splitter Relays
19RealSystem iQ Real Proxy 8
- Access Provider Deployment Goals
- Optimize gateway bandwidth for streaming media
- Eliminate redundant requests for media from
subscribers for content that is back hauled from
the public Internet - Improve playback quality experience by migrating
content closer to the viewing subscriber - Deployed Technologies
- RealSystem Proxy near Telco/Cable cross connects
or at data center
20RealSystem iQ Real Proxy 8 Deployment
Traffic Management
Cache Acquisition (Cold)
21RealSystem iQ Real Proxy 8 Deployment
Cache Playback (Hot)
Live Splitting
22RealSystem iQ RealPlayer 8
- Post-Filters
- Sharpening
- color correction
- graphic equalizer
- Rescaling
- Bandwidth Simulator
- Reverb
- Network Aware
- Monitors Network connection
- Auto-update enabled to obtain latest player
components - Auto-configures for best transport method
- Features
- RV8, RA8, Visualizations, Radio Tuner, Media
Search, localizations for different languages,
etc. see www.real.com for a complete list.
23RealSystem iQReal Client Architecture
RealPlayer
RealJukebox
ActiveX
Netscape Plug-in
Client Core
File Format Plug-in
File System Plug-in
Rendering Plug-in
Generic Plug-ins
Internet/Intranet
24Open Problems...
Overall minimum-distortion optimization
Source/Channel coding Internet traffic problems
IP network
Real Player
Real Server
Real Producer
Real Proxy
Traffic optimization in dedicated delivery
network
Server storage vs. complexity tradeoff
25Source Coding IssuesThe Challange
- Video no adequate distortion measure
- Human retina - an ultimate 1201 lossy compressor
Human eye has around 6M cones 120M rods but
only around 1M ganglion fibers in the optical
nerve.
26Source Coding IssuesThe Challange
- Some other facts from physiology
- visual recognition requires power of 30 of the
cortex - hearing -- uses just 3 of the cortex
- Ultimate video decoder 1/3 of a human
(adequately educated and trained)
27Source Coding Issues
- Constraints
- Bounded delay (1-10 s and/or MB constraint)
- Targeting many discrete, fixed bit rate
connections - Random-access is necessary
- Seek/rewind on pre-recorded media
- Ability to join live stream
- Packetization Each packet must contain
sufficient information to be decoded. - Low complexity decoding
- Real-time encoding
- Need to assume inherently lossy network transport
28Source Coding Issues
- Bounded Delay
- Not as much an issue for Downloadable Media
- Leveraged to solve inhomogeneity in content bit
allocation (VBR), network jitter, random access,
re-transmission requests, etc. - Imposes constraints on look-ahead dependencies
- Imposes constraints on interleaving techniques
for error mitigation - Cannot replicated the remote-control paradigm
of broadcast television.
29Source Coding Issues
- Bandwidth Scalability
- Layered Scalability, e.g. FGS, annex O.
- Introduces additional delay
- Hit in coding efficiency
- Embedded techniques, e.g. FGS, wavelets
- Introduces additional delay
- Possible hit in coding efficiency
- Multi-Encoding e.g. SureStream
- Increased file size and possible delivery costs
- Optimal coding efficiency for targeted rate
- ?
30Source Coding Issues
- How to facilitate random access decoding?
- independent encoding
- pre-encoded transitions
- pre-encoded correlated fragments combined with
Slepian-Wolf encoding - use of multiple-description codes
- something else???
?
31Source Coding Issues
- Packetization
- Not an issue for downloadable media
- Imposes constraints on dependencies for streaming
content leading to a decrease in coding
efficiency - Due to IP and packet reconstruction overhead,
packet size should be nearly as large as the
packet fragmentation limit imposed by routers,
1500 bytes. increased latency of stream - Complexity Scalability
- How does the algorithm scale on limited resource
devices for the encoder and decoder? - Scalable Post-filters
- Limited decode B-frames dropped or I-frame only
- Short-cuts limited ME/MC
32Channel Coding Issues
- IP have been designed to relieve engineers from
thinking about physical connection and channel
coding problems - Ironically enough, with streaming media we now
have to go back to basics - UDP is mainly used packet erasures
- ARQ and ECC
- Client side mitigation techniques
- Ideally, we want the server to do adaptive
channel coding/ transmission. Cost is reduction
in server load. - How much of this work can be done ahead of the
time by encoder? - Is there a way to store all necessary information
in a compact form? Is SureStream the ideal?
33Source/Channel Coding Issues
- Separation Theorem may not hold
- packets have finite lengths
- packets can be sent via multiuser channels
- Should we use joint source/channel coding?
- How to combine distortion due to lossy
compression and one due to channel errors - can we design a universal lossy joint
source/channel code?
34Source/Channel coding problems
- Another way to deal with IP packets is to use
multiple-description coding
- Ozarow (1980) The achievable set of quantuples
(R1,R2 ,d1,d2,d3) is given by
35Summary
- Several embodiments of Media Delivery Systems
exist today, but the technology is still in its
toddler years. - Many more issues, problems, and questions than
answers. - Plenty of opportunities for grad students,
post-docs, and professors! (and job security for
those outside of academia)