Challenges in Wireless Multimedia


1
Challenges in Wireless Multimedia

• CSE Department Seminar Series
• September 26 2003
• Borko Furht

2
OUTLINE

• Scalable Video Source Coding
• Channel Coding and Error Control
• Power-Aware Coding and Transmission Techniques
• Networking Issues
• Rate control
• Multimedia Security
• Application Virtual Workplace

3
Mobile Internet Access
Internet Subscribers (millions)
Source Ericsson
Year
4
Wireless Multimedia Architecture
5
Bandwidth Problem
Bandwidth is like money and sex - only too much
seems to be enough. Arnold Penzias former
chief scientist of Bell Labs
6
Generations of WAN Air Interfaces Based on
Access Technologies

• 1G FDMA (Frequency Division Multiple Access)
• 1980s - each caller has a dedicated frequency
  channel (3 callers use 3 channels)
• 2G TDMA (Time Division Multiple Access) and GSM
  (Groupe Speciale Mobile)
• 1990s - callers timeshare a frequency channel (9
  callers use 3 channels)
• 3G CDMA (Code Division Multiple Access) and
  WCDMA (Wide Code Division Multiple Access)
• 1990s - callers use a shorter bandwidth
• 2000s - spread spectrum. Each code is spread
  randomly broken down and mixed (14 callers use
  the full bandwidth of 1 channel)

7
Data Services
2000
8
M-Commerce Applications

• Transaction Management
• Digital Content Delivery
• Telemetry Services
• Searching for Killer Applications!

9
Transaction Management

• On-line shopping tailored to mobile phones and
  PDAs
• on-line catalogs
• shopping carts
• back office functions
• Initiate and pay for purchases and services
• Micro-transactions - subway fees digital cash

10
Digital Content Delivery

• Information browsing
• weather
• transit schedules
• sport scores
• ticket availability
• market prices
• Downloading entertainment products
• Transferring software high-resolution images
  and full-motion video
• Innovative video applications

11
Telemetry Services

• Wide range of new applications
• Transmission of receipt of status sensing and
  measurement information
• Communication with various devices from homes
  offices or in the field
• Activation of remote recording devices or service
  systems

12
ATT Wireless Welcome to mlife
Get the latest weather forecasts
Get the latest weather forecasts
Find breaking news flight information entertainm
ent..
Get the business and investments news
Get the business and investments news
13
Future of Wireless Technology

• Mobile networks have already begun the migration
  to IP-based networks
• IP as the routing protocol
• 4G New spectrum and Emerging wireless air
  interfaces (very high bandwidth 10 Mbps)
• It may entirely be IP-based and packet-switched
• Increasing usage of wireless spectrum
• On average the number of channels has doubled
  every 30 months since 1985 (Coopers law)

14
Wireless Multimedia Challenges

• Adaptive Decoding - Optimizing rich digital media
  for mobile information devices with limited
  processing power limited battery life and
  varying display sizes
• Error Resilience - Delivering rich digital media
  over wireless networks that have high error rates
  and low and varying transmission speeds
• Network Access - Delivering rich digital media
  without adversely affecting the delivery of voice
  and data services
• Negotiable QoS for IP multimedia sessions as well
  as for individual media components

15
Components of a Wireless Video System
Input Video
Transport Network Layer
Video Encoder
Packetizer
Modulator
Channel Encoder
Wireless Channel
Output Video
Video Decoder
Depacketizer
Demodulator
Channel Decoder
Tradeoff Throughput Reliability Delay
16
Source and Channel CodingTrade-off

• Classic goal of source coding
• Achieve the lowest possible distortion for a
  given target bit rate
• Classic goal of channel coding
• Deliver reliable information at a rate that is as
  close as possible to the channel capacity
• Shannons separation principle
• It is possible to independently consider source
  and channel coding without loss in performance
• The separation principle applies only to
  point-to-point communications and it is not valid
  for multiuser or broadcast scenarios

17
Pragmatic Approach

• Keep the source coder and channel coder separate
  but optimize their parameters jointly
• Key problem in this optimization is the bit
  allocation between the source and channel coder
• Joint source-channel coding schemes
• In the infancy today
• Exploit the redundancy in the source signal for
  channel decoding (Source-controlled channel
  decoding)
• Designing the source codec for a given channel
  characteristic (Channel-optimized source coding)

18
Characteristics of a Wireless Video System

• The capacity of wireless channel is limited by
  the available bandwidth of the radio spectrum and
  various types of noise and interference
• The wireless channel is the weakest link of
  multimedia networks mobility causes fading and
  error bursts
• Resulting transmission errors require error
  control techniques (such as FEC - forward error
  control and ARQ automatic repeat request)

19
The Case for Scalable Video Coding

• In emerging wireless applications multimedia
  data will be streamed
• over various access networks (GPRS UMTS WLANs
  etc.)
• to a variety of devices (PCs TVs PDAs cellular
  phones etc.)
• The transmission of multimedia data need to cope
  with unpredictable bandwidth variations
• due to heterogeneous access technologies of
  receivers (3G 802.11a etc.) or
• due to dynamic changes of network conditions
  (interference etc.)

20
Scalable Video Coding Techniques

• Scalable video coding methods can adapt in real
  time to the bandwidth variations over
  heterogeneous networks and to the terminal
  capabilities while using the same pre-encoded
  system.
• Scalable video coding uses multiple bit streams
  layered video coding
• For example in a two-layer coding the codec
  generates two bit streams
• Base layer the most vital video information
• Enhancement layer the residual information to
  enhance the quality of the base layer image
• This form of two-layer coding is known as SNR
  scalability

21
Scalability Techniques

• Data partitioning
• SNR scalability
• Spatial scalability
• Temporal scalability
• Hybrid scalability

22
Data Partitioning

• Data partitioning is used when two channels are
  available for transmission (it is not true
  scalable coding)
• Divides the bitstream of a single layer into two
  parts or layers.

Multiplexer
Base-layer bitstream
Video in
Output bitstream
Single layer encoder
Data Partitioner
Enhancement- layer bitstream
23
Block DiagramTwo-Layer SNR Scalable Coder
Base layer bitstream
Video in
Base layer Encoder (MPEG 1)
Multiplexer

Output bitstream
Base layer Decoder (MPEG 1)
-
Enhancement layer Encoder (MPEG 2)
Enhancement layer bitstream
24
Adaptive Video Coder Based on 3D-DCT

• Original video cube 8x8x8
• 3D Discrete Cosine Transform

25
Motion Analysis for Various Blocks

• Partition image into NxN inspection areas
• Examine each area for motion content based on
  Normalized Pixel Difference (NPD) between frames
  1 and 8
• Three motion types defined
• No Motion
• Low Motion and
• High Motion
• 3D-DCT block size adapts based on determined
  motion content

26
Example of a Video Hallway Clip

• 8 Frames of luminance (Y) component
• Inspection area size 16x16
• Inspection areas used to determine NPD thresholds
  

27
Video Example 420
Original
Cr120
Cr190
Cr408
28
Architecture of 3D-DCT Adaptive Encoder
29
Example of a Scalable CodingAdaptive 3D-DCT Coder

Original
30
Adaptive 3D-DCT Coder

Layer 1 Cr164 (in vehicles 144
Kbps)
31
Adaptive 3D-DCT Coder

Adding Enhancement Layer 2 Cr96 (For
pedestrians 384 Kbps)
32
Adaptive 3D-DCT Coder

Adding Enhancement Layer 3 Cr54 (for indoor
use 2 Mbps)
33
Channel Coding and Error ControlEffects of
Transmission Errors

• Example 1 The extra insertion bit causing the
  loss of the first GOB
• Example 3 Corruption of the group quantizer
  parameter that resulted in employing the wrong
  quantizer in decoder

• Error-free frame
• Example 2 Corrupted group number
• causing a GOB misplacement

34
Channel Coding and Error Control

• Trade-off between throughput reliability and
  delay
• Forward Error Correction (FEC)
• Automatic Repeat Request (ARQ)
• Error Resilience Techniques for Low Bit Rate
  Video
• Techniques that reduce the amount of introduced
  errors for a given error event (Resynchronization)
  
• Techniques that limit interframe error propagation

35
Recovery From Packet Loss FEC scheme

• Piggyback lower quality stream
• Send lower resolutionaudio stream as
  theredundant information
• For example nominal stream PCM at 64 kbpsand
  redundant streamGSM at 13 kbps.
• Sender creates packet by taking the nth chunk
  from nominal stream and appending to it the
  (n-1)st chunk from redundant stream.

• Whenever there is non-consecutive loss
  thereceiver can conceal the loss.
• Only two packets need to be received before
  playback
• Can also append (n-1)st and (n-2)nd low-bit
  ratechunk

36
Joint Source Coding and Transmission Power
Management

• Goal to limit the amount of distortion in the
  received video sequence while minimizing
  transmission energy
• Combines
• Error resilience and concealment techniques at
  the source coding level and
• Transmission power management at the physical
  layer
• Optimization problem Minimizing the energy
  required to transmit video under distortion and
  delay constraints

37
Joint Source Coding andTransmission Power
Management
Decoder Concealment Strategy
Channel State Information
Controller
Control power
Control coding parameters
Modulator
Channel Encoder
Video Encoder
Video in
Wireless Channel
Demodulator
Channel Decoder
Video out
Video Decoder
Goal to limit the amount of distortion in the
received video sequence
while minimizing transmission
energy
38
Transmission Energy

• Total energy to transmit all the packets in a
  frame
• The algorithm calculates the power needed to
  achieve the desired probability of loss

39
Controlling the Bit Rate

• Most video codecs use variable-length coding
  techniques
• Most existing mobile radio systems transmit at a
  fixed bit rate
• Goal Constant signaling rate leading to a
  different constant bit rate for each modulation
  scheme
• Rate Control Techniques - determine the sending
  rate of video traffic based on the estimated
  bandwidth in the network
• Source-Based Rate Control
• Receiver-Based Rate Control
• Hybrid-Based Rate Control

40
Rate Shaping Techniques

• Techniques that adapt the rate of pre-compressed
  video stream to a target rate constraint
• Rate shaper is an interface (or filter) between
  the compression layer and the network transport
  layer

Variable rate
Constant bit rate
Network Transport Layer
Video in
Compression Layer
Rate Shaper
41
Rate Shapers

• Codec filters
• Frame-dropping filters (dropping BP or I
  frames)
• Layer-dropping filters (in scalable video coding
  schemes)
• Frequency filters (discard DCT coefficients of
  the highest frequency)
• Requantization filters (reqauntizes the DCT
  coefficients with a larger quantizers resulting
  in rate reduction)

42
Multimedia Content Security

• Access control in applications such as
  video-on-demand and videoconferencing so only
  selected users can access the data
• Established encryption algorithms (DES or AES)
  are very complicated and involve large number of
  computations.
• Software implementations of these schemes are not
  fast enough to process the large amount of
  multimedia data
• Hardware implementations require additional costs
  to both data generation and receivers

43
General ArchitectureSelective Encryption System
44
Example of Video EncryptionMPEG Encoder
Secret Key Selective encryption algorithm That
operates on sign bits of DC coefficients
Secret Key Permutation of the Huffman codeword
list
Secret Key Randomly change the sign bits of
motion vectors
45
Example Encrypting Frames of a MPEG-4 Video
Sequence
Original frame Encrypted VLC only Encrypted
FLC only Encrypted VLC and FLC
46
Virtues of the Virtual Workplace

• Universal access to information applications
  services processes and people from any device
  over any network connection - wired wireless or
  Web

47
Virtual Workplace Video Clip

• Wireless Internet and Web
• Wireless appliances
• Security
• Redundant systems
• Wireless applications videoconferencing

48
(No Transcript)
49
The Portable Office
Take the office with you wherever you go
50
Secure Authentication
High Security Authentication including
Bio-Authentication
51
IntegratedMessaging and Communication
Integrated messaging (eg. voice chat) voice to
text with intelligent alerting
52
Information Portability
Access information over any connection wired
or wireless regardless of form factor
53
Business Collaboration
Collaborative capabilities allow on-line
information sharing and communication
54
Business Continuity
Resilient to network interruptions
55
Further Readings

• Hanzo Cherriman and Streit Wireless Video
  Communications IEEE Press 2001.
• IEEE Trans. On Circuits and Systems for Video
  Technology Special Issue on Wireless Video June
  2002.
• Sun and Reibman Compressed Video over
  Networks Marcel Dekker 2001
• Wang Ostermann and Zhang Video Processing and
  Communications Prentice Hall 2002.
• Furht and Ilyas Wireless Internet Handbook
  CRC Press 2003.