Title: ARMOR - A System for Adjusting Repair and Media Scaling for Video Streaming
1ARMOR - A System for Adjusting Repair and Media
Scaling for Video Streaming
- Huahui Wu, Mark Claypool and Robert Kinicki
- Elsevier Journal of Visual Communication and
Image Representation (JVCIR) - Volume 19, Number 8, Pages 489-499 December 2008
2Introduction - Motivation
Video Frames
Repair by Forward Error Correction (FEC)
3Operations Research Concept
- Adjusting Repair and Media Scaling
- Given Network and Application Environment
- For each valid FEC and scaling combination,
measure the video quality - Find the optimal point
4Outline
- Introduction
- Background ?
- Models
- Algorithms
- User Study
- Implementation
- Conclusions
5Video Compression Standard
- MPEG
- Popular compression standard
- Intra-compression and inter-compression
- Three types of frames I, P and B
- Group Of Pictures (GOP)
- ARMOR models MPEG dependencies
6Forward Error Correction (FEC)
- Media-Independent FEC
- Reed-Solomon codes Reed 60
- ARMOR models benefits of FEC for frame
transmission
7Media Scaling (1 of 2)
- Sacrifice data to fit the capacity
- Temporal Scaling (TS)
- Pre-Encoding Temporal Scaling
- Post-encoding Temporal Scaling (below ?)
8Media Scaling (2 of 2)
- Quality Scaling
- MPEG uses quantization in coding to save bits
- Quantization Value (131)
- For example original data 23, 13, 7, 3
- ARMOR models both Temporal Scaling and Quality
Scaling
Quantization Value After Quantization After DeQuantization
3 7, 4, 2, 1 21, 12, 6, 3
6 3, 2, 1, 0 18, 12, 6, 0
12 1, 1, 0, 0 12, 12, 0, 0
9Video Quality Measurements
- Subjective Measurement
- User study, expensive, not practical
- Objective Measurements
- Playable Frame Rate (R)
- Good for Temporal Scaling, not for Quality
Scaling - Peak Signal Noise Ratio (PSNR)
- Good for Quality Scaling, not for Temporal
Scaling - Video Quality Metric (VQM)
- By Institute for Telecommunication science
- Extracts 7 perception-based features
- Only one for frame losses
- Report a distortion value from 0 (no distortion)
to 1 (many) - ARMOR uses both R and VQM
- Includes a user study
10Outline
- Introduction
- Background
- Models?
- Streaming Bitrate Model (cost)
- Video Quality Model (benefit)
- Algorithms
- User Study
- Implementation
- Conclusions
11Parameters and Variables
Video Frames
Repair by Forward Error Correction (FEC)
12Streaming Bitrate Model
- Total streaming bitrate (B), including video
packets and FEC packets - where G is the constant GOP rate
- NPD and NBD are the numbers of transmitting P and
B frames depending on Temporal Scaling level lTS
13Video Quality Model - Overview
- Two distortion factors
- Frame Loss
- Caused by Temporal Scaling and network packet
loss - Appears jerky in the video playout
- Measured by Playable Frame Rate
- Quantization Distortion
- Caused by a high quantization value with Quality
Scaling - Appears visually as coarse granularity in every
frame - Measured by VQM
- Overall Quality
- Distorted Playable Frame Rate
20
14Playable Frame Rate (R)
- Frame Successful Transmission Probability
- Where Frame Size
- Frame Dependencies
- Total Playable Frame Rate
15Distorted Playable Frame Rate (RD )
- Quality scaling distortion varies exponentially
with the quantization level - Distorted Playable Frame Rate
4
16ARMOR Algorithm
- For each Repair and Scaling combination
- Estimate video frame sizes (SI, SP, SB)
- Compute streaming bitrate B and make sure its
under capacity constraint T - Use frame sizes and FEC amount to get
successfully frame transmission rate (qI, qP, qB) - Compute playable frame rate (R)
- Estimate quality scaling distortion (D)
- Compute distorted playable frame rate (RD)
- Exhaustively search all FEC and Scaling
combinations and find optimal quality
17Outline
- Introduction
- Background
- Models
- Algorithms
- User Study ?
- Implementation
- Conclusions
18User Study Goals
- Accuracy of RD
- Correlation with user perceptual quality
- Versus PSNR and VQM?
- Temporal Scaling versus Quality Scaling
- What are the differences?
- Adjusted Repair (FEC) versus No Repair
- Is Adjusted Repair an effective method for
increasing perceptual quality?
19Video Clips
- Compare degraded clips to the original
- Original 30 fps, no quality scaling
- Degraded Combinations of 4 independent factors
(2 options each) - Video and Network environment
- Video content low motion (News)
http//www.youtube.com/watch?vuN3yUm0WZwY or
high motion (Coastguard) http//www.youtube.com/wa
tch?vJQVclZWH5UM - Packet loss rate low loss (1) or high loss (4)
- ARMOR Layer
- Repair adjusted repair or no repair
- Scaling Quality Scaling or Temporal Scaling
- 24 16 combinations for evaluation
20User Study Application
54321
- Two-week volunteer study
- 74 users, most CS undergraduate students (male,
young) - Most LCD, high-rez monitor
Double Stimulus Impairment Scale (DSIS) ITU-R
BT.500-11
21Results Video Quality Metrics (1 of 3)
(Same as original clip)
(Much worse than original clip)
22Results Video Quality Metrics (2 of 3)
- User Score
- Vs.
- VQM Score
- (1 VQM distortion)
23Results Video Quality Metrics (3 of 3)
- User Score
- vs.
- Distorted
- Playable
- Frame Rate
- (RD)
24Results Scaling Methods
ARMOR Prediction (Coastguard)
User Score
- Temporal Scaling versus Quality Scaling
25Results Repair Methods
User Score
ARMOR Prediction (Coastguard)
- Adjusted Repair versus No Repair
26Outline
- Introduction
- Background
- Models
- Algorithms
- User Study
- Implementation ?
- Conclusions
27Implementation Goals
- Provide architecture for ARMOR system
- Validate ARMOR model
- Determine if can make improvements to video
quality in real-time
28Architecture
3
3
2
2
1
2
3
4
1
5
6
7
8
29Experiment Settings
Network (NistNet) Settings Network (NistNet) Settings MPEG Encoder Settings MPEG Encoder Settings
tRTT 50 ms NP 3 frames per GOP
S 1 Kbyte NB 8 frames per GOP
p 0.01 to 0.04 RF 30 frames per sec
- Video clip Paris
- http//www.youtube.com/watch?vXU74KL_72RA
- medium motion and details
- two people sitting, talking, with high-motion
gestures - 1200 CIF (352x288) images
- average I / P / B frame sizes 24.24 KB / 5.20 KB
/ 1.18 KB
30Results
31Conclusions
- Distorted playable frame rate high correlation
with user perceptual quality - Higher than PSNR or VQM
- Adjusting repair improves video streaming quality
significantly - Better than fixed repair and no repair
- Quality Scaling is more effective than Temporal
Scaling - But when bandwidth low and network loss high,
Quality Scaling should be used with Temporal
Scaling - Proof of concept ? ARMOR can be implemented in
real-time to effectively improve streaming quality
32Future Work?
33Future Work
- Implementation of quality scaling
- Implementation of streaming media protocols
- Bandwidth estimation techniques for initial
streaming rate - Alternative repair techniques
- Evaluate with time-varying bandwidth and packet
loss - Classification of video motion and scene
complexity to predict exponential coefficients - User studies to determine if RD works for
different scaling combinations
34ARMOR - A System for Adjusting Repair and Media
Scaling for Video Streaming
- Huahui Wu, Mark Claypool and Robert Kinicki
- Elsevier Journal of Visual Communication and
Image Representation (JVCIR) - Volume 19, Number 8, Pages 489-499 December 2008