Measuring QoE for IPTV Services

1
(No Transcript)
2
Measuring Quality of Experience for Successful
IPTV Deployments

• Dr. Stefan Winkler

3
Outline

• Challenges
• Digital Video Quality Issues
• Traditional Measurements (QoS) vs. Quality of
  Experience (QoE)
• Possible Solutions
• QoE Measurement Approaches
• End-to-end QoE Management
• Conclusions

4
Digital Video Challenges

• Demanding traffic profiles
• High bandwidth streams
• High traffic volumes
• Live, VOD

• Network effects
• Video impacted heavily with minor network
  impairments
• Multi-vendor network complicates diagnosis /
  troubleshooting

Service quality degradations Difficult
diagnosis, troubleshooting Rising management and
OPEX costs Higher customer churn

• High end-user expectations
• Defined with decades of history
• Grow rapidly with HD
• Low tolerance for poor quality

• New architectures
• Sensitive video processing devices create
  possibility for various impairment sources
• Ad-insertion, middleware

5
What Drives End-Users
Source MRG 2007 IPTV Video Quality Survey,
available at http//qoe.symmetricom.com
6
Service Providers View
Source MRG 2007 IPTV Video Quality Survey,
available at http//qoe.symmetricom.com
7
Service Providers View
Source MRG 2007 IPTV Video Quality Survey,
available at http//qoe.symmetricom.com
8
Sources of Video Issues
Consider all elements for true end-to-end solution
9
Compression Artifacts
Original
MPEG-2
H.264
10
PSNR vs. QoE
Same amount of distortion (PSNR) different
perceived quality
Understand model human vision system
11
QoS vs. QoE

• Quality of Experience
• Content impairments
• Blockiness, Jerkiness,
• End-user quality
• Application driven

• Quality of Service
• Network-centric
• Delay, packet loss, jitter
• Transmission quality
• Content agnostic

QoS
QoE
12
QoS vs. QoE
Same network impairments Packet Loss 1 Delay
10ms Jitter 50us Bandwidth 500kbps
Different perceived quality!
13
MDI vs. QoE

• Media Delivery Index (MDI)
• MDI consists of two metrics
• Delay Factor (DF)
• Media Loss Rate (MLR)
• MDI limitations
• MDI assumes constant bit rate (CBR) traffic
• MDI does not consider video payload or content
• MDI values are not intuitive
• MDI doesnt correlate with video quality

14
MDI vs. QoE
MOS
Media Loss
15
QoS/QoE Cycle
Alignment gap
Service provider
End-user
Desired QoE
Targeted QoS
Value gap
Execution gap
Perceived QoE
DeliveredQoS
Perception gap
Adapted from ITU-T Rec. G.1000 and COM12C185E
16
Outline

• Challenges
• Digital Video Quality Issues
• Traditional Measurements (QoS) vs. Quality of
  Experience (QoE)
• Possible Solutions
• QoE Measurement Approaches
• End-to-end QoE Management
• Conclusions

17
Full-Reference Approach
Sender
Receiver

• Comparison of individual video frames
• Offline analysis (capture is required) lab
  applications
• High detail and accuracy
• Alignment procedure

Full Ref. QualityMeasurement
Full reference information
18
No-Reference Approach
Sender
Receiver

• Non-intrusive, in-service measurement
• Real-time monitoring applications
• No alignment required

No-Ref. QualityMeasurement
19
Reduced-Reference Approach
Sender
Receiver

• Monitoring applications
• Correlation of content and network impairments
• Encrypted environments

Reduced Ref.Measurement
Feature Extraction
Feature Extraction
20
Content Network Metrics
(Correlation Engine)
"Vision is the most highly developed of the human
senses, so people are even more sensitive to
flaws in video images than, say, the sound of a
telephone conversation. Ken Wirt, Cisco Vice
President Consumer Marketing, Jan 2008
21
(No Transcript)
22
(No Transcript)
23
Vision Modeling

• Contrast perception
• Visibility of different patterns
• Frequency dependencies
• Masking effects
• Interaction of content and impairments
• Texture, edges, luminance
• Spatial and temporal masking
• Color perception

Sensitivity
Spatial frequency cpd
Temporal frequency Hz
Visibility threshold
Maskingcurve
Thresholdwithoutmasker
Target contrast
Masker contrast
24
End-to-end QoE

• Deep Content Analysis (bitstream)
• Detect content impairments
• Deep inspection to associate content to
  timestamps (eg TS1 I-Frame)

• Deep Content Analysis(pixel by pixel)
• Source content and encoder / transcoder validation

• Network (header or stream) Analysis
• Detect QoS issues
• Content analysis where possible (unencrypted)
• Inspection of QoS to associate timestamps to
  impairments (eg TS1 Packet Loss)

Content Stream Analysis PES inspection PCR
jitter etc.
Content Impairments Blockiness, blur
Jerkiness Freeze/black frame Noise, Color
TS1 I-Frame
Network Impairments Loss Delay Jitter
Bandwidth
Q-Advisor
Correlation Engine
TS1 Packet Loss
Packet Loss -gt I-Frame
Human Vision System Model
VideoQualityReports
25
IPTV QoE Management

• 1. Understand the Service
• Is there an issue?
• Does it matter?

• 2. Understand the Problem
• What does the customer see?
• What is the exact cause?

1.0

• 3. Understand the Solution
• What is the impairment source?

26
Conclusions

• QoE is application-driven
• Measure both network and content impairments
• QoE is user-oriented
• Measure how end-user perceives service issues
• End-to-end quality measurement
• Cover different impairment sources
• Identify problem causes

27
Contact Info
Stefan Winklerswinkler_at_symmetricom.com Company
qoe.symmetricom.com Further Readingstefan.winkle
r.net/book.html