Ingen lysbildetittel - PowerPoint PPT Presentation


PPT – Ingen lysbildetittel PowerPoint presentation | free to view - id: 7473fc-NDEyY


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

Ingen lysbildetittel


Network Media Handling A common platform and scenario based on Virtual worlds, gaming and quality Andrew Perkis, Peter Svensson, Professors Centre for Quantifiable ... – PowerPoint PPT presentation

Number of Views:20
Avg rating:3.0/5.0
Slides: 53
Provided by: Even183


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Ingen lysbildetittel

Network Media Handling
  • A common platform and scenario based on Virtual
    worlds, gaming and quality

Andrew Perkis, Peter Svensson, Professors Centre
for Quantifiable Quality of Service in
Communication Systems (Q2S) Department of
Electronics andTelecommuniations Norwegian
University of Science and Technology
  • Centre of Excellence in Quantifiable Quality of
    Service in Communication Systems Q2S
  • Funded by Norwegian Research Council
  • NOK 30 (EURO 4) million/year
  • 45 people (5 Profs.,, PhDs and
  • Basic research and laboratory experimentation
  • Main Goals
  • Network Media Handling
  • QoS assessment and monitoring
  • QoS mechanisms for dynamic networks.

Common architecture - common usage scenario
  • Process
  • Q2S research plan
  • Common usage scenario
  • Develop work on Virtual worlds and serious
  • Align and coordinate research teams
  • Team
  • Andrew, Peter and Touradj
  • Postdocs
  • PhDs
  • Master students
  • Process
  • Coordination weekly meetings, MindMap
  • Information and communication
  • Q2S research plan
  • Network Media Handling
  • Quality assessment and monitoring
  • Networked Media handling
  • Technology to present, manipulate and evaluate
    multi media content
  • Focus on media representation, error protection,
    perceived quality, functional placement
  • QoS mechanisms for dynamic networks
  • Mechanisms that affect QoS, emphasis on
    heterogeneous and dynamic environment
  • Focus on dependability, security, resource
  • Quality assessment and monitoring
  • Measuring methods and models for QoS
  • Measurement of perceived QoS, measurement methods
    and architecture, and traffic performance

Network Media Handling
  • Focusing on the End-Points
  • Functionality in media representation/distribution
  • Error resilience/protection/concealment
  • Media conversion and scalable/layered coding
  • Multi-modal perception of audio and video

Interaction between Networks
End-Points Optimized media handling in wired and
wireless networks Congestion and admission
control for multimedia traffic Objective network
measurements for end-to-end QoS Quantifiable
perceptual quality metrics and their use
Application Studies Interactive two-way and
multi-party communication High-quality streaming
media services and applications Virtual acoustics
/ 3D audio systems Digital cinema
Multimodal Perceptual Quality Assessment
  • Multimodal Perception
  • Relationship between audio and video
  • Perceptual effects of network impairments
  • Relates to end-user expectation and context

Developing Objective Models Study of realistic
network environments Simulation or controlled
laboratory experiments Laboratory for multimodal
subjective experiments Design of quantifiable
metrics for perceived quality
Use of Objective Models Automated monitoring of
end-user perceived quality Relationship to
measurable end-2-end QoS parameters Using
Perceived QoS to adapt and enhance system
Research topics
Q2S resources and activities
Research results
  • Compilations

End user quality assessment
Congestion control with delivery of erroneous
Classification of data, UEP, FEC
Low delay lossless compression for multichannel
Computational quality measurement for IP-based
Demonstration available during break
Figure 2 Sample stereoscopic video contents use
for measuring the impact of exposure affect on
multimedia quality.
Subjective test methods
4 parameter categories visual, audio, gameplay
and network
visual quality metric for HQ images
Novel Evaluation Method DSAQFA
Stereoscopic Quality assessment
Models and assesment
Economical and technology aspects of end-to-end
Quality of Services provisioning in the Internet
PEAQ ODG measurement
Focus of Attention (FOA) based Visual Quality
FP7 Call 4 proposals
European proposals
35 partners
3D Media Internet Services (3DMEDIA) - NoE
  • Leading WPs
  • European 3D Media Lab integration
  • 3D Media Quality of Experience modeling and
  • Common software, data sets and repositories
  • maintain inventory of participating labs
  • enable strong collaboration
  • Develop methods for subjective assessments
  • Establish virtual 3D MEDIA QoE test facility
  • Determine and measure quality features and models
  • Contribute to standardization

Quality of Experience in Networked Systems and
Technologies (QUEST) - NoE
  • Leading WP
  • Develop an integrated QoE platform based on the
    datasets, measures and methodologies provided by
    other WPs.
  • Explore, design, and implement a certification
    mechanism for quality of experience based on the
    integrated platform.
  • Active participation in other WPs
  • Subjective and objective image, video and
    audio-visual quality assessment with respect to
    different services, applications and scenarios

ENd User quality of experieNCe In multi modAl
neTworked mEdia ENUNCIATE CSA
  • A good common scenario is under developments
  • Substantial involvement
  • Cross disciplinary
  • Needs to enhance network components
  • Has potential for being a platform for many
    integrated projects at Q2S

Research results
  • Individual

High definition assesments
  • Digital Cinema
  • For very high qualities, PSNR slightly tends to
    be best objective metric so far, compared to
    structural similarity based metrics
  • No significant differences in correlation between
  • Comparison effects of source vs. source
    channel distortion on perceived quality
  • At low source distortion levels, PSNR tends to
    overestimate quality with both source channel
  • With increasing source distortion levels, ?
    between PSNR values of source and source
    channel distortion gets smaller

Novel Evaluation Method DSAQFA
  • Compare different types of distortion
  • Usually distortion expressed in terms of PSNR
  • Qualitative characteristics of source and channel
    distortion are different
  • PSNR doesnt scale well

Source distortion
Channel distortion
  • PSNR
  • Is not a good measure when comparing different
    types of distortion
  • Especially for low source distortion ( HQ video)
  • Performance increases for higher source
    distortions ( LQ video)
  • Methodology
  • Well suited for comparing different types of
  • Minimum training nesseray
  • Fast, reliable, no rating scales

Congestion control based on delivery of partly
corrupted packets
  • In the presence of bit errors, network throughput
    can be improved and stabilized by allowing
    delivery of corrupted packets
  • Also the video quality is improved (in terms of
    PSNR), if appropriate resilience mechanisms are
    employed (example results below)

High bit error rate
Throughput Avg. PSNR Trad.
0.95 Mbit/s 32.62 MAC Lite 1.55 Mbit/s
34.30 Low bit error
rate Throughput Avg. PSNR
Trad. 1.41 Mbit/s 34.72
MAC Lite 1.57 Mbit/s 35.22
Modeling and services
  • Computational models for audio and multimedia
    quality assessment
  • Developed a priliminary version of a
    computational model for IP-based audio quality
  • Service and business models
  • Developed a simple financial model demonstrating
    effectiveness of cooperation between competing
    providers for end-to-end quality of services
  • Based on real Internet topology, investigated
    current probability and potential evolution
    scenarios for end-to-end QoS provisioning in the

PEAQ ODG measurement
  • PEAQ ODG measurement results for AAC and Lame
    MP3 codecs with different bitrates and packet
    loss rates

Focus of Attention (FOA) based Visual Quality
  • Focus of attention (FOA) has a great influence on
    visual quality assessment. Compared with image
    quality assessment, the influence of FOA on video
    quality assessment is stronger.
  • Objective FOA model can be constructed based on
    bottom-up and top-down approaches.
  • Our experiments demonstrate that the correlation
    between PSNR and subjective measurements can be
    increased by about 0.33 by integrating FOA into
    quality assessment.
  • Based on certain quality features, a visual
    attention based quality metric (VAQM) is proposed
    and it outperforms the existing metrics, such as
    PSNR, SSIM, NTIA model.
  • The future work is to find and design more
    suitable quality features and consider the
    temporal attention as well as the audiovisual
    attention analysis in the joint audio-video
    quality assessment.

High definition quality evaluations
The scatter plots of MOS versus model predictions
Pearsons correlations and their associated 95
confidence intervals for each metric
  • Creation of mixed-reality representations that
    can be used for interactive future 3D-TV
  • Created contents for Stereoscopic projection
  • contents are currently created
  • Manipulating video sequences to study facial
    motion perception on quality.
  • Program and create interactive design and
    contents for augmented reality

Wendy Ann
Analysis of the impacts of affective elements
  • Examine impact of affective elements in
    multimedia presentation on users perception of
    quality and user experience. The created contents
    will be used to access the impacts of the
    following variables
  • Effects of familiarity in the in the perception
    of enjoyment and quality assessment in a
    stereoscopic environment.
  • currently planning to conduct an experiment to
    determine the impact.
  • Effect of moving Tatcher Illusion technique on
    perception of quality .
  • Effects Familiarity over Photometric
    Inconsistencies. This examines how to effectively
    combine natural and synthetic environment.
  • Research shows that people are excellent at
    identifying faces familiar to them, even from
    very low quality images, but are bad at
    recognizing, or even matching, faces that are
    unfamiliar (Hancock et al. 2000).
  • Write a paper adding emotion-level on the
    users perspective model that affects user
    perception and multimedia quality.

Wendy Ann
Quantifying the strategy taken in an ensemble
delayed hand-clappeing
A parameter called compensation factor was
suggested to capture the amount of early clapping
which performers do to compensate the tempo
decrease caused by latency in a musical
CF can explain Chafe effect Moderate amounts of
delay are beneficial to improve the collaboration
If CF equals a certain amount, this would be the
condition for which the trial will have a stable
tempo. We call the achieved delay time over
measure duration as critical CF
Compensation Factor increases linearly with delay
Stereoscopic image quality metric
  • Incorporating crosstalk, depth and visual
  • Stereoscopic techniques provide a very effective
    depth perception sensation because of using the
    binocular cues. The principle behind is of
    displaying two slightly different perspective
    views for left and right eyes.
  • Crosstalk is probably one of the most annoying
    distortion in visualization stage by using
    stereoscopic techniques.
  • We propose a stereoscopic image quality metric
    which defines explicitly the positive and
    negative depth information in the crosstalk
    artefact psychometric function
  • Original psychometric function
    fracM-m1e-( abx)
  • Modified psychometric function
    fracM-m1e-( abxy) cy
  • Where x, y is the objective measurement of the
    crosstalk and depth stimulus intensity
    separately. Parameter a, b, c control the curve
    location and steepness. m is the minimum on the
    grading scale and M is the maximum.

Stereoscopic image quality
Test matrials Original Psychometric Modified Psychometric
Outdoor 0.940 0.977
Ballet 0.729 0.889
Adile 0.703 0.89
Jungle 0.627 0.82
Train data Test data Original Psychometric Modified Psychometric
outdoor ballet, adile, jungle 0.666 0.778
ballet outdoor, adile, jungle 0.703 0.785
adile outdoor, ballet, jungle 0.735 0.801
jungle outdoor, ballet, adile 0.738 0.807
outdoor, ballet adile, jungle 0.653 0.805
adile, jungle outdoor, ballet 0.812 0.843
ballet, jungle outdoor, adile 0.753 0.800
outdoor, adile ballet, jungle 0.676 0.818
QoE in Internet television
- Which network parameters can be extracted and
quantized? - How can these parameters be related
to the QoE? - How to effectively monitor the last
mile (ISP access network)? - How can we handle
various user equipment?
NTNU Recruitment Game
  • Application of Serious Gaming
  • First Person Game on a Virtual Campus (main
  • Will be used as a recruitment tool for high
    school students in Norway
  • Contains presentations of the different studies
    at IME
  • The goal is to show students what NTNU looks like
    and what you are able to make after 5 years of

Game quality parameters
Research plans
Computational quality measurement for IP-based
  • Computational models for audio and multimedia
    quality assessment for a) IP-based audio, b)
    Videoconferencing, IPTV.
  • Quantification of various impairments on audio of
    video quality
  • Quantification of parameters for various audio
    and video codecs, packet loss rates, packet loss
    concealment algorithms, etc.
  • Subjective studies to investigate (to confirm)
    assumptions about simultaneous effects of
    multiple impairment factors
  • Service and business models
  • Continue working on economical aspects of
    end-to-end Quality of Services provisioning in
    the Internet (SLAs, inter-provider revenue
    distribution, pricing strategies, etc.)
  • Risk assessment methods for IP-networking
  • Business models for new services provisioning (3D
    media applications, multimedia, quality effects)

3D media Internet
  • 3D MEDIA related
  • Develop methods for AV subjective assessments
  • Determine and measure AV quality features and
  • Contribute to standardization
  • PERCEVAL related
  • Experimental design for measuring cross-modal
  • Focus on low-attention and long-term quality
  • AV quality objective metric design and validation

  • Rate control allowing delivery of partly
    corrupted packets
  • More simulating results with more realistic video
    sequences and larger network topologies
  • Analysis of perceptual significance of different
    data elements (media units)
  • Essential in unequal erasure protection schemes
    (UEP FEC, retransmissions)
  • Aim is to improve the methods for classifying
    data elements
  • Audiovisual quality assessment
  • Results to be used for optimizing the trade-off
    between audio and video distortion, as well as
    source and channel distortion
  • Subjective metrics to be turned into objective

Perceived audio-visual quality assessment for
future media
  • Perceived quality assessment for multimodal media
    with complex scenes (HDTV, UHDTV, NHX, 3D, etc.)
  • Prepare appropriate media scenarios and dataset,
    design subjective quality methodologies.
  • Cognitive model for joint audio-visual quality
    assessment and corresponding objective metrics.
  • Spatial and temporal attention analysis, semantic
    multimedia content analysis and the relationship
    with the perceptual quality assessment.
  • Integrated platform for quality of experience
    (QoE) certificate
  • Define certificate mechanism for QoE methods for
    future media.

Virtual worlds
  • In the new age of virtual reality and mixed
    environment, users become part of the synthetic
    environment to generate new forms of experiences.
    How can we effectively maximize our perception of
    quality on these kinds of applications?
  • Synthetic or 3D contents are less expensive than
    natural representations because they are mostly
    pre-defined and can easily be computed in
    real-time. In real-time or live broadcast (e.g.
    news event, weather forecast, live sports) some
    regions are not so salient and therefore possible
    to substitute or combine with synthetic contents.
    How can we effectively combine natural and
    synthetic representations without sacrificing
    quality perception.

Wendy ann
Stereoscopic visual quality
  • How does the artefact of coding or communication
    on different part of the data (left vs right
    multiview, or color vs depth rendering) affect
    the overall stereoscopic quality
  • Same artefect on the different position of left
    and right multiview, or on the color and depth.
    How it affects the people.
  • How about the artefacts on the different depth?
    Is there any difference? It seems that people
    will focus more on the foreground(less depth)
    than background(more depth).

Multimodal Perceptual Quality Assessment of High
Definition Content
  • Codec performance for digital cinema application
  • Conducting subjective quality assessment using
    currently standardized methodology in DCI
    specified Digital Cinema environment
  • Developing subjective assessment methodology that
    take into account multimodal factor
  • Developing objective model that take into account
    multimodal factor

Extra slides
  • Backups

Computational quality measurement for IP-based
audio (1)
  • Quality measurement techniques
  • Subjective (using human subjects)
  • Objective based on comparison with original
    non-degraded signal
  • Objective computational models (when no original
    signal is available for comparision)
  • Focus on objective computational quality
    assessment for IP-based audio
  • Previous work computational quality model for
    narrowband (up to 4 kHz) and wideband (up to 7
    kHz) speech
  • Work on a similar model for IP-based audio in

Computational quality measurement for IP-based
audio (2)
  • 100-point scale is used for quality measurement
    and impairment quantification
  • Impairments are percepually additive
  • Input quantified qualtiy impairments
  • Output uni-directional quality estimate should
    be mapped to a subjective quality measurement
  • Impairments codec, bit rate, packet loss rate,
    packet loss concealment, delay (for interactive
    applications), content dependency,noise, etc.

Computational quality measurement for IP-based
audio (3)
  • Current work
  • Start with quantification of encoding and packet
    loss impairments Q 100 function (codec,
    bitrate) function (loss rate, packet loss
  • Use software implementation of the objective
    accurate PEAQ algorithm, which is based on
    comparison of degraded and original versions of a
  • Qualitify the impairments using experimental PEAQ
    measurement results converting data to the
    perceptually additive 100-point scale.

Some pics
  • Encoding impairment quantification

Loss impairment quantification
Future work (2)
  • We focus more on the talking person and subtitle
    when we are watching TV, but it is difficulty to
    know who is talking by the image analysis or
    speech processing. However, if it is 3DTV
    together with 3D audio, we can use the 3D audio
    to know who is talking now in which position,
    then using this position, we can further know the
    corresponding pixel position in the image, so we
    can differentiate the quality for ROI for the
    better perceived quality

Future work (3)
  • If people feel the depth, then it will keep on.
    However, if he stopped a while, it need time for
    him to feel the depth again. So how the
    stereoscopic affect the people's eye focus. I
    think it will in certain degree make the people
    more stressed since he can not move the eyes as
    frequently as he wants.
  • Besides, if there is depth, maybe people will
    focus on the different things from the pure 2D