Current Trends in Image Quality Perception

1
Current Trends in Image Quality Perception

• Mason Macklem
• Simon Fraser University

2
General Outline

• Examine current image quality standard
• need for improvements on current standard
• Examine common image compression techniques
• potential quality techniques applicable to each
• Discuss further theoretical and constructive
  models

3
Image Compression Model

• Transform an image from one domain into a better
  domain, in which the imperceptible information
  contained in the image is easily discarded
• Goal more efficient representation

4
3 Ways to Improve Compression

• Better domain design better image transforms,
  improve energy compaction
• Imperceptible design better perceptual image
  metric
• Discarded design better image quantization
  methods

5
Current Standard MSE-based

• Mean-Squared Error (MSE)
• Root-Mean-Squared Error (RMS)

• Peak Signal-To-Noise Ratio (PSNR)

6
MSE-based metrics

• Measure image quality locally, ie. pixel-by-pixel
  area
• Not representative of what the eye actually sees
• Returns a single number, intended to represent
  quality of compressed image
• Not accurate for cross-image or cross-algorithm
  comparisons

7
MSE pathologies

• Local (pixel-by-pixel) quality measure
• does not differentiate between constant (not
  noticeable) and varying (noticeable) error-types
• does not take into account local contrast
• assumes no delay or noise in channel
• Known result above error types are treated
  differently by HVS

8
Original bird
Sinusoidal error
Constant error
9
Low contrast no masking
High contrast masking
10
Sinusoidal error (MSE 12.34)
Image offset 1 pixel (MSE 230.7)
Original bird
11
MSE Pathology II Fractal Compression

• Based on theory of Partitioned Iterated Function
  Systems (PIFS)
• uses larger blocks contained in the image to
  represent smaller blocks
• represent smaller blocks using displacement
  vector
• match larger to smaller to maintain contraction
• blocks chosen to minimize MSE
• partly motivated due to promising MSE results

12
Fractal Compression Model

• Divide image into domain and range blocks
• Find closest affine transformation for each range
  image from domain blocks
• Set maximum depth, code all unmatched blocks
  manually (ie. DCT)

• Highly computational, dependent on choice of
  domain and range blocks
• Balance computational and quality requirements
• fewer blocks checked, lower image quality
• slow encoding offset by fast decoding

13

• Models to improve computational complexity
• loosen criteria for matching blocks, ie. take
  first block below a given threshold, take closest
  block within a given radius
• Good MSE/PSNR results not reflected in visual
  appearance of resulting image
• success of fractal compression dependent more on
  internal composition of image than on overall
  model
• if similar blocks are not present in domain
  blocks, then dissimilar blocks will be matched

14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
Better transforms vision models

• Choice of better domain highly dependent on
  visual criteria
• Better quality metric impacts the design stage of
  compression algorithm
• better assessment of visual quality more
  accurate prediction of compression artifacts
• Fractal Compression model depended on inaccurate
  quality model (?)

18
Better Transforms

• Lossless
• All information in reconstructed image is
  identical to original image
• Eg., BMP, GIF
• Lossy
• Discard information in original image to achieve
  higher compression rates
• Strategically discard only imperceptible
  information
• Eg. JPEG, TIF, Wavelet compression
• In network-based applications, more focus is
  given to lossy transforms

19
JPEG

• Split image into 8x8 blocks
• Small enough image sections to assume high
  correlation between adjacent pixels
• Apply 8x8 DCT transform to each block
• Shift energy in each block to uppermost entries
• Quantize, run-length encode
• Quantization lossy step, discard information
• RLE takes advantage of sparseness of result

20
8x8 DCT Matrix
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
JPEG Quantization Matrices

• Divide each entry of the image matrix by the
  corresponding entry in the quantization matrix
• Class of matrices built into JPEG standard
• Contained in the JPEG file, with image information

• Flexibility with
• quantization tables (?)

28
MSE Pathology III DCT
Original image
Sinusoidal error (MSE 12.34)
DCT-based error (MSE 320.6)
29
(No Transcript)
30
(No Transcript)
31
JPEG2000 Wavelet Compression

• New JPEG standard wavelet-based
• Wavelet compression studied extensively for years
• JPEG2000 first attempt at standardizing
• WSQ used to compress fingerprints for FBI
• used in place of JPEG, which quickly blurred
  important information
• Similar compression ratios to JPEG, but with
  higher quality

32
Wavelet Transform

• Alternative to Fourier transform
• Localized in time and frequency
• No blocking/windowing artifacts
• Compact support
• Sums of dilations and translations of (mother)
  wavelet function

33
Multi-resolution Analysis

• Complete nested sequence of function spaces Vj,
  with 0 intersection
• Scale-invariance
• f(t) is in Vj iff f(2t) is in Vj1
• Shift-invariance
• f(t) is in Vj iff f(t-k) is in Vj (k integer)

• Shift-invariant Basis
• V0 has an orthonormal basis (scaling function)
• Difference spaces
• Wavelets basis functions for Wjs
• express function in terms of scaling function and
  wavelets

34
DWT Filter Banks

• DWT banded matrix, with filter coefficients on
  diagonals
• Multiply matrix by input signal
• Highpass filter flip coefficients and alternate
  signs
• Discard even entries to construct output signal

35

• DWT separates function into averages and details
• global and local info
• Two filters highpass and lowpass
• lowpass low frequency (averages)
• highpass high frequency (details)

• Highpass filter decimates constant signal (no
  detail info)
• Lowpass filter decimates oscillating signal (no
  global info)
• Result two signals, half length of original
• most info in lowpass signal

36
DWT Image Compression
columns
rows
37
(No Transcript)
38
Wavelets and Images

• Bottom-up
• imperceptible differences separated into details
• L1 norm applied to 1st quadrant only
• Top-down
• 1st quadrant entries give same general image
• L1 norm applied to detail quadrants
• Both give similar results as MSE-based methods

39
Picture Quality Scale (PQS)

• Parameterized error measure
• separate image into different types of error
• calculate weighted sum, with weights determined
  by curve-fitting subjective results
• Five factors
• normalized MSE (regular and thresholded)
• blocking artifacts
• MSE on correlated errors
• Errors near high-contrast image-transitions

40

• Each factor has associated error image
• Designed so that the contributions to the final
  quality rating can be localized
• Better idea of location of error in compression
  assists the algorithm design-time
• Results equivalent to MSE
• Miyahara, Kotani Algazi (JAIST UCDavis)

(Miyahara, Kotani Algazi)
41
Lessons from PQS

• Start with visual system
• base model on observations of subjects
• Localize information about error
• using pictorial distance representation, rather
  than outputting a number to represent quality
• Need more than MSE-based measures
• PQS fails on same pathologies as MSE

42
Fidelity vs. Quality

• Image Fidelity
• Measured in terms of the closeness of an image
  to an original source, or ideal, image
• eg. MSE-measures, PQS
• Image Quality
• Measured in terms of a single images internal
  characteristics
• Depends on the criteria, application-specific
• eg. Medical Imaging

43
Fidelity-based Approach

• Modelled by IPO (Eindhoven)
• Natural image as conveyer of visual information
  about natural world
• quality based on internal properties of image,
  but only on past experiences of subject
• eg. quality of picture of grass depends on its
  ability to conform to subjects expectations of
  the appearance of grass

44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
IPO model

• Pros
• Very nice theoretically
• Clearly-defined notions of quality
• Based on theory of cognitive human vision
• Flexible for application-specific model

• Cons
• Practical to implement?
• Subject-specific definition of quality
• Subjects more accurate at determining relative
  vs. absolute measurement

48
Next-wave HVS-based