IMAGE QUALITY MEASURES AND THEIR PERFORMANCE

1

• IMAGE QUALITY MEASURES AND THEIR PERFORMANCE
• Ahmet M. Eskicioglu and Paul S. Fisher
• IEEE Transactions on Communications
• Vol. 43. No.12, December 1995

2
INTRODUCTION

• Subjective evaluation is tedious and time
  consuming.
• There is a great deal of interest in developing
  an objective measure.
• Such a measure is needed not only to judge the
  quality of images obtained by a particular
  algorithm, but also for quality judgment across
  various algorithms.
• It is known that the Mean Square Error (MSE), the
  most common objective criterion, or its variants
  do not correlate well with subjective evaluation.
• The HVS is too complex to fully understand with
  present psychophysical means.
• However, the incorporation of even a simplified
  model into objective measures reportedly leads to
  a better correlation with the response of the
  human observers.

3
BIVARIATE IMAGE QUALITY MEASURES
4
IMAGE COMPRESSION TECHNIQUES
5
TEST IMAGES
6
DEFINITION OF SPATIAL FREQUENCY
Row_Freq
Column_Freq
Spatial frequency
7
NILLS DFT-BASED HVS MODEL
for r lt 7
for r ? 7
B number of subimage blocks K normalization
factor such as total energy H(r) response of
HVS M,N number of DFT coefficients Wi subimage
i weighting factor
8
SUBJECTIVE EVALUATION

• Each test image was compressed using 7 ratios
  101 to 701.
• Photographic samples of the degraded images were
  evaluated by 10 observers in an office
  environment.
• Graduate students and faculty members
• They were asked to rank the images in 2 ways
• Within each technique
• Between the 4 techniques
• The mean ranking of the observers was computed by
• where
• sk the score corresponding to the kth ranking
• nk the number of observers with this ranking
• 10 the number of grades in the scale

9
TABLE 4A CORRELATION COEFFICIENT FOR EACH
TECHNIQUE - LENA
Best
10
TABLE 4A CORRELATION COEFFICIENT FOR EACH
TECHNIQUE - GILBERT
Best
11
TABLE 4A CORRELATION COEFFICIENT FOR EACH
TECHNIQUE - FINGERPRINT
Best
12
TABLE 4B CORRELATION COEFFICIENT ACROSS
TECHNIQUES - LENA
13
TABLE 4B CORRELATION COEFFICIENT ACROSS
TECHNIQUES - GILBERT
14
TABLE 4B CORRELATION COEFFICIENT ACROSS
TECHNIQUES - FINGERPRINT
15
NUMERICAL MEASURE RESULTS - I

• The Pearson correlation coefficient
• has values between -1 and 1.
• If the value is closer to -1 or 1, the
  correlation is better.
• Table 4 shows the correlations between objective
  and subjective results.
• The correlations in Part A indicate that the
  objective measures can be put in 3 groups
  according to their performance
• Group I AD, SC
• Group II NK,CQ,LMSE, MD
• Group III WD,PMSE,IF,NAE,NMSE, Lp.
• Group I The measures in this group cannot be
  used reliably with all techniques.
• Group II The measures in this group are
  consistent but they have a poor correlation with
  the observers response for some of the
  techniques.
• Group III NMSE(HVS) is the best for all the
  test images.

16
NUMERICAL MEASURE RESULTS - II

• Part B of Table 4
• Rather disappointing and the information that can
  be extracted is limited.
• As the compression ratio is increased, the
  performance of the measures become much poorer.
• This observation should not be surprising.
• Different techniques introduce different types of
  degradation.
• Since all the metrics combine the pixel
  differences into a single value, one cannot
  expect to know much about the experience of the
  human observer.

17
TABLE 5 A SUBSET OF NMSE (HVS)
best
best
18
GRAPHICAL MEASURES HISTOGRAM
A histogram of the compression error is obtained
by plotting the number of times a specific value
occurs in the difference image versus the
value. Each histogram typically looks like a
Gaussian curve. The more it resembles a spike at
x 0, the greater the fidelity of the
decompressed image.
These histograms were obtained using the absolute
values of the pixels. Normally, the difference
may be a negative or positive value.
Figure 2. Histograms of difference images for
seven compression ratios
(Compression technique JPEG).
19
GRAPHICAL MEASURES HISTOGRAM
Figure 3. Histograms of difference images for
four compression techniques
(Compression ratio 69l)
20
GRAPHICAL MEASURES HOSAKA PLOTS
dS16
21
GRAPHICAL MEASURES HOSAKA PLOTS
Figure 4. Hosaka plots
22
CONCLUSIONS

• The usefulness of a number of objective image
  quality measures are presented.
• A group of objective measures can be reliably
  used to specify the magnitude of degradation in
  decompressed images for a given compression
  technique.
• No objective image quality measure is able to
  produce meaningful results for an evaluation
  across different compression techniques.
• Histograms indicate the degradation as the
  compression ratio is increased.
• Hosaka plots
• Better for blockiness
• No good for blurriness