Image processing in the compressed domain

1
Image processing in the compressed domain
SSIP 2009
Assist.Eng. Camelia Florea, Technical
University of Cluj-Napoca, ROMANIA
2
Contents

• Brief Overview
• JPEG standard coding
• The two ways to process JPEG compressed images
• DCT Coefficients
• JPEG features space for image segmentation
• A DCT-based approach for detecting patients
  identification information
• Bayesian segmentation of hepatic biopsy color
  images in the JPEG compressed domain
• Image enhancement operations in the compressed
  domain
• Compressed domain implementation of fuzzy
  rule-based contrast enhancement

3
Brief Overview

• Compressed domain image processing algorithms in
  encoded JPEG image domain - provide a powerful
  computational alternative to classical.
• This field is in its beginning - the algorithms
  reported in the literature are mostly based on
  linear arithmetic point operations (addition,
  substraction, multiplication).
• Advantage
• no need to decompress/ recompress the whole image
  prior to processing/after processing.
• we compute less data (after quantization many of
  the DCT coefficients are zero).

4
JPEG standard coding

• The color space of the image is converted (RGB to
  YUV)
• The image is divided into 8x8 blocks
• Values are scaled symmetrical towards 0 (from 0,
  255 to -128, 127)
• Each 8x8 block is processed for compression
• DCT is applied on each block gt obtain the DCT
  coefficients (DC and AC)
• DCT coefficients are quantized - small
  coefficients are quantized to zero
• zig-zag scan of the DCT blocks
• RLE (Run Length Encoding) is performed
• finally - entropy coding

5
The image in the JPEG compressed domain
6
The two ways to process JPEG compressed images

• Having a JPEG compressed image is more
  efficiently to process it, without
• performing decompression, pixel level processing,
  and recompression.
• Processing in the compressed domain is made over
  the RLE vectors.
• RLE vector contains data about the variation and
  mean of luminance/color

7
DCT Coefficients

• There are many local texture features embedded in
  the DCT coefficients, reflecting color and
  texture
• the DC coefficient - the average color in a block
  of pixels,
• the AC coefficients - the variance of luminance
  and chrominance

Two dimensional DCT basis functions (N 8).

8
(No Transcript)
9
JPEG features space for image segmentation

• Image segmentation is the technique of
  partitioning an image into units which are
  homogeneous with respect to one or more
  characteristics.
• This can be done on pixel level for highest
  accuracy,
• but also on JPEG block level, in the compressed
  domain, considering the local color and texture
  information roughly needed, is completely present
  in this representation.

10
A DCT-based approach for detecting patients
identification information

• Addressed problem
• implementing an algorithm for detecting patients
  data from JPEG ultrasound images, applied
  directly on DCT coefficients.
• Advantage
• no need to convert medical images/video back to
  the spatial (uncompressed) domain.
• the algorithm can detect textual information
  using the amount of energy, computed using only
  AC coefficients.
• HIPAA recommends to healthcare providers
• the protection of the confidentiality of their
  patients health data.
• Medical information, regarding both
• patients identification and
• their treatment,
• can be
• transmitted and stored
• and, are susceptible of being accessed by
  unauthorized people.
• Images contains textual data about the patient,
  data that requires special security measures when
  disseminating the images.
• They must be handled by authorized health care
  professionals only.

11
A DCT-based approach for detecting patients
identification information

• It is possible to hide or eliminate the patient
  information without processing the image content
  itself (pixels)
• by using only the DCT coefficients.
• The DCT - is one of the best filters for feature
  extraction in the frequency domain it could be
  used here.
• In an ultrasound medical image the areas with
  very high energy amount
• are the regions containing textual
  information.
• Areas with patients data can then be encrypted,
  blurred or eliminated.
• Having a basic knowledge of the ultrasound
  machine used,
• With the same image acquisition conditions,
• gt we can detects (and hide) only the patient
  identification information in the image and keep
  the medical information.

12
Systems architecture
13
Analyzing the grey level variation

• The blocks energy is computed and analyzed for
  each 88 block.
• (EAC - the average of AC coeff. Energy)
• In ultrasound images are many 88 blocks where
  the local variation of the brightness is small
• the background area, and
• the examination area from ultrasound image.
• gt every such block - do not contain text
  information and, under no circumstance,
  information about patients.
• If the 88 blocks exhibit a large variation of
  the grey levels around the average brightness
  value,
• gt we have areas with sudden changes of
  brightness from black to white
• text, or
• cartesian axes, from the ultrasound image.

14
The data hiding algorithm in JPEG compressed
domain

• Compute the EAC.
• If EAC lt ethd gt data from the original image
  are kept (no processing).
• If EAC ethd gt the block has a significant
  content of details,
• areas of interest - need to be processed for
  data selection
• patients identification information - will be
  protected,
• examination data - no processing.
• where ethd represents the optimal selection
  threshold between
• the uniform blocks, and
• the blocks with a significant number of details.

15
High energy blocks
High energy blocks
Medical image resulted just by keeping only the
blocks with high energy.
Apply selection rules
Post Processing
Energy blocks
Selected data to hide.
16
Bayesian segmentation of hepatic biopsy color
images in the JPEG compressed domain

• Addressed problem
• color image segmentation based on
• the color information, and, also on
• the local texture information,
• for each 88 pixel neighborhoods.
• Advantage
• no need to decompress/recompress the whole image
  prior to processing/after processing.
• we compute less data (after quantization many of
  the DCT coefficients are zero).
• This reduced dimensionally feature space makes
  easier the training and implementation of rather
  complex classifiers,
• as e.g. the Bayesian classifier with class
  probabilities modeled by Gaussian mixtures used
  here.

17
Color in RGB vs. YUV space

• Many image representation spaces can be used in
  segmentation process.
• The YUV representation yields certain advantages
  over RGB
• The YUV is the representation used in the JPEG
  standard,
• The YUV provides a clear separation between the
  luminance representation (Y) and color (U,V),
• The luminance information Y, the color
  information U and V exhibits poor correlation
• The image storage format itself provides the
  information needed for an accurate
  identification/segmentation
• e.g. segmentation of the hepatic biopsies into
  tissue vs. microscopic slide and further, of the
  tissue into healthy tissue vs. hepatic fibrosis.

18
Bayesian classification of pixel block in the
discrete cosine transform domain

• We use the Bayes decision rule to classify a DCT
  block into microscopic slide, healthy tissue or
  fibrosis (features space zig-zag scanned
  quantized DCT coefficients).
• A powerful yet simple model for blocks
  classification is the Multivariate Gaussian
  model
• where
• The Bayes decision rules for minimal cost are the
  following
• Typically in the hepatic biopsy there is no
  obvious reason to assume uneven distribution of
  the tissue vs. microscopic slide, and neither of
  fibrosis vs. healthy tissue, therefore, we
  consider

19
Gaussian parameters estimation

• For the classification we need a-priori knowledge
  of the class statistics
• If is square and singular, then its
  inverse does not exist.
• ? This might be the case when the matrix is
  sparse, as is the case when using DCT quantized
  coefficients.
• In these cases, the computation is based on
  computing singular value decomposition of
  (the base of Moore-Penrose pseudo-inverse).
• Any singular values less than a threshold-value
  are treated as zero (gt0.01).
• The determinant of matrix is the product of
  the diagonal singular values.

20
Bayesian segmentation of hepatic biopsy color
images in the JPEG compressed domain

• The training phase of the algorithm
• The statistical properties of the classes used by
  the two classifiers are determined using
  ground-truth images
• As a result the mean values and the covariance
  matrices (as well as their pseudo-inverse) are
  found for each class.
• The test phase of the algorithm
• Each and every 88 block from the microscopic
  compressed image is considered and the blocks are
  processed for classification.
• The segmentation of an image is performed as a
  2-step classification process
• 1st step - Discrimination between microscopic
  slide and hepatic tissue
• 2nd step - Identify the blocks that exhibit
  fibrosis among the hepatic tissue blocks

21
Discrimination between microscopic slide and
hepatic tissue (1st step)

• 1st step is the discrimination between
  microscopic slide and hepatic tissue (with or
  without fibrosis).
• In this case the luminance information gives
  sufficient information for segmentation, and the
  decision rule is
• with Ydct88 the matrix of the
  DCT coefficients

22
Identify the blocks that exhibit fibrosis among
the hepatic tissue blocks (2nd step)

• The hepatic biopsies are treated with Sirius
  stain ? the coloration of hepatic fibrosis
  appears reddish unlike the healthy tissue, of
  beige color.
• This 2-class Bayesian classification is performed
  at block level, but this time, the Y and V
  components are used to compute the two class
  probabilities
• U is not used since it is a measure of the
  dominance of blue
• Color components and luminance information are
  not correlated ? the joint class probabilities
  are the product of the luminance and color
  probabilities
• where Ydct - the 88 block of the luminance
  DCT coefficients
• Vdct - the 88 block of the
  reddish chrominance DCT coefficients.

23
Experimental results
Classifications results using our algorithm and
pixel level algorithm
Patient DCT algorithm Pixel level algorithm Scores of fibrosis
P1 4.15 4.47 1
P2 5.32 7.12 1
P3 7.05 6.23 1
P4 10.7 11.2 2
P5 14.95 14.6 2
P6 16.71 20.5 3
P7 17.84 21 3
24
Experimental results
False acceptance rate (FAR) and false rejection
rate (FRR), for one patient
  1st classifier 1st classifier 2nd classifier 2nd classifier
  FAR FRR FAR FRR
Average 0.75 0.99 2.14 1.81
Worse case FAR 1.41 0.98 3.72 1.95
Worse case FRR 0.82 1.93 1.74 3.59
25
Image enhancement operations in the compressed
domain

• Mostly linear algorithms developed for the
  compressed domain
• Pointwise image addition/substraction
• Constant addition/substraction to each spatial
  position
• Constant multiplication to each spatial position
• Pointwise image multiplication
• Pixel arithmetic can be used to implement a
  number of operations
• cross-fade between two images or video sequences
• image composition - overlaying a forecaster on a
  weather map
• implementation of fuzzy rule-based contrast
  enhancement

26
Alpha-blending between two images
27
Image composition
28
Compressed domain implementation of fuzzy
rule-based contrast enhancement

• implementing a non-linear operator using
  compressed domain processing fuzzy rule-based
  contrast enhancement,Takagi-Sugeno
• Advantage
• no need to decompress/ recompress the whole image
  prior to processing/after processing.
• for the 88 size blocks processed in the
  compressed domain, the processing implies a
  single comparison of the coefficient with the
  threshold (instead of 64 comparisons needed at
  pixel level).

29
Description of the fuzzy rule-based contrast
enhancement algorithm

• The fuzzy rule base of the Takagi-Sugeno fuzzy
  systems comprises the following 3 rules
• R1 IF lu is Dark THEN lv is Darker R1
  IF lu is Dark THEN lvlvd
• R2 IF lu is Gray THEN lv is Midgray ? R2 IF
  lu is Gray THEN lvlvg
• R3 IF lu is Bright THEN lv is Brigter, R3
  IF lu is Bright THEN lvlvb

Input and output membership functions for fuzzy
rule-based contrast enhancement
For any value at the input of our
Takagi-Sugeno contrast enhancement fuzzy system,
in the output image, the corresponding brightness
is obtained by applying the Takagi-Sugeno
fuzzy inference, as
Where , , denote
the membership degrees of the currently processed
brightness to the input fuzzy sets Dark, Gray
and Bright.
30
The adaptive algorithm for contrast enhancement

• To obtain in the compressed domain the same
  processing results as the one given by the
  pixel-level approach
• - the algorithm must be reformulated as a block
  level processing.
• The nonlinear operations, like the thresholding
  in fuzzy rule-based contrast enhancement
  algorithm, must be carefully addressed.
• The DC coefficient gives the average brightness
  in the block
• - is used as an estimate for selecting the
  processing rule for all the pixels in the blocks
  with small AC energy.
• In this algorithm an adaptive minimal
  decompression is used
• - full decompression is no longer needed,
• - but, decompression is used for the block
  having many details, for an improved accuracy of
  processing.

31
The fuzzy set parameters selection using the DC
histogram in the compressed domain

• A reasonable choice for the thresholds values
  , and would be the minimum, the mean
  and the maximum grey level from the image
  histogram.
• Roughly speaking, if the DC coefficients would be
  the only ones used to reconstruct the pixel level
  representation (without any AC information),
• - they would give an approximation of the
  image,
• - with some block boundary effects and some
  loss of details,
• - but, however still preserving the significant
  visual information.
• Therefore,
• - the histogram built only from the DC
  coefficients will have approximately the same
  shape as the grey level histogram.

Histogram of DC coefficients, and at pixel level
(frog.jpg)
32
Experimental results

• The algorithm is applied only on the
  luminance component.
• However, it can be used to enhance color
  images as well, with no change of the chrominance
  components

Input membership function superimposed on the DC
histogram of the Y component
DC histogram of the Y component after fuzzy
contrast enhancement
33
Results for different values ethd
Image ethd EffBlocks MSE
frog.jpg 3 13.75 1.79
woman.jpg 7 4.42 1.44
Lena.jpg 10 9.79 0.014
keyboard.jpg 3 7.29 1.94

• The Mean Squared Error (MSE ) between the
  pixel level processed images and the images
  processed with our algorithm was used as quality
  performance measure.
• The efficiency (EffBlocks) of the proposed
  method formulated above for the compressed
  domain, is evaluated by examining the number of
  blocks processed at pixel level as percent from
  the total number of 88 pixels blocks in the
  image.

34
SVM in the compressed domain