Title: SegmentationBased MotionCompensated Video Coding Using Morphological Filters 1997
1Segmentation-Based Motion-Compensated Video
Coding Using Morphological Filters (1997)
- Paper by
- Demin Wang
- Claude Labit
- Joseph Ronsin
- Presentation by
- Jayson Smith
2Overview
- The paper presents a system for encoding video
The paper presents a system for encoding video
3Morphological FiltersReview
- Morphology operates on pixels as a set.
- The dilation and erosion operations are two
fundamental operations which use a structuring
element to create a new set from an existing set.
Fig Gonzalas
4Morphological FiltersReview (cont)
Opening A by B is the erosion of A by B, followed
by the dilation of the result by B. Closing A by
B is the dilation of A by B, followed by the
erosion of the result by B.
Fig Gonzalas
5Morphological FiltersReview (cont)
Fig Vincent 6
2D morphological filters for binary images can be
extended to grayscale images by treating the
grayscale as stacks of binary sets.
6Morphological FiltersReview (cont)
- Reconstruction is an iteration of the following
operations - Dilate the marker set
- Mask (logical AND) with the reference set.
- Process continues until stable.
Fig Vincent 6
7Morphological FiltersReview (cont)
Fig Vincent 6
Although the prior reconstruction example refers
to binary images, it can be extended to grayscale
as shown above. Here, the reference f is
reconstruted from the marker g. This is called
reconstruction by dilation. Swapping the meanings
of f and g and then eroding the marker will
produce reconstruction by erosion.
8Region Simplification
- There are two key problems in using morphological
filters for images - Small regions with high contrast must be
preserved. - There is a tradeoff in structuring elements
large ones will eliminate important features,
while small ones will not simplify the image. - A hierarchy of structuring elements helps
alleviate these problems.
9Region Simplification
- Spatial Simplification algorithm
- f(r) frame to be encoded, with r(x,y)
- ?(f) open/closing by reconstruction of f(r)
with largest structuring element B0 (12x12
square) - Residual of f(r)- ?(f) is formed, split into
light and dark - Residuals are thresholded to T (8 in experiments)
10Region Simplification
- Spatial Simplification algorithm (continued)
- Perform opening by reconstruction on each
residual - ?B1(gi) is the opening of gi with B1 (6x6 square)
and ?(rec) (X,Y) is the reconstruction by
dilation of X under reference Y. - f1 ?(f) C1 C2, restoring to ?(f) the
important high-constrast areas that were lost in
filtering. - Subtract C1 and C2 from G1 and G2, and re-apply
thresholding and reconstruction using B2 (3x3
square). Re-apply them to f1.
11Region Simplification
- Motion-Preserving Simplification
- Moving components between two frames appear as
linear structures on component edges when taking
the difference between the frames. - Algorithm
- This is the pointwise maximum, and the Bi are
length-five linear structuring elements
horizontal, vertical, and both diagonals. - f2 is then thresholded by T to remove inter-frame
noise (luminous disturbance) - Create these images
12Region Simplification
Motion-Preserving Simplification algorithm
(continued)
f3 recovers moving dark, f4 removes dark
13Image Segmentation
Four steps region growing, region refinement,
region merging, small region elimination
14Image Segmentation
- Region growing flat regions of f4 larger than
3x3 are seeded. Neighboring pixels are added to
the region if their gray levels are within a low
threshold. Threshold is increased and process is
iterated until all pixels are assigned. - Region refinement the regions defined above are
checked against their gray level means, and
considered homogenous if all pixels deviate from
the mean by ltT. If not, the regions are split
until all regions are homogenous.
15Image Segmentation
- Region merging the edges around region
boundaries are checked if the contrast is less
than T/2, the regions are merged. - Small region elimination regions smaller than
0.4 of the total image area are eliminated.
16Region Merging by Motion
Pairs of regions are merged if their compensation
errors are similar.
17DFD Segmentation
The displaced frame difference (DFD) is the
residual of the motion-compensated image the
error left-over once motion has been encoded.
Generally, it has some linear structures and
small spots on a background of 0s.
18DFD Segmentation
- Algorithm
- Form positive and negative indicator images with
thresholding - Open each image with the 4 linear pieces (length
5), then reconstruct the indicator image - This should remove small spots, but could leave
small holes. The indicator is closed to remove
these holes using a 3x3 structuring element. - Segment each region by the gray-level means,
forming first reconstruction of D(r).
19DFD Segmentation
- Algorithm (continued)
- The reconstruction error image is iteratively
segmented in the same fashion until all regions
are found. - Then, the regions are merged regions smaller
than 0.04 of the size are merged into their most
similar regions, and adjacent regions with gray
level mean differences below some threshold are
merged.
20Conclusions
- Encoding method shows segmentation-based video
compression, as opposed to the more ubiquitous
texture-based. - Method relies heavily on morphological filters.
- The encoding scheme achieves similar rates to
that of texture-based encoders - (Foreman clip has bit rate of 256kbps, SNR of
33.12dB in MPEG4, compared with 207kbps, SNR of
30.11dB in this paper, according to CIF 30fps
clip in Liang) - Resources
- Digital Image Processing 2nd Ed., R. Gonzalez and
R. Woods, 2001 - Morphological Grayscale Reconstruction..., L.
Vincent, 1993 - Methods and Needs for Transcoding MPEG-4..., Y.
Liang, 2002