Gray Image Coloring Using Texture Similarity Measures- Thesis Discussion - PowerPoint PPT Presentation

View by Category
About This Presentation
Title:

Gray Image Coloring Using Texture Similarity Measures- Thesis Discussion

Description:

Gray Image Coloring Using Texture Similarity Measures- Master Thesis Discussion Faculty of Computers and Information - Menofia University - Egypt. – PowerPoint PPT presentation

Number of Views:1775

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Gray Image Coloring Using Texture Similarity Measures- Thesis Discussion


1
Gray Image Coloring Using Texture Similarity
Measures
by E. Noura Abd El-Moez Semary Thesis Submitted
in accordance with the requirements of The
University of Monofiya for the degree of Master
of Computers and Information ( Information
Technology )
2
Gray Image Coloring Using Texture Similarity
Measures
Thesis summary on
Supervised by Prof. Mohiy .M.Hadhoud Prof. Nabil .A.Ismail Dr.Waiel .S. Al-Kilani
  • Presented by
  • E. Noura Abd El-Moez Semary
  • For Master degree in Computers and Information
  • IT department, Faculty of Computers and
    information,
  • Menofia University

3
????? ????? ???????? ???????? ?????? ????? ???????
???? ????? ??????
?. ???? ???? ???????? ?.?. ???? ??? ?????? ??????? ?.?. ??? ???? ????? ??? ?????
  • ???? ??
  • ?. ???? ??? ????? ??????? ????
  • ?????? ??? ???? ????????? ?? ???????? ??????????
  • ??? ????????? ????????? - ???? ???????? ?
    ????????? - ????? ????????

4
  • Outlines
  • Introduction
  • Automatic coloring in the literature
  • TRICS Texture Recognition based Image Coloring
    System
  • Results
  • Conclusion
  • Future work
  • Outlines
  • Introduction
  • Automatic coloring in the literature
  • TRICS Texture Recognition based Image Coloring
    System
  • Results
  • Conclusion
  • Future work

5
Introduction Gray image principles
  • Outlines
  • Introduction
  • Automatic coloring in the literature
  • TRICS Texture Recognition based Image Coloring
    System
  • Results
  • Conclusion
  • Future work

6
Introduction Gray image principles
Total size (byte)
Actually No. used colors
Possible No. colors
Image Size (pixel)
Image
60.0 kb
15,998
16,777,216
170 120 px
21.2 kb
246
256
170 120 px
7
Introduction Coloring Problem
  • There are two definitions to describe the gray
    value as an equation of the three basic
    components of RGB color model (red, green, blue)
  • 1 Intensity (most common used)
  • Gray (Red Green Blue) /3
  • 2 Luminance (NTSC standard for luminance)
  • Gray (0.299 Red) (0.587 Green) (0.114
    Blue)

RGB Color R, G, B values Gray value Gray Color
100, 150, 87 128
147, 87, 149 128
149, 147, 87 128
THERE IS NO MATHEMATICAL FORMULA TO CONVERT FROM
GRAY TO RGB COLOR.
8
Introduction Coloring Problem
9
Introduction Coloring Types
  • 1 . Hand coloring
  • Adobe Photoshop and Paintshop Pro
  • Layers
  • Changing Hue
  • BlackMagic, photo colorization software, version
    2.8, 2003

10
Introduction Coloring Types
  • 2 . Semi automatic coloring
  • Pseudocoloring is a common example for semi
    automatic coloring technique

11
Introduction Coloring Types
  • Outlines
  • Introduction
  • Automatic coloring in the literature
  • TRICS Texture Recognition based Image Coloring
    System
  • Results
  • Conclusion
  • Future work
  • 3 . Automatic coloring
  • Transformational coloring
  • Matched image coloring
  • User selected coloring

12
Automatic coloring in the literature1.
Transformational Coloring
  • Outlines
  • Introduction
  • Automatic coloring in the literature
  • TRICS Texture Recognition based Image Coloring
    System
  • Results
  • Conclusion
  • Future work
  • A transformation function Tk is applied on the
    intensity value of each pixel Ig(i,j) resulting
    in the chromatic value Ick(i,j) for channel k

13
Automatic coloring in the literature1.
Transformational Coloring
  • Al-Gindy et al system.
  • Results have unnatural look

A. N. Al-Gindy, H. Al Ahmad, R. A. Abd
Alhameed, M. S. Abou Naaj and P. S. Excell
Frequency Domain Technique For Colouring Gray
Level Images 2004 found in www.abhath.org/html/mo
dules/pnAbhath/download.php?fid32
14
Automatic coloring in the literature2. Matched
image coloring
  • The most similar pixel color is transferred to
    the corresponding gray one by the color transfer
    technique proposed by E.Reinhard

Reinhard, E. Ashikhmin, M., Gooch B. And
Shirley, P., Color Transfer between Images, IEEE
Computer Graphics and Applications,
September/October 2001, 34-40
15
Automatic coloring in the literature2. Matched
image coloring
  • Global matching procedure of T. Welsh et al
  • Local color transfer of Y. Tai et al.
  • All these algorithms fail, when different colored
    regions have similar intensities

T. Welsh, M. Ashikhmin, K. Mueller.
Transferring color to greyscale images. In
Proceedings of the 29th Annual Conference on
Computer Graphics and interactive Techniques, pp
277280, 2002 Y. Tai, J. Jia, C. Tang Local
Color Transfer via Probabilistic Segmentation by
Expectation- maximization, 2005 IEEE Computer
Society Conference on Computer Vision and Pattern
Recognition (CVPR'05), Volume 1, pp. 747-754, 2005
16
Automatic coloring in the literature2. Matched
image coloring
  • Welsh et al proposed also another technique to
    improve the coloring results when the matching
    results are not satisfying. It was achieved by
    asking users to identify and associate small
    rectangles, called swatches in both the source
    and destination images to indicate how certain
    key colors should be transferred

17
Automatic coloring in the literature3. User
selection coloring
  • Outlines
  • Introduction
  • Automatic coloring in the literature
  • TRICS Texture Recognition based Image Coloring
    System
  • Results
  • Conclusion
  • Future work

  • User selection coloring gives high quality colors
  • User dependent color quality
  • Time-consuming
  • Colorization must be fully recomputed for any
    slight change in the initial marked pixels

A. Levin, D. Lischinski, Y. Weiss.
Colorization using optimization. ACM
Transactions on Graphics, Volume 23, Issue 3,
pp.689694, 2004
18
TRICS System Research Objectives
  • Outlines
  • Introduction
  • Automatic coloring in the literature
  • TRICS Texture Recognition based Image Coloring
    System
  • Results
  • Conclusion
  • Future work
  • To simulate the human vision in coloring process
  • To be fully automatic coloring system
  • To spend so little execution time as possible as
    a basic requirement for video coloring.


19
TRICS SystemStructure
Gray image
A
Segmentation
B
Classification
C
Coloring
Colored image
20
TRICS SystemStructure 1. Segmentation Stage
  • Feature extraction (Pixel based )
  • pixel position
  • pixel intensity
  • texture features
  • wavelets coefficients
  • Laws kernels coefficients.

21
TRICS SystemStructure 1. Segmentation Stage
  • 1. Wavelets coefficients
  • Quarter the image size.
  • Up sampling
  • Upper level construction

22
TRICS SystemStructure 1. Segmentation Stage
23
TRICS SystemStructure 1. Segmentation Stage
  • 2. Laws Kernels
  • Level L5 1 4 6 4 1
  • Edge E5 -1 2 0 2 1
  • Spot S5 -1 0 2 0 1
  • Wave W5 -1 2 0 2 1
  • Ripple R5 1 4 6 4 1
  • L5S5

-1 -4 -6 -4 -1
0 0 0 0 0
2 8 12 8 2
0 0 0 0 0
-1 -4 -6 -4 -1
24
TRICS SystemStructure 1. Segmentation Stage
  • Segmentation technique
  • Mean Shift
  • K-mean (Fast k-mean)
  • Adaptive Fast k-mean

D. Comaniciu and P. Meer. Mean shift A robust
approach toward feature space analysis. PAMI,
24(5)603619, May 2002 C.Elkan, Using the
triangle inequality to accelerate k-Means. In
Proc. of ICML 2003. pp 147--153
25
Mean Shift
Region of interest
Center of mass
Mean Shift vector
Objective Find the densest region
26
Mean Shift
Region of interest
Center of mass
Mean Shift vector
Objective Find the densest region
27
Mean Shift
Region of interest
Center of mass
Mean Shift vector
Objective Find the densest region
28
Mean Shift
Region of interest
Center of mass
Mean Shift vector
Objective Find the densest region
29
Mean Shift
Region of interest
Center of mass
Mean Shift vector
Objective Find the densest region
30
Mean Shift
Region of interest
Center of mass
Mean Shift vector
Objective Find the densest region
31
Mean Shift
Region of interest
Center of mass
Objective Find the densest region
32
TRICS SystemStructure 1. Segmentation Stage
  • 1. Mean Shift
  • So slow
  • Many parameters

(170256) - hs16,hr16,m500 - Time 0 34 15 - classes 9 - hs8,hr8,hw4,m500 - Time 0 39 54 - classes 7
33
Fast (Accelerated) K-mean
  • Lemma 1 Let p be a point and let c1 and c2 be
    centers.
  • If E(c1,c2) 2E(p,c1) then
  • E(p,c2) E(p,c1).
  • Lemma 2 Let p be a point and let c1 and c2 be
    centers. Then
  • E(p,c2) max0,E(p,c1) E(c1,c2)

C.Elkan, Using the triangle inequality to
accelerate k-Means. In Proceedings of the 20th
ICML, Washington DC, 2003. pp 147--153
34
TRICS SystemStructure 1. Segmentation Stage
  • 2. Fast K-mean
  • with spatial features
  • structured segmentation
  • Increase no. clusters

35
TRICS SystemStructure 1. Segmentation Stage
  • 2. Fast K-mean
  • without spatial features
  • scattered regions of same cluster
  • disjoint region separation

36
TRICS SystemStructure 1. Segmentation Stage
  • 2. Fast K-mean
  • Small regions (noise)
  • Small regions elimination

37
TRICS SystemStructure 1. Segmentation Stage
  • 3. Adaptive Fast K-mean
  • Clusters number generation
  • Minimum region size estimation

Fully automatic segmentation technique
38
TRICS SystemStructure 1. Segmentation Stage
  • a. Clusters number generation
  • CEC Combined Estimation Criterion
  • If the VRC index, for k clusters, is smaller than
    98 of the VRC index, for k-1 clusters, the CEC is
    not satisfied.
  • If the VRC index, for k clusters, is larger than
    102 of the VRC index for k-1 clusters, or if k1,
    the CEC is satisfied.
  • If the VRC index, for clusters, is smaller than
    102 but larger than 98 of the VRC index for
    clusters, the CEC is satisfied only if TSS for
    k-1 clusters is smaller than 70 of TSS for k
    clusters.


D.Charalampidis, T.Kasparis, Wavelet-Based
Rotational Invariant Roughness Features for
Texture Classification and Segmentation. IEEE
Transactions on Image Processing.Vol.11.No.8
August 2002
39
TRICS SystemStructure 1. Segmentation Stage
Start k1
Gray Image
Fast k-mean
Segmented Image
Calculate CEC
kk1
Yes
CEC Satisfied?
No
Stop Clusters number k
40
TRICS SystemStructure 1. Segmentation Stage
  • b. Minimum region size estimation
  • Split the disjoint regions.
  • Count all regions size.
  • Sort regions size and calculate the step between
    them.
  • Select the regions size of step more than the
    largest image dimension.
  • Consider the minimum region size.


41
TRICS SystemStructure 1. Segmentation Stage
Original gray wavelets Laws
Features
Image
Wavelets
size 170256 M500, EM 324
Laws
A-time adaptive fast k-mean time, F-time
fixed k fast k-mean time, T-time traditional
k-mean , E-time elimination time , EM-time
Elimination time with estimating minimum size
region
42
TRICS SystemStructure Database set1


  • The training set consists of 32 classes of
    Brodatz texture database
  • Each image has a size of 256 256. Each image
    was mirrored horizontally and vertically to
    produce a 512 512 image.
  • The image is split into 16 images of size 128
    128.







256 256 512 512 16 128 128
43
TRICS SystemStructure Database set2
  • The training set consists of 9 classes cloud,
    sky, sea, sand, tree, grass, stone, water, and
    wood
  • Each class has number of samples from 12 to 25
    samples.
  • These samples are taken from real natural images
    as random 64x64 rectangles.







44
TRICS SystemStructure Database
  • Database record
  • Sample
  • Class (level1,level2)
  • 58 Features (6 Moment statistics, 4 Co-ocurance
    measures, 3 Tamura, and 15 wavelets mean, 15
    wavelets variance , 15 wavelets energy for five
    levels wavelets decomposition)
  • Hue







45
TRICS SystemStructure 2. Classification Stage
  • Feature extraction (Region based)
  • Rectangular region
  • Maximum rectangle
  • 64 x 64 rectangle
  • Arbitrary shape
  • Padding rectangle
  • Ying Liu, Xiaofang Zhou, Wei-Ying Ma,
    Extracting Texture Features
  • from Arbitrary-shaped Regions for Image
    Retrieval. 2004 IEEE
  • International Conference on Multimedia and Expo.,
    Taipei, Jun. 2004

46
TRICS SystemStructure 2. Classification Stage
  • Feature extraction (Region based)
  • Region based features
  • GLCM measures (Energy, Entropy, Inertia,
    Homogeneity )
  • Tamura (Coarseness, Contrast , Directionality)
  • Wavelets coefficients for 5 levels
  • Mean and variance
  • Energy

P.Howarth, S.Ruger, Evaluation of texture
features for content-based image retrieval. In
proceedings of the International Conference on
Image and Video Retrieval, Springer-Verlag (2004)
326324 O. Commowick C. Lenglet C.
Louchet, Wavelet-Based Texture Classification
and Retrieval 2003 found in http//www.tsi.enst.f
r/tsi/enseignement/ressources/mti/classif-textures
/ Eka Aulia, Hierarchical Indexing For
Region Based Image Retrieval, Master thesis of
Science in Industrial Engineering, Louisiana
State University and Agricultural and Mechanical
College, May 2005
47
TRICS SystemStructure 2. Classification Stage
  • GLCM and Tamura
  • Scale variant features, not suitable for natural
    textures

48
TRICS SystemStructure 2. Classification Stage
  • Wavelets mean and variance
  • Values were very scattered and the results were
    not accurate for most cases.
  • Wavelets energies
  • Classification accuracy of 92 using
    leave-one-out (each (sub) image is classified
    one by one so that other (sub) images serve as
    the training data) method

49
TRICS SystemStructure 2. Classification Stage
  • Classification technique
  • KNN classifier with (k1,k5,k10,k20)
  • Distance Metric is L2 Euclidean distance
  • k5 gives accuracy up to 94 using N-fold (the
    collection of (sub) images is divided into N
    disjoint sets, of which N-1 serve as training
    data in turn and the Nth set is used for testing)

50
TRICS SystemStructure 2. Classification Stage
51
TRICS SystemStructure 3. Coloring Stage
  • Color model conversion
  • HSV/HSB color model
  • HSI/HLS color model


Change in Saturation Hue 0, Luminance0.5 Change in Brightness Hue 0, Saturation 1 Change in Hue Sat1, Luminance1

Change in Saturation Hue 0, Luminance0.5 Change in Luminance Hue 0, Saturation 1 Change in Hue Sat1, Luminance1
52
TRICS SystemStructure 3. Coloring Stage
  • HSV HSL Channels

53
TRICS SystemStructure 3. Coloring Stage
  • Setting Channels values
  • Brightness
  • The gray image itself
  • Hue
  • One hue value for each texture
  • Saturation
  • HSV 1- brightness HSI 0.5(1-lightness)

54
TRICS SystemStructure 3. Coloring Stage
55
TRICS SystemStructure 3. Coloring Stage
  • Outlines
  • Introduction
  • Automatic coloring in the literature
  • TRICS Texture Recognition based Image Coloring
    System
  • Results
  • Conclusion
  • Future work

56
Results and Conclusion Results
  • Perfect Results
  • Outlines
  • Introduction
  • Automatic coloring in the literature
  • TRICS Texture Recognition based Image Coloring
    System
  • Results
  • Conclusion
  • Future work

57
Results and Conclusion Results
  • Perfect Results

58
Results and Conclusion Results
  • PANN Database

59
Results and Conclusion Results
  • Misclassified results
  • 2 classification levels
  • if the KNN results in 5 classes grass, sea,
    water, grass, sea
  • The traditional solution is the class of the
    grass.
  • The 2 levels classification solution is sea.
  • (Sea and water), (trees and grass), (sky and
    clouds) and (wood and stone) are considered as
    one class in level one.

60
Results and Conclusion Comparisons
HSV/HSB HIS/HLS
61
Results and Conclusion Comparisons
  • Local color transfer
  • Outlines
  • Introduction
  • Automatic coloring in the literature
  • TRICS Texture Recognition based Image Coloring
    System
  • Results
  • Conclusion
  • Future work
  • Global Image Matching

62
Results and Conclusion Conclusion
  • We proposed a new computer coloring technique
    that simulates the human vision in this area.
  • The proposed coloring system is contributed for
    coloring gray natural scenes.
  • The execution time of TRICS is minimized using
    Fast k-mean segmentation technique and the
    results are enhanced by splitting the disjoint
    regions and by eliminating small regions.
  • Outlines
  • Introduction
  • Automatic coloring in the literature
  • TRICS Texture Recognition based Image Coloring
    System
  • Results
  • Conclusion
  • Future work

63
Results and Conclusion Conclusion
  • Clusters number generation algorithm and the
    minimum region size estimation algorithm increase
    the professionalism of the system but also
    increases the time of the execution. And by using
    both of them TRICS becomes a fully unsupervised
    intelligent recognition based coloring system.
  • HSV coloring model is very suitable for our
    system and the coloring results have good natural
    look.

64
Results and Conclusion Conclusion
  • Outlines
  • Introduction
  • Automatic coloring in the literature
  • TRICS Texture Recognition based Image Coloring
    System
  • Results
  • Conclusion
  • Future work
  • We consider our proposed system structure as an
    abstract structure for building any more
    intelligent coloring systems for any other types
    of images
  • Our proposed system results perform the other
    coloring systems.

65
Future work
Gray image
A
  • Outlines
  • Introduction
  • Automatic coloring in the literature
  • TRICS Texture Recognition based Image Coloring
    System
  • Results
  • Conclusion
  • Future work

Segmentation
Features extraction (Joint, wavelets, laws,)
Segmentation (Mean Shift, K-Mean, FCM,..)
B
Classification
Features extraction (Co-occurrence, Tamura,
Wavelets energies)
Classification (K-NN classifier,..)
C
Coloring
Convert image to HSV channels
Convert to RGB
Set Hue, Saturation, and Brightness
Colored image
66
Future work
67
Future work
  • Segmentation and classification stages are
    research areas and any improvement will increase
    the accuracy of the system.
  • Using different types of features and training
    set enables the system for coloring images like
    manmade images, indoors, and people photos .

68
List Of Publications
  • Noura A.Semary, Mohiy M. Hadhoud, W. S.
    El-Kilani, and Nabil A. Ismail, Texture
    Recognition Based Gray Image Coloring, The 24th
    National Radio Science Conference (NRSC2007), pp.
    C22, March 13-15, 2007, Faculty of Engineering,
    Ain-Shams Univ., Egypt.

69
  • Thanks
  • ? ????? ??? ???? ????? ??? ????????
About PowerShow.com