Title: Gray Image Coloring Using Texture Similarity Measures- Thesis Discussion
1Gray Image Coloring Using Texture Similarity
Measures
by E. Noura Abd El-Moez Semary Thesis Submitted
in accordance with the requirements of The
University of Monofiya for the degree of Master
of Computers and Information ( Information
Technology )
2Gray Image Coloring Using Texture Similarity
Measures
Thesis summary on
Supervised by Prof. Mohiy .M.Hadhoud Prof. Nabil .A.Ismail Dr.Waiel .S. Al-Kilani
- Presented by
- E. Noura Abd El-Moez Semary
- For Master degree in Computers and Information
- IT department, Faculty of Computers and
information, - Menofia University
3????? ????? ???????? ???????? ?????? ????? ???????
???? ????? ??????
?. ???? ???? ???????? ?.?. ???? ??? ?????? ??????? ?.?. ??? ???? ????? ??? ?????
- ???? ??
- ?. ???? ??? ????? ??????? ????
- ?????? ??? ???? ????????? ?? ???????? ??????????
- ??? ????????? ????????? - ???? ???????? ?
????????? - ????? ????????
4- Outlines
- Introduction
- Automatic coloring in the literature
- TRICS Texture Recognition based Image Coloring
System - Results
- Conclusion
- Future work
- Outlines
- Introduction
- Automatic coloring in the literature
- TRICS Texture Recognition based Image Coloring
System - Results
- Conclusion
- Future work
5Introduction Gray image principles
- Outlines
- Introduction
- Automatic coloring in the literature
- TRICS Texture Recognition based Image Coloring
System - Results
- Conclusion
- Future work
6Introduction Gray image principles
Total size (byte)
Actually No. used colors
Possible No. colors
Image Size (pixel)
Image
60.0 kb
15,998
16,777,216
170 120 px
21.2 kb
246
256
170 120 px
7Introduction Coloring Problem
- There are two definitions to describe the gray
value as an equation of the three basic
components of RGB color model (red, green, blue) - 1 Intensity (most common used)
- Gray (Red Green Blue) /3
- 2 Luminance (NTSC standard for luminance)
- Gray (0.299 Red) (0.587 Green) (0.114
Blue)
RGB Color R, G, B values Gray value Gray Color
100, 150, 87 128
147, 87, 149 128
149, 147, 87 128
THERE IS NO MATHEMATICAL FORMULA TO CONVERT FROM
GRAY TO RGB COLOR.
8Introduction Coloring Problem
9Introduction Coloring Types
- 1 . Hand coloring
- Adobe Photoshop and Paintshop Pro
- Layers
- Changing Hue
- BlackMagic, photo colorization software, version
2.8, 2003
10Introduction Coloring Types
- 2 . Semi automatic coloring
- Pseudocoloring is a common example for semi
automatic coloring technique
11Introduction Coloring Types
- Outlines
- Introduction
- Automatic coloring in the literature
- TRICS Texture Recognition based Image Coloring
System - Results
- Conclusion
- Future work
- 3 . Automatic coloring
- Transformational coloring
- Matched image coloring
- User selected coloring
12Automatic coloring in the literature1.
Transformational Coloring
- Outlines
- Introduction
- Automatic coloring in the literature
- TRICS Texture Recognition based Image Coloring
System - Results
- Conclusion
- Future work
- A transformation function Tk is applied on the
intensity value of each pixel Ig(i,j) resulting
in the chromatic value Ick(i,j) for channel k
13Automatic coloring in the literature1.
Transformational Coloring
- Al-Gindy et al system.
- Results have unnatural look
A. N. Al-Gindy, H. Al Ahmad, R. A. Abd
Alhameed, M. S. Abou Naaj and P. S. Excell
Frequency Domain Technique For Colouring Gray
Level Images 2004 found in www.abhath.org/html/mo
dules/pnAbhath/download.php?fid32
14Automatic coloring in the literature2. Matched
image coloring
- The most similar pixel color is transferred to
the corresponding gray one by the color transfer
technique proposed by E.Reinhard
Reinhard, E. Ashikhmin, M., Gooch B. And
Shirley, P., Color Transfer between Images, IEEE
Computer Graphics and Applications,
September/October 2001, 34-40
15Automatic coloring in the literature2. Matched
image coloring
- Global matching procedure of T. Welsh et al
- Local color transfer of Y. Tai et al.
- All these algorithms fail, when different colored
regions have similar intensities
T. Welsh, M. Ashikhmin, K. Mueller.
Transferring color to greyscale images. In
Proceedings of the 29th Annual Conference on
Computer Graphics and interactive Techniques, pp
277280, 2002 Y. Tai, J. Jia, C. Tang Local
Color Transfer via Probabilistic Segmentation by
Expectation- maximization, 2005 IEEE Computer
Society Conference on Computer Vision and Pattern
Recognition (CVPR'05), Volume 1, pp. 747-754, 2005
16Automatic coloring in the literature2. Matched
image coloring
- Welsh et al proposed also another technique to
improve the coloring results when the matching
results are not satisfying. It was achieved by
asking users to identify and associate small
rectangles, called swatches in both the source
and destination images to indicate how certain
key colors should be transferred
17Automatic coloring in the literature3. User
selection coloring
- Outlines
- Introduction
- Automatic coloring in the literature
- TRICS Texture Recognition based Image Coloring
System - Results
- Conclusion
- Future work
- User selection coloring gives high quality colors
- User dependent color quality
- Time-consuming
- Colorization must be fully recomputed for any
slight change in the initial marked pixels
A. Levin, D. Lischinski, Y. Weiss.
Colorization using optimization. ACM
Transactions on Graphics, Volume 23, Issue 3,
pp.689694, 2004
18TRICS System Research Objectives
- Outlines
- Introduction
- Automatic coloring in the literature
- TRICS Texture Recognition based Image Coloring
System - Results
- Conclusion
- Future work
- To simulate the human vision in coloring process
- To be fully automatic coloring system
- To spend so little execution time as possible as
a basic requirement for video coloring.
19TRICS SystemStructure
Gray image
A
Segmentation
B
Classification
C
Coloring
Colored image
20TRICS SystemStructure 1. Segmentation Stage
- Feature extraction (Pixel based )
- pixel position
- pixel intensity
- texture features
- wavelets coefficients
- Laws kernels coefficients.
21TRICS SystemStructure 1. Segmentation Stage
- 1. Wavelets coefficients
- Quarter the image size.
- Up sampling
- Upper level construction
-
22TRICS SystemStructure 1. Segmentation Stage
23TRICS SystemStructure 1. Segmentation Stage
- 2. Laws Kernels
- Level L5 1 4 6 4 1
- Edge E5 -1 2 0 2 1
- Spot S5 -1 0 2 0 1
- Wave W5 -1 2 0 2 1
- Ripple R5 1 4 6 4 1
- L5S5
-1 -4 -6 -4 -1
0 0 0 0 0
2 8 12 8 2
0 0 0 0 0
-1 -4 -6 -4 -1
24TRICS SystemStructure 1. Segmentation Stage
- Segmentation technique
- Mean Shift
- K-mean (Fast k-mean)
- Adaptive Fast k-mean
D. Comaniciu and P. Meer. Mean shift A robust
approach toward feature space analysis. PAMI,
24(5)603619, May 2002 C.Elkan, Using the
triangle inequality to accelerate k-Means. In
Proc. of ICML 2003. pp 147--153
25Mean Shift
Region of interest
Center of mass
Mean Shift vector
Objective Find the densest region
26Mean Shift
Region of interest
Center of mass
Mean Shift vector
Objective Find the densest region
27Mean Shift
Region of interest
Center of mass
Mean Shift vector
Objective Find the densest region
28Mean Shift
Region of interest
Center of mass
Mean Shift vector
Objective Find the densest region
29Mean Shift
Region of interest
Center of mass
Mean Shift vector
Objective Find the densest region
30Mean Shift
Region of interest
Center of mass
Mean Shift vector
Objective Find the densest region
31Mean Shift
Region of interest
Center of mass
Objective Find the densest region
32TRICS SystemStructure 1. Segmentation Stage
- 1. Mean Shift
- So slow
- Many parameters
(170256) - hs16,hr16,m500 - Time 0 34 15 - classes 9 - hs8,hr8,hw4,m500 - Time 0 39 54 - classes 7
33Fast (Accelerated) K-mean
- Lemma 1 Let p be a point and let c1 and c2 be
centers. - If E(c1,c2) 2E(p,c1) then
- E(p,c2) E(p,c1).
- Lemma 2 Let p be a point and let c1 and c2 be
centers. Then - E(p,c2) max0,E(p,c1) E(c1,c2)
C.Elkan, Using the triangle inequality to
accelerate k-Means. In Proceedings of the 20th
ICML, Washington DC, 2003. pp 147--153
34TRICS SystemStructure 1. Segmentation Stage
- 2. Fast K-mean
- with spatial features
- structured segmentation
- Increase no. clusters
35TRICS SystemStructure 1. Segmentation Stage
- 2. Fast K-mean
- without spatial features
- scattered regions of same cluster
- disjoint region separation
36TRICS SystemStructure 1. Segmentation Stage
- 2. Fast K-mean
- Small regions (noise)
- Small regions elimination
37TRICS SystemStructure 1. Segmentation Stage
- 3. Adaptive Fast K-mean
- Clusters number generation
- Minimum region size estimation
Fully automatic segmentation technique
38TRICS SystemStructure 1. Segmentation Stage
- a. Clusters number generation
- CEC Combined Estimation Criterion
- If the VRC index, for k clusters, is smaller than
98 of the VRC index, for k-1 clusters, the CEC is
not satisfied. - If the VRC index, for k clusters, is larger than
102 of the VRC index for k-1 clusters, or if k1,
the CEC is satisfied. - If the VRC index, for clusters, is smaller than
102 but larger than 98 of the VRC index for
clusters, the CEC is satisfied only if TSS for
k-1 clusters is smaller than 70 of TSS for k
clusters.
D.Charalampidis, T.Kasparis, Wavelet-Based
Rotational Invariant Roughness Features for
Texture Classification and Segmentation. IEEE
Transactions on Image Processing.Vol.11.No.8
August 2002
39TRICS SystemStructure 1. Segmentation Stage
Start k1
Gray Image
Fast k-mean
Segmented Image
Calculate CEC
kk1
Yes
CEC Satisfied?
No
Stop Clusters number k
40TRICS SystemStructure 1. Segmentation Stage
- b. Minimum region size estimation
- Split the disjoint regions.
- Count all regions size.
- Sort regions size and calculate the step between
them. - Select the regions size of step more than the
largest image dimension. - Consider the minimum region size.
41TRICS SystemStructure 1. Segmentation Stage
Original gray wavelets Laws
Features
Image
Wavelets
size 170256 M500, EM 324
Laws
A-time adaptive fast k-mean time, F-time
fixed k fast k-mean time, T-time traditional
k-mean , E-time elimination time , EM-time
Elimination time with estimating minimum size
region
42TRICS SystemStructure Database set1
- The training set consists of 32 classes of
Brodatz texture database - Each image has a size of 256 256. Each image
was mirrored horizontally and vertically to
produce a 512 512 image. - The image is split into 16 images of size 128
128.
256 256 512 512 16 128 128
43TRICS SystemStructure Database set2
- The training set consists of 9 classes cloud,
sky, sea, sand, tree, grass, stone, water, and
wood - Each class has number of samples from 12 to 25
samples. - These samples are taken from real natural images
as random 64x64 rectangles.
44TRICS SystemStructure Database
- Database record
- Sample
- Class (level1,level2)
- 58 Features (6 Moment statistics, 4 Co-ocurance
measures, 3 Tamura, and 15 wavelets mean, 15
wavelets variance , 15 wavelets energy for five
levels wavelets decomposition) - Hue
45TRICS SystemStructure 2. Classification Stage
- Feature extraction (Region based)
- Rectangular region
- Maximum rectangle
- 64 x 64 rectangle
- Arbitrary shape
- Padding rectangle
-
- Ying Liu, Xiaofang Zhou, Wei-Ying Ma,
Extracting Texture Features - from Arbitrary-shaped Regions for Image
Retrieval. 2004 IEEE - International Conference on Multimedia and Expo.,
Taipei, Jun. 2004
46TRICS SystemStructure 2. Classification Stage
- Feature extraction (Region based)
- Region based features
- GLCM measures (Energy, Entropy, Inertia,
Homogeneity ) - Tamura (Coarseness, Contrast , Directionality)
- Wavelets coefficients for 5 levels
- Mean and variance
- Energy
P.Howarth, S.Ruger, Evaluation of texture
features for content-based image retrieval. In
proceedings of the International Conference on
Image and Video Retrieval, Springer-Verlag (2004)
326324 O. Commowick C. Lenglet C.
Louchet, Wavelet-Based Texture Classification
and Retrieval 2003 found in http//www.tsi.enst.f
r/tsi/enseignement/ressources/mti/classif-textures
/ Eka Aulia, Hierarchical Indexing For
Region Based Image Retrieval, Master thesis of
Science in Industrial Engineering, Louisiana
State University and Agricultural and Mechanical
College, May 2005
47TRICS SystemStructure 2. Classification Stage
- GLCM and Tamura
- Scale variant features, not suitable for natural
textures
48TRICS SystemStructure 2. Classification Stage
- Wavelets mean and variance
- Values were very scattered and the results were
not accurate for most cases. - Wavelets energies
- Classification accuracy of 92 using
leave-one-out (each (sub) image is classified
one by one so that other (sub) images serve as
the training data) method
49TRICS SystemStructure 2. Classification Stage
- Classification technique
- KNN classifier with (k1,k5,k10,k20)
- Distance Metric is L2 Euclidean distance
- k5 gives accuracy up to 94 using N-fold (the
collection of (sub) images is divided into N
disjoint sets, of which N-1 serve as training
data in turn and the Nth set is used for testing)
50TRICS SystemStructure 2. Classification Stage
51TRICS SystemStructure 3. Coloring Stage
- Color model conversion
- HSV/HSB color model
- HSI/HLS color model
Change in Saturation Hue 0, Luminance0.5 Change in Brightness Hue 0, Saturation 1 Change in Hue Sat1, Luminance1
Change in Saturation Hue 0, Luminance0.5 Change in Luminance Hue 0, Saturation 1 Change in Hue Sat1, Luminance1
52TRICS SystemStructure 3. Coloring Stage
53TRICS SystemStructure 3. Coloring Stage
- Setting Channels values
- Brightness
- The gray image itself
- Hue
- One hue value for each texture
- Saturation
- HSV 1- brightness HSI 0.5(1-lightness)
54TRICS SystemStructure 3. Coloring Stage
55TRICS SystemStructure 3. Coloring Stage
- Outlines
- Introduction
- Automatic coloring in the literature
- TRICS Texture Recognition based Image Coloring
System - Results
- Conclusion
- Future work
56Results and Conclusion Results
- Outlines
- Introduction
- Automatic coloring in the literature
- TRICS Texture Recognition based Image Coloring
System - Results
- Conclusion
- Future work
57Results and Conclusion Results
58Results and Conclusion Results
59Results and Conclusion Results
- 2 classification levels
- if the KNN results in 5 classes grass, sea,
water, grass, sea - The traditional solution is the class of the
grass. - The 2 levels classification solution is sea.
- (Sea and water), (trees and grass), (sky and
clouds) and (wood and stone) are considered as
one class in level one.
60Results and Conclusion Comparisons
HSV/HSB HIS/HLS
61Results and Conclusion Comparisons
- Outlines
- Introduction
- Automatic coloring in the literature
- TRICS Texture Recognition based Image Coloring
System - Results
- Conclusion
- Future work
62Results and Conclusion Conclusion
- We proposed a new computer coloring technique
that simulates the human vision in this area. - The proposed coloring system is contributed for
coloring gray natural scenes. - The execution time of TRICS is minimized using
Fast k-mean segmentation technique and the
results are enhanced by splitting the disjoint
regions and by eliminating small regions.
- Outlines
- Introduction
- Automatic coloring in the literature
- TRICS Texture Recognition based Image Coloring
System - Results
- Conclusion
- Future work
63Results and Conclusion Conclusion
- Clusters number generation algorithm and the
minimum region size estimation algorithm increase
the professionalism of the system but also
increases the time of the execution. And by using
both of them TRICS becomes a fully unsupervised
intelligent recognition based coloring system. - HSV coloring model is very suitable for our
system and the coloring results have good natural
look.
64Results and Conclusion Conclusion
- Outlines
- Introduction
- Automatic coloring in the literature
- TRICS Texture Recognition based Image Coloring
System - Results
- Conclusion
- Future work
- We consider our proposed system structure as an
abstract structure for building any more
intelligent coloring systems for any other types
of images - Our proposed system results perform the other
coloring systems.
65Future work
Gray image
A
- Outlines
- Introduction
- Automatic coloring in the literature
- TRICS Texture Recognition based Image Coloring
System - Results
- Conclusion
- Future work
Segmentation
Features extraction (Joint, wavelets, laws,)
Segmentation (Mean Shift, K-Mean, FCM,..)
B
Classification
Features extraction (Co-occurrence, Tamura,
Wavelets energies)
Classification (K-NN classifier,..)
C
Coloring
Convert image to HSV channels
Convert to RGB
Set Hue, Saturation, and Brightness
Colored image
66Future work
67Future work
- Segmentation and classification stages are
research areas and any improvement will increase
the accuracy of the system. - Using different types of features and training
set enables the system for coloring images like
manmade images, indoors, and people photos .
68List Of Publications
- Noura A.Semary, Mohiy M. Hadhoud, W. S.
El-Kilani, and Nabil A. Ismail, Texture
Recognition Based Gray Image Coloring, The 24th
National Radio Science Conference (NRSC2007), pp.
C22, March 13-15, 2007, Faculty of Engineering,
Ain-Shams Univ., Egypt.
69- Thanks
- ? ????? ??? ???? ????? ??? ????????