Introduction to Image Processing

1
Introduction to Image Processing

• CS474/674 Prof. Bebis
• Chapter 1 Sections 2.2, 2.3, 2.4

2
What is the goal of Image Processing?

• Image processing focuses on two major tasks
• Improve image quality for human interpretation
  and high level processing.
• Process images for storage and transmission.

3
Related Areas

• Image Processing
• Computer Vision
• Computer Graphics

4
Image Processing
5
Image Processing (contd)

• Image Enhancement

6
Image Processing (contd)

• Image Restoration

7
Image Processing (contd)

• Example of image restoration
• Hubble telescope
• An incorrect mirror made many of Hubbles
  images useless
• Image processing techniques were used to fix
  this!

8
Image Processing (contd)

• Image Compression

9
Computer Graphics
10
Computer Graphics
Projection, shading, lighting models
Output
Image
Synthetic Camera
11
Computer Vision
12
Computer Vision
Cameras
Images
13
Key Processes in Image Processing
Image Restoration
Morphological Processing
Segmentation
Image Enhancement
Image Acquisition
Representation Description
Object Recognition
Problem Domain
Color Image Processing
Image Compression
14
Image Acquisition
Image Restoration
Morphological Processing
Segmentation
Image Enhancement
Image Acquisition
Representation Description
Object Recognition
Problem Domain
Color Image Processing
Image Compression
15
Image Enhancement
Image Restoration
Morphological Processing
Segmentation
Image Enhancement
Image Acquisition
Representation Description
Object Recognition
Problem Domain
Color Image Processing
Image Compression
16
Image Restoration
Image Restoration
Morphological Processing
Segmentation
Image Enhancement
Image Acquisition
Representation Description
Object Recognition
Problem Domain
Color Image Processing
Image Compression
17
Morphological Processing
Image Restoration
Morphological Processing
Segmentation
Image Enhancement
Image Acquisition
Representation Description
Object Recognition
Problem Domain
Color Image Processing
Image Compression
18
Segmentation
Image Restoration
Morphological Processing
Segmentation
Image Enhancement
Image Acquisition
Representation Description
Object Recognition
Problem Domain
Color Image Processing
Image Compression
19
Representation Description
Image Restoration
Morphological Processing
Segmentation
Image Enhancement
Image Acquisition
Representation Description
1
2
0
3
Object Recognition
Problem Domain
Color Image Processing
Image Compression
20
Object Recognition
Image Restoration
Morphological Processing
Segmentation
Image Enhancement
Image Acquisition
Representation Description
Object Recognition
Problem Domain
Color Image Processing
Image Compression
21
Image Compression
Image Restoration
Morphological Processing
Segmentation
Image Enhancement
Image Acquisition
Representation Description
Object Recognition
Problem Domain
Color Image Processing
Image Compression
22
Color Image Processing
Image Restoration
Morphological Processing
Segmentation
Image Enhancement
Image Acquisition
Representation Description
Object Recognition
Problem Domain
Color Image Processing
Image Compression
23
Applications

• Industrial inspection/quality control
• Surveillance and security
• Face recognition
• Space applications
• Medical image analysis
• Autonomous vehicles
• Virtual reality and much more ...

24
Industrial Computer Vision (Machine Vision)
Industrial computer vision systems work really
well! Make strong assumptions about lighting
conditions Make strong assumptions about the
position of objects Make strong assumptions
about the type of objects
25
Visual Inspection
COGNEX
26
Optical character recognition (OCR)

• Technology to convert scanned docs to text

Automatic check processing
Digit recognition, ATT labs http//yann.lecun.com
/exdb/lenet/
License plate readers http//en.wikipedia.org/wiki
/Automatic_number_plate_recognition
27
Biometrics
28
Login without a password
Face recognition systems now beginning to appear
more widelyhttp//www.sensiblevision.com/
Fingerprint scanners on many new laptops, other
devices
29
Fingerprint Recognition
30
Fingerprint Recognition at
Super-Template Synthesis
super-template
matching
ID
T. Uz, G. Bebis, A. Erol, and S. Prabhakar,
"Minutiae-Based Template Synthesis and Matching
for Fingerprint Authentication", Computer Vision
and Image Understanding (CVIU), vol 113, pp.
979-992, 2009.
31
Hand-based Authentication/Recognition
32
Hand-based Authentication/Recognition at
G. Amayeh, G. Bebis, A. Erol, and M. Nicolescu,
"Hand-Based Verification and Identification Using
Palm-Finger Segmentation and Fusion", Computer
Vision and Image Understanding (CVIU) vol 113,
pp. 477-501, 2009.
33
Iris Recognition
How the Afghan Girl was Identified by Her Iris
Patterns
34
Face Processing Applications

• Face Recognition
• Face Detection
• Gender Classification
• Facial Expression Recognition
• and many more

35
Face Recognition
Challenge appearance changes
http//www.face-rec.org/
36
Face Recognition at

• Visible spectrum
• High resolution, less sensitive to the presence
  of eyeglasses.
• Particularly sensitive to changes in illumination
  direction and facial expression.

• Thermal IR spectrum
• Not sensitive to illumination changes
• Not very sensitive to changes in facial
  expression
• Low resolution, sensitive to air currents, face
  heat patterns, aging, and the presence of
  eyeglasses

visible
LWIR
37
Face Recognition at
Fuse visible with thermal infrared imagery
G. Bebis, A. Gyaourova, S. Singh, and I.
Pavlidis, "Face Recognition by Fusing Thermal
Infrared and Visible Imagery", Image and Vision
Computing, vol. 24, no. 7, pp. 727-742, 2006.
38
Face Detection

• Many new digital cameras now detect faces
• Canon, Sony, Fuji,

39
Face Detection at
Human skin exhibits an abrupt change in
reflectance around 1.4 µm.
J. Dowdall, I. Pavlidis, and G. Bebis, "Face
Detection in the Near-IR Spectrum", Image and
Vision Computing, vol 21, no. 7, pp. 565-578,
2003.
40
Gender Classification

• Useful for collecting demographic data but also
  boosting face
• recognition performance!
• Related applications race classification, age
  classification.

Key challenge choose features that encode gender
information but not identity information!
41
Gender Classification at
Discover gender-specific features using Genetic
Algorithms (GAs)
Original images
Reconstructed using traditional features
Reconstructed using GA-based features
Z. Sun, G. Bebis, and R. Miller, "Object
Detection Using Feature Subset Selection",
Pattern Recognition, vol. 37, pp. 2165-2176,
2004.
42
Facial Expression Recognition
http//www.youtube.com/watch?vM1WgnisIyPQfeature
related
43
Smile detection?
Sony Cyber-shot T70 Digital Still Camera
44
Object Recognition
2D
3D
45
Object Recognition (contd)

• (2) Viewer-centered

• (1) Object-centered

46
Object Recognition at
Synthesize new 2D views of a 3D object using
linear combinations of a set of 2D
reference views
47
Object Recognition at

• reference view 1 reference view 2

novel view recognized

• No 3D models required.
• Predict novel 2D views from known 2D views

W. Li, G. Bebis, and N. Bourbakis, "3D Object
Recognition Using 2D Views", IEEE Transactions on
Image Processing, vol. 17, no. 11, pp. 2236-2255,
2008.
48
Object Recognition at
Reference Views
Recognition Results
49
Segmentation
Separate objects of interest from
background. Typically required before object
recognition!
50
Segmentation at
Iterative Tensor Voting
Motivated by the Gestalt principles of human
visual perception
L. Loss, G. Bebis, M. Nicolescu, and A.
Skurikhin, "An Iterative Multi-Scale Tensor
Voting Scheme for Perceptual Grouping of Natural
Shapes in Cluttered Backgrounds", Computer Vision
and Image Understanding (CVIU), vol. 113, no. 1,
pp. 126-149, January 2009.
51
Image Retrieval

• Combine color, shape, texture etc.

http//corbis.demo.ltutech.com/en/demos/corbis/
52
Visual Surveillance and Human Activity Recognition
53
Human Activity Recognition at

• Recognize simple human actions using 3D head
  trajectories

J. Usabiaga, G. Bebis, A. Erol, Mircea Nicolescu,
and Monica Nicolescu, "Recognizing Simple Human
Actions Using 3D Head Trajectories",
Computational Intelligence (special issue on
Ambient Intelligence), vol. 23, no. 4, pp.
484-496, 2007.
54
Vision-based Interaction and Games
Kinect
Nintendo Wii has camera-based IRtracking built
in. See Lees work atCMU on clever tricks on
using it tocreate a multi-touch display!
55
Traffic Monitoring
http//www.honeywellvideo.com/
56
Smart cars
Mobileye

• Vision systems currently in high-end BMW, GM,
  Volvo models.

57
Vision in space
NASA'S Mars Exploration Rover Spirit

• Vision systems used for several tasks
• Obstacle detection
• Position tracking
• 3D terrain modeling
• For more info, read Computer Vision on Mars by
  Matthies et al.
• International Journal of Computer Vision, 2007.

58
Crater Detection at
Verification
Multi-scale edge detection
Hypotheses
Convex grouping
Line fitting
Ebrahim Emami, Touqeer Ahmad, George Bebis, Ara
Nefian, and Terry Fong, "Crater Detection Using
Unsupervised Algorithms and Convolutional Neural
Networks", IEEE Transactions on Geoscience and
Remote Sensing, vol. 57, no. 8, 2019.
59
Automatic Panorama Stitching
60
3D reconstruction from internet photo collections
see building Rome in a day project at U.
Washington
http//grail.cs.washington.edu/rome/
61
Medical Imaging

Image guided surgery
Skin/Breast Cancer Detection
Enable surgeons to visualize internal structures
through an automated overlay of 3D
reconstructions of internal anatomy on top of
live video views of a patient.
3D imaging MRI, CT
62
A Simple model of image formation
63
What is light?

• The visible portion of the electromagnetic (EM)
  spectrum.
• Approximately between 400 and 700 nanometers.

64
Gama-Ray Imaging

Bone scan
PET scan
Gamma-ray imaging nuclear medicine and
astronomical observations
65
X-Ray Imaging
Chest X-ray
Computer boards
CT scan

X-rays medical diagnostics, industry, astronomy,
etc.
Aortic angiogram
66
Infrared Imaging
Infrared bands light microscopy, astronomy,
remote sensing, industry, and law enforcement.
67
Sonic images

• Produced by the reflection of sound waves off an
  object.
• High sound frequencies are used to improve
  resolution.

68
Range images

• Can be produced by using laser range-finders.
• An array of distances to the objects in the
  scene.

69
Image formation

• There are two parts to the image formation
  process
• The geometry of image formation, which determines
  where in the image plane the projection of a
  point in the scene will be located.
• The physics of light, which determines the
  brightness of a point in the image plane as a
  function of illumination and surface properties.

70
Pinhole camera

• This is the simplest device to form an image of a
  3D scene on a 2D surface.
• Straight rays of light pass through a pinhole
  and form an inverted image of the object on the
  image plane.

(x,y)
(X,Y,Z)
71
Camera optics

• In practice, the aperture must be larger to admit
  more light.
• Lenses are placed in the aperture to focus the
  bundle of rays from each scene point onto the
  corresponding point in the image plane

72
Physics of Light

• Simple model
• f(x,y)i(x,y)r(x,y)
• where
• i(x,y) the amount of illumination
• incident to the scene
• 2) r(x,y) the reflectance from the object

73
CCD (Charged-Coupled Device) cameras

• Tiny solid state cells convert light energy into
  electrical charge.
• The image plane acts as a digital memory that can
  be read row by row by a computer.

74
Frame grabber

• Usually, a CCD camera plugs into a computer board
  (frame grabber).
• The frame grabber digitizes the signal and stores
  it in its memory (frame buffer).

75
Image digitization

• Sampling is measuring the value of an image at a
  finite number of points.
• Quantization is the representation of the
  measured value at the sampled point by an integer.

76
Image digitization (contd)
255
0
77
Effect of Image Sampling

• original image
  sub-sampled by a factor of 2
• sub-sampled by a factor of 4
  sub-sampled by a factor of 8

Note images have been resized for
comparison purposes
78
Effect of Image Quantization

• 256 gray levels (8bits/pixel)
  32 gray levels (5 bits/pixel) 16 gray levels
  (4 bits/pixel)
• 8 gray levels (3 bits/pixel)
  4 gray levels (2 bits/pixel) 2 gray
  levels (1 bit/pixel)

79
Representing Digital Images
The result of sampling and quantization is a
matrix of integer numbers. Here we have an
image f(x,y) that was sampled to produce N rows
and M columns.
80
Representing Digital Images (contd)

• There is no strict requirements about N and M
• Number of quantization levels L 2k
• k is the number of bits/pixel
• The range of pixel values 0, L-1

• The number of bits b required to store an image
• b N x M x k

81
Computer Vision Jobs

• Academia
• MIT, UC-Berkeley, CMU, UIUC, USC UNR!
• National Labs and Government
• Los Alamos National Lab, Lawrence Livermore
  National Lab etc.
• Navy, Air-force, Army
• Industry
• Microsoft, Intel, IBM, Xerox, Compaq, Siemens,
  HP,
• TI, Motorola, Phillips, Honeywell, Ford etc.
• http//www.cs.ubc.ca/spider/lowe/vision.html

82
What skills do you need to succeed in this field?

• Strong programming skills (i.e., C, C, Matlab)
• Good knowledge of Data Structures and Algorithms
• Good skills in analyzing algorithm performance
  (i.e., time and memory requirements).
• Good background in mathematics, especially in
• Linear Algebra
• Probabilities and Statistics
• Numerical Analysis
• Geometry
• Calculus

83
Related Courses at UNR

• CS474/674 Image Processing and Interpretation
• CS485/685 Computer Vision
• CS486/686 Advanced Computer Vision
• CS479/679 Pattern Recognition
• CS482/682 Artificial Intelligence
• CS480/680 Computer Graphics
• CS776 Evolutionary Computation
• Special Topics
• Machine Learning, Biometrics, Neural Networks and
  more.
• Big Data Minor
• https//www.unr.edu/degrees/big-data/minor