Chapter 2 : Imaging and Image Representation

1
Chapter 2 Imaging and Image Representation

• Computer Vision Lab.
• ??????? 3??
• ???

2
Contents

• 2.6 Richness and Problems of Real Imagery
• 2.7 3D Structure form 2D Image
• 2.8 Five Frames of Reference
• 2.9 Other Types of Sensors

• 2.1 Sensing Light
• 2.2 Image Device
• 2.3 Problems in Digital Images
• 2.4 Picture Function And Digital Image
• 2.5 Digital Image
• Formats

3
2.1 Sensing Light

• Device sense and produce different types of
  electromagnetic radio wave, X-ray, microwaves,
  etc.
• Human Eye 400 nm(violet) 800 nm (red)
• Snakes and CCD sensers longer then 800 nm
  (infrared)
• Device to detect very short length X-ray
• Device to detect very long radio waves
• Different Wave lengths of radiation have
  Different Properties
• X-ray Penetrate human bone
• Infrared Not penetrate even clouds

4
2.1 Sensing Light

• Simple model of common photography

(Sun or Flash bulb)
Illuminated by single source
Sense it via chemical on film
Reflects radiation Toward camera

• Wavelengths in the light range result very near
  the surface objects

5
Contents

• 2.1 Sensing Light
• 2.2 Image Device
• 2.2.1 CCD Cameras
• 2.2.2 Image Formation
• 2.2.3 Video Cameras
• 2.2.3 The Human Eye
• 2.3 Problems in Digital Images
• 2.4 Picture Function And Digital Image
• 2.5 Digital Image Formats

• 2.6 Richness and Problems of Real Imagery
• 2.7 3D Structure form 2D Image
• 2.8 Five Frames of Reference
• 2.9 Other Types of Sensors

6
CCD Cameras

• Most flexible and common input device for
  machine-vision systems
• Much like a 35mm file camera Commonly used
  family photo, image plane
• Tiny solid state cells convert energy into
  electrical charge
• All cells fist cleared 0, and integrate their
  response
• Image Plane acts as digital memory read row by
  row by Input Process
• If 500(rows) X 500(cols) / byte sized Gray value
  ¼ of million bytes obtained
• Frame grabber contain memory for the image size
  and control camera

7
CCD Cameras

• Frame Buffer
• Centrol role of Image processong
• High speed image store available Actually Store
  several Images or their derivatives
• Digital Image refer to pixel values as Ir,c
• I array name
• R row
• C Colume

8
2.2.2 Image Formation

• Conceptualized as projection of each point
  through center of projection or lens center on
  image plane
• Actual lenses compound with two refracting
  surface
• Two effects cause blurring of image and limit
  sharpness and the size of the smallest scene
  details that sensed
• Light collector
• ? Circle of Confusion Because Different
  Banding of different color of lights the cone of
  rays actually results in a finite or blurred spot
  
• CCD sensor array is discrete units each sensor
  cell integrates the rays received neighboring
  point

9
2.2.2 Image Formation

• Arrangement of CCD sensor cell
• Linear
• Only need to measure width and where imaging and
  inspecting continuous web
• Flatbed scanner
• Use Cylindrical lens commonly used
• Circular
• Analog dial watches or speedometers
• Scanned image of needle
• ROSA
• Provides hardware solution integrated all the
  light energy
• Quantizing the power of spectrum image
• Chip manufacturing technology

10
2.2.3 Video Camera

• Record sequence of images at a rate of 30 /sec
• To provide smooth human perception, 60 half
  frames /sec (interlacing)
• CCD camera technology for machine vision
  suffered from display standard
• Interlacing of odd and even frames need to give
  smooth picture
• 4 3 size ratio

11
The Human Eye

• Spherical camera with 20mm focal length lens
  focusing the image on the retina
• Iris controls amount of light passing through the
  lens by the size of pupil
• Has one hundred million receptor cells
• Fovea has dense concentration of color receptors,
  called cones
• Away from center, density cones decrease while
  density of black and white receptor, called rod,
  increases
• Three different type of cone is sensitive
  different wavelengths of light

• Ability to smooth perceive a seamless and stables
  3D world
• Saccades of the eyes are necessary for proper
  human visual perception

12
Contents

• 2.6 Richness and Problems of Real Imagery
• 2.7 3D Structure form 2D Image
• 2.8 Five Frames of Reference
• 2.9 Other Types of Sensors

• 2.1 Sensing Light
• 2.2 Image Device
• 2.3 Problems in Digital Images
• 2.4 Picture Function And Digital Image
• 2.5 Digital Image
• Formats

13
2.3.1/5 Geometric Distortion and Clipping or
Warp-Around

• The lens is imperfect the beams are not bent
  exactly as intended
• Barrel distortion for small focal length lenses
  straight line s at the periphery of the scene
  appear to bow away

• Barrel distortion is often observed when short
  focal length lenses are used
• Dark grid intersections at left were actually
  brightest of scene.
• In A/D conversion the bright values were clipped
  to lower values.

14
2.3.3 Blooming

• Because Discrete detectors, such as CCD cells are
  not perfectly insulated from other
• blooming leakage spreads out from a very bright
  region on the image plane, resulting in a bright
  flower in image

15
2.3.2/4 Scattering and CCD Variations

• Scattering
• Beam of radiation bent and dispersed by the
  medium through
• Aerial and satellite images are particularly
  susceptible to such effect
• Caused by water vapor temperature gradients
• CCD Variations
• Imperfect manufacturing
• Variations in the responses of the different
  cells to identical light density
• Precise interpretation of intensity
• I2r,c Sr,cI1r,c Tr,c
• Have some dead cells
• Software remedy assign average response of the
  neighbors

16
2.3.67 Chromatic Distortion and Quantization
Effects

• Chromatic Distortion
• Different wavelengths of light are vent
  differently by the lens
• Scene spot may actually image a few pixels apart
  on the detector
• Example Very sharp black white boundary
  periphery of scene in ramp of intensity change
• Quantization Effects
• The digitization process collects a sample of
  intensity from discrete area maps it to one of
  the discrete set of gray value
• Susceptible mixing and rounding problems

17
Contents

• 2.6 Richness and Problems of Real Imagery
• 2.7 3D Structure form 2D Image
• 2.8 Five Frames of Reference
• 2.9 Other Types of Sensors

• 2.1 Sensing Light
• 2.2 Image Device
• 2.3 Problems in Digital Images
• 2.4 Picture Function And Digital Image
• 2.4.1 Type of Images
• 2.4.2 Image Quantization and Spatial Measurement
• 2.5 Digital Image
• Formats

18
2.4.1 Types of Images

• Convenient concepts of analog image and digital
  images
• Digital image 2D rectangular array of discrete
  values
• Image space and intensity range are quantized
  into a discrete set of values
• Permitting the image to be stored in 2D computer
  memory structure
• Common record intensity 8bit(0255)
• 2 analog image F(x,y) which has infinite
  precision in spatial parameters x and y and
  infinite precision in intensity at each spatial
  point (x, y)
• 3 digital image Ir,c represented by a
  discrete 2D array of intensity samples, each of
  which is represented a limited precision

19
2.4.1 Types of Images

• Mathematical model of an image as a function of
  two real spatial parameters is enormously useful

20
2.4.1 Types of Images

• 4 picture function f(x,y) of a picture as
  a function of two spatial variables x, and y and
  x and y are real values defining points of
  picture and f(x, y) is usually also real value
• 5 gray scale image Monochrome digital
  image Ir,c with one intensity value has 3
  elements
• 6 multispectral image 2D image Mx,y has
  a vector of values at each spatial point or pixel
  (If image is color, vector has 3 elements)
• 7 binary image digital image with all
  pixel values 0 or 1
• 8 labeled image digital image Lr,c
  whose pixel values are symbols from finite
  alphabet (Related concepts thematic image and
  pseudo-colored image)

21
2.4.2 Image Quantization and Spatial Measurement

• 9 nominal resolution of the CCD sensor
  is the scene element that images to a single
  pixel on the image plan
• 10 resolution refer to the sensor in
  making measurements subpixel resolution
  Precision of measurement is a fraction of the
  nominal resolution
• 11 def field of view of a sensor (FOV) is the
  size of the scene can sense (10 inches X 10
  inches),Angular field of view (55 degrees by 40
  degrees)

22
2.5.11 MPEG format for video

• Use appropriate resolution
• Too little produce poor recognition
• Too much slow down algorithm and waste memory

23
2.4.2 Image Quantization and Spatial Measurement

• Spatial quantization effects impose limits on
  measurement accuracy and detectability

24
2.4.2 Image Quantization and Spatial Measurement

• Expect error as bad as 0.5 pixels in the
  placement of a boundary due to rounding of a
  mixed pixel when a binary image
• If we expect to detect certain features in a
  binary image, then we must make sure that their
  image size is at list two pixels in diameter
  this include gaps between objects
• 12 mixed pixel image pixel whose
  intensity represents a sample from a mixture of
  material types in the real world

25
Contents

• 2.6 Richness and Problems of Real Imagery
• 2.7 3D Structure form 2D Image
• 2.8 Five Frames of Reference
• 2.9 Other Types of Sensors

• 2.1 Sensing Light
• 2.2 Image Device
• 2.3 Problems in Digital Images
• 2.4 Picture Function And Digital Image
• 2.5 Digital Image Formats

26
2.5 Digital Image Formats

• Dozen of different formats still in use
• Row data encode image pixels in row-by-row
  (raster order)
• Most recently developed standard formats contain
  a header with mom-image information necessary to
  label the data to decode it

27
2.5.1/2 Image File header Data

• Image File header
• Need to make an image file self-describing so
  that image-processing tools can work with them
• Should contain
• image dimention, type, data , title
• Color table, coding table
• Nice feature not often available
• History section
• Image Data
• Handle only limited types if images binary,
  monochrome, today grow include more image type
  and features
• Pixel size, image size limits between file
  formats
• Multimedia format evolving image data along with
  text, graphics, music, etc.

28
2.5.3 Data Compression

• Reduce the size of an image (30 percent or even
  3 percent of raw size)
• Copression can be lossless and lossy
• Lossless compression original image recovered
  exactly
• Lossy compression loss of quality is perceived
  (but, not always)
• To implement compression
• Include overhead (compression method and
  parameter)
• Loss or Change of a few bit little or no affect
  on
• ( exciting area from signal processing to object
  recognition)
• 13 Lossless decompression methods exists
  to original image, otherwise Lossy

29
2.5.4 Commonly Used Formats

• Colleague or Image data base GIF, JPG, PS
• Scanned and Original Data format GIF, TIFF
• Image/Graphics file formats are still evolving

30
2.5.5 Run-Coded Binary Images

• Efficient for binary or labeled images
• Reduce memory space
• Speed up image operations

31
2.5.6 PGM Portable Gray Map

• Simplest file format
• PBM or PBM (Portable Bit Map)
• family format PBM/PGM,PPM
• Image header encoded ASCII
• Magic Value P2
• ( P2 - gray level,
• P3 - (R,G,B)
• P4 -P4 binary )
• Rows 8
• Cols 16
• Maximum gray value 192

32
2.5.7-8 GIF/TIFF Image File Format

• Tag Image File Format (TIFF/TIF)
• Originated by Aldus Corp.
• Very general and complex
• Used all popular platforms and scanner
• Support multiple images
• ( 1 24 bit / pixel)
• Option available
• (lossy / lossless)

• Graphics Interchange Format(GIF)
• oriented from CompuServ, Inc
• Used on WWW or current DBs
• Only 256 color value available
• Typically sufficient
• Cannot be used high precision color
• More compact 16-color option
• (LZW nonlossy compression)

33
2.5.9 JPEG Image File Format

• JPEG (Joint Photographic Experts Group)
• Provide practical compression of high-quality
  color
• Stream oriented and allow realtime hardware for
  encoding and decoding
• Up to 64K X 64K pixels of 24 bits
• Header contain thumbnail image (up to 64k)
• Achieve high compression, flexible
• but lossy coding scheme Unnoticeable
  degration(1/20)
• Compression work well when has large constant
  regions
• High frequency variation not important
• Compression scheme DCT(Discrete Cosine
  Transform) followed by Huffman coding

34
2.5.10 PostScript

• Family of formats BDC/PDL/EPS
• Using Printable ASCII
• Commonly used to contain graphics or images
  inserted into large document
• PDL page description language
• EPS encapsulated postscript
• Pixel value encoded via 7 bit ASCII ( changed by
  Text Editor)
• 753000 dots / inch grayscale or color
• PDL header contain boundary box of image

35
2.5.11 MPEG format for video

• Stream-oriented encoding scheme for video, text,
  and graphics
• MPEG stands for Motion Picture Experts Group
• MPEG-1
• Primary design for multimedia systems
• Data rate
• Compression audio 0.25 Mbits/s
• Compression video 1.25 Mbits/s
• MPEG-2
• Data rate up to 15Mbits/s
• Handle high definition TV rates
• Compression scheme takes advantage of both
  spatial redundancy (used in JPEG) and temporal
  redundancy (general 1/25, 1/200 possible)
• Motion JPEG compression is not good

36
2.5.12 Comparison of Formats

• Cars TIF file output from the scanner 509,253
  bytes
• Final TIF file had fewer bits in color code
  171,430
• Lossy JPEG clear winner in terms of space (but,
  cost of decoding complexity)

37
Contents

• 2.6 Richness and Problems of Real Imagery
• 2.7 3D Structure form 2D Image
• 2.8 Five Frames of Reference
• 2.9 Other Types of Sensors

• 2.1 Sensing Light
• 2.2 Image Device
• 2.3 Problems in Digital Images
• 2.4 Picture Function And Digital Image
• 2.5 Digital Image Formats

38
2.6 Richness and Problems of real imagery

• Applications of automation solving problem
• Bruises in dark red cherries Irrelevant
  bandwidths of light filtered out
• Moving Object Strobe light for very short
  period time
• Turbine blades Structured light (red and green
  stripe) can make suface mesurement and inspection
  much easier

• The richness enhances human experience but causes
  problems for machine vision
• Intensity and color depends on complex way
• For example
• shiny surfaces
• Shadows
• mutual reflection
• transparent materials
• For recognition of many surfaces or objects,
  color little important relative to shape or
  texture
• Reflection controlled monochrome laser light, but
  be dominated by secondary reflections

39
Contents

• 2.6 Richness and Problems of Real Imagery
• 2.7 3D Structure form 2D Image
• 2.8 Five Frames of Reference
• 2.9 Other Types of Sensors

• 2.1 Sensing Light
• 2.2 Image Device
• 2.3 Problems in Digital Images
• 2.4 Picture Function And Digital Image
• 2.5 Digital Image Formats

40
2.7 3D Structure from 2D Images

• Images processing records complex relationships
  between 3D and 2D structure of the image
• Interposition
• Most important in depth cues
• Relative size
• Vanish point rail road
• Foreshortening Opened door image as trapeziodal
• Texture gradient
• Close can see detail of blade of grass
• Far away only green color

41
Contents

• 2.6 Richness and Problems of Real Imagery
• 2.7 3D Structure form 2D Image
• 2.8 Five Frames of Reference
• 2.8.1 Pixel Coordinate Frame
• 2.8.2 Object Coordinate Frame
• 2.8.3 Camera Coordinate Frame
• 2.8.4 Real Image Coordinate Frame
• 2.8.5 World Coordinate Frame
• 2.9 Other Types of Sensors

• 2.1 Sensing Light
• 2.2 Image Device
• 2.3 Problems in Digital Images
• 2.4 Picture Function And Digital Image
• 2.5 Digital Image Formats

42
2.8 Five frames of reference

• Needed in order to either qualitative or
  quantitative analysis of 3D scenes

43
2.8 Five frames of reference

• Pixel Coordinate Frame
• Each point has integer pixel coordinates
• A falls within pixel aar,ac
• ar and ac are ingefer row and column
• Using only image I, cannot determine which object
  is actually larger in 3D whether or not objects
  are on a collision course

44
2.8 Five frames of reference

• Object Coordinate Frame O
• Used to model ideal objects in both computer
  praphics and computer vision
• Two object frame
• Block Ob
• Pyramid Op
• Coodinate 3D corner point B relative to the
  object coordinate frame xb, 0, zb
• Regardless of how block is posed related to world
• Camera Coordinate Frame C
• Often needed for egocentric (camera centric) view
• Represent just in front of the sensor

45
2.8 Five frames of reference

• Real Image Cordinate Frame F
• Coordinate xf, yf, f
• F focal length
• xf, yf not description of pixels in the image
  array but related to the pixel size and pixel
  position of optical axis in the image
• Frame F contians the picture function digital
  image in the pixel array I
• World Coordinate Frame W
• Needed to relate objects in 3D

46
Contents

• 2.6 Richness and Problems of Real Imagery
• 2.7 3D Structure form 2D Image
• 2.8 Five Frames of Reference
• 2.9 Other Types of Sensors

• 2.1 Sensing Light
• 2.2 Image Device
• 2.3 Problems in Digital Images
• 2.4 Picture Function And Digital Image
• 2.5 Digital Image Formats

47
2.9.1 Microdensitometer

• Slides or film can he scanned by passing a single
  beam of light through the material
• Pros
• Less variation in intensity value
• Many more rows and cols can be obtained
• Cons
• Slow

48
2.9.2 Color and Multispectral images

• The refract film disperse a single beam into 4
  beam falling
• Gain traded for a loss in spatial resolution
• In rotation wheel design, sensing speed traded
  for color sensitivity
• Multispectral Satelite Scanner
• View earth 1 pixel at a time (through a straw)
• Prism produces multispectral pixel
• Image row by scanning boresight
• All rows by motion of satellite in orbit
• Spectrum of intensity value possible classify the
  ground type

49
2.9.3 X-ray

• Record transmit energy at image points in the far
  side of the emitter in same manner as
  mircodensitometer
• CT scanner 3D sensing accomplished

50
2.9.4 Magnetic Resonance Imaging(MRI)

• Produce 3D images if materials
• Is , r ,c
• s slice through the body
• r, c row, col
• Volume element (voxel) about 2mm per side
• Intensity measure chemistry of material
• Magnetic resonance angiography (MRA) produce
  intensity related speed of material at the voxel
• Digital image extracted from 3D MRA data
• Maximum intensity projection, or MIPr,c
  choosing the brightest voxel Is,r,c

51
2.9.5 Range Scanners and Range Images

• LIDAR
• Measure distance by comparing the change of phase
  (delay)
• Estimate the reflectivity of the surface spot
• Produce two registered images
• Range image
• Intensity image
• Slow and expensive

52
2.9.5 Range Scanners and Range Images

• Triangulation
• Bright image point xc, yc corresponding 3D
  point xw, yw, zw
• 3 form of light sheet and 2 equations in 3
  unknowns from the imaging ray
• Solving linear equations yields the location of
  3D surface point