Imaging and Image Representation

1
Imaging and Image Representation

• Dr. Ramprasad Bala
• Computer and Information Science
• UMASS Dartmouth
• CIS 465 Topics in Computer Vision

2
Imaging

• The Human Vision System
• The Imaging System
• The Imaging Device
• Digital Images
• Real Imagery
• Frames of references

3
(No Transcript)
4
The Human Eye

• The eye is a spherical camera
• 20 mm focal length lens focusing on the retina
• The iris provides aperture control (of the pupil)
• One hundred million receptor cells
• Fovea center of the retina has a large
  concentration of color receptors called cones
• The periphery has a concentration of black-white
  receptors called rods
• Three different types of cones
• Saccades important for visual perception

5
(No Transcript)
6
Sensing Light

• Human visual range 400 nanometers (violet) to
  800 nanometers (red) wavelength.
• Wavelength in the light range result from
  generating or reflecting mechanisms very near the
  surface of the object

7
Image receives reflections

• Light reaches surfaces in 3D
• Surfaces reflect
• Sensor element receives light energy
• Intensity counts
• Angles count
• Material counts

8
Imaging devices

• CCD cameras
• Most flexible and common machine vision device
• Instead of chemicals solid-state cells convert
  light energy into electrical charges
• Image plane acts as a digital memory that can be
  read row by row by a computer input process.

9
CCD Camera has discrete elts

• Lens collects light rays
• CCD elts replace chemicals of film
• Number of elts less than with film (so far)

10
Camera Programs Display

• Camera inputs to frame buffer
• Program can interpret data
• Program can add graphics
• Program can add imagery

11
CCD array element arrangement options
12
Problems with Digital Images

• Geometric Distortion
• Imperfect lenses
• Scattering
• Bent radiation
• Blooming
• Leaking of charge
• CCD variation
• Variation in responses in the elements

13
Blooming Problem with Arrays

• Difficult to insulate adjacent sensing elements.
• Charge often leaks from hot cells to neighbors,
  making bright regions larger.

14
8-bit intensity can be clipped

• Dark grid intersections at left were actually
  brightest of scene.
• In A/D conversion the bright values were clipped
  to lower values.

15
Lens distortion distorts image

• Barrel distortion of rectangular grid is common
  for cheap lenses (50)
• Precision lenses can cost 1000 or more.
• Zoom lenses often show severe distortion.

16
Other errors

• Clipping or Wrap around
• Conversion of Analog to Digital signal could
  result in clipping (losing of higher-order bits)
  or wrap around (too large values are wrapped
  around the intensities)
• Chromatic distortion
• Quantization Effects
• Mixing and rounding errors.

17
Picture Functions and Digital Images

• An image is a function of two variables.
• All functional analysis can be used in this view
• Definitions An analog image is a 2D image
  F(x,y) which has infinite precision in spatial
  parameters x and y and in intensity per point.
• A digital image is a 2D image Ir,c represented
  by a discrete 2D array of intensity samples, with
  limited precision.

18
More definitions

• A gray-scale image is a monochrome digital image
  Ir,c with one intensity value per pixel.
• A multispectral image is a 2D image Mx,y which
  has a vector of values at each spatial point or
  pixel. In the case of a color image the vector
  has 3 elements.
• A binary image is a digital image with all pixel
  values 0 or 1.

19
Different Coordinate systems
Cartesian with origin in the image center
Cartesian
Raster
20
Image Quantization

• Each pixel of a digital image represents a sample
  of some elemental region of the scene
• The field-of-view of an image sensor is a measure
  of how much of the scene it can see, would be
  more meaningful as angular field of view, 55o x
  45o.
• The nominal resolution of a CCD sensor is the
  size of the scene element that images to a single
  pixel on the image plane.

21
Resolution

• The resolution of an image can be viewed as to
  how many parts the field of view can be divided,
  which relates to both the capability to make
  precise measurements and to cover a region of a
  scene.
• If precision of measurement is a fraction of the
  nominal resolution, this is called subpixel
  resolution.

22
Resolution is pixels per unit of length

• Resolution decreases by one half in cases at left
• Human faces can be recognized at 64 x 64 pixels
  per face

23
Spatial quantization effects
24
Things to think about

• What spatial resolution is sufficient to resolve
  key features? Would, of course, depend on the
  application.
• Consider exercise 2.6.

25
Many different image file forms

• Portable gray map (PGM) older form
• GIF was early commercial version
• JPEG (JPG) is modern version
• Many others exist
• Do they handle color?
• Do they provide for compression?
• Need to have size parameters pixels

26
PGM image with ASCII info.

• P2 means ASCII gray
• Comments
• W16 H8
• 192 is max intensity
• Can be made with editor

27
JPG current popular form

• Public, not private, standard
• Allows for image compression often 101 or 301
  are easily possible
• 8x8 intensity regions are fit with basis of
  cosines
• Error in cosine fit coded as well
• Parameters then compressed with Huffman coding
• VERY TECHNICAL!

28
(No Transcript)
29
Problems of Real Imagery
30
3D from 2D

• Cognitive psychologist J.J.Gibson provided
  several quantitative models for structure info.
• Interposition is the most important cue in
  obtaining depth cues.
• Relative size is another important cue.
• Parallel lines meet visually vanishing point
• Texture gradient texture varies with distance
  and surface orientation

31
Five frames of reference
W world O object (P, B) C camera F real
image I pixel image
32
Pixel Coordinate Frame

• In the pixel array, each point has integer pixel
  coordinates.
• Pixel coordinate frame can be useful in obtaining
  information about the object such as texture,
  color, intensity etc.
• Pixel coordinates however cannot to be used to
  obtain size of objects and depth information.

33
Object coordinate frame

• An object coordinate frame is used to model ideal
  objects for computer graphics and computer
  vision.
• The coordinates of 3D corner point B relative to
  the object frame are xb,0,zb.
• The object coordinate frame is needed to inspect
  an object, for example, to check if a particular
  hole relatively in the correct place.

34
Camera coordinate frame

• The camera coordinate frame C is often needed for
  egocentric view (cameracentric).
• Helps represent whether the object is in front of
  the camera or moving etc.
• Computer graphics systems allow the user to
  select different camera views of the 3D scene
  being viewed.

35
Real image coordinate frame

• 3D points project to the real image plane at
  coordinates xf,yf,f, where f is the focal
  length. xf,yf are not subscripts of the pixels in
  the image array but relate to the pixel size and
  pixel position along the optical axis. xf,yf are
  based on the world coordinate system.
• Frame F contains the picture function that is
  digitized to form the digital image I.

36
World coordinate frame

• The world coordinate frame is needed to relate
  objects in 3D.
• Humans robots etc operate based on the world
  coordinate system.
• For the next several chapters we will use the
  Pixel coordinate frame.

37
Surface data (2.5D) sensed by structured light
sensor

• Projector projects plane of light on object
• Camera sees bright points along an imaging ray
• Compute 3D surface point via line-plane
  intersection

38
Magnetic Resonance Imaging

• Sense density of certain chemistry
• S slices x R rows x C columns
• Volume element (voxel) about 2mm per side
• At left is shaded image created by volume
  rendering

39
Single slice through human head

• MRIs are computed structures, computed from many
  views.
• At left is MRA (angiograph), which shows blood
  flow.
• CAT scans are computed in much the same manner
  from X-ray transmission data.

40
Other variations

• Microscopes, telescopes, endoscopes,
• X-rays radiation passes through objects to
  sensor elements on the other side
• Fibers can carry image around curves in bodies,
  in machine tools
• Pressure arrays create images (fingerprints,
  butts)
• Sonar, stereo, focus, etc can be used for range
  sensing (see Chapters 12 and 13)