Theoretical Foundations of Multimedia

1
Theoretical Foundationsof Multimedia

• Dr. Robert S. Tannenbaum
• Director, Academic Computing Services
• University of Kentucky
• rst_at_pop.uky.edu

2
Hardware that Enables Multimedia

• Input and Output Devices
• Virtual Reality Devices
• Modems and Network Interfaces

3
Input and Output Devices
Output

• Monitors
• Speakers and MIDI interfaces
• VR helmets and immersive displays
• Keyboards and OCR devices
• Digital cameras, scanners, CD-ROMs
• MIDI keyboards and microphones
• Video cameras and frame grabbers
• Mice, track balls, joysticks, and VR gloves
  and wands

Input
4
Monitors
A simplified cathode ray tube (CRT)
5
Monitors

• Pixel a picture element a dot of color on
  the screen
• Three different phosphors at each pixel to
  create the color
• RGB (Red, Green, Blue)
• CYM (Cyan, Yellow, Magenta)

6
Monitors
Raster Scanning
7
Monitors

• Refresh rate the frequency at which the
  phosphors are excited
• Normally the refresh rate is given in Hertz
• For flicker-free images 75 Hz or faster is
  desirable
• The refresh rate for a projector needs to be
  coordinated with the monitor

8
Monitors

• The digitized image to be displayed must be
  stored in a buffer
• The stored image is said to be bit-mapped,
  because, for monochrome images, the map used
  just one bit per pixel
• Multimedia monitors use 24 bits per pixel (8
  for each color) can define 16 million colors

9
A Good Multimedia Monitor

• Large enough for comfortable viewing, probably
  15 or greater
• Pixel size of no more than 0.28mm
• Refresh rate of at least 75 Hz
• Capable of displaying 24-bit color
• Designed for the CPU and operating system
• Ergonomically comfortable and attractive

10
Speakers and MIDI Interfaces

• Storage of digitized sound files
• Reproduction via digital-to-analog conversion
  sent to a loudspeaker
• Built-in speakers often do not have sufficient
  fidelity
• Low-powered (3- to 5-watt) external speakers or
  head-phones will serve a single user and provide
  excellent fidelity

11
Speakers and MIDI Interfaces

• Storage of synthesizer command files
• Create the sounds by sending the commands to a
  synthesizer
• Musical Instrument Digital Interface (MIDI)
  standard (1982)
• MIDI includes both a hardware and a message
  standard

12
Speakers and MIDI Interfaces

• MIDI hardware standard defines cables,
  connectors, circuits, and electrical signals
• MIDI message standard defines a
• Device number for multiple device systems
• Control segment that gives an instrument-specific
  command
• Data segment containing the information needed by
  that instrument for that command

13
Alphanumeric Keyboards

• For entering commands, text, and data
• Each key is a switch that closes when it is
  depressed, sending a code to the CPU
• The arrangement of the keys may vary
• The most common is QWERTY
• Another arrangement is Dvorak

14
Choosing a Keyboard

• Does it include all of the needed characters,
  including command keys?
• Is it ergonomically comfortable and safe,
  preventing repetitive stress syndrome?

15
Optical Character Recognition (OCR)

• Hardware scans the text image
• Software systematically checks the entire
  image for patterns of light and dark that it
  recognizes as alphabetic, numeric, or punctuation
  characters
• OCR software entails pattern recognition, a
  sophisticated logic problem

16
Optical Character Recognition (OCR)
E e E E e e E E E e e E E E e e E
e e It is relatively easy for a human to
recognize each of these characters as the
letter e. For the pattern recognition
logic in OCR software, this is very
difficult.
17
Digital Cameras and Scanners

• Real Image a portion of what is physically
  present in nature
• Digital Image a representation of a real image
  in which individual points are encoded to
  represent the wavelength and intensity of light
  at that point
• Still Image a single snapshot of an instant
  may be real or digital
• Motion Image a sequence of images that, when
  viewed consecutively at the appropriate rate,
  gives the impression of continuous motion may be
  digital or analog

18
Scanners
Schematic Drawing of a Scanner
19
Digital Cameras
Schematic Drawing of a Digital Camera
20
Digital Cameras and Scanners
Image quality depends on the

• Quality of the optics and the scanning
  mechanism, which determines focus
• Precision of the photosensitive cells, which
  determines the accuracy of the encoding of
  intensity and wavelength data
• Resolution of the instrument in dots per inch,
  which determines graininess
• Amount of storage available, which determines
  the total size of an image that can be digitized

21
Inputting Images
Assuming no compression, 24 bits per pixel
Memory required to store a 5 x 7 snapshot
22
Video Cameras and Frame Grabbers

• Video cameras are similar to digital cameras
• Except that a video camera takes image after
  image continuously
• The output from many video cameras is analog and
  requires digitizing circuitry to make the image
  usable in a computer
• Digital camcorders are now available
• Frame grabber software allows the capture of a
  single still image from the video stream
• Frame grabbed images are of rather low
  resolution, however,

23
Microphones and MIDI Keyboards

• For input of sound
• Microphones capture sound waves from the air as
  an analog signal
• The analog signal must be digitized to be stored
  and then replayed by the computer
• Digitizing at for speech 20,000-40,000 Hz is needed for
  music
• MIDI keyboards usually look like piano
  key- boards with extra switches and controls
• MIDI keyboards encode and transmit musical
  information according to the MIDI standard