Title: Andries van Dam September 5, 2000 Introduction to Computer Graphics 17
1Introduction
2What is Computer Graphics? (1/2)
- Computer graphics generally means creation,
storage and manipulation of models and images
- Such models come from diverse and expanding set
of fields including physical, mathematical,
artistic, biological, and even conceptual
(abstract) structures
Frame from animation by William Latham, shown at
SIGGRAPH 1992. Latham uses rules that govern
patterns of natural forms to create his artwork.
3What is Computer Graphics? (2/2)
- William Fetter coined term computer graphics in
1960 to describe new design methods he was
pursuing at Boeing
- Created a series of widely reproduced images on
pen plotter exploring cockpit design, using 3D
model of human body.
Perhaps the best way to define computer graphics
is to find out what it is not. It is not a
machine. It is not a computer, nor a group of
computer programs. It is not the know-how of a
graphic designer, a programmer, a writer, a
motion picture specialist, or a reproduction
specialist. Computer graphics is all these a co
nsciously managed and documented technology
directed toward communicating information
accurately and descriptively.
Computer Graphics, by William A. Fetter, 1966
4What is Interactive Computer Graphics? (1/3)
- User controls contents, structure, and appearance
of objects and their displayed images via rapid
visual feedback
- Basic components of an interactive graphics
system
- input (e.g., mouse, tablet and stylus, force
feedback device, scanner, live video streams)
- processing (and storage)
- display/output (e.g., screen, paper-based
printer, video recorder, non-linear editor)
- First truly interactive graphics system,
Sketchpad, pioneered at MIT by Ivan Sutherland
for his 1963 Ph.D. thesis
Sketchpad in 1963. Note use of a CRT monitor,
light pen and function-key panel.
5What is Interactive Computer Graphics? (2/3)
Batch (1950s now)
- Before Sketchpad, output via plotters/printers,
input via keypunch, both in batch
Card punching (left). IBM 704 (right) took up a
whole room and was capable of about 4,000
arithmetic operations/second. Cool facts Whirlwi
nd, built in early 50s at MIT, cost 4.5 million
and could perform 40,000 additions/second. Mac
512K, list price 3,195 in 1984, could do
500,000. Today, commodity PCs perform
approximately two or three billion
operations/second.
6What is Interactive Computer Graphics? (3/3)
- Almost all key elements of interactive graphics
system are expressed in first paragraph of
Sutherlands 1963 Ph.D. thesis, Sketchpad, A
Man-Machine Graphical Communication System
The Sketchpad system uses drawing as a novel
communication medium for a computer. The system
contains input, output, and computation programs
which enable it to interpret information drawn
directly on a computer display. Sketchpad has
shown the most usefulness as an aid to the
understanding of processes, such as the motion of
linkages, which can be described with pictures.
Sketchpad also makes it easy to draw highly
repetitive or highly accurate drawings and to
change drawings previously drawn with it
- Today, still use batch mode for final
production-quality video and film (special
effects fx), where one frame of a 24 fps movie
may take 8-24 hours to render on fastest PC!
Render farm
7Environmental (R)evolution (1/6)
- Graphics has been key to technology growth in
evolution of computing environments
- graphical user interfaces (GUIs)
- visual computing, e.g., desktop publishing,
scientific visualization, information
visualization
- Hardware revolution drives everything
- every 12-18 months, computer power improves by
factor of 2 in price / performance Moores Law
- Palm TX, HP I-Paq as full PC
- iPhone, Blackberry for email/internet
- Hallmark singing card, LeapFrog Pad
- graphics memory and network speeds are on even
faster exponentials
- Graphics chips in particular have major
improvements every six to nine months (e.g.
nVidia GeForce series, ATI Radeon series)
8Environmental (R)evolution (2/6)
- Character Displays (1960s now)
- Display text plus alphamosaic pseudo-graphics
- Object and command specification command-line
typing
- Control over appearance coding for text
formatting (.p paragraph, .i 5 indent 5)
- Application control single task
9Environmental (R)evolution (3/6)
- Vector (Calligraphic, Line Drawing)
- Displays (1963 1980s)
- Display line drawings and stroke text 2D and 3D
transformation hardware
- Object and command specification command-line
typing, function keys, menus
- Control over appearance pseudo-WYSIWYG
- Application control single or multitasked,
distributed computing pioneered at Brown via
mainframe host minicomputer satellite
- Term vector graphics survives as scalable
vector graphics library from Adobe and W3C
shapes as transformable objects rather than just
bitmaps
10Environmental (R)evolution (4/6)
- 2D bitmap raster displays for PCs and
workstations
- (1972 at Xerox PARC - now)
- Display windows, icons, legible text, flat
earth graphics
- Note late 60s saw first use of raster
graphics, especially for flight simulators
- Object and command specification minimal typing
via WIMP (Windows, Icons, Menus, Pointer) GUI
point-and-click selection of menu items and
objects, widgets and direct manipulation (e.g.,
drag and drop), messy desktop metaphor - Control over appearance WYSIWYG (which is really
WYSIAYG, What You See Is All You Get)
- Application control multi-tasking, networked
client-server computation and window management
(even X terminals)
11Environmental (R)evolution (5/6)
- 3D graphics workstations (1984 at SGI now)
- Display real-time, pseudo-realistic images of 3D
scenes
- Object and command specification 2D, 3D and nD
input devices (controlling 3 degrees of freedom)
and force feedback haptic devices for
point-and-click, widgets, and direct
manipulation - Control over appearance WYSIWYG (still WYSIAYG)
- Application control multi-tasking, networked
(client/server) computation and window
management
- High-end PCs with hot graphics cards (nVidia
GeForce, ATI Radeon) have supplanted graphics
workstations
- Such PCs are clustered together over high speed
buses or LANs to provide scalable graphics to
drive tiled PowerWalls, Caves, etc.
12Environmental (R)evolution (6/6)
- Classical time-sharing is dead, 1n g n1
- PCs and Workstations merging in distributed
heterogeneous computer networks (e.g., LANs,
WANs, Internet and clusters)
- But file-, print- and compute-servers and network
are still shared
- Client/server computing, component software
technologies are dominant paradigms
- NCs (Network Computers), thin clients attached to
powerful servers reprise dumb terminals and
provide central control havent really taken off
13Versus
Original Macintosh
New iMac 24
2008
1984
Date
24
x .88
2500
2200
Price
CPU
x 382.5
8 MHz
3.06 GHz (Dual)
Memory
x 15625
128KB RAM
2.0GB DDR2 SDRAM
Storage
400KB Floppy
500GB Hard Disk
x 1250000
24 Color
9 Black White
x 2.6
Monitor
1920 x 1200
512 x 342
x 13.2
100 dpi
x 1.5
68 dpi
Mouse
Mouse
same
Devices
Keyboard
Keyboard
same
Desktop WIMP
Desktop WIMP
same
GUI
14New Forms of Computing1990-2008 (1/9)
- Multimedia text and graphics synchronized with
sound and video
- Hypermedia multimedia with hypertextual links
(also called Interactive Multimedia)
- Discredited term Digital Convergence, merging
of digital television and distributed computing,
consumer electronics set-top computers (e.g.,
for Interactive TV, Video-On-Demand) - The Internet and Internet appliances
- Embedded computing (information appliances,
Personal Digital Assistants)
- Ubiquitous/pervasive/ invisible/nomadic
computing, active badges a la Xerox PARC, with
hundreds of devices per person (appliances,
clothing, even body parts) seamless computing
is the dream
15New Forms of Computing1990-2008(2/9)
fully immersive VR(via Head-mounted Displays,
Cave 180 George St.)
CAVE
16New Forms of Computing1990-2008 (3/9)
VOX in Cave and Fishtank VR
17New Forms of Computing1990-2008 (4/9)
ADVISER Mars data visualization
18New Forms of Computing1990-2008 (5/9)
Arterial blood flow and bat flight visualization
19New Forms of Computing1990-2008 (6/9)
Cave Painting
Use feet for navigation, freeing hands for other
uses
20New Forms of Computing1990-2008 (7/9)
Elumens VisionStation
Fishtank VR on a monitor small volume, head-track
ed interactive stereo
21New Forms of Computing1990-2008 (8/9)
- augmented VR(via video see-through optics)
Video or optics superimposes computer-generated
data on real world (e.g., Columbias MARS, led by
our Ph.D., Steve Feiner)
22New Forms of Computing1990-2008 (9/9)
- New Interaction technology
- Inexpensive interaction devices from research lab
into marketplace 2D and 3D graphics no longer
special
- 3D (even time-varying, 4D) interactive
illustrations as clip art/clip models coming
- Kids using computer graphics in rides (e.g.
Aladdin, Pirates of the Caribbean) and gaming
consoles (e.g. Nintendo Wii), with head-mounted
displays and/or force-feedback input devices - New forms of user-interface (UI Lecture)
- 3D Widgets VR demands new interaction
technology
- Gesture-based Interaction (Browns Sketch)
- tablet PCs and electronic whiteboards present new
opportunities for pen-centric, gesture-based
computing
- Multi-touch (NYUs Jeff Han, MS Surface, Apples
iPhone)
- Social interfaces (Microsofts Bob and Clippie
bombed, other avatars may not)
- Agents for indirect control
23Powerful, Inexpensive Processing
- Chips are Key in graphics subsystems
- Advances driven by Moores Law
- price/performance improves 2x every 18 months due
to doubling of number of transistors
- only exponential growth in technology except WWW
- CPU
- Newest processors are 64-bit, dual/quad/8 core
- Server Dual-Core Intel Itanium 2, Dual-Core
Intel Xeon, Dual-Core AMD Opteron, Sun
UltraSPARC T1
- Desktop Intel Core 2 Duo, AMD Athlon64 X2, IBM
G5, Mac ProTM Quad/8-Core
- Graphics subsystems (GPU)
- Commodity cards have taken over the mainstream
market (nVidia GeForce, ATI Radeon)
- Physics subsystems
- nVidia (formerly Ageia) PhysX PPU (Physics
Processing Unit)
- Offloads physics processing from the processor
and GPU
- Artificial Intelligence subsystems
- AIseek Intia Processor
- Accelerated path finding, sensory simulation and
terrain analysis
24Chip Technology Advances in Games Commodity
Graphics Cards
- Last-generation game platforms and set-top boxes
use high-end processors (128-bit architectures,
great graphics capabilities)
- Nintendo GameCube (powered by ATI)
- Sony Playstation 2
- Microsoft XBoxTM
- Current-generation platforms give us
unprecedented performance by utilizing multi-core
processors
- Microsoft Xbox 360 (powered by ATI)
- Sony Playstation 3
- Nintendo WiiTM
- ATI Radeon HD 4800 and nVidia GeForce GTX 280 are
the top graphics cards in 2008 (approx. 200 -
300)
- Significant advances in commodity graphics chips
every 6 months, outrunning CPU chip advances
- AMD Athlon 64 X2 processors have 243 million
transistors
- Radeon HD 4800 chip has over 900 million!
- GPUs are now arguably more powerful than CPUs and
programmable ? CPU OS losing their hegemony as
the GPU co-processor tail wags CPU dog!
- GPUs sometimes used instead of CPUs in clusters
for high-performance computing (e.g.
bio-informatics algorithms)
- Workstation market (and vendors) bowing to
commoditization (e.g., Suns losing client market)
25Application Distinctions
- Two basic paradigms
- Sample-based graphics (left) discrete samples
are used to describe visual information
- pixels can be created by digitizing images, using
a sample-based painting program, etc.
- often some aspect of the physical world is
sampled for visualization, e.g., temperature
across the US
- example programs Adobe Photoshop, GIMP , Adobe
AfterEffects (it came out of CS123/CS224!)
-
- Geometry-based graphics (right) (also called
scalable vector graphics or object-oriented
graphics) geometrical model is created, along
with various appearance attributes, and is then
sampled for visualization (rendering a.k.a image
synthesis) - often some aspect of physical world is visually
simulated, or synthesized
- examples of 2D apps Adobe Illustrator, Adobe
Freehand (formerly by Macromedia), Corel
CorelDRAW
- examples of 3D apps Autodesks AutoCAD2009,
Autodesks (formerly AliasWavefronts) Maya,
Autodesks (formerly Discreets) 3D Studio Max
26Sampled-based Graphics (1/2)
- Images are made up of grid of discrete pixels,
for 2D picture elements
light intensity
1 pixel
- Pixels are point locations with associated sample
values, usually of light intensities/colors,
transparency, and other control information. They
are not little circles or little squares
27Sampled-based Graphics (2/2)
- Samples created directly in paint-type program,
or as sampling of continuous (analog) visual
materials. E.g., photograph can be sampled (light
intensity/color measured at regular intervals)
with many devices including - flatbed and drum scanners
- digital still and motion (video) cameras
- add-on boards such as frame grabbers
- Sample values can also be input numerically
(e.g., with numbers from computed dataset)
- Once an image is defined as pixel-array, it can
be manipulated
- Image editing changes made by user, such as
cutting and pasting sections, brush-type tools,
and processing selected areas
- Image processing algorithmic operations that are
performed on image (or pre-selected portion of
image) without user intervention. Includes
blurring, sharpening, edge-detection, color
balancing, rotating, and warping. Pre-processing
step in computer vision
28Sampling an Image
- Lets do some sampling of CIT building
- A color value is measured at every grid point and
used to color corresponding grid square
- Note this poor sampling and image reconstruction
method creates blocky image
3D scene
0 white, 5 gray, 10 black
29Whats the Advantage?
- Once image is defined in terms of colors at (x,
y) locations on grid, can change image easily by
altering location or color values
- E.g., if we reverse our mapping above and make 10
white and 0 black, the image would look like
this
- Pixel information from one image can be copied
and pasted into another, replacing or combining
with previously stored pixels
30Whats the Disadvantage?
- WYSIAYG (What You See Is All You Get) No
additional information
- no depth information
- cant examine scene from different point of view
- at most can play with the individual pixels or
groups of pixels to change colors, enhance
contrast, find edges, etc.
- But recently, strong interest in image-based
rendering to fake 3D scenes and arbitrary camera
positions. New images constructed by
interpolation, composition, warping and other
operations.
Photo Tourism Exploring photo collections in 3D
(Siggraph 2006)
31Examples of 2D Image Manipulation (1/3)
- There are many things possible with sampled
images uses of computer imaging range from
military applications to entertainment, medicine,
art, and design.
- The news digitally enhanced images are more
and more common. Usually just sharpening, color
balancing, but sometimes much more. To Photoshop
something in - Time Magazines O.J. Simpson cover (below left)
- Reuters doctored Beirut bombing photo (below
right note cloned smoke)
- National Geographics Pyramid cover
- University of Wisconsin application
- Faked NE blackout satellite photo
32Examples of 2D Image Manipulation (2/3)
- People can now mutate into other people or
objects through morphing, and can carry on
conversations in different times and places
- Interactive Digital Photomontage, Siggraph 2004
- The belief in very strong connection between
photorealistic images, still or moving, and
reality is being severed
- no way to tell if news photos are real
photographs
- photographic evidence no longer considered
proof in court of law without clear provenance
of the image
- future of the family photo album?
33Examples of 2D Image Manipulation (3/3)
- In artwork the processes and techniques of
photography and painting are merging in the art
of digital imaging
- Michele Turre the artist, her daughter, and her
mother, all at 3 years of age
- CS123 emphasizes geometry-based graphics but we
do two assignments on sample-based graphics to
learn simple image processing.
- image processing in general includes image
transformation used for feature detection,
pattern recognition, machine/computer vision, and
most recently, for image-based rendering - new field of "digital photography" (a.ka.
computational photography, fauxtography) brings
us many neat tricks with images, some now even
done in camera
34Geometry-Based Graphics
- Geometry-based graphics applications store
mathematical descriptions, or models, of
geometric elements (lines, polygons,
polyhedrons) and associated attributes (e.g.,
color, material properties). Elements are
primitive geometric shapes, primitives for short - Images created as pixel arrays (via sampling of
geometry) for viewing, but not stored as part of
model. Images of many different views are
generated from same model - Users cannot usually work directly with
individual pixels in geometry-based programs as
user manipulates geometric elements, program
resamples and redisplays elements - Increasingly rendering combines geometric and
sample-based graphics, both as performance hack
and to increase quality of final product
35What is Geometric Modeling?
- What is a model?
- Captures salient features (data, behavior) of
thing/phenomenon being modeled
- data includes geometry, appearance attributes
- note similarity to OOP notions
- Real some geometry inherent
- physical (e.g., actual object such as a pump)
- non-physical (e.g., mathematical function,
weather data)
- Abstract no inherent geometry, but for
visualization
- organizational (e.g., company org. chart)
- quantitative (e.g., graph of stock market)
- Modeling is coping with complexity
- Our focus modeling and viewing simple everyday
objects
- Consider this
- Through 3D computer graphics, first time in
human history we have abstract, easily changeable
3D forms. This has revolutionized working process
of many fields science, engineering, industrial
design, architecture, commerce, entertainment,
etc. This has profound implications for visual
thinking and visual literacy
36Lecture Topics
- We manipulated primitive shapes with geometric
transformations (translation, rotation, scale).
These transformations are essential for model
organization, process of composing complex
objects from simpler components. - Hierarchical models and same geometric
transformations are also essential for animation
- Once objects geometry is established, must be
viewed on screen map from 3D to 2D for viewing
and from 2D to 3D for 2D input devices (e.g., the
mouse or pen/stylus) - While mapping from 3D to 2D, object (surface)
material properties and lighting effects are used
in rendering ones constructions. This rendering
process is also called image synthesis
37Decomposition of a Geometric Model
- Divide and Conquer
- Hierarchy of geometrical components
- Reduction to primitives (e.g., spheres, cubes,
etc.)
- Simple vs. not-so-simple elements (nail vs. screw)
Head
Shaft
Point
decomposition
composition
38Hierarchical (Tree) Diagram of Nail
- Object to be modeled is (visually) analyzed, and
then decomposed into collections of primitive
shapes.
- Tree diagram provides visual method of expressing
composed of relationships of model
- Such diagrams are part of 3D program interfaces
(e.g., 3D Studio MAX, Maya)
- As pointer data structure to be rendered, it is
called a scenegraph
root node
Nail
Head (cylinder)
Body
Shaft (cylinder)
Point (cone)
leaf nodes
tree diagram
39Composition of a GeometricModel
Translate
Translate and Scale
Translate and Rotate
Primitives in their own modeling coordinate syst
em
Composition in world (root) coordinate system
- Primitives created in decomposition process must
be assembled to create final object. Done with
affine transformations, T, R, S (as in above
example). - Other composition operators will be briefly
discussed later (e.g., Constructive Solid
Geometry (CSG) with Boolean operators).
40What Kind of Math do We Need?
- Cartesian Coordinates
- Typically modeling space is floating point,
screen space is integer
-
- NBOften, screen coordinates are measured top to
bottom, based on raster scan
-
Integer Grid
x, y Cartesian grid
(0,0)
41Example 3D Primitives
Polyhedron
Polyline
Patch
Sphere
42Conceptual Framework for Interactive Graphics
- Graphics library/package is intermediary between
application and display hardware (Graphics
System)
- Application program maps application objects to
views (images) of those objects by calling on
graphics library
- User interaction results in modification of model
and/or image
- Images are usually means to an end synthesis,
design, manufacturing, visualization,
- This hardware and software framework is more than
4 decades old but is still useful, indeed dominant
Graphics Library (GL)
Application model
Application program
43Graphics Library
- Examples OpenGL, DirectX, Windows Presentation
Foundation
- Primitives
- Attributes
- color
- line style
- material properties for 3D
- Lights
- Transformations
- Immediate mode vs. retained mode
- immediate mode no stored representation, package
holds only attribute state, and application must
completely draw each frame
- retained mode library compiles and displays from
scenegraph, a complex DAG
442D Hardware/Algorithms Outline
- Display Hardware Raster scan vs. random scan
- Drawing primitives by scan conversion
- Lines, polygons, circles and ellipses,
characters, attributes (color, line style, fill
pattern)
- Clipping to clip rectangle three methods
- analytically compute intersections and draw
clipped primitives
- test each pixel and write only if inside
- compute spans inside the primitive and fill
entire span without testing
- Color Table
- indirect specification of (pseudo) color
- color correction, simple types of animation
- BitBlt/RasterOp for operating on blocks of pixels
45Graphics Display Hardware
- Vector (calligraphic, stroke, random-scan)
- still used in some plotters
- Raster (TV, bitmap, pixmap), used in displays and
laser printers
46Vector Architecture
- Vector display works with display list/file
stored in refresh buffer
- Display controller draws all vectors at (often at variable rate) flicker is a problem
472D Raster Architecture
- Raster display stores bitmap/pixmap in refresh
buffer, also known as bitmap, frame buffer can
be in separate hardware (VRAM) or in CPUs main
memory (DRAM) - Video controller draws all scan-lines at
consistent
- 60 Hz separates update rate of the
frame buffer and refresh rate of the CRT
48Drawing Lines
- For horizontal, vertical and diagonal lines all
pixels lie on ideal line special case
- For lines at arbitrary angle, pick pixels closest
to ideal line (Bresenhams midpoint scan
conversion algorithm)
- For thick lines, use multiple pixels in each
column or fill a rotated rectangle
- Sampling continuous line on discrete grid
introduces sampling errors jaggies
49Drawing Filled Polygons
- Find intersection of scanline with polygon edges
- Sort intersections by increasing x
- Fill the polygon between pairs of intersections
(spans)
50Drawing Circles and Ellipses
circle outline
filled ellipse
51Thickness Attribute
- Thick circle drawing by tracing rectangular pen
- Treat such primitives as regions and fill them,
e.g., approximate them as sequences of connected
quadrilaterals
52Drawing Characters (1/2)
- For characters defined by small bitmap
- selectively write pixels of refresh buffer
corresponding to true bits of character bitmap
- Transparent mode dont write 0s BitBlt with
OR (explained later)
- Opaque mode write background color for 0s
- Descenders proportional spacing easily
accommodated
Base line
53Drawing Characters (2/2)
- Outline fonts defined in terms of (mathematical)
drawing primitives (lines, arcs, splines) and
thus scalable, but more CPU intensive (e.g. Adobe
PostScript, Microsoft TrueType) - Font design (typography is highly skilled
specialty, involving graphical and algorithmic
design)
541-bit Bilevel Display
- Digital intensity value Digital to Analog
Conversion (DAC) analog signal to drive
electron beam
- Black White (or any 2 colors, depending on
monitor phosphor color)
- Original Mac resolution was 512x384 pixels, now
from 640x480 up to 2560x1600
n-Bit Display
- 2n intensities or colors 1 (grayscale) or 3
(color) DACs guns
55Image Display SystemLook-up Table
- Any specific 2n colors may be inadequate (n
typically 16-24 in low-end systems)
- Look-up table allows 2n colors out of 224 colors
to be used in one image, some other 2n in another
image
- 224 approx. 16.7 million, exceeds eyes ability
to discriminate (somewhere between 7-10 million)
- Color table is resource managed (usually) by
window manager
56Color Look Up Table Operation
- Pixel value is indexed to color look up table
(CLUT) where color is stored. Here we use only
12 bits (4bits per color) for clarity
typically, 24 bits are used - CLUT look up done at video rates, overlapped with
fetch and DAC!
- In 24-bit true color systems, 3 x 8 bits for R,
G, B each color has its own 8-bit CLUT (0-255)
- CLUT allows variety of effects
- pseudo coloring (LandSat images, stress diagrams,
thermograms)
- fast image changes change table rather than
stored image
- multiple images select or composite/blend
- animation hack
57BitBlt/RasterOp (1/3)
- Logically operate on each pixel in rectangular
source and destination regions in same or
different pixmaps to achieve dynamics, e.g., to
move/scroll windows on screen - RasterOp (Source, Destination) g Destination
- In some implementations either S or D may be
masked, and need not be same size
pixmap
58BitBlt/RasterOp (2/3)
- AND (S,D) S can mask out pixels in D
- OR (S,D) S is non-destructively added to D
used for painting, transparent and kerned
characters (where characters extend beyond their
boxes) not as useful in n-bit systems
Heres how you can use them Lets say you want t
o add some game sprites to background
59BitBlt/RasterOp (3/3)
- Replace (S,D) S destructively replaces D, i.e.,
is deleted and copied on top of D (also called
Move) used for making opaque characters, icons,
scroll - Copy (S,D) as above, but S is not deleted
- XOR (S,D) S selectively inverts D used in 1-bit
systems for rubber-banding/dragging, cheap
cursors
- Note effects in color systems for all but
replace may be weird