Using FPGAs to Supplement RayTracing Computations on the Cray XD1

1
Using FPGAs to Supplement Ray-Tracing
Computations on the Cray XD-1

• Charles B. Cameron

United States Naval Academy Department of
Electrical Engineering United States Naval
Academy 105 Maryland Avenue, Stop 14B Annapolis,
Maryland 21402-5025

• Research supported by
• NASA Goddard Space Flight Center (Code 586)
• NRL Applied Optics Branch (Code 5630)
• DoD High Performance Computing Modernization
  Program at NRL (Code 5593)
• United States Naval Academy
• Xilinx, Inc.

2
Topics

• Ray tracing
• Conventional parallel processing
• Modulo scheduling
• Coordination of sequential and parallel
  processing
• Expected Performance

3
Ray tracing

• MODIS
• Moderate-resolution Imaging Spectroradiometer
• The Intersection Problem
• Finding the Perpendicular
• Refraction
• Reflection

4
MODIS Optical System (Moderate-resolution
Imaging Spectroradiometer)
5
MODIS Optical System

• 485 pinholes
• 400 rays per pinhole
• 241 121 rays reflected from the diffuser
• 5.66 109 rays

6
Ray Directed to a Surface

• MODIS
• Moderate-resolution Imaging Spectroradiometer
• The Intersection Problem
• Finding the Perpendicular
• Refraction
• Reflection
• Coordinate Transformation

7
Calculate the Intercept Point

• MODIS
• Moderate-resolution Imaging Spectroradiometer
• The Intersection Problem
• Finding the Perpendicular
• Refraction
• Reflection
• Coordinate Transformation

8
Find the Normal

• MODIS
• Moderate-resolution Imaging Spectroradiometer
• The Intersection Problem
• Finding the Perpendicular
• Refraction
• Reflection
• Coordinate Transformation

9
Find the Refracted Ray

• MODIS
• Moderate-resolution Imaging Spectroradiometer
• The Intersection Problem
• Finding the Perpendicular
• Refraction
• Reflection
• Coordinate Transformation

10
Find the Reflected Ray

• MODIS
• Moderate-resolution Imaging Spectroradiometer
• The Intersection Problem
• Finding the Perpendicular
• Refraction
• Reflection
• Coordinate Transformation

11
Coordinate Transformation

• MODIS
• Moderate-resolution Imaging Spectroradiometer
• The Intersection Problem
• Finding the Perpendicular
• Refraction
• Reflection
• Coordinate Transformation
• (Hard to visualize this!)

12
Topics

• Ray tracing
• Conventional parallel processing
• Modulo scheduling
• Coordination of sequential and parallel
  processing
• Expected Performance

13
Parallelism
14
Performance (5.66 109 rays)

99.998
5,857
Rate based on a linear regression of results
obtained using a varying numbers of processors.
15
Performance (5.66 109 rays)
16
Efficiency
17
Topics

• Ray tracing
• Conventional parallel processing
• Modulo scheduling
• Coordination of sequential and parallel
  processing
• Expected Performance

18
Operations Required as a Function of Surface,
Aperture, and Interaction Types
Lots of these
Not too many of these
19
Quadratic Equation
Latency
Critical Path (Data-Flow Limit) 88 cycles
20
Modulo SchedulingOne Multiplier
21
Modulo SchedulingOne Multiplier
22
Modulo SchedulingOne Multiplier
23
Modulo SchedulingOne Multiplier
24
Modulo SchedulingOne Multiplier
25
Modulo SchedulingOne Multiplier
26
Modulo SchedulingOne Multiplier
27
Modulo SchedulingOne Multiplier
Equal to the Data-Flow Limit
28
Modulo SchedulingFilling the Pipeline
One collective computation
29
Modulo SchedulingFilling the Pipeline
30
Modulo SchedulingFilling the Pipeline
Multipliers are 100 utilized
No schedule conflicts
31
Modulo SchedulingTwo Multipliers
Two multipliers with two multiplications each
32
Modulo SchedulingTwo Multipliers
One adder with two additions
Two cycles
Maximum efficiency
33
Modulo SchedulingTwo Multipliers
Improved efficiency Up from 25
34
Modulo SchedulingTwo Multipliers
35
Modulo SchedulingTwo Multipliers
36
Modulo SchedulingTwo Multipliers
Less than the Data-Flow Limit
37
Modulo SchedulingTwo Multipliers
Less than the Data-Flow Limit, but double the
throughput.
38
Topics

• Ray tracing
• Conventional parallel processing
• Modulo scheduling
• Coordination of sequential and parallel
  processing
• Expected Performance

39
Cray XD-1

• MPI (Message Passing Interface)
• Master node
• Reads file
• Distributes file
• Collates results

40
One Node of the Cray XD-1

• Open MP (Multi Processing)
• 144 of 220 nodes have a Xilinx Virtex II Pro FPGA
• Opteron processors
• Sequential program
• Depth first
• FPGA
• Pipelined hardware
• Breadth first

41
Topics

• Ray tracing
• Conventional parallel processing
• Modulo scheduling
• Coordination of sequential and parallel
  processing
• Expected Performance

42
Performance
43
Performance
44
Performance
45
Performance
46
Summary

• Modulo scheduling produces 100 efficiency of
  critical resources.
• Sequential processors get a boost from
  supplemental FPGA processing.
• Deep pipelines are efficient only if filled much
  of the time.
• FPGAs beat ASICs only if they can take advantage
  of special problem knowledge.
• Opteron uses 55 W.
• Virtex II Pro FPGA uses 4 W to 45 W.

47
Equations

• Intersection of a Ray with a Plane
• Intersection of a Ray with a Sphere
• Intersection of a Ray with a Conicoid
• Finding the Perpendicular
• Interaction of a Ray with an Optical Surface
• Coordinate Transformations

48
Intersection of a Ray with a Plane
Point in the plane
Initial direction
Final point
Initial point
Normal to the plane
List of equations
49
Intersection of a Ray with a Sphere
Initial direction
Initial point
Final point
List of equations
50
Intersection of a Ray with a Conicoid
Final point
Initial point
Initial direction
List of equations
51
Finding the Perpendicular
Unit Vector Normal to a Sphere
Unit Vector Normal to a Conicoid
List of equations
52
Interaction of a Ray with an Optical Surface
Refraction
Reflection
Initial index of refraction
Final index of refraction
Normal to the plane
Initial direction
List of equations
Final direction
53
Coordinate Transformations
Position in Frame of Reference k
Position in Frame of Reference k1
Rotation and Translation
Rotation
Direction in Frame of Reference k1
Rotation Matrix
Direction in Frame of Reference k
Translation Vector
List of equations