Developing Gears of War in Unreal Engine 3

1
Developing Gears of War in Unreal Engine 3

• Michael Capps, Ph.D.
• President, Epic Games Inc.

2
Outline

• Epic and the Unreal Engine
• A trip down the UE3 rendering pipeline
• Preview of multithreaded renderer
• Optimization techniques for Gears of War
• Lessons Learned

3
Epic Games

• Founded 15 years ago by CEO Tim Sweeney
• About 70 employees (18 engine devs)
• Early success with Jill of the Jungle and Epic
  Pinball
• Multiple hits in the Unreal and Unreal Tournament
  series
• Currently two titles in production

4
In production at Epic
Unreal Tournament 2007 PC, PS3, Xbox 360
Gears of War Xbox 360
5
Unreal Engine History

• In active development for 10 years
• Licensed to external developers since 1997
• More than 50 games have used Unreal Engine
• Unreal Engine 1 (1997-2000)
• PC First-Person Shooters
• Deus Ex, Rune, Undying, Harry Potter
• Unreal Engine 2 (2000-2004)
• PC, Xbox, PS2, Mac, Linux mostly First-Person
  Shooters
• Splinter Cell, Americas Army, Lineage II,
  Rainbow 6

6
Unreal Engine 3

• Unreal Engine 3 (2006-??)
• next-generation consoles and PCs
• 100 shader-driven rendering pipeline
• Per-pixel lighting shadowing everywhere
• No legacy rendering paths
• supports any game type, even massive multiplayer
  and fighting games
• work began 3.5 years and 40 man years ago
• Game Developer Magazine award for best
  engine/middleware 2 years running

7
UE3 Games in Development

• Many unannounced massive-multiplayer and next-gen
  console games
• Studio-wide or multiple-license deals
• Atari
• Disneys Buena Vista Games
• Gearbox Software
• Microsoft
• Midway
• Real Time Worlds
• Silicon Knights
• Vivendi

8
UE3 Announced Projects

• 10tacle Studios Elveon
• Bioware Mass Effect
• Buena Vista Games Turok franchise game
• Mistwalker Lost Odyssey
• Namco Frame City Killer
• NC Soft Exteel
• Real Time Worlds All Points Bulletin
• Silicon Knights Too Human
• WebZen Huxley

9
Consumer expectations rising
560 polygons 4x 256 textures
2,500 polygons 2x 1024 textures
12,000 base 2x 2048 textures
12,000 base 2x 2048 textures
and 2,000,000 high
10
Consumer expectations rising

• Its all about the graphics!
• runVideo( GoW-caves.avi )

11
Next-Gen is Expensive

• What is the actual impact of this detail?
• Cant go from 2.5k to 2m poly chars for free
• Cant go from 100k to 100m scenes for free
• And oh, yeah, we have to draw all that
• Heres how we do it!

12
Outline

• Epic and the Unreal Engine
• A trip down the UE3 rendering pipeline
• Preview of multithreaded renderer
• Optimization techniques for Gears of War
• Lessons Learned

13
The Rendering Pipeline

• General Characteristics
• Usage Cases
• Scene Rendering
• Materials / Shaders
• Lighting and Shadowing
• Postprocessing Effects

14
The Rendering Pipeline

• All rendering is HDR
• 7E3 on Xbox 360, FP16 on PC
• All lighting and shadowing options are orthogonal
• Frequent use of deferred rendering techniques
• Very thorough next-gen feature set

15
Usage Cases

• City-like environments
• Gears of War, Frame City Killer
• approximately 4 million GTA games
• Large outdoor environments
• Unreal Tournament 2007, Lost Odyssey
• dozens of MMOs all also GTA clones
• High-detail indoor environments
• Low system spec games (broad appeal, etc.)
• Supporting all these cases requires a
  general-purpose rendering engine

16
Scene Rendering

• Three primary stages
• Z pre-pass
• One pass for all precomputed lighting
• One pass per dynamic light

17
Scene Rendering

• Z pre-pass
• a.k.a. depth set-up pass
• Uses fast hardware path - no shading
• Complete Z-buffer generated for all opaque
  objects
• Per-object hardware occlusion queries to cull the
  later (expensive) shader rendering

18
Scene Rendering

• One pass for all precomputed and emissive
  lighting.
• For each object, combine
• Directional Light Maps lighting
• Apply three 3-component directional light map
  textures to materials normal map
• Emissive materials
• Materials may produce light independent of any
  light sources

19
Scene Rendering

• One pass per dynamic light
• Render shadowing to stencil, alpha
• Render stencil shadows to stencil buffer
• Render soft shadow-buffer shadows to
  alpha-channel
• We render a screen-space quad over the screen
  extent affected by the shadow
• Does not require re-rendering objects affected by
  shadowing
• Deferred rendering makes shadowing cost dependent
  on of pixels potentially shadowed (not of
  objects)
• Render all objects affected by the light

20
Authoring shaders

• Artist-driven pipeline
• 100 realtime, visual tools
• Artists write shaders by linking Material
  Expressions
• Based on visual node-editing paradigm
• Visually connect color, alpha, and coordinate
  outputs

21
Materials/Shaders

• Programmers add functionality by coding new
  Material Expressions in C, HLSL
• Artists can create extremely complex materials
  using programmer-defined components
• HLSL code is generated on-the-fly in UnrealEd
• In-game shader code is precompiled statically
• No dynamic shader compilation in-game
• No combinatorial explosion in shaders!
• Material Instance framework for reusable
  templates
• Parameters are 100 scriptable

22
Lighting Shadowing

• Fully orthogonal lighting shadowing system
• Artists choose lighting/shadowing technique per
  light
• Artists can customize light/shadow interactions
• Shadow culling options
• Lighting channels
• runVideo( flags.avi )

23
Lighting Techniques

• Light-mapped lights
• All lighting, shadowing is precomputed
• Stored in three (directional) 3-component DXT1
  textures
• Fast supports N lights in 1 pass
• Preserves normal-mapping detail
• Diffuse lighting only
• Real-time lights
• One rendering pass per light/object interaction
• Supports dynamic (view-dependent) specular
  highlights
• Dynamic shadows, light functions

24
Shadowing Techniques

• Static shadows
• Soft shadow occlusion precomputed into
  1-component DXT1 texture
• Light itself can be dynamic (and change in color,
  function, etc) it just cant move
• Dynamic stencil-buffer shadows
• Supports arbitrary moving lights
• Hard-edged

25
Shadowing Techniques

• Dynamic shadow-buffer shadows
• Soft shadowing via 16X oversampling
• Used per-object for shadows of moving objects
• Shadow extent is limited to light-object frustum,
  avoiding scalability issues inherent in
  full-scene shadow buffers

26
Postprocessing Effects

• Modular postprocessing system
• Artist-defined postprocessing passes
• Scriptable via Matinee cinematic engine, Kismet
  visual scripting system
• Components

27
(No Transcript)
28
Gratuitous Rendering Video

• runVideo( GoW-adamshouse.avi )

29
Outline

• Epic and the Unreal Engine
• A trip down the UE3 rendering pipeline
• Preview of multithreaded renderer
• Optimization techniques for Gears of War
• Lessons Learned

30
Multithreaded Systems

• Two heavyweight threads in Unreal Engine 3
• Main thread
• Gameplay updates
• Script execution
• High-level object management
• Catch-all for non performance-critical systems
• Rendering thread
• Perform all scene traversal
• Issue all rendering commands
• Dispatch smaller tasks to helper threads

31
Multithreaded Systems

• Pool of helper threads for lightweight tasks
• Stencil shadow extrusion
• Physics solver
• Data streaming
• Audio
• Coming soon
• Animation updates
• Particle systems

32
New Renderer Preview

• Goals
• If it aint broke, fix it
• Allow rendering to occur in parallel with game
  simulation
• Minimize rendering time for static primitives,
  exploiting precompiled command buffers
• Use a thin Render Hardware Interface (RHI)
  platform layer

Game Systems
Rendering command queue
33
Renderer Details

• Parallel rendering
• Rendering thread runs up to a frame behind the
  game thread
• Game thread updates the scene rendering
  structures
• Rendering resources consume the rendering command
  queue
• Rendering command queue serializes updates to
  rendering thread state with commands to render
  the current state

34
Renderer Details

• Static primitive rendering
• Optimized for platforms that support RHI command
  lists
• We store a RHI command list for each static
  primitive, in different contexts
• The right command list for an object, for the
  right context, is called by the scene traversal
  code

35
Renderer Details

• Render Hardware Interface (RHI)
• A platform independent interface to the
  platform's rendering hardware
• Deals with resources, rendering command, and
  command list resources
• Statically bound to minimize call overhead
• Minimal set of features exposed to implement
  UnrealEngine3's renderer

36
Outline

• Epic and the Unreal Engine
• A trip down the UE3 rendering pipeline
• Preview of multithreaded renderer
• Optimization techniques for Gears of War
• Lessons Learned

37
Lighting Shadowing inGears of War

• Typical scene
• 1-3 hemispherical lights per visible scene
• 20-50 static light-mapped lights per scene
• 1-3 real-time lights per scene
• Artists make tradeoffs between
• Scene complexity
• Dynamic lighting complexity
• Shadowing complexity

38
Performance Tools

• Artists need help to make tradeoff decisions
• Design-time tools
• Level summary statistics
• Per-object statistics
• Number of sections, vertices, triangles, etc
• Memory footprints
• Light/object interactions
• Shadow statistics
• Lighting complexity visualization
• In-Game tools
• Dynamic performance statistics displays
• Memory usage diagnostics

39
Lighting Complexity Visualization

• runVideo( fast-slow.avi )

40
Lighting Complexity Visualization

• Green
• object forces light maps for all lights and does
  not cast dynamic shadows
• Good for unreachable objects
• Yellow
• Receives shadows but does not cast them
• Good for floors, etc., saves unneeded shadow work
• Red
• Only uses light maps if instructed to do so by
  light
• Slowest, most flexible, most correct

41
Lighting Complexity Searches

• Visualization helps only for objects you can see
• Need a way to search for objects by complexity
• Direct cost (triangles, sections, instances,
  etc.)
• Average of lights affecting instances of an
  object
• Triangle cost, a factor of geometric complexity
  and number of passes
• Memory tools (light map resolution, etc.)

42
Outline

• Epic and the Unreal Engine
• A trip down the UE3 rendering pipeline
• Preview of multithreaded renderer
• Optimization techniques for Gears of War
• Lessons Learned

43
Lessons Learned

• One Unified Shadowing Solution scales poorly to
  large-scale game development
• Instead, want many lighting shadowing options
• Must make tradeoffs
• Static vs. Dynamic lighting
• Static vs. Dynamic shadowing
• Soft vs. hard shadow edges (stencil vs shadow
  buffer)
• Scene complexity vs. dynamic lighting/shadowing
  complexity
• Disabling shadows when nonessential to visual
  quality
• Must expose lighting shadowing options
  orthogonally so different tradeoffs can be chosen
  in a single scene

44
Lessons Learned

• Empowering artists to make tradeoffs requires
• Great artist-tools for measuring, understanding
  performance
• Greater emphasis on technical artist role on
  project
• Trusting your artists!
• Make the default options fast
• static lighting, precomputed shadows, etc.
• Force designer to explicitly choose to improve
  visual quality at the expense of performance
• Optimization isnt just a code problem anymore
• Learned this lesson in X05 crunch
• Most of the vital performance tradeoffs are in
  content now

45
Lessons Learned

• Make all rendering features scriptable
• In cinematics
• By artists, using visual scripting tools
• By programmers, via C or scripting language
• In-engine level design tools rule!
• Artists thrive on real-time visual feedback
• Dont want to spend 30 days building a level in
  Mayabefore bringing it in-engine and seeing how
  it performs!
• And of course

46
Lessons Learned

• Next-gen engine development is hard!
• We spent 40 man-years for our full-featured
  next-generation engine
• So you should use UE3 instead of building your
  own -)
• Any questions?