Advanced%20Game%20Development%20with%20the%20Mobile%203D%20Graphics%20API - PowerPoint PPT Presentation

View by Category
About This Presentation
Title:

Advanced%20Game%20Development%20with%20the%20Mobile%203D%20Graphics%20API

Description:

Gain an insight to the Mobile 3D Graphics API and take your games to ... Winding, culling, shading. Perspective correction hint. Fades colors based on distance ... – PowerPoint PPT presentation

Number of Views:167
Avg rating:3.0/5.0
Slides: 67
Provided by: fivedots
Learn more at: http://fivedots.coe.psu.ac.th
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Advanced%20Game%20Development%20with%20the%20Mobile%203D%20Graphics%20API


1
Advanced Game Development with the Mobile 3D
Graphics API
  • Tomi Aarnio, Kari Pulli
  • Nokia Research Center

2
Gain an insight to the Mobile 3D Graphics API and
take your games to the next level
3
Prerequisites
  • Fundamentals of 3D graphics
  • OpenGL or some other modern 3D API
  • Java technology, preferably MIDP

4
Objectives
  • Get a quick overview of the whole API
  • Gain a deeper understanding of selected topics
  • Learn practical tricks not found in the spec

5
Agenda
  • Why should I care?
  • Getting started
  • Low level features
  • Bitmaps and texturing
  • Scene graph
  • Dynamic meshes
  • Animation

6
Agenda
  • Why should I care?
  • Getting started
  • Low level features
  • Bitmaps and texturing
  • Scene graph
  • Dynamic meshes
  • Animation

7
Mobile 3D Graphics API (M3G)
  • Also known as JSR-184
  • Designed for mobile devices
  • Primarily CLDC / MIDP
  • But also CDC

Midlets
Mobile 3D Graphics API
MIDP
Java Virtual Machine (CLDC 1.1)
Graphics Hardware
8
Overcome the performance barrier
Native (C/C) vs. Java on mobiles
Benchmarked on an ARM9 processor
9
Leverage hardware acceleration
  • Benefits for developers
  • Mask hardware differences (3D HW, FPU, DSP,)
  • Exploit any new hardware automatically
  • Benefits for hardware vendors
  • Gives a concrete target which features to offer

10
Speed up development cycles
  • Separate content from code
  • Quick prototyping without programming
  • Artists create looks and behavior
  • Export all in a binary file format
  • Load the file, populate scene graph
  • Commonly needed functionality built in
  • High-level functionality such as scene graph,
    keyframe animation

11
Why a new standard?
  • OpenGL (ES) is too low-level
  • Lots of Java code needed for simple things
  • Java 3D API is too bloated
  • A hundred times larger than M3G
  • Does not fit together with MIDP
  • Tried and failed, but
  • Now knew what we wanted!
  • Basic Java 3D ideas nodes, scene graph
  • Add file format, keyframe animation
  • Remain compatible with OpenGL ES

12
Agenda
  • Why should I care?
  • Getting started
  • Low level features
  • Bitmaps and texturing
  • Scene graph
  • Dynamic meshes
  • Animation

13
Key classes
3D graphics context Performs all rendering
Graphics3D
Can load individual objects and entire scene
graphs (M3G and PNG files)
Loader
Scene graph root node
World
14
Graphics3D
  • Contains global state
  • Target surface, viewport, depth buffer
  • Camera, light sources
  • Rendering quality hints
  • Each renderable object has its own local state
  • Geometry and appearance (material, textures,
    etc.)
  • Transformation relative to parent or world

15
Graphics3D Rendering modes
  • Retained mode
  • Render a scene graph, rooted by the World
  • Take the Camera and Lights from the World
  • Immediate mode
  • Render a branch or an individual node at a time
  • Explicitly give the Camera and Lights to
    Graphics3D

16
Graphics3D How-To-Use
  • Bind a target to it, render, release the target
  • Tip Never mess with a bound target
  • Tip Graphics3D is a singleton (threads!)

void paint(Graphics g) myGraphics3D.bindTarge
t(g) myGraphics3D.render(world)
myGraphics3D.releaseTarget()
17
Graphics3D Rendering targets
Can render to textures also
World
M3G
Graphics3D
Image2D
Graphics
Canvas
MIDP
Image
CustomItem
18
Things to keep in mind
  • Everything is synchronous
  • A method returns only when its done
  • No separate thread for renderer or loader
  • There are no callbacks
  • No abstract methods, interfaces, or events
  • Tip Do not even try to override any methods
  • Scene update is decoupled from rendering
  • render draws the scene, no side-effects
  • animate updates the scene to the given time
  • align applies auto-alignments, e.g.,
    billboards

19
Hello, World
A simplified animation player
import javax.microedition.midlet.MIDlet import
javax.microedition.lcdui.Display import
javax.microedition.lcdui.game.GameCanvas import
javax.microedition.m3g. public class Player
extends MIDlet public void pauseApp()
public void destroyApp(boolean b) public
void startApp() PlayerCanvas canvas new
PlayerCanvas(true) Display.getDisplay(this).s
etCurrent(canvas) try canvas.run()
catch (Exception e) notifyDestroyed()

20
Hello, World
class PlayerCanvas extends GameCanvas
PlayerCanvas(boolean suppress)super(suppress)
public void run() throws Exception
Graphics3D g3d Graphics3D.getInstance()
World w (World) Loader.load("/file.m3g")0
long start, elapsed, time 0 while
(getKeyStates() 0) start
System.currentTimeMillis()
g3d.bindTarget(getGraphics()) try
w.animate(time) g3d.render(w)
finally g3d.releaseTarget()
flushGraphics() elapsed
System.currentTimeMillis()-start time
(elapsed lt 100) ? 100 (int)elapsed if
(elapsed lt 100) Thread.sleep(100-elapsed)

21
Demo
  • Animation playback

21
22
Agenda
  • Why should I care?
  • Getting started
  • Low level features
  • Bitmaps and texturing
  • Scene graph
  • Dynamic meshes
  • Animation

23
Renderable Objects
Made of triangle strips Base class for meshes
(two subclasses)
Mesh
2D image placed in 3D space Good for labels, etc.
Sprite3D
24
Mesh
Composed of submeshes
  • Common buffer of vertex data
  • Submesh has triangle strip indices to vertices
  • One Appearance for each submesh

Mesh
VertexBuffer
coordinates
normals
colors
texcoords
25
Appearance components
Material colors for lighting Can track per-vertex
colors
Material
Blending, depth buffering Alpha testing, masking
CompositingMode
Winding, culling, shading Perspective correction
hint
PolygonMode
Fades colors based on distance Linear and
exponential mode
Fog
Texture matrix, blending, filtering Multitexturing
One Texture2D for each unit
Texture2D
26
Sprite3D
2D image with a position in 3D space
  • Scaled mode for billboards, trees, etc.
  • Unscaled mode for text labels, icons, etc.

Sprite3D
Appearance
CompositingMode
Image2D
Image2D
Fog
27
Rendering tricks
  • Use layers to impose rendering order
  • Appearance contains a layer index (integer)
  • Defines a global ordering for submeshes and
    sprites
  • Tip Disable z-buffering for sky boxes, use
    layers
  • Tip Maximize triangle strip length
  • Even if it requires adding degenerate triangles
  • Better benefits from vertex caching

28
Example Rotating cube
Define raw data for a cube
// Corners of a cube as (X,Y,Z) triplets static
short cubeVertices -1, -1, 1, 1, -1,
1, -1, 1, 1, 1, 1, 1, -1, -1, -1,
1, -1, -1, -1, 1, -1, 1, 1, -1 // A
color for each corner as an (R,G,B)
triplet static byte cubeColors (byte)
255, (byte) 0, (byte) 0, // red (byte)
0, (byte) 255, (byte) 0, // green //
Define the cube as a single triangle strip static
int indices 6,7,4,5,1,7,3,6,2,4,0,1,2,3
static int stripLen 14
29
Example Rotating cube
Copy raw data into Objects
// Fill in VertexArrays from short and
byte int numVertices vertices.length/3 Vertex
Array pos new VertexArray(numVertices, 3,
2) VertexArray col new VertexArray(numVertices,
3, 1) pos.set(0, numVertices,
cubeVertices) col.set(0, numVertices,
cubeColors) // Attach the VertexArrays to a
VertexBuffer // Note the scale (1.0) and bias
(0,0,0) vertices new VertexBuffer() vertices.se
tPositions(pos, 1.0f, null) vertices.setColors(co
l) // Fill in the triangle strip triangles
new TriangleStripArray(cubeIndices,
stripLen) // Create a Mesh with default
Appearance cube new Mesh(vertices, triangles,
new Appearance())
30
Example Rotating cube
Set up a Camera
Camera cam new Camera() // 60-degree field of
view, screen aspect ratio // Near clipping plane
at 1.0, far plane at 1000 float aspect (float)
getWidth() / (float) getHeight() cam.setPerspecti
ve(60.0f, aspect, 1.0f, 1000.0f) // Place the
camera at z5.0 in world space // View vector is
along the negative Z axis transform new
Transform() transform.postTranslate(0.0f, 0.0f,
5.0f) g3d Graphics3D.getInstance() g3d.setCam
era(cam, transform)
31
Example Rotating cube
Clear the buffers, render, animate
g3d Graphics3D.getInstance() g3d.bindTarget(get
Graphics()) try g3d.clear(null)
g3d.render(cube, transform) finally
g3d.releaseTarget() xAngle 1 yAngle 2
zAngle 3 transform.setIdentity() transform.po
stRotate(xAngle, 1.0f, 0.0f, 0.0f) transform.post
Rotate(yAngle, 0.0f, 1.0f, 0.0f) transform.postRo
tate(zAngle, 0.0f, 0.0f, 1.0f)
32
Demo
  • Rotating cube

32
33
Agenda
  • Why should I care?
  • Getting started
  • Low level features
  • Bitmaps and texturing
  • Scene graph
  • Dynamic meshes
  • Animation

34
Add bitmap graphics
  • 3D graphics is mostly vector graphics
  • Mathematical shape definitions
  • Define camera, lights, simulate
  • Computer graphics look, pretty boring
  • Bitmap graphics adds visual richness
  • Background images
  • Texture maps on surfaces
  • Sprites

35
Texturing tricks
  • Tip Use light maps for lighting effects
  • Usually faster than per-vertex lighting
  • Use luminance textures, not RGB
  • Multitexturing is your friend
  • Tip Use the perspective correction hint
  • Almost always faster than adding more vertices
  • But first check if the implementation supports
    it!

36
More bitmap tricks
  • Tip Use background images
  • Can be scaled, tiled and scrolled very flexibly
  • Generally much faster than sky boxes or similar
  • Tip Use sprites as impostors, particles
  • Generally (much) faster than textured quads
  • Unscaled mode is (much) faster than scaled
  • Tip Load your images with the Loader
  • Loader.load(/img.png) outputs Image2D
  • Going through MIDP Image is a waste of memory

37
Demo
  • Backgrounds and texturing

37
38
Agenda
  • Why should I care?
  • Getting started
  • Low level features
  • Bitmaps and texturing
  • Scene graph
  • Dynamic meshes
  • Animation

39
The scene graph
Actually, its just a tree
World
SkinnedMesh
Light
Group
Camera
Group
Group
MorphingMesh
Group
Mesh
Not allowed!
Group
Sprite
40
How to find an object in the scene
UserID defaults to 0
World
Background
find(4)
Group
Light
Camera
Image2D
4
Mesh
Group
Mesh
VertexArray
VertexBuffer
Mesh
Mesh
Appearance
Texture2D
IndexBuffer
Image2D
Diagram courtesy of Sean Ellis, Superscape
41
Node transformations
  • From this node to the parent node
  • Ignored for the root node
  • Composed of four parts
  • Translation T
  • Orientation R
  • Non-uniform scale S
  • Generic 3x4 matrix M
  • Composite C T R S M

C
World
C
Group
C
C
Group
Mesh
C
C
Group
Sprite
42
Node transformations
  • Tip Keep the transformations simple
  • Favor the T R S components over M
  • Avoid non-uniform scales in S
  • Tip How to scale along an arbitrary axis
  • There is no direct support (C T R N-1 S N M)
  • Method 1 R R N-1 and M N M
  • Method 2 Use an extra Group node
  • Tip How to rotate about an arbitrary point
  • Again, no direct support (C T P-1 R P S M)
  • Method 1 Use an extra Group node
  • Method 2 If S is identity, combine into T and M

43
Example Set up a hierarchy
private Group group1, group2 private Mesh cube1,
cube2, cube3 cube1 new Mesh(cubeVB,
triangles, appearance) cube2 new Mesh(cubeVB,
triangles, appearance) cube3 new Mesh(cubeVB,
triangles, appearance) group1 new
Group() group2 new Group() group1.addChild(cu
be1) group1.addChild(cube2) group1.addChild(grou
p2) group2.addChild(cube3)
group1
group2
cube1
cube2
cube3
44
Animate and render the hierarchy
g3d.render(group1, null) cube1.setOrientation(
yAngle, 0.0f, 1.0f, 0.0f ) cube1.setTranslation(
0.0f, 2.0f, 0.0f ) cube2.setOrientation(
-yAngle, 0.0f, 1.0f, 0.0f ) cube2.setTranslation(
0.0f,-2.0f, 0.0f ) group2.setOrientatio
n(-yAngle, 0.0f, 1.0f, 0.0f ) group2.setTranslati
on( -2.0f, 0.0f, 0.0f ) cube3.setOrientat
ion( zAngle, 0.0f, 0.0f, 1.0f ) cube3.setTranslat
ion( -2.0f, 0.0f, 0.0f ) cube3.setScale(
0.5f, 0.5f, 0.5f )
45
Demo
  • Hierarchic transformations

45
46
Agenda
  • Why should I care?
  • Getting started
  • Low level features
  • Bitmaps and texturing
  • Scene graph
  • Dynamic meshes
  • Animation

47
MorphingMesh
For vertex morphing animation
  • A base mesh and several morph targets
  • Result weighted sum of morph deltas
  • Can morph any vertex attribute
  • Vertex positions
  • Colors
  • Normals
  • Texture coordinates

48
MorphingMesh
Example Animating a rabbits face
49
SkinnedMesh
Articulated characters without cracks at joints
  • Stretch a mesh over a hierarchic skeleton
  • The skeleton consists of scene graph nodes
  • Each node (bone) defines a transformation
  • Each vertex is linked to one or more bones

50
SkinnedMesh
Weighted skinning
  • Neutral pose, bones at rest

51
SkinnedMesh
Weighted skinning
  • Bone B rotated 90 degrees

52
SkinnedMesh
Example Animating an arm
53
Agenda
  • Why should I care?
  • Getting started
  • Low level features
  • Bitmaps and texturing
  • Scene graph
  • Dynamic meshes
  • Animation

54
Animation Relevant classes
Storage for keyframes Defines interpolation mode
KeyframeSequence
Controls the playback of one or more animations
AnimationController
A link between sequence, controller and target
AnimationTrack
Base class for all objects that can be animated
Object3D
55
KeyframeSequence
Keyframe is a time and the value of a property at
that time Can store any number of
keyframes Several keyframe interpolation
modes Can be open or closed (looping)
KeyframeSequence
v
t
sequence time
Diagram courtesy of Sean Ellis, Superscape
56
AnimationController
Can control several keyframed animations
together Determines relationship between world
time and sequence time
AnimationController
world time
Diagram courtesy of Sean Ellis, Superscape
57
Animation
Apply value to animated property
Call toanimate(worldTime)
Calculate sequence time from world time
Identifies animated property on this object
Object3D
AnimationController
AnimationTrack
Relates animation controller, keyframe sequence,
and object property together.
KeyframeSequence
v
s
Look up value at this sequence time
Diagram courtesy of Sean Ellis, Superscape
58
Animation
  • Tip You can read back the animated values
  • Use the animation engine for anything you want
  • Much faster than doing interpolation in Java
  • Especially in case of quaternions and splines

59
Example Set up keyframes
KeyframeSequence createKeyframeSequence() //
10 keys, 3 components, spline interpolation
KeyframeSequence ks ks new
KeyframeSequence(10,3,KeyframeSequence.SPLINE)
// x grows linearly, y wiggles up-down, z
remains 0.0f float tb 0.0f, 0.0f, 0.0f
tb0 -4.0f tb1 0.0f
ks.setKeyframe(0, 0, tb) tb0 -3.0f tb1
2.0f ks.setKeyframe(1, 10, tb) tb0
5.0f tb1 0.0f ks.setKeyframe(9, 90,
tb) ks.setDuration(100) // sequence
duration in local time ks.setValidRange(0,
9) // all 10 keyframes are valid
ks.setRepeatMode(KeyframeSequence.LOOP) return
ks
60
Example Set up controller
controller new AnimationController() // The
animation is active between these world
times controller.setActiveInterval(0, 1500) //
Set speed, scale around given time
(0) controller.setSpeed(2.0f, 0) // Create
animation track with keyframes controller ks
createKeyframeSequence() at new
AnimationTrack(ks,AnimationTrack.TRANSLATION)
at.setController(controller) // Attach it to
our cube cube.addAnimationTrack(at)
61
Demo
  • Keyframe animation

61
62
Summary
  • M3G is the 3D API for J2ME technology
  • Will be supported in millions of devices
  • Not just an API
  • M3G also defines a binary file format
  • M3G is flexible
  • Using the immediate mode gives you full control
  • High-level features make your code faster and
    smaller

63
Demo
  • 3D snowboarding

63
64
For More Information
  • M3G specification
  • www.forum.nokia.com/java/jsr184
  • SDKs, additional documentation
  • www.forum.nokia.com/ (click Series 40)
  • developer.superscape.com/ (registration needed)
  • M3G content creation tools
  • www.discreet.com/
  • www.superscape.com/

65
QA
  • Thanks Kari Kangas, Sean Ellis, Nokia M3G team,
    JSR-184 Expert Group

65
66
Advanced Game Development with the Mobile 3D
Graphics API
  • Tomi Aarnio, Kari Pulli
  • Nokia Research Center
About PowerShow.com