MPEG-4 Toward Solid Representation

1
MPEG-4 Toward Solid Representation

• Alain Mignot and Pierre Garneau
• IEEE Trans. on Circuits and Systems for Video
  Tech.,
• Vol. 14, NO. 7, JULY 2004, pp. 967-974.
• Presented by
• Reza Aghaee
• Multimedia Course(CMPT820)
• Simon Fraser University
• March.2005

2
Agenda

• Introduction
• Future of 3D Standards
• Solid Representation MPEG-4s Answer to 3D
  Challenge
• Implications of Rendering Mechanisms
• Impacts on Applications
• Conclusion

3
Introduction

• Traditional 3D Image Creation
• Tessellation
• Models are created of individual objects using
  link points that are made into a number of
  individual polygons
• Geometry
• Polygons are transformed in various ways and
  lighting effects are applied
• Rendering
• Transformed images are rendered into objects with
  very fine details

4
Introduction (cntd.)

• Performance is the number of polygons processed
  per second.
• MPEG-4 is introducing a new and different
  approach.
• This method can be used in various 3D
  applications. (Games/CAD/CAM/CAE)

5
Future of 3D Standards

• 3D Technology is at a Turning Point
• 10 years old OpenGL is reaching its limits.
• Its confirmed by user-programmable parts of the
  pipeline.
• This limited programming capability, in the form
  of vertex and pixel shaders, appears to be the
  easiest solution.

6
Future of 3D Standards

• Shader Programs May Not Be the Solution
• It is actually a step backward, in the opposite
  direction of the beneficial standardization
  process.
• Graphic APIs display 3D scenes on almost any
  device.
• Drivers take care of hardware differences other
  than performance.

7
Future of 3D Standards

• Shader Programs May Not Be the Solution (cntd.)
• A software driver can not simulate most shader
  programs.
• Developers should provide different versions for
  different graphic boards.
• Shader Programs do not bring any additional
  information to the rendering process concerning
  the scene itself.

8
Future of 3D Standards

• Historical Considerations
• Higher performance was often achieved by
  including parts of pipeline into dedicated
  hardware.
• First, later stages of the pipeline were
  performed in hardware and front portion in
  software driver.
• 3D technology should extend its domain on
  functionalities performed by application.

9
Future of 3D Standards

• Limitations of the 3D Pipeline Common to OpenGL
  and DirectX
• Algorithms designed to create the illusion of
  depth and continuity.
• 3D rendering is a set of techniques that have
  nothing to do with geometric objects.
• In real 3D space, objects may influence each
  other even if invisible.

10
Future of 3D Standards

• Limitations of the 3D Pipeline Common to OpenGL
  and DirectX (cntd.)
• Two parallel data structures needed
• One for describing the scene ( objects with
  geometric attributes)
• One for the set of polygons (each object in its
  position relative to observer)
• First structure for managing interactions
• Second structure only for rendering

11
Future of 3D Standards
Traditional 3D Game Production
12
Future of 3D Standards
Traditional CAD rendering
13
Projected Evolution of 3D standards

• Data structure of geometric objects and their
  attributes is common in most applications.
• Most applications share a common framework based
  on geometrical and physical properties of
  objects.
• By including this framework in the rendering
  engine the whole process is well improved.

14
Solid Representation in MPEG-4

• MPEG-4 introduces the purely mathematical
  definition of shapes.
• These shapes are based on algebraic shapes
  combined with arithmetic shapes.
• These shapes are completely independent of
  rendering process.
• These functionalities are referred to as Solid
  Representation.

15
Solid Representation in MPEG-4

• In MPEG-4, solid representation functionalities
  deal with
• Reducing the size of files to be transmitted or
  shared by transmitting constructive commands
  rather than results.
• Increasing the geometrical precision of rendered
  objects by managing a polynomial representation
  of volumes.
• Manipulating complex objects combined by solid
  operations.

16
Solid Representation in MPEG-4

• Solid Primitives
• Any solid object has a 3-D form defined by a
  skin, which delimits the inside from the
  outside.
• If equation of surface is unknown the volume
  will be divided into simpler pieces until they
  have known forms.
• These primitives will then be assembled in order
  to constitute the original complex shape.
• Figure 3 is a complex shape made of a set of
  solid primitives.

Figure 3
17
Solid Representation in MPEG-4

• There are two approaches to define a surface
• A Parametric Equation giving spatial coordinates
  of each surface point.
• Implicit Equation of algebraic surfaces.
• Implicit Equation of a Sphere
• (Px Cx ) 2 ( Py Cy ) 2 (Pz - Cz) 2 - R2
  0
• Eq0 means point is on the surface,for Eqlt0
  point is inside and for Eqgt0 point is outside the
  volume.

18
Solid Representation in MPEG-4

• A quadratic equation (second-degree equation)
  allows to define the entire quadrics family

• An equation of the fourth degree allows to define
  the quartics family

19
Solid Representation in MPEG-4

• Implicit second-degree equation defining quadrics
  iswhere for each point coordinates (X0X3)
  the result isF(X0,,X3) lt 0 F(X0,,X3)
  0 F(X0,,X3) gt 0whether the point is
  inside, on the surface or outside.

20
Solid Representation in MPEG-4

• The point coordinates are made homogeneous by
  adding X3 .
• A surface of fourth-degree will have 35
  coefficients.
• Unbound surfaces should be bounded to be
  processed and displayed.
• Cylinder equation is an example of unbound
  volumes.

21
Solid Representation in MPEG-4

• In MPEG-4 the coefficients of the quadric
  (second-degree implicit surface) may be defined
  by six geometric control points.
• P0,P1 2 contact points on the quadric.
• P2,P3 2 poles of the construction
• tetrahedron.
• P4,P5 2 passing points of the quadric.
• Each point is defined using homogeneouscoordinate
  s allowing the point to be sent to infinity
  (affine geometry)

22
Solid Representation in MPEG-4

• In MPEG-4 two Geometry nodes implement algebraic
  surfaces
• Implicit Node
• Defines the surface by the coefficients of the
  polynomial.
• Quadric Node
• Defines the surface by the six control points
  explained in the previous slide.

23
Solid Representation in MPEG-4

• Arithmetic of Forms is a logical modeling
  system for solid objects.
• Description of a solid object takes the form of a
  solid tree made up of operators and operands.
• Operands are primitives or more complex solid
  objects.
• Operators are mostly union, intersection and
  logical subtraction.

24
Solid Representation in MPEG-4

• Each basic geometric primitive splits space into
  three regions external, boundary, and internal
  coded by integers 0, 1, 2.

• This ternary coding of space tells us the density
  of every point in space.

Constructive Solid Geometry (CGS) tree and
corresponding model 3D
25
Solid Representation in MPEG-4

• There are three basic sets of operators
• General Arithmetic Operators on Densities
• For instance, addition, multiplication, and
  difference of densities of two volumes.
• Below is multiplication of two forms F0 and F1.

26
Solid Representation in MPEG-4

• Three basic Sets of Operators (cntd.)
• Arithmetic Operators with Ternary Logic
• They use ternary logic only.
• Examples are union and intersection.
• Below is ternary intersection of F0 and F1.

27
Solid Representation in MPEG-4

• Three basic Sets of Operators (cntd.)
• Densities Filtering
• In MPEG-4 a set of test functions is applied on
  the root of the solid tree to filter densities
  while keeping the filtering inside the tree.
• Examples can be Equality Filter (F0F1)

Results of Solid Operations
28
Implications of Rendering Mechanisms

• If the volumes must be displayed very precisely
  some techniques can be used to render the output.
• If one accepts less geometric precision, it is
  possible to consider tessellation of implicit
  surfaces before or after applying solid operation
  and rendering.

29
Implications of Rendering Mechanisms

• The functionalities of solid representation in
  MPEG-4 including Implicit, Quadric and SoldRep
  Nodes are completely independent from the
  rendering method.
• All the solid operators are independent from the
  rendering process too.

30
Impacts on Applications

• Compactness
• Suitable for sophisticated web applications
  including online gaming on small-constrained
  devices mobile phones and PDAs.
• Complete model of a historical castle with a
  level of detail from roof frame to door openers
  takes 50 Kb.

31
Impacts on Applications

• The following example is a model solely based on
  SolidRep geometry.

Complete Solid Model of Leihorra villa
Details of the Leihorra villas model
32
Impacts on Applications

• Exact Geometry
• The exact geometry is preserved up to the
  decoder.
• It allows very precise scientific applications as
  CAD/CAM or simulations.
• Below is the simulation of the Canadian Space arm
  from Canadian Space Agency.

33
Impacts on Applications

• Embedded Topology and 3D Properties
• There is a wrap-up of the topology and 3D solid
  properties (e.g. constituent matter and physical
  properties)
• Local manipulation, exploration of the model and
  very accurate collision detection is allowed.

• The picture is an inside view of the villa as the
  result of a solid Operation.

34
Impacts on Applications

• There are many games claiming that the player can
  destroy almost any object to find other hidden
  spaces.
• This model can help implement these models
  without very heavy programming techniques.

Complete Original Model
Model cut with a laser beam
35
Conclusion

• Polygonal pipelines cant present the future of
  3D standards.
• Solid representation can transfer very complex
  models in their most exact geometry and in a very
  compact way.
• Even without a dedicated hardware current CPUs
  are powerful enough to provide real-time
  processing.
• Solid representation is now a part of the MPEG-4
  part-16.
• New APIs will be developed including object
  description to accelerate solid representation.