Streaming Meshes

1
Streaming Meshes
Martin Isenburg UC Berkeley
Peter Lindstrom LLNL
2
Indexed Formats
3
triceratops.obj 2832 vertices 2834
polygonsv 3.661 0.002 -0.738v 3.719
0.347 -0.833v 3.977 0.311 -0.725v 4.077
0.139 -0.654 ? ? ? ? f 2806 2810
2815 2821f 2797 2801 2811 2805f 2789
2793 2802 2796f 2783 2794 2788 ? ?
? ?
2806
4
Original Orderings
showing the first 20to 40 percent of
thetriangle array
5
Large Meshes
3D scans
isosurfaces
6
Large Meshes
3D scans
isosurfaces
7
Input Problem
Indexed Mesh
186 million vertices 2 GB
372 milliontriangles 4 GB
6 GB
8
Input Problem
Indexed Mesh
186 million vertices 2 GB
372 milliontriangles 4 GB
6 GB
9
Input Problem
Indexed Mesh
186 million vertices 2 GB
372 milliontriangles 4 GB
6 GB
10
Papers from 1997 - 2005

• Viz 1997, I/O Optimal Isosurface Extraction,
  Chiang, Silva
• unfortunately, data sets are often given in
  formats that contain indices to vertices
• SIGGRAPH 2000, Out-of-Core Simplification,
  Lindstrom
• operate on a triangle soup in which each
  triangle
• SIAM 2003, Adaptive Vertex Clustering,
  Schaefer, Warren
• we also assume that the mesh is in a triangle
  soup format
• GI 2003, Decimation of Massive Meshes, Wu,
  Kobbelt
• in this format (dubbed triangle soup in 14)
  every triangle
• TVCG 2003, External Memory Mesh, Cignoni et al.
• from a triangle soup (list of faces, not
  indexed)
• SIGGRAPH 2004, Tetra Puzzles, Cignoni et al.,
• we assume triangle soup, a flat list of
  triangles
• SIGGRAPH 2005, Far Voxels, Gobbetti, Marton
• we assume represented as a triangle soup

11
Large Model Distribution
Digital Michelangelo Project Stanford
University
3D Gallery
Visual Computing Lab, ISTI
12
Reading Indexed Input

• one-pass
• memory-map the vertex array
• rely on paging system of OS
• will fail on incoherent input

13
Output Problem
Pieces
14
Writing Indexed Output

• use computer with lots of memory
• Stanford used an SGI Onyx2

• output piece by piece
• concatenate in final pass

• memory-map the mesh
• number of vertices known a priori ?

• output into two separate files
• concatenate in final pass

15
Format Conversion
possibly not IO-efficient
standard indexed formats
186 million vertices 2 GB
372 milliontriangles 4 GB
16
Streaming Formats
17
Streaming Formats

• physical
• water in a pipe
• drip coffee

• geometry

18
Two Types of Streaming

• progressive

?
?
?
19
Non-Progressive Streaming

• consume immediately
• potentially without end
• keep small buffer
• delete data if no longer needed

20
Streaming Mesh Formats
v 1.32 0.12 0.23v 1.43 0.23 0.92v
0.91 0.15 0.62f 1 2 3finalize 2 v
0.72 0.34 0.35f 4 1 3finalize 1
? ? ? ?

• interleave introduce finalize

21
Reading Streaming Formats

• mesh.open ( mesh.sma )while ( event
  mesh.read_event() ) switch ( event )
  case TRIANGLE // look-up v0,
  v1, v2 in hash case VERTEX
  // put new vertex in hash case
  FINALIZE // remove finalized
  vertex from hash
• mesh.close ()

22
Streaming Mesh Definitions

• width
• maximal number of active vertices
• ? how many vertices need to be buffered

• span
• maximal time a vertex stays active
• ? how long vertices remain in the buffer

• do not bloat width and span !!!
• introduce late and finalize early

23
Writing Streaming Formats

• isosurface
• 235 million vertices
• 469 million triangles

over8 Gigabyte

• marching cubes
• extract layer by layer
• output elements as extracted
• finalize vertices of previous layer

Richtmeyer-Meshkov instability simulation at LLNL
24
Coherence in Indexed Meshes
Indexed
Streaming
? visualization of the coherence
? tells us how difficult it is to convert to
stream
25
Layout Diagram Buddha
triangles
vertices
20 MB
26
Layout Diagram Dragon
triangles
vertices
20 MB
27
Layout Diagram Lucy
triangles
vertices
508 MB
28
Layout Diagram St. Matthew
triangles
vertices
6,760 MB
29
Visualizing high vertex spans
30
Topologic and Geometric Sort
depth-first
breadth-first
along one axis
z-order
31
Output Qualities

• low width low span
• sweep-line
• FIFO (queue)

• low width, high span
• block by block
• FILO (stack)

• high width, high span
• random access

32
Streaming with minimal width

• graph theory
• minimization is NP hard
• heuristic spectral sequencing

33
Stream-Processing
34
Stream-Processing Model
35
Stream-Processing Tasks

• tasks that process mesh elements one at a time
• e.g. for each triangle t
• tasks that only require access to local neighbors
• e.g. for each triangle t of vertex v
• tasks that are order independent
• e.g. collapse edges shorter than ?

36
Streaming Simplification
A Stream Algorithm for Wu Kobbelt 03
Large Mesh Simplification Isenburg et al.
03
37
Performance
compared to
External Memory Management and Simplification
of Huge Meshes Cignoni et al. 03
Octree-based External Memory Mesh (figure
courtesy of Paolo Cignoni)
38
Streaming Compression
Streaming Compression of Triangle
Meshes Isenburg, Lindstrom Snoeyink 05
39
Performance
compared to
Out-of-Core Compression of Gigantic Polygon
Meshes Isenburg Gumhold 03
Out-of-Core Mesh
40
Pipelined Streaming
41
Pipelined Stream-Processing

• conventional processing

• super-linear speedup
• minimal end-to-end I/O delay
• optimal disk caching

42
Demo Pipeline
regular volume grid
v 1.32 0.12 0.23v 1.43 0.23 0.92v
0.91 0.15 0.62f 1 2 3finalized 2
v 0.72 0.34 0.35f 4 1 3finalized
1 ? ? ? ?
256
256
256
smextract smclean smsimp smcompress
P2
P3
P1
P4
grid.raw
mesh.smc
43
Conclusion
44
Indexed Formats do not Scale
9 GB
1 MB
45
Small Change Big Benefits
Standard Format
Streaming Formats
46
Current / Future Work

• more stream modules
• streaming re-meshing
• streaming smoothing
• streaming feature extraction
• streaming stripification,

• stream other geometry data
• volumetric meshes / grids
• points in 2D and 3D

47
Streaming Volume Meshes

• format
• like surface mesh format
• straight-forward

• applications
• streaming iso-surface extraction
• streaming simplification
• streaming compression
• streaming mesh refinement,

48
Volume Mesh Compression
compared to
Tetrahedral Mesh Compression with the
Cut-Border Machine
Gumhold et al. 99
torso
662 sec
12 sec
compress
fighter
140 MB
6 MB
1.81
3.56
bits / tet
49
Layouts of Volume Meshes
torso.off
(19 MB)
v 168,930 tet 1,082,723
v 256,614 tet 1,403,504
(25 MB)
fighter.off
rbl.off
v 730,273 tet 3,886,728
(70 MB)
50
Streaming Points

• format
• need spatial finalization
• e.g. no future points in this space

• applications
• streaming surface reconstruction
• streaming re-sampling
• streaming smooting
• streaming compression,

51
Streaming Delaunay
3D Delaunay
2D Delaunay
250 sec
125 sec
2D Delaunay triangulate6 million terrain points
450 MB
4 MB
52
Acknowledgements

• meshes
• Stanford University
• Cyberware
• support
• NSF grant 0429901
• "Collaborative Research Fundamentalsand
  Algorithms for Streaming Meshes."
• U.S. DOE / LLNL W-7405-Eng-48

53
Thank You
Why?
Read the paper!
demo API http//www.cs.unc.edu/isenburg/
sm