Etree

1
Etree A Database-oriented Method for Generating
Large Octree Meshes
David R. OHallaron1,2
Tiankai Tu1
Julio C. López2
tutk_at_cs.cmu.edu
jclopez_at_cs.cmu.edu
droh_at_cs.cmu.edu
1School of Computer Science 2Electrical and
Computer Engineering Department Carnegie Mellon
University
2
Motivation
Goal Make it possible to run large-scale
physical simulations on PCs with limited
physical memory
Approach Index and store the datasets in
databases and compute on the databases
directly
Requires research at the intersection of
computer systems, scientific computing, and
databases
3
Octree mesh generation

• a compromise between structure and modeling power
  
• balance requirement (2-to-1 constraint)
• can be implemented in-core or out-of-core

4
Etree mesh generation overview
A general method for manipulating large
out-of-core octrees by querying databases
application-specific input
element database
unbalanced octree
balanced octree
node database
5
Etree library components

• Etree API a simple Application Programming
  Interface
• Linear quadtree an encoding scheme to assign
  keys to octants
• Auto navigation a mechanism for constructing
  an octree automatically
• Local balancing a technique that speeds up
  balancing operation
• B-tree a database index structure to store and
  access octants on disk

Application (e.g.construct, balance)
Etree API
Etree library
Linear quadtree
Auto navigation
Local balancing
B-Tree
6
Etree API
Unix file I/O style, three classes

• initialization and cleanup example

etree_t etree_open(const char path, int flag,
)

• octant-level operations example

int etree_insert(etree_t ep, location_t loc,
void value)

• octree-level operations example

int etree_balance(etree_t ep, decom_t baldecom)
7
Linear quadtree how it works
y
8
7
h
m
6
5
4
e
g
b
j
l
3
d
f
2
a
c
i
k
1
0
x
3
2
4
7
8
5
6
0
1

• Morton code maps n-dimensional points to
  one-dimensional scalars
• Locational code appends an octants level to
  the Morton code of its left-lower corner

8
Linear quadtree how it works (contd)
ds left-lower corner (2, 2)
y
8
binary form (010, 010)
7
h
m
6
5
interleave the bits to obtain Morton code
4
e
g
b
j
l
3
d
f
010
010
2
a
c
i
k
1
0
00 11 00
x
3
2
4
7
8
5
6
0
1
append the level of d
001100_11
9
Linear quadtree applications
m
h
e
g
b
j
l
d
f
a
c
i
k

• an addressing scheme that clusters nearby
  octants
• finding an octant without knowing its locational
  code
• the order imposed by the locational code is the
  same as the preorder traversal of leafs in octree

10
Auto navigation how it works

• Navigation octree
• guided by an application function
• an in-memory pointer-based octree
• dynamically grows in the depth-first order
• leaf octants are pruned and flushed to disk in
  preorder (increasing locational code order)
• appends the octant to the database to avoid
  database search

octants not yet processed (in memory)
non-leaf octants being decomposed (in memory)
leaf octants (flushed to database)
11
Local balancing how it works

• Operational steps
• partition the whole domain into equal-size
  blocks
• conduct internal balancing to enforce 2-to-1
  constraint within each block (in memory a
  resident blocking array)
• perform boundary balancing to resolve
  interactions between adjacent blocks

12
Correctness of local balancing
Claim Interactions between adjacent blocks are
always absorbed by boundary octants and will not
be propagated into the blocks
Proof See the paper
13
Evaluation questions

• Is the etree method feasible?
• How does the running time vary with the physical
  memory size?
• What is the impact of auto navigation?
• What is the impact of local balancing?

14
Evaluation methodology

• We implemented an etree-based mesh generator to
  generate a family of finite element meshes for
  San Fernando valley earthquake wave propagation
  simulations

Mesh Elements Nodes Slave nodes
SF10 7,940 12,118 4,432
SF5 76,330 105,886 34,858
SF2 1,838,524 2,213,035 407,336
SF1 13,579,124 15,097,365 1,649,855
15
Evaluation setup

• All experiments are conducted on a PIII 1GHz
  machine running Linux 2.4.17.
• The machines physical memory for the
  experiments ranges from 128MB to 880MB
• Before each experiment, two 1.5 GB files are
  sequential scanned to ensure that the operating
  systems buffer cache is flushed

16
Evaluation etree feasibility
All experiments are performed with 128MB physical
memory
Mesh Elements DB size (MB) Time(sec) Thruput(elem/s)
SF10 7,940 2.5 39.9 199
SF5 76,330 24 186.0 410
SF2 1,838,524 583 1,636.7 1,123
SF1 13,579,124 4,300 9,448.8 1,439
Etree-based mesh generator running time and
throughput

• Generating a mesh with 13.6 million elements and
  of size 4.3GB in 2.6 hours seems reasonable
• The overall throughput increases with mesh size

17
Evaluation impact of memory size

• Memory size does not have a significant impact
  on the running time
• The etree method is not relying on the operating
  systems internal caching mechanism to achieve its
  performance

18
Evaluation impact of auto navigation

• Reducing B-tree buffer size does not increase
  the construction time
• Auto navigation is not sensitive to B-tree
  buffer size

19
Evaluation impact of local balancing

• Achieves a speed-up factor ranging from 8 (SF1)
  to 28 (SF10)
• Benefits from the one-time scan of the database
  and the efficient array-based neighbor finding
  algorithm

20
Related work

• General octree algorithms Samet 90
• Octree mesh Shepard Geoges 91, Bern et al.
  90, Young et al. 91, Wang99
• Out-of-core octree method Salmon 97
• Linear quadtree Gargantini 82, Morton 66
• Space filling curve Orenstein 84, Orenstein 86,
  Faloutsos Roseman 89
• Large dataset processing Freitag Loy 99,
  Seamons Winslett 96, Ferreira et al. 99,
  Kurc et al. 01, Choudhary et al. 99
  Parashar Browne 97

21
Conclusion and future work

• Experiment results suggest that the etree method
  can generate large octree meshes on
  memory-limited machines in a reasonable amount of
  time
• Incorporating existing database techniques
  (linear quadtree and B-tree) with new algorithms
  (auto navigation and local balancing) in a
  unified design scheme (the etree) can deliver new
  capability
• We are porting the etree method to commercial
  database systems such as IBM DB2