Efficient Parallel Simulation of Fluid Dynamics on Cartesian Grids

1
Efficient Parallel Simulation of Fluid Dynamics
on Cartesian Grids

• Miriam Mehl
• Ionel Muntean, Tobias Neckel, Tobias Weinzierl
• Computer Science
• TU München

2
Why Cartesian Grids?
numerical efficiency (adaptivity, multigrid)
hardware efficiency ??? (not automat.)
flexibility (adaptivity, marker and cell)
accuracy ??? (not automat.)
physical correctness ??? (not automat.)
3
Numerical Efficiency

• hierarchically structured Cartesian grids
• arbitrarily local adaptivity
• full approximation schemes
• efficient multigrid methods

4
Hardware Efficiency
½ ½ -1

• cell-oriented operator evaluation
• constant difference stencils
• no neighbour relations
• low storage

5
Hardware Efficiency

• Peano curve
• processing order of grid cells
• time locality of data access

6
Hardware Efficiency

• stacks as data structures
• spatial locality of data access

7
Hardware Efficiency

• cache-misses 110 of minimum
• runtime 5 times DiMe (regular grid)

• 3D Poisson
• sphere, adaptive
• 23,118,848dofs

processes Speedup
4 3.73
8 6.85
16 12.93
8
Flexibility

• geometric adaptivity
• Eulerian approach (marker-and-cell)
• complicated changing geometries

9
Accuracy

• geometric adaptivity
• cutting-cell methods
• hierarchical operator generation
• second order accuracy in geometry

10
Physical Correctness

• Verstappen, 2001
• symmetry requirements
• energy and momentum conservation
• FEM

11
Physical Correctness

• FEM-basis u-v-coupled, piecewise linear
• correct velocity interpolation
• dynamical adaptivity, coupling surface

12
Why Cartesian Grids?
numerical efficiency (adaptivity, multigrid)
hardware efficiency (space-filling curves, stacks)
flexibility (adaptivity, marker and cell)
accuracy (adapt., cutting-cell, hier. op. Gen.)
physical correctness (divergence preserv. FE basis)
13
Numerical Results

• Regular grid code F3F
• symmetry preserving FV discretisation
• fully parallel
• full 3D functionality
• platforms up to now
• HLRB2 (SGI Altix 4700)
• TU München Infinicluster (128 CPU Opteron)
• Universität Stutgart Mozart (128 CPU Xeon
  cluster)

14
Numerical Results
15
Numerical Results

• Adaptive grid code Peano
• 2D Navier-Stokes
• parallel Poisson
• platforms up to now
• HLRB2 (SGI Altix 4700)
• TU München Infinicluster (128 CPU Opteron)
• PC cluster

16
Numerical Results

• Free channel flow Re1111
• in preparation to DNS
• boundary layer adaptively refined

17
Conclusion Outlook

• appropriateness of our approach
• concept for adaptive grids Navier-Stokes
• Cartesian grids applications
• next steps
• fully functional 3D parallel adaptive NS-solver
• refinement criteria for turbulent boundary layers
• runtime optimisation on supercomputers