Title: Support for Adaptive Computations Applied to Simulation of Fluids in Biological Systems
1Support for Adaptive Computations Applied to
Simulation of Fluids in Biological Systems
- Immersed Boundary Method Simulation in Titanium
2Objectives
- Provide easy-to-use, high-performance tool for
simulation of fluid flow in biological systems. - Demonstrate the Titanium compiler and language.
- Allow heart simulation on large-scale parallel
machines.
3Outline
- Immersed Boundary Method
- Titanium
- Immersed Boundary Method in Titanium
4Immersed Boundary Method
- Developed at New York University by Peskin
McQueen to model biological systems where elastic
fibers are immersed in an incompressible fluid. - Mammalian heart, blood platelets, sea urchin
embryos - Fibers (e.g., heart muscles) modeled by list of
fiber points - Fluid space modeled by a regular lattice
5Immersed Boundary Method Structure
Fiber activation force calculation
Fiber Points
Interpolate Velocity
Spread Force
Interaction
Fluid Lattice
Navier-Stokes Solver
6IB Method Steps 1 and 2
- Fiber Activation Force calculation
- Application-specific
- For the heart, use an elastic spring law
- Spread Force
- Spread forces from fiber points list to fluid
lattice via the Dirac Delta function.
7IB Method Steps 3 and 4
- Navier-Stokes Solver
- Calculate fluid velocities
- Uses a 3D FFT
- Interpolate velocity
- Gather velocities of fiber points from fluid
lattice via Dirac delta function - Move the fiber points
8Challenges to Parallelization
- Irregular fiber lists need to interact with
regular fluid lattice. - Trade-off between load balancing of fibers and
minimizing communication - Efficient scatter-gather across processors
- Need a scalable elliptic solver
- Plan to uses multigrid
- Eventually addAdaptive Mesh Refinement
- New algorithms under development at LBNL
9Heart Application of the IB Method
- Heart simulation used to design artificial heart
valves
10Outline
- Immersed Boundary Method
- Titanium
- Immersed Boundary Method on Titanium
11Titanium Motivation
- Applications are increasingly complex
- Want classes, overloading, linked data
structures - C is hard to read, modify and tune
- Machines are increasingly complex
- Want compiler help for optimizations
- Want clear performance model and programmer
control - Java is a better C
- Safe strongly typed, garbage collected
- Performance is poor due to
12Titanium for Scientific Computing
- Java dialect for high performance
- Added constructs for performance expressiveness
- Immutable, value classes
- SPMD parallelism with a global address space
- Multidimensional arrays
- Templates
- Region-based memory management
- Compiled to C (no JVM) with lightweight messaging
(Active Messages, LAPI, shmem)
13SPMD Parallelism in Titanium
- Explicitly parallel model
- Fixed number of threads at program startup
- Usually one thread per processor
- Global address space
- Processors can access remote data by reading and
writing through global references (pointers) - Bulk communication happens when copying arrays
- Compiler automatically converts global pointers
local ones (up to 2x speedup for LQI) - Compiler detects synchronizations bugs in barriers
14Value Classes in Titanium
- Java has two distinct kinds of values
- Primitive scalar types boolean, double, int,
etc. - Objects user-defined and library types
- implicit level of indirection (pointer to)
- Titanium adds support for small objects
- Look like classes with immutable keyword
- Stored in place and passed by coping
- Examples
- Complex type
- Points used to index Titanium arrays
15Titanium Arrays
- Java arrays are 1-dimensional
- Arrays of arrays are inefficient
- Titanium adds multidimensional arrays
- Indexed by Points (tuples of ints)
- Algebra over Domains (sets of points)
- Helps with hierarchical algorithms, e.g.,
multigrid - One array may be a subarray of another
- e.g., a is interior of b, or a is all even
elements of b - Foreach loops help compiler optimize arrays
- Within 2x of C for multigrid kernels
- Bulk I/O provided on arrays (2x-40x speedup!)
16Titanium with Other Languages
- Native methods are sometimes useful
- Performance allows for comparisons with other
compilers and additional control - Libraries have interfaced to other systems like
PetSC and ParMetis - Requires understanding of underlying Titanium
implementation in C. - Lower entry cost than Java the native method is
simply included into the generated code
17Titanium Implementation
- Run time system and compiler for
- Uniprocessors
- SMP running POSIX threads
- Clusters with
- Shared memory - SGI Origin cluster (ANL), Tera
MTA - Global Address Space - T3E (NERSC)
- Active Messages - NOW Millennium (UCB)
- LAPI - IBM SP2, SP3 (SDSC)
18Outline
- Immersed Boundary Method
- Titanium
- Immersed Boundary Method on Titanium
19Immersed Boundary Generic Software
- Written by Cowen at NYU
- Implements subset of the IB method adequate for
the heart - Runs on vector machines with shared memory
20Immersed Boundary on Titanium
- IBGS rewritten in Titanium.
- Running since October
- Contractile torus
- runs on Berkeley NOW and SGI Origin
- Needed for heart
- Input file format
- Performance tuning
- Uniprocessor (C code used temporarily in 2
kernels) - Communication
21Contractile Torus Visualization
22Immersed Boundary on Titanium
- Performance Breakdown (torus simulation)
23Immersed Boundary on Titanium
24Future work
- Improve performance
- Especially on SP machines (Blue Horizon)
- Add functionality
- Bending angles, anchorage points, source sinks)
to the software package. - Add adaptability to NS solver (AMR)
- Needed for scaling and more accurate modeling of
fluid features in heart