Title: Using the PETSc Parallel Software library in Developing MPP Software for Calculating Exact Cumulativ
1Using the PETSc Parallel Software library in
Developing MPP Software for Calculating Exact
Cumulative Reaction Probabilities for Large
Systems (M. Minkoff and A. Wagner)ANL (MCS/CHM)
- Introduction
- Problem Description
- MPP Software Tools
- Computations
- Future Direction
2Parallelization of Cumulative Reaction
Probabilities (CRP) with PETScM. Minkoff
(ANL/MCS) and A. Wagner(ANL/CHM)
- Calculation of gas phase rate constants
- Develop a highly scalable and efficient parallel
algorithm for calculating the Cumulative Reaction
Probability, P. - Use parallel subroutine libraries for higher
generation parallel machines to develop parallel
CRP simulation software. - Implementing Miller and Manthe (1994) method for
time-independent solution of P in parallel. - P is determined for an eigenvalue problem with an
operator involving two Greens functions. The
eigenvalues are obtained using a Lanczos method.
The Greens functions are evaluated via a GMRES
iteration with diagonal preconditioner.
3Benefits of using PETSc
- Sparsity PETSc allows arbitrarily sparse data
structures - GMRES PETSc has GMRES as an option for linear
solves - Present tests involve problems in dimensions 3 to
6. Testing is underway using an SGI Power
Challenge (ANL), and SGI/CRAY T3E (NERSC).
(Portability is provided via MPI and PETSc, so
higher dimensional systems are planned for future
work).
4Chemical Dynamics Theory3 angles, 3 stretches6
degrees of freedom
5Chemical Dynamics Theory
- How fast do chemicals react?
- Rate constant k determines it
- dX / dt -k1XY k2ZY
- many rates at work in devices
- rates express interactions in the chemistry
- individual rates are measurable and calculable
- rates depend on T, P.
6Chemical Dynamics TheoryN(E) TrP(E)
- Rates are related to
- Cumulative Reaction Probability (CRP), N(E)
- N(E) 4 Tr
7Chemical Dynamics Theory
- Probability Operator and Its Inverse
- Using probability method calculates a few large
eigenvalues via iterative methods. The iterative
evaluation involves the action of two Greens
function. - Using inverse probability method involves a
direct calculation each iteration to obtain a few
smallest eigenvalues. At each iteration the
action of a vector by the Greens function is
required. This leads to solving linear systems
involving the Hamiltonian.
8Chemical Dynamics Theory
- The Greens functions have the form
- G(E) (E ie - H)-1
- and so we need to solve two linear systems (at
each iteration) of the form - (E ie - H) y x
- where x is known.
- This system is solved via GMRES with
preconditioning methods (initially diagonal
scaling). -
-
9PETSc Portable, Extensible Toolkit for
Scientific Computing
Satish Balay, William Gropp, Lois McInnes, and
Barry Smith MCS Division, Argonne National
Laboratory
- Focus data structures and routines for the
scalable solution of PDE-based applications - Object-oriented design using mathematical
abstractions - Freely available and supported research code
- Available via http//www.mcs.anl.gov/petsc
- Usable in C, C, and Fortran77/90 (with minor
limitations in Fortran 77/90 due to their syntax) - Users manual, hyperlinked manual pages for all
routines - Many tutorial-style examples
- Support via email petsc-maint_at_mcs.anl.gov
10Application Codes Using PETSc
Applications can interface to whatever
abstraction level is most appropriate.
11PETSc Numerical Components
12Sample Scalable Performance
600 MHz T3E, 2.8M vertices
- 3D incompressible Euler
- Tetrahedral grid
- Up to 11 million unknowns
- Based on a legacy NASA code, FUN3d, developed
by W. K. Anderson - Fully implicit steady-state
- Newton-Krylov-Schwarz algorithm with
pseudo-transient continuation - Results courtesy of Dinesh Kaushik and David
Keyes, Old Dominion University
13Computations via MPI and PETSc
145D/T3E Results for Varying Eigenvalue and G-S
Method
15 Parallel Speedup5D/6D ANL/SGI and NERSC/T3E
16Storage Required for Higher Dimensions
17Results and Future Work
- Achieved parallelization with less effort
- Suboptimal but perhaps 2X Optimal Performance
- Testing for 6D and 7D underway.
- MPP CPU and Memory can provide necessary
resources - Many degrees of freedom can be approximated, so
maximum dimension needed is 10. - Develop block structured preconditioning methods