High Performance Computing and Monte Carlo Methods Spring 2005

1
High Performance Computing and Monte Carlo
MethodsSpring 2005

• By Yaohang Li, Ph.D.
• Department of Computer Science
• North Carolina AT State University
• yaohang_at_ncat.edu

2
Course Outline (I)

• Introduction (today)
• Computational Science
• High Performance Computing
• Monte Carlo Methods
• High Performance Computing
• Parallel and Distributed Systems
• MPI and OpenMP
• PBS
• Nature of Monte Carlo Methods
• Review of Relevant Elementary Probability
  Statistics

3
Course Outline (II)

• Direct Simulation
• Error Estimation
• Truncation Error and Statistical Error
• Monte Carlo Integration
• Crude Monte Carlo Integration
• Variance Reduction Techniques
• Random Number Generation
• Test of Randomness
• Sampling for Non-uniform Distribution
• Quasi-Monte Carlo
• Difference from Monte Carlo
• Quasirandom Number Generation

4
Course Outline (III)

• Markov Chain Monte Carlo
• Optimization Problem
• Metropolis Method
• Simulated Annealing
• Simulated Tempering
• Monte Carlo and Linear Algebra
• Greens Function Monte Carlo
• Random Walk
• Quantum Monte Carlo

5
Course Outline (IV)

• Monte Carlo Application in Science and
  Engineering
• Computational Physics
• Computational Material Science
• Nuclear Simulation
• Computational Biology
• Computational Financing
• Visualization

6
What you will learn

• Basic Concepts in Computational Science
• Usage of High Performance Systems
• Principles of Monte Carlo Methods
• Monte Carlo Applications to Science and
  Engineering
• Interdisciplinary Research

7
Administrivia

• Textbook
• Monte Carlo Methods by J. M. Hammersley and D. C.
  Handscomb
• Other handout materials
• Course Web Page
• http//abner.ncat.edu/comp790
• Instructional E-Mail Addresses
• yaohang_at_ncat.edu
• Instructor Yaohang Li, Ph.D.
• Office phone [phone]x117
• Office location 327 McNair
• Office hours
• T, R 200PM 600PM
• W 200PM 400PM
• by appointment

8
Grading

• Grading Policy
• 60 5-10 Assignments (Programming)
• 25 1 Term Paper
• 15 1 Presentation
• Check Online Blackboard System for your Grade

9
How To Get an A in This Course

• Be Aware of Resources
• Check with the course website or the instructor
  about
• Handouts, lectures, grade postings
• Resources online
• Check with classmates about material from missed
  lecture
• Start working on your assignment/project Early
• A story
• How to start virtuous (as opposed to vicious)
  cycle
• How not to cheat

10
What is Computational Science?

• Definition A common characteristic of the field
  is that problems
• Have a precise mathematical model.
• Are intractable by traditional methods.
• Are highly visible.
• Require in-depth knowledge of some field in
  science, engineering, or the arts.
• Computational science is neither computer
  science, mathematics, some traditional field of
  science, engineering, a social science, nor a
  humanities field. It is a blend.

11
Computational Science Community

• Computational science projects are always
  multidisciplinary.
• Applied math, computer science, and
• One or more science or engineering fields are
  involved.
• Computer sciences role tends to be
• A means of getting the low level work done
  efficiently.
• Develop and implementation of algorithms
• Similar to mathematics in solving problems in
  engineering.
• Oh, yuck a service role if the computer science
  contributors are not careful.
• Provides tools for data manipulation,
  visualization, and networking.
• Mathematics role is in providing analysis of
  (new?) numerical algorithms to solve the
  problems, even if it is done by computer
  scientists.

12
Computational Scientist Requirements

• Good understanding of an applied discipline.
• High Performance Computing Architecture
• Development of Numerical Algorithms
• Analysis of the Numerical Algorithms
• Parallelization
• Visualization

13
How Significant are Algorithm and Computer
Improvements?

• There is a race to see if computers can be
  speeded up through new technologies faster than
  new algorithms can be developed.
• Computers have doubled in speed every 18 months
  over many decades. The ASCI program is trying to
  drastically reduce the doubling time period.
• Some algorithms cause quantum leaps in
  productivity
• FFT reduced solve time from O(N2) to O(NlogN).
• Multigrid reduced solve times from O(N3/2) to
  O(N), which is optimal.
• Monte Carlo is used when no known reasonable
  algorithm is available.

14
High-Performance Computing Architectures

• Parallel supercomputers
• Multiple processors per node with shared memory
  on the node (a node is a motherboard with memory
  and processors on it).
• Very fast electrical network between nodes with
  direct memory access and communications
  processors just for moving data.
• SGI Origin 3000, IBM SP4, SUN Enterprise, HP
  Superdome.
• Cluster of PCs
• Take many of your favorite computers and connect
  them with a fast ethernet running 100-1000 Mbs.
• Usually runs Linux, IRIX, True64 UNIX, HP-UX,
  AIX-L, Solaris, or Windows NT with MPI and/or
  PVM.
• Intel, Alpha, or SPARC processors. Intel is most
  common in clusters of cheap micros.

15
Peak Speeds of Selected Computers
16
Top Supercomputers
www.top500.org
17
Network Speeds
18
Introduction

• A Monte Carlo method
• provides approximate solutions to a variety of
  mathematical problems by performing statistical
  sampling experiments on a computer.
• In brief, methods consuming random numbers
• Can be applied to
• Stochastic problems
• Deterministic problems
• Characteristics of Monte Carlo methods
• A natural fit for high-performance computing
  systems
• Monte Carlo methods are stochastic techniques.
• Monte Carlo method is very general.
• We can find MC methods used in many areas from
  economics to nuclear physics to regulating the
  flow of traffic.

19
Monte Carlo Example Estimating p

20
If you are a very poor dart player, it is easy to
imagine throwing darts randomly at the above
figure, and it should be apparent that of the
total number of darts that hit within the square,
the number of darts that hit the shaded part
(circle quadrant) is proportional to the area of
that part. In other words,
21
If you remember your geometry, it's easy to show
that
22
(x, y)
x (random) y (random)
distance sqrt (x2 y2) if
distance.from.origin (less.than.or.equal.to) 1.0
let hits hits 1.0
23
Monte Carlo Applications

• Nuclear reactor design
• Quantum chromodynamics
• Radiation cancer therapy
• Traffic flow
• Stellar evolution
• Econometrics
• Dow-Jones forecasting
• Oil well exploration
• VLSI design
• Protein structure prediction
• Material Science

24
Monte Carlo Methods in Grand Challenges

• Grand Challenges
• fundamental problems in science and engineering
  with potentially broad social, political,
  economic, and scientific impact that can be
  advanced by applying high performance computing
  resources.
• Example of Monte Carlo Methods in Grand
  Challenges
• CHAMMP http//www.epm.ornl.gov/chammp/chammpions
  .html
• Oak Ridge and Argonne National Labs and NCAR
  collaborated to improve NCARs Community Climate
  Model (CCM2).
• A sample visualization of a computer run

25
(No Transcript)
26

• How did Monte Carlo simulation get its name?
• Named for Monte Carlo, Monaco
• where the primary attractions are casinos
  containing games of chance. Games of chance such
  as roulette wheels, dice, and slot machines,
  exhibit random behavior.
• The random behavior in games of chance is similar
  to how Monte Carlo simulation selects variable
  values at random to simulate a model
• The name and the systematic development of Monte
  Carlo methods dates from about 1940s.
• There are however a number of isolated and
  undeveloped instances on much earlier occasions.

27
History of Monte Carlo Method

• In the second half of the nineteenth century a
  number of people performed experiments, in which
  they threw a needle in a haphazard manner onto a
  board ruled with parallel straight lines and
  inferred the value of PI 3.14 from observations
  of the number of intersections between needle and
  lines.
• In 1899 Lord Rayleigh showed that a
  one-dimensional random walk without absorbing
  barriers could provide an approximate solution to
  a parabolic differential equation.

28
History of Monte Carlo method

• In early part of the twentieth century, British
  statistical schools indulged in a fair amount of
  unsophisticated Monte Carlo work.
• In 1908 Student (W.S. Gosset) used experimental
  sampling to help him towards his discovery of the
  distribution of the correlation coefficient.
• In the same year Student also used sampling to
  bolster his faith in his so-called
  t-distribution, which he had derived by a
  somewhat shaky and incomplete theoretical
  analysis.

29
Student - William Sealy Gosset (13.6.1876 -
16.10.1937) This birth-and-death process is
suffering from labor pains it will be the death
of me yet. (Student Sayings)
30
A. N. Kolmogorov (12.4.1903-20.10.1987)
In 1931 Kolmogorov showed the relationship
between Markov stochastic processes and certain
integro-differential equations.
31
History (cont.)

• The real use of Monte Carlo methods as a research
  tool stems from work on the atomic bomb during
  the second world war.
• This work involved a direct simulation of the
  probabilistic problems concerned with random
  neutron diffusion in fissile material but even
  at an early stage of these investigations, von
  Neumann and Ulam refined this particular "Russian
  roulette" and "splitting" methods. However, the
  systematic development of these ideas had to
  await the work of Harris and Herman Kahn in 1948.
  
• About 1948 Fermi, Metropolis, and Ulam obtained
  Monte Carlo estimates for the eigenvalues of
  Schrodinger equation.

32
John von Neumann (28.12.1903-8.2.1957)
33
History (cont.)

• In about 1970, the newly developing theory of
  computational complexity began to provide a more
  precise and persuasive rationale for employing
  the Mont Carlo method.
• Karp (1985) shows this property for estimating
  reliability in a planar multiterminal network
  with randomly failing edges.
• Dyer (1989) establish it for estimating the
  volume of a convex body in M-dimensional
  Euclidean space.
• Broder (1986) and Jerrum and Sinclair (1988)
  establish the property for estimating the
  permanent of a matrix or, equivalently, the
  number of perfect matchings in a bipartite graph.

34
(No Transcript)
35
Summary

• Computational Science
• Requirements of Computational Science
• Introduction to High Performance Computing
  Systems
• Introduction to Monte Carlo Methods
• Definition
• Examples
• History
• Monte Carlo Applications

Learning Monte Carlo Method wouldnt help you win
in Casino.
36
What I want you to do?

• Enjoy your new semester
• Review Slides
• Read the UNIX handbook if you are not familiar
  with UNIX
• No Class on Jan. 11