Parallel Programming with Java - PowerPoint PPT Presentation

Loading...

PPT – Parallel Programming with Java PowerPoint presentation | free to download - id: 71348b-NmVjN



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Parallel Programming with Java

Description:

Title: Computer System Overview Author: Patricia Roy Last modified by: Aamir Shafi Created Date: 6/26/1999 9:48:38 PM Document presentation format – PowerPoint PPT presentation

Number of Views:62
Avg rating:3.0/5.0
Slides: 38
Provided by: Patric561
Learn more at: http://mpj-express.org
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Parallel Programming with Java


1
Parallel Programming with Java
  • Aamir Shafi
  • National University of Sciences and Technology
    (NUST)
  • http//hpc.seecs.edu.pk/aamir
  • http//mpj-express.org

2
Two Important Concepts
  • Two fundamental concepts of parallel programming
    are
  • Domain decomposition
  • Functional decomposition

3
Domain Decomposition
Image taken from https//computing.llnl.gov/tutori
als/parallel_comp/
4
Functional Decomposition
Image taken from https//computing.llnl.gov/tutori
als/parallel_comp/
5
Message Passing Interface (MPI)
  • MPI is a standard (an interface or an API)
  • It defines a set of methods that are used by
    application developers to write their
    applications
  • MPI library implement these methods
  • MPI itself is not a libraryit is a specification
    document that is followed!
  • MPI-1.2 is the most popular specification version
  • Reasons for popularity
  • Software and hardware vendors were involved
  • Significant contribution from academia
  • MPICH served as an early reference implementation
  • MPI compilers are simply wrappers to widely used
    C and Fortran compilers
  • History
  • The first draft specification was produced in
    1993
  • MPI-2.0, introduced in 1999, adds many new
    features to MPI
  • Bindings available to C, C, and Fortran
  • MPI is a success story
  • It is the mostly adopted programming paradigm of
    IBM Blue Gene systems
  • At least two production-quality MPI libraries
  • MPICH2 (http//www-unix.mcs.anl.gov/mpi/mpich2/)
  • OpenMPI (http//open-mpi.org)

6
Message Passing Model
  • Message passing model allows processors to
    communicate by passing messages
  • Processors do not share memory
  • Data transfer between processors required
    cooperative operations to be performed by each
    processor
  • One processor sends the message while other
    receives the message

7
Distributed Memory Cluster
barq.niit.edu.pk (cluster head node)
chenab1
chenab2
chenab3
chenab7
chenab6
chenab4
chenab5
8
Steps involved in executing the Hello World!
program
  1. Lets logon to the cluster head node
  2. Write the Hello World program
  3. Compile the program
  4. Write the machines files
  5. Start MPJ Express daemons
  6. Execute the parallel program
  7. Stop MPJ Express daemons

9
Step1 Logon to the head node
10
Step 2 Write the Hello World Program
11
Step 3 Compile the code
12
Step 4 Write the machines file
13
Step 5 Start MPJ Express daemons
14
Step 6 Execute the parallel program
  • aamir_at_barq/projects/mpj-usergt mpjrun.sh -np 6
    -headnodeip 10.3.20.120 -dport 11050 HelloWorld
  • ..
  • Hi from process lt3gt of total lt6gt
  • Hi from process lt1gt of total lt6gt
  • Hi from process lt2gt of total lt6gt
  • Hi from process lt4gt of total lt6gt
  • Hi from process lt5gt of total lt6gt
  • Hi from process lt0gt of total lt6gt

15
Step 7 Stop the MPJ Express daemons
16
COMM WORLD Communicator
  • import java.util.
  • import mpi.
  • ..
  • // Initialize MPI
  • MPI.Init(args) // start up MPI
  • // Get total number of processes and rank
  • size MPI.COMM_WORLD.Size()
  • rank MPI.COMM_WORLD.Rank()
  • ..

17
What is size?
import java.util. import mpi. .. // Get
total number of processes size
MPI.COMM_WORLD.Size() ..
  • Total number of processes in a communicator
  • The size of MPI.COMM_WORLD is 6

18
What is rank?
import java.util. import mpi. .. // Get
total number of processes rank
MPI.COMM_WORLD.Rank() ..
  • The unique identify (id) of a process in a
    communicator
  • Each of the six processes in MPI.COMM_WORLD has a
    distinct rank or id

19
Single Program Multiple Data (SPMD) Model
  • import java.util.
  • import mpi.
  • public class HelloWorld
  • MPI.Init(args) // start up MPI
  • size MPI.COMM_WORLD.Size()
  • rank MPI.COMM_WORLD.Rank()
  • if (rank 0)
  • System.out.println(I am Process 0)
  • else if (rank 1)
  • System.out.println(I am Process 1)
  • MPI.Finalize()

20
Single Program Multiple Data (SPMD) Model
  • import java.util.
  • import mpi.
  • public class HelloWorld
  • MPI.Init(args) // start up MPI
  • size MPI.COMM_WORLD.Size()
  • rank MPI.COMM_WORLD.Rank()
  • if (rank2 0)
  • System.out.println(I am an even process)
  • else if (rank2 1)
  • System.out.println(I am an odd process)
  • MPI.Finalize()

21
Point to Point Communication
  • The most fundamental facility provided by MPI
  • Basically exchange messages between two
    processes
  • One process (source) sends message
  • The other process (destination) receives message

22
Point to Point Communication
  • It is possible to send message for each basic
    datatype
  • Floats (MPI.FLOAT), Integers (MPI.INT), Doubles
    (MPI.DOUBLE)
  • Java Objects (MPI.OBJECT)
  • Each message contains a tagan identifier

Tag1
Tag2
23
Point to Point Communication
Process 1
Process 2
Process 0
message
Process 3
Process 7
Process 6
Process 4
Process 5
24
Blocking Send() and Recv() Methods
  • public void Send(Object buf, int offset, int
    count,
  • Datatype datatype, int dest, int tag)
    throws MPIException
  • public Status Recv(Object buf, int offset, int
    count,
  • Datatype datatype, int src,
    int tag) throws MPIException

25
Blocking and Non-blocking Point-to-Point Comm
  • There are blocking and non-blocking version of
    send and receive methods
  • Blocking versions
  • A process calls Send() or Recv(), these methods
    return when the message has been physically sent
    or received
  • Non-blocking versions
  • A process calls Isend() or Irecv(), these methods
    return immediately
  • The user can check the status of message by
    calling Test() or Wait()
  • Non-blocking versions provide overlapping of
    computation and communication
  • Asynchronous communication

26
(No Transcript)
27
Non-blocking Point-to-Point Comm
  • public Request Isend(Object buf, int offset, int
    count,
  • Datatype datatype, int dest, int tag)
    throws MPIException
  • public Request Irecv(Object buf, int offset, int
    count,
  • Datatype datatype, int src, int
    tag) throws MPIException
  • public Status Wait() throws MPIException
  • public Status Test() throws MPIException

28
Performance Evaluation of Point to Point
Communication
  • Normally ping pong benchmarks are used to
    calculate
  • Latency How long it takes to send N bytes from
    sender to receiver?
  • Throughput How much bandwidth is achieved?
  • Latency is a useful measure for studying the
    performance of small messages
  • Throughput is a useful measure for studying the
    performance of large messages

29
Latency on Myrinet
30
Throughput on Myrinet
31
Collective communications
  • Provided as a convenience for application
    developers
  • Save significant development time
  • Efficient algorithms may be used
  • Stable (tested)
  • Built on top of point-to-point communications
  • These operations include
  • Broadcast, Barrier, Reduce, Allreduce, Alltoall,
    Scatter, Scan, Allscatter
  • Versions that allows displacements between the
    data

32
Broadcast, scatter, gather, allgather, alltoall
Image from MPI standard doc
33
Broadcast, scatter, gather, allgather, alltoall
  • public void Bcast(Object buf, int offset, int
    count,
  • Datatype type, int
    root) throws MPIException
  • public void Scatter(Object sendbuf, int
    sendoffset, int sendcount, Datatype sendtype,
  • Object recvbuf,
    int recvoffset, int recvcount, Datatype recvtype,
  • int root) throws
    MPIException
  • public void Gather(Object sendbuf, int
    sendoffset, int sendcount, Datatype sendtype,
  • Object recvbuf,
    int recvoffset, int recvcount, Datatype recvtype,
  • int root) throws
    MPIException
  • public void Allgather(Object sendbuf, int
    sendoffset int sendcount, Datatype sendtype,
    Object recvbuf, int
    recvoffset, int recvcount, Datatype recvtype)
    throws MPIException
  • public void Alltoall(Object sendbuf, int
    sendoffset, int sendcount, Datatype sendtype,
  • Object recvbuf,
    int recvoffset, int recvcount, Datatype recvtype)
  • throws MPIException

34
Reduce collective operations
Processes
  • MPI.PROD
  • MPI.SUM
  • MPI.MIN
  • MPI.MAX
  • MPI.LAND
  • MPI.BAND
  • MPI.LOR
  • MPI.BOR
  • MPI.LXOR
  • MPI.BXOR
  • MPI.MINLOC
  • MPI.MAXLOC

35
Reduce collective operations
  • public void Reduce(Object sendbuf, int
    sendoffset,
  • Object recvbuf,
    int recvoffset, int count,
  • Datatype
    datatype, Op op, int root)
  • throws
    MPIException
  • public void Allreduce(Object sendbuf, int
    sendoffset,
  • Object recvbuf,
    int recvoffset, int count,
  • Datatype
    datatype, Op op)
  • throws
    MPIException

36
Collective Communication Performance
37
Summary
  • MPJ Express is a Java messaging system that can
    be used to write parallel applications
  • MPJ/Ibis and mpiJava are other similar software
  • MPJ Express provides point-to-point communication
    methods like Send() and Recv()
  • Blocking and non-blocking versions
  • Collective communications is also supported
  • Feel free to contact me if you have any queries
About PowerShow.com