Shared Memory Parallel Programming - PowerPoint PPT Presentation

Loading...

PPT – Shared Memory Parallel Programming PowerPoint presentation | free to view - id: 175ae5-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Shared Memory Parallel Programming

Description:

Recall we distinguished several kinds of parallel programming models ... A thread can terminate another thread during the exceution: ... – PowerPoint PPT presentation

Number of Views:45
Avg rating:3.0/5.0
Slides: 42
Provided by: barbara179
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Shared Memory Parallel Programming


1
Shared Memory Parallel Programming
  • Pthreads

2
Parallel Programming Models
  • Recall we distinguished several kinds of parallel
    programming models
  • Low-level parallel programming models
  • pthreads for shared memory, MPI for distributed
    memory
  • High-level parallel programming
  • OpenMP for shared memory, HPF for distributed
    memory

3
Parallel Programming Models
  • High-level parallel programming
  • programmer describes parallelism implicitly
  • details of data and computation to be performed
    on each processor determined by compiler
  • different approaches for shared memory and
    distributed memory models

4
Parallel Programming Models
  • Low-level parallel programming
  • programmer must describe parallelism explicitly
  • data and computation to be performed on each
    processor specified exactly by coder
  • different approaches for shared memory and
    distributed memory models

5
Parallel Programming Models
  • Options for shared memory parallel programming
  • OpenMP
  • Vendor-specific directives
  • Mostly superseded by OpenMP
  • Unix utilities
  • Fork, shmget
  • Explicit threading
  • E.g. Pthreads, Solaris UI threads
  • Distributed memory API on shared memory

6
Reminder Threads
  • Process has its own address space
  • A process may be executed by a team of threads
  • A thread shares its address space with other
    threads in same team
  • But thread stack provides space for data local
    (private) to thread
  • Threads used for shared memory parallel
    programming and multitasking

7
Pthreads
  • IEEE POSIX standard developed to replace
    different vendor specific multithreading
    libraries
  • Implementation called Pthreads or Posix threads
  • Available on almost all Unix systems
  • Offers a low-level programming interface
  • No specific compiler support

Library is libpthread on most systems
8
User-Level Multithreading
  • Explicit shared memory parallel programming
  • User fully specifies actions of each thread and
    synchronizes (coordinates) threads
  • The good part control can be used to performance
    benefit, can explicitly consider e.g. data
    locality
  • The bad part can be complex, user has to deal
    with it

9
Applicability of Pthreads
  • POSIX standard shared-memory multithreading
    interface
  • For general multithreaded programming
  • With C, not Fortran
  • Provides primitives for process management and
    synchronization
  • But missing many conveniences

Note std. makes assumptions about memory
consistency
10
Rough Comparison with OpenMP
  • Typically more work to create program
  • Not incremental
  • Can dynamically spawn (create) threads
  • Better support for exception and signal handling
  • Supports unstructured control flow
  • Missing many conveniences such as atomic,
    barrier, flush primitives, reductions,

Suited to different kinds of applications
11
Uses of Pthreads
  • Client-server applications
  • GUI handlers
  • Signal handlers
  • Pipelined applications
  • Can be used wherever OpenMP can be used, but
    possibly with much more effort
  • (Much) higher maintenance costs
  • Many more changes to code

12
Pthreads
  • We only look briefly at a subset of Pthreads
  • For complete information, many good references
    exist
  • Multithreaded Programming With Pthreads, Bill
    Lewis, Daniel J. Berg
  • Pthreads Programming, Bradford Nichols, Dick
    Buttlar, Jacqueline Proulx Farrell, Jackie
    Farrell
  • www.mit.edu/people/proven/pthreads.html

13
What does the user have to do?
  • Decide how to decompose the computation into
    parallel parts
  • Create (and destroy) processes to implement that
    decomposition
  • Add synchronization to make sure dependences are
    respected

14
General Thread Structure
  • Typically, a thread is a concurrent execution of
    a function or a procedure
  • A program needs to be restructured so that
    parallel parts form separate procedures or
    functions

15
Thread Creation
  • int pthread_create
  • (pthread_t new_id,
  • const pthread_attr_t attr,
  • void (func) (void ),
  • void arg)
  • new_id threads unique identifier
  • attr ignore for now
  • func function to be run in parallel
  • arg arguments for function func

16
Example of Thread Creation
  • void func(void arg)
  • int Iarg
  • ..
  • void main()
  • int X pthread_t id
  • .
  • pthread_create(id, NULL, func, X)

17
Example of Thread Creation (contd.)
main()
pthread_ create(func)
func()
18
Example Hello World
include ltpthread.hgt include ltstdio.hgt define
NUM_THREADS 4 void PrintHello(void threadid)
printf("\nd Hello World!\n", threadid)
pthread_exit(NULL) int main (int argc, char
argv) pthread_t threadsNUM_THREADS
int rc, t for(t0 tltNUM_THREADS t)
printf("Creating thread d\n", t)
rc pthread_create(threadst, NULL,
PrintHello, (void )t) if (rc)
printf("ERROR return code from
pthread_create() is d\n", rc)
exit(-1)
pthread_exit(NULL)
19
OpenMP Hello World Example
include omp.h int main(int argc, char argv)
pragma omp parallel
printf(d, Hello, world!\n,omp_get_thread_num())
return 0
20
Compiling Pthreads Applications
  • gcc hello.c lpthread
  • icc hello.c lpthread
  • g hello.c -lpthread

21
Pthread Termination
  • void pthread_exit(void status)
  • Terminates the currently running thread
  • Implicitly called when the function invoked in
    pthread_create returns

22
Cancelling a thread
A thread can terminate another thread during the
exceution int pthread_cancel (pthread_t thread)
int pthread_setcancelstate (int state, int
oldstate) int pthread_setcanceltype (int type,
int oldstype)
23
Example Cancelling a thread
int main() pthread_t e_th
pthread_t f_th int rc /
creates both threads / rc
pthread_create(e_th, NULL, Thread, (void
)e_str) if (rc) return
-1 rc pthread_create(f_th, NULL,
Thread, (void )f_str) if (rc)
return -1 / sleeps a while /
sleep(10) / requests
cancellation / pthread_cancel(e_th)
pthread_cancel(f_th) / sleeps a bit
more / sleep(10)
pthread_exit(NULL)
void Thread(void string) int i
int o_state / disables cancelability
/ pthread_setcancelstate(PTHREAD_CANCEL_DI
SABLE, o_state) / writes five
messages / for (i0 ilt5 i)
printf("s\n", (char )string) /
restores cancelability /
pthread_setcancelstate(o_state, o_state)
/ writes further / while (1)
printf("s\n", (char )string)
pthread_exit(NULL)
24
Thread Joining
  • int pthread_join(
  • pthread_t new_id,
  • void status)
  • Waits for the thread with identifier new_id to
    terminate, either by returning or by calling
    pthread_exit()
  • Status receives the return value or the value
    given as argument to pthread_exit()

25
Thread Joining Example
  • void func(void ) ..
  • pthread_t id int X
  • pthread_create(id, NULL, func, X)
  • ..
  • pthread_join(id, NULL)
  • ..

26
Example of Thread Creation (contd.)
main()
pthread_ create(func)
func()
pthread_ join(id)
pthread_ exit()
27
Attributes
  • pthread_attr_init(pthread_attr_t attr)
  • pthread_attr_destroy(pthread_attr_t attr)
  • pthread_attr_setXXX()
  • pthread_attr_setdetachstate(pthread_attr_t attr,
    int detachstate)
  • pthread_attr_setstacksize (attr, stacksize)

28
Matrix Multiply
  • for( i0 iltn i )
  • for( j0 jltn j )
  • cij 0.0
  • for( k0 kltn k )
  • cij aikbkj

29
Parallel Matrix Multiply
  • All i- or j-iterations can be run in parallel
  • If we have p processors (or cores), assign n/p
    rows to each processor
  • Corresponds to partitioning i-loop
  • Create a thread for each part
  • Threads all execute same function, but on
    different data sets

30
Matrix Multiply Parallel Part
  • void mmult(void s)
  • int slice (int) s
  • int from (slicen)/p
  • int to ((slice1)n)/p
  • for(ifrom iltto i)
  • for(j0 jltn j)
  • cij 0.0
  • for(k0 kltn k)
  • cij aikbkj

31
Matrix Multiply Main
  • int main()
  • pthread_t thrdp
  • for( i0 iltp i )
  • pthread_create(thrdi, NULL, mmult,(void)
    i)
  • for( i0 iltp i )
  • pthread_join(thrdi, NULL)

32
Sequential Jacobi Code
  • for some number of timesteps/iterations
  • for (i0 iltn i )
  • for( j1, jltn, j )
  • tempij 0.25
  • ( gridi-1j gridi1j
  • gridij-1 gridij1 )
  • for( i0 iltn i )
  • for( j1 jltn j )
  • gridij tempij

33
Recall Parallel Jacobi
  • First (i,j) loop nest can be parallelized
  • Second (i,j) loop nest can be parallelized
  • Barrier after each loop nest
  • Give n/p rows to each processor
  • Again, each thread executes same function on its
    own data (data parallel)

34
Pthreads Jacobi Parallel parts (1)
  • void jac_1(void s)
  • int slice (int) s
  • int from (slicen)/p
  • int to ((slice1)n)/p
  • for( ifrom iltto i)
  • for( j0 jltn j )
  • tempij 0.25(gridi-1j gridi1j
  • gridij-1 gridij1)

35
Pthreads Jacobi Parallel parts (2)
  • void jac_2(void s)
  • int slice (int) s
  • int from (slicen)/p
  • int to ((slice1)n)/p
  • for( ifrom iltto i)
  • for( j0 jltn j )
  • gridij tempij

36
Pthreads Jacobi main
  • for some number of timesteps
  • for( i0 iltp i )
  • pthread_create(thrdi, NULL, jac_1, (void
    )i)
  • for( i0 iltp i )
  • pthread_join(thrdi, NULL)
  • for( i0 iltp i )
  • pthread_create(thrdi, NULL, jac_2, (void
    )i)
  • for( i0 iltp i )
  • pthread_join(thrdi, NULL)

37
Synchronizations
  • mutexes - Mutual exclusion lock Block access to
    variables by other threads. This enforces
    exclusive access by a thread to a variable or set
    of variables.
  • joins - Make a thread wait till others are
    complete (terminated).
  • condition variables - data type pthread_cond_t

38
Mutex Example
/ Note scope of variable and mutex are the same
/ pthread_mutex_t mutex1 PTHREAD_MUTEX_INITIALI
ZER int counter0 / Function C / void
functionC() pthread_mutex_lock( mutex1 )
counter pthread_mutex_unlock( mutex1 )
39
Conditional Variables
  • A condition variable is a variable of type
    pthread_cond_t and is used with the appropriate
    functions for waiting and later, process
    continuation.
  • The condition variable mechanism allows threads
    to suspend execution and relinquish the processor
    until some condition is true.

40
Conditional Variable Functions
  • Creating/Destroying
  • pthread_cond_init
  • pthread_cond_t cond PTHREAD_COND_INITIALIZER
  • pthread_cond_destroy
  • Waiting on condition
  • pthread_cond_wait
  • pthread_cond_timedwait - place limit on how long
    it will block.
  • Waking thread based on condition
  • pthread_cond_signal
  • pthread_cond_broadcast - wake up all threads
    blocked by the specified condition variable.

41
Thread Pitfalls
  • Race Conditions
  • Thread safe code
  • Deadlocks
About PowerShow.com