Programming Shared Address Space Platforms using OpenMP - PowerPoint PPT Presentation

Loading...

PPT – Programming Shared Address Space Platforms using OpenMP PowerPoint presentation | free to download - id: 1577ad-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Programming Shared Address Space Platforms using OpenMP

Description:

OpenMP Programming: Example. An OpenMP version of a threaded program to compute PI. ... equal sized chunks of size chunk-size and assigns them in a round-robin fashion ... – PowerPoint PPT presentation

Number of Views:50
Avg rating:3.0/5.0
Slides: 32
Provided by: mka90
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Programming Shared Address Space Platforms using OpenMP


1
Programming Shared Address Space Platforms using
OpenMP
  • Ananth Grama, Anshul Gupta, George Karypis, and
    Vipin Kumar
  • To accompany the text Introduction to Parallel
    Computing'', Addison Wesley, 2003.
  • Some modifications by George Hamer and Ken
    Gamradt 2007-2009

2
OpenMP a Standard for Directive Based Parallel
Programming
  • Pthreads are low-level primitives.
  • This requires the programmer to remember/memorize
    many arcane details.
  • A large class of applications can be efficiently
    supported by higher level constructs/directives.
  • Directive based languages have existed for some
    time, but only recently have they become
    standardized.

3
OpenMP a Standard for Directive Based Parallel
Programming
  • OpenMP is a directive-based API that can be used
    with FORTRAN, C, and C for programming shared
    address space machines.
  • OpenMP directives provide support for
    concurrency, synchronization, and data handling
    while obviating the need for explicitly setting
    up mutexes, condition variables, data scope, and
    initialization.

4
OpenMP Programming Model
  • OpenMP directives in C and C are based on the
    pragma compiler directives.
  • A directive consists of a directive name
    followed by clauses.
  • pragma omp directive clause list
  • OpenMP programs execute serially until they
    encounter the parallel directive, which creates a
    group of threads.
  • pragma omp parallel clause list
  • / structured block /
  • The main thread that encounters the parallel
    directive becomes the master of this group of
    threads and is assigned the thread id 0 within
    the group.
  • Each thread created executes the structured block
    created by the parallel directive.

5
OpenMP Programming Model
  • The clause list is used to specify conditional
    parallelization, number of threads, and data
    handling.
  • Conditional Parallelization The clause if
    (scalar expression) determines whether the
    parallel construct results in creation of
    threads.
  • Degree of Concurrency The clause
    num_threads(integer expression) specifies the
    number of threads that are created.
  • Data Handling
  • The clause private (variable list) indicates
    variables local to each thread.
  • The clause firstprivate (variable list) is
    similar to the private, except values of
    variables are initialized to corresponding values
    before the parallel directive.
  • The clause shared (variable list) indicates that
    variables are shared across all the threads.

6
OpenMP Programming Model
  • A sample OpenMP program along with its Pthreads
    translation that might be performed by an OpenMP
    compiler.

7
OpenMP Programming Model
  • pragma omp parallel if (is_parallel 1)
    num_threads(8) \
  • private (a) shared (b) firstprivate(c)
  • / structured block /
  • If the value of the variable is_parallel equals
    one, eight threads are created.
  • Each of these threads gets private copies of
    variables a and c, and shares a single value of
    variable b.
  • The value of each copy of c is initialized to the
    value of c before the parallel directive.
  • The default state of a variable is specified by
    the clause
  • default (shared) implies a variable shared by
    all threads
  • default (none) implies the state of each variable
    must be explicitly defined.

8
Reduction Clause in OpenMP
  • The reduction clause specifies how multiple local
    copies of a variable at different threads are
    combined into a single copy at the master when
    threads exit.
  • The usage of the reduction clause is
  • reduction (operator variable list).
  • The variables in the list are implicitly
    specified as being private to threads.
  • The operator can be one of , , -, , , , ,
    and .
  • pragma omp parallel reduction( sum)
    num_threads(8)
  • / compute local sums here /
  • /sum here contains sum of all local instances of
    sums /

9
Computing PI using OpenMP
  • All variables are local except npoints
  • There will be 8 threads
  • The value of sum is the reduction of all local
    sum variables at thread completion
  • The program is much easier to write than the
    Pthreads version

10
OpenMP Programming Example
  • /
  • An OpenMP version of a threaded program to
    compute PI.

  • /
  • pragma omp parallel default(private) shared
    (npoints) \
  • reduction( sum) num_threads(8)
  • num_threads omp_get_num_threads()
  • sample_points_per_thread npoints / num_threads
  • sum 0
  • for (i 0 i lt sample_points_per_thread i)
  • rand_no_x (double)(rand_r(seed))/(double)((2ltlt14
    )-1)
  • rand_no_y (double)(rand_r(seed))/(double)((2ltlt14
    )-1)
  • if (((rand_no_x - 0.5) (rand_no_x - 0.5)
  • (rand_no_y - 0.5) (rand_no_y - 0.5)) lt 0.25)
  • sum

11
Specifying Concurrent Tasks in OpenMP
  • The parallel directive can be used in conjunction
    with other directives to specify concurrency
    across iterations and tasks.
  • OpenMP provides two directives - for and sections
    - to specify concurrent iterations and tasks.
  • The for directive is used to split parallel
    iteration spaces across threads. The general form
    of a for directive is as follows
  • pragma omp for clause list
  • / for loop /
  • The clauses that can be used in this context are
    private, firstprivate, lastprivate, reduction,
    schedule, nowait, and ordered.

12
Specifying Concurrent Tasks in OpenMP
  • Computing PI using OpenMP directives
  • The for directive specifies that the loop index
    goes from 0 to npoints
  • The loop index is private by default
  • The only difference between this and the previous
    (serial) version is the dirctives
  • This shows how simple it is to convert a serial
    program into an OpenMP threaded program

13
Specifying Concurrent Tasks in OpenMP Example
  • pragma omp parallel default(private) shared
    (npoints) \
  • reduction( sum) num_threads(8)
  • sum 0
  • pragma omp for
  • for (i 0 i lt npoints i)
  • rand_no_x (double)(rand_r(seed))/(double)((2ltlt14
    )-1)
  • rand_no_y (double)(rand_r(seed))/(double)((2ltlt14
    )-1)
  • if (((rand_no_x - 0.5) (rand_no_x - 0.5)
  • (rand_no_y - 0.5) (rand_no_y - 0.5)) lt 0.25)
  • sum

14
Assigning Iterations to Threads
  • The schedule clause of the for directive deals
    with the assignment of iterations to threads.
  • The general form of the schedule directive is
    schedule(scheduling_class, parameter).
  • OpenMP supports four scheduling classes static,
    dynamic, guided, and runtime.

15
Assigning Iterations to Threads
  • Static
  • The general form is
  • Schedule (static,chunk-size)
  • The technique splits the iteration space into
    equal sized chunks of size chunk-size and assigns
    them in a round-robin fashion
  • The iteration space is split to the number of
    threads if no chunk-size is specified
  • If dim128, the size of each partition is 32
    columns
  • Schedule(static, 16) each partition is 16 columns

16
Assigning Iterations to Threads Example
  • for (i 0 i lt dim i) for (j0 j lt dim
    j) ci,j0 for(k0 k lt dim k)
    ci,jai,kbk,j
  • / static scheduling of matrix multiplication
    loops /
  • pragma omp parallel default(private) shared (a,
    b, c, dim) \
  • num_threads(4)
  • pragma omp for schedule(static)
  • for (i 0 i lt dim i)
  • for (j 0 j lt dim j)
  • Ci,j 0
  • for (k 0 k lt dim k)
  • Ci,j ai, k bk, j

17
Assigning Iterations to Threads Example
  • Three different schedules using the static
    scheduling class of OpenMP.

18
Specifying Concurrent Tasks in OpenMP
  • Dynamic
  • The general form is
  • Schedule(dynamic,chunk-size)
  • Assigned to threads as they become idle
  • The chunk-size defaults to one if none is
    specified
  • Guided
  • The general for is
  • Schedule(guided,chunk-size)
  • Chunk size is reduced exponentially as each chunk
    is dispatched
  • When the number of interations left is less than
    chunk-size, the entire set of iterations is
    dispatched at once
  • The chunk-size defaults to one if none is
    specified
  • Runtime
  • It may be desirable to delay scheduling until
    runtime
  • The environment variable OMP_SCHEDULE determines
    the scheduling class and chunk-size
  • When no scheduling class is specified with the
    omp for directive, the actual scheduling
    technique is not specified and is implementation
    dependent.
  • In this case server restrictions are applied to
    the for loop

19
Parallel For Loops
  • Often, it is desirable to have a sequence of
    for-directives within a parallel construct that
    do not execute an implicit barrier at the end of
    each for directive.
  • OpenMP provides a clause - nowait, which can be
    used with a for directive.
  • In the example that follows, the nowait clause is
    used to prevent idling
  • If the name is in the current_list a thread does
    not have to wait for other threads to finish
    before proceeding to the past_list

20
Parallel For Loops Example
  • pragma omp parallel
  • pragma omp for nowait
  • for (i 0 i lt nmax i)
  • if (isEqual(name, current_listi)
  • processCurrentName(name)
  • pragma omp for
  • for (i 0 i lt mmax i)
  • if (isEqual(name, past_listi)
  • processPastName(name)

21
The sections Directive
  • OpenMP supports non-iterative parallel task
    assignment using the sections directive.
  • The general form of the sections directive is as
    follows
  • pragma omp sections clause list
  • pragma omp section
  • / structured block /
  • pragma omp section
  • / structured block /
  • ...

22
The sections Directive Example
  • The sections directive assigns the structured
    block corresponding to each section to one thread
  • The clause directive may include
  • private, firstprivate, lastprivate, reduction,
    and nowait
  • lastprivate specifies that the last section of
    the sections directive updates the value of the
    variable
  • nowait specifies that there is no implicit
    syncronization among all thread at the end of the
    sections directive
  • It is illegal to branch in and/or out of a
    section block
  • pragma omp parallel
  • pragma omp sections
  • pragma omp section
  • taskA()
  • pragma omp section
  • taskB()
  • pragma omp section
  • taskC()

23
Merging Directives
  • Not merged
  • pragma omp parallel\ default (private) shared
    (n) pragma omp for for(i0 iltn i)
    / body of parallel for /
  • pragma com parallel pragma sections
    pragma omp section taskA()
    pragma omp section
    taskB() / other sections /
  • Merged
  • pragma omp parallel for \ default (private)
    shared (n) for(i0 i lt n i) /
    body of parallel for /
  • pragma omp parallel sections pragma omp
    section taskA() pragma omp
    section taskB() / other sections
    /

24
Nesting parallel Directives
  • Nested parallelism can be enabled using the
    OMP_NESTED environment variable.
  • If the OMP_NESTED environment variable is set to
    TRUE, nested parallelism is enabled.
  • In this case, each parallel directive creates a
    new team of threads.
  • pragma omp parallel for \ default (private)
    shared(a,b,c,dim)\ num_threads(2) for (i0 i lt
    dim i) pragma omp parallel for \ default
    (private) shared(a,b,c,dim) \ num_threads(2)
    for(j0 jlt dim j) ci,j0
    pragma omp parallel for \ default
    (private) shared \ (a,b,c,dim)
    num_threads(2) for(k0 k lt dim k)
    ci,jai,kbk,j

25
Synchronization Constructs in OpenMP
  • OpenMP provides a variety of synchronization
    constructs
  • Synchronization Point pragma omp barrier /all
    wait then release/
  • Single Thread Executions pragma omp
    singleclause list
  • structured block /only a single thread
    executes/
  • pragma omp master
  • structured block /only master thread executes/
  • Critical Sections pragma omp critical (name)
  • structured block /implements critical
    region/ pragma omp atomic expression
    statement /memory update atomic /
  • In-Order Execution pragma omp ordered
  • structured block / serial version execution/
  • Memory Consistency pragma omp flush(list)/
    all threads have same/

26
Synchronization Constructs
  • Producer-Consumer pragma omp parallel
    sections pragma omp parallel section
    / producer thread / task
    produce_task() pragma omp critical
    (task_q) insert_into_queue(task)
    pragma omp parallel section
    / consumer thread / pragma omp
    critical (task_q)
    taskextract_from_queue(task)
    consume_task(task)
  • Cumulative sum cumul_sum0 list0 pragma
    omp parallel for \ private (i)shared
    (cumul_sum, list, n) \ ordered for(i1 i
    lt n i) / other processing on list as
    needed / pragma omp ordered
    cumul_sumi cumul_sumi-1 listi

27
Data Handling in OpenMP
  • One of the critical factors influencing program
    performance is the manipulation of data by
    threads
  • If a thread initializes and uses a variable
    exclusively, then a local private copy should be
    made for the thread.
  • If a thread repeatedly reads a variable that was
    initialized earlier in the program, then a local
    firstprivate copy that inherits the value should
    be made for the thread.
  • If multiple threads manipulate a single piece of
    data, then break these manipulations into local
    operations followed by a single global operation
    using a clause such as the reduction clause.
  • If multiple threads manipulate different parts of
    a large data structure, then break the data
    structure into smaller date structures making
    them private for the manipulating thread.
  • The remaining data items may be shared among all
    threads.

28
OpenMP Library Functions
  • In addition to directives, OpenMP also supports a
    number of functions that allow a programmer to
    control the execution of threaded programs.
  • / thread and processor count /
  • void omp_set_num_threads (int num_threads)
  • int omp_get_num_threads ()
  • int omp_get_max_threads ()
  • int omp_get_thread_num ()
  • int omp_get_num_procs ()
  • int omp_in_parallel()

29
OpenMP Library Functions
  • / controlling and monitoring thread creation /
  • void omp_set_dynamic (int dynamic_threads)
  • int omp_get_dynamic ()
  • void omp_set_nested (int nested)
  • int omp_get_nested ()
  • / mutual exclusion /
  • void omp_init_lock (omp_lock_t lock)
  • void omp_destroy_lock (omp_lock_t lock)
  • void omp_set_lock (omp_lock_t lock)
  • void omp_unset_lock (omp_lock_t lock)
  • int omp_test_lock (omp_lock_t lock)
  • In addition, all lock routines also have a nested
    lock counterpart for recursive mutexes.
  • void omp_int_nest_lock(omp_nest_lock_t lock)

30
Environment Variables in OpenMP
  • OMP_NUM_THREADS This environment variable
    specifies the default number of threads created
    upon entering a parallel region.
  • OMP_SET_DYNAMIC Determines if the number of
    threads can be dynamically changed.
  • OMP_NESTED Turns on nested parallelism.
  • OMP_SCHEDULE Scheduling of for-loops if the
    clause specifies runtime

31
Explicit Threads versus Directive Based
Programming
  • Directives layered on top of threads facilitate a
    variety of thread-related tasks.
  • A programmer is rid of the tasks of initializing
    attributes objects, setting up arguments to
    threads, partitioning iteration spaces, etc.
  • There are some drawbacks to using directives as
    well.
  • An artifact of explicit threading is that data
    exchange is more apparent.
  • This helps in alleviating some of the overheads
    from data movement, false sharing, and
    contention.
  • Explicit threading also provides a richer API in
    the form of condition waits, locks of different
    types, and increased flexibility for building
    composite synchronization operations.
  • Finally, since explicit threading is used more
    widely than OpenMP, tools and support for
    Pthreads programs are easier to find.
About PowerShow.com