Dynamic Multigrain Parallelization on the Cell Broadband Engine (PPoPP 07)

1
Dynamic Multigrain Parallelization onthe Cell
Broadband Engine (PPoPP 07)

• Filip Blagojevic, Dimitrios S. Nikolopoulos,
  Alexandros Stamatakis, Christos D. Antonopoulos
• (Virginia Tech)
• Idea and concept
• Experiment design
• My opinions

2
Idea and concept

• Goal - orchestrating and scheduling parallelism
  at multiple levels of granularity on
  heterogeneous multi-core processors
• Approach - adaptive exploitation and scheduling
  of layered parallelism on the Cell Broadband
  Engine
• a scheduler for applications, e.g., RAxML, with
  layered parallelism on Cell

3
Idea and concept

• Cell Broadband Engine (BE) - heterogeneous cores
  and specialized accelerators on the same chip,
  eight high-frequency specialized execution cores
  with pipelined SIMD capabilities, aggressive data
  transfer architecture
• 8 Synergistic Processing Elements (SPEs) - the
  bulk of the Cells computational Power, an SMT
  Power Processing Element (PPE) - a scheduler of
  computations off-loaded to SPEs
• RAxML - computation but not data intensive,
  embarrassingly parallel at the task-level and
  exhibits intrinsic loop-level parallelism within
  each task

4
Idea and concept

• runtime system and scheduling policies chooses
  the form and degree of parallelism to expose to
  the hardware, in response to workload
  fluctuation
• an event-driven multithreading execution engine
• feedback-guided scheduling policy for dynamically
  triggering and throttling loop-level parallelism
  across SPEs on Cell

5
Idea and concept

• Optimize when off-loading to SPEs
• Scheduling task parallelism
• scheduler oversubscribes the PPE with more than
  two MPI processes
• using a user-level scheduler to interleave
  off-loading across MPI processes

6
Idea and concept

• Scheduling loop parallelism
• using work-sharing constructs similar to OpenMP
• select an SPE to distribute loops to and collect
  the results from other SPEs
• The master accumulates the results from all
  workers and commits the sum to main memory

7
Idea and concept

• Adaptive scheduling
• Mix task parallelism and loop parallelism and
  change parallelization strategies and execution
  policies on the fly
• Application specific, based on observation
  history
• Depend on the degree of task-level parallelism
  exposed by each MPI process and the number of
  tasks waiting for off-loading, switch between TLP
  and TLP-LLP

8
Experiment design

• Parallelizing Across Multiple Cells
• Comparison of Cell with Other Processors
• two Intel Xeon processors with Hyper-threading
  technology
• one 64-bit Power5 processor - a quad-thread,
  dual-core processor with dual SMT cores

9
My opinions

• Scheduling strategy is simple, but explored on
  new architecture. Thus involves several
  implementation considerations
• Use one application throughout the paper to prove
  the idea
• Use one section to describe RAxML-VI-HPC and one
  section describe Cell Broadband Engine