Programming Models and Memory Systems A Brief History of Failure

1
Programming Models and Memory Systems(A Brief
History of Failure?)

• Ian Watson
• School of Computer Science
• University of Manchester
• UK

2
Assumptions

• Single threaded processor performance has reached
  a limit
• Multi-core is the present. Many-core is the
  future
• Many applications will be general purpose
• We dont know how to do (simple?) general purpose
  parallel programming
• But also we dont know how to build the
  parallel hardware

3
Disclaimer

• These are personal opinions
• Some of my comments may imply criticism of
  various approaches
• If I say anything that upsets anyone, apologies.
  No offence is intended!

4
Who Am I?

• The Manchester Dataflow Computer (Gurd Watson
  CACM 1985)
• Originally proposed 1976
• Prototype operational 1981
• Flagship A Parallel Architecture for Declarative
  Programming (ISCA 1988 Watson, Woods, Watson,
  Banach, Greenberg Sargeant)
• UFO United Functions Objects (Sargeant,
  Kirkham Watson circa 1996
• JAMAICA Chip multiprocessor with lightweight
  threads Java run time parallelization using a
  virtual machine. (1999 - .. )

5
Contributions

• 30 years in pursuit of the Holy Grail the
  easily programmable, infinitely extensible,
  general purpose parallel machine.
• Have designed / simulated / built many different
  attempts
• Have explored numerous programming languages,
  programming models and hardware structures.
• Major expertise I know a large number of ways
  how NOT to do it!
• Hopefully some of this expertise is relevant now
  that many-core general purpose computing is set
  to become mainstream

6
Surviving Beliefs I still have some!

• A global view of (at least some) shared memory is
  essential
• It is a (big) mistake to design clever parallel
  hardware and then work out how to program it
• It is usually a mistake to design parallel
  programming languages and then work out how to
  implement them
• The key is to develop the right computational
  model(s) alongside languages hardware
• When it comes to implementing a parallel machine,
  the memory architecture is the critical component

7
Programming Models

• Shared state is inevitable dont try to pretend
  it isnt
• Shared state is the enemy of parallelism use
  sparingly
• Data parallelism is only part of the answer
• Declarative programming is really nice simple
  use it if at all possible
• Transactional Memory is an attractive way of
  handling shared state
• But, above all, dont try to over sell any
  particular approach, it is unlikely that any one
  alone is the answer!

8
Memory Architecture

• First choose a programming model which minimizes
  the need for coherence
• Many-core systems should not attempt to implement
  fine grain coherence across all memory
• Memory needs only to be coherent at certain
  points and then only those parts which are being
  shared
• This ought to be reflected in the programming
  model if at all possible
• Bus snooping and/or global broadcast is
  probably not going to be an extensible solution

9
Conclusions

• Think about systems, not just hardware or
  software
• There is lots of (possibly) relevant work e.g.
• Dataflow (Single Assignment)
• Graph Rewriting (Functional Languages)
• Bulk Synchronous Parallelism (BSP)
• Transactional Memory
• None have yet been the answer but all have merit
• Dont ignore previous work and particularly dont
  re-invent the wheel!
• Keep it simple! (as simple as possible)