Total Order Broadcast and Multicast Algorithms: Taxonomy and Survey Paper by X' Dfago, A' Schiper, a

1
Total Order Broadcast and Multicast Algorithms
Taxonomy and Survey (Paper by X. Défago, A.
Schiper, and P. Urbán)ACM computing Surveys,
Vol. 36,No 4, Dec 2004, pp. 372-421

• Aida Omerovic
• 4. March 2008
• Seminar on Dependable and
• Adaptive Distributed Systems

2
Outline

• Background
• Problem specification
• Classes of ordering mechanisms
• Failure related concepts
• Fault tolerance
• Discussion

3
Background

• Total order broadcast and multicast algorithms
• Both synchronous and asynchrous system models
• Lack of a roadmap for use of the algorithms.
• Lack of generality of existing comparissons.

4
Notions, terms

• Broadcast (messages are sent to all processes)
  vs.
• Multicast (messages are sent to a subset of
  processes)
• Closed vs. open groups (belonging of the sender)
• Single vs. multiple groups (disjoint/overlapping)
  
• ensuring total order at intersection of groups
• Dynamic groups
• Processes join and leave at runtime
• Partitionabe groups
• Splitting of groups into subgroups through
  primary partition membership or partitionable
  membership

5
Motivation

• Concurrency and global control in distributed
  systems
• Total order broadcast a group communication
  primitive
• Ensures that messages sent to a set of processes
  are delivered by all those processes in the same
  order
• Important in clock synchronisation, active
  replication, distributed shared memory,
  distributed mutual exclusion, cooperative
  writing, replicated databases performance

6
Main contributions

• Classification w.r.t. ordering mechanisms
• Characteristic with the strongest influence on
  the behavior
• Definition of five classes of ordering mechanisms
• Survey of approx. 60 published total order
  broadcast algorithms.
• Study of properties and behaviour

7
A correct process def.

• A correct process never expresses any of the
  faulty behaviors
• Crash failures (stops performing any activity)
• Omission failures (omits performing some actions)
• Timing faulures (violation of system time).
  Applies only to synchronous systems.
• Byzantine failures. Performs arbitrary faulty
  behaviour.

8
The problem specification

• The total order broadcast problem specification
• Two primitives
• TO-broadcast(m) For eny message, and any run
  executed at most once!
• TO-deliver(m)
• Properties of total order broadcast
• Validity (if a correct p TO-broadcasts m-gtp
  TO-delivers m)
• Uniform agreement (if a p TO-delivers m-gtall
  correct ps TO-deliver m)
• Uniform integrity (every p TO-delivers m at most
  once and only if m was previously TO-broadcast by
  sender)
• Uniform total order (if processes p and q both
  TO-deliver m and m then p TO-delivers m before
  m iff q TO-delivers m before m)

9
The problem specification cont.

• Properties 1, 2 and 3 satisfied -gt reliable
  broadcast.
• Properties 1 and 2 liveness properties.
  (Property may eventually hold, regardless.)
• Properties 3 and 4 safety properties. (Once
  the property does not hold, it never will).
• Properties 2 and 4 uniform. (Apply to both
  correct and faulty processes.) Costly. Algorithms
  tolerant to Byzantine failures can not guarantee
  any of the uniform properties above.
• Nonuniform Neither 2 nor 4 hold. Apply only to
  correct processes, no restr. on the faulty ones.
  Voting can be a measure.

10
The problem specification cont.

• Alternative uniform processes are those enforced
  by honest processes, correct or not. (Honest
  process behaves according to its specification.)
• An issue contamination. (A faulty process in an
  inconsistent state legally TO broadcasts a
  message, prior to crashing, thus contaminating
  the correct processes.)
• Note satisfies even the strongest specification
  so far.
• This is disallowed by
• gap-free uniform total order (no gaps in the
  delivery sequence.)
• prefix order (history of ane process is a
  prefix of the history of the other.)
• However, contamination can not be avoided in case
  of arbitrary failures (e.g. correct delivery by
  faulty process.)

11
The problem specification cont.

• Other ordering properties include
• FIFO order. Delivery of messages in the order in
  which they are sent (not guaranteed by total
  order).
• Causal order (m precedes m if sending event of m
  precedes the sending event of m).
• Generally broadcast of m before m, implies
  delivery of m before m by correct processes.
• Note these two properties further restrict total
  order property definition by properties related
  to SENDERS.
• Causal order lt-gt FIFO order Local order

12
Classes of ordering mechanisms

• according to how the ordering (e.g. timestamp,
  sequence number) is performed and by whom (type
  of role).
• Process roles sender, destination, sequencer.
• Five classes of total order broadcast algorithms
• Fixed sequencer (sequencer)
• Moving sequencer (sequencer) Token
• Privilege based (sender) Token
• Communication history (sender) Timestamp
• Destinations agreement (destination) Timestamp
• Another distinction is between time-free and
  time-based (physical time) ordering.

13
Classes of ordering mechanisms cont.
Neither of the five is failure tolerant!!!
14
Failure related conceptual issues
Synchronous system a system where upper bounds
on process speed interval and communication
delay, are set. Asynchronous system the two
parameters are unbounded. Timed asynchronous
model asynchronous model with notion of physical
time and assumption that most of the messages
are likely to reach their destination within a
delay d.
15
Failure related conceptual issues cont.
Concensus in asynchronous systems if just a
single process can crash, has no deterministic
solutiuon. Total order broadcast can be
transformed into concensus -gt the impossibility
holds also here! Solution extent the
asynchronous system with oracles. An oracle
provides information that processes can use to
guide their choices.
16
Failure related conceptual issues cont.
Process controlled crash the ability to
artificially force the crash of a
process. Useful in crashing incorrect or suspect
processes. However, a process tolerant algoriths
can only tolerate the crash of a bounded number
of processes. Failures provoked genuine gt
provoking failures degrades the actual fault
tolerance of the system.
17
Fault tolerance mechanisms

• The main fault-tolerance mechanisms algorithms
  rely on
• Failure detection
• Formalized by completness (prevents blocking) and
  accuracy (prevents algorithms from running
  forever without solving the problem)
• Group membership service (manages membership of
  groups of services)
• Provides consistent failure notification
• Resilient communication pattern (avoids any
  potential blocking pattern)
• Message stability (at least one process is
  correct)
• Concensus
• Mechanisms for lossy channels (tokens,
  acknowledgnents)

18
Conclusion

• Problem specification
• Five classes of total order broadcast algorithms
• Failure related concepts
• Fault tolerance mechanisms
• The paper also offers a survey of approx. 60
  algorithms

19
Discussion topics

• Adaptability of the algorithms (e.g. total order
  multicast in dynamic, partitionable groups)
• Synchrony and timeliness
• Performance in the different algorithms
• Fairness in the different algorithms (e.g.
  privilege based)
• Suitability of algorithms for open vs. closed
  groups (e.g. processes have to know of each other
  in priviledge based algorithms)
• Is this approach comprehensive and adequate?
• Not covered yet relevant issues?
• A reflection of this approach in relation to some
  earlier seminar seminar topics? Can the
  principles be adopted elsewhere?

20

• Thats it, folks!