Principles of Reliable Distributed Systems Recitation 8: Paxos (Paxos) Made Simple

1
Principles of Reliable Distributed Systems
Recitation 8 Paxos (Paxos) Made Simple

• Spring 2008
• Alex Shraer

2
Abstract

• The Paxos algorithm, when presented in plain
  English, is very simple.

3
The Model

• Asynchronous
• Benign failures (non-Byzantine)
• Processes may fail by stopping, and restart.
• No limitation on number of failures.
• Messages can be duplicated and lost, but not
  corrupted.

4
Agents

• Three classes
• proposers
• proposing the values
• acceptors
• accepting the values
• learners
• learn the outcome from others, after a value was
  chosen
• A single process may act as more than one agent.

5
Our Goal Understand Paxos

• Safety requirements
• A single value is chosen,
• Only a value that has been proposed may be
  chosen,
• A process never learns that a value has been
  chosen unless it actually has been.
• Liveness requirements (conditional)
• Some proposed value is eventually chosen, and
• If a value has been chosen, then a process can
  eventually learn the value.

6
Choosing a value

• Take I
• Have a single acceptor agent.
• All proposers send proposals to the acceptor.
• Acceptor chooses first proposed value it
  receives.
• Wheres the problem?
• Acceptor may fail (single point of failure).
  No further progress.

7
Choosing a value (contd)

• Take II
• Multiple acceptors
• A proposer sends a proposal to a set of acceptors
• An acceptor may accept the proposed value.
• The value is chosen when a large enough set of
  acceptors have accepted it.
• How large is large enough?
• Majority any 2 majorities have a least one
  acceptor in common
• The same value is chosen by all acceptors
• Works only if an acceptor can accept only one
  value
• Otherwise, an acceptor cannot be sure that every
  other acceptor will choose the same value

8
When to choose?

• If there are no failures or message loss, we want
  a value to be chosen (liveness) even if there is
  only one proposer
• P1 An acceptor must accept the first proposal
  that it receives.
• What happens if several values are proposed by
  different proposers at about the same time?

9
Two proposals
v1
v1
v1
v2
v2
v2

• What value was accepted by the middle process?
• No way to learn which value was chosen
• Unless some acceptor is allowed to change its
  mind!

10
Numbered Proposals

• An acceptor must be allowed to accept gt1
  proposal.
• Keeping track of proposals unique numbers
• In fact, pairs ltcounter, pidgt
• A value is chosen when
• A single proposal with that value has been
  accepted by a majority
• In this case, we also say that the proposal was
  chosen

11
Maintaining Agreement

• We can allow multiple proposals to be chosen, but
• Must guarantee that all chosen proposals have the
  same value (agreement), so
• P2 If a proposal with value v is chosen, then
  every higher-numbered proposal that is chosen has
  value v.

12
Maintaining Agreement (contd)

• To be chosen, a proposal must be accepted by at
  least one acceptor, so we can satisfy P2 by
  satisfying a stronger requirement
• P2a If a proposal with value v is chosen, then
  every higher-numbered proposal accepted by any
  acceptor has value v.

13
Worst Case Scenario
must accept according to P1
must accept according to P1
propose
1, v1
1, v1
v1 was chosen
1, v1
1, v1
Does not hear any message (yet)
Contradicts P2a!
2, v2
2,v2
14
Maintaining Agreement (contd)

• To guarantee P2a, we need to strengthen it
• P2b If a proposal with value v is chosen, then
  every higher-numbered proposal issued by any
  proposer has value v
• Note P2b ? P2a ? P2
• How can P2b be enforced?
• before a value is proposed, the process must
  check whether a value has already been chosen
• how can this be done?

15
Issuing a proposal

• P2c For any v and n, if a proposal with value v
  and number n is issued, then there is a set S
  consisting of a majority of acceptors such that
• either
• no acceptor in S accepts any proposal numbered lt
  n
• or
• v is the value of the highest-numbered proposal
  among all proposals numbered lt n accepted by the
  acceptors in S
• Lets show that if we preserve invariant P2c, then
  P2b is also satisfied

either before or after the issue event
either before or after the issue event
16
Proof that P2c ? P2b

• By induction on n
• Suppose v was chosen and its number is m? n
• this means there is a set C with a majority of
  acceptors that accepted v and m
• And suppose that a proposal is issued according
  to P2c with number n
• Need to show the proposed value is v
• Base (nm) trivial, since only v is proposed
  with m
• Induction assumption all proposals numbered
  m..n-1 have value v
• this means that v is the only possible value
  accepted with m..n-1
• v is the highest numbered value accepted by
  every process?C. v's
  number at these processes is at least m
• According to P2c, there is a majority set S such
  that either (a) or (b)
• S intersects C, therefore at least one process in
  S accepts (in past or future) the value v with
  number ? m .. n-1
• Since no other value has a higher timestamp ltn
  (by induction assumption)
• v must be the value with highest number lt n (that
  has been or will be) accepted by
• an acceptor in S
• the proposed value is v

17
Maintaining P2c (Prepare)

• A proposer needs to contact majority S of
  acceptors
• and then chose the higher numbered value
• But what if after they answer they accept a
  different value with a higher number?
• Need to prevent this
• Lets make them promise that they will not do that
• Choose proposal number n
• Request from all acceptors
• A promise to not accept any proposal numbered lt n
  
• The proposal with the highest number lt n that has
  been accepted (if any)
• Await replies from majority of acceptors

18
Issuing Proposals

• Issue a proposal (v,n) such that
• Either v is the value of the highest-numbered
  proposal,
• Or v is any value (e.g., your input) if no
  proposals were reported
• Is it enough to get the response that no proposal
  lt n was accepted from (only) a majority of
  acceptors?
• yes, because even if others have accepted such
  proposals, their values can never be chosen

19
Acceptors Algorithm

• Can ignore any request without compromising
  safety.
• Should be allowed to respond (for progress)
• Can always respond to prepare.
• Can respond to accept only if it not promised not
  to.
• P1a An acceptor can accept a proposal numbered n
  iff it has not responded to a prepare request
  having a number greater than n.
• Notice that P1a ? P1

20
Optimizations

• If an acceptor receives a prepare request
  numbered n, but already responded to a prepare
  request numbered gt n
• No need to reply! (The value will not be accepted
  anyway)
• A proposer can abandon a request if someone else
  is issue a higher-numbered one
• Instead of ignoring accept/prepare messages, an
  acceptor can reply with a negative response and
  send the higher number it has heard of

21
Acceptor State

• An acceptor must remember
• The value and number of the highest-numbered
  proposal that it has ever accepted
• AcceptVal and AcceptNum
• The number of the highest-numbered prepare
  request to which it has responded (promising not
  to accept anything smaller)
• BallotNum
• Need stable storage
• Why? Homework question

22
Putting it all togetherPhase 1

• Proposer selects a proposal number n and sends a
  prepare request with n to a majority of
  acceptors.
• Acceptor if a prepare request is received with
  number n gt that of any prepare request to which
  it has already responded, it responds to the
  request with a promise not to accept any more
  proposals numbered less than n with the
  highest-numbered proposal (if any) that it has
  accepted.

23
Phase 2

• Proposer if receives a response to prepare(n)
  from a majority of acceptors ? sends an
  accept(n,v) request to each of those acceptors
• v is the value of the highest-numbered proposal
  among the responses, or input value if the
  responses reported no proposals.
• Acceptor if receives an accept request for a
  proposal numbered n, it accepts the proposal
  unless it has already responded to a prepare
  request having a number greater than n.

24
Progress
Prepare(1)
Ack(1,?)
Prepare(4)
. . .
. . .
. . .
. . .
. . .
Prepare(3)
Ack(3,?)
25
Progress (contd)

• A a distinguished proposer is required to
  guarantee progress
• A leader
• The only one allowed to issue proposals in stable
  state

26
Learning a Chosen Value

• Multiple options (communication complexity vs
  fault-tolerance)
• Accepted value sent to all learners
• number of messages acceptors learners
• Acceptors send to a distinguished learner, which
  then sends to all other learners
• number of messages acceptors learners
• not very fault tolerant
• Possible compromise acceptors send to a subset
  of learners

27
Putting it all Together
1
1
1
1
1
2
2
2
(accept, ?1,1? ,v1)
(prepare,1)
. . .
. . .
. . .
(ack, ?1,1? ,r0,)
n
n
n
(accept, ?1,1? ,v1)

• Failure-free run
• Our W implementation
• always trusts process 1

decide v1
28
Leslie Lamport

• At the PODC 2001 conference, I got tired of
  everyone saying how difficult it was to
  understand the Paxos algorithm The current
  version is 13 pages long, and contains no formula
  more complicated than n1 gt n2