CPSC 668 Distributed Algorithms and Systems

1
CPSC 668Distributed Algorithms and Systems

• Fall 2006
• Prof. Jennifer Welch

2
Leader Election in SynchronousRings

• Here is a simple algorithm.
• Group rounds into phases, each phase containing n
  rounds
• In phase i, the processor with id i, if there is
  one, sends a message around the ring and is
  elected.

3
Example of Simple Synchronous Algorithm

• n 4, the smallest id is 7.
• In phases 0 through 6 (corresponding to rounds 1
  through 28), no message is ever sent.
• At beginning of phase 7 (round 29), processor
  with id 7 sends message which is forwarded around
  the ring.

Relies on synchrony and knowing n
4
Analysis of Simple Algorithm

• Correctness Easy to see.
• Message complexity O(n), which is optimal
• Time complexity O(nm), where m is the smallest
  id in the ring.
• not bounded by any function of n!

5
Another Synchronous Algorithm

• Works in a slightly weaker model than the
  previous synchronous algorithm
• processors might not all start at same round a
  processor either wakes up spontaneously or when
  it first gets a message
• uniform (does not rely on knowing n)

6
Another Synchronous Algorithm

• A processor that wakes up spontaneously is
  active sends its id in a fast message (one edge
  per round)
• A processor that wakes up when receiving a msg
  is relay never in the competition
• A fast message carrying id m becomes slow if it
  reaches an active processor starts traveling at
  one edge per 2m rounds
• A processor only forwards a msg whose id is
  smaller than any id is has previously sent
• If a proc. gets its own id back, elects self

7
Analysis of Synchronous Algorithm

• Correctness convince yourself that the active
  processor with smallest id is elected.
• Message complexity Winner's msg is the fastest.
  While it traverses the ring, other msgs are
  slower, so they are overtaken and stopped before
  too many messages are sent.

8
Message Complexity

• Divide msgs into three kinds
• fast msgs
• slow msgs sent while the leader's msg is fast
• slow msgs sent while the leader's msg is slow
• Next, count the number of each type of msg.

9
Number of Type 1 Messages

• Show that no processor forwards more than one
  fast msg
• Suppose pi forwards pj 's fast msg and pk 's fast
  msg. When pk 's fast msg arrives at pj
• either pj has already sent its fast msg, so pk 's
  msg becomes slow, or
• pj has not already sent its fast msg, so it never
  will
• Number of type 1 msgs is n.

pk
pj
pi
10
Number of Type 2 Messages

• (slow sent while leader's msg is fast)
• Leader's msg is fast for at most n rounds
• by then it would have returned to leader
• Slow msg i is forwarded n/2i times in n rounds
• Max. number of msgs is when ids are small as
  possible (0 to n-1 and leader is 0)
• Number of type 2 msgs is at most
• ?n/2i n

n-1
i1
11
Number of Type 3 Messages

• (slow msgs sent while leader's is slow)
• Maximum number of rounds during which leader's
  msg is slow is n2L (L is leader's id).
• No msgs are sent once leader's msg has returned
  to leader
• Slow msg i is forwarded n2L/2i times during n2L
  rounds.
• Worst case is when ids are L to L n-1
• Number of type 3 msgs is at most
• ?n2L/2i 2n

Ln-1
iL
12
Total Number of Messages

• We showed
• number of type 1 msgs is at most n
• number of type 2 msgs is at most n
• number of type 3 msgs is at most 2n
• Thus total number of msgs is at most 4n O(n).

13
Time Complexity of Synchronous Algorithm

• Running time is O(n 2x), where x is smallest id.
• Even worse than previous algorithm, which was O(n
  x)
• Both algorithms have two potentially undesirable
  properties
• rely on numeric values of ids to count
• number of rounds bears no relationship to n, but
  depends on minimum id
• Next result shows that to obtain linear msg
  complexity, an algorithm must rely on numeric
  values of the ids.

14
Comparison-Based LE Algorithms

• We will show that if nodes cannot rely on the
  numeric values of the ids, then any synchronous
  LE algorithm has message complexity ?(n log n).
• How do we formalize "not relying on numeric
  values of ids"?
• Need a definition of "comparison-based", similar
  to that for sorting algs.

15
Definition of Comparison-Based

• First note that in the synchronous model, the
  behavior of an LE algorithm is totally determined
  by the distribution of the ids. Denote execution
  on ring R by exec(R).
• An LE algorithm is comparison-based if, in any
  two "order-equivalent" rings R1 and R2,
  "matching" processors pi in R1 and pj in R2 have
  "similar" behaviors in exec(R1) and exec(R2).

16
Definition of Order-Equivalent

• Rings R1 (x1,x2,,xn) and R2 (y1,y2,,yn) are
  order-equivalent if
• xi lt xj if and only if yi lt yj
• Example

x3
y2
y1
x4
x2
y3
x1
x1
y4
x5
y5
17
Definition of Matching Processors

• Processors pi in R1 and pj in R2 are matching if
  they are the same distance from the processor
  with minimum id.
• Example p0 (with id 18) in R1 and p1 (with id 5)
  in R2, which are 2 hops from p3 (resp., p4)

18
Definition of Similar Behaviors

• Two processors have similar behaviors in two
  executions if, in every round
• one processor sends a message to the left (right)
  if and only if the other one does
• one processor is elected if and only if the other
  one is

19
Synchronous Lower Bound

• Theorem (3.18) For every n 8 that is a power
  of 2, there is a ring of size n on which any
  synchronous comparison-based algorithm sends ?(n
  log n) msgs.
• Proof For each n, construct a highly symmetric
  ring Sn on which every comparison-based algorithm
  takes lots of messages.
• Symmetry means many processors have
  order-equivalent neighborhoods and thus behave
  similarly (including sending msgs)

20
Proof Strategy

• Define highly symmetric ring Sn
• Show that the number of "active" rounds is at
  least n/8
• Show that in the k-th active round, at least
  n/(2(2k1)) msgs are sent.
• Do some arithmetic to show that
• ?n/(2(2k1)) ?(n log n)

n/8
k1
21
Active Rounds

• A round is active if at least one processor sends
  a msg
• A proc. in a synchronous algorithm can
  potentially learn something even in an inactive
  round
• cf. the simple synchronous alg
• But in a comparison-based alg., a proc. can't
  learn about the order pattern of its ring in an
  inactive round
• Note that the k-th active round might be much
  later than the k-th round

22
A Highly Symmetric Ring Sn

• Let pi 's id be rev(i) , the integer whose binary
  representation using log n bits is the reverse of
  i 's binary rep. Example
• For technical reasons, then multiply id by n1

0
10
p1
p0
p2
p3
15
5
23
Why Lots of Active Rounds

• Show number of active rounds, T, in exec(Sn) is
  at least n/8.
• Proof Suppose T lt n/8. Let pi be the elected
  leader.
• Number of T-neighborhoods order-equivalent to pi
  's is at least n/(2(2T1)) (to be shown)
• Since n 8 and T lt n/8, algebra shows
  n/(2(2T1)) gt 1.
• So there is pj ? pi whose T-neighborhood is
  order-equivalent to pi 's.
• Then pj is also elected (to be shown),
  contradiction.

24
Why Enough Msgs Sent in Each Active Round

• Show at least n/(2(2k 1)) msgs are sent in kth
  active round
• Proof Since the round is active, at least one
  proc., say pi, sends a msg.
• There are at least n/(2(2k1)) procs. whose
  k-neighborhood is order-equivalent to pi 's (to
  be shown)
• Each of those procs. sends a msg in kth active
  round, since pi does (to be shown).

25
Why So Many O-E Neighborhoods

• Show every k-nbrhood of Sn has at least
  n/(2(2k1)) order-equivalent k-nbrhoods.
• Proof Sketch Let N be a k-nbrhood (sequence of
  2k1 ids).
• Let j be smallest power of 2 larger than 2k1.
• Break Sn into n/j segments of length j, with one
  segment encompassing N.
• Claim Each segment is order-equivalent to N.

26
Why Similar Behavior in Order-Equivalent
Neighborhoods

• Intuition is that two nodes in order-equivalent
  neighborhoods in the same ring will behave
  similarly, at least until differentiating
  information from beyond those neighborhoods has
  reached them.
• Yet the definition of comparison-based algorithm
  only requires similar behavior for matching
  processors in different (albeit order-equivalent)
  rings.

27
Order-Equivalent Means Similar

• Show that if pi and pj have order-equivalent
  k-nbrhoods, then pi and pj have similar behaviors
  through k-th active round.
• Proof Sketch Construct another ring Sn' such
  that
• pi in Sn behaves same as pj in Sn'
• pj in Sn' behaves similarly to pj in Sn

28
Order-Equivalent Means Similar

• In more detail, construct Sn' such that
• pi 's k-nbrhood in Sn equals pi 's in Sn'
• so behavior is same through k-th active round
• Sn' and Sn are order-equivalent
• pj in Sn' is matching to pj in Sn
• so behavior is similar through k-th active round
• Can be done since ids in Sn are spaced
  (multiplied by n1)
• So pi and pj have similar behavior in Sn through
  k-th active round

29
Anonymous Rings Revisited

• Leader election is impossible in anonymous rings
• No way to break symmetry
• No deterministic algorithm which works in every
  execution
• Another way to break symmetry, which works in
  some (but not all) executions is to use
  randomization.

30
Randomized Algorithm

• In each computation step, the processor receives
  an additional input to its state transition
  function, a random number.

31
Revised LE Problem Definition

• Weakened problem definition compared to original
• At most one leader is elected in every state of
  every admissible execution
• same as previous definition
• At least one leader is elected "with high
  probability".
• weaker than previous definition
• But what does "with high probability" mean?

32
Randomized LE Algorithm

• Assume synchronous model
• Initially
• set id to 1 with probability 1 - 1/n and to 2
  with probability 1/n
• send id to left
• When msg M is received
• if M contains n ids then
• if id is unique maximum in M then elect self
• else not elected
• else append id to M and send to left

33
Analysis of Randomized LE Alg.

• Uses O(n2) msgs
• There is never more than one leader
• Sometimes there is no leader
• leader is only elected if there is exactly one
  processor that sets its id to 2
• How often is there no leader, i.e., what is the
  probability?
• Need some more definitions

34
Random Choices and Probabilities

• Since system is synchronous, an admissible
  execution of the algorithm is determined solely
  by the initial random choices.
• Call this collection random choices
• RC ltr0, r1, , rn-1gt
• where each ri is either 1 or 2.
• Let exec(RC) be the resulting execution.
• Definition For any predicate P on executions,
  PrP is the probability of
• RC exec(RC) satisfies P
• I.e., the proportion of random choices resulting
  in an execution that satisfies P.

35
Probability of Electing a Leader

• Let P be the predicate "there is a leader".
• PrP probability RC contains exactly one 2

36
Improving the Probability of Electing a Leader

• If procs. notice there is no leader, they can try
  again.
• Each iteration of the basic algorithm is a phase.
• Keep trying until success.
• Random choices that define an execution consist
  of an infinite sequence of 1's and 2's, one for
  each proc.
• It is possible that algorithm doesn't terminate.

37
Probability of Not Terminating

• Probability of terminating in a particular phase
  is (1 - 1/n)n-1
• Probability of not terminating in a particular
  phase is 1 - (1 - 1/n)n-1
• Probability of not terminating in k phases is (1
  - (1 - 1/n)n-1)k since phases are independent.
• Last expression goes to 0 as k increases.

38
Expected Number of Phases

• Definition The expected value of a random
  variable T is
• ET ? k PrT k
• Let T be number of phases until termination.
• PrT k
• Prfirst k-1 phases fail kth succeeds
• (1 - (1 - 1/n)n-1)k-1 (1 - 1/n)n-1
• (1 - p)k-1 p , where p (1 - 1/n)n-1
• This is a geometric random variable with expected
  value p-1 lt e.
• So expected number of phases is lt 3.

k