On Fault Tolerance in Wireless Ad Hoc Networks

1
On Fault Tolerance in Wireless Ad Hoc Networks

• Seth Gilbert
• Nancy Lynch Celebration, 2008

2
Nancy Lynch
2002-2008
1997
1994
Late 1980s??
3
FLP Impossibility of distributed consensus with
one faulty process
Consistency
Fault tolerance
DLS Consensus in the Presence of Partial
Synchrony
Simulation Relations, Invariant-based Arguments
Replication
Timing
LT An Introduction to Input / Output Automata
2008
Formal Methods
2004
2000
1996
1992
1988
1984

• Increasingly complex, increasingly dyamic
• Group communication / membership
• Publish / Subscribe
• Peer-to-peer systems
• Wireless ad hoc networks

1980
4
The Virtual Infrastructure Project
5
The Virtual Infrastructure Project

• Papers
• GeoQuorums Implementing Atomic Memory in Mobile
  Ad Hoc Networks, DGLSW, DISC03, DC05
• Virtual Mobile Nodes for Mobile Ad Hoc Networks,
  DGLSSW, DISC03
• Consensus and Collision Detectors in Wireless Ad
  Hoc Networks, CDGNN, PODC05, DC08
• Timed Virtual Stationary Automata for Mobile
  Networks, DGLLN, Allerton05, OPODIS05
• Autonomous Virtual Mobile Nodes, DGSSW,
  DIALM-POMC05
• A Middleware Framework for Robust Applications in
  Wireless Ad Hoc Networks, CDGN, Allerton05
• Reconciling the theory and practice of unreliable
  wireless broadcast, CDGLNN, ADSN05
• Self-Stabilizing Mobile Node Location Management
  and Message Routing, DLLN, SSS05
• Motion Coordination Using Virtual Nodes, LMN,
  CDC05
• The Virtual Node Layer A Programming Abstraction
  for Wireless Sensor Networks, BGLNNS, WWWSNA07

• A Virtual Node-Based Tracking Algorithm for
  Mobile Networks, NL, ICDCS07
• Self-stabilization and Virtual Node Layer
  Emulations, NL, SSS07
• Secret Swarm Unit Reactive k-Secret Sharing,
  DLY, IndoCrypt07
• Virtual Infrastructure for Collision-Prone
  Wireless Networks, CGL, PODC08
• Theses
• Virtual Infrastructure for Wireless Ad Hoc
  Networks, G, PhD 2007
• Air Traffic Control Using Virtual Stationary
  Automata, B, MEng 2007
• Simulation and Evaluation of the Reactive Virtual
  Node Layer, S, MEng 2008
• Virtual Stationary Timed Automata for Mobile
  Networks, N, PhD 2008
• In Progress
• Self-Stabilizing Robot Formations over Unreliable
  Networks, GLMN
• Using Virtual Infrastructure to Adapt Wireline
  Protocols to MANET, W
• Virtual Infrastructure Routing for Mobile Ad Hoc
  Networks, DN

6
Wireless Ad Hoc Networks
7
Wireless Ad Hoc Networks

• environmental monitoring
• intrusion detection
• border monitoring
• fire detection

8
Wireless Ad Hoc Networks

• messaging
• conferences / events
• HikingNet
• TrafficNet

9
Wireless Ad Hoc Networks

• emergency response military
• firefighting
• police response
• terrorism

10
Wireless Ad Hoc Networks
11
Wireless ad hoc networks are really
hard to use.

• Unreliable communication
• Unknown availability

Noise
Lost Messages
Collisions
Unknown topology
Fault prone
Dynamic
Unknown participants
12
Fixed Infrastructure

• Deploy
• Base stations
• Cell towers
• Servers
• Problems
• Too expensive
• Not feasible

13
Virtual Infrastructure
14
Network Layers
Application
Service Service Middleware Wireless
Ad Hoc Network
15
Network Layers
Application
Routing Tracking Virtual
Infrastructure Wireless Ad Hoc Network
16
Building Virtual Infrastructure
17
Building Virtual Infrastructure

• Leader / backup
• Leader sends receives messages for the virtual
  node

• Each participant is a replica.
• Replicas execute a consistency protocol

18
Todays Questions

• What is virtual infrastructure?
• What can you do with it?
• Dynamic distributed coordination.
• Air traffic control
• Does it really work?
• Two simulation studies routing and address
  allocation.

19
Dynamic Distributed Coordination

• Challenging problem
• Highly dynamic environment
• Unreliable network
• Safety-critical applications
• Ideal for Virtual Infrastructure solution
• Static overlay
• Simpler, verifiable algorithms
• Fate-sharing

20
?
21
(No Transcript)
22
Dynamic Distributed Coordination

• Note
• Number of (non-failed) robots unknown.
• Location of other robots unknown.
• Pattern may change over time.

23
Dynamic Distributed Coordination

• In each round
• All robots stop.
• All robots send location info.
• Coordinators exchange info.

• In each round
• Coordinators calculate.
• Coordinators send out targets.
• Robots move to target.

24
Dynamic Distributed Coordination

• Rule 1 If only 1 robot, keep it.

25
Dynamic Distributed Coordination

• Rule 2 If not on the curve and no neighbors on
  the curve distribute evenly all but one.

26
Dynamic Distributed Coordination

• Rule 3 If not on the curve distribute among
  less populated neighbors on the curve.

27
Dynamic Distributed Coordination

• Rule 4 If on the curve distribute among less
  dense neighbors on the curve.

28
Dynamic Distributed Coordination

• Rule 4 If on the curve distribute among less
  dense neighbors on the curve.

29
Dynamic Distributed Coordination

• Rule 5 Distribute robots evenly on the curve in
  each region.

30
Dynamic Distributed Coordination

• Step 1 Eventually, robots cease moving from
  regions off the curve to regions on the
  curve.
• Step 2 If neighbor g is the most dense neighbor
  of u after time t, then u is less dense than g
  after time t1.
• Step 3 Eventually, robots remain always in the
  same region.

31
Dynamic Distributed Coordination

• What happens when something goes wrong?

Too many lost messages Too much churn
INCONSISTENT REPLICAS
Option 1 Design for the very, very worst
case. Option 2 Design a system that can recover
from faults.
32
Emulating Virtual Infrastructure

• Leader Election
• Heartbeats, timeouts
• Resolve leader competitions
• Replica Consistency
• Leader sends checksums of the state.
• If out-of-synch, then re-join.

33
Building Virtual Infrastructure

• Assume that
• A is a self-stabilizing algorithm.
• A is designed for the virtual infrastructure
  abstraction.
• A is executed with the emulator.
• The system begins in an arbitrary (corrupt)
  state.
• Then if the system is eventually well-behaved
• From some point on, the state of A is as if it
  had really executed on a fixed infrastructure.

34
Dynamic Distributed Coordination

• Coordination algorithm is self-stabilizing.
• In each round, all state is recalculated.
• Underlying virtual infrastructure emulation is
  self-stabilizing.
• Implications
• Converges to changing curve.
• Recovers from network instability, lost messages,
  etc.

35
Dynamic Distributed Coordination
Tina Nolte Virtual Stationary Timed Automata for
Mobile Networks PhD 2008
36
Dynamic Distributed Coordination

• Free Flight
• No flight plan, no control towers!
• Each pilot chooses a route independently.
• More efficient
• Adapt to wind currents.
• Avoid turbulence / bad weather.

37
Dynamic Distributed Coordination

• Goal Free Flight
• Each pilot chooses a route independently.
• More efficient
• Adapt to wind currents.
• Avoid turbulence / bad weather.

In the USA, minimum separation 3 miles lateral
distance OR 1000 feet altitude
38
Dynamic Distributed Coordination
Matthew D. Brown Air Traffic Control Using
Virtual Stationary Automata MEng, 2008
39
Todays Questions

• What is virtual infrastructure?
• What can you do with it?
• Dynamic distributed coordination.
• Does it really work?
• Two simulation studies.

40
Simulating Virtual Infrastructure
41
GeoCast
Source
Destination
42
GeoCast
Source
Destination
43
Location Service
Target
hash(id, 1)
hash(id, 2)
44
Location Service
Target
hash(id, 1)
hash(id, 2)
Source
45
Routing

• Two step process
• Lookup destination location.
• Geocast message to destinations region.

46
Simulation Setup

• Number of devices
• 25 / 50 / 100
• Velocity
• 0-20 meters / second
• Mobility model
• Random waypoint
• Pause time 100-900s
• Simulation time
• 1000 seconds

47
Simulation Setup

• GeoCast
• 10 send/receive pairs
• 1 msg every 5 secs
• Routing
• 10 send/receive pairs
• 1 msg every 0.5 secs
• 15 second simulation

48
Mobility and Density
100
100 devices
50 devices
80
Percent of Time Non-Failed
25 devices
60
40
20
200
400
600
800
Pause Time
When density is sufficient, virtual nodes work.
49
Leadership Changes
There is continuous turn-over in the leader.
50
Message Overhead
Most overhead is heartbeats. (Overhead is
negligible.)
51
Geocast Latency Overhead
0.5
0.4
Latency (in seconds)
0.3
0.2
100 devices
0.1
simple Geocast
200
400
600
800
Pause Time
VN-GeoCast is 2-3 times slower than simple
GeoCast.
52
Routing
devices50, pausetime400
Each message requires 3 GeoCast messages.
53
Simulation Summary

• Virtual nodes are stable if
• sufficient density (e.g., 4/region), OR
• low-enough churn
• Message overhead negligible.
• GeoCast latency overhead factor of 2.
• Routing relatively slow.

54
Simulation Summary
Mike Spindel Simulation and Evaluation of
the Reactive
Virtual Node Layer MEng 2008
55
Simulating Virtual Infrastructure
56
Address Allocation

• Challenges
• Highly dynamic. ? No central authority.
• Unreliable network. ? Limited address pool.

57
Simple Scheme

• Each region is allocated a cache of addresses.
• Basic protocol
• Client send REQUEST
• Server reply OFFER
• Client send ACQUIRE
• Server reply ACK
• Renew protocol
• Client send RENEW
• Server reply RACK
• Message forwarding

Virtual Node
Client
REQUEST
OFFER
ACQUIRE
ACK
RENEW
RACK
RENEW
RACK
58
Simulation Setup

• Number of devices
• 160
• MAC Layer
• 802.11
• Models collisions
• Mobility model
• Random waypoint
• Simulation time
• 40000 seconds

250 m
59
Simulation Setup
Number of addresses 30 per region Lease
time 400 seconds Forwarding limit 2 hop -
REQUEST 2 hop - RACK Varying - RENEW
250 m
60
Simulation Setup
61
Message Overhead
Maximum observed
Less than 2-4.5kbps
62
Message Overhead
63
Message Overhead
64
Protocol Performance
65
Protocol Performance
66
Simulation Summary

• Message overhead still negligible.
• Even with collisions
• Backoff
• Bigger simulations
• Simple address allocation scheme
• Reasonably efficient
• Scales well

67
Simulation Summary
Jiang Wu Using Virtual Infrastructure to Adapt
Wirelines Protcols to MANET
68
Summary

• What is virtual infrastructure?
• Dynamic distributed coordination
• Robotic motion coordination
• Self-stabilization
• (Preliminary) simulation results.

69
Distributed Algorithms

• Focus on fault-tolerance
• Replication
• Consistency
• Agreement
• Design principles
• Abstraction / layered design
• IOA / TIOA formalism

70
Congratulations, Nancy, and thank you!!
Seth Gilbert
71
The End