Hierarchical Synchronization and Consistency in GDS

1
Hierarchical Synchronization and Consistency in
GDS

• Sébastien Monnet
• IRISA, Rennes

2
JuxMem Consistency ProtocolCurrently Home Based

• Home node responsible of a piece of data
• Actions on the piece of data ltgt communication
  with the home node

Home
Home
Home node
Client
3
Replicated Home

• The home node is replicated to tolerate failures
• Thanks to active replications all replicas are
  up-to-date

4
Replication
Consistency

• Two layered architecture
• Replication based on classical fault tolerant
  distributed algorithms
• Implies a consensus between all nodes
• Need for replicates in several clusters (locality)

Junction layer
Fault tolerance
Group communication and group membership
Adapter
Atomic multicast
Consensus
Communications
Failure detector
5
Hierarchical
Client
6
Synchronization Point of View

• Naturally similar to data management
• 1 lock per piece of data
• Pieces of data are strongly linked to their locks

SM
Synchronisation manager
Client
7
Synchronization Point of View

• The synchronization manager is replicated the
  same way

8
Synchronization Point of View
Client
9
In Case of Failure

• Failure of a provider (group member)
• Held by the proactive group membership the
  faulty provider is replaced by a new one
• Failure of a client
• With a lock gt regenerate the token
• Without a lock gt do nothing
• Failure of a whole local group
• Very low probability
• As if it was a client (as it is for the global
  group)

10
False Detection

• Blocking unlocking with return code
• To be sure that an operation as performed a
  client has to do something like
• do
• lock(data)
• process(data)
• while(unlock(data) is not ok)
• // here were sure that the action has been taken
  into account

11
Actual JuxMems Synchronization (Sum up)

• Authorization based
• Exclusive (acquire)
• Non exclusive (acquireR)
• Centralized (active replication)
• Strongly coupled with data management
• Hierarchical and fault tolerant

12
Data Updates When?

• Eager (current version)
• When a lock is released update all replicas
• High fault tolerant level / Low performances

Client
13
Data Updates When?

• Lazy (possible implementation)
• Update a local data group when a lock is acquired

Client
14
Data Updates When?

• Intermediate (possible implementation)
• Allow a limited number of local update before
  propagating all the updates to the global level

Client
15
Data Updates When?

• A hierarchical consistency model?
• Local lock
• Global lock

16
Distributed Synchronization Algorithms

• Naïmi-Trehels
• Token based
• Mutual exclusion
• Extented by REGAL
• Hierarchical (Marin, Luciana, Pierre)
• Fault tolerant (Julien)
• Both?
• A fault tolerant, grid aware synchronization
  module used by JuxMem?

17
Open Question and Future Work

• Interface between JuxMem providers and
  synchronization module
• Providers have to be informed of synchronization
  operations to perform updates
• Future work (Julien Sébastien)
• Centralized data / distributed locks?
• Data may become distributed in JuxMem (epidemic
  protocols, migratory replication, etc.)
• Algorithms for token-based non-exclusive locks?
• May allow more flexibility for replication
  techniques (passive or quorum based)

18
Other Open Issues in JuxMem
19
Junction Layer
Consistency

• Decoupled design
• Need to refine the junction layer

Send
Receive
Junction layer
Fault tolerance
20
Replication Degree

• Actual features the client specifies
• The global data group cardinality (i.e number of
  clusters)
• The local data groups cardinality (i.e number of
  replicas in each cluster)
• Desirable features the client specifies
• The criticality degree of the piece of data
• The access needs (model, required perfs)
• A monitoring module
• Integrated to Marins failure detectors?
• Current MTBF, message losses, etc.
• May allow JuxMem to dynamically deduce the
  replication degree for each piece of data

21
Application Needs

• Access model
• Data grain?
• Access patterns
• Multiple readers?
• Locks shared across multiple clusters?
• Data criticality
• Are there different levels of criticality?
• What kind of advice the application can give
  concerning those 2 aspects?
• Duration of the application?
• Traces latency, crashes, message losses?