Consistency And Replication - PowerPoint PPT Presentation

About This Presentation
Title:

Consistency And Replication

Description:

This sequence is allowed with a casually-consistent store, but not with ... 2. Before a release is allowed to be performed, all previous reads and writes by ... – PowerPoint PPT presentation

Number of Views:121
Avg rating:3.0/5.0
Slides: 91
Provided by: baobao3
Category:

less

Transcript and Presenter's Notes

Title: Consistency And Replication


1
Chapter 10
  • Consistency And Replication

2
Topics
  • Motivation
  • Data-centric consistency models
  • Client-centric consistency models
  • Distribution protocols
  • Consistency protocols

3
Motivation
  • Make copies of services on multiple sites,
    improve
  • Reliability(by redundancy)
  • If primary FS crashes, standby FS still works
  • Performance
  • Increase processing power
  • Reduce communication delays
  • Scalability
  • Prevent overloading a single server (size
    scalability)
  • Avoid communication latencies (geographic scale)
  • However, updates are more complex
  • When, who, where and how to propagate the updates?

4
Concurrency Control on Remote Object
  1. A remote object capable of handling concurrent
    invocations on its own.
  2. A remote object for which an object adapter is
    required to handle concurrent invocations

5
Object Replication
  1. A distributed system for replication-aware
    distributed objects.
  2. A distributed system responsible for replica
    management

6
Distributed Data Store
Clients point of view Its data store is capable
of storing an amount of data
7
Distributed Data Store
Data Stores point of view General organization
of a logical data store, physically distributed
and replicated across multiple tasks.
8
Operations on A Data Store
  • Readri(x)b client i or process Pi performs a
    read for data item x and it returns value b
  • Write wi(x)a client i or process Pi performs a
    write on data x setting it to the new value a
  • Operations not instantaneous
  • Time of issue (when request is sent by client)
  • Time of execution (when request is executed at a
    replica)
  • Time of completion (when reply is received by
    client)

9
Example
10
Consistency Models
  • Defines which interleaving of operations is valid
    (admissible)
  • Different levels of consistency
  • strong (strict, tight)
  • weak (loose)
  • Consistency model
  • Concerned with the consistency of a data store
  • Specifies characteristics of valid ordering of
    operations
  • A data store that implements a particular
    consistency model will provide a total ordering
    of operations that is valid according to this
    model

11
Consistency Models
  • Data-centric models
  • Described consistency experienced by all clients
  • Clients P1, P2, P3, see same kind of orderings
  • Client centric models
  • Described consistency only seen by clients who
    request it
  • Clients P1, P2, P3 may see different kinds of
    orderings

12
Data-Centric Consistency Models
  • Strong ordering
  • Strict consistency
  • Linear consistency
  • Sequential consistency
  • Causal consistency
  • FIFO consistency
  • Weak ordering
  • Weak consistency
  • Release consistency
  • Entry consistency

13
Strict Consistency
  • Definition A DDS (distributed data store) is
    strict consistent if any read on a data item of
    the DDS returns the value corresponding to the
    result of the most recent write on x, regardless
    of the location of the processes doing read or
    write
  • Analysis
  • 1. In a single processor system strict
    consistency is for nothing, thats exact the
    behavior of local shard memory with atomic
    reads/writes
  • 2. However, its hard to establish a global time
    to determine whats the most recent write
  • 3. Due to message transfer delays this model is
    not achievable

14
Example
  • Behavior of two processes, operating on the same
    data item.
  • (a) A strictly consistent store.
  • (b) A store that is not strictly consistent.

15
Strict Consistency Problems
Assumption y 0 is stored on node 2, P1 and P2
are processes on node 1 and 2, Due to message
delays, r(y) at t t2 may result in 0 or 1 and
at t t4 may result in 0, 1 or 2 Furthermore
If y migrates to node 1 between t2 and t3 then
r(y) issued at time t2 may even get value 2 (i.e.
.back to the future.).
16
Sequential Consistency (1)
  • Definition A DDS offers sequential consistency,
    if all processes see the same order of accesses
    to the DDS, whereby reads/writes of individual
    processes occur in program order, and
    reads/writes of different ones are performed in
    some sequential order.
  • Analysis
  • 1. Sequential consistency is weaker than strict
    consistency
  • 2. Each valid permutation of accesses is allowed
    iff all tasks see same permutation ? 2 runs of a
    distributed application may have different
    results
  • 3. No global timing ordering is required

17
Example
Each task sees all writes in the same order, even
though not strict consistent.
18
Non-Sequential Consistency
19
Linear Consistency
  • Definition A DDS is said to be linear consistent
    (linearizable) when each operation is
    time-stamped and the following holds The result
    of each execution is the same as if the (read and
    write) operations by all processes on the DDS
    were executed in some sequential order and the
    operations of each individual process appear in
    this sequence in the order specified by its
    program. In addition, if TSOP1(x) lt TSOP2(y),
    then operation OP1(x) should precede OP2(y) in
    this sequence

20
Assumption
  • Each operation is assumed to receive a time stamp
    using a globally available clock, but with only
    finite precision, e.g. some loosely coupled
    synchronized local clocks.
  • Linear consistency is stricter than sequential
    one, i.e. a linear consistent DDS is also
    sequentially consistent.
  • With linear consistency no longer each valid
    interleaving of reads and writes is allowed, the
    ordering has also obey the order implied by the
    time-stamps of these operations.

21
Causal Consistency (1)
  • Definition A DDS is assumed to provide causal
    consistency if, the following condition holds
    Writes that are potentially causally related
    must be seen by all tasks in the same order.
    Concurrent writes may be seen in a different
    order on different machines.
  • If event B is caused or influenced by an
    earlier event A, causality requires that everyone
    else also sees first A, and then B.

22
Causal Consistency (2)
  • Definition write2 is potentially dependent on
    write1, when there is a read between these 2
    writes which may have influenced write2
  • Corollary If write2 is potential dependent on
    write1 ? the only correct sequence is write1
    ?write2.

23
Causal Consistency Example
  • This sequence is allowed with a
    casually-consistent store, but not with
    sequentially or strictly consistent store.

24
Causal Consistency Example
  1. A violation of a casually-consistent store.
  2. A correct sequence of events in a
    casually-consistent store.

25
Implementation
  • Implementing causal consistency requires keeping
    track of which processes have seen which writes.
  • Construction and maintenance of a dependency
    graph, expressing which operations are causally
    related (using vector time stamps)

26
FIFO or PRAM Consistency
  • Definition DDS implements FIFO consistency, when
    all writes of one process are seen in the same
    order by all other processes, i.e. they are
    received by all other processes in the order they
    were issued. However, writes from different
    processes may be seen in a different order by
    different processes.
  • Corollary Writes on different processors are
    concurrent
  • Implementation Tag each write-operation of every
    process with (PID, sequence number)

27
Example
Both writes are seen on processes P3 and P4 in a
different order, they still obey
FIFO-consistency, but not causal consistency
because write 2 is dependent on write1.
28
Example (2)
Two concurrent processes with variable x,y 0
Process P1 Process P2 x1 y1 if ( y0
) print(A) if (x0) print(B)
  • Possible results
  • A
  • B
  • Nil
  • AB?

29
Synchronization Variable
  • Background not necessary to propagate
    intermediate writes.
  • Synchronization variable
  • Associated with one operation synchronize(S).
  • Synchronize all local copies of the data store.

30
Compilation Optimization
int a, b, c, d, e, x, y / variables /int
p, q / pointers /int f( int p, int
q) / function prototype / a x
x / a stored in register /b y
y / b as well /c aaa bb a
b / used later /d a a c / used
later /p a / p gets address of a /q
b / q gets address of b /e f(p,
q) / function call /
  • A program fragment in which some variables may be
    kept in registers.

31
Weak Consistency
  • Definition DDS implements weak consistency, if
    the following hold
  • Accesses to synchronization variables obey
    sequential consistency
  • No access to a synchronization variable is
    allowed to be performed until all previous writes
    have completed everywhere
  • No data access (read or write) is allowed to be
    performed until all previous accesses to
    synchronization variables have been performed

32
Interpretation
  • A synchronization variable S knows just one
    operation synchronize(S) responsible for all
    local replicas of the data store
  • Whenever a process calls synchronize(S) its local
    updates will be updated on all replicas of the
    DDS and all updates of the other processes will
    be updated to its local replica of the DDS
  • All tasks see all accesses to synchronization-vari
    ables in the same order

33
Interpretation (2)
  • No data access allowed until all previous
    accesses to synchronization-variables have been
    done
  • By doing a synch before reading shared data, a
    task can be sure of getting the up to date
    value
  • Unlike previous consistency models weak
    consistency forces the programmer to collect
    critical operations all together

34
Example
Via synchronization you can enforce that youll
get up-to-date values. Each process must
synchronize if its writes should be seen by
others. A process requesting a read without any
synchronization measures may get out-of-date
values.
35
Non-weak Consistency
36
Release Consistency
  • Problems with weak consistency When a
    synch-variable is accessed, the DDS doesnt know
    whether this is done because a process has
    finished writing the shared variables or whether
    it is about reading them.
  • It must take actions required in both cases,
    namely making sure that all locally initiated
    writes have been completed (i.e. propagated to
    all other machines), as well as gathering in all
    writes from other machines.
  • Provide two operations acquire and release

37
Details
  • Idea
  • Distinguish between memory accesses in front of a
    critical section (acquire) and those behind of a
    critical section (release).
  • Implementation
  • When a release is done, all the protected data
    that have been updated within the critical
    section will be propagated to all replicas.

38
Definition
  • Definition A DDS offers release consistency, if
    the following three conditions hold
  • 1. Before a read or write operation on shared
    data is performed, all previous acquires done by
    the process must have completed successfully.
  • 2. Before a release is allowed to be performed,
    all previous reads and writes by the process must
    have been completed
  • 3. Accesses to synchronization variables are FIFO
    consistent.

39
Example
Valid event sequence for release consistency,
even though P3 missed to use acquire and
release. Remark Acquire is more than a lock or
enter_critical_section, it waits until all
updates on protected data from other nodes are
propagated to its local replicas, before it
enters the critical section
40
Lazy Release Consistency
  • Problems with eager release consistency When a
    release is done, the process doing the release
    pushes out all the modified data to all processes
    that already have a copy and thus might
    potentially read them in the future.
  • There is no way to tell if all the target
    machines will ever use any of these updated
    values in the future ? above solution is a bit
    inefficient, too much overhead.

41
Details
  • With lazy release consistency nothing is done
    at a release.
  • However, at the next acquire the processor
    determines whether it already has all the data it
    needs. Only when it needs updated data, it needs
    to send messages to those places where the data
    have been changed in the past.
  • Time-stamps help to decide whether a data is
    out-dated.

42
Entry Consistency
  • Unlike release consistency, entry consistency
    requires each ordinary shared variable to be
    protected by a synchronization variable.
  • When an acquire is done on a synchronization
    variable, only those ordinary shared variables
    guarded by that synchronization variable are made
    consistent.
  • A list of shared variables may be assigned to a
    synchronization variable (to reduce overhead).

43
How to Synchronize?
  • Every synch-variable has a current owner
  • An owner may enter and leave critical sections
    protected by this synchronization variable as
    often as needed without sending any coordination
    message to the others.
  • A process wanting to get a synchronization-variabl
    e has to send a message to the current owner.
  • The current owner hands over the synch-variable
    all together with all updated values of its
    previous writes.
  • Multiple reads in the non-exclusive reads are
    possible.

44
Example
A valid event sequence for entry consistency
45
Summary of Consistency Models
Consistency Description
Strict Absolute time ordering of all shared accesses matters.
Linearizability All processes must see all shared accesses in the same order. Accesses are furthermore ordered according to a (nonunique) global timestamp
Sequential All processes see all shared accesses in the same order. Accesses are not ordered in time
Causal All processes see causally-related shared accesses in the same order.
FIFO All processes see writes from each other in the order they were used. Writes from different processes may not always be seen in that order
(a)
Consistency Description
Weak Shared data can be counted on to be consistent only after a synchronization is done
Release Shared data are made consistent when a critical region is exited
Entry Shared data pertaining to a critical region are made consistent when a critical region is entered.
(b)
  1. Consistency models not using synchronization
    operations.
  2. Models with synchronization operations.

46
Up to Now
  • System wide consistent view on DDS
  • Independent of number of involved processes
  • Mutual exclusive atomic operations on DDS
  • Processes access only local copies
  • Propagation of updates have to be made, whenever
    it is necessary to fulfill requirements of the
    consistency model
  • Are there still weaker consistency models?

47
Client-Centric Consistency
  • Provide guarantees about ordering of operations
    only for a single client, i.e.
  • Effects of an operations depend on the client
    performing it
  • Effects also depend on the history of clients
    operations
  • Applied only when requested by the client
  • No guarantees concerning concurrent accesses by
    different clients
  • Assumption
  • Clients can access different replicas, e.g.
    mobile users

48
Mobile Users
  • The principle of a mobile user accessing
    different replicas of a distributed database.

49
Eventual Consistency
  • If updates do not occur for a long period of
    time, all replicas will gradually become
    consistent
  • Requirements
  • Few read/write conflicts
  • No write/write conflicts
  • Clients can accept temporary inconsistency
  • Examples
  • DNS
  • No write/write conflicts
  • Updates slowly (1 2 days) propagating to all
    caches.
  • WWW
  • Few write/write conflicts
  • Mirrors eventually updated
  • Cached copies (browser or Proxy) eventually
    replaced.

50
Client Centric Consistency Models
  • Monotonic Reads
  • Monotonic Writes
  • Read Your Writes
  • Writes Follow Reads

51
Monotonic Reading
  • Definition A DDS provides monotonic-read
    consistency if the following holds
  • If process P reads the value of data item x, any
    successive read operation on x by that process
    will always return the same value or a more
    recent one (independently of the replica at
    location L where this new read will be done).

52
Example Systems
  • Distributed e-mail database with distributed and
    replicated user-mailboxes.
  • Emails can be inserted at any location.
  • However, updates are propagated in a lazy (i.e.
    on demand) fashion.

53
Example
  • The read operations performed by a single process
    P at two different local copies of the same data
    store.
  • A monotonic-read consistent data store
  • A data store that does not provide monotonic
    reads.

54
Monotonic Writing
  • Definition DDS provides monotonic-write
    consistency if the following holds
  • A write operation by process P on data item x is
    completed before any successive write operation
    on x by the same process P can take place.
  • Remark Monotonic-writing FIFO consistency
  • Only applies to writes from one client process P
  • Different clients -not requiring monotonic
    writing may see the writes of process P in any
    order

55
Example
  • The write operations performed by a single
    process P at two different local copies of the
    same data store
  • A monotonic-write consistent data store.
  • A data store that does not provide
    monotonic-write consistency.

56
Reading Your Writes
  • Definition DDS provides read your write
    consistency if the following holds
  • The effect of a write operation by a process P on
    a data item x at a location L will always be seen
    by a successive read operation by the same
    process.
  • Example of a missing read-your-write consistency
  • Updating a website with an editor, if you want to
    view your updated website, you have to refresh
    it, otherwise the browser uses the old cached
    website content.
  • Updating passwords

57
Example
  1. A data store that provides read-your-writes
    consistency.
  2. A data store that does not.

58
Writes Following Reads
  • Definition DDS provides writes-follow-reads
    consistency if the following holds
  • A write operation by a process P on a data item x
    following a previous read by the same process, is
    guaranteed to take place on the same or even a
    more recent value of x, than the one having been
    read before.

59
Example
  1. A writes-follow-reads consistent data store
  2. A data store that does not provide
    writes-follow-reads consistency

60
Implementing Client Centric Consistency
  • Naive Implementation (ignoring performance)
  • Each write gets a globally unique identifier
  • Identifier is assigned by the server that accepts
    this write operation for the first time
  • For each client two sets of write identifiers are
    maintained
  • Read-set(client C) RS(C)
  • write-IDs relevant for the reads of this client
    C
  • Write-set(client C) WS(C)
  • write-IDs having been performed by client C

61
Implementing Monotonic Reads
When a client C performs a read at server S, that
server is handed the clients read set RS(C) to
control whether all identified writes have taken
place locally at server S. If not, server has to
be updated before reading!
62
Implementing Monotonic Write
  • If client initiates a write on a server S, this
    server S gets the clients write-set in order to
    update server S. A write on this server is done
    according to the times stamped WID.
  • Having done the new write, clients write-set is
    updated with this new write. The response time of
    a client might thus increase with an ever
    increasing write-set.
  • However, what to do if all the reader write-sets
    of a client get larger and larger?

63
Improving Efficiency with RS and WS
  • Major drawback potential sizes of read- and
    write sets ?
  • Group all write- and read-operations of a client
    in a so called session (mostly assigned with an
    application)
  • Every time a client closes its current session,
    all updates are propagated and these sets are
    deleted afterwards

64
Summary on Consistency Models
  • Choosing the right consistency model requires an
    analysis of the following trade-offs
  • Consistency and redundancy
  • All replicas must be consistent
  • All replicas must contain full state
  • Reduced consistency ? reduced reliability
  • Consistency and performance
  • Consistency requires extra work
  • Consistency requires extra communication
  • May result in loss of overall performance

65
Distribution Protocols
  • Replica Placement
  • Permanent Replicas
  • Server-Initiated Replicas
  • Client-Initiated Replicas
  • Update Propagation
  • State versus Operations
  • Pull versus Push Protocols
  • Unicasting versus Multicasting
  • Epidemic Protocols
  • Update Propagation Models
  • Removing data

66
Replica Placement
  • The logical organization of different kinds of
    copies of a data store into three concentric
    rings.

67
Replica Placement
  • Permanent replicas
  • Initial set of replicas. Created and maintained
    by DDS-owner(s)
  • Writes are allowed
  • E.g., web mirrors
  • Server-initiated replicas
  • Enhance performance
  • Not maintained by owner of DDS
  • Placed close to groups of clients
  • Manually
  • Dynamically
  • Client-initiated replicas
  • Client caches
  • Temporary
  • Owner not aware of replica
  • Placed closest to a client
  • Maintained by host (often the client)

68
Update Propagation
69
What to Be Propagated?
  • Propagate only a notification of an update
    (invalidation)
  • Typical for invalidation protocols
  • May include information which part of the DDS has
    been updated
  • Work best, when ratio of reads/write is low
  • Propagate updated data from one replica to
    another
  • Work best, if ratio of reads/writes is high
  • You may also aggregate some update before sending
    them across the network
  • Propagate the update operation to other replicas
    (active replication)
  • This approach called active replication works if
    the size of parameters associated with each
    operation is small compared to the updated data

70
Pull versus Push Protocols
  • Push protocol , i.e. updates are propagated to
    other replicas without those replicas having
    asked for them
  • Used between permanent and server initiated
    replicas, i.e. to achieve a relatively high
    degree of consistence
  • Pull protocol , i.e. a server (or a client) asks
    another server to provide the updates
  • Used by client caches, e.g. when a client
    requests a website, not having updated for a
    longer period of time, it may check the original
    web site, whether updates have been made
  • Efficient when read-to-write ratio is relatively
    low.

71
Pull versus Push Protocols
Issue Push-based Pull-based
State of server List of client replicas and caches None
Messages sent Update (and possibly fetch update later) Poll and update
Response time at client Immediate (or fetch-update time) Fetch-update time
  • A comparison between push-based and pull-based
    protocols in the case of multiple client, single
    server systems.

72
Unicasting
Potential overhead with unicasting in a LAN. Good
for pull-based approach.
73
Multicasting
With multicasting an update message can be
propagated more efficiently across a LAN. Good
for push-based approach.
74
Epidemic Protocols
  • Implementing eventual consistency you may rely on
    epidemic protocols.
  • No guarantees for absolute consistency are given,
    but after some time epidemic protocols will send
    updates to all replicas.
  • Notions
  • An infective is a server with a replica that is
    willingly to spread to other servers, too
  • A susceptible, is a server that has not yet been
    infected, i.e. updated
  • A removed server is a server, that does not want
    to propagate any information

75
Anti-Entropy Protocol
  • Server P picks another server Q at random, and
    subsequently exchanges updates with Q, there are
    3 approaches how to exchange updates
  • P only pushed its own updates to Q
  • P only pulls in new updates from Q
  • P and Q exchange to each other their updates,
    i.e. a push-pull approach

76
Gossip Protocols
  • Rumor spreading or gossiping works as follows
  • If server P has been updated for data item x, it
    contacts another arbitrary server Q and tries to
    push its new update of x to Q.
  • However, if Q got this update already by some
    other server, P is so much disappointed, that it
    will stop gossiping with a prob. 1/k

77
Gossip Protocols (2)
  • Although gossiping really works quite well on
    average, you cannot guarantee that every server
    will be updated.
  • In a DDS with a large number of replicas, the
    fraction s of servers remaining ignorant towards
    an update, i.e. are still susceptible is
  • s e-(k1)(1-s)

78
Analysis of Epidemic Protocols
  • Advantages
  • Scalability, due to limited of update messages
  • Disadvantage
  • Spreading the deleting of a data is quite
    cumbersome, due to an unwanted side effect
  • Suppose, you have deleted on server S data item
    x, but you may receive again an old copy of data
    item x from some other server due to still
    ongoing gossiping

79
Consistency Protocols
  • Primary-Based Protocols
  • Remote-Write Protocols
  • Local-Write Protocols
  • Replicated-Write protocols
  • Active Replication
  • Quorum-Based Protocols

80
Primary-Based Protocols
  • Each data item of a DDS has an associated
    primary, responsible for coordinating write
    operations on x
  • Primary server
  • Fixed,i.e. a specific remote server, i.e. remote
    writing
  • Dynamic, primary is migrated to the place, of the
    next write

81
Remote-Write Protocols (1)
  • Primary-based remote-write protocol with a fixed
    server to which all read and write operations are
    forwarded.

82
Remote-Write Protocols (2)
  • The principle of primary-backup protocol.

83
Local-Write Protocols (1)
  • Primary-based local-write protocol in which a
    single copy is migrated between processes.

84
Local-Write Protocols (2)
  • Primary-backup protocol in which the primary
    migrates to the process wanting to perform an
    update.

85
Replicated-Write Protocols
  • Writes can take place at multiple replicas,
    instead of on only a specific primary server.
  • Active replication
  • Operation is forwarded to all replicas
  • Problem
  • make sure all operations need to be carried out
    in the same order everywhere.
  • Scalability
  • Replicated invocation
  • Majority voting
  • Before reading or writing ask a subset of all
    replicas

86
Replicated Invocation for Active Replication
87
Solutions
  1. Forwarding an invocation request from a
    replicated object.
  2. Returning a reply to a replicated object.

88
Quorum-Based Protocols
  • Preliminaries
  • If a client wants to read or write, it first must
    request and acquire permission of multiple
    servers.
  • Example
  • A DFS with file F being replicated on N servers.
    If an update has to be made, demand, that the
    client first contacts half of the servers plus 1,
    and get them to agree to do his update. Once,
    they have agreed, file F gets a new version
    number F(x.y)
  • To read file F, a client also must contact at
    least half of the servers and ask them, to hand
    out the current version number of F.

89
Giffords Quorum-Based Protocols
  • To read a file F a client must use a read-quorum,
    an arbitrary assemble of NR servers.
  • To write a file F, at least NW servers( the write
    quorum) is required. The following must hold
  • A) NR NW gt N
  • B) NW gt N/2
  • A) Is used to prevent read-write conflicts
  • B) Is used to prevent write-write conflicts

90
Examples
  • Three examples of the voting algorithm
  • A correct choice of read and write set
  • A choice that may lead to write-write conflicts
  • A correct choice, known as ROWA (read one, write
    all)
Write a Comment
User Comments (0)
About PowerShow.com