GemStone Federation Technology

1
GemStone Federation Technology
2
What is federation?

• Ability for one GS repository to communicate with
  another
• Ability for one GS repository to replicate
  objects from another
• replicate may be sychronized

3
What is federation intended to support ?

• Object archiving
• Query server
• Schema synchronization
• Selective fault tolerance

4
Concepts similar to GemBuilder

• Replicates, forwarders, stubs
• Design points for federation
• Site administrators exercise autonomous control
  over their objects
• Developers exercise explicit control over network
  communication

5
Autonomous Control of Objects

• Security
• A site administrator still creates user accounts,
  segments, etc.
• User segments to allow/disallow replication
• Inter-repository communication
• administrator controls who can get out
• administrator controls who can get in

6
Explicit Control of network communication

• There are explicit messages to ...
• login
• generate class definitions
• unstub stubs
• (these can be configured to happen automatically)
• There is an explicit message to ...
• refresh replicates that have been modified
• (this may happen automatically when other network
  communications occurs)

7
What is federation?

• Each repository has its own view of what other
  repositories it can communicate with
• This view is not bi-directional
• Federation must be installed in every repository
  that participates in the federation

8
Remote Object Space

• Is the main object that developers will use to
  build applications
• Is logical container for replicates and
  forwarders from the same remote repository
• Maintains mappings between a replicate/forwarder
  and its primary object
• one replicate or forwarder per primary object in
  a remote object space

9
Remote Object Space (cont.)

• Provides a way to partition replicated objects
• for performance
• for transactional consistency (discussed later)
• Can create multiple remote objects spaces for a
  remote repository

10
Federation and replicates
Database A
Federation permits replicate to remain in
synchronization with its master object
Object Z
ROS
Replicate of Z
Database B
11
GsfRemoteObjectSpace

• To create an instance of GsfRemoteObjectSpace,
  you supply ...
• convenience name
• login paramters
• the name of consistency policy
• an initial size

12
Consistency Policies

• Remote object space provides mechanisms
• communication
• mapping replicates to primaries
• Consistency policy provide the semantics
• timing of refresh
• return types of remote execution
• whether replicates are synchronized
• kind of stub used
• whether synchronized commits required
• Purpose is to provide a tranactionally consistent
  view of replicates within a remote object space

13
Available consistency policies

• Phase I delivers two consistency policies
• readOnlyReplicates
• copyWithoutSynchronization
• Custom policies possible

14
GsfReadOnlyReplicatesPolicy

• Replicates are synchronized with their primary
  objects
• modifications to primary objects are maintained
• across transactions and sessions
• Replicates will be refreshed automatically
• Modifications to replicates are not flushed back
  to primaries
• Usage Scenarios supported
• query server
• selective fault tolerance

15
GsfOperationCopyPolicy

• Replicates are not synchronized
• modifications to primary object are not
  maintained
• primary object may be garbage collected
• used to copy objects from one repository to
  another

16
Terminology

• Coordinator gem
• the session that you explicitly created when you
  logged in to your local repository
• Subordinate gem
• the session that the remote object space creates
  when it logs in
• A coordinator gem may cause the creation of
  multiple subordinate gems
• A subordinate gem only communicates with one
  coordinator gem

17
Federation Gems
Database A
Object Z
Database B
Subordinant Gem
Replicate of Z
Primary Gem
18
Replication

• Send a message to a remote object space to
  acquire a new replicate or refresh existing
  replicates
• To replicate an object, must have a local class
  definition
• can generaated automatically
• Terminology
• replicate
• can be synchronized with its primary object
• primary object
• knows if a synchronized replicate has been
  created for it
• Can ask any object isReplicate
• Can ask primary object exportLocations
• returns id of remote object space
• Can ask remote object space contains anObject

19
Mapping classes

• Can explicitly generate a class at the replicate
  site
• Can automatically generate a class at the
  replicate site
• Can map an existing class at the replicate site

20
Refreshing Replicates

• Primary objects may keep track of their
  modifications since the last time their
  replicates were refreshed
• primary site maintains a dirty set for each
  remote object space
• for some objects, placed in dirty set when
  modified
• for other objects, modification operation is
  logged
• bags, sets, nsc's
• When a remote object space refreshes, only sends
  over what has changed

21
Controlling Replication

• Done by creating an instance of
  GsfReplicationSpec and adding it to a remote
  object space
• Remote object spaces do not share replication
  specs
• Work done at both replicate and primary site for
  efficiency

22
Controlling inst var replication

• To specify inst var replication, replicate site
  must know inst var names from primary site
• create replication spec and add to remote object
  space
• specify inst var replication
• Inst var mapping may also be controlled

23
Stubbing

• To avoid transferring objects across the network
• Can configure remote object space to
  automatically unstub
• can explicitly 'unstub'
• Can send '_isGsfStub'

24
Security

• Still use user accounts and segments to control
  authorization
• To control who can spawn a subordinate gem from
  the replicate site
• GemStoneFederation canSpawnGemBlock
• aRemoteObjectSpace
• To allow a subordinate gem to do work at the
  primary site
• GemStoneFederation allowSpawnedGemBlock
• rosName remoteRepoName remoteUsername

25
Distributed garbage collection

• The primary site will keep primary objects alive
  as long as there are replicates of it
• internal federation objects maintain a strong
  reference
• A remote object space only has weak references to
  replicates
• These weak references are only finalized on a
  markForCollection at the replicate site
• The next time the remote object space
  communicates with the primary site, it tells it
  which replicates have been finalized
• then primary objects can get collected

26
Synchronized commits/aborts

• Is like 2 phase commit
• only thing missing is recovery algorithms
• full 2PC coming
• Application performs commit and abort as before
• e.g., System commitTransaction
• federation automatically communicates remotely if
  necessary

27
Differences between federation GemBuilder

• In the GBS
• maps objects between client and server
• replicates/forwarders are transient
• replicates always up to date
• must be logged in to use replicates/forwarders
• flushing/faulting automatic
• replicates visible to single user
• In federation
• maps objects between server and server
• replicates/forwarders are persistent
• replicates may not be up to date
• replicates/forwarders available without being
  logged in
• flushing/faulting not necessary automatic
• replicates can be shared by multiple users