INNOV15: Life at the Edge: Delivering Effective Data Services for the Extended Enterprise - PowerPoint PPT Presentation

Title: INNOV15: Life at the Edge: Delivering Effective Data Services for the Extended Enterprise


1
INNOV-15 Life at the Edge Delivering Effective
Data Services for the Extended Enterprise
Paul Antosh
Director, Product ManagementReal Time Data
Services pantosh_at_progress.com
2
Agenda
Delivering Effective Data Services for the
Extended Enterprise

• Identify the value of replication
• Key issues of Life at the Edge of the Enterprise
• Examine Solutions working today
• How it works Replication Architecture and more
• Key Features that make a difference
• Getting more information

3
Replication What is it good for?
Replication is easy to understand. That
doesnt mean it is well understood.
Synchronization is the goal. Replication is a
way of synchronizing.
4
Replication What is it good for?

• Scalability HA
• Empower mobile users
• Integrate the new

Embedded SQL DB Option
Client
ShanghaiRemote Plant
Kuala LumpurRemote Plant
ClevelandRemote Plant
5
Agenda
Delivering Effective Data Services for the
Extended Enterprise

• Identify the value of replication
• Key issues of Life at the Edge of the Enterprise
• Examine Solutions working today
• How it works Replication Architecture and more
• Key Features that make a difference
• Getting more information

6
Life at the Edge Centralized Database
Architecture

• Poor QOS

• Whats the root cause?

• Dependency to the centralized data and the network

Im not working!
Remote Site
Data Center
Im slow!
Application
Centralized Data
WAN
7
Life at the Edge More Bandwidth

• Fix the pipe
• Problems with this approach
• Costly
• Most small and mid-size organizations cannot
  afford to do this
• Large organizations COULD try this, but
• STILL no guarantees
• You cant fully control this variable
• Bandwidth only part of the problem. Still have
  central DB as point of failure.

8
Life at the Edge Bulk Data Transfer

• FTP approach
• Problems with this approach
• No intelligence
• Stacking, incremental changes
• Missing one event can cause data corruption
• Highly manual maintenance and troubleshooting
• Hard to recover from failures

9
Life at the Edge Messaging

• Use some form of messaging queue
• Problems with this approach
• Puts much greater load on bandwidth
• Every single data change gets queued up
• No easy way to know if queue becomes corrupted
• Significant changes to application often needed

10
Life at the Edge The Problem Is
...How do I provide the same quality-of-service
to the edge of my enterprise that HQ has?
11
Life at the Edge Distributed Database
Architecture

• Good, consistent QOS
• Distribution of data is the key

Im still working!
Remote Site
Data Center
Im still fast!
Application
Centralized Data
WAN
Data
12
Life at the Edge Issues to Consider

• How many sites/users do you plan on replicating
  too?
• Value increases with each site/user
• Do you have a requirement for disconnected
  operation of your application?
• Full use of the application is important
• Are you experiencing low-bandwidth or
  high-latency networks for replication?
• Connectivity not guaranteed
• Do you have a requirement to slice your data for
  distribution to different locations?
• Snowplow data not needed in Las Vegas

• Do you have access to the schema for your
  database and can you make application changes?
• Sometimes more than just database changes
  required
• What databases are in use?

13
Agenda
Delivering Effective Data Services for the
Extended Enterprise

• Identify the value of replication
• Key issues of Life at the Edge of the Enterprise
• Examine Solutions working today
• How it works Replication Architecture and more
• Key Features that make a difference
• Getting more information

14
Solutions Cole Systems
Order Entry for Lottery Systems

• Issues
• Tracking daily sales info for an occasionally
  connected sales force
• Required to distribute data on a need to know
  basis for each individual sales representative
• Needed to be able to replicate with a large
  number of sites over very low bandwidth
• Solution
• Embedded Hub Spoke replication solution is
  completely invisible to the end user of OrderPad
• Used patented work set technology to partition
  data for each individual sales representative
• Ability to update detailed sales statistics over
  low bandwidth / dial up lines

15
Solutions Synchronics
Retail POS System

• Issues
• Retailers were required to be connect to the
  headquarters for proper reporting
• Slow network connections affected performance at
  the individual retailers
• Individual retailers would occasionally
  disconnect from HQ for increases performance
  created audit nightmare
• Solution
• Embed two-way Hub Spoke replication built into
  CounterPointSQL
• Individual retailers get maximum performance
  while being connected to headquarters
• Audit capability and offline ticketing is enabled
  by database operations even while disconnected

16

• Q Does this work with OpenEdge RDBMS?
• A Adding support for OpenEdge 10.1B
• Full DataXtend product line support
• Start gathering information at Exchange
• Best Practices contact info
• pantosh_at_progress.com
• Office 781-280-4839

17
Agenda
Delivering Effective Data Services for the
Extended Enterprise

• Identify the value of replication
• Key issues of Life at the Edge of the Enterprise
• Examine Solutions working today
• How it works Replication Architecture and more
• Key Features that make a difference
• Getting more information

18
How it works Replication Architecture

• Bi-Directional Replication synchronizes two or
  more sites
• Two Primary Components
• Change Capture Mechanism
• Tracks changes that have occurred to a database.
• Autonomous to Engine (works regardless of engine
  state)
• Replication Engine
• Uses information created by Change Capture to
  replicate all changed user data to another
  database.
• Runs as a service.

19
How it works Change Capture
Your App
Your Database
Data Table N Rows
PK
Control Tables 1 Table per Database Table
Triggers
Control Tables
System Tables
System Tables 40 Tables Set-up Data
20
How it works Change Capture
Record Insert
Customer
PK
Name
Addr
Phone
1
John
3 Main St.
123-3456
(1) Application Inserts Record
2
Kim
25 West St.
456-2343
3
Bob
2 Bush Rd.
789-0987
PDCCustomer
(2) Trigger Fires
PDCID
Customers PK
Timestamp
5
1
2005-08-01 0812
8
2
2005-08-05 1215
(3) Control Record Inserted
13
3
2005-09-01 1729
21
How it works Change Capture
Record Update
Customer
PK
Name
Addr
Phone
1
John
3 Main St.
123-3456
(1) Application Updates Record
2
Kim
25 West St.
456-2343
3
Bob
2 Bush Rd.
789-0987
Chris
PDCCustomer
(2) Trigger Fires
PDCID
Customers PK
Timestamp
5
1
2005-08-01 0812
8
2
2005-08-05 1215
(3) Control Record Updated
2005-09-01 1729
13
3
2005-09-03 1456
22
How it works Replication Architecture

• Engine-Database communication via ODBC
• Engine-Engine Communication via TCP/IP
• Maximize network efficiency through compression
  and scheduling
• Increased security via encryption
• Site autonomy / Automatic discovery

Corporate Data Center
Mobile User/ Remote Office
DataXtend Replication Engine
DataXtend Replication Engine
ODBC
ODBC
23
How it works Bi-directional Replication
Site A
Site B
Customer
Customer
PK
Name
Addr
Phone
PK
Name
Addr
Phone
(1) Sites not in Sync
1
John
3 Main St.
123-3456
1
John
3 Main St.
123-3456
2
Kim
25 West St.
456-2343
2
Kim
25 West St.
456-5555
456-5555
456-5555
(2) Replication Starts
3
Bob
2 Bush Rd.
789-0987
7
Sue
2 Bush Rd.
789-0987
3
Bob
2 Bush Rd.
789-0987
7
Sue
4 Grove St.
789-0987
7
Sue
4 Grove St.
789-0987
3
Bob
2 Bush Rd.
789-0987
(3) Last Replication 2005-08-05 1300
(4) Each site determines delta
PDCCustomer
PDCCustomer
(5) Each site sends delta to other site
PDCID
Customers PK
Timestamp
PDCID
Customers PK
Timestamp
5
1
2005-08-01 0812
5
1
2005-08-01 0812
(6) Replication Ends
2005-08-06 1131
8
2
2005-08-05 1215
8
2
2005-08-06 1131
2005-08-06 1131
(7) Sites in Sync
13
3
2005-09-01 1729
21
7
2005-08-06 1130
13
3
2005-09-01 1729
21
7
2005-08-06 1130
13
3
2005-09-01 1729
21
7
2005-08-06 1130
24
How it works Replication Architecture

• DataXtend RE focus on replication
• Collision
• Conflict
• Horizon
• Node Management
• Change Capture
• Data Slicing
• Implicit Programming Model with the database
• Out-of-the-box Replication

Your App
Data Slicing
Collision detection
Prevent Conflict
Your Database
Node Management
Collisions
Conflict
GUID
GUID
Synchronization
CAM Patent
Peer-to-peer
GUID
GUID
Conflict
Hub Spoke
DataXtend Replication Engine
Change Capture
Conflict
Auto-Balance
Triggers
Control Tables
Bandwidth

ODBC
Encryption
Recovery
System Tables
Heterogeneous
Rollback
Connectivity
25
How it works Horizontal Data Partitioning

• Work Sets One of DataXtend REs most powerful
  features
• Typical Workset One-to-Many

26
How it works Horizontal Data Partitioning

• Typical Workset One-to-Many

27
How it works Horizontal Data Partitioning

• Nested One-to-Many
• Top down

28
How it works Horizontal Data Partitioning

• Nested One-to-Many
• Bottom-up

29
How it works Horizontal Data Partitioning

• Complex Work sets Many-to-Many

Customer drives Salesperson

• Define 1st workset
• Define second workset
• Define dependencies

30
How it works Horizontal Data Partitioning

• Complex Work sets Many-to-Many

31
How it works Vertical Data Partitioning

• Fragments
• Unit of replication is a group of fields
• Changes occur at multiple locations
• Helps avoid
• False Collisions
• Silent Data Corruption
• Optimizes the replication cycle
• Only changed fragments are replicated

State
Zip
City
Site A (winner)
Site B
Results
False Collision
32
How it works Vertical Data Partitioning

• Fragments
• Unit of replication is a group of fields
• Changes occur at multiple locations
• Helps avoid
• False Collisions
• Silent Data Corruption
• Optimizes the replication cycle
• Only changed fragments are replicated

State
Zip
City
Site A (winner)
Site B
Results
Silent Data Corruption
33
How it works Vertical Data Partitioning

• Fragments
• Unit of replication is a group of fields
• Changes occur at multiple locations
• Helps avoid
• False Collisions
• Silent Data Corruption
• Optimizes the replication cycle
• Only changed fragments are replicated

State
Zip
City
Site A (winner)
Site B
Results
Replication with PeerDirect Fragments
34
How it works Database-driven File Replication

• The File Replication feature enables you to
  maintain relationships between database records
  and general files.
• Key Features
• Database Record and File are replicated
  transaction-ally.
• File transfer resume support.
• Relative or Absolute file mapping support.

35
File Directory
Database
DataXtend Replication Engine
Site A
File 3
Row 3
Application Inserts Record
File Directory
Temporary Directory
Database
DataXtend Replication Engine
Site B
File 3
Row 3
File 3.tmp
36
Agenda
Delivering Effective Data Services for the
Extended Enterprise

• Identify the value of replication
• Key issues of Life at the Edge of the Enterprise
• Examine Solutions working today
• How it works Replication Architecture and more
• Key Features that make a difference
• Getting more information

37
DataXtend RE Replication Key Features

• Synchronization via Bi-directional replication
• Connection (session) based replication
• Based on Net Change, not Queue-based
• Only the final image is sent to avoid redundant
  transmissions
• Peer to Peer Update Anywhere Model
• There is no notion of Master/Slave
• Flexible, scalable replication topology
• Peer-to-Peer, Hub Spoke, Multi-tiered
• Collision Avoidance Methodology (Patented - CAM)
• Horizontal (Work Sets), Vertical (Fragments)
• Heterogeneous database support
• E.g., replicate from Oracle to MS SQL Server

38
DataXtend RE Replication Key Features (cont.)

• Configurable Conflict Resolution
• Create custom event handler (e.g., Hub wins over
  Spoke)
• Automatic Partner Site Selection (Patented - DBP)
• Auto-balances replication network
• Simplified recovery from disaster or maintenance
• Simply restore DB, and other sites will catch it
  up
• Embeddable via extensive .NET or C API libraries
• Built-in Rollback Capabilities
• Delivery of error-free data guaranteed
• Horizon Feature
• Determine when changes made to a database have
  been propagated to each site within a replication
  network

39
Agenda
Delivering Effective Data Services for the
Extended Enterprise

• Identify the value of replication
• Key issues of Life at the Edge of the Enterprise
• Examine Solutions working today
• How it works Replication Architecture and more
• Key Features that make a difference
• Getting more information

40
Getting More Information

• Learn more about Progress DataXtend RE
• progress.com/realtime/products/dataxtend_re/index
  .ssp
• Talk to a RepresentativeUS Toll free
  1-800-477-6473Tel 44(0)870 351 7212 Tel
  1-781-280-4000
• progress.com/realtime/get_insight

LEARN MORE
GET MORE INSIGHT
41
Summary

• DataXtend RE has the robust, full-featured
  replication abilities required for enabling Life
  at the Edge
• Take an inventory of
• Whats the real problem?
• What are the constraints?
• What are the goals?
• Replication can greatly improve Life at the Edge
  of the Enterprise
• Distributed Data Management
• Greatly improves remote and disconnected life

42
Questions?

• How does DataXtend RE compare to OpenEdge
  (Fathom) Replication?
• OpenEdge Replication backs up the RDBMS
• DataXtend RE synchronizes databases over a network

Paul AntoshDirector, Product ManagementReal
Time Data Services pantosh_at_progress.com