Data Management in the Cloud - PowerPoint PPT Presentation

Loading...

PPT – Data Management in the Cloud PowerPoint presentation | free to download - id: 6f0749-ZTcxZ



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Data Management in the Cloud

Description:

Data Management in the Cloud – PowerPoint PPT presentation

Number of Views:280
Avg rating:3.0/5.0
Slides: 44
Provided by: Darr128
Learn more at: http://svforum.org
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Data Management in the Cloud


1
Data Management in the Cloud

2
Agenda
  • Cloud fundamentals
  • Market view
  • Cloud vendors
  • Getting data in and out
  • Cloud configurations
  • A Day in the Clouds
  • Security

Its Good Enough
3
What is Cloud Computing?
  • Essential characteristics
  • On-demand self-service
  • Broad network access
  • Resource pooling virtual
  • Rapid elasticity
  • Measured Service pay per use
  • Service models
  • Software-as-a-Service (SaaS)
  • Platform-as-a-Service (PaaS)
  • Infrastructure-as-a-Service (IaaS)
  • Deployment models
  • Private cloud
  • Public cloud
  • Hybrid cloud

Source Draft NIST Working Definition of Cloud
Computing, 8-21-09, version 15 http//csrc.nist.go
v/groups/SNS/cloud-computing/index.html
4
Primary Cloud Computing Services
Infrastructure-as-a-Service Rent-a-server Rent-a-disk
Software-as-a-Service Rent-a-seat Or free
Platform-as-a-Service Middleware stacks Web services for hire Developer tools
5
Public and Private Clouds
Line of Business users
Internet
Company LAN
Private Cloud
Virtualization SLAs
6

Cloud User Surveys Adoption Areas
Q Rate your likelihood to pursue the cloud model
for the following
Collaboration applications
Web applications/Web serving
Data Back-up or Archive services
Business apps (CRM, HR, ERP)
Personal productivity apps
Data/Content Distribution services
Storage capacity on demand
IT Management software
Server capacity on demand
Business Intelligence/Analytics
Application dev/test/deploy platform
IT/Information Security
Source IDC Enterprise Panel, 3Q09, n 263,
September 2009
7
Cloud User Surveys - Benefits
Q Rate the benefits commonly ascribed to the
'cloud'/on-demand model
(Scale 1 Not at all important 5 Very
Important)
Source IDC Enterprise Panel, 3Q09, n 263,
September 2009
8
Cloud User Surveys - Challenges
Q Rate the challenges/issues of the
'cloud'/on-demand model
Source IDC Enterprise Panel, 3Q09, n 263,
September 2009
9
Cloud Vendors
10
Magic Quadrant for Web Hosting and Hosted Cloud
System Infrastructure Services (On Demand), 2009
11
Public Cloud Vendors at a Glance
Company Hypervisor OS Service Contract SLA
Flexiscale (EMEA) Xen Windows, CentOS, Debian, Ubuntu TBD 100
Amazon Xen Windows, Red Hat, Fedora, OpenSolaris, OpenSUSE, Debian, Ubuntu, Gentoo None 99.95
ATT Synaptic VMware Windows, Red Hat Annual 99.7
GNi Hosting Xen, VMware, Microsoft Windows, Red Hat, CentOS, Debian, Gentoo, Ubuntu Monthly 100
IBM Xen, VMware Windows, Red Hat, CentOS, SUSE, AIX Annual Depends on location
RackSpace Xen Red Hat, Fedora, CentOS, Debian, Ubuntu, Gentoo None 100
Savvis VMware Windows, Red Hat, Solaris 10 and x86 Monthly 99.9 up to 99.99
GoGrid Xen Windows, CentOS, Red Hat None 100 with paybacks
High Availability is vastly different between
vendors
Source InfoWeek, From Amazon To IBM, What 12
Cloud Computing Vendors Deliver, Sep 5, 2009
12
Cloud Formations
13
Public Cloud Vendor Configurations
HP/Dell X64 4-16 CPUs, 64GB RAM static IP
address with VM, VLAN
launch instance
save bundle
Ephemeral
14
Server Consolidation using VMware
Virtual Machines
15
Private Clouds through Virtualization
users
Virtual Control Layer
users
Internal Private Cloud
My Data Center A
users
Public Cloud
Internal Private Cloud
My Data Center B
16
Clouds Use Any Commodity Hardware
My momma always said, Clouds are like a box of
chocolates. You never know what you're gonna
get.
Forrest Gump (1994)
17
Public and Private Clouds
  • You get any available server and storage
  • Mostly SANs
  • Selectable CPU speed and memory size
  • Often unknown
  • Memory bus speed
  • RAID configuration unknown
  • Disk rotation speeds and capacities
  • Performance will vary a lot!
  • But is it good enough?
  • In contrast, EDW appliances are carefully
    balanced
  • Ratio of IOPS to memory size/speed to CPU
    size/speed

See http//www.cloudsleuth.net
18
New Skills Needed for Clouds
  • Any hardware configuration planning
  • Strong performance analysis and problem isolation
  • Is it the commodity disk subsystem?
  • Is it the virtual machine tax?
  • Who else is sharing these resources?
  • Is it the BI Tools server or database server?

19
BI-DW Configurations
20
Generic DI/EDW/BI Data Flow
staging
EDW/ mart
System Management, Metadata, Security, Developer
Tools
On premises, inside the firewall
21
Data Mart in the Cloud Data Flow
staging
mart
System Management, Metadata, Security, Developer
Tools
Either public or private cloud
On premises, inside the firewall
22
ETL and Data Cleansing in the Cloud
staging
mart
System Management, Metadata, Security, Developer
Tools ?
Either public or private cloud
On premises, inside the firewall
23
It Can Happen!
staging
mart
System Management, Metadata, Security, Developer
Tools?
On premises, inside the firewall
24
Co-location Minimizes Latencies
1 public cloud vendor
staging
mart
System Management, Metadata, Security, Developer
Tools?
On premises, inside the firewall
25
Getting Data In and Out of a Cloud
26
Data Transfer To/From Cloud
Amazon Web Services Amazon Web Services
0.100 per GB data transfer in
0.150 per GB first 10 TB data transfer out
0.110 per GB next 40 TB data transfer out
0.090 per GB next 100 TB data transfer out
0.080 per GB data transfer out over 150 TB
RackSpace RackSpace
0.08/GB Bandwidth in
0.22/GB Bandwidth out
As of July 2010
27
What Does this Mean to BI?
Report Users MB/day work days GB/month Monthly
20 10 23 4.6 0.69
100 10 23 23 3.45
500 10 23 115 17.25
500 50 23 575 86.25
Batch GB/day work days GB/month Monthly
Extracts 10 30 300 45.00
Extracts 50 30 1500 225.00
Redo log backup 2 30 60 9.00
Full backup 500 4 2000 300.00
Assumptions 500GB data mart Transfer-out at
0.15 per GB/month
28
Data Transfer with Public Clouds
Corporate Data Center
ETL
De-duplicated Compressed Encrypted Secure
RDBMS
SANs
No fees for data transfer inside the same cloud
29
Informatica Cloud ETL and Replication
Source Informatica , Breakfast in the Cloud, May
24,2010 Slideshare
30
Data Integration Considerations
  • Initial loading
  • Just send the tapes (sneaker-net)
  • ETL
  • Are all files available?
  • RDBMS lookups
  • Minimizing data movement
  • Informatica push-down
  • MicroStrategy ROLAP push-down
  • SAS in-database
  • Minimize movement across domains

31
A Day in the Clouds
32
In the Cloud Monday Morning 10am
Data mart
Corporate Data Center
Inspirational source Steve Dine, TDWI, BI in the
Cloud, Nov 5,2009
33
In the Cloud Monday Night 10pm ETL
Data mart
Corporate Data Center
34
In the Cloud Tuesday Morning 3am Reports
Month end surge capacity
Data mart
Corporate Data Center
Inspirational source Steve Dine, TDWI, BI in the
Cloud, Nov 5,2009
35
In the Cloud Tuesday Morning 6am Backup
instance bundle
S3
Data mart
snapshot
rsync
Corporate Data Center
Inspirational source Steve Dine, TDWI, BI in the
Cloud, Nov 5,2009
36
Load Balancing in VMware and Amazon EC2
Live migration
VMware or Xen
VMware or Xen
resource pool
Servers
Inspirational source Steve Dine, TDWI, BI in the
Cloud, Nov 5,2009
37
High Availability in VMware and Amazon EC2
VMware or Xen
VMware or Xen
VMware or Xen
resource pool
Servers
Inspirational source Steve Dine, TDWI, BI in the
Cloud, Nov 5,2009
38
Security
39
Security
  • Physical security
  • Retinal scans, motion-sensors, etc.
  • Database security
  • Encryption, LDAP
  • No network clear text
  • Amazon network security
  • Default all ports are closed
  • You create login key pairs
  • Manages man-in-the-middle and denial-of-service
    attacks
  • Xen Instances cannot access hardware directly

"This building is like a secure bunker, and the
campus is like a military base," Terremark SVP
Norm Laudermilch Inside Terremark's Secure
Government Data Center http//www.informationweek.
com/story/showArticle.jhtml?articleID218700118
40
Amazon Virtual Private Cloud
IP addresses not exposed to Internet
subnets
On premises network
router
VPN gateway
Secure VPN
Source http//news.cnet.com/8301-19413_3-10318114
-240.html?tagmncolposts Source Mike Culver,
Amazon Web Services, Data Warehousing in the
Public Cloud, Smart Data Collective
41
In Summary

42
Good Enough Workloads for Clouds
Workload Public Private
Small-medium data marts X X
Sand box / data labs X
BI tools, ETL tools X X
Development X X
Workload isolation Partners, Systems Integrators X
Non-core HR, CRM, collaboration, eMail, occasional use applications X X
Major applications except highest availability, highest performance X
Short term projects X X
Proof-of-concept, prototypes X X
Quality assurance, software testing X X
43
Summary
  • Cloud adoption and maturity will happen fast
  • Most technology isnt new
  • ISVs gold-rush to clouds
  • Many challenges, many opportunities
  • Elastic scale up and down
  • New workflows, designs
  • Helping Teradata clients get into the cloud
  • Teradata Express for VMware and Amazon Web
    Services

Skeptic
Visionary
About PowerShow.com