Market-Oriented Cloud Computing - PowerPoint PPT Presentation

About This Presentation
Title:

Market-Oriented Cloud Computing

Description:

MarketOriented Cloud Computing – PowerPoint PPT presentation

Number of Views:2577
Avg rating:3.0/5.0
Slides: 54
Provided by: rajkuma
Category:

less

Transcript and Presenter's Notes

Title: Market-Oriented Cloud Computing


1
Market-Oriented Cloud Computing the Cloudbus
Toolkit
2
Market-Oriented Cloud Computing A Vision, Hype,
and Reality of Delivering Computing as the 5th
Utility
  • Dr. Rajkumar Buyya

Cloud Computing and Distributed Systems (CLOUDS)
LabDept. of Computer Science and Software
EngineeringThe University of Melbourne,
Australiawww.cloudbus.orgwww.buyya.comwww.manjr
asoft.com
Major Sponsors/Supporters
3
The Next Revolution in ITThe Big Switch in IT
  • Classical Computing
  • Buy Own
  • Hardware, System Software, Applications often to
    meet peak needs.
  • Install, Configure, Test, Verify, Evaluate
  • Manage
  • ..
  • Finally, use it
  • ....(High Cost)
  • Cloud Computing
  • Subscribe
  • Use
  • - pay for what you use, based on QoS

4
Outline
  • Computer Utilities
  • Vision and Promising IT Paradigms/Platforms
  • Cloud Computing and Related Paradigms
  • Trends, Definition, Cloud Benefits and Challenges
  • Market-Oriented Cloud Architecture
  • SLA-oriented Resource Allocation
  • Global Cloud Exchange
  • Emerging Cloud Platforms
  • Cloudbus Melbourne Cloud Computing Project
  • Summary and Thoughts for Future

5
Subscription-oriented metered, Essential
Utilities and Networks
6
Power Grid Inspiration for Computing? Deliver IT
services as computing utilities to users
7
Computer Utilities Vision Implications of the
Internet
  • 1969 Leonard Kleinrock, ARPANET project
  • As of now, computer networks are still in their
    infancy, but as they grow up and become
    sophisticated, we will probably see the spread of
    computer utilities, which, like present
    electric and telephone utilities, will service
    individual homes and offices across the country
  • Computers Redefined
  • 1984 John Gage, Sun Microsystems
  • The network is the computer
  • 2008 David Patterson, U. C. Berkeley
  • The data center is the computer. There are
    dramatic differences between of developing
    software for millions to use as a service versus
    distributing software for millions to run their
    PCs
  • 2008 The Cloud is the computer Buyya!

8
Computing Paradigms and Attributes Realizing the
Computer Utilities Vision
?
  • Web
  • Data Centres
  • Utility Computing
  • Service Computing
  • Grid Computing
  • P2P Computing
  • Market-Oriented Computing
  • Cloud Computing

  • -Ubiquitous -Reliable
  • Scalable
  • Autonomic
  • Dynamic discovery
  • Composable
  • -QoS
  • -SLA
  • -
  • Trillion business

Paradigms
Attributes/Capabilities
9
Outline
  • Computer Utilities
  • Vision and Promising IT Paradigms/Platforms
  • Cloud Computing and Related Paradigms
  • Trends, Definition, Cloud Benefits and Challenges
  • Market-Oriented Cloud Architecture
  • SLA-oriented Resource Allocation
  • Global Cloud Exchange
  • Emerging Cloud Platforms
  • Cloudbus Melbourne Cloud Computing Project
  • Summary and Thoughts for Future

10
Too popular too many are In Search of Cloud
Computing
Legend Cluster computing, Grid computing,
Cloud computing
11
2009 Gartner IT Hype Cycle of Emerging
Technologies
2008
2007
12
Top 10 for 2010
13
Defining Clouds There are many views for what is
cloud computing?
  • Over 20 definitions
  • http//cloudcomputing.sys-con.com/read/612375_p.ht
    m
  • Buyyas definition?
  • "A Cloud is a type of parallel and distributed
    system consisting of a collection of
    inter-connected and virtualised computers that
    are dynamically provisioned and presented as one
    or more unified computing resources based on
    service-level agreements established through
    negotiation between the service provider and
    consumers.
  • Keywords Virtualisation (VMs), Dynamic
    Provisioning (negotiation and SLAs), and Web 2.0
    access interface

14
Cloud Services
  • Infrastructure as a Service (IaaS)
  • CPU, Storage Amazon.com, Nirvanix, GoGrid.
  • Platform as a Service (PaaS)
  • Google App Engine, Microsoft Azure, Manjrasoft
    Aneka..
  • Software as a Service (SaaS)
  • SalesForce.Com

Software as a Service (SaaS)
Platform as a Service (PaaS)
Infrastructure as a Service (IaaS)
15
Clouds based on Ownership and Exposure
16
(Promised) Benefits of (Public) Clouds
  • No upfront infrastructure investment
  • No procuring hardware, setup, hosting, power,
    etc..
  • On demand access
  • Lease what you need and when you need..
  • Efficient Resource Allocation
  • Globally shared infrastructure, can always be
    kept busy by serving users from different time
    zones/regions...
  • Nice Pricing
  • Based on Usage, QoS, Supply and Demand, Loyalty,
  • Application Acceleration
  • Parallelism for large-scale data analysis,
    what-if scenarios studies
  • Highly Availability, Scalable, and Energy
    Efficient
  • Supports Creation of 3rd Party Services
    Seamless offering
  • Builds on infrastructure and follows similar
    Business model as Cloud

17
Cloud opportunity in short term
18
When will Cloud spending become 50 of IT
spending or reach to a several trillion
business/year?
600?
1000?
30
50
120?
15
2016
2020?
2020?
Buyyas Guestimate!
19
Cloud Computing Challenges Dealing with too many
issues
Virtualization
Energy Efficiency
20
Outline
  • Computer Utilities
  • Vision and Promising IT Paradigms/Platforms
  • Cloud Computing and Related Paradigms
  • Trends, Definition, Cloud Benefits and Challenges
  • Market-Oriented Cloud Architecture
  • SLA-oriented Resource Allocation
  • Global Cloud Exchange
  • Emerging Cloud Platforms
  • Cloudbus Melbourne Cloud Computing Project
  • Summary and Thoughts for Future

21
Market-oriented Cloud Architecture QoS
negotiation and SLA-based Resource Allocation
22
A (Layered) Cloud Architecture
Cloud applications Social computing, Enterprise,
ISV, Scientific, CDNs, ...
User level
Cloud programming environments and tools Web 2.0
Interfaces, Mashups, Concurrent and Distributed
Programming, Workflows, Libraries, Scripting
User-LevelMiddleware
Apps Hosting Platforms
QoS Negotiation, Admission Control, Pricing, SLA
Management, Monitoring, Execution Management,
Metering, Accounting, Billing
Autonomic / Cloud Economy
Adaptive Management
CoreMiddleware
Virtual Machine (VM), VM Management and
Deployment
Cloud resources
System level
23
Outline
  • 21st Century Vision of Computing
  • Promising Computing Paradigms
  • Cloud Computing and Related Paradigms
  • Trends, Definition, Characteristics, Architecture
  • Market-Oriented Cloud Architecture
  • SLA-oriented Resource Allocation
  • Global Cloud Exchange
  • Emerging Cloud Platforms
  • Cloudbus Melbourne Cloud Computing Project
  • Summary and Thoughts for Future

24
Some Commercial-Oriented Cloud platforms/technolog
ies
System Property AmazonEC2 S3 GoogleApp Engine MicrosoftAzure ManjrasoftAneka
Focus IaaS IaaS/PaaS IaaS/PaaS PaaS
Service Type Compute (EC2), Storage (S3) Web apps Web and non-web apps Compute/Data
Virtualisation OS Level Xen Apps container OS level/Hyper-V Resource Manager and Scheduler
Dynamic Negotiation of QoS None None None SLA-oriented/Resource Reservation
User Access Interface EC2 Command-line Tools Web-based Administration Console Windows Azure portal Workbench, Tools
Web APIs Yes Yes Yes Yes
Value-added Service Providers Yes No Yes No
Programming Framework Amazon Machine Image (AMI) Python .NET framework Multiple App models in.NET languages
25
Many Cloud Offerings Good, but new
issues-vendor lock in, scaling across clouds
26
InterCloud Global Cloud Exchange and Market Maker
27
Outline
  • Computer Utilities
  • Vision and Promising IT Paradigms/Platforms
  • Cloud Computing and Related Paradigms
  • Trends, Definition, Cloud Benefits and Challenges
  • Market-Oriented Cloud Architecture
  • SLA-oriented Resource Allocation
  • Global Cloud Exchange
  • Emerging Cloud Platforms
  • Cloudbus Melbourne Cloud Computing Project
  • Summary and Thoughts for Future

28
Cloudbus_at_CLOUDS Lab Melbourne Cloud Computing
Initiative
  • Market-Oriented Clouds
  • SLA-based Resource Management
  • Global Cloud Exchange Elements Brokers
  • Aneka .NET-based Cloud Computing
  • PaaS for Enterprise and Public Clouds
  • Scaling Across Clouds (Meta Brokering)
    Harnessing Compute resources
  • Federation of clouds for application scaling
    across distributed resources
  • 3rd Party Cloud Services (e.g., MetaCDN)
    Harnessing Storage resources
  • Building Content Delivery Networks using
    different vendors Storage Clouds
  • Green Clouds / Data Centers
  • Energy Efficiency and QoS Oriented Resource
    Allocation
  • CloudSim Toolkit for Simulation of Clouds
  • Design and evaluation for resource management
    policies algorithms

29
(No Transcript)
30
Aneka .NET-based Cloud Computing
  • SDK containing APIs for multiple programming
    models and tools
  • Runtime Environment for managing application
    execution management
  • Suitable for
  • Development of Enterprise Cloud Applications
  • Cloud enabling legacy applications
  • Portability for Customer Apps
  • Enterprise ? Public Clouds
  • .NET/Win ? Mono/Linux

31
QoS Negotiation in Aneka
Meta Negotiation Registry

3. Matching
DB
DB
DB
Registries
1. Publishing
2. Publishing, Querying
MN Middelware
MN Middelware
Gridbus Broker
Aneka
4. Session Establishment
Meta-Negotiation
Meta-Negotiation
Amadeus Workflow
Handshaking
Handshaking
Alternate Offers Negotiation Strategy
Alternate Offers Negotiation Strategy
Local SLA Template
WSDL
Local SLA Template
5. Negotiation
Party 1
Party 2
API
Service Consumer
Service Provider
6. Service Invocation
32
Aneka components
public DumbTask ITask public void
Execute()
Aneka enterprise Cloud
for(int i0 iltn i) DumbTask task
new DumbTask() app.SubmitExecution(task)
work units
internet
work units
Aneka Worker Service
Aneka Manager
internet
Aneka User Agent
33
Aneka Virtual Resource Pools Integration
  • XenServer Pool
  • Provisioning over private Cloud managed by Xen
    Server
  • VMWare Pool
  • Provisioning over private Cloud managed by VMWare
  • Amazon EC2 Pool
  • Provisioning over public Cloud provider Amazon
    EC2

Executors
private enterprise network
internet
publicly available resources (physical and
virtual)
Private Cloud
VPN (virtual resources)
Executors/Schedulers
Client Libraries
Public Cloud
34
Aneka Case Studies
35
User scenario GoFront(unit of China Southern
Railway Group)
Application Locomotive design CAD rendering
36
Providing a scalable architecture for TitanStrike
on-line Gaming Portal
37
DNA MicroArray Data Analysis for BRCA (Brain
Cancer gene profiles)
Aneka on Public Cloud Amazon EC2
38
Experiments on Amazon EC2
  • Master image Aneka container with scheduling and
    task model file staging services deployed on
    Windows Server 2003
  • Worker image Aneka Container with task execution
    services deployed on RedHat Linux
  • Execution time (in minutes)

c1.medium
39
Building 3rd Party Cloud Services Harnessing
Storage Clouds
  • Building Next-Gen Content Delivery Networks

40
Motivations
  • Content Delivery Networks (CDNs) such as Akamai
    place web server clusters in numerous
    geographical locations huge upfront
    investment
  • to improve the responsiveness and locality of the
    content it hosts for end-users.
  • However, their services are priced out of reach
    for all but the largest enterprise customers.
  • Hence, we have developed an alternative approach
    to content delivery by leveraging infrastructure
    Storage Cloud providers at a fraction of the
    cost of traditional CDN providers pay as you
    go

41
MetaCDN Harnessing Storage Clouds for Content
Delivery (Broberg, Buyya, Tari, JNCA 2009)
42
Meta Brokering Harnessing Compute Clouds for
Application Scaling
  • Extending market-oriented Grid Ideas with Cloud
    computing

43
Building a Grid of Clouds ? Scaling across Clouds
Grid Information Service
Grid Resource Broker
Application
R2
R3
R4
R5
RN
Grid Resource Broker
R6
R1
Resource Broker
Grid Information Service
44
Gridbus Broker Scheduling Applications Across
Clouds and other IT Resources
Application Development Interface
Single-sign on security
Algorithm1
SchedulingInterfaces
AlgorithmN
Plugin Actuators
Data Store


Access Technology
SRB
Grid FTP
45
Experiment Setup DBC Scheduling with Optimize
for (1) Time (2) Cost
  • Workload
  • A parameter sweep synthetic application (100
    jobs), each job is modeled to execute 5 minute
    with variation of (/-20 sec.).
  • QoS Constraints Deadline 40 min. and Budget 6
  • Resources
  • US
  • Europe
  • Australia

R
R2
Information Service
R1
R4,5
Resource Broker
46
Resources Price (multiplier for clarity)
Organization Resource Details Rate (Cents per second1000 ) Total Jobs Total Jobs
Organization Resource Details Rate (Cents per second1000 ) Time-Opt Cost-Opt
Georgia State University, US snowball.cs.gsu.edu 8 Intel 1.90GHz CPU, 3.2 GB RAM, 152 GB HD, Linux 90 (0.09) 32 11
H. Furtwangen University, Germany unimelb.informatik.hs-furtwangen.de 1 Athlon XP 1700 CPU, 767 MB RAM, 147 GB HD 3 4 5
University of California-Irvine, US harbinger.calit2.uci.edu 2 Intel P III 930 MHz CPU, 503 MB RAM, 32 GB HD 2 8 10
University of Melbourne, Australia billabong.csse.unimelb.edu.au 2 Intel(R) 2.40GHz CPU, 1 GB RAM, 35 GB HD 6 8 10
University of Melbourne, Australia gieseking.csse.unimelb.edu.au 2 Intel(R) 2.40GHz CPU, 1 GB RAM, 71 GB HD 6 8 10
Amazon EC2 ec2-Medium instance 5 EC2 Compute Units, 1.7 GB RAM, 350 GB HD 60 14 16
Amazon EC2 ec2-Medium instance 5 EC2 Compute Units, 1.7 GB RAM, 350 GB HD 60 13 16
Amazon EC2 ec2-Small instance 1 EC2 Compute Unit, 1.7 GB RAM, 160 GB HD 30 7 11
Amazon EC2 ec2-Small instance 1 EC2 Compute Unit, 1.7 GB RAM, 160 GB HD 30 6 11
  Total Price / Budget Consumed Total Price / Budget Consumed 5.04 3.71
  Time to Complete Execution Time to Complete Execution 28 min 35 min
Amazon charges for 1 hour even if you use VM
for 1 sec. We should force Amazon to change
Charging Policy from 1hr block to actual usage!
Or invent a 3rd party service that manages this
by leasing smaller slots.
47
Results of Execution on Cloud and other
Distributed Resources
Organization Resource Details Rate (Cents per second1000 ) Total Jobs Total Jobs
Organization Resource Details Rate (Cents per second1000 ) Time-Opt Cost-Opt
Georgia State University, US snowball.cs.gsu.edu 8 Intel 1.90GHz CPU, 3.2 GB RAM, 152 GB HD, Linux 90 (0.09) 32 11
H. Furtwangen University, Germany unimelb.informatik.hs-furtwangen.de 1 Athlon XP 1700 CPU, 767 MB RAM, 147 GB HD 3 (0.003) 4 5
University of California-Irvine, US harbinger.calit2.uci.edu 2 Intel P III 930 MHz CPU, 503 MB RAM, 32 GB HD 2 8 10
University of Melbourne, Australia billabong.csse.unimelb.edu.au 2 Intel(R) 2.40GHz CPU, 1 GB RAM, 35 GB HD 6 8 10
University of Melbourne, Australia gieseking.csse.unimelb.edu.au 2 Intel(R) 2.40GHz CPU, 1 GB RAM, 71 GB HD 6 8 10
Amazon EC2 ec2-Medium instance 5 EC2 Compute Units, 1.7 GB RAM, 350 GB HD 60 14 16
Amazon EC2 ec2-Medium instance 5 EC2 Compute Units, 1.7 GB RAM, 350 GB HD 60 13 16
Amazon EC2 ec2-Small instance 1 EC2 Compute Unit, 1.7 GB RAM, 160 GB HD 30 7 11
Amazon EC2 ec2-Small instance 1 EC2 Compute Unit, 1.7 GB RAM, 160 GB HD 30 6 11
  Total Price / Budget Consumed Total Price / Budget Consumed 5.04 3.71
  Time to Complete Execution Time to Complete Execution 28 min 35 min
Amazon charges for 1 hour even if you use VM
for 1 sec.
QoS Constraints Deadline 40 min. and Budget 6
48
Resources Consumed by Cost and Time Opt.
Strategies
Cost-Opt
Time-Opt
UniMelb .006
EC2-m .06
UniMelb .006
EC2-m .06
UCi .002
EC2-s .03
EU .003
EC2-s .03
Georgia .09 (most expensive)
QoS Constraints Deadline 40 min. and Budget 6
Time Cost
Budget Consumed 5.04 3.71
Time to Complete 28 min 35 min
49
Experimental Evaluation is too much of work and
expensive for computing researchers?
  • CloudSim Performance Evaluation Made Easy
  • Repeatable, scalable, controllable environment
    for modelling and simulation of Clouds
  • No need to worry about paying IaaS provides
    CloudSim is FREE!

50
The CloudSim Toolkithttp//www.cloudbus.org/cloud
sim/
51
Outline
  • Computer Utilities
  • Vision and Promising IT Paradigms/Platforms
  • Cloud Computing and Related Paradigms
  • Trends, Definition, Cloud Benefits and Challenges
  • Market-Oriented Cloud Architecture
  • SLA-oriented Resource Allocation
  • Global Cloud Exchange
  • Emerging Cloud Platforms
  • Cloudbus Melbourne Cloud Computing Project
  • Summary and Thoughts for Future

52
Summary
  • Several Computing Platforms/Paradigms are
    promising to deliver Computing Utilities vision
  • Cloud Computing is the most recent kid in the
    block promising to turn vision into reality
  • Clouds built on SOA, VMs, Web 2.0 technologies
  • Many exciting business and consumer applications
    enabled.
  • Market Oriented Clouds are getting real
  • Need to move from static pricing to dynamic
    pricing
  • Need strong support for SLA-based resource
    management
  • 3rd party Composed Cloud services starting to
    emerge
  • Building Grids using Clouds is much more
    realistic.
  • Extension of idea can lead to ? Global Cloud
    Exchange

53
Dozens of Open Research Issues
  • (Application) Software Licensing
  • Seamless integration of private and Cloud
    resources
  • Security, Privacy and Trust
  • Cloud Lock-In worries and Interoperability
  • Application Scalability Across Multiple Clouds
  • Clouds Federation and Cooperative Sharing
  • Global Cloud Exchange and Market Maker
  • Dynamic Pricing
  • Dynamic Negotiation and SLA Management
  • Energy Efficient Resource Allocation and User QoS
  • Power-Cost and CO2 emission issues
  • Use renewable energy follow Sun and wind?
  • Regulatory and Legal Issues

54
Convergence of Competing Paradigms/Communities
Needed
?
  • Web
  • Data Centres
  • Utility Computing
  • Service Computing
  • Grid Computing
  • P2P Computing
  • Cloud Computing
  • Market-Oriented Computing

  • Ubiquitous access
  • Reliability
  • Scalability
  • Autonomic
  • Dynamic discovery
  • Composability
  • QoS
  • SLA
  • Trillion business
  • Who will own it?

Paradigms
Attributes/Capabilities
55
Thanks for your attention!
  • Are there any
  • Questions?
  • Comments/ Suggestions

We Welcome Cooperation in RD and Business!
http/www.gridbus.org www.Manjrasoft.com rbuyya_at_
unimelb.edu.au raj_at_manjrasoft.com
56
References
  • Blueprint Paper!
  • R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg, I.
    Brandic, Cloud Computing and Emerging IT
    Platforms Vision, Hype, and Reality for
    Delivering Computing as the 5th Utility, Future
    Generation Computer Systems (FGCS) Journal, June
    2009.
  • Aneka Documents
  • http//www.manjrasoft.com/
  • The Grid Economy Paper
  • R. Buyya, D. Abramson, S. Venugopal, The Grid
    Economy, Proceedings of the IEEE, No. 3, Volume
    93, IEEE Press, 2005.
  • MetaCDN Paper
  • James Broberg, Rajkumar Buyya, and Zahir Tari,
    MetaCDN Harnessing 'Storage Clouds' for High
    Performance Content Delivery, Journal of Network
    and Computer Applications, ISSN 1084-8045,
    Elsevier, Amsterdam, The Netherlands, 2009.
  • CloudSim Keynote Paper
  • R. Buyya, R. Ranjan and R. Calheiros, Modeling
    and Simulation of Scalable Cloud Computing
    Environments and the CloudSim Toolkit Challenges
    and Opportunities, Proceedings of the 7th High
    Performance Computing and Simulation (HPCS 2009)
    Conference, Leipzig, Germany, June 21 - 24, 2009.

57
Solutions for Cloud Computing
58
Backup
59
Gridbus Service Broker (GSB)
  • A resource broker for scheduling task farming
    data-intensive applications with static or
    dynamic parameter sweeps on global Grids and
    Clouds.
  • It uses computational economy paradigm for
    optimal selection of computational and data
    services depending on their quality, cost, and
    availability, and users QoS requirements
    (deadline, budget, T/C optimisation)
  • Key Features
  • A single window to manage control experiment
  • Programmable Task Farming Engine
  • Resource Discovery and Resource Trading
  • Optimal Data Source Discovery
  • Scheduling Predications
  • Generic Dispatcher Grid Agents
  • Transportation of data sharing of results
  • Accounting

60
workload
Gridbus User Console/Portal/Application Interface
App, T, , Optimization Preference
Gridbus Broker
Gridbus Farming Engine
Schedule Advisor
Trading Manager
RecordKeeper
Dispatcher
Grid Explorer
TM TS

GE GIS, NWS
Core Middleware
Grid Info Server
RM TS
G

Data Catalog
Data Node
C

U
G
Globus enabled node.
L
A
Amazon EC2/S3 Cloud.
61
s
62
Market-Oriented Scheduling Experiments
63
Scheduling for DBC Cost Optimization
64
Resource Scheduling for DBC Time Optimization
65
Execution Console Setting QoS
66
Aneka Cloud
Xen Server - Capacity 10 VMs
Aneka
VMWare - Capacity 5 VMs
Provision Service
Amazon Clouds
67
AnekaXen
Suspend VM (10)
Xen Server - Capacity 10 VMs
Aneka
Suspend VM (4)
VMWare - Capacity 5 VMs
Provision Service
Amazon Clouds
68
AnekaXenEC2
Suspend VM (10)
Xen Server - Capacity 10 VMs
Aneka
Suspend VM (5)
VMWare - Capacity 5 VMs
Provision Service
Release VM (9)
Amazon Clouds - Cost 20 cents per instance
Write a Comment
User Comments (0)
About PowerShow.com