Resources and Services Virtualization without Boundaries (RESERVOIR)

1
Resources and Services Virtualization without
Boundaries (RESERVOIR)

• www.reservoir-fp7.eu

Alex Galis University College London
a.galis_at_ee.ucl.ac.uk
The research leading to these results has been
partially funded by the European Community's
Seventh Framework Programme (FP7/2007-2013) under
grant agreement n 215605.
2
The RESERVOIR Vision

• The Next Generation Infrastructure for Service
  Delivery
• Provide revolutionary foundation for a new
  European infrastructure where resources and
  services can be transparently and dynamically
  managed, provisioned and relocated like utilities
  virtually without borders
• No single facility/provider can create a
  seemingly infinite infrastructure capable of
  serving massive amounts of users at all times,
  from all locations
• Federation of clouds
• Leverage the diversity factor to achieve
  economies of scale
• Leverage locality
• Analogies exists in areas outside services
• Electrical power delivery capacity can be
  shifted to guarantee supply and lower costs
• Roaming cellular communications Talk wherever
  you are
• Impact
• Enable utility-like deployment of services,
  relieving the service consumer from awareness of
  the IT attributes while assuring QoS and security
  guarantees
• Aim to create the basis for future service
  products

3
Approach

• Focus on technologies that enable to build
  cooperating computing clouds
• Connect computing clouds to create an even bigger
  cloud
• Integration of virtualization technologies with
  grid computing driven by new techniques for
  service management ? ?The Service Oriented
  Infrastructure (SOI) equation

• Building on this equation we will architect and
  implement a platform for supporting complex
  services, which
• Enables dynamic deployment of complex multi-tier
  services across heterogeneous administration
  domains
• Uses virtualization of servers, storage and
  network to allow migration without borders
• Supports service definition, SLA management,
  accounting and billing

4
RESERVOIR Partners and Competencies
Partner Role Comment
IBM HRL Technology Project Lead, Virtualization/SOA Infrastructure
Telefonica ID Technology Service Technology, Billing Infrastructure
UCM Technology Grid, Dynamic Allocation Technology
Thales Technology Security, Virtualization Infrastructure, Hosting
SAP Use-Cases Use-Cases, Contribution to Requirement an Standards
Sun Microsystems Use-Cases Tech Contribution to Standards, Java Services, Monitoring
DATAMAT Technology Service Management Technologies
University Lugano Technology Partner, Monitoring and SLA Management
University UMEA Technology Monitoring, Measuring and Billing Technology
University Messina Technology Grid Experience, Testbed Development,
University College London Technology Service Management, Virtualization Technology
CETIC Technology Security
OGF Standardization Grid and Virtualization Standards
5
The Reservoir Architecture

• Monitor service and enforce SLA compliance by
  managing capacity of Service Components (VEEs)
  or/and size of Service Tiers
• Deals with translation/mapping of service
  concepts/metrics (response time) to
  infrastructure concepts/metrics (VEE size)

• Monitor VEEs and find best VEE placement that
  meet constraint satisfaction problem
• Deals federation of domains

6
Project Structure
7
A2 VEE Infrastructure

• Virtual Machine Technologies (IBM)
• Improve performance of VEE execution for typical
  RESERVOIR workloads
• Provide VEEMS enablement layer for virtual
  machines
• Relocation Enablement (IBM)
• Network Virtualization
• Storage Virtualization
• Java Service Containers (Sun)
• Provide VEEMS enablement layer for virtual java
  service containers

8
A3 VEE Management

• VEE Provisioning and Supervision (UCM)
• Image management
• Monitoring
• Allocation Policy Management (Datamat)
• Policy based placement and migration
• Federation of Management Domains (UCM)
• Built atop WSRF interfaces to access remote VEE
  Supervisors
• Push new and leverage existing OGF/DMTF/OASIS
  standards
• Interoperability between administrative domains
  and scheduling heuristics on federated and
  utility architectures.

9
A4 Service Management

• Service Definition (UCL)
• Design a new service description language that
  will allow the description of service interfaces,
  service lifecycle, interface bindings to
  implementations, service deployment, SLA
  requirements for a service, rules for VEEs
  (re)configuration and (re)organisation and
  service components distribution and configuration
• Revisit the service lifecycle definition to
  accommodate the influence of virtualisation
• Extend tools available for service design (for
  example the Eclipse Web Tools Platform)
• Standardize the service description language
• Service Management (TID, UCL)
• SLA monitoring across administrative domains and
  service-oriented architectures.
• Integrate monitoring with resource allocation and
  scheduling and take explicit account of the
  potentially synchronous nature of service
  invocations.
• Automatic deployment of services based on complex
  service definition
• Accounting, Billing and Payment (TID)
• Accounting and billing arrangements for
  outsourced services are based on raw machine
  resource consumption (CPU-time, storage capacity
  etc)
• RESERVOIR will pursue the definition of a
  framework that allows accounting and billing in
  terms of the services that were completed, taking
  into consideration the quality of service that
  was provided.

10
A5 Experimentation and Validation

• Testbed (UniMe)
• Create the necessary environment for testing and
  validation
• Support the execution of use cases
• Scenario 1 eGov application (Thales)
• Automatic adjustment of resources and domains
  cooperation.
• Scenario 2 SAP business application (SAP)
• Business application oriented use cases and the
  opportunities to execute them on a flexible
  infrastructure.
• Scenario 3 Utility computing (Sun)
• Deploy arbitrary operating system and application
  stacks on remote resources. Provide secure and
  seamless access to them. Adjust resource
  allocation on-demand without the end user
  noticing disruption of service
• Scenario 4 Telco application (TID)
• Hosting web sites that deals with massive access
  (e.g., the Olympics games)
• High degree of personalization and support for
  mashups

11
Results so far

• RESERVOIR Architecture
• http//ec.europa.eu/information_society/events/cf
  /item-display.cfm?id583
• 10 presentations at conferences workshops
• 2 papers fully based on the IPR generated in the
  project 5 papers partially based on the IPR
  generated in the project
• www.reservoir-fp7.eu

12
(No Transcript)
13
The Evolution of the Power Grid

• Make your own infrastructure
• Not the companys main business but a
  considerable competitive advantage

• The utility industry
• Metering
• Limited reach
• Reproducible (yet costly)

• Efficient distribution
• Federation of providers
• The diversity factor
• Economies of scale

14
The Evolution of the Compute Grid
todays commercial clouds have not been open
and general purpose, but instead been mostly
proprietary and specialized for the specific
internal uses (e.g., large-scale data analysis)
of the companies that developed them. The idea
that we might want to enable interoperability
between providers (as in the electric power grid)
has not yet surfaced will move towards a
mix of microproduction and large utilities, with
increasing numbers of small-scale producers
co-existing with large-scale regional producers,
and load being distributed among them dynamically
Theres Grid in then thar Clouds - Ian Foster

• Make your own infrastructure
• Not the companys main business but a
  considerable competitive advantage

• The utility industry
• Metering
• Limited reach
• Reproducible (yet costly)

• Efficient distribution
• Federation of providers
• The diversity factor
• Economies of scale

15
SOI Grid Computing
Grid node or Service Site
Physical Resources
Service Tasks
16
SOI Grid Computing Virtualization
Virtual Execution Environment (VEE)
Improved isolation, Relax dependencies, Well
defined billing units
17
SOI Grid Computing Virtualization BSM
Policy 1 If possible keep VEEs fromthe same
organization in the same physical box
18
SOI Grid Computing Virtualization BSM
Policy 1 If possible keep VEEs fromthe same
organization in the same physical box
Policy 2 Turn off underutilized physical boxes
19
SOI Grid Computing Virtualization BSM
Policy 1 If possible keep VEEs fromthe same
organization in the same physical box
Policy 2 Turn off underutilized physical boxes
Local optimizations (within a single site)
placement, power, etc.
20
SOI Grid Computing Virtualization BSM
Boundaries
Policy 3 If possible keep VEEs in
owning organization
21
SOI Grid Computing Virtualization BSM
Boundaries
Policy 3 If possible keep VEEs in
owning organization
Policy 4 If possible keep VEEs in least number
of external organizations
22
SOI Grid Computing Virtualization BSM
Boundaries
Policy 3 If possible keep VEEs in
owning organization
Policy 4 If possible keep VEEs in least number
of external organizations
23
SOI Grid Computing Virtualization BSM
Boundaries
Policy 5 Follow the service customer
Migration across sites ? Global optimizations
placement, cost, bandwidth, etc.
24
Virtualize the Network
Create virtual networks connecting VEEs
regardless of physical server location
25
Virtualize the Network and the Storage
Enable secure access to relevant data regardless
of storage location