Model-Driven Engineering for Development-Time QoS Validation of Component-based Software Systems

1
Model-Driven Engineering for Development-Time QoS
Validation of Component-based Software Systems

• James Hill, Sumant Tambe Aniruddha Gokhale
• Vanderbilt University, Nashville, TN, USA
• IEEE International Conference Workshop on
• Engineering of Computer Based Systems (ECBS 07)
• March 26- 29, 2007
• Tucson, AZ, USA

2
Motivation Model-Driven Engineering

• Historically, distributed real-time embedded
  (DRE) systems were built directly atop OS
  protocols

Applications
Operating System Communication Protocols
Hardware Devices
3
Motivation Model-Driven Engineering

• Historically, distributed real-time embedded
  (DRE) systems were built directly atop OS
  protocols

Applications

• Traditional methods of development have been
  replaced by middleware layers to reuse
  architectures code
• Viewed externally as Service-Oriented
  Architecture (SOA) Middleware

Operating System Communication Protocols
Hardware Devices
4
Motivation Model-Driven Engineering

• Historically, distributed real-time embedded
  (DRE) systems were built directly atop OS
  protocols

Applications

• Traditional methods of development have been
  replaced by middleware layers to reuse
  architectures code
• Viewed externally as Service-Oriented
  Architecture (SOA) Middleware

• Model-driven engineering (MDE) techniques coupled
  with domain-specific model languages (DSMLs) are
  increasingly being used to address the challenges
  of building large-scale component-based systems

Operating System Communication Protocols
Hardware Devices
5
Motivation Current Model-Driven Engineering
Techniques

• Many existing MDE techniques for large-scaled
  component-based system focus primarily on

• Structural issues
• Component representation (i.e., interfaces
  attributes)
• Component assembly packaging
• System deployment configuration
• Functional operational issues
• Model checking (e.g., checking for functional
  correctness)
• Runtime validation of performance (e.g., running
  simulations at design-time, or empirical
  benchmarks at integration time)

There remains a major gap in evaluating system
quality of service (QoS) at different phases of
development to allow design flaws to be rectified
earlier in the development lifecycle.
6
Serialized Phasing is Common in Large-scale
Systems (1/2)
System infrastructure components developed first
Application components developed after
infrastructure is mature
7
Serialized Phasing is Common in Large-scale
Systems (2/2)
Finished development
System integration testing
Integration Surprises!!!
8
Complexities of Serialized Phasing
Still in development
Ready for testing

• Complexities
• System infrastructure cannot be tested adequately
  until applications are done

9
Complexities of Serialized Phasing
Overall performance?

• Complexities
• System infrastructure cannot be tested adequately
  until applications are done
• Entire system must be deployed configured
  properly to meet QoS requirements
• Existing evaluation tools do not support what
  if evaluation

It is hard to address these concerns in processes
that use serialized phasing
10
QoS Concerns with Serialized Phasing
Meet QoS requirements?

• Key QoS concerns
• Which DCs meet the QoS requirements?

11
QoS Concerns with Serialized Phasing
Performance metrics?

• Key QoS concerns
• Which DCs meet the QoS requirements?
• What is the worse/average/best time for various
  workloads?

12
QoS Concerns with Serialized Phasing
System overload?

• Key QoS concerns
• Which DCs meet the QoS requirements?
• What is the worse/average/best time for various
  workloads?
• How much workload can the system handle until its
  QoS requirements are compromised?

It is hard to address these concerns in processes
that use serialized phasing
13
Promising Solution Approach Emulate Behavior
using Next Generation System Execution Modeling
Tools
Component Workload Emulator (CoWorkEr)
Utilization Test Suite (CUTS) Workflow

• While target system under development
• Use a domain-specific modeling language (DSML) to
  define validate infrastructure specifications
  requirements
• Use DSML to define validate application
  specifications requirements
• Use middleware MDD tools generate DC metadata
  so system conforms to its specifications
  requirements
• Use analysis tools to evaluate verify QoS
  performance
• Redefine system DC repeat

Enables testing on target infrastructure
throughout the development lifecycle http//www.dr
e.vanderbilt.edu/hillj/docs/publications/CUTS-RTC
SA06.pdf
14
Case Study Distributed Stock Application (DSA)
(1/2)
Usage Patterns by User Type Usage Patterns by User Type Usage Patterns by User Type
Type Percentage Response Time (msec)
Basic (Client A) 65 300
Gold (Client B) 35 150

• Implementation Details
• All components are to implemented as CORBA
  Component Model (CCM) components using
  ACETAOCIAO 5.1 middleware
• Target architectures includes 3 separate hosts
  running Fedora Core 4.
• Each component in the DSA is schedule to complete
  is development at different times in development
  lifecycle

15
Case Study Distributed Stock Application (2/2)
Replaceable with real component
CCM
Microsoft .NET
Realistic behavior workload
Realistic user behavior

• Challenges of Continuous QoS Validation
• Emulating Business Logic Emulated components
  must resemble their counterparts in both
  supported interfaces behavior
• Realistic Mapping of Emulated Behavior Behavior
  specification should operate at a high-level of
  abstraction map to realistic operations
• Technology Independence Behavior specification
  should not be tightly coupled to a programming
  language, middleware platform, hardware
  technology, or MDE tool

16
Domain-Specific Modeling Languages for
Continuous QoS Validation

• Component Behavior Modeling Language (CBML)
• a high-level domain-specific modeling language
  for capturing the behavior of components
• e.g., its actions states

• Workload Modeling Language (WML)
• a domain-specific modeling language for capturing
  workload, used to parameterize the actions on
  CBML

CBML WML were both developed using GME
(http//www.isis.vanderbilt.edu/projects/GME)
17
The Component Behavior Modeling Language

• Context
• Components behavior can be classified as
  communication internal actions
• Need to define these actions as close as possible
  to their real counterpart (i.e., Challenge 1)
• Research Contributions of CBML
• Based on the semantics of Input/Output (I/O)
  Automata
• i.e., contains representative elements
• Behavior is specified using a series of action to
  state transitions
• Transitions have preconditions that represent
  guards effects have postconditions that
  determine the new state after an action occurs
• Variables are used to store state can be used
  within pre postconditions

18
Domain-Specific Extensions to CBML

• Context
• Some aspects of component-based systems are not
  first-class entities in I/O automata
• e.g., lifecycle events monitoring notification
  events
• Extended CBML (without affecting formal
  semantics) to support domain-specific extensions
• Domain-Specific Extensions
• Environment events input actions to a component
  that are triggered by the hosting system rather
  than another component
• Periodic events - input actions from the hosting
  environment that occur periodically

environment
periodic
19
Ensuring Scalability of CBML

• Context
• One of the main goals of higher-level of
  abstraction is simplicity ease of use
• It is known that one of the main drawbacks of
  automata languages is scalability
• Usability Extensions
• Composite Action contains other actions helps
  reduce model clutter
• Repetitions - determines how many times to repeat
  an action to prevent duplicate sequential actions
• Log Action - an attribute of an Action element
  that determines if the action should be logged
• Tool Specific GME add-on that auto-generates
  required elements (e.g., states)

20
The Workload Modeling Language
generic action

• Context
• CBML as a standalone language is sufficient
  enough to capture behavior of a component
• For emulation purposes, it does not capture the
  reusable objects of a component its workload
  (i.e., Challenge 2)
• Research Contributions of WML
• Middleware, hardware, platform, programming
  language independent DSML
• Used to define workload generators (workers) that
  contains actions to represent realistic
  operations
• Defined using a hierarchical structure that
  resembles common object-oriented programming
  packaging techniques that are consistent with
  conventional component technologies

no payload
executable actions
workload generator
21
Integrating WML Models with CBML Models

• Context
• CBML WML are standalone DSMLs with a distinct
  purpose
• i.e., model behavior workload, respectively
• WML is designed to complement CBML by provide
  CBML with reusable operations that can map to
  realistic operations

• Integration Enablers
• WML worker elements have same model semantics as
  variables
• WML actions have same modeling semantics as CBML
  actions
• Allows WML elements to be used in CBML models

CBML
WML action
WML
22
Integrating Behavioral and Structural DSMLs

• Context
• Structural DSML (e.g., the Platform Independent
  Component Modeling Language (PICML)) capture the
  makeup of component-based systems
• There is no correlation between a components
  ports its behavior
• Integration Enablers
• Defined a set of connector elements that allow
  structural DSMLs to integrate with (or contain)
  CBML models
• Input ports directly connect to Input Action
  elements
• Output actions have the same name as their output
  port to reduce model clutter
• i.e., prevent many to one connections

input connections
output name
23
Code Generation for Emulation

• Context
• The generation technique should not be dependent
  on the underlying technology
• Although we are targeting CUTS, would should be
  able to generate emulation code for any
  benchmarking framework
• Generation Enablers
• Emulation layer represents the application
  layers business logic where elements in WML
  used to parameterize CBML behavior are mapped to
  this layer
• Template layer acts as a bridge between the
  upper emulation layer lower benchmarking layer
  to allows each to evolve independently of each
  other
• Benchmark layer the actual benchmarking
  framework (e.g., CUTS)

component method
emulation
template / benchmarking
24
Lessons Learned

• Using a DSML based on a mathematical formalism to
  define behavior of components helps in specifying
  unambiguous behavior when generating code
  configuration files for emulation simulation.
• Separating the workload, behavior, structural
  models allows all to evolve independently of each
  other. Moreover, it encourages the same behavior
  model to be supported in multiple structural
  models to increase portability, flexibility, and
  usability.
• Using generative programming with templates that
  are parameterized by actions from behavioral
  models allows the DSML to easily be adapted to
  different environments of execution

25
Future Work
use value of input events

• Extend CBML with a simple programming language
  that will allow developers to use component
  properties elements when defining behavior
• e.g., event members
• Extend the current capabilities of both modeling
  languages and code generators to handle
  evaluation of real components
• i.e., benchmarking them using realistic input
  data)
• Extend the capabilities of WML such that existing
  class objects in the target programming language
  can be used in WML models
• i.e., enable real implementation code to be
  generated directly from models.

real component needs real data
26
Concluding Remarks

• We described a MDE generative approach to address
  the challenges of evaluating component-based
  system QoS throughout the development lifecycle
  instead of delaying it to integration time
• Our approach included two DSMLs, namely CBML
  WML, which allows use to generate realistic
  applications for emulation early in the
  development lifecycle
• Our approach facilitates continuous system
  integration as real components are completed,
  they replace the emulated components
• Likewise, as more is learned about the system,
  components behavior can be refined regenerated

Tools are available as open-source can be
downloaded from http//www.dre.vanderbilt.edu/CoS
MIC http//www.dre.vanderbilt.edu/CUTS
27
Questions