ContextSpecific Middleware Specialization Techniques for Optimizing Software Productline Architectur

1
Context-Specific Middleware Specialization
Techniques for Optimizing Software Product-line
Architectures
Arvind S. Krishna, Aniruddha S. Gokhale, Douglas
C. Schmidt Institute for Software Integrated
Systems, Dept of EECS Vanderbilt University
Nashville, TN, USA Venkatesh P. Ranganath, John
Hatcliff Dept of Computer and Information
Sciences Kansas State Univ Manhattan, KS, USA
Eurosys06, Leuven, Belgium April 18-21, 2006
2
Middleware for Product Line Architectures
Air Frame
FLIR
AP
HUD
GPS
Nav
IFF
Domain-specific Services
Common Middleware Services

• Middleware factors out many reusable
  general-purpose domain-specific services from
  traditional DRE application responsibility
• Essential for product-line architectures (PLAs)
• e.g., Boeing Boldstroke Avionics mission
  computing PLA for Boeing fighter aircrafts (F-15,
  F/A-18, AV-8B, UCAV/JUCAS)
• DRE system with 100 developers, 3,000 software
  components, 3-5 million lines of C
• Used as open experimentation platform

3
Middleware for Product Line Architectures
Air Frame
FLIR
AP
HUD
GPS
Nav
IFF
Domain-specific Services
Common Middleware Services

• Middleware factors out many reusable
  general-purpose domain-specific services from
  traditional DRE application responsibility
• Essential for product-line architectures (PLAs)
• However, standards-based, general-purpose,
  layered middleware is not yet adequate for the
  most demanding mission-critical PLA-based DRE
  systems

4
Middleware for Product Line Architectures
Air Frame
FLIR
AP
HUD
GPS
Nav
IFF

• Middleware factors out many reusable
  general-purpose domain-specific services from
  traditional DRE application responsibility
• Essential for product-line architectures (PLAs)
• However, standards-based, general-purpose, layers
  middleware is not yet adequate for the most
  demanding mission-critical PLA based DRE
  systems

Soln Middleware Specialization for PLA-based DRE
systems
5
Middleware Specialization Evaluation Criteria

• Premise
• Application of specialization techniques should
  result in considerable improvements in QoS over
  above horizontal general-purpose middleware
  optimizations
• Handcrafting specializations infeasible for
  large-scale DRE systems need for tools and
  processes
• Specializations should have minimal impact on
  standards compliance (APIs)

• Evaluation Criteria
• Use TAO (www.dre.vanderbilt.edu/TAO) as gold
  standard with several general-purpose
  optimizations
• Set performance improvements 30 to 40
  improvement from application of specializations
  cumulatively
• Turning on just one/two optimizations might
  improve performance by 10 to 15

6
Opportunities for Middleware Specialization

• Dimension 1 Specification-imposed generality
• Standards-based general purpose middleware
  functionality defined by specifications such as
  CORBA, J2EE etc

• Certain functionality can be excessive for PLAs
• e.g., layered demultiplexing, leading to
  unnecessary performance overhead

• Challenge automatically remove
  specification-imposed generality when its not
  needed
• Goal is to devise techniques that apply to any
  standards compliant middleware, not just an
  implementation

4
3
2
1
7
Opportunities for Middleware Specialization

• Dimension 2 Middleware framework generality
• General-purpose middleware implementations need
  to work across applications that have varying
  functional QoS requirements
• Accommodate variability by providing hooks

• e.g., for different protocol, concurrency
  demultiplexing strategies

• Hooks add overhead ?indirections dynamic
  dispatching
• PLAs however require one alternative one
  protocol

Thread-pool, Single-threaded, Thread-per
connection

• Challenge Automatically specialize middleware
  frameworks to eliminate unnecessary hooks
• Goal is devise techniques applicable to
  distributed systems that apply common patterns

TCP/IP, VME, SCTP, SHMIOP
8
Opportunities for Middleware Specialization

• Dimension 3 Platform generality
• Middleware implementations run on different
  hardware/OS/compiler platforms
• Platforms provide certain optimizations that can
  be leveraged to enhance QoS

• Challenge Automatically discover PLA deployment
  platform characteristics to improve QoS
• Goal is to devise techniques that apply to any
  host infrastructure middleware (e.g., ACE or
  JVMs) targeting heterogeneous OS, compiler,
  hardware platforms

gcc 3.2 (no exceptions), timesys kernel
Green-hills compiler, vxWorks platform
9
Bold Stroke PLA Scenario
Goal Select representative DRE system, e.g.,
rate based events for control information
operations that transmit common data

• Example PLA configuration Basic Single Processor
  (BasicSP) DRE system scenario based on Boeing
  Bold Stroke challenge problems from DARPA PCES
  MoBIES

• Timer Component Triggers periodic refresh rates

• GPS Component Generates periodic position
  updates

• Airframe Component Processes input from the GPS
  component feeds to Navigation display

• Navigation Display Displays GPS position updates

ACE_wrappers/TAO/CIAO/DAnCE/examples/BasicSP
CoSMIC/examples/BasicSP
10
Identifying Ahead of Time System Invariants
Specification Invariance
Single method interfaces Sends same operation on
wire
Framework Invariance
Deployment Invariance
Does not support native exceptions
Protocol A specific protocol used
A specific Reactor used
11
Specializing Dispatch Resolution
Dim 1 Specification Imposed generality
Dim 2 Framework generality
Dim 3 Deployment generality

• Invariants
• The same operation is invoked on a given
  connection

Normal layered path

• Request Processing Paths
• Normal layered path ?Uses general-purpose
  optimization for request lookup
• Optimized fast path ? Bypasses middleware layers
  to directly perform operation invocation
• 15 processing steps across three layers in
  ACETAO middleware

Optimized Fast path processing
This specializations is an example of Memoization
Layer-folding specialization, where a
pre-computed result is cached used to bypass
middleware layers
12
Specializing Reactor Component
Dim 1 Specification Imposed generality
Dim 2 Framework generality
Dim 3 Deployment generality

• Invariants
• Middleware components, such as Reactor and
  protocol components remain same for a PLA
  scenario

• Reactor Framework
• Reactor separates event detection from
  demultiplexing
• 5,000 LOC across ACETAO modules
• Applies bridge pattern to support different
  reactors

• Specialize Bridge Pattern
• Remove indirection, e.g., Reactor_Impl base class
  completely (all virtual methods concrete)
• No changes to component interface doesnt break
  compatibility

This specialization is an example of Aspect
Weaving specialization, where crosscutting
middleware features are customized
13
Deployment Platform Specializations
Dim 1 Specification Imposed generality
Dim 2 Framework generality
Dim 3 Deployment generality

• Invariants
• Hardware, OS, compiler characteristics do not
  change

• Deployment Platform Generality
• Platform characteristics affect QoS
• TANGO ?Timesys linux gcc
• ACE ?vxWorks Greenhills compiler
• Certain versions of gcc do not support exceptions

• Specialize Middleware for Deployment platform
• Empirically (via benchmarks) determine the right
  set hardware, OS compiler characteristics
  suitable for the deployment platform (similar to
  research done by Yotov et.al)

This specialization is a variation of Constant
Propagation where the constant (platform
characteristic) is used to tailor the middleware
14
Feature Oriented CUStomizer (FOCUS)
FOCUS addresses specialization challenges by
building specialization language, tool, process
to capture automate middleware specializations
Middleware Instrumentation Phase

• Capture specialization transformations via FOCUS
  specialization language

• Annotate middleware source code with
  specialization directives

• Create a domain-specific language (DSL) to
  capture middleware variability

• 1,000 Perl SLOC Parser weaver
• 2,500 XML SLOC specialization files
• 50 (files) annotations

Middleware Specialization Phase

• Analyses determines the type of specializations
  applicable

• FOCUS transformation engine selects the
  appropriate transformations uses the
  annotations to automate specializations

15
FOCUS Specialization Language (FSL)

• FOCUS uses an XML DTD to create a DSL for
  capturing specializations

• Capability to perform code
• substitutions
• Devirtualize interfaces
• Replace base classes with derived
• class
• FSL Approach
• Capability to do
• on code

• Capability to specialize base implementations
• Framework specialization requires code
• to be copied from derived to base classes
• FSL Approach
• Copy code from a specific
• point to a specific point between different files
  
• ? start copying
• ? hook to stop copying
• ? destination

• Capability to weave code at
• specified points
• The layer folding specializations
• require code to be woven in along
• the request processing path
• FSL Approach
• that specifies the
• annotated point where
• data is specified

FSL
ACE_Reactor_Impl
ACE_Select_Reactor_Impl itute
Select_Reactor.hrce HOOK-START
HOOK-END
HOOK-COPY
FORWARD_DECL include
Select_Reactor.h
16
Specialization Experimental Setup

• Goals
• Application of specialization techniques should
  result in considerable improvements in QoS over
  above horizontal general-purpose middleware
  optimizations

• TAO baseline
• Active demultiplexing perfect hashing for O(1)
  request demultiplexing
• Buffer caching direct collocation optimization
• Optimized configuration for different ORB
  components

Specialized TAO Middleware

• Experiment Setup
• Pentium III 850 Mhz processor, running Linux
  Timesys 2.4.7 kernel, 512 MB of main memory, TAO
  version 1.4.7 compiled with gcc 3.2.3
• Timers at the client within ORB used to collect
  data
• Used Emulab testbed

17
Results for Layer-folding Specialization
Dim 1 Specification Imposed generality
Dim 2 Framework generality
Dim 3 Deployment generality

• Experiment
• End-to-end latency measures for
• General-purpose optimized TAO with active
  demultiplexing perfect hashing
• Specialized TAO with layer folding specialization
  enabled

Average path measures improved by 40
Average end-to-end measures improved by 16
Worst case path measure improved by 20
Worst case end-to-end latency improved by 14
Dispersion improves by a factor of 1.5 for both
cases

• Path specialized latency measures
• Path defined as code-path when a request is
  received until the upcall is dispatched on the
  skeleton

Specialization applied at the server side (can
also be applied at the client side)
18
Aspect Weaving Specialization Results
Dim 1 Specification Imposed generality
Dim 2 Framework generality
Dim 3 Deployment generality

• Experiment
• Client/server communicate by sending a simple
  long
• End-to-end latency measures for
• General-purpose optimized TAO and
• Reactor, Protocol frameworks specialized with
  FOCUS compared

• Average Measures
• Reactor 4.5
• Protocol 5

Jitter better specialization does not compromise
predictability

• Result Synopsis
• Average latency improves by 5 for both protocol
  reactor specializations

Specialization also applied to other frameworks
in TAO Messaging, Wait Strategies
Worst case measures improved compared to
general-purpose TAO

• Jitter improves, though not considerably

• 99 values closer to average worst-case values
  are better indicating better predictability

19
Results for Autoconf Specialization
Dim 1 Specification Imposed generality
Dim 2 Framework generality
Dim 3 Deployment generality
Average latency improves 17

• Deployment platform specialization
• Exception support
• Loop unrolling optimization
• During ORB configuration, autoconf used to use
  emulated exceptions loop unrolling optimization

Predictability improved over general-purpose
middleware
Worst-case measures improve by 50 µs
Future research will extend research on these
empirical optimizations to more model-driven
approaches for performance prediction
20
Cumulative Specialization Results
Layer folding, deployment platform, memoization,
constant propagation

• Specification related
• Layer folding
• Memoization
• Constant propagation (ignoring endianess)
• Framework
• Aspect weaving (Reactor protocol)
• Deployment
• Loop unrolling emulated exceptions

Average end-to-end measures improved by 43
Worst-case measures improved by 45
Jitter results twice as good as general-purpose
optimized TAO

• End-to-end client side throughput improved by
  65.
• Results exceeded the hypothesis evaluation
  criteria

21
Evaluating FOCUS Pros Cons

• Strengths
• Provides a lightweight, zero (run-time) overhead
  middleware specialization
• Designed to work across different languages
  (e.g., Java C)
• KSU applying FOCUS specializations to Java ORBs
• XML-based rule capture
• Easy language extension, ability to add new
  features easily
• If/when C aspect technologies mature, can
  transform them into aspect rules via XSLT
  transforms
• Execute transformations via scripting
• Integration with QA tools code generation from
  models

• Drawbacks
• Doesnt provide full-fledged language parser,
  i.e., join points identified via annotations or
  via regular expressions
• Need to synchronize annotations with
  specialization files, so modifying source code
  requires change to specialization files
• Ameliorated via distributed continuous QA
  Limitation exists even with aspects
• Correctness of transformations have to be
  validated externally unlike AspectJ
• Need higher level tools to validate combinations
  of specializations

FOCUS available in latest ACETAO distribution in
ACE_wrappers/bin/FOCUS
22
Lessons Learned (1/2)

• Need to document specialization interplay
• Specializations applied such that no
  dependencies/conflicts exists
• As more specializations are developed it is
  necessary to document dependencies between
  specializations

• QoS benefits implementation specific
• Optimizations improve QoS over above
  general-purpose optimizations
• Other ORBs, e.g., JacORB, do not use such
  optimizations performance gains in this case
  would be more
• Preliminary results Daugherty on ZEN show that
  speed up was better compared to TAO

TAO uses active demultiplexing perfect hashing
for O(1) demux time
Daugherty Gary Daugherty, A Proposal for the
Specialization of HA/DRE Systems, ACM PEPM 04
23
Lessons Learned (2/2)

• Specializations have potential to improve task
  schedulability
• Specializations help high priority tasks finish
  ahead of their time to complete
• Tasks with priority 50 finish early, increasing
  time available for scheduling priorities with 35
  10
• Specializations can benefit both hard real-time
  soft softer real-time tasks

Specializations increase slack in the system

• Adaptation with specializations can adversely
  affect QoS
• Specializations do not consider any form of
  recovery if invariance assumptions fail
• Adaptation requires loading general-purpose code,
  add checks along request processing path
  increases jitter for DRE systems

24
Future Work ?System Optimizations

• FOCUS approach applied to middleware
  optimizations

• Future work will focus on identifying system
  level (middleware, platform, application)
  specializations

• Goal is to drive the specialization process to
  optimize systems layer-to-layer

• Capturing invariants in models and using
  generative technologies to drive specializations

• Other QoS parameters

25
Concluding Remarks

• Resolving the tension between
• Generality ? Middleware is designed to be
  independent of particular application
  requirements

• Specificity ? PLAs are driven by the functional
  QoS requirements for each product variant (using
  SCV analysis)

Common Middleware Services
Specialized Middleware Stack

• Development of reusable specialization patterns

• Identifying specialization points in middleware
  where patterns are applicable

• Domain-specific language (DSL) tools process
  for automating the specializations

• Latency improvements of 45

• www.dre.vanderbilt.edu

26
QUESTIONS ?
27
FOCUS Research Goals
Domain specific language for capturing
specializations minimize accidental complexity
Ensuring transformations (1) do not incur
additional overhead (2) do not compromise
portability
Develop a process that enables role separation
Process to support specialization with evolution
Applying specializations improve latency 30
40
Language agnostic leverage annotations

• Contributions The goal of this work isnt on
  specialization languages per se, but instead to
  quantify the improvements that come from applying
  various types of specializations to middleware
• Transformational approaches Stratego-XT, DMS
  support a broader range of specializations

28
FOCUS Specialization Phase
Annotations help identify join points
Annotated middleware source
Transformed middleware source
//File Reactor.h //_at__at_ REACTOR_SPL_INCLUDE_HOOK /
/ Code woven by FOCUS include
"ace/Select_Reactor.h" // END Code woven by
FOCUS class Reactor public int
run_reactor_event_loop () // Other public
methods .... private // Code woven by FOCUS
ACE_Select_Reactor reactor_impl_ // End Code
woven by FOCUS // ......
//File Reactor.h //_at__at_ REACTOR_SPL_INCLUDE_HOOK cl
ass Reactor public virtual int
run_reactor_event_loop () // Other public
methods .... private ACE_Reactor_Impl
reactor_impl_ // ......
include concrete header file
Removed virtual keyword
Replaced ACE_Reactor_Impl with ACE_Select_Reactor

• These specializations dont affect framework
  business logic, just its structure
• Existing framework is still available when
  developers need OO capabilities

29
Automating Layer Folding Specialization
Dim 1 Specification Imposed generality
Dim 2 Framework generality
Dim 3 Deployment generality

• Automating Specialization via FOCUS
• On a per connection basis determine the target
  dispatcher
• For subsequent requests, directly use the
  dispatcher to process requests

Normal path the dispatch is cached in the process
TAO_DISPATCH_OPT_ADD_HOOK
1. if (__dispatch_resolved__)
2. 3 //First invocation normal path
4 __dispatch_resolved_ 1 5
6 else 7 // All invocations go
through --- Optimized path 8
this-request_disaptcher__.dispatch () 9

FOCUS enables specification of specializations in
corresponding language syntax
Directly use cached dispatcher
30
Automating Reactor Specialization
Dim 1 Specification Imposed generality
Dim 2 Framework generality
Dim 3 Deployment generality
TAO_REACTOR_SPL_COMMENT_HOOK_ENDok ACE_NEW_RETURN (impl,
ACE_Select_REACTOR (.),
0)

• Automating Specialization via FOCUS
• , weave code that creates
  specialized Reactor
• to move implementation from
  Select_Reactor to base Reactor component
• comment unspecialized code

Transformations preserve hooks Minimizes clutter
in middleware source
ACE_Reactor_Impl TAO_Advanced_Resource_Factory
allocate_reactor_impl (void) const
ACE_Reactor_Impl impl 0 //_at__at_
TAO_REACTOR_SPL_COMMENT_HOOK_START switch
(this-reactor_type_)
//_at__at_ TAO_REACTOR_SPL_COMMENT_HOOK_END
31
Automating Platform Specializations
Dim 1 Specification Imposed generality
Dim 2 Framework generality
Dim 3 Deployment generality

• Deployment platform specialization
• Exception support
• Autoconfs AC_COMPILE_IF_ELSE macro to check if
  compiler supports native exception
• Loop unrolling optimization
• Autoconfs AC_RUN_IF_ELSE macros to compile run
  benchmark to quantify memcpy() performance
  improvements

Provide actual source language that will be
compiled run
Variable that will be checked to set a
compilation flag
ACE_CACHE_CHECK(if ACE memcpy needs loop
unrolling, ace_cv_memcpy_loop_unroll, AC_RUN_I
FELSE(AC_LANG_SOURCE( /// . Program that
will run the benchmark to //// determine if an
optimization is better
Directive to run the benchmark
This approach can be applied automatically to
discover OS-specific system calls compiler
settings that maximize QoS
32
Concluding Remarks

• Specification-imposed specializations
• Layer-folding
• Constant propagation
• Memoization

• Framework specializations
• Aspect Weaving techniques
• Bridge ?Reactor
• Template method ? Protocol
• Strategy ?Messaging, Wait, Demultiplexing

• Development of reusable specialization patterns

• Deployment platform specializations
• Unroll copy loops
• Use emulated exceptions
• Leverage zero-copy data transfer buffers

• Identifying specialization points in middleware
  where patterns are applicable

• Domain-specific language (DSL) tools process
  for automating the specializations

33
Research Challenges

• Resolving the tension between
• Generality ? Middleware is designed to be
  independent of particular application
  requirements

• Specificity ? PLAs are driven by the functional
  QoS requirements for each product variant (using
  SCV analysis)

Common Middleware Services
Standards-based, General-purpose, Layered
Middleware Architecture
34
Related Research
Current research on specializations does not
address how techniques tools can be applied to
middleware to improve QoS
35
FOCUS Instrumentation Phase

• Middleware Annotations
• Middleware developer annotates middleware with
  hooks
• Hooks represented as meta-comments, i.e.,
  opaque to compilers

ACE_Reactor_Impl TAO_Advanced_Resource_Factory
allocate_reactor_impl (void) const
ACE_Reactor_Impl impl 0 //_at__at_
TAO_REACTOR_SPL_COMMENT_HOOK_START switch
(this-reactor_type_)
//_at__at_ TAO_REACTOR_SPL_COMMENT_HOOK_END
11
12
rce.cpp" 13 14
TAO_REACTOR_SPL_COMMENT_HOOK_START
15 TAO_REACTOR_SPL
_COMMENT_HOOK_END 16
17 18 23 ....

Annotations help identify join points relieve
FOCUS from Implementing a full fledged language
parser
36
Lessons Learned (2/2)

• Specializations have potential to improve task
  schedulability
• Specializations help high priority tasks finish
  ahead of their time to complete
• Tasks with priority 50 finish early, increasing
  time available for scheduling priorities with 35
  10
• Specializations can benefit both hard real-time
  soft softer real-time tasks

Specializations increase slack in the system

• Adaptation with specializations can adversely
  affect QoS
• Specializations do not consider any form of
  recovery if invariance assumptions fail
• Adaptation requires loading general-purpose code,
  add checks along request processing path
  increases jitter for DRE systems

• QoS improvements are scenario specific
• All our specializations improve path latencies
  considerably than end-to-end latency More the
  specialized code path is traversed, greater is
  QoS improvement

37
Applicability of Specialization Approaches
Dim 1 Specification Imposed generality
Dim 2 Framework generality
Dim 3 Deployment generality

• Resolving Specification Imposed Generality
• Applicable to standards compliant CORBA
  middleware
• Layer folding specialization ? CORBA
  demultiplexing Other layered demultiplexing
  approaches

• Avoiding (de)marshaling checks ? middleware
  standards such as J2EE, .NET that target
  heterogeneous deployment

4
3
2
Little endian
Big endian
1
38
Applicability of Specialization Approaches
Dim 1 Specification Imposed generality
Dim 2 Framework generality
Dim 3 Deployment generality
Resolving Framework generality ?Specialize design
pattern
Reactor Framework specialization ? Bridge
pattern Protocol framework specialization ?
Template method pattern

• Techniques to distributed systems that apply
  these patterns (www.dre.vanderbilt.edu/POSA)

Thread-pool, Single-threaded, Thread-per
connection
TCP/IP, VME, SCTP, SHMIOP
39
Applicability of Specialization Approaches
Dim 1 Specification Imposed generality
Dim 2 Framework generality
Dim 3 Deployment generality

• Dimension 3 Resolving Deployment generality
• Host Infrastructure middleware (ACE, JVMs)
  targeting heterogeneous OS, compiler hardware
  characteristics
• Systems concerned with QoS

gcc 3.2 (no exceptions), timesys kernel
Green-hills compiler, vxWorks platform
40
Comparing FOCUS with Related Work

• FOCUS Characteristics
• FOCUS design motivated by template
  code-generation approaches generative
  programming techniques
• Allows annotations to be directly placed in code
  (placed as special comments)
• Annotations explicitly identify variability
  support evolution
• A light-weight specialization approach (does not
  implement a full-blown language parser)