FourA Component Adaptation and Assurance DARPA ITS Kickoff Phoenix, AZ August 1999 107

1
Four-A Component Adaptation and
AssuranceDARPA ITS Kick-offPhoenix, AZAugust
1999 (107)

• Bill Scherlis
• CMU
• scherlis_at_cs.cmu.edu, 412-268-8741

2
Four-A Technical Objectives

• Enable Assured Adaptation of Software Systems
• 1. Systematically improve code-level software
  assurance
• Systematically improve tolerance, safety, and
  other properties by transformation
• Uses analyses and component specifications to
  improve the extent of formal assurance of those
  properties
• 2. Preserve software assurance through structural
  adaptation
• Avoid re-verification of tolerance and other
  properties as software components and systems
  evolve.
• Programmer guides adaptation the tool formally
  analyzes and carries out changes, preserving
  assurance where possible.
• 3. Apply to flexible distributed systems
• Systematically introduce and adjust mobility and
  self-adaptation

3
1. Systematic Improvement of Assurance

• Analysis to check and assure tolerance and safety
  properties
• Composable properties (examples)
• Thread-safety and race conditions
• Array bounds, exceptions, extended type safety,
  and other code safety properties
• Cf. Extended Static Checking (ESC)
• Transformations to improve software with respect
  to tolerance and safety properties
• Introduce redundancies
• Insert/remove audits, checks, logging
• Insert techniques for graceful degredation
• Support for iterative development of intrusion
  tolerant systems

4
2. Assurance through Adaptation

• Structural adaptation
• Examples
• API change Data representation change
• Class hierarchy restructuring Signature change
• Introduce self-adaptation Mobility
• Encapsulation
• Systematic reorganization of component
  encapsulation boundaries
• Improve performance
• Enable functional change by localizing and
  isolating related code
• Improve robustness, availability, assurance,
  ....
• Additional examples
• Split into phases / stages
• Cloning to produce specialized variants
• Merging of related functions
• Replication for robustness

5
AWT 1.0 ? AWT 1.1

• ! /bin/sed -f
• updateAWT Update from deprecated 1.0 AWT
  method names to new 1.1 names
• Usage updateAWT old-file gt new-file
• Example updateAWT MyClass.java gt tmp.java
• In this example, tmp.java should eventually
  be renamed to
• MyClass.java once the conversion has been
  looked over for accuracy.
• Bug fixes (1/7/97)
• - fixed erroneous double left paren that
  prevented enable,
• disable, show, and hide from being changed
• - added show(ltexpressiongt) --gt
  setVisible(ltexpressiongt)
• - added enable(ltexpressiongt) --gt
  setEnabled(ltexpressiongt)
• - added getBoundingBox --gt getBounds for
  Polygon class
• - added getClipRect --gt getClipBounds for
  Graphics class
• - removed size --gt getSize because of
  conflict with Vector class

• CheckboxGroup
• s/\ltgetCurrent\( (\)/getSelectedCheckbox\1/g
• s/\ltsetCurrent\( (\)/setSelectedCheckbox\1/g
• Choice, List
• s/\ltaddItem\( (\)/add\1/g
• s/\ltallowsMultipleSelections\(
  (\)/isMultipleMode\1/g
• s/\ltclear\( (\)/removeAll\1/g
• s/\ltcountItems\( (\)/getItemCount\1/g
• s/\ltdelItem\( (\)/remove\1/g
• s/\ltisSelected\( (\)/isIndexSelected\1/g
• s/\ltsetMultipleSelections\( (\)/setMultipleMod
  e\1/g
• Scrollbar
• s/\ltgetLineIncrement\( (\)/getUnitIncrement\1/
  g
• s/\ltsetLineIncrement\( (\)/setUnitIncrement\1/
  g
• s/\ltgetPageIncrement\( (\)/getBlockIncrement\1
  /g
• s/\ltsetPageIncrement\( (\)/setBlockIncrement\1
  /g
• s/\ltgetVisible\( (\)/getVisibleAmount\1/g

6
Assured Software Change

• Structural change in practice
• Costly
• Changes can be distributed throughout a system.
• Complex analysis (program understanding) is
  required.
• Risky
• Invariants and specifications are not present.
• Many code elements may need to be changed.
• Code elements may be inaccessible for analysis or
  change.

7
Assured Software Change

• Structural change in practice
• Avoided
• Why are we stuck with bad structural design
  decisions?
• Decisions are made early
• Consequences are understood late
• They often start wrong and stay wrong
• Why do we tolerate brittleness?
• Code rot persistence of abstractions beyond
  their time.
• Why do commercial APIs accrete?
• Why does ad hoc code persist?
• Why is it so costly to navigate structural
  trade-offs?
• Revise interface and component structure
• Trade-off generality and performance

8
Additional Four-A Challenges

• Make wrapper/sandbox boundaries more flexible
• Extreme cases Sandboxed Java applet vs.
  wants-to-be-trusted ActiveX component
• Enable adaptation to circumstances and policy
• Reduce low-level vulnerabilities
• E.g., threading, pattern compliance, null refs,
  name binding, etc.
• Reduce the complexity and brittleness resulting
  from adaptation
• Assure introduction of tolerance features
• Sustain properties thru (dynamic) change without
  revalidation
• Support as-running configuration management for
  self-adaptive systems
• (CM for self-adaptive and mobile systems)
• Manage through analytic views

9
Four-A Premises

• Work from code level toward spec
• Why Code as ground truth. Snapshot problem.
• Why Legacy code. Exploit and improve partial
  specs.
• Why Manage detail design.
• Use partial information about components in a
  system
• Why Trade secret (COTS). Security. Distributed
  development.
• Cf. whole-program analysis
• Rely on encapsulation, type safety, compositional
  props
• Java, (modified) beans
• Why Scalability. Manipulation soundness.
• Automate administrative change
• Why Appropriate roles for programmers and tools.
  Adoptability.
• Why Tune for performance, security, robustness

10
Four-A Underlying Technologies(Adaptation,
Analysis, Annotation, Auditing)

• Semantics-based program manipulation
• Source and design level
• Structural manipulations
• Run-time manipulations
• Meta-manipulations
• Analysis and models
• OO effects, mutability, uniqueness, aliasing,
  uses, . . .
• Annotation and specification
• Mechanical properties
• Component frameworks
• Semantic patterns
• Analytic views and auditing
• Consistent views
• Tools for assured adaptation of Java components
• Information loss and chain of evidence

11
3. Candidate Application Context

• Flexible distributed systems
• Highly distributed, component-based (Java)
  systems
• Components
• Can migrate, self-adapt, spawn or merge with
  others
• Component framework
• Syntactic and semantic conventions intended to
  achieve compositionality
• E.g., Beans, COM
• Consequences of scale, distribution,
  self-adaptivity
• Increased reliance on compositional properties of
  frameworks
• Increased difficulty of code understanding,
  system snapshot
• Increased need for managed change
• Increased difficulty of sustaining assurance

12
Examples

13
Example Move Field

• Checks
• C is descendent of A.
• If A is interface, f must be public static final.
• Shadowing A and B have no use of ancestral f.
• Unshadowing No f field in B (capture Cs f
  uses).
• D (and other sibs) have no uses of f.
• Initializer code can be reordered, by field type.
• Reordering is acceptable for interleaved
  constructor and field code.
• Actions
• Adjust access tags
• Handle special cases
• Caveats
• Visibility in D and other sibs
• Visibility in Cs subs
• Promises introduced
• Changes in binary compatibility

• Move field f from class C to class A.

Programmers can do this using drag-and-drop.
14
Example Highlighting

• Original code is hard-wired
• Two kinds of errors are highlighted in a
  visualization
• Specific analysis code checks for each error
• Specific looks for highlighting each error type
• New code is designed to distribute responsibility
• How many highlighters
• How they look
• What analyses are highlighted
• How the highlighting is controlled
• From ad hoc (hardwired) to robust (in the
  pattern)
• And back again
• Or sustain the pattern
• (Pattern as expression of invariants)

15
The Ad Hoc Scaffold
highlighting code
16
Abstract two highlighting classes
17
Superclass for common aspects
18
Accept multiple highlighters in View
19
Client specifies highlighting
20
Summary of changes

• A. Abstract highlighting specifics into two new
  classes
• 0. Inline uses of local variables
• 1. Add empty classes
• 2. Move uses of highlighting conditions and
  styles to new classes
• B. Create a highlighting superclass with common
  parts
• 3. Add an empty abstract superclass
• 4. Define abstract method headers
• 5. Make a shadow field for highlighting style in
  each class
• 6. Initialize the field in each constructor
• 7. Pull the fields to the superclass
• 8. Move getter method to superclass
• 9. Make Highlighter a public class
• 10. Make highlighting classes anonymous.

• C. Handle variable set of highlighters in viewer
• 11. Add an array field with the two highlighters
  in it
• 12. Use the array instead of the individual
  highlighters
• 13. Roll two uses into a loop
• 14. Common subexpression elimination of array
  accesses.
• 15. Initialize array in constructor, rather than
  at its declaration
• 16. Convert the array into a vector
• 17. Abstract out common vector addition code
• 18. Change name of method to reflect genericity
• 19. Move highlighter creation to analysis
• 20. Inline field to constructor.
• D. Move choice of highlighters to client

21
New inner classes LimitedHighlighter
EffectsHighlightere
22
Technical Elements

23
The Four-A Technologies

• Semantics-based program manipulation
• Structural manipulations
• Run-time manipulations
• Meta-manipulations
• Analysis and models
• OO effects, mutability, uniqueness, aliasing,
  uses
• Annotation
• Mechanical properties
• Component frameworks
• Semantic patterns
• Analytic views and auditing
• Consistent views
• Tools for assured adaptation of Java components

24
Manipulations

• Manipulations enable systematic structural change
• Trade-off generality and performance
• Sacrifice (or introduce) abstractions
• Reorganize component boundaries
• Introduce or adjust run-time (later stage)
  manipulations
• Managed self-adaptivity
• Manipulations are idiomatic program evolution
  steps
• Precise expression of patterns of evolution or
  refactorings
• Enable rapid/dynamic structural change (fluid
  programming)
• Enable model-based programming (analytic views)
• Tool role
• Programmer Design intent, exploration of
  structural options
• Tool Mechanical details, soundness, design
  record

25
Manipulation Techniques(Examples, 1)

• Boundary movement
• Code relocation (expression, statement, method,
  class)
• Abstract/unfold (method, variable, class)
• Clone (class, method, etc.)
• Data representation change
• Shift
• Idempotency, Projection
• Destructive operations
• Frequency change
• Pass separation
• Tabulation/closure
• Hierarchy restructuring
• Hoist
• Insert
• Split/clone

26
Manipulation Techniques(Examples, 2)

• Staging, specialization, splitting
• (Partial evaluation)
• Merging and generalization
• Pass separation
• Self-adaptation
• (OO method dispatch)
• (RTCG)
• Meta-manipulation
• Polyvariance and domain-tolerance
• Mobility
• Relocation
• Replication

27
Building on Manipulations

• Analytic Views
• The challenge of locality
• Cf. AOP, SOP, etc.
• Analytic View
• Extract thru meaning-preserving manipulations.
• Idea
• Because self-adaptation and mobility are
  introduced and managed using systematic
  manipulations, we can potentially assure that the
  actual executing code base at any moment is an
  analytic view of some baseline code base.
• Snapshots and audit
• Capture information about an as-running system
• Narrow the range of analytic views that model
  that system

28
Specifications for mechanical properties

• Manipulations require analyses
• Example
• Manipulation Reorder code
• Analyses Effects, aliasing (may-equal and
  uniqueness), uses.
• Avoid whole-program analysis
• Analyses require mechanical assertions
• Promises about components and their elements

29
Properties specified by assertions

• Mechanical properties specified (examples)
• Read/write effects in OO systems
• Enable reordering
• Use aliasing and uniqueness information
• Unique references
• Tolerate temporary loss of uniqueness (borrowed)
• Structure declarations
• Precise control over uses
• Mutability
• Promises as a currency of flexibility
• Promises change less frequently than code
• Tools identify potential promises
• Programmer chooses which to offer clients
• Programmer can request specific promises
• Tool manages dependency and validation information

30
The version forest
E.g.,400,000 nodes10,000 versions
Initial version
Each transition represents a manipulation
Latest release
A growing tip in the tree
(abandoned)
Latest snapshot
Shared manipulations
Experimental Configuration
31
Information ManagementThe Internal
Representation (IR)

• Features
• Global name spaces
• Entitles (fluid.ir.IRNode)
• Attributes (fluid.ir.SlotInfo)
• Types
• Versioning
• Several policies
• Possible at cell level
• Configurations
• Dependencies
• Notification
• Tracking
• Conventional wrappers
• Attribute patterns navigable ordered trees,
  etc.
• Collaboration support
• Persistence
• Fine-grained concurrency policy
• Surrogacy

32
Project Management

33
Impact Measures

• Localization and change management
• Comparison with AOP, SOP, and other localization
  methods. What can be localized?
• Performance improvements
• From structural manipulations How much
  improvement?
• From self-adaptation How much improvement?
• Audit events
• How wide is the range of analytic views?
• Case studies
• Usage analysis studies How much work at the
  margin?
• Individual and collaborative
• Design tool to support incremental adoption
• Employ design record and usage instrumentation to
  evaluate usage
• Capture development patterns What can be
  captured?
• Common refactorings

34
Task Schedule

• Year 1
• Tool user interface for analysis and manipulation
• Semantic annotations and analyses for Java beans
• Year 2
• Meta-manipulations for self-adaptation (or other
  domain)
• Assurable component bean concept
• Design record to manage assurance information,
  audit events
• Year 3
• Audit-constrained analytic views
• Quantifiable performance-improvement measures
• Model-based manipulation through views

35
Issues and Risks

• Ongoing formalization of Java, beans, EJB, etc.
• Identified composability failures
• How will these architectures evolve?
• Comprehesiveness measures of manipulation set
• Cf. refactoring library comprehensiveness
• How many manipulations are needed?
• Analytic views
• Will there be a significant improvement in
  locality?
• Extent of trusted infrastructure
• Define aggregate manipulations by composition
• What is the extent of the basis set?
• Meaning preservation
• How precise are meanings (errors?,
  performance?,)?

36
Transition

• Build on mainstream commercial technologies
• Java, beans, etc.
• Build on existing Fluid infrastructure for Java
  analysis, manipulation, engineering process,
  design information mgmt
• Platform (UI, IM, VM, syntax) is also usable for
  other languages
• Commercial interest in Four-A technology
  components
• Make usability/adoptability a priority from the
  outset
• Enable experimentation without high adoption cost
• E.g., gesture-based interface where possible
• Provide potential adopters with evaluation
  measures

37
Reprise Assurance and Adaptation

• Systematic Adaptation
• Interactive programmer-guided program
  manipulation
• Controlled introduction of self-adaptive features
• Operate on specs as well as code
• Structural views. Fine-grained versions. Design
  record mgmt.
• Semantics based
• Supported by program analysis
• Supported by mechanical component specifications
• Why systematic?
• Coordinate many related simultaneous
  administrative changes
• Achieve useful levels of assurance despite
  partial specifications
• Why semantics based?
• Enable assurance
• Enable management/understanding/audit

38
Reprise Key Ideas

• Structural manipulations
• Reduce the difficulty and risk of structural
  change
• Enable systematic introduction of safety and
  tolerance features
• Provide safety and performance
• Meta-manipulations
• Controlled introduction of self-adaptive features
• Analytic views
• Model-based code management
• Audit
• Narrow the range of views
• Component mechanical specifications
• Avoid the need for whole-program analysis in a
  component-based world