More Flexible Software By Favoring Implicit Calls and Implicit Communication

1
More Flexible Software By Favoring Implicit
Callsand Implicit Communication

• by Karl Lieberherr
• Northeastern University, CCIS/PRL
• Joint work with Bryan Chadwick, Ahmed Abdelmeged
  and Therapon Skotiniotis

2
Overview

• Why is traversal abstraction important?
• generalized data abstraction What is AP?
• expression problem
• AP-F and AP-P by example
• Container capacity checking
• Translation tasks
• Implementation Demeter-F for AP-F and Demeter-P
  for AP-P
• Conclusions

3
Important Flexible Software Topics To Which We
Contribute with Demeter

• Generalizing the Data Abstraction Problem
• Contributing to Solving the Expression Problem
• Better Abstractions for Traversals

4
Generalizing Data Abstraction
well-accepted

• Data Abstraction Principle
  the implementation of objects can be changed
  without affecting clients provided the interface
  holds.
• Adaptive Programming (AP) Principle
  the interface of objects can be changed
  without affecting clients provided the Demeter
  interface holds.

5
Expression Problem

• Expression Problem (named by Phil Wadler), which
  separates structure from computations performed
  on that structure, while still enabling modular
  extensions to both the structure and the
  computations.
• We address the Expression Problem by practicing
  structure-shy programming loosely couple
  structure and computations.
• Supports modular extensions to both structure and
  computations.

6
Traversal Abstraction

• Traversal of an object is fundamental to any
  computation on the object.
• Express WhereToGo using a domain-specific
  language for navigation control (a-la XPath).
• But traversals perform diverse computations
  Interpretation and Translation.

7
Lack of Traversal Abstraction

• leads to tight coupling between structure and
  computations.
• hinders using the adaptive programming principle
• hinders solving the expression problem

8
Traversal Abstraction

• A new way of thinking about traversal
  abstraction.
• Explain with Law of Demeter Talk only to your
  friends.
• New Listen only to your friends. Your friends
  satisfy all your needs! Two ways
• AP-F F for Functional communicate through
  arguments.
• AP-P P for interPosition communicate through
  variables.

9
What happens if you dont use traversal
abstraction?

• You write a lot of boiler plate code! Related
  work Scrap Your Boilerplate (SYB) started by
  Laemmel and Peyton-Jones.
• You tightly couple structure (changes frequently)
  and computation (more stable). That is against
  common sense.
• You always deal with the worst-case scenario of
  AP-F or AP-P.

10
Related work in Bern

• Magritte Project (Lukas Renggli (Diplomarbeit),
  Stephane Ducasse, Adrian Kuhn)
• Magritte supports end-user customization of meta
  models. Facilitates creation of infrastructure
  for changed meta models (editors, views,
  persistency tools).
• AP supports customization of meta models
  including the application-specific code.

11
A Quick Intro to AP-F

• An AP-F program is defined by three sets of
  functions which adapt behavior of a predefined
  traversal with a multiple dispatch function. The
  three sets of functions
• transformers
• builders
• augmentors

12
Intro to AP-F

• Think of object computation as moving information
  down (augmentors) and pushing up (builders and
  transformers).
• Allow transformation at each node (transformers).
• Send information down as needed (augmentors).
• Default behavior copy object.

13
Motto of AP-F

• Law of Demeter Talk only to your friends.
• Law of AP-F Listen only to your friends and
  selectively receive information from your
  superiors.
• You might not need all the information you get
  from your friends. You only take what you need
  and what the communication protocol requires.

14
Motto of AP-F
augmentors
A
object graph
Listen to C and D Receive from A
B
builders/ transformers
C
Does NOT Receive from A
Receive from A
D
E
Receive from A
15
DemeterF implements AP-F

• Terminology map AP-F to Java
• transformers -- apply methods
• take one argument
• default identity
• builders combine methods
• several arguments (from your friends)
• default copy
• augmentors update methods
• two arguments (where, what to pass down)
• default identity (return second argument)

16
Class DiagramNested Container Capacity Checking
(CCC)
17
Illustration of combinefor capacity constraint
violation
C
(w1w2w3w4,2)
Cons
(w1w2w3w4,1)
E
(w4,0)
(w1w2w3,1)
Cons
C
(w2w3,1)
Cons
(w1,0)
(w2w3,0)
Cons
(0,0)
E
Empty
(w1,0)
E
(w3,0)
Cons
(w2,0)
both containers (C) violate capacity constraints
(w2,0)
E
Empty
(0,0)
18
Illustration of combinefor capacity constraint
violation
19a
19
C
(w1w2w3w4,2)
Cons
(w1w2w3w4,1)
E
(w4,0)
1
20
2
18a
3
18
3a
13a
(w1w2w3,1)
Cons
after blue arrow combine is active (like after)
17a
C
(w2w3,1)
4
12a
17
Cons
(w1,0)
5
13
14
12
16a
(w2w3,0)
Cons
15a
6
(0,0)
E
Empty
(w1,0)
11a
7a
15
16
11
E
(w3,0)
Cons
(w2,0)
both containers (C) violate capacity constraints
8
7
10a
9a
(w2,0)
E
Empty
(0,0)
9
10
19
Container-specific code for CCC
AP-F Listen through arguments AP-P Listen
through variables
Manual Traversal

• Pair check()
• Pair p items.check()
• return p.add(0,(p.w gt cap)?10)
• Pair combine(Container c, Integer i, Pair wv)
• return wv.add(0,(wv.w gt c.cap)?10)
• void after(Container c)
• myWeight c.myWeight
• totalViolations ((c.capltc.myWeight)?10)

TALK TO YOUR FRIENDS
AP-F
AP-P
LISTEN TO YOUR FRIENDS
20
AP-F for CCC in DemeterF

• class Check extends IDb
• // generic combines for passing up pairs
• Pair combine(Object o, Pair f, Pair r) // pair
  addition
• return f.add(r.w, r.v)
• Pair combine(Object o, Pair f) return f //
  pass it up
• Pair combine(Object o) return Pair.make(0,0)
  // init
• // container capacity checking specific
  combines
• Pair combine(Element e) return
  Pair.make(e.weight,0)
• Pair combine(Container c, Integer i, Pair wv)
• return wv.add(0, (wv.wgtc.cap)?10)

21
Calling the Function Object

• Pair check(Item i)
• return new
• Traversal(new Check()).traverse(i)

22
Class Diagram with Noise for CCC
Noise is very common because the objects are used
for several purposes. 0 outgoing has-a edges 1
outgoing has-a edge 2 outgoing has-a edges
23
Any program change?

• class Check extends IDb
• // generic combines for passing up pairs
• Pair combine(Object o, Pair f, Pair r) // pair
  addition
• return f.add(r.w, r.v)
• Pair combine(Object o, Pair f) return f //
  pass it up
• Pair combine(Object o) return Pair.make(0,0)
  
• // container capacity checking specific
  combines
• Pair combine(Element e) return
  Pair.make(e.weight,0)
• Pair combine(Container c, Integer i, Pair wv)
• return wv.add(0, (wv.wgtc.cap)?10)

24
Any Change needed to CCC program?
default operation 1 send up Pair
combine(Object o, Pair f) return f 2 add
Pair combine(Object o, Pair f, Pair r) return
f.add(r.w, r.v) 3 init Pair combine(Object
o) return Pair.make(0,0) 4 problem specific
25
Benefit of Traversal Abstraction

• NO CHANGE! although meta model changed
  significantly.

26
Quick Intro to AP-P

• AP-P is concerned about achieving the most
  structure-shy yet reusable form of communication
  among different pieces of traversal advice.
• The approach proposed by AP-P for structuring
  communication is to assign a set of interposition
  variables in the visitor class to store
  information related to a specific type of
  objects.

27
Quick Intro to AP-P

• Pieces of traversal advice communicate through
  interposition variables which give access to the
  innermost enclosing object of a given class.
• An important benefit communication can take
  place even across different visitors.
• An interposition variable is a context sensitive
  variable. Its context is defined by the traversal.

28
Intro to AP-P

• Think of computation as traversing the object and
  collecting information in a visitor object.
• Use both regular visitor variables as well as
  interposition variables.
• Interposition variables facilitate implicit
  communication.
• Default behavior traverse object.

29
CCC for AP-P in DemeterP

• class Checker extends Visitor
• public int totalViolations 0
• _at_Interposition (className "Container")
• public int myWeight 0
• public void after(Element e) myWeight
  e.weight
• public void after(Container c)
• myWeight c.myWeight
• totalViolations((c.capltc.myWeight)?10)
• public Pair getReturnValue() return new
  Pair(myWeight, totalViolations)
• Pair check(Item i)
• return cg.traverse(i, "from Container to
  Element",new Checker())

30
Current Traversal Abstraction Restriction

• Abstract traversal code
• non-cyclic objects
• explicit and implicit cyclic objects cause
  problems

31
Module Interface Implementation
DemeterInterface
uses (imports)
module M2
module M1
faceM1 mentM1
faceM2 mentM2 AdapConM1
uses (imports)
faceM1 mentM1
module M1
faceM1 mentM12
faceM1 mentM1
faceM1 mentM13
change implementation
change interface
DemeterInterface(M1)
32
What Hinders Creation of Flexible Software

• Manually following rules like Follow the
  structure, follow the grammar.
• Actively call traversal methods (explicit
  traversal problem).
• Also leads to manual passing of arguments
  (explicit argument problem).

33
Type Unifying information flow
T combine(A a, T t)
A
object graph
T apply(T t) T combine(B b, T t1, T t2)
B
C
D
T apply(T t) T combine(C c, T t1)
T apply(D d)
E
T apply(E e)
34
Type Unifying information flow
T combine(A a, T t)
A
object graph
T apply(T t) T combine(B b, T t1, T t2)
B
C
D
T apply(T t) T combine(C c, T t1)
T apply(D d)
E
T apply(E e)
35
Type Unifying information flow
T combine(A a, T t)
A
object graph
T apply(T t) T combine(B b, T t1, T t2)
B
C
D
T apply(T t) T combine(C c, T t1)
T apply(D d)
E
T apply(E e)
36
Type Unifying information flow
T combine(A a, T t)
A
object graph
T apply(T t) T combine(B b, T t1, T t2)
B
C
D
T apply(T t) T combine(C c, T t1)
T apply(D d)
E
T apply(E e)
37
Type Unifying information flow
T combine(A a, T t)
A
object graph
T apply(T t) T combine(B b, T t1, T t2)
B
C
D
T apply(T t) T combine(C c, T t1)
T apply(D d)
E
T apply(E e)
38
semantics apply-combine expression
Tree ltngt Node ltleftgt Tree
ltrightgt Tree. Node IdentityApply
ltogt Object.

• apply(combine(this,
• apply(combine(left,)),
• apply(combine(right,))
• ))
• Translate tree object into apply-combine
  expression.

39
Still follow the grammar?

• Might have to write customized methods for every
  node. Extreme case.
• Encode local information about structure in
  personalized methods.

40
Simple applicationProgram transformation

• Old
• E Num Var Op Call
• Op Plus Equals.
• Equals .
• New
• E Bool.
• Bool True False.

class BoolTrans extends IDf static E newtrue
Call.parse(( 1 1)), static E newfalse
Call.parse(( 1 0) ) E apply(True t) return
newtrue E apply(False t) return newfalse

apply for transformation of result returned by
builder
41
de Bruijn indices
for later

• Old
• Var Sym.
• Sym Ident.
• New
• Var Sym Addr.
• Addr Integer.

class AddrTrans extends IDf Var apply(Var
var, SymList senv) return new
Addr(senv.lookup(var)) class SymExtender
extends IDa SymList update(Lambda l, SymList
senv) return senv.push(l.formals)
42
Related Work

• Modularity First A Case for Mixing AOP and
  Attribute Grammars, Pavel Avgustinov, Torbjorn
  Ekman, Julian Tibble, Programming Tools Group,
  University of Oxford, AOSD 2008
• SYB home page
• AP home page

43
Conclusions

• Rely more on your friends. Listen to them on the
  agreed upon communication channels like your
  Facebook wall.
• Dont tightly couple structure (volatile) and
  computation (more stable).
• Use AP-F and AP-P ideas as your design notations.
  To execute your designs DemeterF home page.

44
Operation of AP-F
45
AP-F for SDG

• AP-F
• think of generic propagation first
• add specific processing

46
generic propagation in SDG

• push down predicate, retrieve an object
  satisfying predicate.
• push down assignment, combine integer or formula.
• push down predicate.

47
Design space for Demeter-F programs
ID
use builders and transformers
48
managing software development

• Learn by evolving a software product an
  algorithmic financial derivative trading game
• Learn by working in a small team using pair
  programming
• Learn by integrating software from the teams

49
SDG Features

• outcome determined by owner/buyer
• type
• outcome
• raw material (owner)
• finished product (buyer)
• pay (quality)
• outcome determined by environment
• type
• outcome
• pay

50
SDG Features

• outcome determined by owner/buyer
• type T
• set of relations of rank k
• exactly rank k, at most rank k.
• size of set, e.g., 2.
• use a predicate to constrain permitted relations,
  e.g. only OR relations -gt CNF
• outcome
• raw material (owner)
• CSP(T) formula
• finished product (buyer)
• assignment to variables of CSP(T) formula.
• pay (quality)
• satisfaction_ratio
• all-or-nothing satisfaction_ratio gt price10
• outcome determined by environment
• type
• outcome
• pay

51
SDG Features

• synchronous rounds in fixed order p1 p2 p3 p1
  p2 p3 Can buy and offer when it is your turn. On
  each turn, must buy derivative or re-offer all
  existing derivatives at lower price.
• asynchronous Can buy and offer at any time. Time
  interval RV. In every RV, must buy derivative or
  re-offer all existing derivatives at lower price.

52
SDG Features

• Security

53
Domain Design for SDG

• Domain Specific Languages
• Need languages for
• types
• raw material
• finished product
• configuring the game
• game moves
• game rules?

54
Five Deep Questions in ComputingJeannette Wing,
CACM Jan. 2008

• PNP
• What is computable?
• What is intelligence?
• What is information?
• How can we build complex systems simply?

55
Deep?

• Speak to the foundations of the field.
• Understanding and expanding the frontiers of
  computing.
• PNP would have profound practical consequences,
  shaking the foundations of cryptography.
• P!NP profound theoretical consequences.

56
SDG and PNP

• Is there an assignment that satisfies at least k
  clauses is in NP and PNP if this problem has a
  polynomial time algorithm.

57
What is intelligence?

• Can a computer learn to play SDG well without
  being programmed explicitly?
• What are the algorithms that the computer would
  invent?
• Does it require intelligence to play the game
  well?

58
How can we build complex systems simply?

• Can we build systems with simple and elegant
  designs that
• are easy to understand, modify and evolve
• yet still provide the functionality we want today
  and dream of for tomorrow?

59
Programming in DemeterF

• Every programming technology has difficulties
  associated with it.
• Programmers then use style rules to avoid those
  difficulties.
• Examples
• null pointer exceptions in Java. Rule A X.
  -gt A XOpt. XOpt X EmptyX.
• excessive structural dependencies in OO. Rule
  Law of Demeter.

60
DemeterF Example

• when I write
• Integer combine(Object o, Integer n1, Integer n2)
  return n1n2
• I intend that n1 and n2 are returned by an apply
  or combine method. They should not be accidental
  integers that happen to be parts of object o.

61
Possible style ruleCapture Avoidance Rule

• The return type of a combine method should not
  appear in the structure being traversed (Do not
  return a traversed type).
• Example If the structure contains Integer parts
  and the purpose is to add, we could use Float to
  count (see project 3).

62
Implementation of Capture Avoid.Terminal Buffer
Rule with Built-ins

• The terminal buffer rule (TBR) says that all
  terminal classes should be buffered by a class
  that has the terminal class as a part class. TBR
  improves readability of programs.
• violation City ltnamegt String ltzipgt Integer.
• with TBR City CityName Zip. CityName String.
  Zip Integer.

63
Terminal Buffer Rule with Built-ins

• CityName and Zip are made built-in classes so
  that the traversal does not go further.
• Then we are free to use Integer as return type of
  combine.

64
Dispatch function of DemeterF

• chooses most specific method
• more information means more specific
• only list arguments up to last one that is needed