Design and Implementation of Object Oriented Dynamic Programming Languages

1
Design and Implementation of Object Oriented
Dynamic Programming Languages

• Jonathan Bachrach
• Greg Sullivan
• Kostas Arkoudous

2
Who am I?

• Jonathan Bachrach
• PhD in CS neural networks applied to robotic
  control
• Experience in real-time high performance
  electronic music
• Involved in design of Sather and Dylan
• Five years experience writing Dylans runtime and
  compiler at Harlequin

3
The Seminar

• 6.894, 26-311, 1-2.30, 3-0-9, 3 EDPs
• Theme
• Proto running example
• Lectures
• Readings Presentations
• Panels
• Assignments
• Projects

4
Today

• Taste of the Seminar
• Evolutionary Programming
• Proto
• Language Design in the New Millenium
• Language Implementation Techniques
• Seminar Details

5
Tomorrow

• Motivation and overview of OODL
• Implementation perspective
• Gregs favorite topics

6
Language Renaissance

• Perl
• Python
• MetaHTML
• PHP
• TCL
• Cecil
• C
• Java

• JavaScript
• Sather
• Rebol
• Curl
• Dylan
• Isis
• Limbo

7
The Stage

• Increasing Demands for Quick Time to Market
• Moores Law for Hardware but is software keeping
  up?
• Most Time Spent after initial deployment
• Complex environments
• Distributed
• Agents
• Real world
• Continuous, non-deterministic, dynamic inputs

8
Conventional Programming

• Assume user knows exactly what they want before
  programming
• Assume that specifications dont change over time
• Problem Forces premature decisions and
  implementation effort
• Example Java Class Centric Methods

9
Evolutionary Programming

• Need to play with prototypes before knowing what
  you really want
• Support rapid prototyping
• Smooth transition to delivery
• Requirements change all the time
• Late binding allowing both developers and program
  alike to update behavior

10
Language DesignUser Oriented Goals

• Perlis A language that doesn't affect the way
  you think about programming, is not worth
  knowing.
• What user-oriented goals would you suggest would
  result in the best language?

11
Proto

• Goals
• Examples
• Relation
• Definition
• State

12
Proto Hello World

• (format out hello world)

13
Proto Goals

• Simple
• Productive
• Powerful
• Extensible
• Dynamic
• Efficient
• Real-time

• Teaching and research vehicle
• Electronic music is domain to keep it honest

14
Proto Ancestors

• Language Design is Difficult
• Leverage proven ideas
• Make progress in selective directions
• Ancestors
• Scheme
• Cecil
• Dylan

15
Proto ltgt Scheme

• Concise naming
• Procedural macros
• Objects all the way

• Long-winded naming
• Rewrite rule only
• Only records

16
Proto ltgt Cecil

• Prefix syntax
• Scheme inspired special forms

• Infix syntax
• Smalltalk inspired special forms

17
Proto ltgt Dylan

• Prefix syntax
• Prototype-based
• Procedural macros
• Rationalized collection protocol / hierarchy
• Always open
• Predicate types

• Infix syntax
• Class-based
• Rewrite-rule only
• Conflated collection protocol / hierarchy
• Sealing
• Fixed set of types

18
Object Orientation

• Assume you know OO basics
• Motivations
• Abstraction
• Reuse
• Extensibility

19
Prototype-based OO

• Simplified object model
• No classes
• Cloning basic operation for instance and
  prototype creation
• Prototypes are special objects that serve as
  classes
• Inheritance follows cloned-from relation

20
Proto OO MM

• (dv ltpointgt (isa ltanygt))
• (slot ltpointgt (point-x ltintgt) 0)
• (slot ltpointgt (point-y ltintgt) 0)
• (dv p1 (isa ltpointgt))
• (dm ((p1 ltpointgt) (p2 ltpointgt) gt ltpointgt)
• (isa ltpointgt
• (set point-x ( (point-x p1) (point-x p2)))
• (set point-y ( (point-y p1) (point-y p2))))

21
Language DesignUser Goals -- The ilities

• Learnability
• Understandability
• Writability
• Modifiability
• Runnability
• Interoperability

22
Learnability

• Simple
• Small
• Regular
• Gentle learning curve
• Perlis Symmetry is a complexity reducing
  concept seek it everywhere.

23
Proto Learnability

• Simple and Small
• 16 special forms if, seq, set, fun, let, loc,
  lab, fin, dv, dm, dg, isa, slot, ds, ct, quote
• 7 macros try, rep, mif, and, or, select, case
• Gentle Learning Curve
• Graceful transition from functional to
  object-oriented programming
• Perlis Purely applicative languages are poorly
  applicable.

24
Proto Special Forms

• IF (IF ,test ,then ,else)
• SEQ (SEQ ,_at_forms)
• SET (SET ,name ,form) (SET (,name ,_at_args)
  ,form)
• LET (LET ((,var ,init) ) ,_at_body)
• FUN (FUN ,sig ,_at_body)
• LOC (LOC ((,name ,sig ,_at_body) ) ._at_body)
• LAB (LAB ,name ,_at_body)
• FIN (FIN ,protected-form ,_at_cleanup-forms)
• DV (DV ,var ,form)
• DM (DM ,name ,sig ,_at_body)
• DG (DG ,name ,sig)
• ISA (ISA (,_at_parents) ,_at_slot-inits)
• SLOT (SLOT ,owner ,var ,init)
• sig (,_at_vars) (,_at_vars gt ,var)
• var ,name (,name ,type)
• slot-init (SET ,name ,value)

25
Understandability

• Natural notation
• Simple to predict behavior
• Modular
• Models application domain
• Concise

26
Proto Understandability

• Describable by a small interpreter
• Size of interpreter is a measure of complexity of
  language
• Regular syntax
• Debatable whether prefix is natural, but its
  simple, regular and easy to implement

27
Writability

• Expressive features and abstraction mechanisms
• Concise notation
• Domain-specific features and support
• No error-prone features
• Internal correctness checks (e.g., typechecking)
  to avoid errors

28
Tradeoff One Abstraction ltgt Writability

• Abstraction can obscure code
• Example abuse of macros
• Concision can obscure code
• Example APL

29
Tradeoff TwoDomain Specific ltgt Simplicity

• Challenge is to introduce simple domain specific
  features that dont hair up language
• CL has reader macros for introducing new token
  types

30
Proto Error Proneness

• No out of language errors
• At worst all errors will be be caught in language
  at runtime
• At best potential errors such as no applicable
  methods will be caught statically earlier and in
  batch
• Unbiased dispatching and inheritance
• Example Method selection not based on
  lexicographical order as in CLOS

31
Design Principle TwoPlanned Serendipity

• Serendipity
• M-W the faculty or phenomenon of finding
  valuable or agreeable things not sought for
• Orthogonality
• Collection of few independent powerful features
  combinable without restriction
• Consistency

32
Proto Serendipity

• Objects all the way down
• Slots accessed only through calls to generics
• Simple orthogonal special forms
• Expression oriented
• Example
• Exception handling can be built out of a few
  special forms lab, fin, loc,

33
Modifiability

• Minimal redundancy
• Hooks for extensibility included automatically
• No features that make it hard to change code later

34
Proto Extensible Syntax

• Syntactic Abstraction
• Procedural macros
• WSYWIG
• Pattern matching
• Code generation
• Example
• (ds (unless ,test ,_at_body)
• (if (not ,test) (seq ,_at_body)))

35
Proto Multimethods

• Can add methods outside original class
  definition
• (dm jb-print ((x ltnodegt)) )
• (dm jb-print ((x ltstrgt)) )

36
Proto Generic Accessors

• All slot access goes through generic function
  calls
• Can easily redefine these generics without
  affecting client code

37
Runnability

• Features for programmers to control efficiency
• Analyzable by compilers and other tools
• Note this will be a running theme!

38
Tradeoff ThreeRunnability ltgt Simplicity

• Much of a language design can be dedicated to
  providing mechanisms to control efficiency (e.g.,
  sealing in Dylan)
• Obscure algorithms
• Perlis A programming language is low level when
  its programs require attention to the
  irrelevant.

39
Proto Optional Types

• All bindings and parameters can take optional
  types
• Rapid prototype without types
• Add types for documentation and efficiency
• Example
• (dm format (s msg (args )) )
• (dm format ((s ltstreamgt)(msg ltstrgt) (args )) )

40
Proto Pay as You Go

• Dont charge for features not used
• Pay more for features used in more complicated
  ways
• Examples
• Dispatch
• Just function call if method unambiguous from
  argument types
• Otherwise require dynamic method lookup
• Protos bind-exit called lab
• Local exits are set goto
• Non local exits must create a frame and stack
  alloc an exit closure

41
Interoperability

• Portability
• Foreign Language Interface
• Directly or indirectly after mindshare

42
The Rub

• Support for evolutionary programming creates a
  serious challenge for implementors
• Straightforward implementations would exact a
  tremendous performance penalty

43
The Game

• Every Problem in CS can be Solved with another
  Level of Indirection -- ancient proverb
• Every Optimization Problem can be Viewed as
  Removing a Level of Indirection -- jb
• Example Procedures ltgt Inlining

44
Implementation Techniques

• System code techniques
• Often called runtime optimizations
• Turbo charges user code
• Example caching
• User code rewriting techniques
• Often called compile-time optimizations
• Example inlining

45
Implementing Prototypes

• Problem
• No classes only objects
• But objects need descriptions of slots and state
• Would be too expensive for each object to have a
  copy of these descriptions
• A solution
• Create classes called traits on demand
• When you are cloned or
• When slots are added to you
• Objects contain their values and share traits
  through a traits pointer

46
Proto Traits on Demand
47
Implementing Generic Dispatch

• Multimethod dispatch
• Considers all arguments
• Orders methods based on specializer type
  specificity
• Naïve description
• Find applicable methods
• Sort applicable methods
• Call most specific method
• Problem too expensive

48
Dispatch Technique OneDispatch Cache

• Hierarchical cache
• Each level discriminates one argument using
  associative mechanism
• Keys are concrete object-traits
• Values are either
• Cache for remaining arguments or
• Method to call with given arguments
• Problems
• Too many indirections
• generic call method call
• key and value lookups

49
Engine Node Dispatch

• Glenn Burke and myself at Harlequin, Inc. circa
  1996-
• Partial Dispatch Optimizing Dynamically-Dispatche
  d Multimethod Calls with Compile-Time Types and
  Runtime Feedback, 1998
• Shared decision tree built out of executable
  engine nodes
• Incrementally grows trees on demand upon miss
• Engine nodes are executed to perform some action
  typically tail calling another engine node
  eventually tail calling chosen method
• Appropriate engine nodes can be utilized to
  handle monomorphic, polymorphic, and megamorphic
  discrimination cases corresponding to single,
  linear, and table lookup

50
Engine Node Dispatch Picture
Define method \ (x ltigt, y ltigt)
end Define method \ (x ltfgt, y ltfgt)
end Seen (ltigt, ltigt) and (ltfgt, ltfgt) as inputs.
51
Dispatch Technique TwoDecision Tree

• Intelligently number traits with ids
• Children tend to be in contiguous id range
• Label all traits according to most applicable
  method
• Construct decision tree constructing largest
  class id ranges by merging
• Implement with simple numeric operations and JIT
  code generation
• Problem
• Generic call method call
• Lost inlining opportunities

52
Class Numbering
53
Binary Search Tree Picture

• From Chambers and Chen OOPSLA-99

54
Code Rewriting Technique OneInlining Dispatch

• Inline when number of methods is small
• When methods themselves are small
• Example List operations
• Twist Partially inline dispatch when there is a
  spike in the method frequencies
• Example Numeric operations
• Problem Lose downstream type inference
  possibilities because result of call gets merged
  back into all other possibilities

55
Inlining Dispatch Example One

• (dm empty? ((x ltlstgt)) f)
• (dm empty? ((x nil)) t)
• (dm null? ((x ltlstgt))
• (empty? X))
• gt
• (dm null? ((x ltlstgt))
• (if ( x nil) t
• (if (isa? X ltlstgt) f
• (error ))))

56
Inlining Dispatch Example Two

• (dm sum1 (x y)
• ( ( x y) 1))
• gt
• (dm sum1 (x y)
• (let ((t (if (and (isa? x ltintgt) (isa? y
  ltintgt))
• (i x y)
• ( x y))))
• (if (isa? t ltintgt)
• (i t 1)
• ( t 1))))

57
Code Rewriting Technique TwoCode Splitting

• Clone code below typecase so each branch can run
  with tighter types
• Problem potential code explosion

58
Code Splitting Example

• (dm sum1 (x y)
• ( ( x y) 1))
• gt
• (dm sum1 (x y)
• (if (and (isa? x ltintgt) (isa? y ltintgt))
• (i (i x y) 1)
• ( ( x y) 1)))

59
Summary

• Touched on evolutionary programming
• Introduced Proto
• Discussed language design
• Sketched out several implementation techniques to
  gain back performance

60
Todays Reading List

• Dijkstra Goto harmful
• Hoare Hints on PL design
• Steele Growing a Language
• Gabriel Worse is Better
• Chambers Synergies Between Object-Oriented
  Programming Language Design and Implementation
  Research
• Chambers The Cecil Language

• Norvig Adaptive Software
• Cook Interfaces and Specifications for the
  Smalltalk-80 Collection Classes
• XP www.extremeprogramming.com
• Lee Advanced Language Implementation
• Queinnec Lisp In Small Pieces

61
Open Problems

• Extensible enough language to stem need for
  domain specific languages
• Best of both worlds of performance and dynamism
• Simplest combination of language implementation

62
Credits

• Craig Chambers notes on language design for UW
  CSE-505

63
Course

• Seminar style format
• Encourage discussion
• Identify and make progress towards the solution
  of open problems
• Course mostly not about language design
• Will use proto as a point of discussion
• Will talk about language design implications
  along the way

64
Prerequisites

• 6.001 (SICP)
• 6.035 (Computer Language Engineering)
• 6.821 (Programming Languages) preferred
• Or permission of instructor

65
Lectures

• Intro I II
• Interpreters and VMs
• Objects and Types
• Runtime Object Systems
• Reflection
• Memory Management

• Macros
• Type Inference
• OO Optimizations
• Partial Evaluation
• Dynamic Compilation
• Proof-Based Compilation
• Practical Aspects

66
Presentation Talking Points

• Enumerate design parameters
• Identify open problems and make progress towards
  their solution
• Identify and understand tradeoffs
• Note language design implications

67
Paper Presentations

• Each student will present one or two papers
  judged important to the course
• Each presentation will be about a half hour
• A reading list is available on the course web
  site
• Other research can be presented at instructors
  discretion

• Three slots in the following categories
• Language Design, Interpreters VMs
• Types and Object Systems, Reflection, and Memory
  Management
• Macros, Type Inference, and OO Optimizations,
  Partial Evaluation and Dynamic Compilation

68
Panels

• Local experts
• System, Compiler, Language Design
• Positions
• Open Problems
• Secret Tricks Revealed
• Q A

69
Assignments

• Small one week projects
• Metacircular interpreter
• Simple VM
• Fast isa?
• Fast method dispatch
• Slot layout

• in Proto such as
• Simple GC
• Type inferencer
• Specialization
• Feedback guided dispatch
• Profile guided inlining

70
Project

• Substantial project involving original language
  design and/or implementation
• Produce a design and project plan
• Working implementation
• Present an overview of their project
• 10 page write-up

71
Grades

• Will be based on the deliverables and class
  participation

72
First Assignment

• Survey
• Due by Thursday