Lisp:%20Where%20Do%20We%20Come%20From?%20What%20Are%20We?%20%20Where%20Are%20We%20Going?%20Peter%20Norvig%20NASA%20Ames%20Research%20Center

1
LispWhere Do We Come From? What Are We?
Where Are We Going?Peter NorvigNASA Ames
Research Center
Paul Gauguin, 1898
2
The answer is?(and the Y combinator)
3
Where Do We Come From? (1958-1989 the first 30
years)

• John McCarthy
• Notation, Functional Programming, Statements as
  Expressions, Incremental Compilation
• MIT/Stanford/CMU/Xerox
• Language of choice for many elite programmers
• Language of Artificial Intelligence
• Lisp Machines
• Tool of choice A religion? Commercialization
• Lucid/Franz/Harlequin/Digitool/...
• Many of benefits to stock HW (without religion)

4
What Are We?(1990s the last ten years)

• Groups and Fields can be defined in terms of
• Shared Community
• Shared Tools
• Shared Literature
• Ecological Niche

5
We are a Shared Community

• But is the community healthy?
• Defensive stance (like Apple or Linux)
• Minority, with some zealots, past-looking
• Numbers holding steady(?)
• But the world is consolidating on C/Java
• See http//www.norvig.com/Lisp-retro.html
• C/Java way out in front
• Lisp comparable to Perl on Books, Usenet, URLs
• But Lisp far behind in Job postings

6
We Have a Set of Shared Tools

• Interactive RAD
• Garbage Collection
• Everything is an Object
• Higher-order Functions
• Emacs
• Lisp Machines
• Macros, Meta-Programming
• Custom Small Languages
• Generic Functions, Informal Interfaces
• Mostly Functional Programming

7
Our Tools are Under-Appreciated

• Six Lasting Concepts from tPoP
• Simplicity and Clarity L/J2
• Generality L/J1.5
• Evolution L/J3
• Interfaces L/J0.5
• Automation (under-appreciated) L/J10
• (Programs writing other programs, tests, etc.)
• Notation (under-appreciated) L/J5
• (Little Languages)

Amazon.com rank 2,481Mentions of Lisp a few
8
We Have a Shared Literature

• Books, Web
• Good choices
• Source Code
• Lost LM code!!
• Journals and Magazines
• Not influential

9
We Inhabit an Ecological Niche

• Artificial Intelligence
• Recent competition from C, Java, MATLAB, etc.
• Solving the hard problems
• Very competitive niche
• RAD
• Lisp is losing the high ground
• CAD, Education
• But
• Lisp is the most adaptable language ever
• The key to survival is to find a new niche
• In 30 years Lisp will likely be ahead of C/Java
  (but behind something else)

Amazon.com rank 690Mentions of Lisp 0
10
Where Are We Going?(the next 10 to 100 years)

• How to predict the future?
• Look at published theories
• What niche can we inhabit?
• Look at some new ones

11
Worse Is Better(Gabriel 1990)

• Simple is Better
• First provide the right 50 capability
• Users will grow it to 90
• Lisp too right thing, CL too big, Scheme too
  small, large entry barrier (50 of population
  below median)
• Prediction Hard for Lisp to compete, even with
  Win Big strategy

12
Crossing the Chasm(Moore 1991)

• Useful is Better
• Early adopters dont predict mainstream adopters
• Be useful in a vertical industry
• Then move into the mainstream
• Lisp as Tool Too early-adopter
• Prediction Lisp can Help companies cross (Viaweb)

13
Innovators Dilemma(Christensen 1997)

• Cheaper is Better
• Innovation moves up from the low end, often with
  new business model or manufacturing process
• The best companies can lead tech development, and
  still miss out
• Lisp Firms overtaken by inferior C, etc.

14
Internet Time(eVeryone 1998)

• First is Better
• Rush to market, get market share
• Lisp not keeping up
• Prediction need for premier RAD tool Lisp
  techniques could be part of solution

15
Where is Lisp Winning?

• NASA Remote Agent (and various rovers)
• Yahoo Viaweb Shopping (Paul Graham)
• Ascent Datamining Tool
• Nichimen (nee Symbolics) Graphics Systems
• ATT, Lucent, other 24x7 telecom switches
• NASA, Boeing, Airline planning/scheduling
• Reasoning Systems Y2K program analysis tools
• Prototyping by Wizards (many)Answer Where
  There Are Lisp Programmers

16
Where Is Lisp Losing?

• The worst thing we can do is to stand still as
  a community, and that is what is happening. -
  Richard Gabriel (in 1991, and since then?)
• Three strikes and youre out - Cordell Green
  (on LispUnixCategory Theory)
• RAD development as good as Lisp
• And better for traditional GUIs
• Standardization falling far behind
• GUI, HTTP, TCP/IP, Threads, binariesAnswer
  need a 10? or 100? breakthrough

17
What Niche Can We Inhabit?

• The High Ground in 10? or 100? improvements
• Look at three possibilitiesWe means Lisp
  Community Not necessarily Lisp

18
(1) First-class Design Patterns

• Design Patterns are hot
• At least in bookstore for actual production
  use???
• Design Patterns are
• Descriptions of what experienced designers know
• Hints/reminders for choosing classes and methods
• Higher-order abstractions for program
  organization
• To discuss, weigh and record design tradeoffs
• To avoid limitations of implementation language
• Lisp is the best host for first class patterns

19
First-class Design PatternsLevels of
Implementation

• InvisibleSo much a part of language that you
  dont notice(e.g. when class replaced struct in
  C, no more Encapsulated Class pattern)
• InformalDesign pattern in prose refer to by
  name, butMust be implemented from scratch for
  each use
• FormalImplement pattern itself within the
  languageInstantiate/call it for each useUsually
  implemented with macros

20
Applying Lisp toDesign Patterns

• 16 of 23 GOF patterns are either invisible or
  simpler
• First-class types (6) Abstract-Factory,
  Flyweight, Factory-Method, State, Proxy,
  Chain-Of-Responsibility
• First-class functions (4.5) Command, Strategy,
  Template-Method, Visitor, Iterator
• Macros (1.5) Interpreter, Iterator
• Method Combination (2) Mediator, Observer
• Multimethods (1) Builder
• Modules (1) Facade

21
Constructing and DeconstructingPrograms into
Patterns
Construction Grammars Whats X doing Y?
construction
22
(2) Adaptive, Teachable Systems

• Not enough programmers to go around
• But many users, domain experts, critics
• Architect systems in which
• Programmers specify initial architecture, specs
• Users say good or bad (reinforcement
  learning)
• Users can correct answers (supervised learning)
• Users can script new behavior, then improve
• Program learns from many users (statistics)
• Programs have ontology of user goals, situations

23
Adaptable, Teachable SystemsInterfaces vs.
Performance

• Black Box
• Input/Output spec.
• Cant talk about performance
• Open Implementation
• Performance tweaking, T
• Designer picks good tweaks
• Adaptive Software
• Performance feedback, F
• Feedback informs tweaks

24
Adaptable, Teachable SystemsDecomposing Into
the Right Components

• Aspect-Oriented Programming
• Deal with one aspect at a time (performance,
  parallelism, replication, etc.)
• http//www.parc.xerox.com/aop
• Subject-Oriented Programming
• A subject is a collection of classes
• http//www.research.ibm.com/sop
• Intentional Programming
• Dont bother with syntax express intentions
• http//www.research.microsoft.com/research/ip/

25
(3) Self-verifying and Repairing Systems

• Model Checking, other Formal Methods
• Formally prove properties of program
• Helps with concurrency, etc.
• Model-Based Reasoning
• Say what there is, not how to do things with it
• Separate reasoning engines for the what
• Adaptive to changes in environment, system

26
Conclusions

• Lisp has always allowed us to imagine the
  abstractions we want, and then build a program
  with them (see On Lisp)
• Now it is time to imagine the program development
  and use system we really want, and then evolve
  all our programs starting with it
• Even if the resulting system is not in Lisp, its
  users will essentially be Lisp Programmers