People, Poetry, and Culture: Reinventing Software Development

1
People, Poetry, and CultureReinventing Software
Development

• Dr. David West
• OTUG Distinguished Lecture Series
• July 2001

2
What shall we talk about?

• Past
• Some history and context
• 1968
• Present
• Contemporary dialogs
• Future
• Presuppositions
• Reinventing Software Development

3
History Plato on Calculation
If you cannot calculate you cannot speculate on
future pleasure and your life will not be that of
a human, but that of an oyster or a
jellyfish. Philebus, 21
4
History - Calculation

• Algebra
• India (terminological / abbrev.), Arabia
  (symbolic)
• Europe (particular reasoning to global reasoning)
• Calculus
• Rome (pebbles, basic operations of arithmetic)
• Newton / Leibniz (differential, integral)
• Vector, tensor, predicate

5
History Binary Arithmetic

• Leibniz 1679
• W.H. Eccles F. W. Jordan flip-flop
• C.E. Wynn-Williams, 1932 thyatron tube binary
  electronic counting machine
• John von Neumann 1945 binary representation
  for computers
• (George Boole Logic)

6
History - Calculators

• Abacus
• 1617 John Napier (Napiers Bones)
• 1623 Wilhelm Shickard (calculating clock)
• 1642 - Blaise Pascal (the Pascaline)
• 1673 Leibniz (four function digital)
• 1885 Burroughs (adding and listing machine)

7
History Aristotles Dream
Suppose every instrument could by command or by
anticipation of need execute its function on its
own suppose (like the carvings of Daedelus or
the figurines of Hephaestus which, the poet says,
could take on a life of their own) that spindles
could weave of their own accord, and plectra
strike the strings of zithers by themselves then
craftsmen would have no need of hand-work, and
masters have no need of slaves.
8
History - Automata

• Hero of Alexandria
• Al Jazzari (sequential automata)
• Jacques de Vaucanson, 1738, (programmable
  sequential automata the Digesting Duck)
• Henri-Louis Jaquet-Droz, 1770, (android musician)

9
History - computer

• Jacquards Loom
• Babbage -- Lady Lovelace
• Hollerith
• Watson and IBM (Harvard Mark I)
• SSEC (conditional branching, subroutines, stored
  program)
• Eniac
• Vannevar Bush, Mauchly, Eckert, von Neuman

10
Recap Essential Historical Ideas

• One Zero and place-value math
• Two algebraic thought
• Three logical thought
• Fourth algebraic and binary logic
• Fifth set theory
• Sixth mathematical logic (predicate calculus)
• Seventh Turing Machine

11
The End of History

• Amazing body of knowledge and understanding
• Of computation
• Of computers
• Of Automata
• Formally defined deterministic mechanical and
  electro-mechanical systems
• A bad case of hubris
• Rationalists thought they had found the answer to
  life, the universe, and everything. Ask the
  Dolphins.

12
Context - Metaphor

• Quine and the fringes of science
• MacCormac
• Diaphor poetry
• Epiphor prose
• Lexical usage
• West
• Paraphor (melded concepts of Kuhns paradigm
  and metaphor) between epiphor and lexical

13
Life Cycle of Metaphors (MacCormac)

• Diaphor poetry
• what if was like
• Epiphor prose
• an atom can be visualized as a small planetary
  system with electrons (planets) orbiting a
  central nucleus (sun), the orbits corresponding
  to
• Lexical usage
• Both the human brain and the electronic computer
  are instances of physical symbol systems, hence
  the brain IS a kind of computer.

14
Off the Beaten Track (West)

• Paraphor
• When a metaphor becomes a paradigm
• Kuhns notion of paradigm
• Lexical use of metaphor
• Metaphor becomes part of cultural perspective
• Values associated with paraphor become the values
  of the culture
• Rational is good emotional is bad
• Control is essential chaos is evil
• Efficiency is a virtue waste is a vice

15
Context Paraphors We Knew and Loved

• Spinning our fates (drop spindle / potters wheel)
• God The Watchmaker (clock)
• Wind and Water (dynamics)
• Blowing off steam (heat engine)
• I compute, therefore I am (digital computer)
• J. David Bolter, Turings Man, Chapel-Hill UNC
  Press, 1984

16
We Hold These Truths

• Philosophers Stone
• Everything it touched turned to gold
• Single idea powerful enough to explain everything
• Rational Formalism Paraphor for modernity

17
Descartes Dream

• November 10, 1619 crawled into a wall stove,
  warmed up, had a vision.
• Unification and illumination of the whole of
  science, even the whole of knowledge, by one and
  the same method the method of reason.
• Mirabilis scientiae fundamenta
• Accept only what is clear in ones own mind
• Split large difficulties into smaller ones
• Argue from simple to complex
• Check, when done

18
Context Leibnizs Dream

• Characteristica universalis - universal method
  whereby all human problems, whether of science,
  law or politics, could be worked our rationally,
  systematically, by logical computation.
• De arte combinatoria a general method in which
  all truths of the reason would be reduced to a
  kind of calculation. At the same time this would
  be a sort of universal language or script, but
  infinitely different from all those projected
  hitherto for the symbols and even the words in
  it would direct the reason and errors, except
  those of fact, would be mere mistakes in
  calculation.

19
Context Leibniz on Binary Math
Omnibus ex hihil ducendis sufficit unum.
(With one, everything can be drawn out of
nothing.)
Saw binary arithmetic in the 64 Hexagrams of the
I Ching (Yin / Yang). Tried to convert the
Emperor of China (an avid mathematician) to
Christianity via arguments that the Unity stood
for God and the Zero for the Void, that the
Supreme Being had drawn every existing thing from
out of the void just as unity and zero suffice to
represent every possible number.
20
Context Bertrand Russel
The most astonishing feature of modern science
is its return to Pythagorism.
the Universe is ruled by the laws of a
physical world whose properties could be
expressed as abstract concepts, the numbers,
themselves developed by the efforts of human
thought which once it had attained the capacity
of extreme abstraction could act on the objects
of this same universe.
21
Context Maurice dOcagne on Calculation
Calculation is important in the domain of theory
as it is in matters of practice. All the
material progress of our civilisation (sic)
derives directly or indirectly from Science, and
the progress of Science itself constantly depends
on calculation. This is not only the case for
the so-called exact sciences but also for
those which were considered experimental or
observational
22
On the Eve of a Conference

• Rational Machines, Rational Theory
• Rational Universe
• Rule governed (classical binary logic coupled
  with mathematics)
• Top-down centralized control (God as the master
  control module)
• Deterministic cause and effect
• Objective reality not dependent on observer(s)
• Rational Software
• Rational Process

23
1968 We have seen the enemy and he is us.
(Pogo)

• NATO Conference on Software Engineering
• Inevitable application of prevailing paraphor to
  software development
• Software artifacts are machines
• Deterministic, cause and effect, formally
  describable
• People are potential machines
• After installation of proper formalisms
  (education/training)
• With appropriate management discipline
• Systems are Cartesian
• Model machine
• Possible to define a formal syntax and grammar
  capable of unambiguous description of the
  implemented machine.

24
Inspiration from an Architect Christopher
Alexander - I
"Today more and more design problems are reaching
insoluble levels of complexity. At the same
time that the problems increase in quantity,
complexity, and difficulty, they also change
faster than before.
25
Inspiration from an Architect Christopher
Alexander - II
It is not hard to see why the introduction of
mathematics into design is likely to make
designers nervous. What they do not realize,
however, is that modern mathematics deals at
least as much with questions of order and
relations as with questions of magnitude
becoming a very powerful tool indeed if it is
used to explore the conceptual order and pattern
which a problem presents to its designer.
26
Inspiration from an Architect Christopher
Alexander - II
logic in the widest sense, refers to something
far more general. It is concerned with the form
of the abstract structures, and is involved the
moment we make pictures of reality and then seek
to manipulate these pictures so that we may look
further into the reality itself. It is the
business of logic to invent purely artificial
structures of elements and relations and then,
because the logic is so tightly drawn, we gain
insight into the reality which was previously
withheld from us.
27
Plato on Decomposition
First, perceiving and bringing together under
one Idea the scattered particulars, so that one
makes clear the thing which he wishes to do...
Second, the separation of the Idea into
classes, by dividing it where the natural joints
are, and not trying to break any part, after the
manner of a bad carver... I love these processes
of division and bringing together ... and if I
think any other man is able to see things that
can naturally be collected into one and divided
into many, him I will follow as if he were a
god. Plato
28
Diderot on Analysis and Synthesis
bring the discoveries and organize them, so
that men may be enlightened, While it is
important to disassemble, to resolve the whole
into its parts, it is also indispensable to
combine, to situate, and to re-assemble these
diverse, separated fragments that are collected.
This creates a new whole, now intelligible to
the inquisitive mind, before putting them in
the mind of others according to its functional
individuality and its characteristic law of
action and development.
29
Engineering - I
" software development today is in a
pre-engineering phase analogous in many respects
to the pre-engineering phases of the now
traditional engineering disciplines." "We do not
seem to be even trying to learn from the history
of the classical engineering fields."
30
Engineering - II
"The essential factor which accounts for the very
substantial difference between design correctness
yesterday and today in classical engineering
fields it seems clear to me, is a scientific and
mathematical foundation and the fact that
practitioners apply that foundation regularly and
as a matter of course in their work."
31
Engineering - III
"The scientific and mathematical foundations
referred to above are provided by Newton's laws
(for the civil and mechanical engineer) and
Maxwell's laws (for electrical engineers). The
comparable foundations for software engineering
in the truest sense of the word engineering is
provided by the work of Floyd, Hoare, Djikstra
and others, but has not yet come into widespread
use in practice. We are still living and working
in the pre-engineering days of software
development."
32
Engineering - IV

• Robert L. Barber - IEEE Annals of the History of
  Computing 194, Oct.-Dec. 1997.
• 29 years since advent of SE and it still is not
  being done?
• Why
• Doesnt work
• Isnt worth the effort to try

33
Engineering V (Winograd)

• There is a given reality out there which we
  come across during software development. By
  analyzing facts of this reality, we obtain
  requirements for the software
• The essential task of the software developer is
  starting from the problem defined in that reality
  to find a correct solution in the form of a
  program system.
• It is possible to separate the production of
  software from its use. Software engineering is
  concerned with the production of software on the
  basis of fixed requirements.

34
Engineering V - (Winograd, cont.)

• Software production is based on models
  representing reality. Models should map reality
  correctly.
• The whole process is largely independent of
  individuals. For one and the same problem,
  different developers should arrive at the same
  results. The developers should be
  interchangeable.

35
Engineering V (Winograd, cont.)

• Communication should be restricted and regulated
  via fixed interfaces. The division of labor can
  be worked out on an ad hoc basis. Subject to
  technical feasibility, any desired parts of the
  production process can be automated.
• The developers responsibility covers only
  proper construction of the product in accordance
  with the requirements specification. Any ethical
  considerations that go beyond this are quite
  separate from the technical aspects of the work.

36
Contemporary Dialogs

• The artificial intelligence of computer
  scientists
• A political pamphlet from the middle ages.
• Greece vs. Rome
• CASE and automated development
• Object Orientation
• eXtremely Agile methodologies
• Information Ecologies / Communities of Practice
• Complexity Theory
• Humanity

37
The AI of Computer Scientists

• Over the holidays, Alan (Newell) and I created
  an intelligent program. Herbert Simon
• Perceptrons cant compute I can mathematically
  prove it. Marvin Minsky (And while I am at it I
  will mathematically prove that hummingbirds cant
  fly.
• Dreyfus and Dreyfus What Computers Cant Do

38
A political pamphlet from the middle ages.

• 1979 - DeMillo, Lipton, Perlis
• Social construction of proofs
• Slide title Djikstras response
• 1989 Fetzer again challenges the efficacy of
  proofs and formal methods, evoking similar
  firestorm
• Heuristics versus Formal Methods
• David Parnas Top Down Design how and why to
  fake it.

39
Greece vs. Rome (Robert L. Glass)

• Greece
• Informal
• Codes
• Uses structured-stuff
• Minimum documentation
• Uses things as tools
• Democratic
• Empirical/inductive
• Intuitive
• Substance

• Rome
• Formal
• Manages project
• Uses Structured Stuff
• Maximum documentation
• Uses people as tools
• Imperial
• Analytical/deductive
• Logical
• Form

40
Process

• Better process better products!
• SEI / CMM, Capers Jones, TQM
• Formal process assumes formal systems
• Empirical studies fail to show either increased
  quality or increased efficiency
• Robert A. Frosch (1969)
• Absence of detailed specification
• Hard part first
• Satisficing solutions
• Rapid iterations

41
Object Orientation

• Language Wars
• Smalltalk is the only true OO language and it
  fails to fully implement object concepts
• Method Wars
• CRC versus RUP
• Patterns or Pattern Languages
• Inspiration from an architect
• Timeless Way of Building versus A Pattern Language

42
Inspiration from an Architect Christopher
Alexander
A building or town will be alive to the extent
that it is governed by the Timeless Way. To seek
the Timeless Way we must first know the Quality
Without a Name. To reach the Quality Without a
Name we must then build a living Pattern Language
as a gate. Once we have the gate, we can pass
through it to the practice of the Timeless
Way. And yet the Timeless Way is not complete,
and will not fully generate the Quality Without a
Name, until we leave the gate behind.
43
XP versus the Agile Alliance
Agile says we value individuals and interactions
over processes and tools. XP says put the
individuals together and have them work this
way. Agile says we value working software over
comprehensive documentation. XP says write
software the first day and every day, test the
first day and every day, refactor the first day
and every day. Agile says We value customer
collaboration over contract negotiation. XP
says Sit the customer down with the programmers
and have them steer the project every single
day. Agile says We value responding to change
over following a plan. XP says Set up intense
and rapid feedback and govern yourselves by
it. The fact is, XP's real viewpoint is so
radical that it can't even be properly expressed
by comparing with those wimpy weightless Agile
preferences.
44
Data is Dead Information is Comatose

• J Arsac on Information (ns)
• Data is what can be stored in a DBMS
• Information is the assembly and presentation of
  data according to formally defined rules
• Information Ecologies (Nardi and ODay)
• The Social Life of Information (John Seeley Brown)

45
J. Arsac on Information
A piece of information is a written formula
capable of imparting knowledge I distinguish
amongst the various sources and reserve the term
information solely for those which I can write
down. Anything that can be measured can be
written down I refuse to give the name
information to ways of acquiring knowledge that I
am unable to describe explicitly.
46
Information Ecology
We define an information ecology to be a system
of people, practices, values, and technologies in
a particular local environment.
System dense interrelationships Diversity
essential to health evolution Coevolution all
members adapt and create in response to
change Keystone species essential role for
smooth function Locality local habitation and
a name.
47
Social Life of Information

• Process
• Discrete
• Well-defined
• Isolable
• Black box
• Longitudinal (sequential)
• Information centric

• Practice
• Diffuse, distributed
• Ill-defined
• Holistic
• Complex
• Lateral and longitudinal
• Experience / knowledge centric

48
Software Development and Reality Construction

• Dreyfus brothers ( Merleau Ponty and Dilthey)
• Winograd (Heidegger, Maturana and Flores)
• Coyne (Derrida)
• Floyd

49
Christiane Floyd, et. al.

• Challenge and refutation of rationalist
  assumptions listed by Winograd.
• No objective reality all reality is socially
  constructed
• Software development is a moral and social
  process, an exploration of a constructed reality
• Artifact and use are inextricably intertwined and
  mutually defining.
• The map is not the territory.
• Communication is essential multidirectional,
  multi-vocal, and context specific.
• Ethics and morality are essential aspects of the
  development tasks not add-ons.

50
Not Complicated. Complex!

• Formal systems are merely complicated
• A complex adaptive system is
• An ensemble of independent agents
• Who interact to create an ecosystem
• Whose interaction is defined by the exchange of
  information.
• Whose individual actions are based on some system
  of internal rules
• Who self-organize in nonlinear ways to produce
  emergent results
• Who exhibit characteristics of both order and
  chaos
• Who evolve over time.

51
Adaptive Development Model LifeCycle - Highsmith

• Does not Manage (bounds, directs, nudges,
  confines)
• Relies on emergent order not imposed order
• Component (outcomes / products) not task based
• Highly iterative
• Unpredictable (deviations guide us to the
  correct solution)

52
Highsmith ADMLC Graphic
53
Recap

• Software Engineering was the last hurrah of the
  Age of Reason.
• Some good results general dissatisfaction
  complete failure in achieving objectives
• Nothing intrinsically wrong with the foundation
  ideas they are just applied to the wrong
  domain.
• Long tradition of loyal opposition
• Time to reinvent software development

54
Reinvention - Scope

• Philosophy
• Values
• Products
• Systems architecture / infrastructure
• Components
• Applications
• Systems
• Process
• Community

55
Philosophy

• Metaphysics
• Objective reality is overrated, and irrelevant at
  the level of applications and systems
• Software development IS reality construction
• Complexity Science (emergence will win out)
• Epistemology
• Context is everything, syntax is convenience
• Ineffable knowledge exists and is essential

56
Values
Open source
Formality, NO!! Discipline, yes.
Man, the measure of all things.
People First!
57
Products

• Systems architecture / infrastructure
• Espalier (David Gelernter)
• Components
• Objects, primitive objects, atomic objects
• Applications
• Affordances
• Systems
• Information ecologies

58
Infrastructure
Node
Virtual Node
Just Add Web.
59
Objects
node
Objectionary
60
Process

• Explore to improve
• Hybrid of eXtreme Programming ADMLC

61
Community

• Software Craft
• Raw materials (objects, infrastructure)
• Hand tools
• Ineffable knowledge
• Supporting Guild (The Order)
• Enculturation
• Apprenticeship
• Community of Practice
• Extended family
• Users, managers, ecology members

62
Becoming

• Pilgrimage
• The Ashram
• Apprenticeship
• Lore and wisdom (reuse)
• Rites-of-Passage
• Novice, enabler, coordinator, listener
• In the World but not of It

63
The Business Case

• Not going to change the world
• Recognized group
• Dorsai (Gordon Dickson)
• Amish lt The Order lt Mormons
• Value
• Order of magnitude (10x) better outcomes
• 2x 5x typical fees
• Assets
• Open source everyone can have access to
  tangible assets
• Ineffable knowledge proprietary assets