Modern Software Engineering Practice COMP37321 Lecture 1: Introduction and Background1

1
Modern Software Engineering Practice
COMP37321Lecture 1 Introduction and
Background(1)

• The term Software Engineering was introduced
  during a NATO conference in the late 1960s1,
  so discipline now more mature
• There has been, and still is, much debate, theory
  (and counter theory), many competing methods,
  tools, techniques, much obsolescence of ideas
• The debate has turned away from an original
  emphasis on formal methods to
  process-oriented approaches2.
• Any effective software engineering process must
  be capable of Ensuring that consistency is
  provided in both the artefacts and in the manner
  of their development. (My definition but
  consistent with most others)
• In conventional engineering disciplines, product
  and process partitioning have been a successful
  techniques
• Modern SE techniques are process oriented
• 1 http//homepages.cs.ncl.ac.uk/brian.randell/
  NATO/
• 2 "The Future of Software Engineering" ,
  Anthony Finkelstein (Ed.), ACM Press 2000, Order
  number is 592000-1, ISBN 1-58113-253-0. ACM
  E-Store http//store.acm.org/acmstore

2
Introduction and Background(2)

• Any effective software engineering approach must
  be capable of Ensuring that consistency is
  provided in both the artefacts and in the manner
  of their development.
• More specifically, a set of artefacts (code,
  documents, specifications, etc.) can be produced
  by any team that adheres to a Software
  Engineering process, with some guarantee that
  they will be delivered in a controlled manner
  (e.g. time-scale, cost) with a defined quality.
• Emphasis is on consistency, i.e. repeatability,
  in the same way that a scientific experiment is
  repeatable given the same starting conditions,
  method and ingredients
• n.b. Scientific Method (a four step process)
  relies on experiments (in step 4) that
  demonstrate principles, theories and facts.
• - If an experiment demonstrates a theory once,
  it should also demonstrate it every time the same
  experiment is performed.
• - When this is not the case, either the
  experiment is not correctly performed, or the
  theory is incorrect.
• - If all (known) experiments successfully verify
  a proposed theory, and the theory meets all other
  criteria of scientific objectivity, the theory is
  said to be proved.
• The term quality in the context of software is
  arguably problematic - see later lecture

3
Introduction and Background(3) Formality and
its Significance

• Formal Methods were one of the original
  underpinnings of SE.
• Formal Methods enable a theory to be proven about
  a specification, an implementation to be proved
  to satisfy a specification
• Much research effort was expended by their
  developers and proponents.
• Relatively simply notation techniques, e.g.
  grammars, finite state automata provide
  constructs that enable simple structures to be
  represented formally and embody a method for
  their use
• Formal Methods, e.g. VDM and Z provide
  constructs for defining the properties of
  small-scale structures and also constructs
  suitable for formalising larger-scale and more
  complex structures
• There are many examples of the effective use of
  Formal Methods especially by hardware
  designers3 and also by some software
  developers4.

4
Introduction and Background(4)

• Adoption of Formal Methods by mainstream IT
  software developers has been problematic
• Problems with their adoption have centred upon
• Lack of adequate tools, e.g. tools to prove a
  theory in a specification, to prove that an
  implementation satisfies a specification, to
  translate automatically from a specification into
  an implementation (i.e. completely translate
  automatically)
• Lack of adequate mathematical sophistication in
  developers.
• Developers can produce software without any
  obligation to prove the properties of the
  resulting system
• Incompatibility with conventional techniques.
• Conventional techniques emphasise the need for
  software validation (suitability for purpose)
  rather than verification, any verification is
  done by testing

5
Introduction and Background(5)

• High costs associated with their use.
• Tool development and provision, training,
  learning curve associated with additional
  representations due to specification, additional
  relationships between specification and
  implementation
• Overselling by their advocates.
• For high-integrity systems it may be necessary
  to prove properties of the specification and the
  implementation which IS possible when using a
  formal method
• For many systems, suitability for purpose and
  correctness due to systematic testing are
  sufficient
• 3http//www.csl.sri.com/papers/c/o/computer96/co
  mputer96.html
• 4http//fme03.isti.cnr.it/hall.pdf

6
Introduction and Background(6) Limitations of
Formality

• An informal statement, e.g. a requirement
  expressed in English, cannot be translated into a
  formal statement by formal means-
• Formal methods can prove that an implementation
  satisfies a formal specification, but they cannot
  prove that a formal specification captures a
  user's intuitive informal understanding of a
  system.
• Thus, formal methods can be used to verify a
  system, but not to validate a system.
• The distinction is that validation shows that a
  product will satisfy its operational mission,
  while verification shows that each step in the
  development satisfies the requirements imposed by
  previous steps5.
• 5 Barry W. Boehm, Software Engineering
  Economics, Prentice- Hall, Inc., 1981.

7
Introduction and Background(7) Formality in
Modern SE

• However, it is now generally agreed that an
  increase in formality provides a basis for
  improving the engineering of software systems.
• Although not themselves widely adopted, Formal
  Methods have influenced modern notations, e.g.
  UML, XML.
• - UML is itself formalised, however, the
  approach originally used 6 has been criticised,
  e.g. in 7, as being
• Difficult for mere UML users to understand
• Ambiguous about the relations between models
  represented in UML (even internally consistent
  models) and the subject that is being modelled
• Lacking a justification that explains why the
  presented set of modelling concepts is necessary
  and sufficient (to represent the scope of UML
  models)
• The latest version, UML 2.08 is designed to
  support MDA (see later lecture on modelling).
• 6http//www.uml.crespim.uha.fr/documentation/ver
  sion1.0/semantics/semantics_c1.html
• 7http//icwww.epfl.ch/publications/documents/IC_
  TECH_REPORT_200344.pdf
• 8 http//www.uml.org

8
Introduction and Background(8) Engineering,
Software and SE.

• What makes Software Engineering difficult?
• Most texts mention the scale and complexity of
  the systems involved, but useful systems are
  built out of simpler components in a modular
  manner so must be able to engineer on
  increasingly smaller scales(my italics)
• Few texts mention the nature of software
  itself, i.e. software is not built from physical
  materials using physical processes, hence unusual
  relationship of SE to other (more mature)
  traditional engineering disciplines.
• Traditional engineering disciplines are generally
  limited by the physical interconnection
  constraints of the building blocks used to
  construct artefacts, and benefit from the
  presence of a set of well understood (within that
  particular engineering discipline) and
  standardised building blocks.

9
Introduction and Background(9)

• Example
• When fabricating semiconductors, the majority of
  connections can only be made in two dimensions
  (at the moment!)
• - A whole degree of freedom (and hence
  complexity) is unavailable.
• Larger-scale structures registers, adders,
  multipliers, etc. can be easily reused between
  designs.
• With software, the nature of the interconnections
  between elements provides an unlimited number of
  degrees of freedom, and hence complexity.
• Also, the lack of truly universal standardisation
  for the building blocks of software (e.g.
  components) provides a barrier to making more
  complex (where complex is synonymous with useful)
  software structures out of simpler, well
  understood elements which will have well-defined
  characteristics.
• Aside Object-orientation provides a basis for
  considerations of the limits of the degrees of
  freedom available to software developers. It has
  been argued that OO introduces symmetries in
  the structure of software written in OO
  languages, e.g. due to the imposition of a class
  construct in class-based OO programming
  languages, in subtyping, in operator overloading
  (both also independently of OO), in double
  dispatch, even in the format of string written in
  a programming language

10
Introduction and Background(10) Aside Continued

• In addition to symmetry such discussions
  introduce the notion of symmetry breaking by OO
  design patterns
• The argument is that an OO design pattern
  breaks a symmetry, e.g. the symmetry due to a
  class
• There are, however, arguably deeper symmetries
  that software exhibits, e.g. the symmetry due
  to the system of types that a programming
  language permits to be legal, i.e. by enforcing
  typing constraints via a type checker
• In this deeper context, a design pattern cannot
  break any symmetry due to a languages type
  system, it merely enables a constraint imposed by
  given organisation of classes to be obviated by
  an alternative (but equivalent) organisation
  which is the design pattern
• Arguably, there are symmetries and hence
  design patterns for each programming language
  paradigm, e.g. the symmetry due to procedure
  function abstractions in imperative procedural
  languages, and hence the design pattern
  program design by recursive descent9
• More on Design Patterns in a later lecture
• http//www.jot.fm/issues/issue_2003_09/article3
  
• 9http//en.wikipedia.org/wiki/Recursive_descent_
  parser

11
Introduction and Background(11)

• Most significant advances in effective modern SE
  practice are due to wider recognition that
  process of software development can be predicted,
  planned, managed and controlled as in any other
  branch of engineering10
• Software execution may be internal to a computer,
  but design process should be visible to all (i.e.
  including all levels of management)
• Transition from analysis of requirements to
  software specification/design is most crucial
  stage
• All parts of specification/design subjected to
  review by most able experienced software
  architects/engineers
• Large implementation teams then share experience
  judgement of best available architects/engineers
  
• Involves inspections, walkthroughs, reviews, etc,
  
• At final stage, i.e. code review, formal
  comments in form of assertions, preconditions,
  post-conditions and invariants
• 10 Hoare. C. A. R. How Did Software Get So
  Reliable Without Proof?
• The use of natural language comments should,
  arguably, be discouraged. Formal comments provide
  verifiable properties expected from the code
  and help the construction of a convincing
  argument that software actually works.

12
Introduction and Background(12)

• Testing is benchmark of reliability in quality
  and control of modern, i.e. conventional,
  production engineering
• Tests are applied as early as possible, at all
  stations in production line
• Tests are designed rigorously to maximise
  likelihood of failure (and to detect faults as
  soon as possible)
• Each test is a test of the methods used to
  produce a product, processes, production lines,
  machine tools, parameter settings, operating
  disciplines
• If a test fails, faulty product(s) are NOT
  repaired (even if only one is faulty)
• Instead, whole production line is is re-examined,
  inspected, adjusted, or even closed until root
  cause of defect is found and eliminated
• In science, a theory is tested similarly, i.e.
  series of most rigorous possible experiments is
  devised that aim explicitly and exclusively to
  disprove the theory
• In SE, (by analogy with conventional engineering
  disciplines and science in general) there must be
  a test strategy in advance and in all possible
  detail at start of project
• Tests must be as severe as possible (to make it
  unlikely an error in design is undetected)
• When software is implemented (and passes all
  tests immediately) unlikely that any inherent
  defect in methods of software construction (or
  systematic lapse in their application) c.f.
  Clean room strategy11
• 11http//en.wikipedia.org/wiki/Harlan_Mills

13
Introduction and Background(13)

• Real value of tests is NOT in detecting bugs
  (c.f. Dijkstra program testing can only reveal
  presence of bugs, never their absence)
• Tests should detect 1) inadequacy in methods of
  development, 2) lack of concentration and skills
  of software designers and implementors

14
Introduction and Background(14) Summary

• SE can be characterised as a relatively
  immature discipline when compared to more
  conventional engineering disciplines.
• The mature aspects of SE are exemplified by
  SEs early underpinnings (especially Formal
  Methods), strong semi-formal methods (e.g. JSD,
  JSP), and most recently the adoption of
  process-oriented approaches which exploit
  formalised (usually graphical) notations, e.g.
  UML.
• Although not widely adopted in mainstream IT
  development, Formal Methods have arguably
  influenced increasing formality, e.g. in
  notations used by mainstream developers today.

15
Introduction and Background(15)

• More generally, it can be argued that what is
  expected of SE has increased significantly since
  the term was first introduced (diversity, scale
  and complexity of problems solutions, rate of
  change in mechanisms, tools and techniques used)
• Most SE texts refer to SE as being concerned with
  large-scale, complex software systems
• But, even small-scale components in conventional
  engineering disciplines are engineered, thus,
  it should be likewise in SE, i.e. even the
  smallest component, e.g. a type definition, a
  variable declarations, a method signature, a
  method body, etc should all be capable of being
  engineered
• Expectations of users of software products have
  also arguably changed fundamentally
• All software now expected to be robust and
  reliable (not just O/S and bundled applications
  e.g. word processor, database, spreadsheet etc).
• Underlying theory and practice of modern SE now
  drawn from different disciplines, utilise a wide
  range of current and developing technologies, so
  effective SE has to be woven together from an
  unusual combination of sources.

16
Introduction and Background(16)

• Best general-purpose techniques of management,
  quality control and safety engineering are
  insufficient (useless?) alone
• Need 1) an agreed understanding of field of
  endeavour 2) common conceptual framework (and
  terminology) for discussing cause and effect,
  action and consequence, etc
• Research in programming methodology (over many
  years, even before SE) seeks to establish a
  conceptual framework and theoretical basis to aid
  systematic derivation (and justification) for
  every design decision
• via a rational and explicable train of reasoning
• Past research in programming methodology has had
  important effect on way software is now
  constructed
• Structured programming (Boe?hm-Jacopini
  theorem) http//delivery.acm.org/10.1145/370000/36
  5646/p366-bohm.pdf?key1365646key22674460911col
  lGUIDEdlGUIDE,ACMCFID10941068CFTOKEN5926917
  6
• Data types and strict type checking
• Information hiding
• Object-orientation (see later lectures)
• The next lecture Software Engineering as a
  Process