IT607 - Software Engineering

1
IT607 - Software Engineering

• Kavi Arya
• KReSIT, IIT-Bombay

2
Beware of the computer!

• From software verticals to embedded systems
• Growing number of critical applications

3
Designer Productivity

• The Mythical Man Month by Frederick Brooks 75
• More designers on team gt lower productivity
  because of increasing communication costs between
  groups
• Consider 1M transistor project- Say, a designer
  has productivity of 5000 transistor/mth- Each
  extra designer gt decrease of 100 transistor/mth
  productivity in group due to comm. costs
• 1 designer 1M/5000 gt 200mth
• 10 designer 1M/(104100) gt 24.3mth
• 25 designer 1M/(252600) gt 15.3mth
• 27 designer 1M/(272400) gt 15.4mth
• Need new design technology to shrink design gap

Source Embedded System Design Frank Vahid/ Tony
Vargis (John Wiley Sons, Inc.2002)
4
Software Chronic Crisis
W. Wayt Gibbs, Software Chronic Crisis,
Scientific American, September 1994

• Studies in the USA have shown
• For every 6 new large-scale software systems put
  into operation, 2 others are cancelled!
• The average software development project over
  shoots its schedule by half (and larger projects
  generally do worse).
• Around 75 of all large systems are operating
  failures i.e. do not function as intended or are
  not used at all. Source of USA figures
  Software Productivity Research

5
The Problem

• 1/4 of large software projects are canceled
• Average software project 50 over cost
• 3/4 of large systems are operating failures
• Software in high demand
• Cell phone 30 kLOC
• 4-speed transmission 20 kLOC
• B-2 Stealth Bomber 3.5 MLOC

6
Current Situation

• However, today the vast majority of code is still
  hand crafted from programming languages by
  artisans using techniques they neither measure
  nor are able to repeat consistently.
• But academics have made some progress in formal
  methods and in instituting software engineering
  degrees.
• Industry has made some progress towards market
  structures and technology supporting reusable
  software parts.
• Even so, a research innovation typically requires
  18 years to become standard industrial practice.
• SEI finding

7
Current trends The decade ahead

• A combination of academic industry and government
  is needed to hoist software development to the
  level of an industrial age engineering discipline
  within the decade (2005)
• As we move into the Information Age, error-free
  software will become the expected norm.
• This will be especially true for embedded
  software in consumer products.
• The amount of s/w in consumer products is
  doubling every 2 years e.g. TV 500Kb, Shaver 2Kb
• Existing problems in attempts to develop
  error-free Real-Time s/w for defense applications
  do not give us confidence (Gilles Kahn of INRIA)

8
Some Solutions

• Progress on Software Process, e.g. the SEIs
  Capability Maturity Model (CMM)
• 5 Level Model - 1 is Chaotic, 2 is Repeatable, 3
  is Defined, 4 is Managed and 5 is Optimising
• But 75 of companies are at Level 1 and 24 are
  at Levels 2 and 3.
• USA Space Shuttle software maintenance is at
  Level 5.

9
Progress on Software Reuse

• There is some work on component based
  development, reusable parts and standardisation
  to support reuse
• But more research is needed before this becomes
  widespread industrial practice.

10
Causes of Failure

• Shifting requirements
• Denver had 20M changes after construction
  began
• New or legacy system dependencies
• Poor specification
• High complexity, coupling
• Large size
• Lack of calendar time
• Insufficient tools and techniques
• Poor management

11
Some Progress

• Shifting requirements Prototype-first
• System dependencies Architectures, processes
• Poor Specification Formal methods
• High complexity Domain analysis, architectures
• Large size Modular decomposition
• Lack of calendar time Processes
• Insufficient tools and techniques More work
• Poor management Books

12
Formal Methods

• The science behind software engineering
• Based on sets, Boolean logic, predicate logic
• Or graph theory, automata theory, probability
  and statistics

13
New Processes

• XP lightweight evolutionary process
• On-site customer, prototypes
• Always a working system, trade time for
  features
• Write test cases first
• Cleanroom Dont let bugs in
• Dont execute code (and maybe dont compile!)
• Independent verification group
• Analyze quality statistically
• Only integrate verified components
• And others

14
Evaluation The Capability Maturity Model

• SEIs Capability Maturity Model (CMM)
• Levels 1 (chaos) to 5 (repeatable, predictable)
• Increased productivity and quality, lower risk
• Understand and fix process problems
• Most organizations are at level 1

15
Does It Work?

• Raytheon went from level 1 to 2 to 3 87 to 92
• 15 projects saved 15M
• 2x productivity, 7.7x ROI
• Motorola 1993 report
• 34 projects assessed at all CMM levels
• Level 5 vs level 1 defect rate 10x lower,
  cycle time 8x shorter, productivity 3x better
• 6.77x ROI
• Also no improvement at level 3, costs are high

16
But

• Companies can fool the rating
• Discourages companies from hard projects
• Doesnt encourage valuable projects
• CMM is a poor predictor for challenging
  projects
• Honeywell (CMM level 5) and QRAS

17
Characteristics of a Good Process

• Should be precisely defined no ambiguity about
  what is to be done, when, how, etc.
• It must be predictable can be repeated in other
  projects with confidence about its outcome
• Predictable with respect to effort, cost
• Project A Web-bawd library applications done by
  3 persons in 4 months
• ? another project B (guest house bookings),
  similar in complexity should also take about 12
  person months.

18
A Good Process

• Predictable for quality with respect to number
  and type of defects, performance,
• Predictable process is said to be under
  statistical control , where actual values are
  close to expected values
• It supports testing and maintainability
• Maintenance by third party
• Follow standards, provide necessary documentation
• This characteristic differentiates between
  prototype and product

19
A Good Process

• Facilitates early detection of and removal of
  defects
• Defects add to project cost
• Late detection/correction is costly
• It should facilitate monitoring and improvement
• Based on feedback
• Permit use of new tools, technologies
• Permit measurements

20
Projects

• Some Ideas Ekalavya Infrastructure (Prof. DB
  Phatak)
• Build Ekalavya core infrastructure using
  software engineering principles
• Core infrastructure 2 projects
• Modules 2 projects
• Some Ideas J to E Simulator (capacity planning
  tool) Prof. Umesh Bellur
• Architect, design , build.
• Some Ideas Affordable ERP System (Prof. Kavi /
  Prof. Bernard)
• Think of an ERP for Kirana stores
• 15M shops threatened by Malls
• Each shop supporting 5-6 people
• Some Ideas Prof. Kavi Arya
• Customer Relationship Management Tool
  (brandable)
• Architect, design , build.

21
References

• Softwares Chronic Crisis by W. Wayt Gibbs 2003
• W. Wayt Gibbs, Software Chronic Crisis,
  Scientific American, September 1994
• Topics in Tools and Environments for a
  Distributed Cooperative Work, Koichiro Ochimizu.
  Japan Adv. Inst. of Science and Technologies
  School of Information Science
• Software Engineering A Practitioners Approach,
  Roger Pressman, McGraw Hill International
  Edition.
• Misc web-based resources

22
Software Applications

• system software
• application software
• engineering/scientific software
• embedded software
• product-line software
• WebApps (Web applications)
• AI software

23
SoftwareNew Categories

• Ubiquitous computingwireless networks
• Netsourcingthe Web as a computing engine
• Open sourcefree source code open to the
  computing community (a blessing, but also a
  potential curse!)
• Also (see Chapter 32)
• Data mining
• Grid computing
• Cognitive machines
• Software for nanotechnologies

24
Legacy Software
Why must it change?

• software must be adapted to meet the needs of new
  computing environments or technology.
• software must be enhanced to implement new
  business requirements.
• software must be extended to make it
  interoperable with other more modern systems or
  databases.
• software must be re-architected to make it viable
  within a network environment.

25
Software Evolution

• The Law of Continuing Change (1974) E-type
  systems must be continually adapted else they
  become progressively less satisfactory.
• The Law of Increasing Complexity (1974) As an
  E-type system evolves its complexity increases
  unless work is done to maintain or reduce it.
• The Law of Self Regulation (1974) The E-type
  system evolution process is self-regulating with
  distribution of product and process measures
  close to normal.
• The Law of Conservation of Organizational
  Stability (1980) The average effective global
  activity rate in an evolving E-type system is
  invariant over product lifetime.
• The Law of Conservation of Familiarity (1980) As
  an E-type system evolves all associated with it,
  developers, sales personnel, users, for example,
  must maintain mastery of its content and behavior
  to achieve satisfactory evolution.
• The Law of Continuing Growth (1980) The
  functional content of E-type systems must be
  continually increased to maintain user
  satisfaction over their lifetime.
• The Law of Declining Quality (1996) The quality
  of E-type systems will appear to be declining
  unless they are rigorously maintained and adapted
  to operational environment changes.
• The Feedback System Law (1996) E-type evolution
  processes constitute multi-level, multi-loop,
  multi-agent feedback systems and must be treated
  as such to achieve significant improvement over
  any reasonable base.

Source Lehman, M., et al, Metrics and Laws of
Software EvolutionThe Nineties View,
Proceedings of the 4th International Software
Metrics Symposium (METRICS '97), IEEE, 1997, can
be downloaded from http//www.ece.utexas.edu/per
ry/work/papers/feast1.pdf
26
A Layered Technology
Software Engineering
tools
methods
process model
a quality focus
27
A Process Framework
Process framework Framework activities work
tasks work products milestones deliverables QA
checkpoints Umbrella Activities
28
Framework Activities

• Communication
• Planning
• Modeling
• Analysis of requirements
• Design
• Construction
• Code generation
• Testing
• Deployment

29
Umbrella Activities

• Software project management
• Formal technical reviews
• Software quality assurance
• Software configuration management
• Work product preparation and production
• Reusability management
• Measurement
• Risk management

30
The Process ModelAdaptability

• the framework activities will always be applied
  on every project ... BUT
• the tasks (and degree of rigor) for each activity
  will vary based on
• the type of project
• characteristics of the project
• common sense judgment concurrence of the project
  team

31
The CMMI

• The CMMI defines each process area in terms of
  specific goals and the specific practices
  required to achieve these goals.
• Specific goals establish the characteristics
  that must exist if the activities implied by a
  process area are to be effective.
• Specific practices refine a goal into a set of
  process-related activities.

32
Process Patterns

• Process patterns define a set of activities,
  actions, work tasks, work products and/or related
  behaviors
• A template is used to define a pattern
• Typical examples
• Customer communication (a process activity)
• Analysis (an action)
• Requirements gathering (a process task)
• Reviewing a work product (a process task)
• Design model (a work product)

33
Process Assessment

• The process should be assessed to ensure that it
  meets a set of basic process criteria that have
  been shown to be essential for a successful
  software engineering.
• Many different assessment options are available
• SCAMPI
• CBA IPI
• SPICE
• ISO 90012000

34
Assessment and Improvement
35
Personal Software Process (PSP)

• Recommends five framework activities
• Planning
• High-level design
• High-level design review
• Development
• Postmortem
• stresses the need for each software engineer to
  identify errors early and as important, to
  understand the types of errors

36
Team Software Process (TSP)

• Each project is launched using a script that
  defines the tasks to be accomplished
• Teams are self-directed
• Measurement is encouraged
• Measures are analyzed with the intent of
  improving the team process

37
The Primary Goal of Any Software Process High
Quality
Remember High quality project
timeliness Why? Less rework!
38
Prescriptive Models

• Prescriptive process models advocate an orderly
  approach to software engineering
• That leads to a few questions
• If prescriptive process models strive for
  structure and order, are they inappropriate for a
  software world that thrives on change?
• Yet, if we reject traditional process models (and
  the order they imply) and replace them with
  something less structured, do we make it
  impossible to achieve coordination and coherence
  in software work?

39
The Waterfall Model
40
The Incremental Model
41
The RAD Model
42
Evolutionary Models Prototyping
Quick plan
communication
Modeling Quick design
Deployment delivery feedback
Construction of prototype
43
Evolutionary Models The Spiral
44
Evolutionary Models Concurrent
45
Still Other Process Models

• Component based developmentthe process to apply
  when reuse is a development objective
• Formal methodsemphasizes the mathematical
  specification of requirements
• AOSDprovides a process and methodological
  approach for defining, specifying, designing, and
  constructing aspects
• Unified Processa use-case driven,
  architecture-centric, iterative and incremental
  software process closely aligned with the Unified
  Modeling Language (UML)

46
The Unified Process (UP)
inception
elaboration
inception
47
UP Phases
48
UP Work Products
49
The Manifesto for Agile Software Development

• We are uncovering better ways of developing
  software by doing it and helping others do it.
  Through this work we have come to value
• Individuals and interactions over processes and
  tools
• Working software over comprehensive documentation
  
• Customer collaboration over contract negotiation
• Responding to change over following a plan
• That is, while there is value in the items on the
  right, we value the items on the left more.

Kent Beck et al
50
What is Agility?

• Effective (rapid and adaptive) response to change
• Effective communication among all stakeholders
• Drawing the customer onto the team
• Organizing a team so that it is in control of the
  work performed
• Yielding
• Rapid, incremental delivery of software

51
An Agile Process

• Is driven by customer descriptions of what is
  required (scenarios)
• Recognizes that plans are short-lived
• Develops software iteratively with a heavy
  emphasis on construction activities
• Delivers multiple software increments
• Adapts as changes occur

52
Extreme Programming (XP)

• The most widely used agile process, originally
  proposed by Kent Beck
• XP Planning
• Begins with the creation of user stories
• Agile team assesses each story and assigns a cost
• Stories are grouped to for a deliverable
  increment
• A commitment is made on delivery date
• After the first increment project velocity is
  used to help define subsequent delivery dates for
  other increments

53
Extreme Programming (XP)

• XP Design
• Follows the KIS principle
• Encourage the use of CRC cards (see Chapter 8)
• For difficult design problems, suggests the
  creation of spike solutionsa design prototype
• Encourages refactoringan iterative refinement
  of the internal program design
• XP Coding
• Recommends the construction of a unit test for a
  store before coding commences
• Encourages pair programming
• XP Testing
• All unit tests are executed daily
• Acceptance tests are defined by the customer
  and excuted to assess customer visible
  functionality

54
Extreme Programming (XP)
55
Adaptive Software Development

• Originally proposed by Jim Highsmith
• ASD distinguishing features
• Mission-driven planning
• Component-based focus
• Uses time-boxing (See Chapter 24)
• Explicit consideration of risks
• Emphasizes collaboration for requirements
  gathering
• Emphasizes learning throughout the process

56
Adaptive Software Development
57
Dynamic Systems Development Method

• Promoted by the DSDM Consortium (www.dsdm.org)
• DSDMdistinguishing features
• Similar in most respects to XP and/or ASD
• Nine guiding principles
• Active user involvement is imperative.
• DSDM teams must be empowered to make decisions.
• The focus is on frequent delivery of products.
• Fitness for business purpose is the essential
  criterion for acceptance of deliverables.
• Iterative and incremental development is
  necessary to converge on an accurate business
  solution.
• All changes during development are reversible.
• Requirements are baselined at a high level
• Testing is integrated throughout the life-cycle.

58
Dynamic Systems Development Method
DSDM Life Cycle (with permission of the DSDM
consortium)
59
Scrum

• Originally proposed by Schwaber and Beedle
• Scrumdistinguishing features
• Development work is partitioned into packets
• Testing and documentation are on-going as the
  product is constructed
• Work occurs in sprints and is derived from a
  backlog of existing requirements
• Meetings are very short and sometimes conducted
  without chairs
• demos are delivered to the customer with the
  time-box allocated

60
Scrum
61
Crystal

• Proposed by Cockburn and Highsmith
• Crystaldistinguishing features
• Actually a family of process models that allow
  maneuverability based on problem
  characteristics
• Face-to-face communication is emphasized
• Suggests the use of reflection workshops to
  review the work habits of the team

62
Feature Driven Development

• Originally proposed by Peter Coad et al
• FDDdistinguishing features
• Emphasis is on defining features
• a feature is a client-valued function that can
  be implemented in two weeks or less.
• Uses a feature template
• ltactiongt the ltresultgt ltby for of togt a(n)
  ltobjectgt
• A features list is created and plan by feature
  is conducted
• Design and construction merge in FDD

63
Feature Driven Development
Reprinted with permission of Peter Coad
64
Agile Modeling

• Originally proposed by Scott Ambler
• Suggests a set of agile modeling principles
• Model with a purpose
• Use multiple models
• Travel light
• Content is more important than representation
• Know the models and the tools you use to create
  them
• Adapt locally