Software Cost Estimation and Quality Management Sommerville Chapters 23, 24, 25

1
Software Cost Estimation and Quality Management
Sommerville - Chapters 23, 24, 25

• Lecture to cover following-
• Estimation Techniques
• Algorithmic Cost Modeling
• Quality Assurance and Standards
• Quality Planning and Control
• Process Improvement

2
Software Cost Estimation

• Predicting the resources required for a software
  development process
• Issues
• Productivity
• Estimation techniques
• Algorithmic cost modelling
• Project duration and staffing

3
Software Cost Components

• Hardware and software costs
• Travel and training costs
• Effort costs (dominant in most projects)
• Salaries of engineers involved in the project
• Social and insurance costs
• Effort costs must take overheads into account
• costs of building, heating, lighting
• costs of networking and communications
• costs of shared facilities (e.g library, staff
  restaurant, etc.)

4
Programmer Productivity

• A measure of the rate at which individual
  engineers involved in software development
  produce software and associated documentation
• Not quality-oriented although quality assurance
  is a factor in productivity assessment
• Essentially, we want to measure useful
  functionality produced per time unit

5
Productivity Measures

• Size related measures based on some output from
  the software process. This may be lines of
  delivered source code, object code instructions,
  etc.
• Function-related measures based on an estimate of
  the functionality of the delivered software.
  Function-points are the best known of this type
  of measure

6
Measurement Problems

• Estimating the size of the measure
• Estimating the total number of programmer months
  which have elapsed
• Estimating contractor productivity (e.g.
  documentation team) and incorporating this
  estimate in overall estimate

7
Lines of Code

• What's a line of code?
• The measure was first proposed when programs were
  typed on cards with one line per card
• How does this correspond to statements as in Java
  which can span several lines or where there can
  be several statements on one line
• What programs should be counted as part of the
  system?
• Assumes linear relationship between system size
  and volume of documentation

8
Productivity Comparisons

• The lower level the language, the more
  productive the programmer
• The same functionality takes more code to
  implement in a lower-level language than in a
  high-level language
• The more verbose the programmer, the higher the
  productivity
• Measures of productivity based on lines of code
  suggest that programmers who write verbose code
  are more productive than programmers who write
  compact code

9
Function Points

• Based on a combination of program characteristics
• external inputs and outputs
• user interactions
• external interfaces
• files used by the system
• A weight is associated with each of these
• The function point count is computed by
  multiplying each raw count by the weight and
  summing all values

10
Function Points

• Function point count modified by complexity of
  the project
• FPs can be used to estimate LOC depending on the
  average number of LOC per FP for a given language
• LOC AVC number of function points
• AVC is a language-dependent factor varying from
  200-300 for assemble language to 2-40 for a 4GL
• FPs are very subjective depend on estimator.
• Automatic function-point counting is impossible

11
Productivity Estimates

• Real-time embedded systems, 40-160 LOC/P-month
• Systems programs , 150-400 LOC/P-month
• Commercial applications, 200-800 LOC/P-month
• In object points, productivity has been measured
  between 4 and 50 object points/month depending on
  tool support and developer capability

12
Estimation Techniques

• There is no simple way to make an accurate
  estimate of the effort required to develop a
  software system
• Initial estimates are based on inadequate
  information in a user requirements definition
• The software may run on unfamiliar computers or
  use new technology
• The people in the project may be unknown
• Project cost estimates may be self-fulfilling
• The estimate defines the budget and the product
  is adjusted to meet the budget

13
Estimation Techniques

• Algorithmic cost modelling
• Expert judgement
• Estimation by analogy
• Parkinson's Law
• Pricing to win

14
Algorithmic Cost Modelling

• Cost is estimated as a mathematical function of
  product, project and process attributes whose
  values are estimated by project managers
• Effort A SizeB M
• A is an organisation-dependent constant, B
  reflects the disproportionate effort for large
  projects and M is a multiplier reflecting
  product, process and people attributes
• Most commonly used product attribute for cost
  estimation is code size
• Most models are basically similar but with
  different values for A, B and M

15
Expert Judgement

• One or more experts in both software development
  and the application domain use their experience
  to predict software costs. Process iterates
  until some consensus is reached.
• Advantages Relatively cheap estimation method.
  Can be accurate if experts have direct experience
  of similar systems
• Disadvantages Very inaccurate if there are no
  experts!

16
Estimation by Analogy

• The cost of a project is computed by comparing
  the project to a similar project in the same
  application domain
• Advantages Accurate if project data available
• Disadvantages Impossible if no comparable
  project has been tackled. Needs systematically
  maintained cost database

17
Parkinson's Law

• The project costs whatever resources are
  available
• Advantages No overspend
• Disadvantages System is usually unfinished

18
Pricing to win

• The project costs whatever the customer has to
  spend on it
• Advantages You get the contract
• Disadvantages The probability that the customer
  gets the system he or she wants is small. Costs
  do not accurately reflect the work required

19
Estimation Methods

• Each method has strengths and weaknesses
• Estimation should be based on several methods
• If these do not return approximately the same
  result, there is insufficient information
  available
• Some action should be taken to find out more in
  order to make more accurate estimates
• Pricing to win is sometimes the only applicable
  method

20
Estimation Accuracy

• The size of a software system can only be known
  accurately when it is finished
• Several factors influence the final size
• Use of COTS and components
• Programming language
• Distribution of system
• As the development process progresses then the
  size estimate becomes more accurate

21
Quality Management

• Managing the quality of the software process and
  products
• Issues
• Quality assurance and standards
• Quality planning
• Quality control
• Software measurement and metrics

22
Software Quality Management

• Concerned with ensuring that the required level
  of quality is achieved in a software product
• Involves defining appropriate quality standards
  and procedures and ensuring that these are
  followed
• Should aim to develop a quality culture where
  quality is seen as everyones responsibility

23
What is Quality?

• Quality, simplistically, means that a product
  should meet its specification
• This is problematical for software systems
• Tension between customer quality requirements
  (efficiency, reliability, etc.) and developer
  quality requirements (maintainability,
  reusability, etc.)
• Some quality requirements are difficult to
  specify in an unambiguous way
• Software specifications are usually incomplete
  and often inconsistent

24
Quality Management Activities

• Quality assurance
• Establish organisational procedures and standards
  for quality
• Quality planning
• Select applicable procedures and standards for a
  particular project and modify these as required
• Quality control
• Ensure that procedures and standards are followed
  by the software development team
• Quality management should be separate from
  project management to ensure independence

25
ISO 9000

• International set of standards for quality
  management
• Applicable to a range of organisations from
  manufacturing to service industries
• ISO 9001 applicable to organisations which
  design, develop and maintain products
• ISO 9001 is a generic model of the quality
  process must be instantiated for each
  organisation

26
ISO 9000 Certification

• Quality standards and procedures should be
  documented in an organisational quality manual
• External body may certify that an organisations
  quality manual conforms to ISO 9000 standards
• Customers are, increasingly, demanding that
  suppliers are ISO 9000 certified

27
Quality Assurance and Standards

• Standards are the key to effective quality
  management
• They may be international, national,
  organizational or project standards
• Product standards define characteristics that all
  components should exhibit e.g. a common
  programming style
• Process standards define how the software process
  should be enacted

28
Importance of Standards

• Encapsulation of best practice- avoids
  repetition of past mistakes
• Framework for quality assurance process - it
  involves checking standard compliance
• Provide continuity - new staff can understand
  the organisation by understand the standards
  applied

29
Process and Product Quality

• The quality of a developed product is influenced
  by the quality of the production process
• Particularly important in software development as
  some product quality attributes are hard to
  assess
• However, there is a very complex and poorly
  understood between software processes and product
  quality

30
Process-based Quality

• Straightforward link between process and product
  in manufactured goods
• More complex for software because
• The application of individual skills and
  experience is particularly important in software
  development
• External factors such as novelty of an
  application or need for an accelerated
  development schedule may impair product quality
• Care must be taken not to impose inappropriate
  process standards

31
Quality Planning

• A quality plan sets out the desired product
  qualities and how these are assessed and define
  the most significant quality attributes
• It should define the quality assessment process
• It should set out which organisational standards
  should be applied and, if necessary, define new
  standards

32
Software Quality Attributes
33
Quality Control

• Checking the software development process to
  ensure that procedures and standards are being
  followed
• Two approaches to quality control
• Quality reviews
• Automated software assessment and software
  measurement

34
Quality Reviews

• The principal method of validating the quality of
  a process or of a product
• Group examined part or all of a process or system
  and its documentation to find potential problems
• There are different types of review with
  different objectives
• Inspections for defect removal (product)
• Reviews for progress assessment(product and
  process)
• Quality reviews (product and standards)

35
Quality Reviews

• A group of people carefully examine part or all
  of a software system and its associated
  documentation.
• Code, designs, specifications, test plans,
  standards, etc. can all be reviewed.
• Software or documents may be 'signed off' at a
  review which signifies that progress to the next
  development stage has been approved by management.

36
Quality Reviews

• Objective is the discovery of system defects and
  inconsistencies
• Any documents produced in the process may be
  reviewed
• Review teams should be relatively small and
  reviews should be fairly short
• Review should be recorded and records maintained

37
Software Measurement and Metrics

• Software measurement is concerned with deriving a
  numeric value for an attribute of a software
  product or process
• This allows for objective comparisons between
  techniques and processes
• Although some companies have introduced
  measurement programmes, the systematic use of
  measurement is still uncommon
• There are few standards in this area

38
Software Metric

• Any type of measurement which relates to a
  software system, process or related documentation
• Lines of code in a program, the Fog index, number
  of person-days required to develop a component
• Allow the software and the software process to be
  quantified
• Measures of the software process or product
• May be used to predict product attributes or to
  control the software process

39
Metrics Assumptions

• A software property can be measured
• The relationship exists between what we can
  measure and what we want to know
• This relationship has been formalized and
  validated
• It may be difficult to relate what can be
  measured to desirable quality attributes

40
Internal and External Attributes
41
Product Metrics

• A quality metric should be a predictor of
  product quality
• Classes of product metric
• Dynamic metrics which are collected by
  measurements made of a program in execution
• Static metrics which are collected by
  measurements made of the system representations
• Dynamic metrics help assess efficiency and
  reliability static metrics help assess
  complexity, understandability and maintainability

42
Dynamic and Static Metrics

• Dynamic metrics are closely related to software
  quality attributes
• It is relatively easy to measure the response
  time of a system (performance attribute) or the
  number of failures (reliability attribute)
• Static metrics have an indirect relationship with
  quality attributes
• You need to try and derive a relationship between
  these metrics and properties such as complexity,
  understandability and maintainability

43
Measurement Analysis

• It is not always obvious what data means
• Analysing collected data is very difficult
• Professional statisticians should be consulted if
  available
• Data analysis must take local circumstances into
  account

44
Process Improvement

• Understanding existing processes
• Introducing process changes to achieve
  organisational objectives which are usually
  focused on quality improvement, cost reduction
  and schedule acceleration
• Most process improvement work so far focused on
  defect reduction. This reflects the increasing
  attention paid by industry to quality
• However, other process attributes can be the
  focus of improvement

45
Process Improvement Stages

• Process analysis
• Model and analyse (quantitatively if possible)
  existing processes
• Improvement identification
• Identify quality, cost or schedule bottlenecks
• Process change introduction
• Modify the process to remove identified
  bottlenecks
• Process change training
• Train staff involved in new process proposals
• Change tuning
• Evolve and improve process improvements

46
Process and Product Quality

• Process quality and product quality are closely
  related
• A good process is usually required to produce a
  good product
• For manufactured goods, process is the principal
  quality determinant
• For design-based activity, other factors are also
  involved especially the capabilities of the
  designers

47
Quality Factors

• For large projects with average capabilities,
  the development process determines product
  quality
• For small projects, the capabilities of the
  developers is the main determinant
• The development technology is particularly
  significant for small projects
• In all cases, if an unrealistic schedule is
  imposed then product quality will suffer

48
Process Analysis and Modelling

• Process analysis
• The study of existing processes to understand the
  relationships between parts of the process and to
  compare them with other processes
• Process modelling
• The documentation of a process which records the
  tasks, the roles and the entities used
• Process models may be presented from different
  perspectives

49
Process Analysis and Modelling

• Study an existing process to understand its
  activities
• Produce an abstract model of the process. You
  should normally represent this graphically.
  Several different views (e.g. activities,
  deliverables, etc.) may be required
• Analyse the model to discover process problems.
  Involves discussing activities with stakeholders

50
Process Analysis Techniques

• Published process models and process standards
• It is always best to start process analysis with
  an existing model. People then may extend and
  change this.
• Questionnaires and interviews
• Must be carefully designed. Participants may tell
  you what they think you want to hear
• Ethnographic analysis
• Involves assimilating process knowledge by
  observation

51
The SEI Process Maturity Model

52
Maturity Model Levels

• Initial
• Essentially uncontrolled
• Repeatable
• Product management procedures defined and used
• Defined
• Process management procedures and strategies
  defined and used
• Managed
• Quality management strategies defined and used
• Optimising
• Process improvement strategies defined and used

53
Key Process Areas
54
SEI Model Problems

• It focuses on project management rather than
  product development.
• It ignores the use of technologies such as rapid
  prototyping.
• It does not incorporate risk analysis as a key
  process area
• It does not define its domain of applicability

55
The CMM and ISO 9000

• There is a clear correlation between the key
  processes in the CMM and the quality management
  processes in ISO 9000
• The CMM is more detailed and prescriptive and
  includes a framework for improvement
• Organisations rated as level 2 in the CMM are
  likely to be ISO 9000 compliant

56
Process Classification

• Informal
• No detailed process model. Development team chose
  their own way of working
• Managed
• Defined process model which drives the
  development process
• Methodical
• Processes supported by some development method
  such as HOOD
• Supported
• Processes supported by automated CASE tools

57
Process Applicability