Software Cost Estimation

1
Software Cost Estimation

• Seth Bowen
• Samuel Lee
• Lance Titchkosky

2
Outline

• Introduction
• Inputs and Outputs
• Methods of Estimation
• COCOMO
• Conclusion

3
Cost Estimation Is Needed

• 55 of projects over budget
• 24 companies that developed large distributed
  systems (1994)
• 53 of projects cost 189 more than initial
  estimates
• Standish Group of 8,380 projects (1994)

4
Cost Estimation

• An approximate judgment of the costs for a
  project
• Many variables
• Often measured in terms of effort (i.e., person
  months/years)
• Different development environments will determine
  which variables are included in the cost value
• Management costs
• Development costs
• Training costs
• Quality assurance
• Resources

5
Cost Estimation Affects

• Planning and budgeting
• Requirements prioritization
• Schedule
• Resource allocation
• Project management
• Personnel
• Tasks

6
Cost Estimation During the Software Life Cycle

• Cost estimation should be done throughout the
  software life cycle to allow for refinement
• Need effective monitoring and control of the
  software costs to verify and improve accuracy of
  estimates
• At appropriate level of detail
• Gathering data should not be difficult
• Success of a cost estimate method is not
  necessarily the accuracy of the initial
  estimates, but rather the rate at which estimates
  converge to the actual cost

7
Who is the Estimator?

• Someone responsible for the implementation
• Can compare previous projects in organization to
  current project
• Usually experienced
• Someone from outside the organization
• Can provide unbiased estimate
• Tend to use empirical methods of estimation
• Difficulties
• Lack of confidence that a model will outperform
  an expert
• Lack of historical data to calibrate the model

8
General Steps and Remarks

• Establish Plan
• What data should we gather
• Why are we gathering this data
• What do we hope to accomplish
• Do cost estimation for initial requirements
• Decomposition
• Use several methods
• There is no perfect technique
• If get wide variances in methods, then should
  re-evaluate the information used to make estimates

9
General Steps and Remarks (cont.)

• Do re-estimates during life cycle
• Make any required changes to development
• Do a final assessment of cost estimation at the
  end of the project

10
Software Cost Estimation Process

• Definition
• A set of techniques and procedures that is used
  to derive the software cost estimate
• Set of inputs to the process and then the process
  will use these inputs to generate the output

11
Inputs and Outputs to the Estimation Process

• Classical view of software estimation process
  Vigder/Kark

12
Inputs and Outputs to the Estimation Process
(Cont.)

• Actual cost estimation process Vigder/Kark

13
Cost Estimation Accuracy

• To determine how well a cost estimation model
  predicts
• Assessing model performance
• Absolute Error (Epred Eact)
• Percentage Error (Epred Eact) / Eact
• Mean Magnitude of Relative Error

14
Cost Estimation Methods

• Algorithmic (Parametric) model
• Expert Judgment (Expertise Based)
• Top Down
• Bottom Up
• Estimation by Analogy
• Price to Win Estimation

15
Algorithmic (Parametric) Model

• Use of mathematical equations to perform software
  estimation
• Equations are based on theory or historical data
• Use input such as SLOC, number of functions to
  perform and other cost drivers
• Accuracy of model can be improved by calibrating
  the model to the specific environment

16
Algorithmic (Parametric) Model (Cont.)

• Examples
• COCOMO (COnstructive COst MOdel)
• Developed by Boehm in 1981
• Became one of the most popular and most
  transparent cost model
• Mathematical model based on the data from 63
  historical software project
• COCOMO II
• Published in 1995
• To address issue on non-sequential and rapid
  development process models, reengineering, reuse
  driven approaches, object oriented approach etc
• Has three submodels application composition,
  early design and post-architecture

17
Algorithmic (Parametric) Model (Cont.)

• Putnams software life-cycle model (SLIM)
• Developed in the late 1970s
• Based on the Putnams analysis of the life-cycle
  in terms of a so-called Rayleigh distribution of
  project personnel level versus time.
• Quantitative software management developed three
  tools SLIM-Estimate, SLIM-Control and
  SLIM-Metrics.

18
Algorithmic (Parametric) Model (Cont.)

• Advantages
• Generate repeatable estimations
• Easy to modify input data
• Easy to refine and customize formulas
• Objectively calibrated to experience
• Disadvantages
• Unable to deal with exceptional conditions
• Some experience and factors can not be quantified
• Sometimes algorithms may be proprietary

19
Expert Judgment

• Capture the knowledge and experience of the
  practitioners and providing estimates based upon
  all the projects to which the expert
  participated.
• Examples
• Delphi
• Developed by Rand Corporation in 1940 where
  participants are involved in two assessment
  rounds.
• Work Breakdown Structure (WBS)
• A way of organizing project element into a
  hierarchy that simplifies the task of budget
  estimation and control

20
Expert Judgment (Cont.)

• Advantages
• Useful in the absence of quantified, empirical
  data.
• Can factor in differences between past project
  experiences and requirements of the proposed
  project
• Can factor in impacts caused by new technologies,
  applications and languages.
• Disadvantages
• Estimate is only as good experts opinion
• Hard to document the factors used by the experts

21
Top - Down

• Also called Macro Model
• Derived from the global properties of the product
  and then partitioned into various low level
  components
• Example Putnam model

22
Top Down (Cont.)

• Advantages
• Requires minimal project detail
• Usually faster and easier to implement
• Focus on system level activities
• Disadvantages
• Tend to overlook low level components
• No detailed basis

23
Bottom - Up

• Cost of each software components is estimated and
  then combine the results to arrive the total cost
  for the project
• The goal is to construct the estimate of the
  system from the knowledge accumulated about the
  small software components and their interactions
• An example COCOMOs detailed model

24
Bottom Up (Cont.)

• Advantages
• More stable
• More detailed
• Allow each software group to hand an estimate
• Disadvantages
• May overlook system level costs
• More time consuming

25
Estimation by Analogy

• Comparing the proposed project to previously
  completed similar project in the same application
  domain
• Actual data from the completed projects are
  extrapolated
• Can be used either at system or component level

26
Estimation by Analogy (Cont.)

• Advantages
• Based on actual project data
• Disadvantages
• Impossible if no comparable project had been
  tackled in the past.
• How well does the previous project represent this
  one

27
Price to Win Estimation

• Price believed necessary to win the contract
• Advantages
• Often rewarded with the contract
• Disadvantages
• Time and money run out before the job is done

28
COCOMO 81

• COCOMO stands for COnstructive COst MOdel
• It is an open system First published by Dr Barry
  Bohem in 1981
• Worked quite well for projects in the 80s and
  early 90s
• Could estimate results within 20 of the actual
  values 68 of the time

29
COCOMO 81

• COCOMO has three different models (each one
  increasing with detail and accuracy)
• Basic, applied early in a project
• Intermediate, applied after requirements are
  specified.
• Advanced, applied after design is complete
• COCOMO has three different modes
• Organic relatively small software teams
  develop software in a highly familiar, in-house
  environment Bohem
• Embedded operate within tight constraints,
  product is strongly tied to complex of hardware,
  software, regulations, and operational
  procedures Bohem
• Semi-detached intermediate stage somewhere
  between organic and embedded. Usually up to 300
  KDSI

30
COCOMO 81

• COCOMO uses two equations to calculate effort in
  man months (MM) and the number on months
  estimated for project (TDEV)
• MM is based on the number of thousand lines of
  delivered instructions/source (KDSI)
• MM a(KDSI)b EAF
• TDEV c(MM)d
• EAF is the Effort Adjustment Factor derived from
  the Cost Drivers, EAF for the basic model is 1
• The values for a, b, c, and d differ depending on
  which mode you are using

31
COCOMO 81

• A simple example
• Project is a flight control system (mission
  critical) with 310,000 DSI in embedded mode
• Reliability must be very high (RELY1.40).  So we
  can calculate
• Effort 1.403.6(319)1.20 5093 MM
• Schedule 2.5(5093)0.32 38.4 months
• Average Staffing 5093 MM/38.4 months 133 FSP

32
COCOMO II

• Main objectives of COCOMO II
• To develop a software cost and schedule
  estimation model tuned to the life cycle
  practices of the 1990s and 2000s
• To develop software cost database and tool
  support capabilities for continuous model
  improvement
• From Cost Models for Future Software Life Cycle
  Processes COCOMO 2.0," Annals of Software
  Engineering, (1995).

33
COCOMO II

• Has three different models
• The Application Composition Model
• Good for projects built using rapid application
  development tools (GUI-builders etc)
• The Early Design Model
• This model can get rough estimates before the
  entire architecture has been decided
• The Post-Architecture Model
• Most detailed model, used after overall
  architecture has been decided on

34
COCOMO II Differences

• The exponent value b in the effort equation is
  replaced with a variable value based on five
  scale factors rather then constants
• Size of project can be listed as object points,
  function points or source lines of code (SLOC).
• EAF is calculated from seventeen cost drivers
  better suited for today's methods, COCOMO81 has
  fifteen
• A breakage rating has been added to address
  volatility of system

35
COCOMO II Calibration

• For COCOMO II results to be accurate the model
  must be calibrated
• Calibration requires that all cost driver
  parameters be adjusted
• Requires lots of data, usually more then one
  company has
• The plan was to release calibrations each year
  but so far only two calibrations have been done
  (II.1997, II.1998)
• Users can submit data from their own projects to
  be used in future calibrations

36
Importance of Calibration

• Proper calibration is very important
• The original COCOMO II.1997 could estimate within
  20 of the actual values 46 of the time. This
  was based on 83 data points.
• The recalibration for COCOMO II.1998 could
  estimate within 30 of the actual values 75 of
  the time. This was based on 161 data points.

37
Is COCOMO the Best?

• COCOMO is the most popular method however for any
  software cost estimation you should really use
  more then one method
• Best to use another method that differs
  significantly from COCOMO so your project is
  examined from more then one angle
• Even companies that sell COCOMO based products
  recommend using more then one method. Softstar
  (creators of Costar) will even provide you with
  contact information for their competitors
  products

38
COCOMO Conclusions

• COCOMO is the most popular software cost
  estimation method
• Easy to do, small estimates can be done by hand
• USC has a free graphical version available for
  download
• Many different commercial version based on COCOMO
  they supply support and more data, but at a
  price

39
Conclusions

• Project costs are being poorly estimated
• The accuracy of cost estimation has to be
  improved
• Data collection
• Use of tools
• Use several methods of estimation

40
References

• Boehm B., Clark B., Horowitz E., Madachy R.,
  Shelby R., Westland C. (1995). Cost Models for
  Future Software Life Cycle Processes COCOMO 2.0,
  Annals of Software Engineering.
  http//sunset.usc.edu/research/COCOMOII/Docs/stc.p
  df.
• Boehm B., Clark B., Horowitz E., Madachy R.,
  Shelby R., Westland C. (1995). An Overview of the
  COCOMO 2.0 Software Cost Model.
  http//sunset.usc.edu/research/COCOMOII/Docs/stc.p
  df.
• Boehm B., Chulani S., Clark B. (1997).
  Calibration Results of COCOMO II.1997.
  http//sunset.usc.edu/publications/TECHRPTS/1998/u
  sccse98-502/CalPostArch.pdf.
• Boehm B., Chulani S., Clark B. (1997).
  Calibrating the COCOMO II Post Architecture
  Model. http//sunset.usc.edu/Research_Group/Sunita
  /down/calpap.pdf.
• Boehm B., Chulani S., Reifer D., The Rosetta
  Stone Making COCOMO 81 Files Work With COCOMO
  II. http//sunset.usc.edu/publications/TECHRPTS/19
  98/usccse98-516/usccse98-516.pdf.
• Chulani, S. (1998). Software Development Cost
  Estimation Approaches A Survey. IBM Research.
• Humphrey, W.S. (1990). Managing the Software
  Process. Addison-Wesley Publishing Company, New
  York, NY.

41
References

• Hussein, A. (2001). Introduction to Software
  Process Management. University of Calgary,
  Calgary, Canada. http//sern.ucalgary.ca/courses/S
  ENG/621/W01/intro.ppt.
• Londeix, B. (1987). Cost Estimation for Software
  Development. Addison-Wesley Publishing Company,
  New York, NY.
• Pressman, R.S. (2001). Software Engineering A
  Practitioners Approach. McGraw-Hill Higher
  Education, New York, NY.
• Vigder, M. R. and Kark, A. W. (1994). Software
  Cost Estimation and Control. Software Engineering
  Institute for Information Technology.
  http//wwwsel.iit.nrc.ca/seldocs/cpdocs/NRC37116.p
  df.
• Wu, L. (1997). The comparison of the Software
  Cost Estimating Methods. University of Calgary,
  Calgary, Canada. http//sern.ucalgary.ca/courses/s
  eng/621/W97/wul/seng621_11.html.
• Basic COCOMO Software Cost Model.
  http//www.jsc.nasa.gov/bu2/COCOMO.html.
• COCOMO 2, Softstar Systems. http//www.softstarsys
  tems.com/cocomo2.htm.
• Answers to Frequently Asked Questions, Softstar
  Systems. http//www.softstarsystems.com/faq.htm.

42
Questions and Discussion