CSC444 Software Engineering

1
CSC444Software Engineering
Release Planning
2
Capability Maturity Model

• Developed at Carnegie-Melon Software Engineering
  Institute by Watts Humphrey.
• Classifies an organizations process maturity
  into 5 levels.
• Each level is a group of practices.
• The CMM is a roadmap for process improvement.
• Should have substantially all practices in place
  for a lower level before proceeding to the next
• Can be certified to a certain CMM level
• Some similarities to
• Malcolm Baldrige
• ISO 9000
• Not universally agreed to be a good thing
• But everyone agrees to pretend

3
CMM Levels
4
(No Transcript)
5
Relationship to ISO9000

• ISO 9000
• Set of quality standards
• Subset relate to software development
• In essence
• Must document the process
• Must maintain quality records
• These are auditable to ensure the process is
  being carried out
• The process can be anything

6
Relationship to Top10

• Practices necessary to achieve CMM Level 2
  (Repeatable).
• With enough Level 3 (defined) added to attain
  ISO9000.
• With some Level 4 (quantitatively managed)
  sprinkled in where most effective
• Defect arrival / departure rates
• Estimates versus actuals
• Metrics on process step completion
• Defect attribution

7
Planning

• Most important aspect of CMM Level 2
• Common flaws
• Make no plans
• Make a plan, but dont track it
• Use Microsoft Project

8
Why Plan?

• Not always a good thing
• If no expected date
• If no other expectations (e.g., expected
  functionality)
• Planning can only slow you down
• Required when
• External pressures come to bear on release dates
• Usually only happens a bit later in a software
  companys business evolution
• Not right at the start
• Necessary for crossing the chasm

9
Crossing the Chasm, Geoffrey Moore (1991)
10
Gantt Charts Considered Harmful
11
Planning Essentials

• What are we building?
• By when will it be ready?
• How many people do we have?

Answer these and nothing more not who will be
doing what? nor what are the detailed tasks
required? nor in what order must the tasks be
performed?
12
Implementation Plans

• Once planning is complete can then transform into
  a detailed plan
• E.g., Microsoft Project
• Detailed plan should not contradict the release
  plan
• Not all of the project needs details beyond
• Who do we assign it to
• But some parts do
• These plans may not be necessary
• If no great inter-dependencies that cant be
  worked out as you go
• They hinder change as they are so cumbersome to
  change

13
Of Mice and Men

• The essence of planning is uncertainty
• Plans never go according to plan
• Must embrace change, not close our eyes to it
• Therefore
• Must track the plan always
• Must react quickly to adverse situations
• Must embrace changes in direction
• Must re-plan quickly

14
Internal Changes

• Estimation errors
• Initial estimates contain a significant
  (one-sided) margin of error.
• As plan progresses, variance in estimates lower
• Developer-power leaving the project
• Illness
• Parental leave
• Resignations
• Budget cuts
• Unexpected vacation plans
• Unexpectedly low work hours
• Unexpectedly low productivity

15
External Changes

• New (big) customer desiring new functionality
• Competitor release a product
• Collaboration opportunities
• Acquisitions and mergers
• Sudden changes in customer needs
• E.g., regulatory changes affecting them

16
The Difficult Question

• What are we building?
• May be hard for 1st release
• Subsequent releases will have a big wish list
• Pick the best looking ones, most demanded ones
• Marketing product management decision
• What will make us the most sales?
• By when will it be ready?
• Too soon
• Customers wont be ready for a new release
• Wont want to install
• Wont want to learn
• Wont want to pay for it
• Too late
• Customers will forget about you
• Competitors will pass you
• Foregone revenues
• How many developers?
• Usually fixed for next release

17
A Common Happening

• Often organizations will answer all three
  questions, but not address the difficult one.
• Development management will want to please their
  masters, and will tend to agree to too much in a
  spirit of gung-ho!
• Some managers firmly believe that over-commitment
  is the road to productivity.
• Its a stretch, but well pull it off
• Coders will say it cant be done
• but thats all they ever say.
• Massive sate of denial will set in.
• Everybody will hope for a miracle
• Nobody will accept any blame
• Development management we told you it would be a
  stretch
• Coders we said it could never be done
• Marketing you should have said something earlier
• CEO you all told me it was going fine
• Yourdons Death March.

18
The SolutionGood Planning!

• Does not need to happen this way.
• But need courage and conviction.
• Need common sense
• Is it even feasible to do whats asked by the
  date required?
• Dont give an off-the-cuff answer
• Even if it is obviously impossible
• Put together a plan and demonstrate the facts.

19
Release Planning ReviewThe Product Lifecycle
20
Eliciting Potential Requirements

• Starts with a wish list
• Stated as business requirements
• Features or architectural enhancements

21
A Simple Release Plan
Dates Coding phase Jul.1Oct.1 Beta
availability Nov.1 General availability Dec.1
Capacity days available Fred 31
ecd Lorna 33 ecd Bill 21
ecd total 317 ecd Requirement days
required AR report 14 ecd Dialog
re-design 22 ecd Thread support 87
ecd total 317 ecd Status Capacity 317
effective coder-days Requirement 317 effective
coder-days Delta 0 effective coder days
22
Sizing the Available Resources

• Who can work on the next release?
• Must have skills and familiarity to contribute
• For how long?
• Must count workdays in the coding phase
• Each resource available all that time?
• Subtract estimated vacation
• How dedicated to the next release
• Work factor w
• Converts 8-hour (nominal, arbitrary) days to time
  available to code and unit test during the coding
  phase
• E.g. w0.6 means can dedicate 0.6x8 4.8
  hours/workday
• Accounts for
• Other tasks, sickdays, meetings, weekends, ...
• Range is 0 .. 9, usually 0.6 or so

23
Sizing Potential Requirements

• Cost / Benefit analysis
• Cost financial opportunity person days
• Sizing in ECDs
• Inherent size of the work item
• Who will work on it
• The productivity of that person on that work item
• Ensure units are well-understood (more later)

24
The Capacity Constraint

• After all is done in a release
• Actual Resource Used Sum of Actual Times for
  Features
• This is always true. It is a constraint.
• In planning, knowing that this must work out at
  the end, can estimate both sides and force the
  estimates to be equal
• Resource Estimate Sum of Feature Estimates

25
A Geometric Analogy - Capacity
26
A Geometric Analogy - Requirement
27
A Geometric Analogy - Capacity Constraint
It Must All Fit!
28
Release Planning

• What to build F
• By when to build it T F ? N x T
• Using how many people N
• Need to build an initial plan that respects the
  capacity constraint
• Need to continuously update the plan to maintain
  its adherence to the capacity constraint.

29
Most Common Problem

• Comes from either
• not knowing
• knowing but hoping for the best (Yourdon Death
  March)
• (can happen initially or as we go)

30
Dealing with Issues as they Arise
Developer leaves the team
31
Other Happenings
32
Organizational Issues

• Management must appreciate that software
  development carries with it certain inherent
  risks.
• The business of a software organization is to
  manage and adapt as possibilities continuously
  become reality.
• Ranting and raving is unproductive
• With good data, good managers will make good
  decisions.

33
The Quantitative Capacity Constraint

• Post-Facto, the following relationship must hold.
• But, it requires careful definition.

We define carefully so that we know what it is we
are trying to estimate, and how to compare
actuals against estimates for post-mortem.
34
Number of Workdays T

• The number of full-equivalent working days for
  fork to dcut.
• Subtracts
• Weekends
• Statutory holidays
• Company Days
• Subtracts anything we know in advance that nobody
  is expected to work.

35
Developer Power N

• The average number of dedicated developers per
  workday working during the T-day period.
• Dedicated Developer?

36
Work Time Dedicated Time

• Work Time or Body Time
• Defined as 8 hours per workday
• Excludes weekends, stat. holidays, vacation
  entitlement.
• E.g., 9-to-6 with 1 hour for lunch.
• Dedicated Time
• Uninterrupted hour equivalents.
• Time dedicated to adding new features to the
  release.
• Uninterrupted Time
• 4 hrs with 30 min. of constant interruptions
• Not 3.5 hrs of dedicated uninterrupted time
  more like 2
• 2 hrs with NO interruptions at all

37
Examples of Dedicated Losses

• Maintenance (tracking down and fixing defects) on
  previous releases
• Other simultaneous projects
• Team-leader duties ( helping others)
• Meetings
• Training
• Unexpected, non-made-up days off (e.g., sick
  days)
• Sales/marketing support
• Loss of flow due to interruptions

38
Measuring N

• Assume each developer understands the concept of
  a dedicated uninterrupted hour.
• Get each of the n developers to record how many
  dedicated uninterrupted hours they spent in total
  during the coding phase.
• hi is whats in the time tracking system for the
  ith developer.

39
Attributing N

• di is the number of days available during the
  coding phase
• vi is the number of vacation days they took
  during the coding phase
• hi is as before
• Substitute to get back to

40
Example

• Bob called in sick for 2 days accounted for in h
• Bob took an afternoon off, but worked on the
  weekend to make up for it accounted for in h

41
Features
fk dedicated hours / 8 it took to code the kth
feature.
42
Post-Mortem

• Imagine a time-tracking system that could track
• hi,k,d
• dedicated (uninterrupted) hours spent
• by the ith developer
• on the dth day
• doing coding work on the kth feature
• each such quantum would appear on both sides of
  F N x T constraining them to be equal.
• See section 5.10 in book for proof.

43
Example Release Plan

• Sample Deterministic Release Plan

44
The Stochastic Capacity Constraint
45
Estimates

• Estimates are never 100 certain
• E.g, if we estimate a feature at 20 ECDs
• Not saying will be done in 20 ECDs
• But then what are we saying?
• Are we confident in it?
• Is it optimistic?
• Is it pessimistic?
• A quantity whose value depends upon unknowns (or
  upon random chance) is called a stochastic
  variable
• Release planning contains many such stochastic
  variables.

46
Confidence Intervals

• Say we toss a fair coin 5000 times
• We expect it to come up heads ½ the time 2500
  times or so
• Exactly 2500?
• Chance is only 1.1
• 2500?
• Chance is 50
• If we repeat this experiment over and over again
  (tossing a coin 5000 times), on average ½ the
  time it will be more, ½ the time less.
• 2530?
• Chance is 80
• 2550?
• Chance is 92
• These (50, 80, 92) are called confidence
  intervals
• With 80 confidence we can say that the number of
  heads will be less than 2530.

47
Stochastic Variables

• Consider the work factor of a coder, w.
• When estimating in advance, w is a stochastic
  variable.
• Stochastic variables are described by statistical
  distributions
• A statistical distribution will tell you
• For any range of w
• The probability of w being within that range
• Can be described completely with a probability
  density function.
• X-axis all possible values of the stochastic
  variable
• Y-axis numbers gt 0
• The probability that the stochastic variables
  lies between two values a and b is given by the
  area under the p.d.f. between a and b.

48
PDF for w

• Probability that 0.5 lt w lt 0.7 66
• Looks to be fairly accurate.
• Has a finite probability of being 0
• Has not much chance of being much greater than
  1.2 or so
• Drawing such a curve is the only real way of
  describing a stochastic variable mathematically.

49
Parameterized Distributions

• So, Bill, heres a piece of paper, could you
  please draw me a p.d.f. for your work factor?
• Nobody knows the distribution to this level of
  accuracy
• Very hard to work with mathematically
• Usual method is to make an assumption about the
  overall shape of the curve, choosing from a few
  set shapes that are easy to work with
  mathematically.
• Then ask Bill for a few parameters that we can
  use to fit the curve.
• Because we are not so sure on our estimates
  anyways, the relative inaccuracy of choosing from
  one of a set of mathematically tractable p.d.f.s
  is small compared to the other estimation errors.

50
e.g., a Normal for w

• Assume work factors are adequately described by a
  bell-shaped Normal distribution.
• 2 points are required to fit a Normal
• E.g., average case and some reasonable worst
  case.
• Average case half the time less, half the time
  more 0.6
• Worst case 95 of the time w wont be that bad
  (small) 0.4
• Normal curves that fits is N(0.6,0.12).

area 68
51
Maybe not Normal

• Normals are easiest to work with mathematically.
• May not be the best thing to use for w
• Normal is symmetric about the mean
• E.g., N(0.6,0.12) predicts a 5 best case of
  0.8.
• What if Bill tells us the 5 best case is really
  1.0?
• Then cant use a Normal
• Would need a skewed (tilted) distribution with
  unsymmetrical 5 and 95 cases.
• Normal extends to infinity in both directions
• Finite probability of w lt 0 or w gt 10

52
Estimates

• Most define our quantities very precisely
• E.g., for a feature estimate of 1 week
• Post-Facto
• What are the units?
• 40 hours? Longer? Shorter? Dedicated? Disrupted?
  One person or two? ...
• Dealt with this last lecture in great detail
• Stochastic
• 1 week best case?
• 1 week worst case?
• 1 week average case?
• Need a p.d.f
• Depending upon these concerns, my 1 week maybe
  somebody elses 4 weeks.
• Very significant issue in practice

53
The Stochastic Capacity Constraint

• T is fixed
• F and N are both stochastic quantities.
• Can only speak about the chance of the goo
  fitting into the rectangle
• Say F400, N10, T40 are we good to go?
• Cannot say.
• Need precise distributions to F and N to answer,
  and then only at some confidence level.

54
Summing Distributions

• F and N are sums and products over many
  contributing stochastic variables.
• E.g.
• F f1 f2
• If f1 and f2 have associated statistical
  distributions, what is the statistical
  distribution of F?
• In general, no answer.
• Special case f1 and f2 are both Normal
• Then F will be Normal as well.
• Mean of F will be the sums of the means of f1 and
  f2
• Standard deviation of F will be the square root
  of the sums of the squares of the standard
  deviations of f1 and f2.
• How about f1 f2?
• Figet about it! Huge formula, result is not a
  Normal distribution
• One needs statistical simulation software tools
  to do arithmetic on stochastic variables.

55
Law of Large Numbers

• If we sum lots and lots of stochastic variables,
  the sum will approach a Normal distribution.
• Therefore something like F is going to be pretty
  close to Normal.
• E.g., 400 features summed
• N will also be, but a bit less so
• E.g., 10 ws summed

56
Delta Statistic

• D(T) N ? T ? F
• If we have Normal approximations for N and F, can
  compute the Normal curve for D as a function of
  various Ts.
• We can then choose a T that leads to a D we can
  live with.
• Interested in
• Probability D(T) ? 0
• The probability that all features will be
  finished by dcut.
• In choosing T will want to choose a confidence
  interval the company can live with, e.g., 80.
• Then will pick a T such that D(T) ? 0 80 of the
  time.

57
Example Picking T

• F is Normal with mean 400 and 90 worst case 500
• N is Normal with mean 10 and 90 worst case 8
• Cells are D(T) N ? T ? F at the indicated
  confidence level
• Note transitions through 0.

58
Choices for T

• To be 95 certain of hitting the dates, choose T
  60 workdays
• Or... If we plan to take 40 workdays, only 5 of
  the time will be late by more than 20 workdays
• To be 80 sure, T 49
• To gamble, for a 25 fighting chance, make T 33.

59
Shortcut

• Ask for 80 worst case estimates for everything.
• If F NxT using the 80 worst case values, then
  there is an 80 chance of making the release.
• The Deterministic Release Plan is based on this
  approach.
• If you also ask for mean cases for everything,
  can then fit a Normal distribution for D(T) and
  can predict the approximate probability of
  slipping.

60
Initial Planning

• Start with a T
• Choose a feature set
• See if the plan works out
• If not, adjust T and/or the feature set an
  continue

61
Adjusting the Release Plan

• Count on the w estimated to be too high and
  feature estimates to be too low.
• Re-adjust as new data comes in.
• Can pad the plan by choosing a 95 T.
• Will make it with a high degree of confidence
• May run out of work
• May gold plate features
• Better to have an A-list and a B-list
• Choose one T such that, e.g.,
• Have 95 confidence of making the A list
• Have 40 confidence of making the AB list.

62
Risk Tolerance

• Say a plan is at 60
• Developer may say
• Chances are poor 60 at best
• An entrepreneurial CEO will say
• Looking great! At least a 60 chance of making
  it.
• Should have an explicit discussion of risk
  tolerance

63
Loading the Dice

• Can manage to affect the outcome.
• Like a football game
• Odds may be 3-to-1 against a team winning
• But by making a special effort, the team may
  still win
• In release planning
• Base the odds on history
• But as a manager, dont ever accept that history
  is as good as you can do!
• E.g., introduce a new practice that will boost
  productivity
• Estimate will increase productivity by 20
• Dont plan for that!
• Plan for what was achieved historically.
• Manage to get that 20 and change history for
  next time around.

64
Example Stochastic Release Plan

• Sample Stochastic Release Plan