EEEGEF 492'15 Software Process Metrics

1
EEE/GEF 492.15 Software Process Metrics
Royal Military College of Canada Electrical and
Computer Engineering

• Professor Colin Wortley
• wortley_at_rmc.ca
• 613-541-6000 ext. 6493

Dr. Terry Shepard shepard_at_rmc.ca 613-541-6000
ext. 6031
2
Why Software Metrics?

• Will they answer the question of how much
  software weighs?

3
(No Transcript)
4
Le kilogramme est l'unité de masse il est égal
à la masse du prototype international du
kilogramme.

• Louis XVI chargea un groupe de savants d'établir
  un nouveau système de mesure. Leurs travaux
  aboutirent à la création du système métrique
  décimal , qui est devenu le SI tel que nous le
  connaissons aujourd'hui. L'idée de départ de la
  commission royale (qui comprenait des notables
  tels que Lavoisier) était de créer une unité de
  masse qui porterait le nom de grave . Par
  définition, cette unité serait la masse d'un
  litre d'eau à la température de congélation
  (c'est-à-dire pratiquement 1 kg). Elle serait
  représentée par un étalon de masse.
• Après la Révolution, le nouveau Gouvernement
  républicain reprit l'idée du système métrique,
  mais en y apportant des changements notables. Par
  exemple, comme un grand nombre de mesures de
  masse effectuées à cette époque concernaient des
  masses bien plus petites que un kilogramme, le
  Gouvernement décida que l'unité de masse serait
  le gramme . Cependant, un étalon d'un gramme
  étant aussi difficile à utiliser qu'à établir, il
  décida de représenter l'unité de masse par un
  étalon d'un kilogramme. Cet étalon serait connu
  sous le nom de kilogramme des archives . Vers
  1875, l'unité de masse fut redéfinie comme
  kilogramme , et fut représentée par un nouvel
  objet dont la masse était pratiquement identique
  à celle du kilogramme des archives.
• La décision du Gouvernement républicain eut
  peut-être des motivations politiques  après
  tout, n'était-il pas composé des mêmes personnes
  qui ont condamné Lavoisier à être guillotiné...
  Quoi qu'il en soit, du fait de cette décision
  malheureuse, nous avons hérité d'une unité de
  base dont le nom comporte un préfixe.

http//www1.bipm.org/fr/si/history-si/name_kg.html
5
Le kilogramme est l'unité de masse il est égal
à la masse du prototype international du
kilogramme.

• The reason why "kilogram" is the name of a base
  unit of the SI is an artefact of history.
• Louis XVI charged a group of savants to develop a
  new system of measurement. Their work laid the
  foundation for the "decimal metric system", which
  has evolved into the modern SI. The original idea
  of the king's commission (which included such
  notables as Lavoisier) was to create a unit of
  mass that would be known as the "grave". By
  definition it would be the mass of a litre of
  water at the ice point (i.e. essentially 1 kg).
  The definition was to be embodied in an artefact
  mass standard.
• After the Revolution, the new Republican
  government took over the idea of the metric
  system but made some significant changes. For
  example, since many mass measurements of the time
  concerned masses much smaller than the kilogram,
  they decided that the unit of mass should be the
  "gramme". However, since a one-gramme standard
  would have been difficult to use as well as to
  establish, they also decided that the new
  definition should be embodied in a one-kilogramme
  artefact. This artefact became known as the
  "kilogram of the archives". By 1875 the unit of
  mass had been redefined as the "kilogram",
  embodied by a new artefact whose mass was
  essentially the same as the kilogram of the
  archives.
• The decision of the Republican government may
  have been politically motivated after all, these
  were the same people who condemned Lavoisier to
  the guillotine. In any case, we are now stuck
  with the infelicity of a base unit whose name has
  a "prefix".

http//www1.bipm.org/fr/si/history-si/name_kg.html
6
http//encyclopedia.thefreedictionary.com/Kilogram
me

• The international prototype of the kilogram seems
  to have lost about 50 micrograms in the last 100
  years, and the reason for the loss is still
  unknown (reported in Der Spiegel, 2003 26). The
  observed variation in the prototype has
  intensified the search for a new definition of
  the kilogram. Although it is accurate to state
  that all other objects in the universe have
  gained 50 micrograms per kilogram, this
  perspective is counterintuitive and defeats the
  purpose of a standard unit of mass.

7
Measuring is hard

• Even old, widely used, well established metrics
• have a complicated history
• are still changing
• What is a metric?
• the type of property being measured?
• eg. lines of code
• the unit used to measure a property?
• eg. a standard for counting lines of code in Java
• the result of measuring a property?
• eg. the number of lines of code in a particular
  program
• In software, the term metric is used in all
  three senses
• most often in the first two senses

8
Software does have weight-usually not a very
interesting property

• Software weight depends on how it is represented
  paper, chips, circuit boards, chalk, CD, shrink
  wrap,
• Software can exert very large forces
• Changing the software may not change its weight

• choosing useful metrics for software is a hard
  problem
• there are too many choices
• - we will let Walker (not Winston) Royce choose
  for us

9
four scales of measurement

• The four scales are defined by three qualities
• magnitude, equal intervals, and zero
• Most basic is the nominal scale,
• represents only names
• has none of the three qualities. 
• eg. a list of students, the set of names in a
  program
• Second is the ordinal scale
• has magnitude only
• values are ordered but arithmetic operations are
  pointless
• eg. house numbers on a street, ranking of
  software versions by market position
• Third is the interval scale
• possesses both magnitude and equal intervals
• values can be usefully added or subtracted
• eg. temperature in degrees F, ranking of software
  versions by date of release
• Fourth is the ratio scale.
• contains all three qualities
• values can be usefully multiplied or divided
• eg. temperature in degrees K, number of lines of
  code in a module

an example
10
The following table shows the width and height of
ISO A and B paper formats, as well as the ISO C
envelope formats. The dimensions are in
millimeters
A8, A9, A10
11
The ISO paper size concept
.
which scale is being used?
why does software not use A0 size paper? - all
other branches of engineering do
12

• Is a process metric different from a product
  metric?

examples?
rest of this lecture will focus on process
metrics
13
Properties of Process Metrics

• provide objective measures of
• project progress
• qualities of software products
• provide the means for estimating the cost and
  schedule to complete a project with desired
  levels of qualities
• accuracy improves over time
• each metric has two dimensions
• a static value
• a dynamic trend
• measuring change (trend) is the most important
  aspect of metrics for project management

process metrics provide reality checks
14
Seven Core Process Metrics ( Walker Royce)

• Management Indicators
• Work and Progress
• Budgeted Cost and Expenditures
• Staffing and Team Dynamics
• Quality Indicators
• Change Traffic and Stability
• Breakage and Modularity
• Rework and Adaptability
• MTBF and Maturity

15
M1 work and progress

• purpose
• iteration planning
• plan vs. actuals
• management indicator
• perspectives
• SLOC
• function points
• object points
• scenarios
• test cases
• Software Change Orders (SCOs)

16
Work and Progress
100
planned progress 40
Work Plan
current time
actual progress 25
progress
schedule variance 15
time
100
17
M2 - budgeted costs and expenditures

• purpose
• financial insight
• plans vs. actuals
• management indicator
• perspectives
• cost per month
• full-time staff per month
• of budget expended
• earned value calculations

18
Planned and Actual Cost ? Progress
actual progress?
Expenditure Plan
100
current time
planned cost 40
expenditure
actual cost 15
cost variance -25
time
100
19
Earned Value Example
The earned value at the current time is the
planned cost of the actual progress. The negative
cost variance represents an under-budget
situation.
100
planned cost 40
expenditure (planned and actual)
Expenditure Plan
current time
actual progress 25
actual cost 15
schedule variance 15
cost variance -10
time
100
20
M3 staffing and team dynamics

• purpose
• resource plan vs. actuals
• hiring rate
• attrition rate
• perspectives
• people per month added
• people per month leaving

21
Staffing and Team Dynamics
planned
actual
Staffing
the two charts dont correspond ???
22
M4 - change traffic and stability

• purpose
• iteration planning
• management indicator of schedule convergence
• perspectives
• SCOs opened versus SCOs closed
• by type (0,1,2,3,4)
• by release / subsystem / component

23
Change Traffic and Stability
Open SCOs
SCOs
Closed SCOs
time
does this graph represent stability?
24
M5 breakage and modularity

• purpose
• convergence
• software scrap
• quality indicator (maintainability)
• perspectives
• reworked SLOC per change
• by type (0,1,2,3,4)
• by release / subsystem / component

25
Breakage and Modularity
Healthy
Breakage ()
design changes
implementation changes
time
26
M6 rework and adaptability

• purpose
• convergence
• software rework
• quality indicator (maintainability)
• perspectives
• average hours per change
• by type (0,1,2,3,4)
• by release / subsystem / component

27
Rework and Adaptability
Rework (/change)
implementation changes
Sick
design changes
time
28
M7 MTBF and maturity

• purpose
• test coverage / adequacy
• robustness for use
• quality indicator (reliability)
• perspectives
• failure counts
• by release / subsystem / component
• test hours until failure (MTBF)
• by release / subsystem / component

29
MTBF and Maturity
MTBF (hours/failure)
Reliability Growth
time
30
MTBF and Maturity
MTBF (hours/failure)
Reliability Decay
time
31
Characteristics of Good metrics
32
Characteristics of Good Metrics

• meaningful to the customer, manager and performer
• demonstrates quantifiable (real) correlation
  between process changes and business performance
• everything boils down to (cost reduction,
  revenue growth and margin increase)
• objective and unambiguous
• displays trend
• natural by-product of the process
• supported by automation

33
References

• Royce, Walker. Software Project Management,
  Addison-Wesley, 1998. Chapter 13.