Inspections

1
Inspections
EE491.05
Maj JW Paul
From a presentation by Diane Kelly Dr Terry
Shepard
2
Experience is something you get just after you
really need it
3
Critique Schedule v3.7
Due Discussion Leader SilverBullet
16 Sept 22 Sept Paul FM x 2 30
Sept 6 Oct Oldford Toenders Quality x 2
14 Oct 20 Oct Hutter
McLellan Testing/Bugs 28 Oct 3 Nov
Pollard Girard SWEng 12 Nov 17 Nov
Maurice
4
Review

• Name the primary change in DND software
  specification standards
• a move from defining content to defining the
  process
• What drove this change?
• it wasnt working

5
Software Mathematics

• Do questions HS appendix H.2
• 3a,d, 6a,d 8a,d 10b, 12c,
• 18a, 19a
• Due Monday, 22 Sept

6
Lab 1 questions?
7
Outline

• What is a defect?
• Inspection in Industry
• The 3 Ms of Inspection
• Eight Maxims of Inspection

Todays Class
8
Overloading a Defect
error
issue
botch
problem
defect
incident
delusion
fault
bug
elision
complaint
finding
goof
slip
mistake
flaw
failure
trouble
boner
error
anomaly
gripe
glitch
blunder
howler
oversight
9
A problem is a problem

• Defect or opportunity?
• Help system could be improved
• Scale issues
• system exceeds time limit
• several components involved
• Difficulty in categorizing 35
• are all typing errors the same?
• syntactic vs semantic

10
But what if

• scope/details change after initial report
• variable used multiple times but only guarded
  (range checked) in one case
• missing/unnecessary guard? (special case)
• setting of a flag indicating a fatal error is
  commented out
• mistake?
• undocumented change of error handling style?
• kluge to stop the program from terminating?

11
What is a Defect?

• not a fault (i.e. doesnt lead to failure)
• incorrect comment
• not an error
• new version of a component or a platform
• sometimes not clear
• dead code that may have a purpose
• really an enhancement
• limitations on current functionality

12
Simple definitions

• Gilb 4
• a defect is identified as something that is
  incorrect
• anything that impacts or interferes with
  maintaining or improving quality and production
• Fredericks and Basili 29
• any fault, failure, or error occurring in a
  software system

13
Definitions from standards 36

• Any condition or characteristic in any supplies
  or services furnished by the contractor under the
  contract that is not in compliance with the
  requirements of the contract. GATUS federal
  govt

14
Definitions from standards 36

• The non-fulfillment of intended usage
  requirements. ISO8402, QMPP
• Any non-conformance of a characteristic with
  specified requirements. MIL-STD 105, QMPP
• A substandard condition. CSM Centre for Systems
  Management

15
Robert Grady at HP 37

• A defect is a deviation from the product
  specification or an error in the specification if
  the error could have been detected and would have
  been corrected. If the error could not possibly
  have been detected, or it could have been
  detected and would not have been corrected, then
  it is an enhancement, not a defect. Defects do
  not include typographical or grammatical errors
  in the engineering documentation.

16
Our Definition
A defect is an observable property of code or
other work product that could diminish the
desired level of any quality factor defined for
the software system.
Quality factors (Ilities) Timeliness,
Functionality, Cost, Correctness, Reliability,
Maintainability Usability, Flexibility,
Adaptability, Testability, Readability,
Portability, Device independence, Self
containedness, Reusability, Interoperability,
Efficiency, Device efficiency, Integrity,
Security, Accuracy, Robustness, Completeness,
Consistency, Accountability, Accessibility, Human
Engineered, Self descriptive, Fault tolerant,
Structuredness, Conciseness, Legibility,
Augmentability, Modifiability, Understandability,
Clarity, Resilience, Validity, Generality,
Minimality, Modularity, Expandability,...
17
Defect Classification

• Why would we want to do this?
• to identify trends
• to help with solutions
• to lay the blame
• More to follow

18
Inspections
19
Inspection use in Industry
General Inspection
Code Inspection
Don't Inspect
But 50 of development time is spent in debugging
and testing!!
20
Some quotes...

• At best, the state of practice is we inspect our
  key components
• At worst, its we dont have the time to do
  inspections at all
• The value of inspection is probably the topic on
  which computing research agrees more consistently
  than any other

21
Why Inspect

• Finding faults early is cheaper
• 1991 BNR showed 65 to 90 of operational
  (execution-time) defects are detected by
  inspection at 1/4 to 2/3 the cost of testing 13
• Other arguments for inspection
• better control of process
• higher quality
• reduced defect rates

22
What makes inspections hard?

• People issues
• Mentally demanding
• Specific skills in short supply
• Moderator needs special people skills

• Organization issues
• Time pressures
• Attitudes
• Lack of ownership
• Unclear/multiple goals

23
Qualities of Good Inspectors

• Meticulousness
• Patience
• Knowledge
• Focus
• Objectivity
• Structuredness

• Editor
• Detective
• Expert

24
Industry Standard Inspection

• Fagan (1976 1986) inspection
• defines a process
• defines roles for the participants
• suggests feedback to improve the development
  process
• Fagan approach to inspections has become an
  industry standard

25
Fagan-style inspections

• Preparation/entry and exit criteria
• 3-6 people including Moderator
• Control the process
• inspection rate100 to 200 loc/hr
• meeting length max 2 hr
• number of meetings per day max 2
• Measure the process
• report classify time spent rates

26
Gilb and Graham
variation on Fagan
27
New Approaches Fagan may not be best 25

• Research industry suggest varying
• goals
• meetings
• techniques
• size of teams
• entry criteria
• exit criteria
• when to inspect
• activities to include/exclude

28
Some tests of Fagan

• Votta 5
• two-person meetings are enough?
• Porter Votta 26
• collection meetings produce no net improvement
  in terms of meeting losses and meeting gains
• Johnson Tjanjono 17
• meetings help to detect false positives, not help
  find more defects

29
Another test

• Laitenberger et al 28 (DaimlerChrysler Germany)
• each inspector prepares and presents a model of
  the requirements document under inspection
• defects presented in context of their model
• used two inspectors plus the author
• experiment concludes that meeting results are
  better than individual results

30
More than one meeting?

• Porter, Siy, Toman, Votta 8
• varied the the number of reviewers, the number of
  teams inspecting the code unit, and the
  requirement of fixing defects between first and
  second teams inspections
• one of the conclusions careful what you change -
  you may not get what you wished for... (change to
  process is not as important as technique)

31
Review

• How does this apply to Lab 1

32
Questions?
33
Next Class

• 3 Ms of Inspections
• Mechanics
• Metrics
• Management
• Lab1 finish math assignment