Software Metrics/Quality Metrics - PowerPoint PPT Presentation


PPT – Software Metrics/Quality Metrics PowerPoint presentation | free to download - id: 3c0d26-YzkyM


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

Software Metrics/Quality Metrics


Software Metrics/Quality Metrics Software Quality Metrics: Product Pre-Release and Post-Release Process Project Data Collection Product Characteristics – PowerPoint PPT presentation

Number of Views:544
Avg rating:3.0/5.0
Slides: 30
Provided by: cseSpsuE
Learn more at:


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Software Metrics/Quality Metrics

Software Metrics/Quality Metrics
  • Software Quality Metrics
  • Product
  • Pre-Release and Post-Release Process
  • Project
  • Data Collection

Product Characteristics
Project Characteristics
Process Characteristics
Software Metrics
  • Software Product
  • All the deliverables
  • Focus has been on Code, but interested in all
  • Product metrics include concerns of complexity,
    performance, size, quality, etc.
  • Pre-Release Post Release Processes
  • Focus has been on Pre-Release processes
  • Easiest staring point for metrics Testing and
    the number of bug found
  • Process metrics used to improve on development
    and support activities
  • Process metrics include defect removal
    effectiveness, problem fix response time, etc.
  • Project
  • Cost
  • Schedule
  • HR staffing and other Resources
  • Customer Satisfaction
  • Project metrics used to improve productivity,
    cost, etc.
  • Project metrics include cost such as effort/size,
    speed such as size/time, etc.
  • Project and Process metrics are often

Will talk about this more
Function point
Product Quality Metrics
  • What are all the deliverables ?
  • Code and Help Text
  • Documentation (function, install, usage, etc. in
    requirements design specifications)
  • Education (set-up/configure, end-user, etc.)
  • Test Scenarios and Test Cases
  • Quality Question (mostly intrinsic to the
    product but affects external customer
    satisfactions )
  • When/where does it fail how often
  • how many defect rate

GQM (one more time from Basili)
  • A reminder on generating measurements
  • In coming up with metrics think of GQM
  • Whats the goal
  • Whats the Question
  • Whats the metric
  • Goal is Improved Quality
  • Question What is the Post Release Defect Rate?
  • Metric Number of problems found per user months

Some Definitions of Error to Failure
  • Error human mistake that results in incorrect
    software (one or more fault or defect)
  • Defect or Fault a mistake in the software
    product that may or may not be encountered
  • Problem a non-functioning behavior of the
    software as a result of a defect/fault in the
  • Note that an error can cause one or more defects,
    and a defect can cause one or more problems. But
    a problem may never surface even if there is a
    defect which was caused by a human error.

When/Where Do Product Failures Occur
  • When/Where are somewhat intertwined
  • Right away e.g. happens at install
  • Sometimes - e.g. happens at initialization-config
  • Sometimes e.g. happens at certain file access
  • Generalized Metric
  • Mean time to failure (MTTF)
  • Difficult to assess
  • What should be the goal (8 hours, 30 days, 6
    months), or should we just say --- lessen the
    failure rate?
  • Hard to test for and analyze (especially- prod.
    education, doc., etc.)
  • Applies better for simple logic (like stays up
    for z amount of time)

Meantime to failure for install problem should
probably be close to 0
Product Defects and Defect Rate
  • Most of the metric has been asked in terms of
    code but should be more inclusive
  • Defect Volume How many defects (for the complete
    product - not just for code)
  • Defect Rate defects/(opportunity of defect)
  • Defects of all kind or by type (e.g. code, test
    cases, design, etc.)
  • Defects by severity (not quite a rate more by
  • Opportunity of defect by ( also used to assess
  • Code loc, function point, module
  • Documentation pages, diagrams
  • Education or training of power point slides
    (doc) or amt. of time (delivery)

Code Defect Opportunity (LOC)
  • Using Lines of code (loc) problems
  • Executable, non-executable (comments)
  • Test cases and scaffolding code
  • Data and file declaration
  • Physical line or logical line
  • Language difference (C, C, assembler, Visual
    Basic, etc.)

Possible Code Defect Rate Metrics
  • Often used
  • Valid Unique Defect per line of executable and/or
    data code released(shipped)
  • IBMs total valid unique defects / KSSI
  • Total valid unique defects / KCSI (only changed
  • Valid Unique Defect of high severity per line
    of executable and/or data code released (shipped)
  • What about all in-line comments should they
    not count ? These provide opportunity of defects
    too. (especially for pre and post condition
  • What about Help text ?

Product Quality Metric (User View)
  • Defect rate is not as useful from user
  • What type of problems do users face?
  • screen interface
  • data reliability/(validity)
  • functional completeness
  • end user education
  • product stability - crashes
  • error message and recovery
  • Inconsistencies in the handling of similar
  • How often are these types of defect encountered?
  • ---- counted with -- MTTF -- means more
    to users?
  • Possible metric is Problems per User Month(PUM)
  • user month is dependent on length of period and
    the number of users (this takes some tracking
  • More Broader Customer Satisfaction issues
  • CUPRIMDSO capability, usability, performance,
    rel. etc. (IBM),
  • FURPS functionality, usability, reliability,
    etc. (HP)

Begin Function Point
  • Separate Segment

Function Point (product size or complexity)
  • Often used to assess the software complexity
    and/or size
  • May be used as the opportunity for defect part
    of defect rate
  • Started by Albrecht of IBM in late 70s
  • Gained momentum in the 90s with IFPUG as
    software service industry looked for a metric
  • Function Point does provide some advantages over
  • language independence
  • dont need the actual lines of code to do the
  • takes into account of many aspects of the
    software product
  • Some disadvantages include
  • a little complex to come up with the final
  • consistency (data reliability) sometimes varies
    by people

Function Point Metric via GQM
  • Goal Measure the Size(volume) of Software
  • Question What is the size of a software in terms
    of its
  • Data files
  • Transactions
  • Metrics
  • amount/difficulty of Functionalities to
    represent size/volume
  • consider Function Points ---- (defined in this

What kind of validity problem might you
encounter? construct applicability,
predictive relational content coverage?
FP Utility
  • Where is FP used?
  • Comparing software in a normalized fashion
    independent of op. system, languages, etc.
  • Benchmarking and Prediction
  • size .vs. cost
  • size vs development schedule
  • size vs defect rate
  • Outsourcing Negotiation

  • Identify and Classifying
  • Data (or files/tables)
  • Transactions
  • Evaluation of Complexity Level
  • Compute the Initial Functional Point
  • Re-assess the range of other factors that may
    influence the computed Functional Point and
    Compute the Function Point

1) Identifying Classifying 5 Basic Entities
  • Data/File
  • Internally generated and stored (logical files
    and tables)
  • Data maintained externally and requires an
    external interface to access (external interfaces
  • Transactions
  • Information or data entry into a system for
    transaction processing (inputs)
  • Information or data leaving the system such as
    reports or feeds to another application (outputs)
  • Information or data displayed on the screen in
    response to query (query)

Note - What about tough algorithms and other
function oriented stuff? (We take of
that separately in a separate 14 Degree of
2) Evaluating Complexity
  • Using a complexity table, each of the 5 basic
    entity is evaluated as
  • low
  • average
  • high
  • Complexity table uses 3 attributes for decisions
  • of Record Element Types (RET) e.g. employee
    data type, student record type ---- of file
  • of unique attributes (fields) or Data Element
    Types (DET) for each record e.g. name, address,
    employee number, and hiring date would make 4
    DETs for the employee file
  • of File Type Referenced (FTR) e.g an external
    payroll record file that needs to be accessed

5 Basic Entity Types uses the RET, DET, and
FTRfor Complexity Evaluation
For Logical Files and External Interfaces (DATA)
of RET 1-19 DET 20-50 DET
50 DET 1
Low Low
Ave 2 -5 Low
High 6 Avg
High High

For Input/Output/Query (TRANSACTIONS)
of FTR 1-4 DET 5 -15 DET
16 DET 0 - 1
Low Low
Ave 2 Low
High 3 Avg
High High

  • Consider a requirement ability or functionality
    to add a new employee to the system.
  • (Data) Employee information involves, say, 3
    external file that each has a different Record
    Element Types (RET)
  • Employee Basic Information file has employee data
  • Each employee record has 55 fields (1 RET and 55
  • Employee Benefits records file
  • Each benefit record has 10 fields (1 RET and 10
    DET) - LOW
  • Employee Tax records file
  • Each tax record has 5 fields ( 1 RET and 5 DET)
    - LOW
  • (Transaction) Adding a new employee involves 1
    input transaction which involves 3 file types
    referenced (FTR) and a total of 70 fields (DET).
    So for the 1 input transaction the complexity is

Function Point (FP) Computation
  • Composed of 5 Basic Entities
  • input items
  • output items
  • inquiry
  • master and logical files
  • external interfaces
  • And a complexity level index matrix

Logical files
Ext. Interface
3) Compute Initial Function Point
  • Initial Function Point
  • S Basic Entity x Complexity Level Index

all basic entities
Continuing the Example of adding new employee
- 1 external interface (average) 7 - 1
external interface (low) 5 - 1 external
interface (low) 5 - 1 input (high) 6
Initial Function Point 1x7 1x5 1x5 1x6
4) More to Consider
  • There are 14 more Degree of Influences (DI) on
    a scale of 0 - 5
  • data communications
  • distributed data processing
  • performance criteria
  • heavy hardware utilization
  • high transaction rate
  • online data entry
  • end user efficiency
  • on-line update
  • complex computation
  • reusability
  • ease of installation
  • ease of operation
  • portability (supports multiple sites)
  • maintainability (easy to change)

Function Point Computation (cont.)
  • Define Technical Complexity Factor (TCF)
  • TCF .65 (.01 x DI )
  • where DI SUM ( influence factor value)
  • So note that .65 TCF 1.35

Function Point (FP) Initial FP x TCF
Finishing the Example Suppose after considering
14 DIs, our TCF 1.15, then Function
Point Initial FP x TCF 23 x 1.15 26.45
Defect Rate Defects/FP by CMM Levels
  • C. Jones estimated defect rates by SEIs CMM
    levels through the maintenance life of a software
  • CMM Level 1 organizations 0.75 defect/FP
  • CMM Level 2 - 0.44
  • CMM Level 3 0.27
  • CMM Level 4 0.14
  • CMM Level 5 0.05

Be careful of this type of claims use it with
End Function Point
  • Separate Segment

Pre-Release Process Quality Metrics
  • Most common one is from testing (Defect
    Discovery Rate)
  • defects found (by severity) per time period
    ( per dev. phase)
  • Compare defect arrivals by time by test phase
  • looking for stabilization (what would the
    curve look like?)
  • looking for a decreasing pattern
  • Compare number of defects by products
  • those with high number of problems found during
    pre-release tend to be buggy after release
    (interesting phenomenon)
  • Other Pre-Release quality metric Defect Removal
    Effectiveness (e.g. Via inspection)
  • defects removed / ( total latent defects)
  • latent defects are estimated how estimated?
    --- go back later with defects found in the field

Post-Release Product and Process
  • Post-Release Product
  • of Problems per Usage-Month ( of PUM)
  • Post-Release Fix Process
  • Fix Quality Number of Fix bugs/ Total number of
  • Very sensitive if fix quality is not close to
  • Post-Release Process Quality
  • Problem backlog total of problems unresolved
  • by severity
  • by arrival date
  • Problem Backlog Index of problems resolved /
    of arrivals per some time period such as week
    or month
  • Average Fix Response Time ( from problem open to
    close )
  • These metrics are usually compared with a goal
  • average response time on severity 1 problem is 24
  • problem backlog index is between 1.3 and .8 (.8
    may be problem!)

Collecting Data
  • Decide on what Metrics are to be used
  • measuring what (validity of measure)
  • whats the goal (validity of measure)
  • Decide on how to collect the data
  • clearly defining the data to be collected
  • assure the recording is accurate (reliability)
  • assure the classification is accurate
  • Decide on tools to help in the collection
  • source code count
  • problem tracking

Data Collection Methodology (Basili Weiss)
  • Establish the goal of the data collection
  • Develop a list of questions of interest
  • Establish data categories
  • Design and test data collection mechanism (e.g.
  • Collect and check the reliability data
  • Analyze the data