Lecture Notes

1
Software Quality Assurance
Software Quality Assurance Lecture Notes M. D.
Dykton 21 April 2003 University of Maryland
Baltimore County CS 345 Software Engineering
2
Outline

• What is software quality?
• What is software quality assurance?
• How do you create software quality?

3
Software Quality Why Bother?

• Importance of Software
• Ever increasing levels of system functionality
  are embedded in software, not hardware
• For complex, software-intensive systems
• 1970sestimated functionality 20 software/80
  hardware
• 2000sestimated functionality 80 software and
  growing
• Size, complexity and criticality of software is
  growing rapidly
• Quality is not easily produced by accident
  anymore
• Quality of work/reputation of organizations
  frequently depends upon producing quality
  software

4
Quality What is it?

• The Institute of Electrical and Electronics
  Engineers' (IEEE) Standard Glossary of Software
  Engineering Terminology defines quality as (1)
  the degree to which a system, component, or
  process meets specified requirements, and (2)
  customer or user needs or expectations.
• Kitchenham states quality is "hard to define,
  impossible to measure, easy to recognize."

5
Software Quality

• Multi-faceted concept, not simply defined
• Fundamentally, quality implies fitness for
  intended use
• In part, implies software meets a specification
  or set of requirements
• Also includes many additional attributes which
  may or may not be explicitly captured in a
  specification

• Correctness
• Safety
• Security
• Reliability
• Resilience
• Robustness
• Efficiency
• Complexity

• Reusability
• Learnability
• Usability
• Testability
• Understandability
• Modifiability
• Portability
• Maintainability

6
Software Quality Assurance

• Quality Management System
• Policy

• Three Principal Elements of SQA
• Software Quality Assurance
• Policy Establishment
• Software Quality Planning
• Software Quality Control

• Quality Guidance
• Standards
• Practices Procedures

Process Refinement

• Quality Controls
• Process enforcement

Feedback
Project-specified plans developed from QA guidance

Project 1 QA Plan
Project 2 QA Plan
Project n QA Plan
7
Software Quality Assurance Policy
Establishment

• High-level organization-based statement of
  software quality policy
• Goals
• Rationale
• Relationship to other QA policies standards
• Specific policies and procedures
• Roles and responsibilities
• Organizational changes
• Establishment of a Quality Assurance
• Management System

8
Software Quality Assurance Organization JHU/APL
Example
Steering Committee
Laboratory-level management oversight, sets policy
Software Engineering Process Group (SEPG)
Laboratory-level group that defines policy
Implementation and process improvement
Department-level group that defines detailed
policies, processes and procedures
Quality Council or Dept. SEPG
Software Quality Assurance Manager (SQAM)
Oversees and enforces department-level
policies, processes and procedures handles
process tailoring, support infrastructure mgmt,
staff training
Mini-SQAMs or QA Teams
Group-level QA support
9
SQA Process Improvement

• Software Engineering Institute (SEI) Software
  Capability Maturity Model (SW-CMM) classifies
  five levels of process
• Level 1 Initial Ad hoc, unpredictable software
  process
• Herding Cats.
• Level 2 Repeatable Focus is on effective
  software project management processes for cost,
  schedule and functionality
• Project planning
• Project tracking and oversight
• Requirements management
• Quality Assurance
• Configuration management
• Subcontract management

10
SQA Process Improvement(continued)

• Level 3 Defined Software management and
  engineering processes are standardized and
  documented
• Emphasis is on formal procedures to ensure
  defined process is followed
• Organizational product definition and focus
• Software product engineering
• Integrated software management
• Reviews
• Intergroup coordination
• Level 4 Managed Measures of the software
  process and product quality collected, and
  software process and products are quantitatively
  understood and controlled
• Software quality management
• Quantitative process management

11
SQA Process Improvement(continued)

• Level 5 Optimizing Level 4 process coupled with
  planned and funded process improvement program
• Process change management
• Technology change management
• Defect prevention

12
Software Quality Planning

• Software Policy Procedure considerations
• Risk management
• Sound software management and engineering
  practices
• Measurement program

13
SQA and Risk Management

• Risk management is an integral part of the
  process to develop software quality
• Quality is not freeQA activities costs time and
  moneytrade-offs are necessary
• Quality assurance activities are risk reduction
  efforts
• A one-size-fits-all quality assurance plan is
  rarely feasible
• QA process tailoring is necessary
• Risk management is an approach to intelligent
  process tailoring
• Risk Management
• Risk Identification
• Risk Analysis and Assessment
• Risk Planning Mitigation
• Risk Tracking

14
SQA and Risk Management(continued)

• SEI Software Risk Taxonomy
• Use risk taxonomy as a checklist to identify
  risks and potential risk mitigation activities
  (i.e., QA process and products)

• Program Constraints
• Resources
• Contract
• Project Interfaces

• Product Engineering
• Requirements
• Design
• Code and Unit Test
• Integration and Test
• Engineering Specialties

• Development Environment
• Development Process
• Development System
• Management Process
• Management Methods
• Work Environment

See Managing Risk Methods for Software Systems
Development, Elaine M. Hall, Addison Wesley
Longman, Inc., 1998, p.76, Table 4.1
15
Software Quality Control

• Process Enforcement
• Independent agent (i.e., Software Quality
  Assurance manager/team)
• Quantitative Metrics
• Data Collection / Archiving
• Review / Audit (process and products)
• Authority to act
• Process Assessment
• Process Improvement Feedback

16
Software Metrics

• Necessary, underused and hard
• Forms one of the pillars of prove-able quality
• Software and its development is complex and
  multi-dimensional, hard to understand and measure
• Attributes of Good Metrics
• Useful
• Meaningful
• Quantifiable
• Measurable
• Repeatable
• Metrics should be linked to risk mitigation
  activities and TE program (both product and
  process)

17
Software Metrics(continued - Example)
See Software Metrics A Rigorous Practical
Approach, Norman E. Fenton and Shari Lawrence
Pfleeger, PWS Publishing Company, 1997, p.76.
18
Software Metrics(continued)

• Scope of software metrics process, products or
  resources
• Project management
• Cost and level-of-effort estimation
• Productivity measures
• Quality attributes
• Reliability measures
• Performance
• Defect tracking
• Structural and complexity metrics
• Software engineering process metrics

19
Summary

• Software Quality is important software controls
  life-and-death decisions, has enormous economic
  consequences, affects reputations, etc.
• An independent Quality Management System is vital
  component of an effort to produce quality
  software - Quality needs a stakeholder
• Complete requirements determination is a
  critical first step, include both overt and
  implied requirements
• Quality assurance is a product of effective risk
  management
• Tailor project management and software
  engineering practices to mitigate quality-related
  risks
• Measurement program demonstrate progress toward
  quality objectives using metrics, testing and
  other measurements

20
References

• Managing Risk Methods for Software Systems
  Development, Elaine M. Hall, Addison Wesley
  Longman, Inc., 1998.
• Software Metrics A Rigorous Practical
  Approach, Norman E. Fenton and Shari Lawrence
  Pfleeger, PWS Publishing Company, 1997.
• Software Quality, Theory and Management. Gillies,
  Alan C., International Thomson, Computer Press,
  1997.
• Risk Mangement Processes for Software Engineering
  Models, Marian Myerson, Artech House, 1996.
• Software Engineering, Ian Sommerville, 5th
  Edition, Addison-Wesley Publishing Company, 1996.
• Software Quality Assurance A Practical Approach,
  Ernest Wallmuller, Prentice Hall International
  (UK) Ltd., 1994.
• ISO 9001 and Software Quality Assurance, Darrel
  Ince, McGraw-Hill Book Company, 1994.
• Handbook of Software Quality Assurance, Edited by
  G. Gordon Schulmeyer and James I. McManus, 2nd
  Edition, Van Nostrand Reinhold Company, 1992.
• Managing the Software Process, Watts S. Humphrey,
  Addison-Wesley Publishing Company, 1989/1990.
• Applications Strategies for Risk Analysis,
  McGraw-Hill Software Engineering Series, Robert
  N. Charette, Intertext Publications, 1990.
• Software Product Assurance Techniques for
  Reducing Software Risk, William L. Bryan and
  Stanley G. Siegel, Elsevier Science Publishing
  Co., Inc., 1988.
• Characteristics of Software Quality, Barry W.
  Boehm, John R. Brown, Hans Kaspar, Myron Lipow,
  Gordon J. MacLeod and Michael J. Merritt,
  North-Holland Publishing Company, 1978
• Kitchenham, Barbara, and Shari Lawrence Pfleeger.
  "Software Quality The Elusive Target." IEEE
  Software 13, 1, Jan. 1996 12-21.