SOFTWARE QUALITY ASSURANCE

1
310313SOFTWARE ENGINEERING

• SOFTWARE QUALITY ASSURANCE

2
LEARNIING OBJECTIIVES

• 1. Understand the quality management process and
  the central
• process activities of quality assurance, quality
  planning and
• quality control.
• 2. Understand the importance of standards and
  metrics in the
• quality management process.
• 3. Know what is software configuration
  management, why it is
• important and what are its major activities.
• 4. Understand the principles of software process
  improvement and
• why process improvement is worthwhile.

3
SOFTWARE QUALITY ASSURANCE OUTLINE
10, 12

• Software Quality Assurance
• Life Cycle Role
• Purpose and Impoetance
• Techniques Standards and Metrics
• Achieving Software Quanlity
• Achieving Product Quality
• Design Quality Metrics
• Formal Approaches
• Program Quality Metrics
• Achieving Project Quality
• Reviews
• Software Configuration Management
• Achieving Process Quality
• ISO 9001/9000-3
• SEI Process Capability Maturity Model
• Achieving People Quality
• The People Capability Maturity Model

4
SOFTWARE QUALITY ASSURANCE LIFE CYCLE ROLE
5
THE PURPOSE SOFTWARE QUALITY ASSURANCE

• Quality assurance consists of those procedures,
  techniques, and tools applied by professionals to
  ensure that a product meets or exceeds
  pre-specified standards during its development
  cycle. E.H. Bersoff

quality assurance Defines organizational
standards that lead to
high quality software. quality
planing Selects appropriate standards and
tailors them to
a specific software product. quality control
Ensures standards are followed. Continuous
quality improvement should be the overall goal.
6
WHY SQA ACTIVITIES PAY OFF
cost to find and fix a defect
log scale
7
SQA AN UMBRELLA ACTIVITY
Standards and Procedures
Metrics and Measurement
The foundation of quality assurance.
Methods and Tools
Testing
Software Quality Assurance Techniques
Software Configuration Management
Formal Technical Reviews
8
SOFTWARE QUALIITY ASSURANCE TECHNIQUES

• Includes all techniques used to improve the
  quality of the software
• 1. developing and applying standards
• To ensure repeatability of the development
  process
• 2. developing and using metrics and measurement
  procedures
• To meet designgoals and improve the software
  development process
• 3. using appropriate methods and tools for system
  development
• To achieve a high quality system
• 4. doing formal technical reviews at each step
• To uncover quality problems and to sign-off
• 5. using software configuration management and
  change control
• To ensure changes are managed and controlled
• 6. performing multi-tiered testing
• To ensure effective defect detection. (BUT
  testing is not a panacea)

9
SQA STANDARDS

• Why are software standards important for SQA ?

• 1. They encapsulate best (or most appropriate)
  practices
• Acquired after much trial and error helps
  avoid previous mistakes
• 2. They provide a framework around which to
  implement SQA process
• Ensures that best practices are properly
  followed
• 3. They assist in ensuring continuity of project
  work
• Reduces learning effort when starting new work
• product standards define the characteristics all
  product artifacts should exhibit so as to have
  quality
• process standards define how the software
  process should be conducted to ensure quality
  software

Standards Handbook needed!
Each project needs to decide which standards
should be ignored used as is modified
created.
10
SQA TECHNIQUES METRICS

• A metric is any type of measurement that relates
  to a software system product , process or related
  artifact

Why are metrics important for Software Quality
Assurance? 1. Metrics can be used to control
(i.e., plan and manage) the development
process (e.g., effort expended, elapsed
time,budget spent, etc.). 2. Metrics can be used
to predict an associated product quality
(e.g., cyclomatic complexity can predict ease of
maintenance). Metrics are the only objective way
to measure quality attributes of software
otherwise it is all opinion and guess work.
11
ACHIEVING SOFTWARE QUALITY
set of activities
How can we achieve software quality?
measurement feedback
Process
template
software engineers
set of artifacts
Product
Project
People
participate
result
education training
management monitoring
testing measurement
12
ACHIEVING SOFTWARE QUALITY(contd)

• 1. We have a set of standards and quality
  attributes that a software product must meet.
• There are design goal to achieve.

• 2. We should be able to measure the quality of a
  software product.
• There is a way to determine how well the
  productconforms to the design goal.
• 3. We should track the values of the quality
  attributes.
• So that it is possible to assess ,over time, how
  well we are doing in achieving the design goal.
• 4. We use information about software quality to
  improve the quality of future software products.
• There is feedback into the software development
  process.

13
ACHIEVING PRODUCT QUALITY DESIGN GOALS

• Recall that a design goal is an (external)
  attribute that we want
• the system to have such as
• safety understandability portability
• security testability usability
• reliability adaptability reusability
• resilience modularity efficiency
• robustness maintainability learnability
• Usually, we can only asses whether a system has
  one of these
• attributes (i.e., meets its design goals) after
  it is completed!
• However, we would like to assess whether we are
  achieving a
• design goal during the systems development? How
  to do it?

Try to use internal attributes to assess
(predict) whether the system will meet its design
goals.
14
ACHIEVING PRODUCT QUALITY DESIGN GOALS(contd)

• Examples of possible relationships between
  external andinternal attributes

External(design goal) maintainability reliability
portability usability
Internal(measurable) of parameters cyclomatic
complexity lines of code of error
messages length of user manual
Problems 1. hard to formulate and validate
relationships between internal and external
attributes 2. software metrics must be
collected, calibrated, interpreted
15
ACHIEVING PRODUCT QUALITY DESIGN QUALITY METRICS

• For a design component, a key design goal is
  maintainability.

• For design components maintainability is related
  to
• cohesion - how closely related is the
  functionality of the component?
• coupling - how independent is the component?
• understandability - how easy is it to understand
  what the component does?
• adaptability - how easy is it to change the
  component?
• Problem most of these cannot be measured
  directly, but it is reasonable to infer that
  there is a relationship between these attributes
  and the complexity of a component
• measure complexity

How?
16
ACHIEVING PRODUCT QUALITY DESIGN QUALITY METRICS

• 1) Structural fan-in/fan-out
• fan-in the number of calls to a component by
  other components
• fan-out the number of components called by a
  component
• high fan-in gt high coupling
• high fan-out gt calling component has high
  complexity

• 2) Informational fan-in/fan-out
• consider also the number of parameters passed
  plus access to shared data structures
• complexity component-length x (fan-in x
  fan-out)2
• It has been validated using the Unix system
• It is a useful predictor of effort required for
  implementation

17
ACHIEVING PRODUCT QUALITY DESIGN QUALITY METRICS

• 3) IEEE Standard 982.1-1988
• looks at subsystem properties (number of
  subsystems and degree of coupling)database
  properties (number of attributes and classes)
• Compute a design structure quality indexDSQI
  (0-1)
• Used to compare with past designs if DSQI is
  too low,further design work and review may be
  required

• We can also consider changes made throughout the
  lifetime of the software and compute how stable
  the product is (i.e., how many changes have been
  made in subsystems in the current release)
• Define a software maturity indexSMI (0-1)
• As SMI approaches 1, the product begins to
  stabilize

18
IEEE STANDARD 982.1-1988
This slide is for information only you are not
required to know this!

• S1 the total number of subsystems defined in
  the program architecture
• S2 the number of subsystems whose correct
  function depends on the source of data input or
  that produces data to be used elsewhere
• S3 the number of subsystems whose correct
  function depends on prior processing
• S4 the number of database items (includes data
  objects and all attributes that define objects)
• S5 the total number of unique database items
• S6 the number of database segments (different
  records or individual objects
• S7 the number of subsystems with a single entry
  and exit

19
IEEE STANDARD 982.1-1988
This slide is for information only you are not
required to know this!

• Program structureD1 1 if the architecture was
  developed using a distinct methodD1 0
  otherwise
• Subsystem independence D2 1 - (S2/S1)
• Subsystems not dependent on prior processing D3
  1 - (S3/S1)
• Database size D4 1 - (S5/S4)
• Database compartmentalization D5 1 - (S6/S4)
• Subsystem entrance/exit characteristic D6 1 -
  (S7/S1)
• DSQI ?wiDi wi relative weighting of each Di

20
IEEE STANDARD 982.1-1988
This slide is for information only you are not
required to know this!

• SMI MT - (Fa Fc Fd)MT
• MT the number of subsystems in the current
  release
• Fc the number of subsystems in the current
  release that have been changed
• Fa the number of subsystems in the current
  release that have been added
• Fd the number of subsystems in the preceding
  release that were deleted in the current release

21
ACHIEVING PRODUCT QUALITY PROGRAM QUALITY METRICS
For an implementation component, the key quality
attributes are reliability and difficulty of
implementation.

• Some approaches to measure reliability and/or
  difficulty a) Halsteads Software
  Science
• Looks at operators and operands in an
  implementation component and calculates values
  for component volume, V, component difficulty, D,
  and effort, E, required to implement the
  component.
• n1 number of unique operators in a component
• n2 number of unique operands in a component
• N1 the total number of operators
• N2 the total number of operands
• L N1 N2 (component length)
• V L log2(n1 n2) (component volume in
  bits)
• D (n1/2) (N2/n2) (difficulty of implementing
  the component)
• E V D (effort required to implement the
  component)

22
ACHIEVING PRODUCT QUALITY PROGRAM QUALITY METRICS

• 2) McCabes Complexity Metric
• Looks at control flow in a component.
• Cyclomatic Complexity measures components
  logical complexity
• an indication of the testing difficulty of a
  component
• Studies have shown a distinct relationship
  between the Cyclomatic Complexity of a component
  and the number of errors found in the source
  code, as well as the time required to find and
  fix errors.

• 3) Other program quality metrics
• line of code (LOC) length of identifiers
• depth of conditional nesting

Standards can be established to avoid complex
components and/or highlight problem components.
23
ACHIEVING PRODUCT QUALITY FORMAL APPROACHES

• a) Proving programs/specifications correct
• Logically prove that requirements have been
  correctly transformed into programs
• (e.g., prove assertions about programs)
• b) Statistical Quality Assurance
• Categorize and determine cause of software
  defects
• Use 80-20 rule ? 80 of defects can be traced to
  20 of causes
• Isolate and correct 20 of the causes
• effort directed to things that cause the
  majority of defects
• c) The Cleanroom Process
• A combination of above two approaches

24
ACHIEVING PROJECT QUALITY REVIEWS

• The principal method of validating the quality of
  a project.

requirements walkthroughs
Leads to early discovery of defects
Requirements capture
analysis walkthroughs
Analysis
design walkthroughs
Design
Requirements, Analysis and Design introduce
50-60 of all defects.
code walkthroughs
Implementation
test plan review
Formal technical reviews can uncover 75 of these!
Testing
25
ACHIEVING PROJECT QUALIITYSOFTWARE CONFIGURATION
MANAGEMENT (SCM)
26
SOFTWARE CONFIGURATION MANAGEMENT (SCM)
10.2

• Software Configuration Management is an activity
  executed throughout the system life cycle to
  control the change to the products and life cycle
  artifacts.

To what do we want to control changes?
Software Configuration Management plans
programs specifications documents/manuals
procedures data
identify
support
control
report
audit
configuration item an artifact to which we want
to control changes
27
SCM ITEM ITEM IDENTIFCATION AND DESCRIPTION
10.3.1

• Each configuration item must be identified and
  described
• a unique name
• configuration item type
• project identifier
• change and/or version information
• resources the configuration item provides or
  requires
• pointer to actual configuration item
• The configuration items also must be organized
  (i.e., need to define the relationships that
  exist between configuration items)
• object diagram ltpart-ofgt domain model
• domain model ltrelated-togt use-case model

This is metadata about configuration items
This information is use to define and construct a
software library (database) that stores, manages
and tracks configuration items.
28
SCM ITEMS VERSIONS

• A configuration item usually evolves throughout
  the software engineering process (i.e., it will
  have several versions)
• An evolution graph can be used to describe
  aconfiguration items change history

• version
• Configuration itemk is obtained by modifying
  configuration itemi
• usually a result of bug fixes, enhanced system
  functionality, etc.
• itemk supercedes original itemi created in a
  linear order
• branch
• A concurrent development path requiring
  independent configuration management
• variant
• Different configurations that are intended to
  coexist
• e.g., different configurations depending on
  operating system type
• Oracle for Windows Oracle for Linux

29
SCM ITEMS EVOLUTION GRAPH
version
1.0
branch line
branch line
2.0
2.0.2.1
2.0.1
2.1
2.0.2.2
2.0.2
2.2
2.0.2.3
3.0
parallel development
mainline development
30
SCM SUPPORT SOFTWARE LIBRARY
10.3.5

• A software library provides facilities to store,
  label, identify versions and track the status of
  the configuration items

Used for
developers workspace
used by a single developer
everyday development
check-in/check-out of configuration items
Stores tracks
master directory
used by other developers
promotions
after SQA
Stores tracks
software repository
releases
for users
31
SCM BASELINES
10.3.2

• A baseline is a time/phase in the software
  development (usually a project milestone) after
  which any changes must be formalized (e.g., go
  through a formal change control procedure).

• To become a baseline, the configuration item must
  first pass a set of formal review procedures
  (e.g., formal code review, documentation review,
  etc.).
• It then becomes part of the project software
  library.
• After this a check-out procedure is applied
  to the item(i.e., access to and change of the
  configuration item is controlled)
• Any modified configuration item must again go
  through a formal review process before it can
  replace the original (baselined) item.
• Requires a version control mechanism.

32
SCM BASELINES
Requirements gathering
Requirements Specification
Analysis
Analysis Specification
Design
Design Specification
Implementation
Source code, manuals
Testing
Test Plans/Procedures/Data
Operational System
33
SCM CHANGE CONTROL
STOP

• change request submitted byusers/developers
• change control authority evaluates,decides and
  issues an engineeringchange order if approved
• configuration item is checked-out,changed,
  checked-in after SQA,and version controlled
• configuration item made availablefor use
  (promotion release)

Uncontrolled change rapidly leads to chaos !
34
SCM CHANGE CONTROL

• change request from user/developer
• change control authority evaluates and decides
• request is queued for action change request is
  denied
• assign individuals to configuration items user
  is informed
• check-out configuration items
• make changes
• review (audit) the change
• check-in the configuration items that have
  been changed
• establish a baseline for testing
• perform quality assurance and testing activities
• promote changes for inclusion in next release
• rebuild appropriate version of software
• review (audit) the change to all configuration
  items
• include changes in release
• distribute the release

can be applied tointernal requests(i.e., user
developer)andexternal requests(i.e., user
end-user)
35
SCM AUDIT AND STATUS REPORTING

• Audit ensures that changes have been properly
  implemented.

• Have the proper steps and procedures been
  followed? checklist
• Usually done by Quality Assurance (QA) group if
  SCM is a formal activity.

Status reporting keeps all parties informed and
up-to-date on the status of a change.

• Status reporting is a communication mechanism
  among project members to help keep them
  coordinated.
• Status reporting allows management/users to
  determine who made
• what changes, when and why.

36
SCM BENEFITS

• Reduces the effort required to manage and effect
  change
• Resulting in improved productivity

• Leads to better software integrity and security
• Resulting in increased quality
• Generates information about the process
• Resulting in enhanced management control
• Maintains a software development database
• Resulting in better record keeping and
  tracking

37
ACHIEVING PROCESS QUALITY

• Unlike manufactured products, software
  development has some unique factors that affect
  its quality
• software is designed, not manufactured
• software development is creative, not mechanical
• individual skills and experience have a
  significant influence
• external factors (application novelty, commercial
  pressure)
• Software development processes are organization
  specific
• Sometime peopletechnology may be more important
  than process
• Insufficient resources will always adversely
  affect quality

38
ACHIEVING PROCESS QUALIITY ISO 9001/9000-3
39
ACHIEVING PROCESS QUALITY ISO 9001/ 9000-3
Specific a generic quality model
Focus is on process management
The ISO quality Model need to be customized
for a particular organization

• certification is easy can be a marketing ploy

40
ISO 9001/9000-3 STANDARD (1/3)
This slide is for information only you are not
required to know this!

• A. Quality system framework
• 1. Management responsibility
• quality policy defined, documented, understood,
  implemented and maintained
• responsibilities and authorities for all
  personnel defined
• in-house verification resources defined, trained
  and funded
• a designated manager ensures that the quality
  program is implemented and maintained
• 2. Quality system
• requires a documented quality system be
  established
• should be an integral process throughout the
  entire life cycle
• 3. Internal quality system audits
• requires audits be planned and performed
• results communicated to management
• deficiencies corrected
• 4. Corrective action
• requires causes of non-conforming product be
  identified
• potential causes of non-conformance eliminated
• procedures changed as a result

41
ISO 9001/9000-3 STANDARD (2/3)
This slide is for information only you are not
required to know this!

• B. Quality Life Cycle Activities
• 1. Contract review
• contracts are reviewed to determine whether the
  requirements are adequately defined, agree with
  the bid, and can be implemented
• 2. Purchasers requirements specification
• procedures to identify and collect purchasers
  requirements
• 3. Development planning
• production processes are defined and planned
• 4. Quality record
• quality assurance process is planned
• 5. Design and implementation
• procedures to control and verify the design

• includes planning design
• activities, identifying inputs and outputs,
  verifying the design and controlling design
  changes
• 6. Testing and validation
• requires systems/products to be tested and
  validated before use
• records of testing are kept
• 7. Acceptance
• procedures for acceptance should be established
• 8. Replication, delivery and installation
• procedures for the above should be established
  and maintained
• 9. Maintenance
• requires that maintenance activities be performed
  as specified

42
ISO 9001/9000-3 STANDARD (3/3)
This slide is for information only you are not
required to know this!

• C. Quality System Supporting Activities

• 6. Tools and Techniques
• are applied appropriately to support the
  development process
• 7. Purchasing
• purchased products conform to their specified
  requirements
• 8. Included software product
• should be verified and maintained
• 9. Training
• training needs should be identified and training
  provided since related tasks may require
  qualified personnel
• training records should be maintained

• 1. Configuration management
• process of development and maintenance should be
  documented and controlled
• 2. Document control
• distribution and modification of documents should
  be controlled
• 3. Quality records
• quality records should be collected, maintained
  and dispositioned
• 4. Measurement
• where appropriate, adequate statistical
  techniques should be identified and used to
  verify the acceptability of process capability
  and product characteristics
• 5. Rules, practices and conventions
• are in place and followed

43
ACHIEVING PROCESS QUALITYTHE SEI CAPABILITY
MATURITY MODEL (CMM)
12.3

• SEU-CMM intended to help a software
  organizationimprove their software development
  processes.

• Level 1 Organization Initial process (ad hoc)
• no formal procedures, no cost estimates, no
  project plans,no management mechanism to ensure
  procedures are followed
• Level 2 Organization Repeatable process
  (intuitive)
• basic project controls intuitive methods used
• Level 3 Organization Defined process
  (qualitative)
• development process defined and institutionalized
• Level 4 Organization Managed process
  (quantitative)
• measured process process database established
• Level 5 Organization Optimizing process
• improvement feedback rigorous defect-cause
  analysis and prevention

Focus is on process improvement
44
CAPABILITY MATURITY MODEL (CMM)
This slide is for information only you are not
required to know this!

• Key Processes in Place
• Level 1 Initial process
• Level 2 Repeatable process
• Requirements Management
• Software Project Planning
• Software Project Tracking Oversight
• Software Subcontract Management
• Software Quality Assurance
• Software Configuration Management
• Level 3 Defined process
• Organization Process Focus
• Organization Process Definition
• Training Program
• Integrated Software Management
• Software Product Engineering
• Intergroup Coordination
• Peer Reviews

• Level 4 Managed process
• Quantitative Process Management
• Software Quality Management
• Level 5 Optimizing process
• Fault Prevention
• Technology Change Management
• Process Change Management

45
ACHIEVING PROCESS QUALITY NOTES

• It is possible for a Level 1 organization to
  receive ISO 9000 certification!
• A Level 3 organization would have little
  difficulty in obtaining ISO 9000 certification.
• A Level 2 organization would have significant
  advantage in obtaining ISO 9000 certification.
• excellent software developers (e.g., space
  shuttle) attain around level 3-4 only

46
ACHIEVING PEOPLE QUALITYPEOPLE CAPABILITY
MATURITY MODEL (PCMM)

• Intended to improve knowledge and skill of people.

• Level 1 Initial
• no technical or management training
  providedstaff talent not a critical resource
  no organizational loyalty
• Level 2 Repeatable
• focus on developing basic work practices staff
  recruiting, growth and development important
  training to fill skill gaps performance
  evaluated
• Level 3 Defined
• focus on tailoring work practices to
  organizations business strategic plan to locate
  and develop required talent skills-based
  compensation
• Level 4 Managed
• focus on increasing competence in critical
  skills mentoring team-building quantitative
  competence goals evaluation of effectiveness of
  work practices
• Level 5 Optimizing
• focus on improving team and individual skills
  use of best practices

Focus is on people improvement
47
SQA THE BOTTOM LINE

• an organization should have a quality manual
  which documents its quality assurance procedures

• each project should have a quality plan which
  sets out the quality attributes that are most
  important for that project and how they will be
  assessed
• an organization should have established standards
  for documentation
• mechanisms (processes) should be established that
  monitor compliance with all quality requirements
  of the organization
• reviews are the primary means of carrying out
  quality assurance
• metrics can be used to highlight anomalous parts
  of the software which may have quality problems

48
SUMMARY SOFTWARE QUALITY

• Quality software does not just happen!

• Quality assurance mechanisms should be built into
  the software development process
• Developing quality software requires
• Management support and involvement
• Gathering and use of software metrics
• Policies and procedures that everyone follows
• Commitment to following the policies and
  procedures even when things get rough!

Testing is an important part of quality
assurance, but its not all there is to obtaining
a quality software product.