Test process and test process development

1
Software testing standards ISO/IEC 29119 and
33063

• ISO/IEC 29119 software testing
• ISO/IEC 33063 assessment of testing
• Ossi Taipale, Heikki Uusitalo, Risto Nevalainen
• FISMA r.y, ISO/IEC SC7

2
JTC 1 SC7 Structure
LCPHAG
Life Cycle Process Harmonization Advisory Group
SC7
Secrétariat
WG26
Software Testing
3
Software testing standards ISO/IEC 29119
Contents of the ISO/IEC 29119 testing
standard Part 1 Concepts and Definitions Part 2
Testing Process Part 3 Test Documentation Part 4
Test Techniques
4
ISO/IEC 29119 Part 1 Concepts and Definitions
Scope This standard covers definitions and
concepts in systems and software testing.
Example, test case set of test case
preconditions, inputs (including actions, where
applicable), and expected results, developed to
determine whether or not the covered part of the
test item has been implemented correctly. NOTE A
test case is the lowest level of test input (i.e.
test cases are not made up of test
cases). Generic process Testing can be described
as a process. A process is a set of interrelated
or interacting activities that transforms inputs
into outputs. The objective of this standard is
to present and describe a generic testing
process. Life cycle models Agile Evolutionary Se
quential (i.e. the waterfall model)
5
ISO/IEC 29119 Part 1 Concepts and Definitions
Static testing Testing can be carried out on a
product without executing the product on a
computer. This is called static testing in this
standard and in many areas of the industry,
although other standards (e.g. IEEE1028, Standard
for Software Reviews and Audits) may more
specifically call this reviews, walkthroughs or
inspections. For static testing this standard
acknowledges and identifies the role of the
tester in these activities even though they may
be owned by other groups or standards within a
project. This is because the static testing
activities are considered highly important for
complete lifecycle testing and test involvement
has been shown to be critical for early defect
detection, reduced overall project costs and an
improved ability to meet schedule demands.
6
ISO/IEC 29119 Part 1 Concepts and Definitions
The relationship between the generic test
sub-process, test levels and test types.
7
ISO/IEC 29119 Part 1 Concepts and Definitions
The Role of Testing in Verification and
Validation This standard addresses only parts of
the verification and validation activities. Other
standards, e.g. ISO/IEC 12207, Software life
cycle processes, address other verification or
validation activities. Testing, as described in
this standard, addresses both dynamic and static
testing as well as associated sub processes and
activities. This standard does not address,
except in passing, demonstration activities or
validation and verification. To provide complete
validation and verification of a product an
organization will need to use this standard in
conjunction with other standards as part of a
comprehensive engineering program. See Annex A
for a diagram of verification and validation
activities. (IEEE 1012, Standard for Software
Verification and Validation)
8
ISO/IEC 29119 Part 1 Concepts and Definitions
Hierarchy of Verification and Validation
activities
9
ISO/IEC 29119 Part 2 Test Process
The multi-layer test process Three levels, the
approach is risk-based
10
ISO/IEC 29119 Part 2 Test Process
The multi-layer model showing all test processes
11
ISO/IEC 29119 Part 2 Test Process
Example test management process relationships
12
ISO/IEC 29119 Part 2 Test Process
13
ISO/IEC 29119 Part 2 Test Process
Dynamic test processes
14
ISO/IEC 29119 Part 2 Test Process
15
ISO/IEC 29119 Part 3 Test Documentation
Test Documentation, includes templates and
examples of test documentation that are produced
during the test process.
The test documentation in this International
Standard describes the output of the processes
specified in part two of this standard.
16
ISO/IEC 29119 Part 4 Test Techniques
The test techniques in this International
Standard are used to derive the test cases
executed as part of the dynamic testing process
specified in part two of this standard.
17
ISO/IEC 29119 Part 4/Part 2 Test Design and
Implementation Process
18
ISO/IEC 29119 Part 4 Test Techniques
Test Coverage Measurement The coverage measures
defined in this part of the standard are based on
test design technique coverage. Coverage levels
can range from 0 to 100. Quality
Characteristics Software testing can be carried
out to collect evidence that required quality
criteria have been satisfied by a test item.
Required quality characteristics should be
specified in the test basis. Definitions of
quality characteristics could be derived from
ISO/IEC 25010 System and Software Product Quality
Requirements and Evaluation (SQuaRE) System and
Software Quality Models.
19
ISO/IEC 29119 Part 4 Test Techniques
Mapping Quality Characteristics (ISO/IEC 25010)
to Test Design Techniques and Types of Testing
20
ISO/IEC 29119 Part 5 NWIP, Keyword-Driven Testing
Keyword Word or term used as a reference to a
specific action performed during the execution of
a test case.
21
ISO/IEC 33063 Testing assessment
22
ISO/IEC 33063 Testing assessment
23
ISO/IEC 33063 Testing assessment
Assessment indicators