Static Techniques (on code) - PowerPoint PPT Presentation

Loading...

PPT – Static Techniques (on code) PowerPoint presentation | free to view - id: b6782-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Static Techniques (on code)

Description:

possibly used for very critical parts of code or in high risk software! ... They are usually related to other quality attributes than correctness. ... – PowerPoint PPT presentation

Number of Views:152
Avg rating:3.0/5.0
Slides: 106
Provided by: beriogi
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Static Techniques (on code)


1
Static Techniques (on code)
  • (Static Analysis of code)
  • performed on the code without executing the code

2
Categories of Static Techniques
  • Manual
  • (Semi) Mechanised

3
Static techniques vs. code testing
  • Code testing tries to characterize set of
    executions throughout one test case --- (minimal)
    coverage (of input similar executions by using
    classes of input data, of paths, others) is the
    most important issue
  • Static techniques characterize once set of
    executions thats the reason to call qualify
    them as static techniques
  • Static techniques are usually used within a
    verification activity (i.e they may come before
    testing)
  • Static techniques and testing have complementary
    advantages and disadvantages additionally, some
    static techniques during the testing to support
    the test case design

4
Informal analysis techniquesCode walkthroughs
  • Recommended prescriptions
  • Small number of people (three to five)
  • Participants receive written documentation from
    the designer few days before the meeting
  • Predefined duration of meeting (few hours)
  • Focus on the discovery of defects, not on fixing
    them
  • Participants designer, moderator, and a
    secretary
  • Foster cooperation no evaluation of people
  • Experience shows that most defects are discovered
    by the designer during the presentation, while
    trying to explain the design to other people.

5
Informal analysis techniquesCode inspection
  • A reading code technique aiming at defect
    discovery
  • Based on checklist (also called defect-guessing),
    e.g.
  • use of uninitialized variables
  • jumps into loops
  • nonterminating loops
  • array indexes out of bounds

6
Defect Guessing
  • From intuition and experience, enumerate a list
    of possible defects or defect prone situations
  • Defect guessing can also be used to write test
    cases to expose those defect

7
Defect Guessing Esempio
  • Nel caso di array o stringhe, ogni indice è
    compreso nei limiti della dimensione
    corrispondente?
  • Ci si riferisce ad una variabile non
    inizializzata?
  • Per i riferimenti attraverso puntatore/riferimento
    , la corrispondente area di memoria è stata
    allocata (dangling reference problem)?
  • Una variabile (eventualmente riferita tramite
    puntatore) ha tipo diverso da quello usato dal
    programma?
  • Esistono variabili con nome simile (pratica
    pericolosa)?

8
Defect Guessing Esempio
  • I calcoli coinvolgono tipi diversi e
    inconsistenti (ad es., stringhe e interi)?
  • Esistono delle inconsistenze nei calcoli misti
    (ad es., interi e reali)?
  • Esistono dei calcoli che coinvolgono tipi
    compatibili ma di precisione differente?
  • In unassegnazione (xexp, xexp), il valore
    sinistro ha rappresentazione meno precisa del
    valore destro?
  • È possibile una condizione di overflow o
    underflow (ad esempio nei calcoli intermedi)?
  • Divisione per zero?

9
Defect Guessing Esempio
  • Nelle espressioni che contengono vari operatori
    aritmetici, le assunzioni sullordine di
    valutazione e di precedenza degli operatori sono
    corrette?
  • Il valore di una variabile esce dallintervallo
    ragionevole? (ad es., il peso dovrebbe essere
    positivo, )
  • Ci sono usi sbagliati dellaritmetica fra interi,
    in particolare delle divisioni?
  • Stiamo prendendo in considerazione adeguatamente
    la precisione finita dei reali?

10
Defect Guessing Esempio
  • Gli operatori di confronto sono usati
    correttamente?
  • Le espressioni booleane sono corrette (uso
    appropriato di and, or, not)?
  • Nelle espressioni che contengono vari operatori
    booleani, le assunzioni sullordine di
    valutazione e di precedenza degli operatori sono
    corrette?
  • Gli operandi di unespressione booleana sono
    booleani?
  • La maniera in cui viene valutata lespressione
    booleana ha impatto sul significato
    dellespressione stessa (pratica pericolosa)?

11
Correctness proofs
12
A program and its specification (Hoare notation)
true begin read (a) read (b) x a
b write (x) end output input1 input2
proof by backwards substitution
13
Proof rules
Notation for If Claim 1 and Claim 2 have been
proven, one can deduce Claim3
Claim1, Claim2 Claim3
14
Proof rules for a language
F1S1F2, F2S2F3 F1S1S2F3
sequence
if-then-else
Pre and cond S1 Post,Pre and not cond S2
Post Pre if cond then S1 else S 2 end if
Post
while-do
I and cond S I I while cond loop S end
loop I and not cond
I is called the loop invariant
15
Correctness proof
  • Partial correctness proof
  • validity of Pre program Post guarantees that
    if the Pre holds before the execution of program,
    and if the program ever terminates, then Post
    will be achieved
  • Total correctness proof
  • Pre guarantees program termination and the truth
    of Post
  • These problems are undecidable!!!

16
Example
input1 gt 0 and input2 gt 0 begin read (input1)
read (input2) xinput1 yinput2 div
0 while x gt y loop div div 1 x x
- y end loop write (div) write
(x) end input1 div input2 x
17
Discovery of loop invariant
  • Difficult and creative because invariants cannot
    be constructed automatically
  • In the previous example
  • input1 div y x and yinput2

18
Combining correctness proofs
  • We can prove correctness of operations (e.g.
    operations on a class)
  • Then use the result of each single proof to make
    proof for complex modules containing these
    operations or complex combinations of these
    operations

19
Example
module TABLE exports type Table_Type (max_size
NATURAL) ? no more than max_size entries may
be stored in a table user modules must
guarantee this procedure Insert (Table in out
TableType ELEMENT in ElementType) procedur
e Delete (Table in out TableType ELEMENT in
ElementType) function Size (Table in
Table_Type) return NATURAL provides the current
size of a table end TABLE
20
Having proved these
true Delete (Table, Element) Element ?
Table Size (Table) lt max_size Insert (Table,
Element) Element ? Table
We can then prove properties of programs using
tables For example, that after executing the
sequence Insert(T, x) Delete(T, x) x is not
present in T
21
An assessment ofcorrectness proofs
  • Still not used in practice
  • However
  • possibly used for very critical parts of code or
    in high risk software!
  • assertions (any intermediate property) may be the
    basis for a systematic way of inserting runtime
    checks (instead of checking values of variables)
  • proofs may become more practical as more powerful
    support tools are developed
  • knowledge of correctness theory helps programmers
    being rigorous
  • well written post conditions can be used to
    design test cases

22
Symbolic execution
  • Can be viewed as a middle way between testing and
    pure verification (but it is anyway a
    verification technique)
  • Executes the program on symbolic values (symbolic
    expressions)
  • One symbolic execution corresponds to many usual
    program executions

23
Example (1)
Consider executing the following program with
xX, yY, aA (symbolic binding) x y
2 if x gt a then a a 2 else y x
3 end if x x a y
Build the control graph!
24
Example(2)
  • When control reaches decisions, in general,
    symbolic values do not allow to select a branch
  • One can choose a branch, and record the choice in
    a path condition
  • Result
  • lt lta A, y Y 5, x 2 Y A 7gt
  • lt1, 3, 4gt , Y 2 lt A gt

symbolic binding
execution path
path condition
25
Symbolic execution rules (1)
symbolic state ltsymbolic_binding,
execution_path, path_conditiongt
  • read (x)
  • removes any existing binding for x and adds
    binding x X, where X is a newly introduced
    symbol
  • write (expression)
  • output(n) computed_symbolic_value_expression (n
    counter initialized to 1 and automatically
    incremented after each write statement)

26
Symbolic execution rules (2)
  • x expression
  • construct symbolic value of expression, SV
    replace existing binding of x with xSV
  • After execution of a statement of a path that
    corresponds to an edge of control graph, append
    the edge to execution path

27
Symbolic execution rules (3)
  • if cond then S1 else S2 endif
  • while cond loopendloop
  • condition is symbolically evaluated
  • eval(cond)
  • If it is possible to state eval(cond) ? true or
    eval(cond) ? false then execution proceeds by
    following the appropriate branch
  • otherwise, make a choice of true or false, and
    conjoin eval(cond) (resp., not eval(cond) to
    the path condition

28
Symbolic execution and testing
  • The path condition describes all data are
    required for the program execution follows the
    execution path
  • Usage of symbolic execution for testing
  • identify one execution path (I.e. a sequence of
    arrows in a control graph)
  • symbolically execute the execution path (if
    possible)
  • chose data that satisfy the path condition
  • These data allow to execute that path and
    therefore are a test case for the path.

29
Example (1)
found false counter 1 while (not found)
and counter lt number_of_items loop if table
(counter) desired_element then found
true end if counter counter 1 end
loop if found then write ("the desired
element exists in the table") else write
("the desired element does not exist in
the table") end if
30
Example (2)
1,2,3,5,6,2,4 is not possible!
31
Why so many approaches to testing and analysis?
  • Testing versus static techniques
  • Formal versus informal techniques
  • White-box versus black-box techniques
  • Fully mechanised vs. semi-mechanised techniques
    (for undecidable properties)
  • view all these as complementary

32
OO Unit Code Defect Testing
33
Test Case Design Objectives
  • Coverage is the always the key point (what things
    are covered by the test cases).
  • Minimality of this coverage is the other key
    point (do not write two distinct test cases for
    discovering same defects).

34
Some techniques for making test cases
  • General defect testing
  • The tester looks for plausible defects (i.e.,
    aspects of the implementation of the system that
    may result in defects). To determine whether
    these defects exist, test cases are designed to
    exercise the code.
  • Specialized defect testing Class Testing and
    Class Hierarchy
  • Inheritance does not obviate the need for
    thorough testing of all derived classes. In fact,
    it can actually complicate the testing process.
  • Scenario-Based Test Design (defect but also
    acceptance testing)
  • Scenario-based testing concentrates on what the
    user does, not what the product does. This means
    capturing the tasks (via use-cases) that the user
    has to perform, then applying them and their
    variants as test cases.

35
Two levels of test
  • Test of one class
  • Test single operations (white and black box)
  • Test sequences of operations (random test)
  • Test sequences of states (behavior test)
  • Test of sequences of operations and states is
    because in object-oriented code, expressing the
    expected-behavior or what the program does in
    term of input and output is not possible
    therefore, the codomain R is described by
    sequences of operations or states
  • Test of several classes with interacting objects

36
Esempio
Pre pulsante P premuto Post la richiesta R è
memorizzata
qualè peso è possibile dimmi pulsanti da
illuminare
37
Esempio
avviarediminuire fermare
38
Esempio
Le operazioni sono indipendenti a parte alcuni
vincoli che possono essere espressi con pre e
post condizioni
open setupAccount deposit
39
Behavior Testing
The tests to be designed should achieve all state
coverage KIR94. That is, the operation
sequences should cause the Account class to make
transition through all allowable states
Black box!
40
OO Software Inter-Class Testing
  • Inter-class testing to exercise interactions
    between classes
  • For each class, use the list of class operations
    to generate a series of random test sequences of
    operations. The operations will send messages to
    other classes.
  • For each message that is generated, determine the
    destination class and the corresponding
    operations.
  • For each operation in the destination class (that
    has been invoked by incoming messages), determine
    the messages that it transmits.
  • For each of the messages, determine the next
    level of operations that are invoked and
    incorporate these into the test sequence

Black box if based on sequence diagrams!
White box if based on class code!
41
(No Transcript)
42
OO Software Additional tests
  • New issues
  • inheritance
  • genericity
  • polymorphism
  • dynamic binding
  • Open problems still exist

White box!
43
How to test classes of a hierarchy?
  • Flattening the whole hierarchy and considering
    every class as a totally independent unit
  • does not exploit incremental class definition
  • Finding an ad-hoc way to take advantage of the
    hierarchy

44
A simple technique test case design for class
hierarchy
  • A test case that does not have to be repeated for
    any heir
  • A test case that must be performed for heir class
    X and all of its further heirs
  • A test case that must be redone by applying the
    same input data, but verifying that the output is
    not (or is) changed
  • A test case that must be modified by adding other
    input parameters and verifying that the output
    changes accordingly

45
Black Box testing concurrent and real-time
software
  • Non-determinism (of events driving the control
    flow) inherent in concurrency affects
    repeatability of failures
  • For real-time software (i.e. with time
    constraints), a test case consists not only of
    usual input data, but also of the time when such
    data are supplied (events)

output
input
events
46
Software Testing in the large
47
Software Testing Strategy
In the small component testing
unit test
integration test
Defect testing
In the large
system test
Validation (acceptance) test
Elaborated from Pressman
48
Software Architectures and Integration Testing
  • Software architectures provides at least the
    structure of a complex software therefore, it is
    natural to perform testing on integrated code by
    following the architecture

Layered
Call-return
Object-oriented
49
What should be tested!
  • We have tested units according to the expected
    behavior and (some forms) of unexpected behavior
  • This is largely related to functional
    requirements and to discovery related defects in
    code units testing as such is therefore related
    to the correctness of code and is performed on
    code with possible inputs (EB) or potential
    inputs (P), without attention to how inputs are
    provided and where the software is installed
  • What about non-functional requirements? They are
    usually related to other quality attributes than
    correctness.
  • We need to talk about a software system (not just
    the software but the software installed and
    running) for talking about the software running
    in its environment! The software system is then
    part of the whole system and it is therefore a
    subsystem of it (as many others).

50
Software System and Software
software code
BB/WB
system
system
Operating systems, Middleware,Compilers,
Interpreters, N of installation of the same
module, environment parameters
software system
software system (sub-system)
BB/WB
BB/WB
BB/WB
Assumptions during the requirement engineering!
Perfect technology assumption!
51
Separate concerns in testing
  • Testing for correctness is not enough, e.g.
  • Overload (stress) testing (reliability)
  • Robustness testing (test unexpected situations)
    (safety)
  • Performance testing (test response time)
  • These tests are typically related to some
    software quality attributes and non-functional
    requirements and usually performed on the
    software system (or subsystems) not on single
    units

52
Testing Activities in the Software Process
System Requirements
System Test
planned
The word system refers to the whole system
and to the software system. The software system
implicitly encompasses hardware and allocation of
software on hardware.
planned
53
Levels of Testing
Type of Testing
Performed By
  • Programmer
  • Development team Independent Test Group
  • Independent Test Group
  • Customer/End Users
  • Low-level testing
  • Unit (module) testing
  • Integration testing
  • High-level testing
  • System testing
  • Acceptance testing

54
Integration Testing
55
Unit Testing
  • Done on individual units (or units of modules)
  • Test unit with white-box and black-box
  • Requires stubs and drivers
  • Unit testing of several units can be done in
    parallel

56
What are Stubs, Drivers ?
  • Stub
  • dummy module which simulates the functionality of
    a module called by a given module under test
  • Driver
  • a module which transmits test cases in the form
    of input arguments to the given module under test
    and either prints or interprets the results
    produced by it

e.g. to test B in isolation
e.g. module call hierarchy
Driver
Stub for C
57
Esempio
Come si fa a fare il test di integrazione?
Read (x,y,a) call P(x,y) if xthen write (test
ok) else write (test not ok)
P(x,y) x y 2 if x gt a then a a
2 else y x 3 a F(x, y) end if x x
a y
Driver
P
x y 2 if x gt a then a a 2 else y
x 3 a STUB_F(x, y) end if x x a y
x
y
Stub for F
If xlty then yx5 If xgty then yx7
58
Esempio
Come si fa a fare il test della seguente unità?
Esecuzione simbolica, dirà che prima della
chiamata di F(x,y) xY2, aA e yY5 (con
Y2lta)
x y 2 if x gt a then a a 2 else y
x 3 a F(x,y) end if x x a y
Definizione dei test cases di P (per esempio
Black Box) Es. A5, Y1, X7 Per cui deve essere
noto loutput di F(3,6)
59
Esempio
Come si fa a fare il test di integrazione?
P(x,y) x y 2 if x gt a then a a
2 else y x 3 a STUB_F(x, y) end
if x x a y
Driver
P
x
y
Stub for F
Possibili Inputs per F Indicare manualmente i
possibili outputs
Identificare i test cases per P
60
Esempio
Come si fa a fare il test di integrazione?
Read (x,y,a) call P(x,y) if xthen write (test
ok) else write (test not ok)
P(x,y) x y 2 if x gt a then a a
2 else y x 3 a F(x, y) end if x x
a y
Driver
P
x y 2 if x gt a then a a 2 else y
x 3 a STUB_F(x, y) end if x x a y
x
y
Stub for F
Write (x) Read (y)
61
Integration Testing
  • Test integrated modules (i.e. integrated code)
  • Usually focuses on interfaces (i.e. calls and
    parameters passing) between modules (defects are
    in the way modules are called)
  • Largely architecture-dependent

62
Integration Test Approaches
  • Non-incremental ( Big-Bang integration )
  • tests each module independently (black and white
    box)
  • combines all the modules to form the integrated
    code in one step, and test (usually black-box)
  • Incremental
  • instead of testing each module in isolation, the
    next module to be tested is combined with the set
    of modules that have already been tested
  • With two possible approaches Top-down, Bottom-up

63
Comparison
  • Non-Incremental
  • requires more stubs,drivers
  • module interfacing defects are detected late
  • finding defects is difficult
  • Incremental
  • requires less stubs, drivers
  • module interfacing defects detected early
  • finding defects is easier
  • not all modules should be implemented for
    starting test
  • results in more thorough testing of modules

64
Top-down Integration
  • Begin with the top module in the module call
    hierarchy (represented as a structure chart)
  • Stub modules are produced
  • But stubs are often complicated
  • The next module to be tested is any module with
    at least one previously tested superordinate
    (calling) module (depth first or breadth first
    ways)
  • After a module has been tested, one of its stubs
    is replaced by an actual module (the next one to
    be tested) and its required stubs

65
Example of a Module Hierarchy
66
Top-down Integration Testing
Example
Stub B
Stub C
Stub D
67
Top-down Integration Testing
Example
Stub C
Stub D
Test cases written for A are reused Test cases
white and black box for B should be combined with
test cases written for A
Stub F
Stub E
68
Bottom-Up Integration
  • Begin with the terminal (leaves) modules (those
    that do not call other modules) of the modules
    call hierarchy
  • A driver module is produced for every module
  • The next module to be tested is any module whose
    subordinate modules (the modules it calls) have
    all been tested
  • After a module has been tested, its driver is
    replaced by an actual module (the next one to be
    tested) and its driver

69
Example of a Module Hierarchy
70
Bottom-Up Integration Testing
Example
Driver E
Driver F
71
Bottom-Up Integration Testing
Example
Driver B
B
Test cases white and black box for B
72
Comparison
  • Top-down Integration
  • Advantage
  • a skeletal version of the program exists early
  • Disadvantage
  • required stubs could be expensive
  • Bottom-up Integration
  • Disadvantage
  • the program as a whole does not exist until the
    last module is added

No clear winner
  • Effective alternative -- use hybrid of bottom-up
    and top-down
  • - prioritize the integration of modules based on
    risk
  • - highest risk modules are integration tested
    earlier
  • than modules with low risk

73
Regression Testing
  • Re-run of previous test cases to ensure that
    software already tested has not regressed to
    earlier defects after making changes (or
    integration) to the software
  • Regression testing can also be performed during
    the entire life a software
  • Reusability of test cases is the key point!

74
Types of System and Acceptance Testing
75
(Sub)System Testing
  • Process of attempting to demonstrate that system
    (or subsystem) does not meet its original
    requirements and objectives as stated in the
    requirements (specification) document i.e. it is
    a defect testing
  • Usually, it is not only a code testing but a test
    on the software system
  • Test cases derived from
  • system objectives, user scenarios, possibly
    during system engineering or early requirement
    engineering
  • requirement document and software requirements
    specification (analysis model)
  • expected quality stated during the design
    engineering
  • additional aspects related to deployment of code

76
Usual Types of Software System Testing
  • Volume testing
  • to determine whether the system can handle the
    required volumes of data, requests, etc.
  • Load/Stress testing
  • to identify peak load conditions at which the
    system will fail to handle required processing
    loads within required time spans
  • Usability (human factors) testing
  • to identify discrepancies between the user
    interfaces of the system (software) and the human
    engineering requirements of its potential users.
  • Security Testing
  • to show that the systems security requirements
    can be subverted

77
Usual Types of Software System Testing
  • Performance testing (also as code testing)
  • to determine whether the system meets its
    performance requirements (eg. response times,
    throughput rates, etc.)
  • Reliability/availability testing
  • to determine whether the system meets its
    reliability and availability requirements (here
    availability is related to failure however
    availability may only be related to out of
    service situations not necessarily related to
    failures)
  • Recovery testing
  • to determine whether the system meets its
    requirements for recovery after a failure

78
Usual Types of Software System Testing
  • Installability testing
  • to identify ways in which the installation
    procedures lead to incorrect results
  • Configuration testing
  • to determine whether the system operates properly
    when the software or hardware is configured in a
    required manner
  • Compatibility testing
  • to determine whether the compatibility
    (interoperability) objectives of the system have
    been met
  • Resource usage testing
  • to determine whether the system uses resources
    (memory, disk space, etc.) at levels which exceed
    requirements
  • Others

79
Alpha and Beta Testing
  • Acceptance testing performed on the developed
    software before its released to the whole user
    community.
  • Alpha testing
  • conducted at the developer site by End Users (who
    will use the software once delivered)
  • tests conducted in a controlled environment
  • Beta testing
  • conducted at one or more customer sites by the
    End Users
  • it is a live use of the delivered software in
    an environment over which the developers has no
    control

80
Stop conditions for Defect Test
81
When to Stop Defect Testing ?
  • Defect testing is potentially a never ending
    activity!
  • However, an exit condition should be defined,
    e.g.
  • Stop when the scheduled time for testing expires
  • Stop when all the test cases execute without
    detecting failures
  • but both criteria are not good

82
Better Code Defect Testing Stop conditions
  • Stop on use of specific test-case design
    techniques.
  • Example Test cases derived from
  • 1) satisfying multiple condition coverage and
  • 2) boundary-value analysis and
  • 3) .
  • .
  • all resultant test cases are eventually
    unsuccessful (i.e they do not lead to failures)

83
Better Code Defect Testing Stop condition 1
  • Sia ND il numero dei difetti
  • Inserire nello unit un insieme di difetti NDI
  • Far eseguire il test (a qualcuno che non conosce
    i difetti inseriti) attraverso un certo numero di
    tecniche
  • Lefficacia di tale test è quindi
  • NumeroDifettiInseritieScoperti/NumeroDifettiInseri
    ti
  • (NDIS/NDI)
  • Nellipotesi che i difetti siano simili si può
    dire che
  • (NDS/ND)(NDIS/NDI) e quindi
  • NDNDINDS/NDIS ove NDS è il NumeroDifettiScopert
    i

84
Better Code Defect Testing Stop condition 2
  • Miglioramento con due gruppi indipendenti di test
    che trovano X e Y difetti, di cui Q sono comuni
  • ND, numero totale dei difetti, è quindi pari a
    NDXY/Q (poiché si ipotizza che X/NDQ/Y)

85
System Testing Stop condition
  • Stop in terms of failures (rate) to be found and
    therefore in term of time to be spent in testing
  • This stop condition is closely related to the
    reliability of the software system

86
Test Automation
87
Steps in Test Cases definition and Execution
88
Test tools
89
Debugging
  • Task for locating and correcting defects in
    software code
  • It can start when a failure has been detected
  • It usually performed during test
  • Need sometime the definition of an intermediate
    concept, error i.e. a situation leading to the
    failure and due to the defect
  • Requires closing up the gap between a fault and
    failure
  • watch points
  • intermediate assertions

What is seen from an external observer
defect (fault) ? error ? failure
The cause
What is recognized as non correct situation
discovering
discovering
90
Autodebugging, System management and
Fault-tolerance
  • Detect errors and alert on them, may stop or may
    not stop the execution (leading to failure)
  • Detect errors and undertake a fault management
    strategy (recovery, alternatives etc.) that
    allows to tolerate the fault!

91
Performance, Reliability TestingQuality
Assessment for subjective quality attributes
92
Performance
  • Types of performance analysis (can also be
    applied to code)
  • Worst case analysis
  • focus is on proving that the (sub)system response
    time is bounded by some function of the external
    requests and parameters
  • Average behavior
  • Analytical vs. experimental approaches, an both
    may concern the (software) system
  • Queue models, statistics and probability theory
    (Markov chains)
  • Simulation
  • Others

93
Correctness review
  • Correcteness is an absolute feature of software
    with a binary result (the software is correct,
    the software is not correct)
  • Typically, correcteness is expressed in term of
    functional requirements or component
    specifications derived from functional
    requirements
  • Less important for real systems where hypotheses
    (made during requirement engineering or as
    perfect technology assumptions) on which these
    systems are built are only true in probability or
    sometime false
  • Correcteness can be reformulated as
  • Reliability (probability to work without failures
    over a time period)
  • Robustness (management of unexpected situations
    (i.e. failures elsewhere))
  • Safety (probability that something does not
    happen)

94
Reliability (1)
  • There are approaches to measure reliability on a
    probabilistic basis, as in other engineering
    fields, i.e. the probability the (software)
    system will work without failure over a period of
    time and under some conditions (shortly,
    probability to do not fail within a time frame)
  • Unfortunately, there are some difficulties with
    this approach
  • independence of failures does not hold for
    software

If xgt0 then write(y) else (write(x) write(z))
X is wrongly assigned to 7 instead of 6 Z is
wrongly assigned
X is wrongly assigned to 0 instead of 1 Z is
wrongly assigned
95
Reliability (2)
  • Reliability is firstly concerned with measuring
    the probability of the occurrence of failures
  • Meaningful parameters include
  • average total number of failures observed at time
    t AF(t)
  • failure intensity FI(t)AF'(t)
  • mean time to fail at time t MTTF(t)1/FI(t)
  • mean time between failures MTBF(t)MTTF(t)MTTR(t)
    (MTTR corresponds to time needed after a
    failure, to repair)
  • Time is the execution time but also the calendar
    time (because in part of the software system can
    be shared with other software systems)

96
Basic reliability model
  • Assumes that the decrement per failure observed
    (i.e., the derivative with respect to the number
    of observed failures) of the failure intensity
    function is constant
  • i.e., FI is a function of AF
  • FI(AF) FI0 (1 - AF/AF8)
  • where FI0 is the initial failure intensity and
    AF8 is the total number of failures
  • The model is based on optimistic hypothesis that
    a decrease in failures is due to the fixing of
    defects, sources of those failures

97
AF law
AF
Basic model
FI(0)
FI(t)
t
98
Uso del modello base
Stima Af e ? Calcola il tempo t per cui
Af(t)Af(T) 1 ove T è il tempo cui si è
arrivati con il test e si sono osservati Af(T)
failure (quindi AfAF(T) indica il numero di
failure ancora osservabili) Fare il test per
almeno affinché il sistema sia eseguito per
almeno t-T in modo da osservare un ulteriore
failure, se vi sono ancora difetti
99
Assessment of subjective (less factual) quality
attributes
  • Quality assessment on code of subjective quality
    attributes
  • Consider quality attribute like maintainability,
    reusability, understandability
  • There is the need of metrics

100
Internal and external attributes of quality
Software quality attributes (also called external
attributes)
Internal quality attributes
101
McCabe's source code metric
  • Cyclomatic complexity C on the control graph is C
    e - n 2p
  • Where e is edges, n is nodes, p is
    connected components
  • McCabe contends that well-structured modules
    (i.e. high quality) have C in range 3 .. 7, and C
    10 is a reasonable upper limit for the
    cyclomatic complexity of a single module
  • confirmed by empirical evidence

102
Halstead's software science
  • Tries to measure some software qualities by
    measuring some quantities on code, such as
  • n1, number of distinct operators in the program
  • n2, number of distinct operands in the program
  • N1, number of occurrences of operators in the
    program
  • N2, number of occurrences of operands in the
    program

N n1 log2 n1 n2 log2 n2 (length of the
program) V (N1N2) log2 (n1n2) (volume of the
program) --- error in Pressman, N instead of N1N2
103
Halstead's software science
  • Other than software qualities, quantities on code
    can be used to estimate interesting features of
    code
  • Mental Effort (effort required to understand and
    further develop a program)
  • E (n1) (N2) (N1N2) log2(n1n2) / 2(n2)
  • Estimated Number of Defects
  • B E(2/3) / 3000

104
Esempio
if ( A gt 1) and ( B 0 ) then X X / A if
( A 2 ) or ( X gt 1) then X X 1
n1 6 (inclusi operatori logici) N1 8 n2
3 N2 7
E(n1) (N2) (N1N2) log2(n1n2) / 2(n2) E6 7
(87) log2(63)/2 3 333 (circa)
105
Conclusioni
  • Testing in generale (anche in relazione con la
    verifica e la validazione e la più generale
    assicurazione di qualità, distinta in previsione
    della qualità e valutazione della qualità)
  • Testing Componenti Convenzionali (Black White
    box)
  • Tecniche statiche (di Verifica) e Conventional
    Unit Code Testing (Inspection, Walkthrougth,
    Symbolic Execution, Correcteness Proof)
  • Testing Componenti Object-Oriented
  • Testing in the large (Integration and System
    Testing)
  • Testing attributi soggettivi di qualità (diversi
    da correttezza, affidabilità, robustezza, safety
    e prestazioni)
About PowerShow.com