Software Reliability Growth Models Incorporating Fault Dependency with Various Debugging Time Lags

1
Software Reliability Growth Models Incorporating
Fault Dependency with Various Debugging Time Lags

• Chin-Yu Huang, Chu-Ti Lin, Sy-Yen Kuo, Michael R.
  Lyu, and Chuan-Ching Sue
• Computer Science Engineering Department
  
• The Chinese University of Hong Kong
• Shatin, Hong Kong

The 28th Annual International Computer Software
and Application Conference (COMPSAC 2004)
2
Outline

• Introduction
• Reviews of software fault detection and
  correction process
• Considering failure dependency debugging time
  lag in software reliability modeling
• Numerical examples
• Conclusions

3
Introduction

• The software is becoming increasingly complex and
  sophisticated. Thus reliability will become the
  main goal for software developers.
• Software reliability is defined as the
  probability of failure-free software operation
  for a specified period of time in a specified
  environment.
• Over the past 30 years, many Software Reliability
  Growth Models (SRGMs) have been proposed for
  estimation of reliability growth of products
  during software development processes.

4
Introduction (contd.)

• One common assumption of conventional SRGMs is
  that detected faults are immediately removed.
• In practice, this assumption may not be realistic
  in software development.
• The time to remove a fault depends on the
  complexity of the detected faults, the skills of
  the debugging team, the available man power, or
  the software development environment, etc.
• Therefore, the time delayed by the detection and
  correction processes should not be neglected.

5
Introduction (contd.)

• On the other hand, there are two types of faults
  in a software system mutually independent faults
  and mutually dependent faults.
• mutually independent faults are on different
  program paths.
• mutually dependent faults can be removed only if
  the leading faults are removed.
• In this paper, we will incorporate the ideas of
  failure dependency and time-dependent delay
  function into software reliability growth
  modeling.

6
Basic Assumptions of Most SRGMs

• The fault removal process follows the Non-
  homogeneous Poisson Process (NHPP).
• The software system is subject to failures at
  random times caused by the manifestation of
  remaining faults in the system.
• All faults are independent and equally
  detectable.
• Each time a failure occurs, the corresponding
  fault is immediately and perfectly removed. No
  new faults are introduced.

7
Issues of Fault Detection and Correction
Processes in SRGMs

• It is noted that the assumption (4) may not be
  realistic in practice.
• Actually, most existing SRGMs can be
  reinterpreted as delayed fault-detection models
  that can model the software fault detection and
  correction processes.
• We can remove the impractical assumption that the
  fault-correction process is perfect and establish
  a corresponding time-dependent delay function to
  fit the fault-correction process.

8
Definition 1

• Given a fault-detection and fault-correction
  process, one defines the delay-effect factor,
  f(t), to be a time-dependent function that
  measures the expected delay in correcting a
  detected fault at any time.

9
Definition 2

• An SRGM is called a delayed-time NHPP model if it
  obeys the following assumptions
• The fault detection process follows the NHPP.
• The software system is subject to failures at
  random times caused by the manifestation of
  remaining faults in the system.
• All faults are independent and equally
  detectable.
• The rate of change of mean value function (MVF)
  of the number of detected faults is exponentially
  decreasing.
• The detected faults are not immediately removed.
  The fault correction process lags the fault
  detection process by a delay-effect factor f(t).

10
Fault Detection and Correction Processes in SRGMs

• Based on the above assumptions (1)-(4), the
  original MVF of NHPP model is
• where a is the expected number of initial
  faults, and r is the fault detection rate.
• From the assumption (5) in definition 2 and above
  equation, the new MVF can be depicted as
• We thus derive the following theorem.

11
Theorem 1

• Given a delay-factor, f(t),we have
• The fault-detection intensity of the
  delayed-time NHPP SRGM is
• where agt0 and rgt0.

12
Three Conventional SRGMs

• In the following, we will review three
  conventional SRGMs based on NHPP can be directly
  derived from Definition 1, Definition 2, and
  Theorem 1.
• Goel-Okumoto model
• Yamada delayed S-shaped model
• Yamada Weibull-type testing-effort function
  model

13
Goel-Okumoto model

• This model, first proposed by Goel and Okumoto ,
  is one of the most popular NHPP model in the
  field of software reliability modeling.
• If , then
  (Definition 1)
• we have
  (Theorem 1)
• mean value function (MVF) (Definition 2)

14
Yamada delayed S-shaped model

• The Yamada delayed S-shaped model is a
  modification of the NHPP to obtain an S-shaped
  curve for the cumulative number of failures
  detected such that the failure rate initially
  increases and later decays .
• If , then
  (Definition 1)
• we have
  (Theorem 1)
• MVF
  (Definition 2)
• 
  .

15
Yamada Weibull-type testing-effort function model

• Yamada et al. proposed a software reliability
  model incorporating the amount of test-effort
  expended during the software testing phase. The
  distribution of testing-effort can be described
  by a Weibull curve.
• If
  , then (Definition 1)
• a) we have
  (Theorem 1)
• b) Mean value function
  (Definition 2)
• 
  .

16
Considering Failure Dependency in Software Fault
Modeling

• Assumption
• The fault detection process follows the NHPP.
• The software system is subject to failures at
  random times caused by the manifestation of
  remaining faults in the system.
• The all detected faults can be categorized as
  leading faults and dependent faults. Besides, the
  total number of faults is finite.
• The mean number of leading faults detected in the
  time interval (t, t?t is proportional to the
  mean number of remaining leading faults in the
  system. Besides, the proportionality is a
  constant over time.

17
Considering Failure Dependency in Software Fault
Modeling (contd.)

• The mean number of dependent faults detected in
  the time interval (t, t?t) is proportional to
  the mean number of remaining dependent faults in
  the system and to the ratio of leading faults
  removed at time t and the total number of faults.
  Besides, the proportionality is a constant over
  time.
• The detected dependent fault may not be
  immediately removed and it lags the fault
  detection process by a delay-effect factor f(t).
  That is, f(t) is the time delay between the
  removal of the leading fault and the removal of
  the dependent fault(s).
• No new faults are introduced during the fault
  removal process.

18
Considering Failure Dependency in Software Fault
Modeling (contd.)

• From assumption (3) (4), we have a a1 a2 .
• Besides, a1 P a and a2 (1-P) a., and m(t)
  m1(t) m2(t)
• Where
• a expected number of initial faults.
• a1 total number of leading faults .
• a2 total number of dependent faults.
• P portion of the leading faults.
• m(t) the MVF of the expected number of initial
  faults.
• m1(t) the MVF of the expected number of leading
  
• faults detected in time (0, t.
• m2(t) the MVF of the expected number of
  dependent
• faults detected in time (0, t.

19
Considering Failure Dependency in Software Fault
Modeling (contd.)

• From assumption (4), the number of detected
  leading faults is proportional to the number of
  remaining leading faults. Thus we obtain the
  following differential equation.
• ,
  a1 gt0, 0ltrlt1.
• Solving the above differential equation under the
  boundary condition m1(t) 0, we have

20
Considering Failure Dependency in Software Fault
Modeling (contd.)

• Similarly, from assumption (5) (7), we obtain
  the following differential equation.
• In the following, we will give a detail
  description of possible behavior of f(t).

21
Case 1

• If , we obtain
• 
  
• ?
• From assumption (3), the MVF of the expected
  number of initial faults is

22
Case 2

• If , we obtain
• 
  
• ?
• From assumption (3), the MVF of the expected
  number of initial faults is

23
Case 3

• If , we
  obtain
• Similarly, we can obtain the MVF of the expected
  number of initial faults by means of the same
  steps.

24
Numerical Examples

• Two Real Software Failure Data Sets
• Ohbas data set (1984)
• Data set from RVLIS project in Taiwan (2003)
• Criteria for Models Comparison
• Mean Square of Fitting Error
• Mean Error of Prediction
• Noise
• Performance Analysis
• We only consider Case 1 Case 2 as illustration.
• Parameters of all models are estimated by using
  the method of LSE and MLE.

25
Data Description

• The first data set (DS1)
• The system was a PL/I database application
  software consisting of approximately 1,317,000
  lines of code. During 19 weeks, 47.65 CPU hours
  were consumed and about 328 software faults were
  removed.

26
Data Description (contd.)
Table 1 Number of failures per week from DS1.
Where CNF Cumulative number of failures.
27
Data Description (contd.)

• The second data set (DS2)
• RVLIS, a detection system used to monitor the
  level of water within the reactor vessel. The
  coding language is VersaPro 2.03 and the
  development platform is GE FANUC PLC 9030. It
  took 25 weeks to complete the test. During the
  test phase, 230 software faults were removed.

28
Data Description (contd.)
Table 2 Number of failures per week from DS2.
Where CNF Cumulative number of failures.
29
Criteria for Models Comparison (1)

• Noise
• where ri is the predicted failure rate.
• Mean Square of Fitting Error (MSE)
• where mi is the observed number of faults by time
  ti.
• A smaller MSE indicates a smaller fitting error
  and better performance.

30
Criteria for Models Comparison (2)

• Mean Error of Prediction (MEOP)
• where ni is the observed cumulative number of
  failures at time si and mi is the predicted
  cumulative number of failures at time si, ik,
  k1,, n.

31
Performance Analysis
Table 3 Comparison results of different SRGMs
for Case1-DS1.
32
Performance Analysis (contd.)

• Case 1 DS1
• The proposed model estimates P0.55. That is,
  the software may contain two categories of
  faults, 55 are leading faults and 45 are
  dependent faults.
• The possible values of P are also discussed and
  listed in Table 3.
• The proposed model provides the lowest MSE MEOP
  if compared to the Goel-Okumoto model and the
  Yamada delay S-shaped model.
• We can see that the proposed model provides the
  best fit and prediction for the second data set.

33
Performance Analysis (contd.)
Table 4 Comparison results of different SRGMs
for Case1-DS2.
34
Performance Analysis (contd.)

• Case 1 DS2
• The proposed model estimates P0.71. That is,
  the software may contain two categories of
  faults, 71 are leading faults and 29 are
  dependent faults.
• The possible values of P are also discussed and
  listed in Table 4.
• The proposed model provides the lowest MSE MEOP
  if compared to the Goel-Okumoto model and the
  Yamada delay S-shaped model.
• We can see that the proposed model provides the
  best fit and prediction for the second data set.

35
Performance Analysis (contd.)
Table 5 Comparison results of different SRGMs
for Case2-DS1.
36
Performance Analysis (contd.)

• Case 2 DS1
• The proposed model estimates P0.72 for this data
  set. And it indicates that the software may
  contain two categories of faults, 72 are leading
  faults and 28 are dependent faults.
• The possible values of P are also discussed and
  listed in Table 5.
• The proposed model almost provides the lowest
  MEOP if compared to the Goel-Okumoto model and
  the Yamada delay S-shaped model.
• Overall, the MVF of proposed model provides a
  good fit to this data.

37
Performance Analysis (contd.)
Table 6 Comparison results of different SRGMs
for Case2-DS2.
38
Performance Analysis (contd.)

• Case 2 DS2
• The proposed model estimates P0.77 for this data
  set. And it indicates that the software may
  contain two categories of faults, 77 are leading
  faults and 23 are dependent faults.
• The possible values of P are also discussed and
  listed in Table 6.
• On the other hand, we know that the inflection
  S-shaped model is based on the dependency of
  faults by postulating the assumption some of the
  faults are not detectable before some other
  faults are removed. Therefore, it may provide us
  some information for reference.

39
Performance Analysis (contd.)

• Case 2 DS2 (contd.)
• After the simulation, we find that the estimated
  value of inflection rate (which indicates the
  ratio of the number of detectable faults to the
  total number of faults in the software) is 0.598
  for DS2. It indicates that the growth curve is
  slightly S-shaped.
• On the average, the proposed model performs well
  in this actual data.

40
Conclusions

• In this paper, we incorporate both failure
  dependency and time-dependent delay function into
  software reliability assessment.
• Specifically, all detected faults can be
  categorized as leading faults and dependent
  faults.
• Moreover, the fault-correction process can be
  modeled as a delayed fault-detection process and
  it lags the detection process by a time-dependent
  delay.
• Experimental results show that the proposed
  framework to incorporate both failure dependency
  and time-dependent delay function for SRGM has a
  fairly accurate prediction capability.