Towards Destructive Software Testing

1
Towards Destructive Software Testing

• Kiumi Akingbehin, PhD
• Department of Computer and Information Science
• University of Michigan-Dearborn

By Michael J. Dudley
2
Introduction

• Conventional Software Testing Definitions
• the activity or process which shows or
  demonstrates that a program or system performs
  all intended functions correctly
• the activity of establishing the necessary
  confidence that a program or system does what
  it is supposed to do, based on the specifications
  and requirements set by the user

3
Introduction

• Conventional Software Testing Definitions
• the process of executing a program/system with
  the intent of finding errors
• any activity aimed at evaluating an attribute or
  capability of a program or system and determining
  that it meets its required results

4
Destructive Software Testing (DST)

• DST does not conform to any of the previous
  definitions proposed.
• DST can be performed without knowledge of the
  original requirements of a software product.
• Although some knowledge of the requirements may
  help in developing a good comprehensive
  destructive testing strategy.

5
Basic Principles of DST

• Is not a replacement for conventional testing.
• Should be performed in addition to conventional
  testing.
• Acknowledges the fact that users of a software
  product will sometimes not use the software
  correctly.
• Software Misuse

6
Software Misuse

• Improper or incorrect
• Input data will be supplied
• Letters entered when numbers are expected.
• Commands will be typed
• Commands that the software was not written to
  handle.
• GUI sequences will be applied
• Alternate path that wasnt expected.

7
Destructive Testing Adage

• Adage
• a proverb or wise saying made familiar by long
  use
• An popular adage
• garbage in, garbage out
• This adage is not good enough for high quality,
  robust, and reliable software.
• A better adage for DST
• garbage in, proper predictable behavior out.
• DST improves the quality of a software product.

8
Destructive Testing Terminology

• Destructive Hardware Testing (DHT)
• Hardware systems are destroyed as part of testing
• Example Automobile safety testing
• The usual practice is to subject the automobile
  to an actual accident where the vehicle is
  heavily damaged or destroyed
• DST was chosen to be in compliance with
  corresponding concept of DHT

9
DST Definitions

• DST is testing that
• assures proper or predictable software behavior
  when the software is subject to improper usage or
  improper input
• attempts to crash a software product
• tries to crack or break a software product
• checks the robustness of a software product

10
Destructive Testing Confusion

• The Term Destructive Testing
• Should not be confused with the same term used
  for conventional software testing
• Has been used in the past to indicate software
  that fails conventional testing

11
Conventional TestingStrategies to DST

• Strategies that can be used for DST

• Black Box Testing
• Top-Down Testing
• Regression Testing
• Interface Testing
• Equivalence Partitioning
• Test Cases
• Quality Assurance
• Quality Control

• Acceptance Testing
• Boundary Value Testing
• Loop Testing
• Stress Testing
• Alpha Testing
• Beta Testing
• System (High Order) Test

12
Conventional TestingStrategies to DST

• Strategies that can, with modifications, be used
  for DST

• White Box Testing
• Bottom-Up Testing
• Basis Path Testing
• Security Testing
• Defect Testing

• Smoke Testing
• Performance Testing
• Unit (Module) Testing
• Integration Testing
• Object-Oriented Testing

13
Conventional TestingStrategies to DST

• Strategies that cannot be used for DST
• Verification Testing
• Validation Testing
• Benchmark Testing
• For more information on testing strategies
• Hetzel, Bill, The Complete Guide to Software
  Testing, John Wiley, 1988.
• Myers, G. J., The Art of Software Testing, John
  Wiley, New York, 1979.
• Pressman, R. S, Adaptable Process Model Software
  Test Specification, www.rspa.com, 2005.

14
Incorporation of DST into Specifications

• Requirements for a software system can be written
  to promote DST
• Such requirements, by definition, are
  non-functional
• Functional requirements by nature fall into the
  category of conventional testing that can not be
  tested using destructive testing
• Verification Testing
• Validation Testing
• Benchmark Testing

15
Incorporation of Destructive Testinginto
Specifications

• Mandatory destructive testing clauses
• The software must be designed in such a way that
  proper behavior is obtained when the software is
  subject to improper usage or improper input.
• No data shall be lost if the software prematurely
  terminates as a result of other system failures.
• The software shall not prematurely or
  unintentionally terminate as a result of any
  combination of user keyboard or mouse input.

16
Incorporation of Destructive Testinginto
Specifications

• Mandatory destructive testing clauses
• The software shall never accept or process
  invalid input data.
• The software shall always produce proper output
  data regardless of the validity or correctness of
  input data.

17
Incorporation of Destructive Testinginto
Specifications

• For Specific software products, it is important
  to explicitly define
• proper software behavior
• improper software behavior
• improper usage
• improper input data
• proper output data

18
Future work

• Author is currently working with a team to
  develop requirements specification for an example
  case study involving a data conversion program.
• Subsequent to the completion of the requirements
  specification and implementation of the software,
  test cases will be developed for destructive
  testing of the software.

19
Conclusion

• The goal of conventional software testing is to
  ensure a software product correctly performs all
  the functions specified in the requirements
  specification.
• The goal of destructive testing is to ensure a
  software product exhibits proper behavior when
  subject to improper usage or improper input.
• Ongoing work includes the development of
  requirements specification that mandate
  destructive testing of a case study software
  product.

20
Conclusion

• Destructive testing is a reflection of the fact
  that user will sometimes use a software product
  in an improper manner.
• Destructive testing does not replace conventional
  testing
• Supplements conventional testing
• Performed in addition to conventional testing
• Destructive testing cannot be detrimental
• Destructive testing can only be beneficial.

21
Questions