2.2 Software Myths

1
2.2 Software Myths

• Myth 1. The cost of computers is lower than that
  of analog or electromechanical devices.
• Hardware is cheap compared to other
  electromechanical devices
• However cost of software, with reliability and
  maintenance, is enormous
• e.g. Space-Shuttle software has 400,000 words
  (relatively small) but costs NASA approximately
  100,000,000 a year to maintain.
• Software Costs can become exorbitant over time.

2
Myth 2. Software is easy to change.

• Yes changes are easy to make -- but hard to make
  without introducing errors.
• Every change must be verified and rectified
• Becomes more brittle with changes
• We become hesitant to change software over time
  -- recognizing

3
Myth 3 Cont.

• Little available data on software reliability vs.
  non-computer systems
• British Royal Signals and Radar Establishment
  analyzed software for highly safety critical
  purposes.
• 10 of modules or functions deviated from
  original design.
• Deviations found even in tested software.
• 1 in 200 new modules had errors with observable
  effects on performance.
• Integer overflow errors.
• Complete error elimination is a hard and lofty
  goal to achieve.

4
Myth 3 Cont.

• These are not just teething problems but
  chronic ones over tens or hundreds of hours of
  use
• e.g. (1) Therac-25 worked correctly thousands of
  times before first know overdose occurred.
• (2) Space Shuttle -- NASA invested
  enormous effort and resources since 1980
• yet 16 severity-level 1 software errors have been
  discovered (errors that would result in loss of
  shuttle or crew
• 8 errors remained in code used in flights, though
  not encountered

5
Myth 3/ Cont.

• 12 errors with lower severity triggered during
  flight 3 threatened mission, 9 had to be worked
  around
• ALL DESPITE THE SOPHISTICATION OF NASAs software
  development and verification program.

6
Myth 3/ Cont.

• Redundancy is not a solution as in the case of
  hardware wearout.
• Zero-Defect software is false claim.
• Usually not enough time to perfect software
  costs also severe.
• Computers may be more reliable but not
  necessarily safer.

7
Myth 4 Increasing software
reliability will increase safety

• Software errors may not be related to safety at
  all
• Compliance with requirements specification may
  not remove errors
• Safety-critical software errors can often be
  traced to Requirements
• That is, software is doing exactly what it is
  supposed to do.
• Software may be correct and 100 reliable -- yet
  responsible for serious accidents.
• RELIABILITY DOES NOT EQUAL SAFETY

8
Myth 5 Testing software or proving (using
formal verification techniques) software correct
can remove all the errors.

• Software limitations well known
• Exhaustive testing is impossible
• Only a relatively small part of the state space
  can be covered
• Despite improved testing techniques, no
  breakthroughs
• Mathematical proofs advanced -- but even
  arguments for impossibility of complete proof of
  correctness
• Mathematical verification of software may be
  possible in the future.

9
Myth 5 /cont.

• Correct behavior of software must be specified in
  a formal mathematical language.
• May be as difficult and error-prone as the code.
• Software errors often involve overload -- outside
  the realm of specification
• Intricate software interactions complicate the
  issue.
• In summary, most safety-related software errors
  can be traced to the requirements

10
Myth 6 Reusing Software increases safety

• Reuse of proven software may increase
  reliability, but has little or no effect on
  safety
• May even decrease safety because of the
  complacency it engenders
• Specific hazards of new implementation may not
  have been considered
• Examples include

11
Therac-20 parts reused for Therac-25 with same
error, but causing two deaths

• Error did not have serious consequences in
  Therac-20.
• Resulted in occasional blown fuse -- not massive
  overdose
• Never detected or fixed in Therac-20

12
Air Traffic Control Software

• Successful in US for many years but not in Great
  Britain
• Was not developed for zero degrees longitude
  along the Greenwich Meridian
• Manchester plopped on top of Warwick

13
Aviation Software written for Northern Hemisphere
has problems in Southern Hemisphere

• Software written for US F-16s caused problems
  when reused in aircraft flown over the Dead Sea
  where altitude is below sea level.
• Safety is not a property of software alone, but
  of the software design, and environment where
  software is used.
• Application, environment, and system-specific.

14
Myth 7 Computers reduce risk over mechanical
systems

• Argument 1-- can check parameters through finer
  control more often
• Counter -- Finer control allows operation under
  smaller safety margins
• No way to test adequately

15
Myth 7 /cont.
Argument 2-- automated system allow operators to
work farther away from hazardous areas. More
accidents due to operators entry into 7 - 7
environment. Humans enter unsafe hazardous
environments Became unsafe to enforce robot
shutdown protocols
16
Myth 7/cont.

• Argument 3 -- eliminating operators eliminates
  human errors
• Argument 4 -- Computers have the potential to
  provide better information to operators and
  thus improve decision making
• Argument 5 -- Software does not fail.