PROPANE An Environment for Examining the Propagation of Errors in Software

1
PROPANEAn Environment for Examining
thePropagation of Errors in Software

• Martin Hiller, Arshad Jhumka, Neeraj Suri
• Chalmers University of Technology
• Göteborg, Sweden
• hiller, arshad, suri_at_ce.chalmers.se

2
Overall Objectives

Software reliability can be provided/increased by
adding mechanisms that detect and correct data
errors ? wrappers, assertions, etc. (especially
for black-box software)
A
C
E
D
B

F
Given a software system and limited resources one
would like to concentrate work on the most
vulnerable/exposed parts of the software, i.e.
? Where do upcoming errors propagate?
3
Outline

• Error Propagation Basic Approach
• Examples of results obtained by using PROPANE
• Aircraft arrestment system
• Overview of PROPANE
• Tool suite
• Requirements limitations
• Conclusions some future directions

4
Error Propagation
Error A system state which is different from
the state in a correct execution of the system
(i.e, not mutations or software defects)
C
A
E
B
D
F
Modules in a software system have different
levels of exposure and different ability to
break error propagation (i.e., different levels
of error containment) examining error
propagation gives the developer a picture/profile
of these levels.
5
Basic Approach ? Error Injection

• Generate Golden Run, i.e., an error free
  reference run
• Generate Injection Run, i.e., a run in which an
  error (i.e. erroneous system state) is injected
• Compare Injection Run with Golden Run to see
  which parts of the system were corrupted by the
  injected error

A
C
E
D
B
F
probe
injection location
PROPANE Propagation Analysis Environment
? By instrumenting the target software, PROPANE
can, during execution, log individual variables
and events and inject errors into individual
variables.
6
Example Aircraft Arrestment System
Cable
Tape drum (original)
Tape drum (mirror)
ms_slot_nbr
i
CLOCK
CALC
mscnt
pulscnt
slow_speed
PACNT
stopped
DIST_S
TIC1
TCNT
Rotation sensor
Pressure sensor
Pressure valve
Pressure valve
SetValue
OutValue
TOC2
ADC
PRES_A
V_REG
PRES_S
IsValue
Computer
Target system overview
Target software overview
7
Results Generated by PROPANE

• From low level to high level information
• Signal/Variable Plots
• Propagation Signatures
• Propagation Graphs
• Propagation Summaries
• Data compilations that can be used for further
  analysis

8
Signal/Variable Plots
One-cycle single-bit upset at t 1500 ms
SetValue erroneous at t 1539 ms
9
Propagation Signatures

• Each injected error generates a Propagation
  Signature showing when and where the error
  propagated

10
Propagation Graphs
Incoming errors from locations earlier in the
propagation path
error counttmin / tavg / tmax
Probedlocation
Outgoing errors to locations later in the
propagation path

• The Propagation Graph is generated in three
  different formats
• PROPANE native format
• GML (Graph Markup Language)
• dot (part of the graphviz tool suite from ATT
  Research)

11
Propagation Graph for PACNT
Starting point
Propagation path
Each arc carries information regarding number of
propagated errors and propagation time
Variables along the propagation trajectory
12
Propagation Summary for PACNT
Probed location(variable)

• For all incoming arcs to one node
• Total error count and error rate
• Combined tmin / tavg / tmax

Propagation Graphs and Propagation Summaries
indicate which parts of the observed system state
that are most vulnerable to the injected errors?
perhaps a good location for error
detection/recovery mechanisms?
13
Error Propagation from PACNT
ms_slot_nbr
i
Highest Error Rate
CLOCK
CALC
mscnt
pulscnt
Lowest Error Rate
slow_speed
PACNT
stopped
DIST_S
TIC1
TCNT
SetValue
OutValue
PRES_S
PRES_A
TOC2
ADC
V_REG
IsValue
14
Overview
15
PLPROPANE Library

• Static C Library providing the injection/logging
  API
• propane_inject() / Injects an error /
• propane_log_var() / Variable probe /
• propane_log_event() / Event probe /
• Target must be instrumented with these functions
  and linked with the library
• Experiments are then set up using description
  files
• Error types (what errors to inject, e.g.,
  bit-flips)
• Error triggers (when to inject errors, e.g.,
  once/periodically)
• Active probes (perhaps not all instrumented
  variables/events have to be logged)

16
Instrumentation Example
Original code
Instrumented code
int spherical_volume( double radius )
double volume volume 4.0 (PI
pow(radius, 3.0)) / 3.0 return volume
int spherical_volume( double radius )
double volume / Injection location for
radius / propane_inject( IL_SPHERE_VOL,
radius,
PROPANE_DOUBLE ) / Probe the value of
radius / propane_log_var( P_RADIUS,
radius ) volume 4.0 (PI
pow(radius, 3.0)) / 3.0 / Probe the value
of volume / propane_log_var( P_VOLUME,
volume ) return volume
At this point, instrumentation is unfortunately
still a manual task. However, automation is the
next step in the development of PROPANE.
17
PSCPROPANE Setup Creator

• Once the target is instrumented, experiments can
  be set up
• Given information on injection locations, probes,
  error types and occurrences, PSC will create
• description files for running experiments
• description files for data extraction and analysis

18
PCDPROPANE Campaign Driver

• After instrumentation and setup, the PCD runs the
  experiments
• Invokes instrumented target executable containing
  the PL and all links to external modules, e.g.,
  environment simulator
• Provides continuous information on experiment
  status and remaining work
• User control
• Pause
• Abort
• Skip

19
PDEPROPANE Data Extractor

• Extracts and analyses data from the raw readouts
  produced during experiments
• Perform Golden Run Comparisons, i.e., compare an
  injection run with a clean reference run to
  trace errors, generating a Propagation Signature
• Collapse multiple propagation signatures into
  Propagation Graphs and Propagation Summaries
• Prepare/compile data for further external
  analysis (using e.g. MatLab)

20
Requirements Limitations

• Programming language
• The library itself is written in C ? the language
  in the target system must be able to interface
  with C-libraries
• Environment simulator
• Stimuli to the target system must be provided by
  an environment simulator
• PROPANE provides basic interfacing functionality
  such that control of the EnvSim can be made part
  of the experiment setup (plug-ins)
• Target instrumentation
• Difficulties with real-time applications that
  cannot be made to run in simulated time
• Current implementation focused towards
  single-node SW
• Distributed SW not considered at this point

21
Conclusions

• PROPANE enables experimental evaluation of error
  propagation in SW
• Propagation profiles indicate which parts of
  system state that are vulnerable ? aid for
  equipping SW with error detection and recovery
  mechanisms
• We envision that PROPANE can be used as a design
  stage aid for analysis of single node SW
• Limitations apply for some apsects of real-time
  and distributed SW

22
Some Future Directions

• Increased automation
• E.g. automatic instrumentation of target software
• Extended analysis capabilities
• E.g. provide your own analysis plug-ins
• Open source
• Possibly in the near future

23
Further Information(and future download area)

• www.ce.chalmers.se/LDC/DEEDS