Designing User Interfaces Spring 1999 - PowerPoint PPT Presentation

1 / 52
About This Presentation
Title:

Designing User Interfaces Spring 1999

Description:

Logic problems. Math errors. Non-testable requirement or paths. Weaknesses ... Logic and structure problems. Math and data errors. Weaknesses. Interface and ... – PowerPoint PPT presentation

Number of Views:34
Avg rating:3.0/5.0
Slides: 53
Provided by: robos
Category:

less

Transcript and Presenter's Notes

Title: Designing User Interfaces Spring 1999


1
SE 746-NT Embedded Software Systems
Development Robert Oshana Lecture
35 For more information, please
contact NTU Tape Orders NTU Media
Services (970) 495-6455
oshana_at_airmail.net
tapeorders_at_ntu.edu
2
Embedded Systems Testing Part 2
From Beatty ESC 2002
3
Agenda
  • More verification techniques
  • Code inspection checklist
  • Unit test standards

4
Testing
  • Organized process of identifying variances
    between actual and specified results
  • Goal zero significant defects

5
Testing axioms
  • All software has bugs
  • Programs cannot be exhaustively tested
  • Cannot prove the absence of all errors
  • Complex systems often behave counter-intuitively
  • Software systems are often brittle

6
Finding spec/design problems
  • Reviews / Inspections / Walkthroughs
  • CASE tools
  • Simulation
  • Prototypes

Still need consistently effective methods !
7
Testing Spec/Design Reviews
  • Can be formal or informal
  • Completeness
  • Consistency
  • Feasibility
  • Testability

8
Testing Evaluating methods
  • Relative costs
  • None
  • Low
  • Moderate
  • High
  • General effectiveness
  • Low
  • Moderate
  • High
  • Very high

9
Testing Code reviews
  • Individual review
  • Effectiveness high
  • Cost Time low, material - none
  • Group inspections
  • Effectiveness very high
  • Cost Time moderate, material - none

10
Testing Code reviews
  • Strengths
  • Early detection of errors
  • Logic problems
  • Math errors
  • Non-testable requirement or paths
  • Weaknesses
  • Individual preparation and experience
  • Focus on details, not big picture
  • Timing and system issues

11
Step by step execution
  • Exercise every line of code or every branch
    condition
  • Look for errors
  • Use simulator, ICE, logic analyzer
  • Effectiveness moderate dependent on tester
  • Cost time is high, material is low or moderate

12
Functional (Black Box)
  • Exercise inputs and examine outputs
  • Test procedures describe expected behavior
  • Subsystems tested and integrated
  • Effectiveness is moderate
  • Cost time is moderate, material varies

Tip where functional testing finds problems look
deeper in that area !
13
Functional (Black Box)
  • Strengths
  • Requirements problems
  • Interfaces
  • Performance issues
  • Most critical/most used features
  • Weaknesses
  • Poor coverage
  • Timing and other problems masked
  • Error conditions

14
Functional test process
  • ID requirements to test
  • Choose strategy
  • 1 test per requirement
  • Test small groups of requirements
  • Scenario broad sweep of many requirements
  • Write test cases
  • Environment
  • Inputs
  • Expected outputs
  • Traceability

15
Structural (White box)
  • Looks at how code works
  • Test procedures
  • Exercise paths using many data values
  • Consistency between design and implementation
  • Effectiveness high
  • Cost time is high, material low to moderate

16
Structural (White box)
  • Strengths
  • Coverage
  • Effectiveness
  • Logic and structure problems
  • Math and data errors
  • Weaknesses
  • Interface and requirements
  • Focused may miss big picture
  • Interaction with system
  • Timing problems

17
Structural (White box)
  • Test rigor based on 3 levels of Risk (FAA)
  • C reduced safety margins or functionality
  • Statement coverage
  • Invoke every statement at least once

18
Structural (White box)
  • Test rigor based on 3 levels of Risk (FAA)
  • B Hazardous Decision Coverage
  • Invoke every statement at least once
  • Invoke every entry and exit
  • Every control statement takes all possible
    outcomes
  • Every non-constant Boolean expression evaluated
    to both a True and a False result

19
Structural (White box)
  • Test rigor based on 3 levels of Risk (FAA)
  • A Catastrophic Modified Condition Decision
    Coverage
  • Every statement has been invoked
  • Every point of entry and exit has been invoked

20
Structural (White box)
  • Every control statement has taken all possible
    outcomes
  • Every Boolean expression has evaluated to both a
    True and a False result
  • Every condition in a Boolean expression has
    evaluated to both True/False
  • Every condition in a Boolean expression has been
    shown to independently affect that expressions
    outcome

21
Unit test standards
  • What is the white box testing plan?
  • What do you test?
  • When do you test it?
  • How do you test it?

22
Structural test process
  • ID all inputs
  • ID all outputs
  • ID all paths
  • Set up test cases
  • Decision coverage
  • Boundary value analysis
  • Checklist
  • Weaknesses

23
Structural test process
  • Measure worst case execution time
  • Determine worst case stack depth
  • Bottom up

24
Integration
  • Combines elements of white and block box
  • Unexpected return codes or acknowledgements
  • Parameters boundary values
  • Assumed initial conditions/state
  • Unobvious dependencies
  • Aggregate functionality

25
Integration
  • Should you do thiswhen?
  • Depends on the complexity of the system
  • Boundary values of parameters in functions
  • Interaction between units
  • interesting paths
  • Errors
  • Most common

26
Verification
  • Verify the structural integrity of the code
  • Find errors hidden at other levels of examination
  • Outside of requirements
  • Conformance to standards

27
Verification
  • Detailed inspection, analysis, and measurement of
    code to find common errors
  • Examples
  • Stack depth analysis
  • Singular use of flags/variables
  • Adequate interrupt suppression
  • Maximum interrupt latency
  • Processor-specific constraints

28
Verification
  • Strengths
  • Finds problems that testing and inspection cant
  • Stack depth
  • Resource sharing
  • Timing
  • Weaknesses
  • Tedious
  • Focused on certain types of errors

29
Verification
  • Customize for your process/application
  • What should be checked
  • When
  • How
  • By whom

30
Stress/performance
  • Load the system to maximumand beyond!
  • Helps determine factor of safety
  • Performance to requirements

31
Stress/performance
  • Examples
  • Processor utilization
  • Interrupt latency
  • Worst time to complete a task
  • Periodic interrupt frequency jitter
  • Number of messages per unit time
  • Failure recovery

32
Other techniques
  • Fault injection
  • Scenario testing
  • Regression
  • Critical functions
  • Most functionality with the least tests
  • Automation
  • Risk of not re-testing is higher than the cost
  • Boundary value testing

33
Tools
ICE Simulator Logic analyzer
Step through code X X X
Control execution X X
Modifying data X X
Coverage X X X
Timing analysis X X X
34
Code Inspection Checklist
35
Code Inspection Checklist
  • Code correctly implements the document software
    design
  • Code adheres to coding standards and guidelines
  • Code is clear and understandable
  • Code has been commented appropriately
  • Code is within complexity guidelines
  • Cyclomatic complexity lt 12

36
Code Inspection Checklist
  • Macro formal parameters should not have side
    affects (lint message 665)
  • Use parenthesis to enhance code robustness, use
    parenthesis around all macro parameters (665,
    773)
  • Examine all typecasts for correct operation
  • Examine affects of all implicit type conversions
    (910-919)

37
Code Inspection Checklist
  • Look for off-by-one errors in loop counters,
    arrays, etc
  • Assignment statements within condition
    expressions (use cchk)
  • Guarantee that a pointer can never be Null when
    de-referencing it
  • Cases within a switch should end in a break (616)

38
Code Inspection Checklist
  • All switch statements should have a default case
    (744)
  • Examine all arguments passed to functions for
    appropriate use of pass by value, pass by
    reference, and const
  • Local variables must be initialized before use
  • Equality test on floating point numbers may never
    be True (777)

39
Code Inspection Checklist
  • Adding and subtracting floats of different
    magnitudes can result in lost precision
  • Insure that division by zero cannot occur
  • Sequential multiplications and divisions may
    produce round-off errors

40
Code Inspection Checklist
  • Subtracting nearly equal values can produce
    cancellation errors
  • C rounds towards zero is this appropriate here
    ?
  • Mathematical underflow/overflow potential
  • Non-deterministic timing constructs

41
Unit test standards
42
Unit test standards
  • 1. Each test case must be capable of independent
    execution, i.e. the setup and results of a test
    case shall not be used by subsequent test cases
  • 2. All input variables shall be initialized for
    each test case. All output variables shall be
    given an expected value, which will be validated
    against the actual result for each test case

43
Unit test standards
  • 3. Initialize variables to valid values taking
    into account any relationships among inputs. In
    other words, if the value of a variable A affects
    the domain of variable B, select values for A and
    B which satisfy the relationship
  • 4. Verify that the minimum and maximum values can
    be obtained for each output variable (i.e. select
    input values that produce output values as close
    to the max/min as possible)

44
Unit test standards
  • 5. Initialize output variables according to the
    following
  • If an input is expected to change, set its
    initial value to something other than the
    expected result
  • If an output is not expected to change, set its
    initial value to its expected value
  • 6. Verify loop entry and exit criteria

45
Unit test standards
  • 7. Maximum loop iterations should be executed to
    provide worst case timing scenarios
  • 8. Verify that the loss of precision due to
    multiplication or division is within acceptable
    tolerance

46
Unit test standards
  • 9. The following apply to conditional expressions
  • OR expressions are evaluated by setting all
    predicates FALSE and then setting each one
    TRUE individually
  • AND expressions are evaluated by setting all
    predicates TRUE and then setting each one
    FALSE individually

47
Unit test standards
  • 10. Do not stub any functions that are simple
    enough to include within the unit test
  • 11. Non-trivial tests should include an
    explanation of what is being tested

48
Unit test standards
  • 12. Unit test case coverage is complete when the
    following criteria are satisfied (where
    applicable)
  • 100 function and exit coverage
  • 100 call coverage
  • 100 statement block coverage
  • 100 decision coverage
  • 100 loop coverage
  • 100 basic condition coverage
  • 100 modified condition coverage

49
Unit test checklist
Common coding error checks Status Note(s)
Mathematical expression underflow/overflow ltPass/Fail/NAgt
Off-by-one errors in loop counters ltPass/Fail/NAgt
Assignment statements within conditional expressions ltPass/Fail/NAgt May be detected by compiler, lint, cchk
Floats are not compared solely for equality ltPass/Fail/NAgt Lint message 777
Variables and calibrations use correct precision and ranges in calculations ltPass/Fail/NAgt
50
Unit test checklist
Common coding error checks Status Note(s)
Pointers initialized and de-referenced properly ltPass/Fail/NAgt
Intermediate calculations are not stored in global variables ltPass/Fail/NAgt
All declared local variables are used in the function ltPass/Fail/NAgt May be detected by compiler or lint
Typecasting has been done correctly ltPass/Fail/NAgt
Unreachable code has been removed ltPass/Fail/NAgt Lint message 527
51
Common coding error checks Status Note(s)
All denominators are guaranteed to be zero (no divide by 0) ltPass/Fail/NAgt
Switch statement handle every case of the control variable (have DEFAULT paths). Any cases that fall through to the next case are intended to do so ltPass/Fail/NAgt Lint message 744, 787 Fall through 616
Static variables are used for only one purpose ltPass/Fail/NAgt
All variables have been properly initialized before being used assume a value of 0 after power up ltPass/Fail/NAgt
52
SE 746-NT Embedded Software Systems
Development Robert Oshana 10 minute
break For more information, please
contact NTU Tape Orders NTU Media
Services (970) 495-6455
oshana_at_airmail.net
tapeorders_at_ntu.edu
Write a Comment
User Comments (0)
About PowerShow.com