Software Independent Verification and Validation IV - PowerPoint PPT Presentation

Loading...

PPT – Software Independent Verification and Validation IV PowerPoint presentation | free to download - id: 12fbc5-ZDJiM



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Software Independent Verification and Validation IV

Description:

Verification (Are we building the product right? ... Independent Verification & Validation ... methods sufficient to verify and validate software requirements? ... – PowerPoint PPT presentation

Number of Views:148
Avg rating:3.0/5.0
Slides: 37
Provided by: bja88
Learn more at: http://ses.gsfc.nasa.gov
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Software Independent Verification and Validation IV


1
Software Independent Verification and
Validation (IVV) NASA IVV Facility Fairmont,
West Virginia Judith N. Bruner Acting
Director 304-367-8202 judith.n.bruner.1_at_gsfc.nasa.
gov
2
Content
  • Why are we discussing IVV?
  • What is IVV?
  • How is IVV done?
  • IVV process
  • Why perform IVV?
  • Summary
  • Points of Contact

3
Why are we discussing IVV?

4
Setting the Stage
  • In the 90s, the Commanding General of theArmys
    Operational Test and Evaluation Agency noted that
    90 percent of systems that were not ready for
    scheduled operational tests had been delayed by
    immature software.

5
Software Chaos
The Standish Group examined 8,380 Software
Projects.
16 Successful
53 Challenged
- In Budget
-
Over budget
by 189
- On Time
- Late by 222
- Meets Requirements
- Missing 39 of Capabilities
- User involved
31 Cancelled
Note For Large Companies
- Development
- 9 were Successful
- 61.5 Challenged
-
Over budget
by 178
- Late by 230
- Missing 58 of Capabilities
- 29.5 were Cancelled
6
Error Densities
Design Implementation
23
Requirements Specification
Installation Commissioning
68
9
7
Increasing Cost of Changes
The cost to correct an software error multiplies
during the development lifecycle.
Cost scale factor
(Normalized to Requirements Phase)

8
What is IVV?

9
Independent Verification and Validation (IVV)
  • Independent
  • Technical IVV prioritizes its own efforts
  • Managerial Independent reporting route to
    Program Management
  • Financial Budget is allocated by program and
    controlled at high level such that IVV
    effectiveness is not compromised
  • Verification (Are we building the product
    right?)
  • The process of determining whether or not the
    products of a given phase of the software
    development cycle fulfill the requirements
    established during the previous phase
  • Is internally complete, consistent and correct
    enough to support next phase
  • Validation (Are we building the right product?)
  • The process of evaluating software throughout its
    development process to ensure compliance with
    software requirements. This process ensures
  • Expected behavior when subjected to anticipated
    events
  • No unexpected behavior when subjected to
    unanticipated events
  • System performs to the customers expectations
    under all operational conditions

10
Independent Verification Validation
  • Software IVV is a systems engineering process
    employing rigorous methodologies for evaluating
    the correctness and quality of the software
    product throughout the software life cycle
  • Adapted to characteristics of the
    target program

11
How is IVV done?

12
IVV Activities Throughout Lifecycle
13
IVV Life Cycle Functions
  • IVV Process provides tools and analysis
    procedures appropriate to each phase of the
    software development life cycle
  • Formulation Phase
  • Is development process sound, repeatable, and
    managed?
  • Requirements Phase
  • Verify that system and software requirements are
    correct, complete, traceable and testable
  • Analyze system-level requirements Are test
    plans and acceptance criteria sufficient to
    validate system requirements and operational
    needs?
  • Are testing methods sufficient to verify and
    validate software requirements?
  • Are the correct software development, management,
    and support processes in place?
  • Design Phase
  • Does the design support the requirements?
  • Are test plans and test environments sufficient
    to verify and validate software and operational
    requirements?
  • Does the design have any characteristics that
    will cause it to fail under operational
    scenarios? What solutions are appropriate?

14
IVV Life Cycle Functions (cont.)
  • Typical IVV functions by Software life-cycle
    phase (cont.)
  • Coding Phase
  • Does the code reflect the design?
  • Is the code correct?
  • Verify that test cases trace to and cover
    software requirements and operational needs
  • Verify that software test cases, expected
    results, and evaluation criteria fully meet
    testing objectives
  • Analyze selected code unit test plans and results
    to verify full coverage of logic paths, range of
    input conditions, error handling, etc.
  • Test Phase
  • Analyze correct dispositioning of software test
    anomalies
  • Validate software test results versus acceptance
    criteria
  • Verify tracing and successful completion of all
    software test objectives
  • Operational Phase
  • Verify that regression tests are sufficient to
    identify adverse impacts of changes

15
IVV Testing Involvement
  • IVV identifies deficiencies in programs test
    planning
  • Program changes their procedures to address
    deficiencies vice IVV independently test
  • IVV may independently test highly critical
    software using an IVV testbed
  • Whitebox
  • Stress
  • Endurance
  • Limit
  • Developer motivated to show software works
  • IVV attempts to break software

16
IVV Process

17
IVV Process
Integrates IVV into program Provides IVV
funding Resolves Exception issues
Reflects IVV in program mgmt plan Agrees to data
transfer plan Reflects agreement in subcontracts
IVV in phase with development
18
IVV Scope
  • Scope is determined so as to minimize the risk
    within the Programs IVV budget. Effort is
    based on
  • Criticality and risk of system functions
    performed/managed by software
  • Budget limitations

Programs IVV budget
19
CARA Scoring Methodology
20
CARA Criticality
Sample Criticality Evaluation Criteria
21
CARA Risk
Sample Risk Driver Criteria
22
Requirements Analysis IALs
23
Design Analysis IALs
24
Code Analysis IALs
25
Test Analysis IALs
26
IVV Is Process As Well As Product Oriented
27
IVV Increases Program Awareness
28
Staffing Paradigm
29
Why perform IVV?

30
IVV Benefits
Technical
Management
31
Summary

32
IVV Key Points
  • IVV works with the Project
  • Goal is project success
  • IVV is an engineering discipline
  • IVV processes are defined and tailored to the
    specific program
  • Mission, operations and systems knowledge is used
    to perform engineering analyses of system
    components
  • IVV is most effective when started early
  • 70 of errors found in testing are traceable to
    problems in the requirements and design
  • IVV works problems at the lowest possible level
  • Primarily work via established informal
    interfaces with the development organization -
    working groups, IPTs, etc.
  • Elevate issues only when necessary

33
IVV Approach Efficiently Mitigates Risk
  • It is not necessary or feasible to perform all
    IVV analyses on all software functions
  • IVV resources allocated to reduce overall
    exposure to operational, development, and
    cost/schedule risks
  • Software functions with higher cirticality and
    development risk receive enhanced levels of
    analysis (CARA process)
  • Systems analyses performed to reduce costly
    interface and integration problems
  • Process analyses performed to verify ability to
    produce desired result relative to program plans,
    needs and goals
  • IVV working interfaces promote timely problem
    resolution
  • Proactive participation on pertinent development
    teams
  • Emphasis on early identification of technical
    problems
  • Engineering recommendations provided to expedite
    solution development and implementation

34
Analyses Are Value Added and Complementary - Not
Duplicative
  • Analyses performed from a systems perspective
    considering mission needs and system use, hazards
    and interfaces
  • Discipline experts assigned to perform analysis
    across all life cycle phases
  • Horizontal specialty skills are matrixed across
    IVV functional teams to verify correct systems
    integration
  • Specialized tools and simulations perform complex
    analyses
  • IVV testing activities complement developer
    testing enhancing overall software confidence
  • Developer testing focuses on demonstrating
    nominal behavior, IVV testing activities try to
    break the software
  • Overall program integration, test and
    verification approach analyzed for completeness,
    integrity and effectiveness

35
Why use NASA IVV Facility?
  • Software IVV, as practiced by the NASA Software
    IVV Facility, is a well-defined, proven, systems
    engineering discipline designed to reduce the
    risk in major software developments.

36
NASA IVV Facility Points of Contact
  • Judy Bruner
  • Acting Director
  • 304-367-8202
  • judith.n.bruner.1_at_gsfc.nasa.gov
  • Bill Jackson
  • Deputy Director
  • 304-367-8215
  • bill.jackson_at_ivv.nasa.gov
About PowerShow.com