Hany H. Ammar - PowerPoint PPT Presentation

About This Presentation
Title:

Hany H. Ammar

Description:

Introduction to Risk Management And Software ... – PowerPoint PPT presentation

Number of Views:145
Avg rating:3.0/5.0
Slides: 45
Provided by: JudiCo1
Category:

less

Transcript and Presenter's Notes

Title: Hany H. Ammar


1
??? ???? ?????? ?????? ????? ??? ? ???????
??????? ??? ???? ????
Introduction to Risk Management And Software
Architecture Risk Assessment
  • Hany H. Ammar
  • LANE Department of Computer Science and
    Electrical EngineeringWest Virginia University,
    Morgantown, West Virginia, USA, and
  • Faculty of Computers and Information, Cairo
    University, Cairo, Egypt

2
OUTLINE
  • Risk Management
  • Software Architecture Risk Assessment
  • Maintainability-based risk
  • Conclusions
  • Next Steps

3
Risk Management
4
(No Transcript)
5
For NASA Programs
  • RISK MANAGEMENT An organized, systematic
    decision-making process that efficiently
    identifies risks, assesses or analyzes risks, and
    effectively reduces or eliminates risks to
    achieving program goals.
  • RISK A Program Risk is any circumstance or
    situation that poses a threat to crew or vehicle
    safety, Program controlled cost Program
    controlled schedule or major mission objectives,
    and for which an acceptable resolution is deemed
    unlikely without a focused management effort

6
Risk Management Cycle Identify Identify that a
risk exits and give it a meaningful
name. Analyze Determine the severity of the risk
according to the risk matrix. If the risk is
negligible (low to medium severity, low
likelihood of occurrence), stop here. However, if
the risk could cause damage to the system or the
system's users, continue. Plan Decide how to
combat the risk based on the risk's severity and
likelihood of occurrence. Mitigate Follow the
plan formulated in the previous phase as closely
as possible to combat the risk. If this approach
does not work, return to the previous phase and
make a new plan. If the plan does work, continue
analyzing the risk to determine whether it has
been reduced to an acceptable severity
level. Track Once the risk has been mitigated to
an acceptable severity level, the risk should be
tracked to ensure the continued control of the
risk. If at any time the risk seems to resurface,
the risk management cycle should begin again,
starting with the analysis phase.
7
(No Transcript)
8
Risk Definition
  • According to NASA Software Safety Technical
    Standard, risk is defined as
  • exposure to the chance of injury or loss. It is
    a function of the possible frequency of
    occurrence of the undesired event, of the
    potential severity of resulting consequences, and
    of the uncertainties associated with the
    frequency and severity.
  • For software intensive systems, a risk is a
    combination of a likelihood of occurrence of an
    abnormal event or failure and the potential
    consequences or severity of that event or failure
    to a system's operators, users, or environment

9
Risk Matrix
Severity Likelihood of Occurrence Likelihood of Occurrence Likelihood of Occurrence Likelihood of Occurrence
Severity Probable Occasional Remote Improbable
Catastrophic High High High-Medium Medium
Critical High High-Medium Medium Medium-Low
Marginal High-Medium Medium Medium-Low Low
Negligible Medium Medium-Low Low Low
10
(No Transcript)
11
NASA IVV Facility
  • NPD 2820.1C for Software IVV Policy
    states"Task the IVV Facility in Fairmont,
    West Virginia to manage the performance of all
    IVV for software identified per the established
    criteria, and for any other safety critical
    software (as defined in NASA-STD-8719.13)"

12
IVV Function
  • Software Independent Verification Validation
    (IVV) is a systems engineering process employing
    rigorous methodologies for evaluating the
    correctness and quality of the software product
    throughout the software life cycle.
  • Software IVV is adapted to the characteristics
    of the project. Different projects require
    different level of IVV

13
IVV Lifecycle Activities
System Requirements Review
Preliminary Design Review
System Test
Mission Readiness Review
System Retirement
Critical Design Review
S/W FQT
Launch
Initial IVVP Signed
Baseline IVVP Signed
  • - IVV provides support and reports for Project
    milestones
  • Technical Analysis Reports document major phases
  • IVVP is updated to match changes in Project

IVV Provides CoFR
IVV Final Report
Concept Phase 2.0
Requirements Phase 3.0
Design Phase 4.0
Implementation Phase 5.0
Test Phase 6.0
Operations Maintenance Phase 7.0
IVV Phase Independent Support 1.0
Note numbers correspond to IVV WBS
  • Life-cycle IVV is designed to mesh with the
    Project schedule and provide timely inputs to
    mitigate risk
  • Dialog between the IVV Facility and the Project
    must begin before SRR
  • For most Projects, IVV ends (and the Final
    Report is delivered) on or about MRR. Some
    Projects have extended S/W development
    post-launch or major upgrades/maintenance (e.g.
    Shuttle, MER)

14
Software Project Resolution
Project Resolution is commonly categorized into
three resolution types
  • Successful Projects
  • Completed and operational, and
  • On Schedule
  • On Cost
  • With all originally specified features and
    functions
  • Challenged Projects
  • Completed and operational, but
  • Behind Schedule-------? Project Risk
  • Over Cost---------------? Project Risk
  • With fewer features and functions than originally
    specified
  • ------------? Product Risk
  • Failed Projects
  • Cancelled before completion or never implemented

15
Software CHAOS
  • The Standish Group has examined 30,000 Software
    Projects in the US since 1994. This "CHAOS"
    research has revealed a decided improvement in IT
    project management with the implementation of
    standards and practices such as IVV. This
    improvement correlates with the rise in project
    success depicted in the chart below

Project Resolution History (1994-2000)
The Standish Group International, Inc. Extreme
CHAOS (2001) - The 2001 update to the CHAOS
report. http//www.standishgroup.com/sample_resear
ch/PDFpages/extreme_chaos.pdf
16
Error Detection/Correction
  • Early error detection and correction are vital.
    The cost to correct software errors multiplies
    during the software development lifecycle. Early
    error detection and correction reduce costs and
    save time.

Direct Return on Investment of Software
Independent Verification and Validation
Methodology and Initial Case Studies, James B.
Dabney and Gary Barber, Assurance
Technology Symposium, 5 June 2003.
17
IVV Characteristics
  • Includes Risk Identification and Mitigation
    Techniques
  • Provides Independent Evaluation / Assessment of
  • Are we building the product right? Verification
  • Are we building the right product? Validation
  • Requires Technical, Managerial and Financial
    Independence
  • Makes a value added contribution, everyone shares
    the same mission success objective
  • For NASA Management - Provides Mission Assurance
  • For Project Management - Provides Unbiased Source
    of Help
  • Helps deliver
  • Risk Identification and Mitigation
  • Increased Quality and Safety
  • Improved Timeliness and Reliability
  • Reduced Rework Cost
  • NPD 8730.4 Requires NASA programs and projects
    that contain mission or safety critical software
    to document decisions concerning the use of IVV.

18
OUTLINE
  • Risk Management
  • Software Architecture Risk Assessment
  • Reliability-based risk
  • Performance-based risk
  • Maintainability-based risk
  • Component Ranking
  • Conclusions
  • Next Steps

19
Software Architecture Risk Assessment
  • This work is funded in part by grants to West
    Virginia University Research Corp. from the NSF
    (ITR) Program, and from the NASA Office of Safety
    and Mission Assurance (OSMA) Software Assurance
    Research Program (SARP) managed through the NASA
    Independent Verification and Validation (IVV)
    Facility, Fairmont

20
Project Overview
  • Risk Assessment of software architecture
    components, usage scenarios, and requirements
  • Risk definition is based on
  • Frequency of abnormal events
  • Severity or consequences of events
  • Reliability-based risk,
  • Performance-based risk
  • Requirementbased risk
  • Severity Analysis
  • Maintainability-based risk

What keeps satellites working 24/7 ?
21
Software Architecture Risk Assessment
  • An architecture-based approach for risk
    assessment
  • Components\connectors, requirements, and
    scenario risk,
  • Define several types of risk factors
  • Reliability-based Risk IEEE Trans. on Rel
    2001, on SE, 2002, 2003
  • Probability of failure
  • Severity or Consequences of this failure
  • Maintainability-based Risk RAMS 06, ICSM 05,
    ICSM 04
  • Probability of performing maintenance task
  • Cost of performing this task
  • The losses caused by low system maintainability
    can be
  • High cost of maintenance effort
  • Loss of the system by aging
  • Performance-based Risk IEEE Trans. SE, Jan.
    2005
  • Probability of missing timing or performance
    requirements
  • Severity or Consequences

22
Importance / Benefits
  • Identify the high risk components of the system
    in terms of Reliability/Maintainability/Performanc
    e

23
Software Architecture Risk Assessment Methodology

Requirements Model
System Architecture Model
Components Ranking
Components Risk Factors
24
OUTLINE
  • Risk Management
  • Software Architectures
  • Software Architecture Risk Assessment
  • Maintainability-based risk
  • Conclusions
  • Next Steps

25
Maintainability-based Risk
  • ICSM 05 AbdelMoez, Goseva, Ammar, Mili,
    Fuhrman, Architectural level Maintainability
    Based Risk Assessment
  • RAMS 06 AbdelMoez, Goseva, Ammar, Methodology
    for Maintainability Based Risk Assessment, Jan
    2006.

26
Importance / BenefitsMaintainability-based Risk
  • According to Pigoski, 60-80 of the system
    budget is spent on maintenance
  • Enhancements (perfective/ adaptive maintenance)
    account for 78-83 of the maintainer effort

27

Importance / BenefitsMaintainability-based Risk
  • Unisys holds the NASA contract to maintain and
    support 14 million lines of ground software for
    the space shuttle
  • There were 3,800 requirement changes made to the
    software after the loss of Challenger. These
    changes resulted in 900 software releases, of
    which 30 applied to the mission-control center
    with 3 of these being major upgrades
  • Reference
  • IEEE Software, Vol.6,  No.1,  pp. 116-119

http//hebb.cis.uoguelph.ca/dave/343/L
ectures/introduction.html
28
Importance / BenefitsTrade off Analysis for
Perfective Maintenance
Risk Factor
Components Risk
Components
Patterns
Maintainability risk for perfective maintenance
(open source case study Borg)
29
Software Architecture Risk Assessment
Methodology Maintainability-based Risk
SW Architecture
Requirments maturity Index / change / error
reports
(3) Estimate Size of Change (SC)
(1) Estimate components Initial Change
Probability (ICP)
(2) Estimate Change Propagation (CP) probabilities
ICPicpi
SCsci/j
CPcpi/j
(4) Estimate component risk factor
30
Maintenance change propagation
Outgoing maintenance
Incoming maintenance
31
Estimating Change Propagation
V1
C2
C1
1
V11
V12
V13
Change
Change in Provided Service
0
  • Change Propagation Probabilities matrix CPcpij
  • cpij is the probability that a change in Ci
    due to corrective/ perfective maintenance
    requires a change in Cj while maintaining the
    overall function of a system S
  • cpij P(Cj ? Cj' Ci ? Ci' S
    S' )
  • cpij is estimated by

32
Estimating Size of Change
  • Size of change SCscij
  • scij is defined as the ratio between the
    number of affected methods of the receiving
    component caused by the changes in the interface
    elements of the providing components and the
    total number of methods in the receiving
    component
  • scij is estimated by

33
CM1 Maintainability-Based Risk in Adaptive
Maintenance Context
34
Case Study NASA CM1 UML Model Structure Diagram
  • The UML-RT Model of CM1 was
  • Developed by WVU students (Nathan, Tom and
    Rajesh, Summer 2004) based on the CM1 software
    design specification

35
Change Propagation Probabilities for CM1
  • The Change Propagation probabilities CP is
    estimated using the CM1 UML model
  • The Change Propagation probabilities CP can be
    automatically estimated from UML-RT models, or
    java source

36
Size of Change for CM1
  • The Size of Change metrics SC is estimated using
    the CM1 UML model
  • The Size of Change metrics SC Probabilities CP
    can be automatically estimated from UML-RT
    models, or Java source

37
Software Architecture Risk Assessment
Methodology Maintainability-based Risk
SW Architecture
Requirments maturity Index / change / error
reports
(3) Estimate Size of Change (SC)
(1) Estimate components Initial Change
Probability (ICP)
(2) Estimate Change Propagation (CP) probabilities
ICPicpi
SCsci/j
CPcpi/j
(4) Estimate component risk factor
38
Maintainability-based Risk For corrective
maintenance (case study CM1)
ICP is estimated using error reports data
39
Prioritizing Corrective Maintenance Tasks for CM1
40
Maintainability-based Risk Maintainability risk
for Adaptive maintenance (case study CM1)
ICP is estimated using change reports data
41
Tool Support
42
Technology Readiness Level The Software
Architecture Risk Assessment Tool Support
43
The tool will be developed as a web application
44
Conclusions
  • Risk Management is vital to the success of
    projects and products
  • A risk Assessment process is needed
  • Software Architecture is a major determinant of
    software quality
  • Software Architecture can be used to manage
    project and product risks
  • Development of a methodology and a process for
    software architecture risk assessment
  • Continued development of a software architecture
    risk assessment tool to support the methodology

45
Papers Published
  • Vittorio Cortellessa, Katerina Goseva-Popstojanova
    , Kalaivani Appukkutty, Ajith R. Guedem, Ahmed
    Hassan, Rania Elnaggar, Walid Abdelmoez, Hany H.
    Ammar, Model-Based Performance Risk Analysis,
    IEEE Transactions on Software Engineering,
    January 2005, (Vol. 31, No. 1), pp.3-20.
  • Katerina Goseva-Popstojanova, Ahmed Hassan, Ajith
    Guedem, Walid Abdelmoez, Diaa Eldin M. Nassar,
    Hany Ammar, Ali Mili, "Architectural-Level Risk
    Analysis Using UML", IEEE Transactions on
    Software Engineering, October 2003 (Vol. 29, No.
    10), pp.946-960.
  • S. Yacoub, H. Ammar, A Methodology for
    Architectural-Level Reliability Risk Analysis,
    IEEE Transactions on Software Engineering, Vol.
    28, No. 6, June 2002.
  • W. AbdelMoez, K. Goseva-Popstojanova, H.H.
    Ammar, Methodology for Maintainability-Based
    Risk Assessment, Proc. of the 52nd Annual
    Reliability Maintainability Symposium (RAMS
    2006), Newport Beach, Ca., January 23-26, 2006.
     
  • Israr P. Shaik , W. Abdelmoez, R. Gunnalan, M.
    Shereshevsky, A. Zeid, H.H. Ammar, A. Mili, C.
    Fuhrman, Change Propagation for Assessing Design
    Quality of Software Architectures, Proc. of 5th
    IEEE/IFIP Working Conference on Software
    Architecture (WICSA), Pittsburgh, Pa., USA,
    November 6-9, 2005.
  • AbdelMoez, W., I. Shaik, R. Gunnalan, M.
    Shereshevsky, K. Goseva-Popstojanova, H.H. Ammar,
    A. Mili, C. Fuhrman, Architectural level
    Maintainability Based Risk Assessment, Proc. of
    poster papers in IEEE International Conference on
    Software Maintenance (ICSM 2005), Budapest,
    Hungary, September 25-30,2005.
  • W. Abdelmoez, D. M. Nassar, M. Shereshevsky, N.
    Gradetsky, R. Gunnalanm and H. H. Ammar, Bo Yu,
    and Ali Mili "Error Propagation in Software
    Architectures". In Proceedings of the 10th
    International Symposium on Software Metrics
    (METRICS'04), September 11 - 17, 2004 , IEEE
    Comp. Soc., pp 384-393
  • Abdelmoez, W., M. Shereshevsky, R. Gunnalan, H.
    H. Ammar, Bo Yu, S. Bogazzi, M. Korkmaz, A. Mili,
    "Software Architectures Change Propagation Tool
    (SACPT), 20th IEEE International Conference on
    Software Maintenance (ICSM'04) September 11 -
    14, 2004 , Chicago, Illinois, IEEE Comp. Soc., pp
    517
  • A. Hassan, K. Goseva-Popstojanova, and H. Ammar,
    UML Based Severity Analysis Methodology,
    Proceedings of the 2005 Annual Reliability and
    Maintainability Symposium (RAMS 2005),
    Alexandria, VA, January 2005.
Write a Comment
User Comments (0)
About PowerShow.com