Program

1
Manage Risks
Evaluate Risk Handling Options
Program Requirements
Assess Risks
Establish Cost Schedule/ Perf Impacts
Manage Risks
Evaluate Subcontractor Risks
Establish Indicators
Evaluate Indicators
Management Action

• TPMs
• Metrics
• Cost Reports
• Schedules
• Reports
• Watchlists
• Demonstrations

Management Review
Reassess Risks
Evaluate New Handling Options
2
(No Transcript)
3
Risk Management Plan

• Risk Management Strategy - See Risk Management
  Best Practices notes
• Responsibilities

- Who identifies - Who constructs abatement plan
options - Who selects options to implement - Who
updates identification, assessment, and
abatement plans
4
Risk Management Plan (Cont.)

• Risk management activities
• Identification methods (structured baseline)
• Assessment methods (definitions, QRA?)
• Abatement planning (waterfall chart
  withaccompanying text)
• Monitoring/Decision Making

5
Risk Management Process Flow
Program Milestones
LOA SRR SDR/SFR
PDR
Customer
List of Options
Rev. Appr.
Program Manager
Rev. Appr.
Systems Eng
IPT Leads
Risk Abatement Plans
Risk Assessment
Technical Exp.
Subcontractor
Risk Fac.
Revised Database
RMP/Process
TNG/Com.
RiskList
Database
6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
Decision Making Cycle
Define Decision
Bayes Theorem
Collect/Capture Data
Assess Uncertaintyin Data
DetermineValue of Reducing Uncertainty
Make Decision
Gather AdditionalData
N
Y
15
Derivation of Bayes Theorem
C is a universe of outcomes such that P(C) 1.
B
A
C
P(A) Area A / Area CP(AB) P(A and B) /
P(B), so by substitution, P(BA) P(B and A) /
P(A)P(AB) P(B) P(BA) P(A)P(AB)
P(BA) P(A) / P(B)
Bayes Theorem
16
Bayes Theorem (Cont.)
Outcome 1
P(A) P(A) 1,P(B) P(B) 1P(B) P(BA)
P(A) P(BA) P(A)P(AB)
P(BA) P(A) P(BA) P(A) P(BA) P(A)
B
A
B
Outcome 2
Outcome 3
B
A
B
Outcome 4
Bayes TheoremRestated
17
Example
.63.18.09.02.03.05
No Redesign
Original Design Meets Cust. Req.
.7
Minor Redesign
.2
.9
.1
Major Redesign
Original Design Does NOT Meet Cust. Req.
No Redesign
.2
.1
.3
Minor Redesign
.5
Major Redesign
1
18
Bayesian Estimation

• In general, there are two types of approaches to
  estimation of parameters
• Frequentist - Estimates an unknown
  parameter based only on observed data and an
  adopted model - Characterized by
  scientific objectivity
• Bayesian - Estimates an unknown parameter
  by appropriately combining prior intuition
  or knowledge with information from observed data
  - Characterized by subjective nature of prior
  opinion
• Each approach is valid when applied under
  specific circumstances.Neither approach will
  uniformly dominate the other

19
Bayes Theorem Exercise
Many combat aircraft carry air-to-air missiles.
A warning light will notify the pilot if the
missile is defective. If the missile is
defective an alternative missile may be used or
the mission may need to be aborted. The missile
may or may not be defective when the warning
light is on. Given the warning light indicates a
bad missile - what is the probability the missile
is really defective? Approach Define Event A as
the missile is a dud. Define Event B as the
warning device signals the missile is a dud. From
product specs it is known that P(A) .0001
i.e., one missile in 10000 is a dud. P(BA)
.9998 i.e., the probability of the avionics
hardware and software detecting and reporting a
defective missile is .9998 P(Bnot A) .002
i.e., probability of a false warning is .002
20
Present Value Notes
100 1 YR. _at_ 10 110 100(.1) 100
100(1.1) 2 YR. 100(1.1)(1.1)
3 YR. 100(1.1)(1.1)(1.1) Future Value
(Present Value)(1R)n at year n Present Value
Future Value/(1R)n Present Value i1?n
(Future Value _at_yr i) / (1R)i (All
Years) Net Present Value Income - Expenses
- Initial Outlay I1/1R I2/(1R)2...
In/(1R)n Note R is the Hurdle Rate
21
Example 1 Present Value
R 6 Jan 1, 2002 Dec 31, 2003 Dec 31,
2004 Dec 31, 2005
X
Cost Income
100K 150K 200K
PV 100K/(1.06)2 150K/(1.06)3 200K/(1.06)4
373K
22
Example 2 Present Value
Rate
6
7
8
5
Year Beg. 2002 End 2003 End 2004
End 2005
Cost Income
X
100K 150K 200K
PV 100K/(1.06)(1.07) 150K/(1.06)(1.07)(1.08)
200K/(1.06)(1.07)(1.08)(1.05)
23
Question
How can you determine a funding upper bound for a
risk abatement alternative during
development? First - Should you be trying to
minimize risk or maximize profit?
24
Probability of Win Based on Bid
Probability of Winning
1
.8
.6
.4
.2
-10 -8 -6 -4 -2 0
2 4 6 8 10
Below
Above
Expected Competitive Bid
25
Return Based on Bid
Return
10
-10 -8 -6 -4 -2 0
2 4 6 8 10
Below
Above
Expected Competitive Bid
26
Expected Return Based on Bid
Return Discounted by Prob. of Win
.
-2.2
Marginal Return X Prob. Win
-10 -8 -6 -4 -2 0
2 4 6 8 10
Below
Above
Expected Competitive Bid
27
Technology RoadmapDevelopment Process
28
Roadmap Development Overview
Baseline Force Capability
Preliminary Ranking of Configuration Options
Technology Push
Pre-Design Studies
Requirements Analysis
Other Considerations
Master Configuration Roadmap
29
Roadmap Development Activities
30
Baseline Capability

• Missions/Tasks for Each Own Force Aircraft Must
  Be Identified Based on Warfare Objectives
• Evaluations of Effectiveness Must Account for
  Mission/Tasks within Air-to-Air and Air-to-Ground
  Categories. Summations of Kills Over Time
  Across Both Categories Are Particularly
  Meaningful.

31
Sample Multirole Aircraft Missions/Tasks
Warfare Objectives to Be Achieved
Top-Level Multirole Aircraft Requirements
Identification
Massive Firepower In-Theater At a Pace and in
Numbers Sufficient to Field an Over-Whelming
Force
1. In-theatre Sortie Generation 2.
Supportable 3. Kill Armored Vehicles 4.
Destroy Critical Chokepoints 5. Destroy Enemy
Aircraft in Air and on Ground 6. Destroy
Enemy Air Defenses 7. Reconstitutable
Direct Support of the Ground Campaign Slow the
Enemys Offensive Temp until Friendly Ground
Forces Can Mass and Counter the Advance
Establish Air Superiority Denying the Enemy Use
of His Airpower and Defenses
Reconstitution The ability to Reconstitute a
Credible Defense Faster Than Any Potential
Opponent Can Generate an Over-Whelming Offense
32
(No Transcript)
33
(No Transcript)
34
Technology Push

• Sources of Options
• Government Labs - Defense Industry
• Commercial Organizations - Educational Institutes
• Upgrade Option Categories
• Airframe - Electronic Warfare
• Propulsion - Avionics Architecture
• Fuel Carriage - Comm/Nav/Ident
• Materials - Weapons Int./Carriage
• Takeoff/Landing Systems - Mission Planning
• Flight Controls - Training Systems
• Pilot Vehicle Interface - Reliability/Maint./Supp
  .
• Sensors - C3I

35
Technology Upgrade Option Candidates

• Airframe
• Structure
• Wing
• Tail
• Inlet
• Electroluminescent Strip Lighting
• Signature Improvements
• Propulsion
• Improved Engine
• Nozzle
• Thrust Vectoring

• Electronic Warfare- Threat Warning- Threat Info
  Management- Dispensers- Expendables- Towed
  Countermeasures- Electronic Countermeasures
• Avionics Architecture- Data Bus- Processor
  Memory- Processor Throughput- Tactical Data
  Management

36
Technology Upgrade Option Candidates(Cont.)

• Fuel Carriage
• External Tanks
• Conformal Tanks
• Dorsal Tanks
• Materials
• Composites
• Advanced Metals
• Takeoff/Landing
• Gear
• Drag Chute
• Braking
• Thrust Reversers
• Longer Life Tires

• Comm/Nav/Ident
• Anti-Jam Voice Data Link
• Global Positioning System
• Improved IFF
• Infrared Navigation
• Weapons Int/Carriage
• Standoff Weapon
• Air-to-Air Missiles
• Air-to-Surface Weapon
• Digital Store Management System
• Mission Planning
• Automated Planning Aids

37
Technology Upgrade Option Candidates(Cont.)

• Flight Controls
• Terrain Following/Avoidance
• Ground Proximity Warning
• Pilot Vehicle Interface
• Color Moving Map
• Night Vision
• Helmet-Mounted Display
• Tactical Situation Awareness Display
• Sensors
• Radar
• Infrared Targeting
• All Weather Targeting
• Day/Night Capability
• Laser Designators
• Synthetic Aperture Radar

• Training Systems
• Pilot
• Mission
• Reliability/Maint./Supp.
• Modular Avionics
• Auxiliary Power Unit
• Integrated Diagnostics Testing
• C3I
• Real-Time Intelligence to Cockpit
• On-Board Data Fusion

38
Format For Technology Summaries

• Description
• Basic System Characteristics
• Integration Issues
• Manufacturers
• Benefits/Costs
• Operational Improvements
• Performance Impacts
• Reliability and Maintainability
• System Impacts (Weight, Volume, Cooling, Power)
• Development Cost
• Unit Cost
• Status
• Development Required
• Availability
• Risk and Other Considerations

39
Other Considerations
Political Drivers

• Known/Suspected Threat Change Plans
• Projected New Technology Capability Must Be
  Accepted and Embrace by Both Military and
  Civilian Leadership
• Technology Must Allow Appropriate
  Self-Sufficienecy
• Technology Must Be Compatible with Manpower,
  Training, and Skill Levels

40
Other Considerations (Contd)
International Market Competitiveness

• Must Enhance Future Market Potential
• In-Country Support Concept Must Be Workable
• Availability to Tailor Subsystems to Specific
  Foreign Customer Needs (Example Adaptability to
  Existing Weapons)

Budget/Timeframe

• Must Be Affordable in Relation to Baseline
  Aircraft Cost
• Technology Must Be Low to Moderate Risk in
  Timeframe Planned for Implementation

41
Preliminary Ranking
42
Quantifying Technology Upgrade Benefit
43
Rankings of Sample TechnologiesSuggests Most
Promising Options
44
Pre-Design Studies

• Cost - Estimate Life Cycle Cost Components of
  Technology
• Availability - Determine Earliest Implementation
  Timeframe
• Feasibility - Assessment of Technology Impact on
  Design Margins (i.e., Available Volume, Weight,
  Cooling, and Electrical Power). Define Feasible
  Upgrade Options without Regard to Implementation
  Time Sequence

45
Life Cycle Cost Composition
46
(No Transcript)
47
Requirements Analysis
48
Principal Steps in ComprehensiveRequirements
Analysis Assessments
1. Identify Decision Factors within Broad
Decision Categories (Note This Step Can Be
Performed Before or During Pre-Design
Studies) 2. Quantify Decision Factors for Each
Configuration 3. Analyze Customer Preferences
for Each Decision Factor (Note This Step Can
Also Be Performed Before or During
Pre-Design) 4. Assign Weights to Decision
Factors 5. Score Each Configuration (Sum
Weights x Preferences) 6. Perform Sensitivity
Analysis on Weights If Configuration Scoring is
Close)
49
Sample Configuration Decision Categories
Air Vehicle
Effectiveness
Cost
Risk
Threat Acquisition Avoidance Hit Avoidable Given
Acquisition Sortie Survival Given Hit Target
Acquisition Target Kill Given Acquisition Kills
per Sortie Targets Killed Over Time
Flyaway Weapon System Procurement Program
Acquisition Life Cycle
Technical Cost Schedule Producibility Support
ability Management
50
Utility Functions - Preference Indicators

• Utility Functions Provide a Good Technique for
  Translating Diverse Criteria Into a Common Scale.
  (i.e., Range in NMi, MTBF in Hours, etc.)
• Utility Scores Range From 0 to 1 With 0 Being
  Least Preferred and 1 Being Most Preferred.

Examples
Utility for Range
Utility for MTBF
1
1
Threshold Objective
Threshold Objective
Range in MNi
MTBF in hours
51
Hints for Determiningthe Shape of Utility
Functions
1
1
After Establishing the Minimum Requirements and
Goal, Draw Neutral Preference Position as Shown
Neutral Preference
Critical, Risk Prone
Non-Critical, Risk Average
Req Decision Factor Goal
1
2
Divide Decision Factor into Quartiles and Assess
25, 50, and 75 Points Relative to Neutral
Preference
Req Decision Factor Goal
52
Final Steps in Development of Master
Configuration Roadmap

• Time Phase Prioritized Technologies with Yearly
  Budgets for Upgrades (i.e., Pace Upgrade
  Implementations with Planned Budgets)
• Resequence Time Phased Prioritized Technology
  Upgrades as Necessary Dependent on Implementation
  Availability for New or Emerging Technologies

53
(No Transcript)
54
How to Determine a FundingProfile for Program
Phases
QUESTION How to select the development cost so
that development and production cost is
minimum?
Key Factors Design Margins Extent of
Risk Reduction
Development Production Cost
Development Cost
55
Relating Risk Resolution toDevelopment Cost
.
H
Extent of Risk Resolution
Extent of Risk Resolution
.
L
Design Margins Required
Development Cost
When risks are resolved development cost
increases.
56
Relating Design Margins to Production Cost
.
H
Extent of Risk Resolution
Extent of Risk Resolution
.
L
Design Margins Required
Development Cost
When design margins are increased, production
costs increase.
Production Cost
Design Margins Required
57
Relating Production Cost and RiskResolution to
Development Cost
H
Extent of Risk Resolution
Extent of Risk Resolution
L
Design Margins Required
Development Cost
.
Production Cost
Production Cost
.
Design Margins Required
Development Cost
58
Observation
When development cost goes up (due to risk
resolution in order to decrease design margins),
production cost goes down.
Question How can development and production
cost be balanced?
59
How to Determine Development Cost
Sum the development and production cost and graph
as a function of development cost.
60
Summary of Course

• Review different Systems Engineering
  sub-processes from the perspective of
• Existing industry standard (i.e., current
  state-of-the art)
• Emerging trend not widely implemented
• Idea for improvement

61
Fear Tigers Not Mice
Work to identify risks and decide how to best
handle them. In the absence of periodic focus on
risks your program teams will focus on
accomplishable tasks.
62
Concluding Thoughts
Each of you can implement systems engineering
and/or risk management process improvements in
areas you are responsible for, assuming existing
company procedures do not already dictate a
procedure. When company procedures exist, a new
procedure pilot will many times be a good tack
for gaining approval to proceed. Suggestions for
improvements in areas outside your responsibility
may be considered and not implemented. Choose
wisely!