Recent NAFCOM Extensions and Improvements - PowerPoint PPT Presentation

1 / 37
About This Presentation

Recent NAFCOM Extensions and Improvements


Uses cost estimating relationships (CERs) which correlate historical costs to ... Gemini. Hawkeye. INTELSAT-IV. LANDSAT-1. Lunar Orbiter. Mariner-10. Mariner-4 ... – PowerPoint PPT presentation

Number of Views:402
Avg rating:3.0/5.0
Slides: 38
Provided by: juliee6


Transcript and Presenter's Notes

Title: Recent NAFCOM Extensions and Improvements

Recent NAFCOM Extensions and Improvements
Presented by SAIC NASA Cost Symposium April 14,
  • NAFCOM Introduction
  • Benchmarking
  • CG Improvements
  • Latest NAFCOM Enhancements

  • The Systems Technologies Operation of SAIC, under
    contract to NASA MSFC, develops cost estimating
    tools and databases.
  • Development and maintenance of NAFCOM and REDSTAR
    are funded under the Engineering Cost Analysis
    Techniques Development (ECATD) contract.
  • Versions of NAFCOM produced under this contract
    are disseminated agency-wide to NASA and widely
    within the Air Force.

NAFCOM Description
  • NAFCOM is a parametric estimating tool for space
  • Uses cost estimating relationships (CERs) which
    correlate historical costs to mission
    characteristics to predict new project costs.
  • It is based on historical NASA and Air Force
    space projects.
  • It is intended to be used in the very early
    phases of a development project.
  • NAFCOM can be used at the subsystem or component
    levels and estimates development and production
  • NAFCOM is applicable to various types of missions
    (manned spacecraft, unmanned spacecraft, and
    launch vehicles).
  • There are two versions of the model a government
    version that is restricted and a contractor
    releasable version.

NAFCOM Evolution
  • Since 1990, ten versions of NAFCOM have been
    developed and distributed across NASA and other
    government agencies.

  • Fully functional cost model with user defined WBS
    and data access
  • CERs built automatically within NASCOM using 1st
    Pound method
  • Database contained 91 data points
  • Total re-write of all NAFCOM program code
  • Complexity Generators for all subsystems
  • Major user interface improvements
  • Database contains 122 data points
  • Cost Risk Analysis Module
  • CER Improvements
  • SOCM
  • Component level Complexity Generator
  • First non-weight based CERs for five subsystems
    (Complexity Generators)
  • Government and contractor versions distributed
  • Database contained 114 data points
  • Allowed online searches and copying of data
  • Cost estimates developed in spreadsheets with
    CERs created by individuals
  • Database contained 70 data points
  • Combined NASA and Air Force data
  • Enhanced search and filtering of data
  • Standardized WBS elements created
  • Database contained 102 data points
  • NASCOM database in hardcopy only
  • Estimators hand-entered data into spreadsheets
  • Database contained 65 data points

Benchmarking Overview
  • At the direction of MSFC, SAIC worked with
    Lockheed, Boeing, and Northrop Grumman to
    benchmark NAFCOM with relevant and recent
    completed missions.
  • Atlas V, Delta IV, RS-68, EOS-Aqua, Genesis, and
    GOES were benchmarked in three separate phases of

  • Results showed close comparisons to actual costs
    at top level, except for Atlas V, which is
    attributed to new engine (RD-180).
  • Estimates 6 above actual cost for Delta IV, for
  • At lower levels, NAFCOM consistently
    underestimated avionics costs in the first phase
    of benchmarking.

  • The results gleaned from benchmarking were used
    to modify NAFCOM estimating algorithms and
    approaches in order to increase the models
    estimating accuracy.
  • Subsequent comparisons for missions benchmarked
    in prior phases supported evidence for improved
    estimating capability.
  • Delta IV estimates improved from 11 higher than
    actual cost to 6 higher than actual cost.

Assessment of Model Validity
  • SAIC was tasked to assess the validity of
    acquisition estimating techniques, specifically
  • An iterative validation process was developed to
    indicate areas of improvement and then track the
    impact of those improvements at both the CER
    level and the total estimating capability level.

Assessment of Model ValidityBaselining NAFCOM
  • SAIC developed sixteen estimates using NAFCOM
    including manned, unmanned, and launch vehicles.
  • These estimates were compared to actuals at both
    the subsystem and total cost levels.
  • Identified potential trends in estimating
    inaccuracies (consistent over or under).

NAFCOM Baseline Estimates
NAFCOM Baseline Estimates
NAFCOM Baseline Estimates
Assessment of Model ValidityNAFCOM Complexity
  • SAIC has conducted a statistical analysis of
    actual costs versus estimateds cost for each
    Complexity Generator CER.
  • This analysis provides an overall measure of
    predictability including standard error, R2,
    coefficient of variance, and other metrics
    related to the estimated versus actual.
  • The data was then stratified to highlight
    potential trends in estimating inaccuracies, e.g.
    mission type, weight, etc.

CER Segregation CCDH
Improvement MethodologyMission Class
  • Stratified subsystem databases using attribute
  • Earth Orbiting Lite, Earth Orbiting, Planetary,
    Launch Vehicles, and Manned.
  • Previously, NAFCOM used a single variable with
    differing values for each mission class.
  • The use of attribute or dummy variables is more
    statistically correct, allows for a better fit,
    and does not mask the potential counter-balancing
    effects of a single mission class variable.

Improvement MethodologyRemoval of Outliers
  • In a few cases, outliers were removed from the
  • A t-test was used to remove outliers the
  • follows a t-distribution with n-k-1 degrees
    of freedom,
  • where n is the number of data points, k is
    the number of variables, ln y is the natural log
    of the actual cost of the potential outlier,
    est(ln y(l)) is the estimate of the potential
    outlier y using the model without y, and se(l) is
    the standard error of the model without y.
  • Using a 95 critical value, any data point with
    p-value less than 1 in 1,000 is rejected as an
    outlier and removed.

Improvement MethodologyRemoval of Outliers
  • Removal of data points was infrequent, at most
    three data points were removed from a CER because
    they were outliers.
  • No large missions (e.g., Shuttle) were removed.
  • Almost all missions removed were small satellites.

Improvement MethodologySubsystem Specific
  • Other subsystem-specific improvement methods were
    applied on a case-by-case basis.

Attitude Control Improvements
  • Stratified the Attitude Control database using
    mission class dummy variables.
  • Microsat was proven to be a statistical outlier
    for the Attitude Control subsystem and removed.
  • Adjusted database for old technology.
  • Removed all missions with launch dates prior to
  • Assumption based on introduction year for first
    Intel 4004 microchip, with a four-year lag.

Attitude Control ImprovementsAdjustment for New
  • Missions removed

Orbiter Mariner-10 Mariner-4 Mariner-6 Mariner-8 M
odel-35 OSO-8
AE-3 Apollo CSM Apollo LM ATS-1 ATS-5 ATS-6 Centau
r-D DSCS-II Gemini
Pioneer-10 S3 S-IVB SMS-1 Surveyor TACSAT TIROS-M
CCDH Improvements
  • Stratified the CCDH database using mission class
    dummy variables.
  • Microsat was proven to be a statistical outlier
    for the CCDH subsystem and removed.
  • Also looked at REX but data point was not a
    significant outlier.
  • Adjusted database for old technology.
  • Removed all missions with launch dates prior to
  • Assumption based on introduction year for first
    Intel 4004 microchip, with a four-year lag.

Electrical Power Improvements
  • Stratified the Electrical Power database using
    mission class dummy variables.
  • Microsat, Hawkeye, and Radcal were proven to be
    statistical outliers for the Electrical Power
    subsystem and removed.

Reaction Control Improvements
  • Stratified the Reaction Control database using
    mission class dummy variables.
  • Also tried the following variables as inputs, but
    the fit was not improved
  • Propellant Type
  • ISP
  • Launch Year

Structures Improvements
  • Stratified the Structures database using mission
    class dummy variables.
  • Added a dummy variable for Large, Inert
  • Added a Year of Technology variable.
  • SME was removed due to its being built primarily
    from spare parts.
  • UFO and GPSMYP were removed due to them being
    follow-ons of previous missions.
  • Mariner-4 was removed due to old technology
    (oldest data point in the Structures database).

SRM Kick Motors Improvements
  • Stratified the SRM Kick Motors database using
    mission class dummy variables.
  • Unmanned
  • Manned
  • Retro Rockets
  • Upper Stage
  • P-78 was proven to be statistical outlier for the
    SRM Kick Motor subsystem and removed.

Thermal ControlImprovements
  • Improvement in fit due to incorporation of dummy
    variables for mission stratification, and a
    refinement of technical rating.
  • Several missions were removed.
  • SME Thermal control was built from spare parts.
  • GPSMYP Follow-on to GPS-1.
  • Removed NATO-III, Intelsat-III, MACSAT, DSP, and
    Voyager for being outliers.

Crew Accommodation and ECLSSImprovements
  • Added data points to the CERs.

Propulsion and OMSImprovements
  • Added attribute variables for environment and

NAFCOM CG Improvement Summary
  • Reviewed all subsystem CERs.
  • Modified all the complexity generator equations.
  • Added new cost drivers (e.g., weight/volume
    ratio added to
  • ECLS CER).
  • - Stratified database according to mission type.
  • - Removed a few statistical outliers.
  • - Incorporated launch year (year of
  • - Changed technical rating definitions.
  • Removed old missions (pre-1976) for some
  • (e.g., CCDH).
  • Changes improved goodness-of-fit statistics.

NAFCOM is continually improved to increase
estimating credibility.
Summary of Improvements
Additional data points added to Crew
Accommodations and ECLSS CERs.
Example CER Estimates for COBE, Before
Improvements and After
New NAFCOM Features
  • New Template Wizard Options (e.g. CTV Only, Probe
    Only, etc.)
  • WBS Generator
  • User Defined Elements
  • Stage/System Insert and Delete
  • Risk Dollar Allocation Reports
  • Engine Risk (LRE, CCP, and Turbojet)
  • Pop-Up Notes Feature

New version of NAFCOM is now available.
Backup Slides
New Template Wizard Options
Additional templates have been developed and
incorporated into the model (1) CTV Only (2)
Upper Stage Only (3) Inner Planetary Orbiter
Only (4) Inner Planetary Flyby Spacecraft
Only (4) Inner Planetary Flyby Probe Only (5)
Inner Planetary Flyby Lander Only (6) Outer
Planetary Orbiter Only (7) Outer Planetary Flyby
Spacecraft Only (8) Outer Planetary Flyby Probe
WBS Generator
  • The WBS Generator is an interface to provide the
    user the flexibility of starting from scratch
    along with the automated functions and
    convenience of the modern NAFCOM.
  • To begin users are prompted to enter the number
    of stages/systems to include in their WBS.
  • The stages/systems are created automatically with
    a total roll-up, hardware rollup, one blank
    element, and the system integration elements.
  • The user can drag and drop, cut, copy, and paste
    to populate the stages/systems.
  • The user can also add instruments, operations
    cost, or launch services to the WBS from the
    Wizard interface.

User Defined Element
  • The User Defined Element Type allows the user to
    bypass the NAFCOM methodology.
  • It provides an additional estimating option.
  • Known costs or custom equations can be defined to
    estimate a particular element.
  • When using the User Defined Element Type the user
    will be allowed to use simple mathematical
    commands to enter costs or equations.
  • The User Defined input screen allows the user to
    input equations for DD, STH, and Flight Unit.
    The user can also create an equation that
    contains any combination of the variables W, X, Y
    and Z.
  • The User Defined Element Type is accessible via
    the Input Subsystem menu item.

Risk WBS Allocation Report
  • Risk Dollars can now be allocated by WBS element.
  • Risk Dollars is the amount of additional dollars
    (above the mean) required to fund a program at
    some appropriate level of confidence.
  • The two new Risk Reports are
  • Risk Allocation Report Xth Percentile
  • Provides a WBS allocation of Risk Dollars only by
    WBS element calculated by the 70th, 80th, or 90th
  • Risk Allocation Report Xth Percentile Total
  • Provides a WBS allocation of Risk Dollars and
    Mean costs by WBS element calculated by the 70th,
    80th, or 90th Percentile.
Write a Comment
User Comments (0)