Exploration Systems RFI 04-01: Methodology and Results

1
Exploration Systems RFI 04-01Methodology and
Results
2
RFI Evaluation Process Phase I Scoring
STEP 1 Contractors submit white papers in three
categories (responses due NLT 5/20)
Design Principles, Objectives, and Guidelines
Crosscutting Design Drivers/ Architecture Elements
Program Management, Acquisition, and Interfaces
STEP 2 Eval teams utilize web-based document
review tool in OExS Product Data Management
system to assign scores and WBS/tech tags as file
attributes (5/20-31)
HRT Evaluation Team 1 (60 Heads)
HRT Evaluation Team 2 (60 Heads)
Constellation Evaluation Team (10 Heads)
Scoring Criteria
STEP 3 Scored RFI responses are passed to teams
in Requirements, Human Robotic Technology, and
Constellation for use in program content and
management definition
Demonstrated Effectiveness / Technical
Maturity 1- 5
Innovativeness / Variation from Historical
Approach 1- 5
Potential Improvement in Schedule, Cost,
Risk 1- 5
RFI Review Period May 20 31
3
RFI Evaluation Process Phase I Focus Area Mapping
Results are mapped into RFI Focus Areas to
facilitate retrieval and use for subsequent BAAs
and RFPs
4
RFI Evaluation Process Phase I WBS Mapping
Results are mapped to WBS Tier 1 to
facilitate retrieval and use for subsequent BAAs
and RFPs
5
RFI Evaluation Process Phase I Tech Tagging
RFI responses are tagged for relevance to certain
technology types to facilitate retrieval and use
for subsequent BAAs and RFPs
6
Web-Based Evaluation System Process Map
STEP 1 Login to the RFI Portal
STEP 2 Get your Review Task
STEP 3 Complete your Evaluation

• Original RFI submission page
• Download document
• Review document

Task a
Task b

• Create RFI Review Document
• Fill out evaluation criteria

Email notification
Review Task Page
START
Login
Review Completed Evaluation
Task c
RFI Portal WWW Browser https//naccsli3.nis.nasa.
gov
Work-list
START
Login
Task d
Perform Task Complete

• An evaluator can use either path (
  or ) to complete an evaluation
• Click on the any of the above process blocks or
  arrows to see details
• At anytime you can get to your Review Task
  through your work-list

START
START
Map
Home
FAQs
7
Scoring CriteriaDemonstrated Effectiveness /
Technological Maturity

• SCORE OF 1 This paper identifies concepts,
  technologies, or approaches that comply with the
  basic principles of physics, systems integration,
  or program management, but that have not been
  fully formulated.
• SCORE OF 2 This paper identifies concepts,
  technologies, or approaches that have been
  formulated through analytical or experimental
  proof-of-concept, but that have not been
  validated in a relevant environment or
  small-scale project.
• SCORE OF 3 This paper identifies concepts,
  technologies, or approaches that have been
  validated in a relevant environment or
  small-scale project, but have not been
  demonstrated in full prototype form or in the
  development phase of a major program.
• SCORE OF 4 This paper identifies concepts,
  technologies, or approaches that have been
  demonstrated in full prototype form or in the
  development phase of a major program, but have
  not been carried through the full life cycle of a
  development program or mission operations.
• SCORE OF 5 This paper identifies concepts,
  technologies, or approaches that have been
  successfully carried through the full life cycle
  of a development program and/or mission
  operations.

8
Scoring CriteriaInnovativeness / Variation from
Historical Approach

• SCORE OF 1 This paper identifies concepts,
  technologies, or approaches that have been
  broadly employed in previous development
  programs, that met the baseline requirements of
  these programs, but did not substantially improve
  functional performance or state-of-the-art.
• SCORE OF 2 This paper identifies concepts,
  technologies, or approaches that have been
  broadly employed in previous development programs
  with the effect of substantially exceeding
  baseline requirements and/or substantially
  improving functional performance or
  state-of-the-art.
• SCORE OF 3 This paper identifies concepts,
  technologies, or approaches that have been
  employed occasionally in previous development
  programs with the effect of substantially
  exceeding baseline requirements and/or
  substantially improving functional performance or
  state-of-the-art. Though they proved to be
  useful assets to the programs that employed them,
  these concepts, technologies, or approaches have
  not been infused into a broad number or diversity
  of programs.
• SCORE OF 4 This paper identifies concepts,
  technologies, or approaches that have been
  previously proposed for infusion into development
  programs. These concepts, technologies, or
  approaches would potentially alter functional
  performance or state-of-the-art to the extent
  that they would fundamentally alter previously
  conducted system-wide trades in development
  programs. These concepts, technologies, or
  approaches may have advocates in the development
  community, but are not broadly known or accepted.
  Alternatively, the paper may propose the
  utilization of concepts, technologies, or
  approaches that have not been previously
  considered for their potential contributions to
  the capabilities of space systems.
• SCORE OF 5 This paper identifies concepts,
  technologies, or approaches that have not been
  previously proposed. These concepts,
  technologies, or approaches would potentially
  alter functional performance or state-of-the-art
  to the extent that they would fundamentally alter
  system-wide trades in development programs.
  Alternatively, the paper may propose concepts,
  technologies, or approaches that have not been
  considered as potential elements of a space
  system or space exploration architecture, and
  that might revolutionize the capabilities of
  developed systems.

9
Scoring CriteriaPotential Improvement in Cost,
Schedule, Risk

• SCORE OF 1 This paper identifies concepts,
  technologies, or approaches that have been
  employed in the past and that caused programs to
  fail to meet baseline targets for cost, schedule,
  and/or risk. Alternatively, the concepts,
  technologies, or approaches advocated by this
  paper have not been previously employed but
  present unjustifiable threats to a program that
  would be run within cost, schedule, and/or risk
  constraints.
• SCORE OF 2 This paper identifies concepts,
  technologies, or approaches that have been
  employed in the past and that posed substantial
  threats to baseline targets for cost, schedule,
  and/or risk. If not previously employed, the
  concepts, technologies, or approaches advocated
  by this paper would force programs to accept
  substantial threats to cost, schedule, and/or
  risk baselines.
• SCORE OF 3 This paper identifies concepts,
  technologies, or approaches that helped enable
  previous programs to meet baseline targets for
  cost, schedule, and/or risk. If not previously
  employed, the concepts, technologies, or
  approaches advocated by this paper would have a
  neutral effect on the cost, schedule, and/or risk
  baselines of programs exhibiting conventional
  analogs of the concepts, technologies, or
  approaches.
• SCORE OF 4 This paper identifies concepts,
  technologies, or approaches that allowed previous
  programs to positively exceed baseline targets
  for cost, schedule, and/or risk. If not
  previously employed, the concepts, technologies,
  or approaches advocated by this paper have a high
  probability of substantially improving the cost,
  schedule, and/or risk performance of programs
  employing conventional analogs.
• SCORE OF 5 This paper identifies concepts,
  technologies, or approaches that would reduce the
  cost, schedule, and/or risk of developing a
  system so much that the total system-of-systems
  or proposed acquisition strategy would be
  fundamentally improved or accelerated.

10
RFI Results Summary Observations

• Wealth of great material will be available in
  database.
• Comprehensive trade studies or architectural
  concepts
• Articulations of difficult lessons learned in
  prior NASA programs
• Engineering data management process and IT a
  major focus.
• Some of these included collaboration tools
• Others extended this to modeling and simulation
• A recurring theme was the strong recommendation
  to consider and demonstrate technology X early in
  the lifecycle.
• Examples of X included EVA suits, autonomous
  robotics, vehicle health management, simulation
  infrastructure, software development tools, and
  many others
• It was difficult for many respondents to treat
  this as an RFI, not an RFP. Many respondents
  submitted what amounted to proposals to commence
  or resume technical studies or development
  programs.
• Many submissions were recycled technical reports.
  Though content did not respond directly to the
  RFI focus areas, many were relevant.

11
RFI Results High Score Distribution
12
RFI Results Design Principles Distribution
13
RFI Results Cross-Cutting Drivers Distribution
14
RFI Results Program Management Distribution
15
RFI Results WBS Distribution
Systems Engineering Integration tagged for
gt50 of Evaluated Submissions
16
Phase I Feedback to RFI Submitters
Notification of Evaluation Online evaluation
database will automatically notify submitter when
RFI submission has been fully evaluated.
Results Analysis Posting Team of Headquarters
managers will query database to (1) identify
exceptional proposals for concepts, technologies,
or approaches (2) review metadata that describes
total complement of submissions (3) probe
certain clusters of papers based on mapping to
WBS, focus areas, technology types, and scoring
criteria. A document describing the results of
this analysis will be posted to the Exploration
Systems Acquisition Information website by June
15.
Briefing of Results Analysis at Industry Day RFI
Evaluation Process managers will brief results
analysis at NASA Headquarters Industry Day, to be
held on June 18.
17
WBS Relevance Mapping Phase II Utilization
RFI responses are passed to teams in
Requirements, Human Robotic Technology, and
Constellation for use in program content
formulation and management definition
Requirements Formulation and Architecture
Definition Activities
HRT BAA/NRA System-of-Systems Challenges
Constellation Program Structure Management,
Acquisition Strategy, CEV RFP
BAA Concept Exploration Refinement
HRT BAA/NRA Technology Gap Filling