Test & Evaluation/Science &Technology Net-Centric Systems Test (NST) Focus Area Overview

1
Test Evaluation/Science TechnologyNet-Centric
Systems Test (NST)Focus AreaOverview

• NAVAIR Pt. Mugu
• Executing Agent

Gil Torres
2
TE/ST Program Overview

• The TE/ST program
• Exploits new technologies and processes to meet
  important TE requirements
• Expedites the transition of new technologies from
  the laboratory environment to the TE community.
• Leverages commercial equipment, modeling and
  simulation, and networking innovations to support
  TE
• Examines emerging requirements derived from joint
  service initiatives
• Identifies needed technology areas and develops a
  long-range roadmap for technology insertion
• Leverages research efforts from the highly
  developed technology base in the DoD labs, test
  centers, other government agencies, industry, and
  academia

3
TRMC OrganizationOSD Test Resource Management
Center
Director
Deputy Director Joint Investment, Policy,
Planning
Deputy Director Strategic Planning
Deputy Director Budget Resources
Policy
CTEIP Program
JMETC
ST Program
4
Net-Centric Systems Test (NST) Focus Area
Overview
5
NST Vision
Accelerate the delivery of Mission Ready
Net-Centric capabilities to the warfighter
Net-Centric
Tomorrows Plug Test Net-Centric Environment
Todays Netcentric Environment

• Stove-Piped Information
• Centralized Control
• Unique Software Solutions
• Data Not Shared
• Inefficiency

• Net-Centricity
• Decentralized Control
• Enterprise Services
• Shared Data
• Autonomous Agent
• Enterprise Architecture
• Web 2.0

NST PortfolioNST TE Advanced Technologies/Capabi
lities
Broad Agency Announcement
NST TE Needs Technology Gaps
6
NST TE Needs(1 of 2)

• Recreate the Net-Centric Test Battlespace
• Emulate tactical edges
• Test mission threads with limited participants
• Integration/Responsiveness of Virtual Component
  with Live Component
• Integration of Cyber/Information Operations with
  Kinetic Simulations
• Representation of the Network Infrastructure and
  Network Environment
• Representation of Cyber threat to NSUT
• Simulate/Stimulate Irregular Warfare for urban
  environment
• Verification Validation (VV) of the Synthetic
  Battle Environment (SBE)

7
NST TE Needs(2 of 2)

• Measure Analyze Net-Centric Test Environment
• Monitor/Analyze Critical Chaining Issues
• Improving Near Real Time Analysis
• Improve Instrumentation Analyzing Cyber/IO
• Manage Net-Centric Test Battlespace
• Quality of Service to Critical Nodes
• Synchronize Synthetic Environments
• Manage Test Environment for Net Based Systems
• Manage Tactically Dynamically Configured Networks

8
NST ChallengesTechnologies to. . . .

• Recreate Net-Centric Test Battlespace
• Flexible, Scalable LVC Environment For JMe
  Testing
• GIG SOA Sim / Stim Capabilities
• Integrate Validate Net-Centric Simulations
  SoS
• Emulate Red Cyber Warfare Capabilities
• Automate C2 Decision Process

• Measure Analyze Net-Centric Test Environment
• Analyze Joint Mission Threads Near Real-Time
• Evaluate Net-Centric Data Services
• Automated NR-KPP Analysis
• Measure Joint Mission Effectiveness (JMe)
• Analyze Visualize Information Assurance
  Operations
• Manage Net-Centric Test Battlespace
• Dynamically Manage Test Infrastructure
• Automated Planning and Scenario Development
• Cross Domain Solution Over Distributed Test
  Infrastructure

9
NST Roadmap Recreate the Net-Centric Test
Battlespace
TE/ST Investments
NE2S Network Effects Emulator System TRCE
TENA in a Resource Constrained Environment

• Emulate Tactical Edges

• Test Mission Thread w/Limited Participants

• Technologies to VV NST Env.

• Integration/Responsiveness of VC with
  Theatre/Live Component

• Representation of Cyber Threat to NSUT

• Sim/Stim Irregular Warfare

• Representation of Network Infrastructure and Env.

NE2S

• Integrate Internet into Family of Tested Nets

TRCE

• Integrate IO with Kinetic Sims

Challenges
Emulate Red Cyber Warfare Capabilities
Flexible, Scalable LVC Environment For JMe Testing
Integrate Validate Netcentric Simulations SoS
Automate C2 Decision Process
GIG SOA Sim / Stim Capabilities
TE Capability
Recreate the Net-Centric Test Battlespace
10
NST Roadmap Measure and Analyze the Net-Centric
Test Environment
TE/ST Investments
NECM NST Evaluation Capability Module NEIV
Net-Centric Environment Instrumentation
Visualization NSAT NR-KPP Solution Architecture
Toolset
NECM
NEIV
NSAT

• Monitor/Analyze Critical Chaining Issues

• Improving Near Real Time Analysis

• Analyze impact of Cyber and IO

Challenges
Analyze Joint Mission Threads Near Real-Time
Measure Joint Mission Effectiveness (JMe)
Evaluate Netcentric Data Services
Analyze Visualize Information Assurance
Operations
Automated NR-KPP Analysis
TE Capability
Measure and Analyze the Net-Centric Test Env.
11
NST Roadmap Manage the Net-Centric Test
Battlespace
TE/ST Investments
MENSA Middleware Enhancements for Net-Centric
Simulation Architecture ANSC Analyzer for NST
Confederations MLSCLS Multi-Level Security
Cross Layer Scheme PAM Policy Based Adaptive
Network Management RRM Rapid Reconfiguration
Module
MENSA
ANSC
MLSCLS

• Synch Synthetic Environments

PAM

• Manage Test Env. For Net-based systems

RRM

• Manage Tactically Dynamic Configured Networks

• QoS to Critical Nodes

Challenges
Dynamically Manage Test Infrastructure
Cross Domain Solution Over Distributed Test
Infrastructure
Automated Planning and Scenario Development
TE Capability
Manage the Net-Centric Test Battlespace
12
TE/ST Project Critical Factors

• TE Need (Gaps)
• Address a known TE technological gap.
• Understanding of the current TE capabilities and
  techniques
• Understand capabilities and techniques for the
  future of TE
• ST Challenge
• What needs to be done to effectively address the
  TE Need
• Efficiencies and test capacity improvements as a
  result of the technology developed or matured
• Results of project will significantly advance
  technology and thereby significantly enhance
  current test capabilities
• High Risk-High Payoff
• The project improves testing rigor and
  capabilities relevant to Net-Centric technology
  improvements
• Reduces the cost of government operations
• Provides an improved capability to test operators

13
TE/ST Project Critical Factors

• Wide Application
• The technology can be applied at multiple sites,
  on multiple systems, in multiple organizations
  simultaneously
• The technology effectively supports a distributed
  LVC Net-Centric test event
• Transition Strategy
• Identify potential partners, future consumers of
  the technology (government lab, agency, test
  activity, industry, etc).
• Identify the intended use of the technology, and
  a plan to obtain buy-in to the technology from
  potential transition partners
• Demonstrate a concise and sensible plan that will
  create a higher probability of a successful
  transition to the identified transition partners

14
Project Execution Plan (PEP)

• PEP required contents
• Purpose and Work Scope
• TE Needs
• ST Challenges
• Technology Readiness Levels
• Budget
• Project Schedule
• Project Specifications
• Phase Exit Criteria
• Project Transition
• List of Deliverables

15
Project Specifications
16
Netcentric Systems Test Technology Investments
Transitions (1 of 7)
17
Netcentric Systems Test Technology Investments
Transitions (2 of 7)
18
Netcentric Systems Test Technology Investments
Transitions (3 of 7)
19
Netcentric Systems Test Technology Investments
Transitions (4 of 7)
20
Netcentric Systems Test Technology Investments
Transitions (5 of 7)
21
Netcentric Systems Test Technology Investments
Transitions (6 of 7)
22
Netcentric Systems Test Technology Investments
Transitions (7 of 7)
23
Questions?
24
T-E Web Monthly Reports

• Monthly reporting timeliness is critical
• Timeliness of report submission is important to
  effectively communicate project status
• Submission to Deputy EA required by the 5th of
  the month for prior month
• Content
• Executive Summary of the project
• Funding OE
• Activities for the month just completed
• Significant accomplishments
• Planned efforts for the next month
• Problems/Issues
• Brief explanation on use of funds

25
FAM Monthly Reports

• Focus Area Monthly (FAM) reports
• EA required to submit to PMO on the last Friday
  of each month
• Highlight significant (positive and negative)
  events or issues in projects
• Not a substitute for more timely notification to
  PMO of highly significant issues or events
• Issues requiring immediate action
• Events that could result in bad press
• FAM reports are now auto-generated from T-E Web,
  so significant accomplishments and issues are
  very important to report each month

Keep the Focus Area Monthly Reports simpleno
lengthy description of normal, scheduled
activitiesjust good news and problems
26
Back-Up Slides
27
Weekly Activity Reports

• Due to EA every Wednesday by noon Pacific Time
• Sample Content
• Netcentric Systems Test (NST) NST Evaluation
  Capability Module (NECM) The NECM project is
  developing an automated object-relationship graph
  algorithm . . . .
• This technology will automate the arrangement of
  operational activities, control logic, sequence
  flows, and . . .
• This algorithm enables improved user interaction
  to create a. . .
• During the week of 11 January 2010, the NECM team
  implemented an initial Joint Personnel Recovery
  (JPR) joint mission thread (JMT). . .
• The future efforts will include. . .
• Status/Assessment Green Early engagement of
  transition partners to define appropriate use
  case to verify useful test technologies, and
  project is operating on schedule and on budget.

28
TE/ST Program Roles and Responsibilities

• TE/ST Program Manager
• Provides guidance to focus areas to ensure
  disciplined project selection and funding process
• Provides program funds for administering and
  executing TE/ST Program
• Reviews technical execution of focus areas and
  projects to ensure TE/ST Program objectives are
  being met
• Reviews financial execution of focus areas and
  projects to ensure best value and fiscal
  responsibility
• Provides TESTWeb and the TE/ST Acquisition
  Community Connection (ACC) Special Interest Area
  to support exchange of financial, technical, and
  programmatic information throughout the TE/ST
  Program
• TE/ST Program Office Staff
• Supports the TE/ST Program Manager
• Reviews focus areas and projects to ensure
  technical objectives are being met
• Reviews focus areas and projects to ensure
  financial objectives are being met
• Provides recommendations to the TE/ST Program
  Manager on Program related matters
• Interfaces with EAs, Deputies, WG members and PIs
  on behalf of TE/ST Program Manager as required

29
TE/ST Program Roles and Responsibilities

• Focus Area Executing Agents
• Primary point of contact for TE/ST Program
  Office within focus area
• Maintain awareness of DoD TE needs within focus
  area and technology developments related to focus
  area
• Actively seek out high payoff TE/ST projects
  that address critical DoD TE needs and mitigate
  identified TE risks
• Issue Requests for Information (RFIs) and Broad
  Agency Announcements (BAAs) through contractual
  channels
• Responsible for technical and financial execution
  of projects approved by TE/ST Program Manager
• Chair for focus area working group (WG) and
  responsible for maintaining Tri-Service WG TE
  and ST representation
• Coordinate replacements with TE/ST Program
  Manager
• Responsible for maintaining current project and
  focus area information in TESTWeb and TE/ST ACC
  Special Interest Area

30
TE/ST Program Roles and Responsibilities

• Focus Area Working Group Members
• Act as government leads on projects where there
  is no intrinsic government lead
• Provide coordination of TE/ST projects with
  Service related efforts
• Identify and prioritize TE needs
• Support the EA on source selection efforts
• Review ongoing focus area projects
• Identify transition opportunities for projects
• Project Principal Investigators
• Execute projects approved by TE/ST Program
  Manager
• Conduct research efforts
• Provide technical progress reports to EA
• Identify and actively pursue transition
  opportunities for TE/ST projects
• Responsible for maintaining accurate financial
  information in TESTWeb

31
TE/ST Program Roles and Responsibilities

• Government Lead
• Attend major focus area reviews (when able)
• Provide guidance to the project to ensure a
  disciplined engineering approach
• Review technical execution of project to ensure
  NST Program objectives are being met gaps are
  being satisfied
• Provides quick review of monthly reports to
  ensure accuracy
• Interfaces with EAs, Deputies, WG members and
  Principle Investigators as required
• Maintain awareness of identified TE needs within
  the NST focus area and technology developments
  related to focus area
• Support the EA on source selection efforts
• Support EA in Identifying transition
  opportunities for project

32
Publications

• Professional publications are encouraged by the
  TRMC and PMO
• Provides opportunities for transition of
  technology
• Raises awareness of TE/ST Program to new
  communities
• Public Release of Technical Information
• EAs adhere to local procedures for release of
  technical information in EAs focus area
• Copies of all technical information to be
  released are to be sent to the TE/ST Principal
  Scientist prior to release
• Use the ACC Special Interest Area
  https//acc.dau.mil/CommunityBrowser.aspx
• Include acknowledgement of the TE/ST Program in
  all publications
• Example The authors would like to thank the
  Test Resource Management Center (TRMC) Test and
  Evaluation/Science and Technology
  (TE/ST) Program for their support.  This work
  is funded (or is partially funded) by the TE/ST
  Program through the (Put your organizations name
  here) contract number xxxxxx.

33
Release of Programmatic Information

• The PMO must review and approve release of
  programmatic information prior to transmission
  outside the program
• Applies to detailed project status, funding
  levels and execution, schedule, problem areas,
  and potential management issues
• Also applies to Solicitations (BAAs, PRDAs, RFIs)
• Not intended to restrict information exchange
  between EAs or core focus area working groups
• This applies to written and verbal release of
  information
• For publications, PMO requires a minimum of 10
  days before EAs next suspense on the publication
• For verbal or email Request for Information, call
  or email PMO for approval before information is
  released
• Option is to refer requestor to PMO for the
  programmatic information

34
Technology Readiness Levels (TRL)
35
TRL (Cont)
36
Definitions

• Breadboard A breadboard is an experimental
  model composed of representative system
  components used to prove the functional
  feasibility (in the lab) of a device, circuit,
  system, or principle without regard to the final
  configuration or packaging of the parts.
  Breadboard performance may not be representative
  of the final desired system performance, but
  provides an indication that desired performance
  can be achieved.
• Brassboard A brassboard is a refined
  experimental model that provides system level
  performance in a relevant environment. The
  brassboard should be designed to meet (at a
  minimum) the threshold performance measures for
  the final system.
• Prototype A prototype is a demonstration system
  that has been built to perform in an operational
  environment. A prototype device meets or exceeds
  all performance measures including size, weight,
  and environmental requirements. This device is
  suitable for complete evaluation of electrical
  and mechanical sub-system form, fit, and function
  design and characterization of the full system
  performance.

37
Accounting Definitions

• Obligations
• The amount of an order placed, contract awarded,
  service rendered, or other transaction that
  legally reserves a specified amount of an
  appropriation or fund for expenditure
• Accruals
• The costs incurred during a given period
  representing liabilities incurred for goods and
  services received, other assets acquired and
  performance accepted, prior to payment being made
• Disbursements
• The charges against available funds representing
  actual payment and evidenced by vouchers, claims,
  or other documents approved by competent
  authority
• Expenditures
• The total of disbursements plus accruals