AEGIS Surface Warfare Closed-Loop Systems Engineering Using Operational Realistic Test

1
AEGIS Surface WarfareClosed-Loop Systems
EngineeringUsing Operational RealisticTest
Evaluation

• CDR Mike Good, US Navy
• PEO Integrated Warfare Systems
• Littoral Warfare Manager
• 22 October 2003

Robert Doyle CSC Advanced Marine Senior Engineer
Mark Davis Planning Consultants, Inc SUW Systems
Engineer
Calvin Foster Northrop-Grumman IT Deputy TE
Manager
2
AGENDA

• Challenge
• AEGIS Flight IIA Destroyer Configuration
• Closed-loop SE / TE Process Team
• At-Sea Testing
• Summary

3
CHALLENGE

• Demonstrate DDG 51 Flight IIA Surface Warfare
  (SUW) Performance vs. Massed Fast Patrol Boat
  (FPB) Raid

4
FLIGHT IIA SUW CONFIGURATION
Phalanx Block 1B Close In Weapons System Surface
Mode
Engagement
5/62 Caliber Gun
Armed Helo w/Hellfire Missile .50 Cal MG
Standard Missile Surface Mode
Standard Missile Surface Mode
Armed Helo w/Radar, FLIR
SPY-1D Radar
Mk 46 Optical Sight System
SPS-67 Radar
SQS-53 Sonar
Sensor
5
WHY CLOSED-LOOP?

• Directly Link Requirements to Results
• Instills Rigor At-Sea Test Time More Efficient
• Ensures Capture of Critical Data
• Ensures Operational Technical Balance
• Enhances Visibility of Results
• Developer
• Program Office
• Fleet

6
DOD-STD-2167A INTERPRETED FOR AEGIS
COMPUTER PROGRAM IMPLEMENTATION
COMPUTER PROGRAM TESTING
SYSTEM DEFINITION AND DESIGN
COMPUTER PROGRAM DEFINITION AND DESIGN
SYSTEM INTEGRATION TESTING
PRODUCT BASELINE
ALLOCATED BASELINE
OPERATIONAL TEST OPERATIONAL TEST
PLAN, PROCEDURES, REPORT
OPERATIONAL REQUIREMENTS DEFINITION TOP
LEVEL REQUIREMENTS
2167A PROCESSES
SYSTEM REQUIREMENTS ANALYSIS AND SYSTEM DESIGN
CPAP
SYSTEM DEMONSTRATION SYSTEM QUALIFICATION
TEST (SQT) TEST PLAN, PROCEDURES, REPORT
SOFTWARE REQUIREMENTS ANALYSIS
SYSTEM REQUIREMENTS DEFINITION A SPEC
PRELIMINARY DESIGN
MRP
DETAILED DESIGN
SYSTEM TEST EVALUATION, MULTI-ELEMENT
INTEGRATION TEST (MEIT) TEST DOCUMENTATION
ELEMENT REQUIREMENTS DEFINITION B1 SPEC
CODING AND CSU TESTING
CSC INTEGRATION AND TESTING
SDR
CSCI TESTING
COMPUTER PROGRAM PERFORMANCE REQUIREMENTS
DEFINITION PPS, CPRS
ENGINEERING TEST AND EVALUATION (ETE) TEST
MATRICES, PROCEDURES, REPORT
SYSTEM INTEGRATION AND TESTING
PDR
LEGEND
NAVY REVIEW
ELEMENT COMPUTER PROGRAM INTEGRATION AND
TEST TEST PROCEDURES, REPORT
COMPUTER PROGRAM DESIGN PDD, CPDD, IDS, CPDP
MULTIPLE ITEMS REVIEWED BY ELEMENT
LOCKHEED MARTIN INTERNAL NAVY INVITED
CDR CRITICAL DSGN REV CPAP CMPTR PRGRM
ACCEPT PNL CPDD CMPTR PRGRM DSGN DOCMT CPDP
CMPTR PRGRM DEV PLAN CR CODE REV CSC
CMPTR S/W COMPONENT CSCI CMPTR S/W CONFIG
ITEM CSU CMPTR S/W UNIT DDR DETAILED
DSGN REV ETE ENGNRNG TESTS EVAL IDS
INTERFACE DSGN SPEC IPR IN-PROCESS
REV MDF MODULE DEV FOLDER MEIT
MULTI-ELEM INTEG TEST MRP MISSION READINESS
PNL PDD PRGRM DSGN DOCMT PDR PRELIM
DSGN REV PPS PRGRM PERF SPEC SDR SYS
DSGN REV SQT SYS QUAL TEST TPR TEST PRGRM
REV
CDR
IPR
UNIT TEST MODULE/CSU TEST, PROCEDURES, MDF
COMPUTER PROGRAM DETAILED DESIGN MDF
CODE MODULE/CSUs AND DATA
CODE MODULE.CSUs AND DATA
7
AEGIS COMBAT SYSTEMS ENGINEERING PROCESS
SEC Test Objective Development
Element/System Test
Test Plan/Execute
Report Out
Issue Closure
WSPA Issues
Baseline Definition/CP Development
Performance Assessment
30 Day WSPA
Performance Review Board (As Required) IWS 1B1
Dep (Chair) DD (Secretariat) PHD COR LM WAMs ATRC
SWDG AEGIS Techrep
Retest Requirements Additional Data
Required Ongoing Investigations Training Physics
LIMS Impact Assessment TTP Development TTP
Viability
60 Day Performance Review Board IWS 1B1 Dep
(Chair) COR (Secretariat) PHD DD LM WAMs ATRC SWDG
AEGIS Techrep
New Requirements Future Baseline Design
8
CLOSED LOOP SE / TE
System Engineering Requirements Operational
Requirements DT/OT Deficiencies New
Capability Previous Test Results
Identify Performance Deficiencies Identify
Training Deficiencies Simulation/Validation Retire
Completed TOs
Report System Performance Quicklooks Test
Objective Assessment WSPA/TESPA Semi-Annual Prog.
Office Review WSPR/TESPR
Test Objectives (TOs) Measures of
Effectiveness Test Kinematics
TO Assignment to Test Event
Scenario Development Performance Predictions
Observations Results
Test Execution and Data Analysis Plan
Testing
Test Plan Certification
9
SUW SYSTEMS ENGINEERING COUNCIL
AEGIS CSE / TE SUW SEC Panel
SEC Admin/Support

• Test Objective WG
• TO Development
• Edit/Revise
• Initial Approval
• Submit/Input into the TO Database

• Scenario WG
• Target/Ordnance Requirements
• Surface Warfare
• Warfare Analysis
• Modeling Simulation
• Range Requirements IPT
• Target Requirements IPT
• Ordnance Requirements IPT
• Warfare Analysis/MS IPT
• Element/AWS Reps/SME IPT

• Data Analysis WG
• (Pre-Event Post-Event)
• DMAP
• WSPA/TESPA
• TORs
• Test Reports
• Quicklooks
• Final Analysis
• Sensor Performance
• Element Performance

New/Different TO
PEOIWS 1B1C CSE Review
ORG View
More Data
Improvements
1 Week
Scenario Development
TO Development
Scenario Certification
Event View
Scenario Finalization
TO Assignment
TO Approval
Scenario Pre-Cert
3 Months
1-2 Months
3 Months
Conclusions Recommendations
Formal Report
Conduct Event
Analyze Event
Final TP Due To The Ship
Draft TP
DMAP
10
SYSTEMS ENGINEERING AT-SEA TE

• Test Execution and Analysis
• Confirm All Systems (Ship and Range) Are Ready
• Execute Test
• Gather Weapons, Range and Target Data
• Conduct Quick-Look Analysis Onsite
• Conduct Detailed Data Analysis Working Group
• Close Loop With Updated Discrepancies and
  Findings For Future Testing...

• Test Objectives
• Derived From Operational Requirements Document
  (ORD) and TE Master Plan (TEMP)
• Match to Validated Threat
• Create Measures of Performance
• Verify Capability to Test

• Test Planning
• Insert TOs Into Combat System Ship Qualification
  Trial (CSSQT) SUW Test Phase
• ID Upgrade Range Capabilities, Target
  Performance and Data Collection Systems

11
READINESS BUILDING BLOCKS
Surface Events
DDG 84 801 OT HSMST Event AFWTF, Feb 03 GWS,
Armed Helo, CDF, SM-2 and CIWS BLK IB
Goal SuccessfulOT
DDG 84 CSSQT/801 DT HSMST Event AFWTF, Nov 4-5,
02 GWS, Armed HELO, CDF, CIWS BLK IB, SM-2
C2 Tactics Training Range
Verify Test Plan Can Be Executed, Range Can
Support
Range Risk Reduction With Multiple Targets
Transfer Targets to AFWTF, Conduct Range Risk
Reduction Dry Run Oct 10, 02 5 Targets
Range
Readiness Building Blocks
Provides Introduction of CIWS BLK IB Into Ship
Defense Capabilities, Risk Reduction, Complex
Scenario, Multiple Elements, Validate Procedure,
Command/Control
DDG 83 CSSQT HSMST Event PMTC, Mar 02 Planned
GWS, CIWS BLK IB Actual GWS Non-Firing, CIWS
BLK IB
Engineering Work C2 Tactics
DDG 82 CSSQT/801 DT HSMST PMTC, Mar 02 Planned
GWS, Armed Helo, CDF, 5 Tgts Actual GWS, Armed
Helo, CDF, 4 Tgts
Provides Test Plan Proof of Concept, Complex
Scenario, Multiple Elements, Validate Procedure,
Command/Control
Planned Range Actual Engineering
DDG 81 CSSQT HSMST Event AFWTF, Feb 02 Planned
GWS, 3 Target Actual Non-Firing/Ships RHIBS
Planned Provide Range Risk Reduction
Opportunity Actual Identified HSMST Mode Issues
12
DDG 84 HSMST DEVELOPMENTAL TEST
Armed Helicopter Engage

• DT Events 11/4-5
• Manned Raid
• Unmanned Raid
• Live Fire Event 11/16
• Present 5 HSMST RHIBs
• CDF Queue OTH
• Armed Helo Search
• Ship Coordinates Layered Defense
• Helo w/ Hellfire Missiles
• STANDARD missile
• 5/62 Gun
• CIWS Blk 1B

GWS Hazard Area
Standard Missile GWS Engage
CIWS Engage
CDF Queue
T1
T2
T3
T4
T5
13
SUMMARY

• Conducting Small Boat Defense Testing Is More
  Complex Than Most Appreciate
• Operationally Meaningful Testing, With Measurable
  Results, is Critical to ID of Deficiencies/Solutio
  ns
• End-To-End Detect to Engage Evaluation Requires
  Holistic Organizational and Technical Approach
• Collaboration With Fleet, SWDG and OPNAV
  Leveraged Limited Resources to Address Critical
  Capability

Build a Little, Test a Little, Learn a Lot