LargeScale Simulation

1
Large-Scale Simulation
Information Sciences Institute, Computational
Sciences Division
22 March 2007Bob Lucasrflucas_at_isi.edu
2
Outline

• Background
• JFCOMs Urban Resolve
• ISIs Contribution
• Connecting to the Real World

3
Outline

• Background
• Experimentation
• STOW SF Express
• Scud Hunts
• Urban Operations
• JFCOMs Urban Resolve
• ISIs Contribution
• Connecting to the Real World

4
Experimentation

• As told to me by Gen Welch, USAF Ret.
• Training brings old experience to new people
• Experimentation creates new experiences
• Effects of new sensors or weapons
• New tactics or doctrine
• Military experiments are NOT games!!!

5
STOW and SF Express

• How I got interested in this stuff
• Synthetic Theatre of War (STOW)
• 1990s campaign level simulation
• Goal of 50K entities in mid-90s
• Network B/W limited
• SF Express project
• Partnership with Randy Garrett
• Use DARPA developed HPC systems
• SPAWAR (Caltech and JPL)
• Supported over 100K tanks

6
Scud Huntsback at the turn of the century
Iraqi Al Hussein (SCUD) Missiles
7
Urban OperationsCurrent focus!
8
Outline

• Background
• JFCOMs Urban Resolve
• Terrain
• Entities
• Large-scale Computing
• Experiments
• ISIs Contribution
• Connecting to the Real World

9
Urban Resolve Experiment
Joint Urban Operations Experiments Modeling and
simulation infrastructure Human-in-the-loop
10
Terrain Box
11
View of a City
12
Lots of People
13
Growth in Entities
2,000,000
Future experiments require orders of magnitude
larger more complex battlespaces
AO-00 pushed the limits of networked PCs
1,000,000
SPP Proof of Principle DARPA / CalTech Demo
500,000
107,000
50,000
12,000
3,600
UE 98-1 (1997)
SAF Express (1997)
Urban Resolve (2006)
J9901 (1999)
AO-00 (2000)
JSAF/SPP (2003)
14
Simulators

• Entity level
• JSAF
• Lockheed Martin, kinetic military
• Culture
• Lockheed Martin, civilians
• OneSAF OTB
• SAIC, US Army
• SOAR
• SOAR Tech, SOF spies
• SLAMEM
• Toyon, Sensors

Aggregate level SEAS Simulex, PEMSII
15
Heterogeneous Computing
Event Control
Pucker
16
Geographically Distributed
17
Data Intensive Too
Fully distributed logging at point of
generation RTI Interceptor captures simulation
events Archiver stores eventsto local
disk Decoderd stores events to local relational
database Binary DB for R/T queries SQL DB
for after action
18
Example of Experiments
Near-term - UR2005 Tactics E.g., Counter
Mortar/Rocket Long-term - UR2015 Direct research
expenditures Sensor platforms Doctrine How
to best employ future systems
19
Experimental Sensor Architecture
20
Outline

• Background
• JFCOMs Urban Resolve
• ISIs Contribution
• Scalable computing
• Scalable communication
• Data logging and management
• Data analysis tools
• Future architectures
• Connecting to the Real World

21
Scalable Computing
22
Scalable Communications
23
Data Collection
Dimensions organize data along lines of
interest Multiple dimensions Explore crosscutting
relationships Hierarchical dimensions Provide
summarization by defining partial orderings Drill
down, roll up, slice dice
24
Analysts Data Model

• Quantitative measurements
• Sensor effectiveness
• Damage Assessment
• Measurement aggregation operations
• Count
• Sum
• Max / Min
• Mean (count sum)
• Variance (count, sum, sum squared)
• Covariance

25
Distributed Data Analysis

• Data cubes are distributed across multiple
  compute nodes
• SDG provides summing managers to aggregate across
  nodes
• Sum cubes on demand based on user query through
  web interface
• Able to aggregate while the federation is running
• Low communication overhead
• Cube data size is typically much smaller the log
  data size

26
Urban Resolve Experiment - JFCOM
Distributed computation Virginia J9,
TEC California SPAWAR Ohio GLENN cluster 128
nodes Hawaii KOA cluster 128 nodes Networks Giga
bit within clusters 20 mbit site to site Data
volume Terabyte of data from 2 week exercise X10
x100 future capability Near real-time and
after action queries
27
Analysis Tools
Define customized dimensions tailored to analyst
needs Quickly create hierarchical dimensions with
tree-table drag-n-drop interface Reads in UR2015
DIS enumerations in csv format Writes to SDG
relational database dimension tables
28
Graphics Processors

• Theres more to COTS computing than just PCs
• Graphics Processing Units (GPU)
• Game Chips (IBM/Sony/Toshiba Cell)
• GPUs are intriguing
• Higher compute power than IA32
• Getting faster, faster ?
• GP GPU not trivial
• Non-standard architecture
• No standard languages
• HPCMP 2007 DHPI awarded to JFCOM

29
Opportunity for Graphics Processorsin Urban
Resolve
30
Proposed Line-of-Sight Algorithm
31
Outline

• Background
• JFCOMs Urban Resolve
• ISIs Contribution
• Connecting to the Real World
• Why Bother?
• Easy stuff
• Hard stuff
• Dreams

32
Experimentation Reality

• Less simulation
• More credible experimentation
• Allows for greater VV of experimentation
• Sore point today
• Allows real World to leverage sim S/W
• Exploit O(109) dollar training enterprise
• Experimentation is smaller
• At least data logging and analysis tools

33
Its Already HappeningJFCOM Sentient World
Mathematical Models
Inverse Problem

Emergent Behaviors
System Level

• Validation
• Shapes of Curves

Validity of Behaviors
Emergent Behaviors
Inverse Problem
Group Level
Forward Problem
Bottom-up Approach
Individual Level

• Advantages
• Validity of Emergent Behaviors
• Explanation at the individual behavior level
• Intervention at the individual level

Actions
34
Straightforward Stuff

• Logging real World events
• Conceptually be no different than simulated
  events
• Contribute simulation technology to real World
• Interest-managed communications
• Simulation provides framework for displaying
  data
• E.g. display plans for Blue forces
• Scalable Data Grid offers tools for querying
  data

35
Research Problems

• Assimilate real data into models
• What do simulated entities do in between
  updates?
• Can agents learn to emulate and anticipate?
• Aggregate vs. entity level assimilation
• Does the overall traffic density need to be
  correct?
• Do individual vehicles have to be correct?
• Analogous to problems with multiple resolutions
  in simulation
• Is it thus intractable?

36
My Ultimate Dream

• Simulation-based decision aid
• Assimilate real world data into simulated world
• Take it off-line
• Add Blue plans (red too from spies?)
• Speed-up the clock
• Run multiple simulations to get alternate
  futures
• Automatically extract the story from each
• From the ensemble, determine likely outcomes
• Inform decision makers
• Much of the above are open research problems
• Others have similar visions
• Course of Action Analysis (COAA)
• Proactive Intelligence (PAINT)
• Deep Green

37
Summary

• We want to play ?
• So would the rest of the experimentation World!