Joint JSBJVB Components for

1
05S-SIW-051
Joint JSB-JVB Components for Composable Mission
Space Environments for Sensors and C4ISR

Mr. Russ Moulton rmoulton_at_jrmtech.com Mr.
David Geyer dgeyer_at_virtc.com Dr. Gary
Eiserman geiserman_at_virtc.com Mr. Jeff
Wallace jwallace_at_envoytekinc.com Mr. Wayne
Civinskas wayne.civinskas_at_lmco.com Mr. Mark
Henneberry mhenneberry_at_virtc.com Mr. Peter
Wickis peter.wickis_at_lmco.com Ms. Deborah
Wilbert deborah.wilbert_at_lmis.com Dr. Chris
Fink cfink_at_jrmtech.com Mr. Ken
George kgeorge_at_jrmtech.com
2
Background

• Need Statement
• Currently, creating large scale environments for
  credible sensor and C4ISR experiments involves
  costly and time-consuming re-development "from
  scratch" to satisfy specific program requirements
• In response, this effort is developing
  break-through
• Architectures
• APIs
• Proof-of-concept components

For rapid, low-cost composition of high-fidelity
sensor and C4ISR mission space environments
3
Basic Phenomenology
Basic spectral quantities
Meteorology (Irradiance) At-Object (Emission
Reflection) Atmospheric Thermal
Radiance Atmospheric Scattered Radiance Atmospheri
c Transmission
4
Phenomenology
Sensor-Environment Interactions
NVG, FLIR, MTI, SAR COMM SIGINT
Spectral Solar, Lunar, Star, Sky Manmade
Irradiances
Direct Reflected Passband Irradiance
1D or 3D Model Atmospheric Propagation Modeling
Scattered Attenuated Direct Diffuse Spectral
Irradiance Atm Emission
Rsurf,s

• EO/IR/RF Signature Synthesis
• Material System-encoded DBs
• Spectral BRDF dir. radiance
• Transient Thermal Modeling
• (Diurnal cycle, solar loading, conduction,
  convection, radiative cooling, dynamic vehicle
  states)
• TargetCultural Feature Model RCS(s) terrain s0

Tsurf Rsurf(az,el,l) s(az,el,f) or s0(az,el,f)
or Emitter Power
Computing at-aperture signatures from many points
on the Targets Clutter Objects, as well as
surrounding background
5
Overview of Modular Approach
Break-down phenomenologies into
logically-separable components within a 3D scene
space

• Fast Radiative Transport Engine
• Fundamental, used frequently by other routines
• Can be augmented with DB query capability for
  occlusions
• Spectral Meteorological Engine
• Propagates natural and man-made irradiances using
  transport engine
• Signature Synthesis Engines
• Temp(surface, time) high fidelity case can be
  precomputed or farmed out to separate process
• Planckian or non-Planckian emission
• Parameterized BRDF / RCS lookup (freq,
  polarization, aspect angle)

6
Advantages of a Modular Architecture

• Advantages
• Facilitates Distributed, Parallel
  GPU-processing acceleration
• Provides interfaces for integrating other
  phenomenology model codes
• Ensures consistency / interoperability
• Allows focused optimization on the real
  bottlenecks

7
Block Diagram of Modules
8
Block Diagram of Modules
Environmental Influences Module
At-Surface Irradiances, Wind speed, T, P, H BCs
GetEnvInfuences (3Dpt) GetNatIrrad(),
GetManMadeIrrad(),
TOA Irrad, Geometry
Transport Engine
Signature Synthesis Modules
Surface Irrad, BCs BRDF, Emissivity, Geometry
Surface Temperature RCS
UpdateSignature()
Transport Engine
At-aperture Radiance Modules
Surface Irrad Surface Temp BRDF RCS Ptrans
R_total (vect, passband)
Radiance(), Radiance2D, SAR2D()
Transport Engine
9
Terrain
Traditional CTDB Terrain DB
Correlated High-Resolution Material-texture
encoded 3D DB

• Traditional CTDB used for mobility, fast LOS
  checks
• A correlated Material-encoded DB for use with
  sensor models
• Library architectures for dialable-fidelity-perfo
  rmance processing/serving from the
  material-texture DB
• Architectures leverage high-performance COTS GPUs

10
Signature Services
Correlated Material-texture encoded, higher
resolution 3D DB
Rendering Libraries

• Can use COTS GPUs
• 16 x 16 samples Low-fidelity, extremely fast
  performance
• 512 x 512 samples High-fidelity Sensor Image
  Rendering

11
RenderRadiance()

• Sensor federate calls two routines
• RenderTargetRadiance() synthesizes 2D spectral
  radiance field from material-system-encoded
  target model, at any arbitrary LOS, atmospheric
  state
• Render an entity from any view in EO/IR/RF
• RenderBkgndRadiance() to render spectral the
  background from any view EO/IR/RF
• Virtual Sims
• See target/background just as man-in-loop
• Constructive Sims
• Operate on images
• ACQUIRE model
• DT, DC

12
Integrated Architecture
Sensor Model
START
END
Apply detector response
Make initial query
Composed EO/IR at-aperture services
Viewing (unit) vector wavelength bandwidth fidelit
y, etc
Total at- aperture L(l)
Radiance from independent sources
Radiance from dependent sources
Signature Updates
View vector
3d point
Material/Property-encoded Atmospheric DB Terrain
DB Entity DB
Transport Library
13
API-level Composability IR/RF
// SE DB LOADING Initialization LoadAtmosphere(a
tm.file, ssATM) LoadTerrain(terrain.file,
ssTERRAIN) LoadEntity(MIG23.file, ssMIG23)
LoadEntity(T72.file, ssT72) LoadSensors(sensor.f
ile, SENSOR) // SIMULATION TIME
LOOP AdvanceTime() UpdateEntityState(ssMIG23)
UpdateEntityState(ssT72) GetEnvInfluences(altWGS
84, spec_metGROUND) //T72 also on ground of
course! GeEnvInfluences(MIG 3D position,
spec_metMIG23) UpdateSignature(ssTERRAIN)
UpdateSignature(ssMIG23) UpdateSignature(ssT72
) UpdateSensor(sens0) UpdateSensor(sens1)
GetLOSAtmospherics(sens0, ssMIG23)
GetLOSAtmospherics(sens0, ssT72) GetLOSAtmosp
herics(sens1, ssMIG23) GetLOSAtmospherics(se
ns1, ssT72) Radiance2D RenderRadiance(sens
0) SAR2D VectorSAR(sens1) ComputeAcquirePa
rams()
14
Material/Property-encoded Data Representations At
mosphere Terrain Entities
15
Atmospheric Environment Modeling
Propagation Services
Temperature (x,y,z) Pressure (x,y,z) Relative
Humidity (x,y,z) H2O (x,y,z) O3 (x,y,z) CH4
(x,y,z) N20 (x,y,z) CO (x,y,z) Cloud Type
(x,y,z) None, Cumulus, Alto-stratus,
Stratus-Strato-Cumulus, Nimbo-stratus Rai
n Rate, etc
1D Vertically Stratified or 3D Grids
16
3D Atmosphere Rep
Pre-processing stage___________________

path-and-angle- independent extinction, radiance,
and scattering coefficients per layer (cell).
Given 3D grid of cells, each with its own temp,
pressure, humidity, H20 and aerosol concentrations
, etc
Temp(z) Pressure(z) RelHum(z) H2O(z) etc.
NOTE Spectral !!
17
Propagation Services
Correlated Spectral Band Atmospherics
18
Propagation Services
Correlated Spectral Band Atmospherics
19
Material Systems
Material System Concept In SEDRIS EDCS
MATERIAL Homogenous composition of matter with
specific values of intrinsic (context-independent)
physical properties
Convection _at_ 19.2 degC, 0.44 m/s, Direct and
diffuse solar, and radiative cooling
LAYER is 1 or more materials, each of arbitrary
aggregate mass in the layer.
Asphalt 0.05 m
Gravel 0.05 m
1D MATERIAL SYSTEM Set of layers of arbitrary
thickness, with context-dependent boundary
conditions.
30Loam 70Clay70 0.82m
Granite_Bedrock 15 m
Provides hook for physical property
attributions throughout the terrain.
Concept incorporated in EDCS
Isothermal Conduction at 60 deg F
20
Material Systems

• 1D Material System
• Thermal Solver Mode (energy balance, transient)
• Thermal Boundary Conditions at Top and Bottom
  (Convective, Conductive, Insulating, etc..)
• Number of Layers
• Material ID, Thickness of each layer

• Material
• Bulk Properties Density, Thermal Conductivity,
  Specific Heat, Latent Heats of Fusion,
  Evaporation, Sublimation, Solar Absorptivity,
  Lambertian Emissivity, BRDF model to use, Angle
  at which DHR measured.
• Spectral Properties BRDF parameters such as
  DHR, Specular, Lobe Width, Shininess

21
Material Properties
Material / Material System Physical Property
Database
22
BRDF Models
All Spectral
23
Material/Property-encoded Data Representations At
mosphere Terrain Entities
24
Automated Material Classification
Signature Predictive Classifier (SPC)
Spatial Correlation Refiner (SCR)
Iterative Context Adjuster (ICA)
or
Picks hypothesis material-systems by fitting
at-aperture signature predictions from library of
material systems to n-channel pixel data
Resolves ambiguities by using nearest neighbor
pixels/spatial data to refine material assignments
Uses geographical and cultural information to
further refine material system library/values
25
Automated Material Classification
Original RGB Image
Synthesized Visible Image
Material Classifications
Signature Predictive Classifier (SigSim in
reverse on 200 Material Systems)
Material system file (5-channel .mcm above)
Generated RGB image from hypothesized material
systems (SigSim)
26
Environmental Influences -- Spectral Irradiance
Fields Ambient Boundary Conditions
27
Meteorology Module
Spectral Irradiance Modeling
Exo-atmosphere solar lunar irradiance data from
Modtran 4.0
Transmission applied to this path
Hemispherical-integration for Isky
28
Meteorology Module
Star Ephemeris and Irradiance
SigSim provides stellar ephemeris for over
300,000 stars Fetches star magnitudes from
catalogue, and corresponding integrated radiant
intensity Apply relevant filter function and
invert equation to obtain temperature Then
computes stars spectral power densities from
Planck equation
29
Meteorology Module
SigSim Spectral Irradiance Output
Example 0.4-2.2 micron passband
30
Meteorology Module
SigSim Solar Spectral Irradiance Output
Visible region, at 1km altitude, local noon
31
Meteorology Module
SigSim Lunar Spectral Irradiance Output
Visible region, at 1km altitude, local noon
32
Meteorology Module
Spectral Meteorology Modeling

• Vertically stratified or 3D Gridded Atmospheric
  Data Structure
• Arbitrarily fine Layer or Grid Size
• ? e.g. 3 km and 1 km OaSES / EnviroFed Cells
• Function for Pre processing of cell spectral
  coefficients, or on-the-fly
• via SigSim Modtran- and Radtran-based algorithms
  
• performed at each 20 min OASES update 100 cells
  lt 60 secs
• Function for surface spectral irradiances
  prediction at any altitude
• via very fast SigSim path-integrals and/or
  hemispherical integration to Top-of- Atmosphere
  (TOA)
• spectral irradiance predictions gt 30 Hz

33
Atmospheric Propagation Modeling
34
SigSim Real-time 3D Atmospherics
Propagation Module
Real-time atmospheric propagation library
computes arbitrary LOS EO/IR /RF passband
transmission and atmospheric radiance/noise
35
Propagation Module
SigSim Spectral Atmospherics
36
Propagation Module
37
Propagation Module
38
Signature Modeling
39
SigSim 1D Fast Transient Thermal Modeling
Automated Material Classification
40
Observables (Signature) Federate
41
Background RF
Background RCS Modeling
Via Ulaby/Dobson Parameterization
42
OTF Thermal
Thermal Modeling Medium Fidelity Parameterized

• Approach - Break up vehicle into n isolated,
  active thermal regions based on boundary
  conditions, each with two, 1D basis functions
  TL(z,t), TH(z,t) and one 2D weighting function
  H(x,y)
• Can do 1D thermal transient solution with dynamic
  BCs in Real Time 1D thermal systems at 1000 Hz
  on PCs

43
OTF Thermal
SigSim 3D Fast Transient Thermal Modeling
Empirical or 3D model Steady State thermal
gradients defined for primary active regions of
vehicle model Gradient curve-fit by percentage
mixture of two 1D material systems Active and
ambient boundary conditions based on actual
vehicle dynamic states Dynamic run-time gradient
is modulation of the two 1D MS Error
quantifiable in spatial and temporal terms,
manageable
44
OTF Thermal
1-D solution covering active BC TH(z,t) at
(xH,yH)
Z-direction, heat flow
1-D solution covering ambient BC TL(z,t) at
(xL,yL)
Temperature anywhere on the hood, at any time can
be represented by a linear combination of these
two 1-D basis functions, TH(z,t) and TL(z,t),
weighted by H(x,y)
Th(x,y) TL(x,y)arbitrary units
H(x,y) is obtainable either from empirical
measurement (FLIR) or 3D predictive modelsthermal
imagery data (FLIR, etc.)
Distance from (xH,yH)
45
OTF Thermal
SigSim 3D RT Transient Thermal ModelingVia Q3D
ViXsen Dual-MS-mixture
46
Target RCS
47
RF Synthesis
48
OTF Sigs
49
Observables (Signature) Federate
T72 moving T0 sec.
T72 stopped and engine idling T1 min.
T72 engine idling T2 min.
T72 engine idling T3 min.
T72 engine offT5 sec. post-idle
T72 engine offT50 sec. post-idle
T72 engine offT10 min. post-idle
50
Signature Services
Correlated Material-texture encoded, higher
resolution 3D DB
Rendering Libraries

• Can use COTS GPUs
• 16 x 16 samples Low-fidelity, extremely fast
  performance
• 512 x 512 samples High-fidelity Sensor Image
  Rendering

51
RenderRadiance()

• Sensor federate calls two routines
• RenderTargetRadiance() synthesizes 2D spectral
  radiance field from material-system-encoded
  target model, at any arbitrary LOS, atmospheric
  state
• Render an entity from any view in EO/IR/RF
• RenderBkgndRadiance() to render spectral the
  background from any view EO/IR/RF
• Virtual Sims
• See target/background just as man-in-loop
• Constructive Sims
• Operate on images
• ACQUIRE model
• DT, DC

52
(No Transcript)
53
(No Transcript)
54
(No Transcript)
55
(No Transcript)
56
(No Transcript)
57
(No Transcript)
58
(No Transcript)
59
Deliverables

• Joint FOM for sensors C4ISR
• JSB, MATREX/FCS, JVB Compatible
• Data Distribution Manager (DDM) design
• Scalable fidelity and performance
• High-entity-count simulations
• Federate/Service Architectures
• Common, high-performance for signatures
  propagation
• Proof-of-concept components
• Technology Demonstration
• Rapid deployment
• Low-cost
• High-entity-count

60
Common FOM
Attenuation
SimulationService
Observables
EO
IR
61
Common FOM (contd)

• Attenuation
• PreviousInteractionID
• Direction
• Coefficients
• Array of attenuation coefficients, wavelengths
  and incidence angles
• TargetPlatformID
• SensorID

62
Common FOM (contd)

• Observables
• Sensor information
• Target information
• Observables.EO
• TargetPresentedArea
• EOContrastInBand
• ImageNumber
• AverageContrast
• BackgroundEmittance

• Observables.IR
• TargetPresentedArea
• TemperatureDifferenceInBand
• ImageNumber
• AverageDeltaTemperature
• BackgroundTemperature

63
Conceptual Model
Sensor
3 at-target signature
Observables
1 sensor characteristics
3 attenuation
Propagation
2 ground truth
Target
64
Initial Federation Architecture
Physics-Based API (SigSim JSB))
Physics-Based API (SigSim JSB))
Signature Data
RTI
hlaEval Data Analysis
Collected Data
65
Alternate Architecture
Recompose functionality to support different
performance or fidelity goals
Observables ServicePhysics-Based API
Signature Data
RTI
hlaEval Data Analysis
Collected Data
66
DDM
67
Data Distribution Manager (DDM)

• A common data distribution management (DDM)
  approach for addressing high-entity-count
  (30,000) scalability requirements for JSB,
  JVB/MATREX and JDEP
• The design of the common DDM approach included
  review of the existing JVB/MATREX, MC02 and
  proposed JSB CSE DDM approaches for addressing
  the unique requirements of both the air and
  ground domains
• DDM is a filter mechanism that enables federates
  to limit the data they receive to only the data
  they are interested in
• Standard HLA filter mechanism (Declaration
  Management) only allows filtering based on the
  type of data (e.g. give me all Targets)
• Using DDM we are able to allocate the processing
  for a subset of targets to each Propagation
  federate instance

68
DDM Architecture

• As targets are instantiated in the federation and
  as their values are updated they are associated
  with an update region
• Each propagation server subscribes to a specified
  region or range of regions
• The RTI sends all data of the appropriate type
  with a region value of X to the propagation
  federate that is subscribed to the region X

69
DDM Architecture (contd)

• Targets can be assigned to regions as they are
  instantiated, not geographically
• This allows us to run N propagation federates
  where each of those federates receive data for
  1/Nth of the targets
• This ensures that all propagation federates will
  be equally loaded
• Targets can also be assigned geographically
• This enables shorter range, ground based sensors
  to only receive signals from sources that are
  physically close enough for them to be seen

70
DDM Architecture (contd)

• As sensors are instantiated in the federation
  they publish a DDM region number that indicates
  which DDM region they will subscribe to
• When a propagation federate produces an
  interaction for a sensor, it only publishes it to
  the DDM region for that sensor
• This significantly reduces the amount of incoming
  interaction traffic received by each sensor
  federate

Sensors are served signals from each of
the propagation servers via their own DDM regions
DDM 2
Target DDM Regions
Targets
71
Technology Demonstration

• Compose a proof of concept test federation using
  components from
• JSB Common Synthetic Environment
• MATREX hlaEval data analysis tool
• JDEP Model Engine Architecture
• (Sensor, Propagation, TargetGenerator and 3D
  Viewer federates)

72
3D View of Test Scenario
73
Propagation Data
74
Observables Data
75
Observables Data
76
Conclusion

• Developed a set of core components
• relatively quick and easy incorporation into
  different federation architectures
• current implementations could be replaced with
  minimal impact on the functioning of the
  remaining components
• Design approach used here could be
• naturally extended to other sensing modalities
• used as a model for designing composable
  components to simulate other physical processes
• Leveraged to support an efficient, physics-based
  EOIR Server

77
Future Directions Leveraging these concepts
into rapidly-composable EO/IR Server Architectures
78
Virtual FLIR w/o Target
Virtual FLIR with Target
EO/IR Sensor Server
Target-only Pixels
79
EO/IR Sensor Server
White Boxes False Alarms Red Box Target
Detection
CGF Sensor Perception via TAA
80
(No Transcript)