Dynamic Data-Driven Application Systems (DDDAS)

1
Dynamic Data-Driven Application Systems (DDDAS)

• What is DDDAS?
• Dynamic Data Driven Application Systems
• DDDAS is a new paradigm.
• Old ways Application simulations use static data
  input and start up.

2
DDDAS (continued)

• Application simulations are able to
• RECEIVE AND RESPOND to ONLINE
• physical data and measurements and/or
• control the measurements.

3
Why DDDAS?

• Computer models were not designed to deal with
  dynamic conditions as the simulations already
  started.
• Example simulating wild fire. We need to know
  which community needs to be evacuated. The wild
  fire simulation needs to adjust based on the wind
  direction and other factors.

4
Why DDDAS (continued)

• Also fueled by advances in
• Applications and algorithms for parallel and
  distributed platforms.
• Computational steering and visualization.
• Computing.
• Networking.
• Sensors and data collection.
• System software technologies.

5
What DDDAS Needs

• Three research areas
• Application data driven technology.
• Algorithm dynamic data injection and data
  perturbation tolerance.
• System software support for dynamic
  environments.
• Static applications can be changed into a more
  useful
• dynamic data driven applications.

6
DDDAS General Properties

• Whats new in DDDAS?
• Feedback and control interactions between
• computations and the physical measurement
  systems.
• There are three ways to interact with DDDAS
• computation
• Human interaction.
• Physical system interaction.
• Computational infrastructure interaction
  (machines and their connections).

7
DDDAS Interactions
8
DDDAS Key Characteristics

• Key characteristics that need to be addressed
• Time dependency and/or real time aspect.
• Data streams in addition to data sets.
• Interactive visualization and steering.

9
DDDAS Example

• Tsunami simulation.
• (http//www.pgc.nrcan.gc.ca/geoscapevictoria)
• Where we get the data from?
• Tsunami can be caused by tectonic plate
• movement, or sea floor quakes.
• Data sources GPS stations, sea floor sensors.

10
DDDAS Example 1 (continued)
11
DDDAS Example 1 (continued)
12
DDDAS Example 1 (continued)
13
DDDAS Example 2

• Air traffic simulation.
• (http//www.simlabs.arc.nasa.gov/cvsrf/atcs.html)
• Where we get the data from?
• We need the world location for all the aircrafts,
  as well
• as their schedules. Outside data weather
  simulations.
• Data source Aircraft transponders.

14
DDDAS Example 2 (continued)
15
DDDAS Example 3

• Medical imaging and simulation.
• (http//www.bitc.gatech.edu/bitcprojects/eye_sim/e
  ye_surg_sim.html)
• The Georgia Institute of Technology and the
  Medical College of
• Georgia.
• We need the real life model of the object of
• interest. We can do this by using Intensity
• Modulated Radiation Therapy (IMRT).
• Simulating eye surgery. To blind or not blind.

16
DDDAS Example 3 (continued)
17
DDDAS Example 3 (continued)
18
DDDAS Researches Past, Present, and Future.

• Reference taken from DDDAS.org Home Page,
• NSF Official DDDAS Page, NSF 2000
• Workshop on Dynamic Data-Driven Application
• Systems, and Performance Engineering
• Technology Notes.

19
DDDAS Past and Present
20
Past Experience

• February 17, 2000 Meteorologists missed
  predicting the track and magnitude of a major
  storm in January 24-25, 2000, that blanketed
  major cities from South Carolina to New England.
• May 7, 2000 The National Park Service started a
  controlled burn near Los Alamos National
  Laboratory. Within a day, the fire was labeled a
  wildfire.

21
What is being sought from DDDAS?

• DDDAS Capability simulation applications that
• can dynamically accept and respond to field
• data and measurements, and can control such
• measurements in a dynamic manner through
• the symbiotic measurement and simulation
• systems.

22
How DDDAS Affects Our Present and Future

• Examples of Applications benefiting from
• DDDAS
• Engineering Design and Control aircraft design,
  oil exploration, semiconductor manufacturing,
  structural engineering, computing systems
  hardware and software design and runtime.
• Medical customized surgery, radiation treatment,
  BioMechanics/BioEngineering

23
How DDDAS Affects Our Present and Future
(continued)

• Economy Production planning and control,
  financial trading (stock market, portfolio
  analysis).
• Crisis Management and Environmental Systems
  Weather, hurricane/tornado prediction system,
  floods, fire propagation, transportation systems
  (emergency planning, accident response)

24
Challenges in Enabling DDDAS Capabilities

• Application Simulations Developments
• Algorithms
• Computing Systems Support

25
Challenges in Enabling DDDAS Capabilities
(continued)

• Applications
• Ability of the application to interface with
  measurement systems.
• The stream data might introduce new modalities to
  describe the system, like a different level of
  the physics involved, in cases where the analysis
  is about a physical system.
• Ability to dynamically select the application
  components depending on the dynamically streamed
  data.

26
Challenges in Enabling DDDAS Capabilities
(continued)

• Algorithms
• Stable to dynamically injected data/ tolerant to
  perturbations of dynamic input data.
• Visualization with a human in the loop feedback
  to the simulations.
• Handling data uncertainties continuous
  sensitivity analysis

27
Challenges in Enabling DDDAS Capabilities
(continued)

• Computing Systems Support
• Support environments where the application
  requirements change during the execution
  depending on the streamed data/dynamic execution
  support on heterogeneous environments.
• Extended spectrum of platforms Grid Computing
  and beyond.

28
What is Grid Computing?
29
What is Grid Computing? (continued)

• Grid computing enables the virtualization of
  distributed computing and data resources such as
  processing, network bandwidth and storage
  capacity to create a single system image,
  granting users and applications seamless access
  to vast IT capabilities.
• Just as an Internet user views a unified instance
  of content via the Web, a grid user essentially
  sees a single, large virtual computer.
• At its core, grid computing is based on an open
  set of standards and protocols that enable
  communication across heterogeneous,
  geographically dispersed environments. With grid
  computing, organizations can optimize computing
  and data resources, pool them for large capacity
  workloads, share them across networks and enable
  collaboration.

30
What is Grid Computing? (continued)

• Like the Web, grid computing keeps complexity
  hidden multiple users enjoy a single, unified
  experience.
• Unlike the Web, which mainly enables
  communication, grid computing enables full
  collaboration toward common business goals.
• Like peer-to-peer, grid computing allows users to
  share files.
• Unlike peer-to-peer, grid computing allows
  many-to-many sharing not only files but other
  resources as well.
• Like clusters and distributed computing, grids
  bring computing resources together.
• Unlike clusters and distributed computing, which
  need physical proximity and operating
  homogeneity, grids can be geographically
  distributed and heterogeneous.
• Like virtualization technologies, grid computing
  enables the virtualization of IT resources.
• Unlike virtualization technologies, which
  virtualizes a single system, grid computing
  enables the virtualization of vast and disparate
  IT resources.

31
Why DDDAS Now?

• DDDAS has the potential to revolutionize
• science, engineering, medicine, economy,
• management systems, and etc.
• We have technological progresses that has
  advances
• the level of overcoming the challenges
• - computing speed (terascale, uni- and
  multi-processor systems, Grid Computing, Sensor
  Networks).

32
Why DDDAS Now? (continued)

• - System software
• - Applications (parallel and grid computing)
• - Algorithms (parallel and grid computing,
  numeric
• and non-numeric techniques, data assimilation,
• and chaotic Monte-Carlo method).

33
Summary and What the Future Holds

• NGS Next Generation Software (1998 - )
• Develops systems software supporting dynamic
  resource execution

34
Summary and What the Future Holds (continued)

• SES Scalable Enterprise Systems Program (1999,
  2000-2003)
• Geared toward commercial applications.
• ITR Information Technology Research (NSF-wide
  project)
• Has been used as an opportunity to support
  DDDAS-related efforts.

35
Summary and What the Future Holds (continued)

• Simultaneous advances on the models, methods, and
  algorithms that underpin the components and on
  their systematic integration to target strategic
  applications are crucial for realizing the
  potential of DDDAS.
• But hardware, software, and Cyber Infrastructure
  alone are insufficient to achieve this goal.

36
Case Study Architecture of the DDDAS Wildfire
Model
37
Major Components of the Model

• Coupled atmosphere/fire model
• Legacy NCAR code
• Combined with the latest techniques, such as
  OpenMP and Multigrid
• Data acquisition
• From the Internet GIS maps, past fire
  information, weather
• Field information photos taken from aircraft,
  field sensors
• Visualization and user interface
• runs on PDAs or cell phones in the field
• Dynamic Data Assimilation control module
• Incorporates data from the field
• Bayesian data assimilation
• Data assimilation steers the data acquisition
• Guaranteed secure communication infrastructure

38
The NCAR Coupled Atmosphere/Fire Model

• The interaction of fire and atmosphere is
  important
• Heat flux from the fire to the atmosphere
  produces fire wind
• Wind facilitates the fire spread
• Traditional fire models cannot represent this
  interaction
• Wildfires are difficult to model
• Limited computational resources
• An meteorological model is coupled with an fire
  spread model
• The atmosphere model is based on the Clark-Hall
  Atmospheric Model
• Fire model can be an empirical model, or a more
  realistic Stochastic Reaction-Diffusion Equation
  Fire Model
• Represents the important interaction between fire
  and atmosphere, more accurate and closer to
  reality

39
Data Acquisition

• Geographical, weather, and fire information
  available from the Internet
• Weather information NOAAPORT broadcast, MesoWest
  weather stations, and the Rapid Update Cycle
  (RUC) weather system by NOAA
• Past fire information the GeoMAC project at USGS
• Fuel type national database
• Advancement of the fire front
• infrared pictures taken from aircraft
• GPS for aircraft position
• 3-axis inertial measurement to get the pointing
  direction of the camera
• Field fire and weather information data
• Sensors arbitrarily placed in the vicinity of a
  fire for point measurements of fire and weather
  information

40
Visualization and User Interface

• Simulation results (prediction) will be
  transferred to the firefighters in the field
• Need an easy and portable way to visualize the
  simulation result
• Firefighters dont want to carry a notebook when
  fighting fires
• PDA (or cell phone) and Java is the natural
  choice
• Limited computing power and memory
• Needs Java-based graphic software

41
Why Dynamic Data Assimilation?

• Wildfire is a complex process with lots of
  uncertainties, such as wind, humidity,
  temperature
• Neither empirical model nor physical model cannot
  represent the wildfire very well
• Lots of parameters cannot be measured accurately
• All in all, the system is heavily non-linear and
  ill-posed
• Sequential Bayesian data assimilation can be used
  to guide the simulation

42
How Dynamic Data Assimilation works?

• The state of the system at any time - physical
  variables and parameters of interest at mesh
  points
• Time-state vector x - snapshots of system states
  at different points in time
• The knowledge of the time-state of the system -
  probability density function p(x)
• p(x) is represented by a ensemble of time-state
  vectors x1, x2, , xn
• Number of system states maintained size of the
  ensemble number of snapshots
• Thousands of simulation run simultaneously

43
Sequential Bayesian Filtering

• Current state of the model
• prior probability density
• Incorporate data from the field
• Measurements vector y
• How the data is derived from x p(yx)
• Posterior probability density
• The system advances in time from the posterior
  probability density until new data arrives this
  process is called an analysis cycle

44
Standard Approach of Data Assimilation by
Ensemble Filter

• Initialization generate initial ensemble by a
  random perturbation of initial conditions
• Repeat the analysis cycle
• Advance ensemble statesto a target time by
  solvingthe model PDEs in time
• Inject data with time-stampsequal to the target
  time modify ensemble states by a Bayesian
  update

45
Overall Pictures
46
References

• Geoscape Victoria http//www.pgc.nrcan.gc.ca/geosc
  apevictoria/
• Air Traffic Controls Simulator http//www.simlabs.
  arc.nasa.gov/cvsrf/atcs.html
• Eye Surgery Simulation http//www.bitc.gatech.edu/
  bitcprojects/eye_sim/eye_surg_sim.html
• DDDAS by Dr. Craig Douglas http//www.dddas.org/
• DDDAS by NSF http//www.nsf.gov/cise/cns/darema/dd
  _das/index.jsp
• Jan Mandels DDDAS Website http//www-math.cudenve
  r.edu/jmandel/dddas03/