Data Assimilation Research Testbed Tutorial - PowerPoint PPT Presentation


PPT – Data Assimilation Research Testbed Tutorial PowerPoint presentation | free to download - id: 672c59-NmNhM


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation

Data Assimilation Research Testbed Tutorial


Title: Data Assimilation Research Testbed Tutorial Author: ncar Last modified by: Tim Hoar Created Date: 2/29/2012 8:57:13 PM Document presentation format – PowerPoint PPT presentation

Number of Views:3
Avg rating:3.0/5.0
Date added: 31 October 2019
Slides: 40
Provided by: ncar


Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Data Assimilation Research Testbed Tutorial

Open Source Ensemble Kalman Filtering the Data
Assimilation Research Testbed - DART
Tim Hoar, Jeffrey Anderson, Nancy Collins, Kevin
Raeder, Hui Liu, Glen Romine NCAR Institute for
Mathematics Applied to Geophysics
10 years in 12 hours
  • Sunday Afternoon (4 hours)
  • Introductions / making teams
  • Configure environment
  • DART WWW-site
  • Download DART
  • DART_LAB Matlab-based exercises to learn DA
  • Monday Morning (4 hours)
  • Recap of yesterday / questions
  • Select chapters of the DART tutorial
  • Moving from toy models to large models
  • Monday Afternoon (4 hours)
  • Diagnostics
  • Testing Strategies 1 observation, please.
  • Real Observations
  • CLM

CAHMDA-V July 2012 pg 2
Sunday Afternoon
  • Introductions I need to know where to start and
    what to cover. As you introduce yourself, please
    let me know how much experience you have with
    each of the following
  • Unix/Linux command line
  • Shell programming (i.e. csh, ksh, sh)
  • vi / emacs / kedit /
  • Matlab / NCL / IDL / NCO / R
  • Ensemble Data Assimilation Theory
  • I cannot give individual attention to this many
    people during these exercises. We will need to
    make teams of 3 that will need to work together
    during this tutorial. We need to make teams that
    have some experience in each of the skills listed
    above. When I tell you to go stand in a corner,
    do not take it personally! I mean no disrespect!

CAHMDA-V July 2012 pg 3
Configuring your nix environment on x50
  • Your HOME directory
  • Customizations dotfiles
  • Compilers, Libraries, inconsistencies
  • Your shell csh, sh, bash, ksh, tcsh
  • Your commands PATH, aliases
  • Your favorite EDITOR
  • Remote logins, X forwarding Windows
  • Batch jobs
  • Being nice lots of people on one machine
  • Being graceful orphaned processes

CAHMDA-V July 2012 pg 4
DART home page
  • The most useful (to me) pull-down menus
  • Getting Started
  • Documentation
  • Diagnostics
  • Miscellany Platform-specific Notes

CAHMDA-V July 2012 pg 5
Download DART
  • Register for the DART code really.
  • Actually download the code
  • we will cheat
  • svn - making modifications with NO FEAR
  • DART file tree / schematic
  • DART documentation
  • DART tutorial
  • Review DART interface requirements
  • DART build mechanism mkmf

CAHMDA-V July 2012 pg 6
What is Data Assimilation?
Observations combined with a Model forecast

  • to produce an analysis.

Overview article of DART Anderson, Jeffrey, T.
Hoar, K. Raeder, H. Liu, N. Collins, R. Torn, A.
Arellano, 2009 The Data Assimilation Research
Testbed A Community Facility. Bull. Amer.
Meteor. Soc., 90, 12831296. doi10.1175/2009BAMS2
CAHMDA-V July 2012 pg 7
Ensemble Filter for Large Geophysical Models
1. Use model to advance ensemble (3 members here)
to time at which next observation becomes
Ensemble state estimate after using previous
observation (analysis)
Ensemble state at time of next observation (prior)
CAHMDA-V July 2012 pg 8
Ensemble Filter for Large Geophysical Models
2. Get prior ensemble sample of observation, y
h(x), by applying forward operator h to each
ensemble member.
Theory observations from instruments with
uncorrelated errors can be done sequentially.
CAHMDA-V July 2012 pg 9
Ensemble Filter for Large Geophysical Models
3. Get observed value and observational error
distribution from observing system.
CAHMDA-V July 2012 pg 10
Ensemble Filter for Large Geophysical Models
4. Find the increments for the prior observation
ensemble (this is a scalar
problem for uncorrelated observation errors).
Note Difference between various ensemble filters
is primarily in observation increment calculation.
CAHMDA-V July 2012 pg 11
Ensemble Filter for Large Geophysical Models
5. Use ensemble samples of y and each state
variable to linearly regress observation
increments onto state variable increments.
Theory impact of observation increments on each
state variable can be handled independently!
CAHMDA-V July 2012 pg 12
Ensemble Filter for Large Geophysical Models
6. When all ensemble members for each state
variable are updated, there is a new analysis.
Integrate to time of next observation
CAHMDA-V July 2012 pg 13
DART_LAB Matlab-based tutorial
  • DART/DART_LAB/presentations
  • Section 1 the 1D perspective
  • Section 2 impacting an unobserved state
  • Section 3 sampling error and localization
  • Section 4 perturbed observations (EnKF)
  • DART/DART_LAB/matlab
  • Section 1 gaussian_product, oned_ensemble,
  • Section 2 twod_ensemble, run_lorenz_63,
  • Section 3 run_lorenz_96
  • Section 4 oned_ensemble, twod_ensemble,
    oned_model, run_lorenz_63 and run_lorenz_96 all
    allow selection of EnKF.

Im going to focus on
CAHMDA-V July 2012 pg 14
Ensemble Filter for Large Geophysical Models
A generic ensemble filter system like DART just
needs 1. A way to make model forecasts 2. A
way to compute forward operators, h.
CAHMDA-V July 2012 pg 15
Thats all for today .
CAHMDA-V July 2012 pg 16
Monday morning
  • Introductions and lies which of these is true
    about me?
  • I have seen all 7 continents.
  • I am a competitive square dancer.

CAHMDA-V July 2012 pg 17
Monday morning
  • Questions from yesterday
  • netCDF ncdump, ncview
  • Matlab customizations for DART
  • DART tutorial (in the interest of time, were
    skipping a lot at your leisure go back and be
  • Sections 1,2,3 in their entirety (but quickly)
  • Section 4 skip to pg 29
  • Section 5 only pg 15
  • Section 7 introduces lorenz_96 (L96) pg 10
  • Section 8 sampling error (L96) pgs 10,13
  • Section 9 inflation (L96) after pg 15
  • Section 11 building DART
  • Section 14 observation quality control
  • Section 18 not knowing the truth
  • Scripting for standalone executables large

CAHMDA-V July 2012 pg 18
Monday Afternoon
  • Diagnostics
  • State-space (useful if you know the truth)
  • Observation-space (useful in general)
  • link_obs example with dev/POP
  • obs_diag example wth dev/WRF
  • Testing Strategies
  • Lorenz_96 perfect model experiment
  • Observation sequence file creation
  • Ameriflux data
  • CESM multi-instance facility
  • Modifying the run script
  • CLM variable specification picking a state
  • Adding new DART kinds/types
  • Observation operators

CAHMDA-V July 2012 pg 19
Reasons to NOT reinvent the wheel
  • DART has proven methods to address the most
    common (and some not-so-common) issues affecting
    the performance of ensemble filters
  • Inflation
  • Anderson, J. L., 2009
  • Spatially and temporally varying adaptive
    covariance inflation for ensemble filters.
  • Tellus A, 61, 72-83 doi10.1111/j.1600-0870.2008.
  • Anderson, J. L., 2007.
  • An adaptive covariance inflation error correction
    algorithm for ensemble filters.
  • Tellus A, 59, 210-224.  doi10.1111/j.1600-0870.20
  • Novel algorithms
  • Anderson, J.L., 2010
  • A Non-Gaussian Ensemble Filter Update for Data
  • Monthly Weather Review, 138, 4186-4198,
  • Anderson, J. L., 2007
  • Exploring the need for localization in ensemble
    data assimilation using a hierarchical ensemble
  • Physica D, 230, 99-111.  doi10.1016/j.physd.2006.
  • Parallelization

CAHMDA-V July 2012 pg 20
More reasons to NOT reinvent the wheel
  • Diagnostics
  • Many routines/methods are provided to explore the
    performance of the assimilation.
  • Native ability to explore value of different
    types of observations
  • Immediate ability to perform perfect model
  • Documentation
  • Each code module has a companion HTML document to
    describe its use and purpose.
  • http//
  • All documentation/code available online
  • Workshop materials
  • Self-paced tutorials included in the download
  • Portable, tested on many platforms
  • Free, open source.
  • Too many platforms/compilers to bother listing.
  • Distributed and maintained with subversion.
  • Can exploit, but does not need MPI.

CAHMDA-V July 2012 pg 21
Creating the initial ensemble of CLM.
Replicate what we have N times. Use a unique (and
different!) realistic DATM for each. Run them
forward for a long time.
model time
spun up
Getting a proper initial ensemble is an area of
active research.
a long time
We dont know how much spread we NEED to capture
the uncertainty in the system.
CAHMDA-V July 2012 pg 22
The ensemble advantage.
You can represent uncertainty.
In a free run, the ensemble spread frequently
With a good assimilation ensemble spread
ultimately remains stable and small enough to be
observation times
CAHMDA-V July 2012 pg 23
Atmospheric Reanalysis
Assimilation uses 80 members of 2o FV CAM forced
by a single ocean (Hadley NCEP-OI2) and
produces a very competitive reanalysis.
O(1 million) atmospheric obs are assimilated
every day.
1998-2010 4x daily is free and available. Contact
500 hPa GPH Feb 17 2003 Contours 5200m5700m by
CAHMDA-V July 2012 pg 24
Code to implement all of the algorithms discussed
is freely available from
CAHMDA-V July 2012 pg 25
(No Transcript)
DART is used at
43 UCAR member universities More than 100 other
  • Public domain software for ensemble Data
  • Well-tested, portable, scalable, extensible,
  • Models
  • Toy to HUGE
  • Observations
  • Real, synthetic, novel
  • An extensive Tutorial
  • With examples, exercises, explanations
  • People The DAReS Team

Moving towards coupled assimilationfor earth
system models.
Tim Hoar, Nancy Collins, Kevin Raeder, Jeffrey
Anderson, NCAR Institute for Math Applied to
Geophysics Data Assimilation Research Section
Steve Yeager, Mariana Vertenstein, Gokhan
Danabasoglu, Alicia Karspeck, and Joe
Tribbia NCAR/NESL/CGD/Oceanography
Hypothesis Need Ensemble of Atmospheres to Force
Ensemble Assimilation for Ocean
500 hPa GPH Feb 17 2003
  • Case 1 23 POP members forced by a single
  • Case 2 48 POP members forced by 48 CAM/DART
  • Case 2 Generates additional ocean spread,
    improved analyses.

DATM 2D forcing from CAM assimilation
Current POP Assimilation from within the Climate
Earth System Model - CESM
World Ocean Database T,S observation counts
These counts are for 1998 1999 and are
FLOAT_SALINITY   68200              
FLOAT_TEMPERATURE 395032            
DRIFTER_TEMPERATURE 33963              
MOORING_SALINITY   27476             
MOORING_TEMPERATURE  623967                 BOT
TLE_SALINITY   79855            
BOTTLE_TEMPERATURE   81488                    
CTD_SALINITY  328812                  
CTD_TEMPERATURE  368715                    
STD_SALINITY     674                 
STD_TEMPERATURE     677                  
XCTD_SALINITY    3328                
XCTD_TEMPERATURE    5790                
MBT_TEMPERATURE   58206                 
XBT_TEMPERATURE 1093330                 APB_TEM
PERATURE  580111
  • temperature observation error standard deviation
    0.5 K.
  • salinity observation error standard deviation
    0.5 msu.

Ensemble Spread for Pacific 100m XBT
Spread of the climatological ensemble
Twice as much!
100m Mooring Temperature RMSE Pacific
POP/CAM as good or better RMSE
Physical Space 1998/1999 SST Anomaly from
POP forced by observed atmosphere (hindcast)
Coupled Free Run
48 POP 48 CAM
Fully coupled assimilation will need data from
all models at the same time
This is a very CESM-centric view of fully coupled
data assimilation.
DART works with many geophysical models
Global Atmosphere models CAM Community
Atmosphere Model NCAR CAM/CHEM CAM with
Chemistry NCAR WACCM Whole Atmosphere
Community NCAR Climate Model AM2 Atmosphere
Model 2 NOAA/GFDL NOGAPS Navy Operational
Global US Navy Atmospheric Prediction
System ECHAM European Centre Hamburg
Model Hamburg Planet WRF Global version of
WRF JPL MPAS Model for Prediction
Across NCAR/DOE Scales (under development)
DART works with many geophysical models
Regional Atmosphere models WRF/ARW Weather
Research and NCAR Forecast Model WRF/CHEM W
RF with Chemistry NCAR NCOMMAS Collaborative
Model for NOAA/NSSL Multiscale Atmospheric
Simulation COAMPS Coupled Ocean/Atmosphere US
Navy Mesoscale Prediction System CMAQ Communit
y Multi-scale Air Quality EPA COSMO Consortium
for Small-Scale DWD Modeling
DART works with many geophysical models
Ocean models POP Parallel Ocean
Program DOE/NCAR MIT OGCM Ocean General
Circulation MIT Model ROMS Regional Ocean
Modeling Rutgers System (under
development) MPAS Model for Prediction
Across DOE/LANL Scales (under
development) Land Surface models
CLM Community Land Model NCAR (under
DART works with many geophysical models
Upper Atmosphere/Space Weather models
ROSE NCAR TieGCM Thermosphere
Ionosphere NCAR/HAO Electrodynamic
GCM GITM Global Ionosphere Thermosphere
Model Michigan