ASSESSING THE COSTS AND BENEFITS OF INCREASING ECOSYSTEM MODEL COMPLEXITY USING DATA ASSIMILATION

1
ASSESSING THE COSTS AND BENEFITS OF INCREASING
ECOSYSTEM MODEL COMPLEXITY USING DATA ASSIMILATION
Marjorie Friedrichs, Raleigh Hood, Jerry Wiggert
and Ed Laws
Research Questions
How should we determine how much ecosystem model
complexity is required to best explain bulk
biogeochemical properties and fluxes? How much
ecosystem model complexity is required to best
explain observations from the JGOFS Arabian Sea
study?
2
Outline

• Methods
• Mechanistic vs. empirical ecosystem models
• Data assimilation
• Results
• Comparison 1 Optimize all parameters
• Comparison 2 Cost function ratio
• Comparison 3 Predictive cost function ratio
• Conclusions, implications and future work

3
Methods Ecosystem model descriptions
bN
agr(P(1-r)Z) rP(1-r)Zf
aZ

• Eco_4 McCreary et al. (2001) PZND 10
  parameters
• Eco_5 Hood et al. (2001) PZNDDON 16
  parameters
• Eco_8 Christian et al. (2002) 2P,2Z,2D,N,A 19
  parameters
• Emp_4 PZND 20 parameters

4
Problem How do we determine whether
differences in model results
are due to differences in model structure, or
differences relating to parameter
choices? How do we compare these models in
an objective way? How much ecosystem model
complexity is supported by the available data?
Approach
Variational Adjoint Method of Data Assimilation

5
Methods Variational Adjoint Method
parameter optimization/weighted least-squares

• cost function measure of model-data misfit
• cost function ? Wp (Pdat-Pmod)2 Wz
  (Ndat-Nmod)2
• ? Minimize cost function by adjusting model
  parameters
• ? Assess model performance based on the magnitude
  of
• the cost function

6
JGOFS Arabian Sea Process Study 1995
Available data Cruise observations nitrate
(6) chlorophyll (5) production (5) zooplankton
(4) Sediment trap 1 year time series
X WHOI mooring Sediment Trap S7
station
7
Comparison 1 Optimize all parameters
When all parameters are optimized, Eco_4 and
Emp_4 fit the data equally well? Important to
objectively optimize models prior to comparison
8
Note Parameters selected on the basis of cost
function sensitivity
The mechanistic model fits the data better than
an empirical model with the same number of d.o.f.
9
Comparison 2 Cost Ratio (Mechanistic/Empirical)
Eco_4 0.32 .02 Eco_5 0.28 .03 Eco_8
0.28 .02

• If only 2 to 4 parameters are optimized, the
  mechanistic models all fit the data 70 better
  than the empirical model

10

• If we estimate too many parameters we
  are merely fitting noise in the data ? If we
  estimate more than 3 parameters, Eco_4 yields
  predictive costs lower than the optimal Emp_4
  predictive cost

11
Comparison 3 Predictive Cost Function Ratio
Eco_4 0.71 .05 Eco_5 0.68 .05 Eco_8
0.66 .05

• If only 3 or 4 parameters are optimized, the
  mechanistic models all perform 30 better than
  the empirical model

12
Summary Three comparisons
Cost Function Optimize all parameters
Cost Ratio Optimize key parameters
Predictive Cost Ratio Optimize key parameters
There is no improvement in model-data fit with
the addition of more ecosystem model complexity
13
Conclusions

• Data assimilation is required to ensure were
  not simply comparing the degree of tuning
• - the variational adjoint method
• A 4-component empirical model can reproduce
  data as well as a 4-component mechanistic model
• Two comparison methods
• 1. Compare to an empirical model with the
  same
• number of free parameters
• the cost function ratio (mechanisticempirical)
• 2. Perform cross-validation experiments
• - the predictive cost function
• In the Arabian Sea, the more complex ecosystem
  models did not perform better than the simplest
  model

14
Future Work

• Analysis in Equatorial Pacific
• Simultaneous assimilation of Equatorial Pacific
• and Arabian Sea data
• Larger range of model types 4 to 24 state
  variables
• Participating modeling groups include
• Anderson/McGillicuddy Dusenberry/Doney/Moore
  Fujii/Chai
• Hood/Laws Schartau/Armstrong Spitz
  Wiggert/Christian