Smart Initialization

1
Smart Initialization

• Mike Romberg

2
Lecture

• Smart Init what, why, how?
• Configurability
• Structure of Smart Init Modules
• Override Procedures
• Examples of Algorithms
• Enhancements
• Laboratory

3
What is it?

• Derives sensible weather elements from model
  data. Creates the IFP databases from the D2D
  data.
• Works on models, not MOS

4
Why is it needed?

• Models do not provide sufficient spatial
  resolution.
• Models do not provide set of needed sensible
  weather elements.

5
Derived Surface T
6
Characteristics of Smart Init

• Field tailorable algorithms
• Can add new weather elements
• Can add local models, if the netCDF files are in
  the correct D2D-style format
• Dependencies automatic
• Grids are generated as D2D data arrives

7
How it works
8
Configurability (serverConfig/localConfig)

• INITMODULES
• Maps algorithms modules to models
• INITMODULES MesoEta MESOETAU,
  MESOETAS, Eta ETA, LAPS
  LAPS
• INITMODULES smart init module D2D
  Source Databases,

9
Configurability (serverConfig/localConfig)

• INITSKIPS
• Can skip certain model runs (INITSKIPS)
• INITSKIPS RUC 1,2,4,5,7,8,10,11,13,14,16,
  17,19,20,22,23
• INITSKIPS D2D ModelName
  hour1,hour2,hour3 to skip

10
Configurability (serverConfig/localConfig)

• Number of D2D Versions
• Not directly associated with smart init
• Does NOT override D2D purging.
• Default is 2 versions.
• D2DDBVERSIONS ETA 3, NGM 1
• D2DDBVERSIONS D2D model name
  numOfVersions,

11
Configurability (serverConfig/localConfig)

• Smart init modules
• Can override and provide own
• Coded in Numerical Python
• One file per new model or changed model

12
Smart Init structure

• File locations etc/BASE, etc/SITE
• Override files through etc/SITE
• Uses Python inheritance

tells ifpServer what to run
import Forecaster
13
Concept of Source/Destination Database (needed
in defining new smart init modules)

• Source database D2D netCDF file
• Destination database IFP

14
Format of Smart Init File

• Class Statement
• Levels (function)
• Calc Functions
• def calcT(arguments)
• Helper Functions (if needed)
• Main Declaration

15
Smart Init Inheritance
16
Format of smart init file
Class Statement and Levels
from Init import class EtaForecaster(Forecaster)
def __init__(self)
Forecaster.__init__(self, "ETA", "Eta") def
levels(self) return "MB1000", "MB950",
MB900", "MB850", "MB800", "MB750",
"MB700","MB650","MB600
17
Format of smart init file
Calc Functions
def calcSnowAmt(self, T, FzLevel, QPF, topo)
m1 less(T, 9) m2
greater_equal(T, 30) snowr T -0.5
22.5 snowr where(m1, 20, snowr)
snowr where(m2, 0, snowr) snowamt
where(less_equal(FzLevel - 1000, topo
3.048), snowr QPF, 0) return snowamt
18
Format of smart init file
Example of a helper function
def linear(self, xmin, xmax, ymin, ymax, we)
m (ymax ymin) / (xmax xmin 0.0000001)
b ymin m xmin return m we b
19
Format of smart init file
Main declaration
def main() EtaForecaster().run() if
__name__ "__main__" main()
20
calc Arguments. WARNING These are different
than smart tool arguments
21
calc Arguments (cont.)
ctime Time from the source database grid currently being calculated, as a time range tuple (startTime, endTime) in seconds since epoch
mtime Time in the destination database grid currently being calculated, as a time range tuple (startTime, endTime), in seconds since epoch
stime Number of seconds from the model basetime currently being calculated. (Seconds since the model analysis time.)
22
calc Arguments

• NOT the Fcst database as in smart tools
• Can use any combination of source/destination
  weather elements in the calcXXX functions.
• At present time, cannot easily have multiple
  input source databases

23
Overriding Algorithms

• Why override?
• I dont like existing algorithms.
• I need additional sensible wx elements.

def calcSnowAmt()
def calcSnowAmt()
24
Steps to Override

• Create new smart initialization module.
• If you modify the ones in etc/BASE they will go
  away when you upgrade!
• Test out new smart initialization modules.
• Put it into routine operations.

25
Format of Override File

• Similar to existing smart init modules.
• Subtle differences, mainly due to inheritance.

26
Format of override file
from Eta import class MyEtaForecaster(EtaForecas
ter) def __init__(self)
EtaForecaster.__init__(self) def
calcSnowAmt(self, T, QPF) m2
less_equal(T, 32) snowamt where(m2,
10.0QPF, 0) return snowamt def main()
MyEtaForecaster().run() if __name__
main__ main()
27
Testing your new module

• Can run from command line
• ifpInit t modeltime algFile
• ifpInit t modeltime a algFile
• Model time in yyyymmdd_hhmm format.
• -a switch forces all grids to be generated
• ifpInit t 20011116_1200 a MyEta

28
Testing your new module

• Run from the command line to see progress and
  problems.
• Check the log files for errors
• /data/logfiles/dateDirectory/EtaInit
• Check the data in the GFE.

29
Okay, it works, now what?

• Put it into operational use by
• Redefining INITMODULES in localConfig
• serverConfig.INITMODULESMyEta ETA
• del serverConfig.INITMODULESEta
• Dont forget to disable the original module!

30
Adding New Algorithms

• Define new algorithm module
• In localConfig, add new weather elements to
  destination database(s) before testing.
• Test
• Modify localConfigs INITMODULES.

31
Adding Local Models

• netCDF files must have certain information in
  order for GFESuite to recognize them
• If displayable on d2d, probably close to ok.
• Need geographic information in netCDF.
• Inherit from Forecaster, look at Eta.py for
  example.
• Similar procedure write module, test, modify
  INITMODULES.
• May need to modify D2DDIRS.
• Dont forget - you can see these models on
  GFESuite without smart init!

32
Useful functions in Init.py
def linear(self, xmin, xmax, ymin, ymax, we) --
linear interpolation def FtoK(self, t) --
convert F to Kelvin def KtoF(self, t) -- convert
Kelvin to Farhenheit def esat(self, temp) --
saturation vapor pressure def self._empty -
returns grid of all zeros def self.levels() -
returns list of levels (MB500, MB450,) def
self._minus - return grid of all 1s def
self.pres - return list of levels as numbers
(500, 450,)
33
Accessing Data Types

• Scalar
• T, simply a numerical grid
• Vector
• tuple, V0 is magnitude, V1 is direction
• Weather
• tuple, W0 is grid, W1 is key
• Key is set of Weather ugly strings
• Grid value indexed into key gives real value
• Smart Tool lecture/lab covered specifics of types.

34
Gotchas

• Conditional statements
• Unlike regular Python, both the true and the
  false are always executed in where statements
• Standard Python
• if x ! 0 y z / xelse y 0.0
• Numerical Python
• where(not_equal(x, 0.0), z/x, 0.0))
  where(not_equal(x, 0.0), z/(x0.00001), 0.00)

35
Future Enhancements

• Ability to access multiple sources
• Useful for model blending
• Syntax identical for numerical smart tools and
  smart initialization
• Fairly close now.

36
Examples of Algorithms - sky
def calcSky(self, gh_c, rh_c, topo)
only use the first levels (up to MB600)
gh_c gh_c9,, rh_c rh_c9,,
rh_c where(less(rh_c, 50), 0, rh_c)
m1 logical_and(greater(rh_c, 50), less(rh_c,
70)) m2 logical_and(greater(rh_c, 70),
less(rh_c, 85)) m3 greater(rh_c, 85)
skylayer where(m1, self.linear(50, 70, 0,
25, rh_c), where(m2,
self.linear(70, 100, 25, 100, rh_c),
where(m3, 100, rh_c)))
37
Examples of Algorithms - sky
skylayer skylayer / 100 sky
skylayer0 for i in xrange(1,
skylayer.shape0) sky sky
skylayeri - sky skylayeri sky
clip(sky, 0, 1) sky sky 100
return sky
Sowhat is wrong with this algorithm?
38
Laboratory Exercises

• Override the Eta algorithm for T
• Simply use 2m FHAG directly
• Enhance to use dry adiabatic rate for topo
  corrections
• Add a new model (UKMET)
• Add UKMET to localConfig
• Calculate QPF, Wind, and T for UKMET