Methods using the ultraviolet absorption of ozone in the Huggins bands have been extremely successfu

1
How Well Do OMIs UV-Based Retrievals Sample the
Lowermost Troposphere?
Introduction Methods using the ultraviolet
absorption of ozone in the Huggins bands have
been extremely successful in producing estimates
of the total ozone column, providing accuracies
to 12 in situations where they can be compared
to the best ground-based solar occultation
methods. The scattering, reflection, and
absorption of UV radiation have been the basis of
the TOMS, GOME, SCIAMACHY, and OMI instruments.
Until recently, only very coarse resolution of
the vertical structure of ozone distribution in
the vertical has been attempted. Recently,
methods using multi-wavelength optimal estimation
have shown promise with both GOME and now OMI (X.
Liu, et al., JGR, 2005 X. Liu et al., Aura Sci.
Team Mtg.) Alternatively, a lower-atmospheric
ozone partial column has been estimated using
other measurements. A very promising development
has been the continued development of OMI minus
MLS (Ziemke et al., 2006) method by using
sophisticated trajectory modeling to spread the
limited MLS observational opportunities forward
in time and space (Schoeberl et al., 2008). On
the other hand, air pollution forecasters and
ozone regulatory bodies have insisted that they
would like very detailed description of ozone
concentrations. They would prefer useful
measurements describing the first ca. 0.81.5 km
(daytime mixed layer depth) for remote sensing
to be relevant. A more achievable goal is to map
ozone within the lowest 23 km (including an
upper ozone-transport layer, one that is
frequently mixed, albeit over a few days, by
fair-weather clouds. This is a demanding
requirement for UV techniques, since these
partial column are the similar to, sometimes
slightly larger, than the current ozone column
precision. Furthermore, the basic TOMS-style
retrieval technique which has been improved
over decades is based on fundamentals of
radiative transfer that suggest sensitivity of UV
methods to smog ozone is much lower than other
ozone. An often-shown graphical characterization
of the retrieval sensitivity (in the vertical) of
the OMI/TOMS technique, as estimated by the
forward-calculation theory (P.K. Bhartia, Aura
Science Mtg., 2007 ) is shown to the right. This
is valid for clear-sky conditions. Sensitivities
are ca. 0.2 for 01 km, ca. 0.27 for 12 km,
and ca. 0.35 for 23 km. This is particularly
discouraging when one seeks of lt 10 Dobson units
(DU) of integrated ozone partial column in the
lower atmosphere within a background of 350 DU
total column (mostly in the stratosphere).
OMI-MLS Tropospheric Ozone Residual techniques
have settled for the easier task of tropospheric
ozone, 35 DU, in hopes that such estimates will
be of some use in describing ozones pollution
and climate impacts. Despite these theoretical
estimates, investigators have repeatedly found
variations in ozone column (TCO) or tropospheric
ozone residuals (TOR) that correlate surprisingly
well with surface ozone (Fishman, Vukovich, , J.
Appl. Meteor., 1987, Fishman et al., AGU Spring
2008) or lower-tropospheric ozone soundings
(e.g., Chatfield et al., GRL, 2004).
Robert B. Chatfield, NASA Ames Research Center
Robert.B.Chatfield_at_nasa.gov MS 245-5 Moffett
Field, CA 94305 USA P.K. Bhartia, NASA GSFC Mark
Schoeberl,NASA GSFC Xiong Liu, NASA GSFC, UMBC,
Smithsonian CfA
Samuel Oltmans, NOAA GMD Robert Esswein, NASA
Ames and BAER Inst David Tarasick, AQRB,
Environment Canada Anne Thompson, Pennsylvania
State Univ
The Goal 0-1.5 km Smog O3 A difficult goal is
to map air pollution in the lowermost troposphere
with detail exceeding this model simulation,
performed using a 36-km grid. Better detail,
showing details of 510 km within cities is
useful to improve local understanding, but this
scale is useful in understanding and predicting
regional impacts 1502500 km downwind. Courtesy
Georg Grell, NOAA.
Sensitivity of the Observed Portion of the
Atmosphere Above Cloud
Sensitivity of the Common Total-Ozone Product
Including Climatology Below Cloud
Sensitivity of the Common Total-Ozone Product
Including Climatology Below Cloud
compare to s(pl)
Sensitivity of X. Liu Many-? Optimal Estimation
of Total Column O3 Xiong Liu (with Kelly Chance,
Mike Newchurch, and P.K. Bhartia) have estimated
total ozone column and specific
height-distributed sub-columns (discussed below)
using the spectral distribution of absorption in
the Huggins bands. A limited number of sondes
satisfied our criteria of coincidence one degree
latitude and one degree longitude away and on the
same day. (Very likely the hour was appropriate,
as Liu has concentrated on retrievals at
instances when there were sondes. We can likely
extend the number of coincidences.)
s(pl)
Sensitivity of Version-3 OMI/TOMS to Ozone at
Differing Levels of the Atmosphere The basic
question of sensitivity to the lower troposphere
in present distributed products begins here. The
version-3 ozone product is reported with an
effective cloud pressure and an amount of ozone
that can be filled below cloud to provide a
map-able old-standardproduct TCO. These
quantities take into account partial cloudiness
and sensed UV radiation from within clouds with
simple linear approximations. We analyzed
observed-only ozone to cloud tops, and used
only appropriate level contributions computed for
IONS and SHADOZ sondes, under restrictive
conditions of burst at or above 10 hPa. (Above
left) Sensitivity estimates, depicted on the
horizontal axis, for the observed-only ozone
have a decrease towards the surface similar to
the the forward radiative-transfer calculation,
but the statistics find enough information to
describe a curve upward towards the surface!
150-hPa ozone is sampled with excellent
sensitivity (probably limited by the sonde
accuracy), but sensitivity above 50 hPa
decreases (we speculate that pumps and
pump-pressure corrections vary significantly).
Above-burst ozone, estimated using the McPeters
et al. (JGR, 2005) climatology, does not show
useful correlated variation, but see concluding
comments below. With care, we may add more
sondes and categorize groups of sondes to
highlight the most important variables .
(Above right) Surprisingly, the inclusion of
climatological (a-priori) ozone near the surface
increases sensitivity in this sample. Ozone
invisibility may correlate with meteorological
variability and high or low ozone. Note,
however, the large spread between the two dashed
lines indicating two standard error estimates.
Ozone column is typically estimated with R2 gt
0.94, and a standard deviation 2 of the mean.
This study is preliminary, and we would prefer a
bit more feedback and experience before quoting
more on accuracies. Direct solar occultation
measurements are no doubt a more direct measure
of accuracy for OMI retrievals in the special
circumstances of clear skies. Note for these
cases that clear skies seem to be correlated with
ozone in the lowermost troposphere. Nevertheless,
the relative success of the fitting at this stage
points to a useful role for sondes.
(Interesting tidbit) When we omit the
estimation of a constant (like 130.8 or 141.6 DU)
above, the sensitivity of the near-burst region
increases! This forces higher sensitivities in
some regions in this case the linear regresion
reaches for the relatively low-variance layer
above 50 hPa. Similar considerations may or may
not explain the dips in sensitivity in the
mid-troposphere an analysis of variances should
be carried out.
Methods Previous Work, Basic Concepts, and
Difficulties At the 2007 Aura Science Meeting,
we reported a bit-too-simple technique that tried
to compare the SHADOZ and IONS ozonesonde
datasets with the then-current version of the
Schoeberl OMIMLS technique (figures to the
right). The basic idea carries through to the
technique presented below. For Schoeberl TOR,
one forms a regression with TORi as the dependent
variable (l.h.s) and information summed up from
the sonde reports (partial ozone columns for
atmospheric layers defined by pressure difference
(?pl) on the right-hand side multiplied by an
averaged mixing ratio for that layer ?il (in fact
one characterizes the layers with the appropriate
integration over sounding points.) One seeks a
set of sensitivity parameters al that portray
empirically the way that the layer DU small
partial-column estimates co-vary with the total
TOR estimate. Since soundings quite variable in
shape, there is information with which to inform
the empirical estimate of the sensitivity terms
al. Note The estimates are independent of any
radiative transfer theory, retrieval theory, or
practical expedients employed in dealing with
clouds, aerosols, or subtle, remaining instrument
effects. Symbolically, Where i indexes
sonde/OMI pairs, l indexes pressure layers ?pl
and associated layer-mean pressures pi , cil
indicates mixing ratios from sounding
measurements, ti indicates an above-burst
estimated ozone partial column, C is a constant
for the whole regression, and ?i are the errors
associated with the match-up. The sensitivity
coefficients are the al , each estimated
separately. Several problems were identified
with this estimate first it needlessly conflated
effects of the TCO estimation algorithm with the
effects of the stratosphere-elimination (TOR)
algorithm. It was clearly desirable to work both
with and without the stratospheric-elimination.
Secondly, it became clear that estimation for
adjacent layers al was correlated, possibly
leading to over-fitting and very scattered
layer-sensitivity estimates. Thirdly, it seemed
clear that there must be more information for
certain levels than others, but it was not clear
how to estimate the information content and
therefore the appropriate layer choices.
Fourthly, there were clear indications of
different effects associated with different
stations, possibly due to the operation and
instrumental character of the ozonesondes, or
conceivably due to meteorological and physical
effects on the OMI retrieval somehow related to
the ozonesonde launch site. Certain stations like
Sable Island often seemed to stand out. Lastly,
the error terms were most directly associated
with the TOR estimates (lhs), not errors in the
ozonesonde measurements (consistently high or low
sensitivity to ozone throughout the sounding, or
associated with individual layers). Cross-Validat
ed, Statistically Guided, Smoothing Technique.
Recent developments in statistical theory and
available software have solved many of these
problems, and so we have begun to address the
difficulties. The general linear model
software gam contained in the mgcv package
devised and written by Simon Wood (Chapman and
Hall Press, 2007, R documentation, 2008) allows
us to estimate a variable coefficients model
that defines constrains the parameters al to be
evaluated values of a smooth spline function
s(pl) , with the smoothness and the evaluation of
error contributions to be synthesized in one
statistical estimation. The sensitivity is
estimated as a function of layer the fit is over
many soundings. The formula that was used to
seek a best rationalization of the sondes
vertical information to the estimate by OMI of a
total column was Here ?i stands for the total
column ozone (TCO), or, as we shall see, other
partial columns describing tropospheric ozone.
The term s(pl) is constrained in its curvature
(second derivative), not its deviation from any
starting or preferred value (sensitivity 0,
sensitivity 1, sensitivity estimated from an
asymptotically appropriate forward model, or
whatever). If correlation does not establish a
sufficient regression effect, the statistical
model prefers a constant term Ci. This means
that sensitivities normally lie between 1
(excellent correlation with insignificant noise)
and 0 (no information is provided by the
variability found in the sounding). The method
is essentially linear regression, and so there
should (see below) be one solution, directly
achievable, albeit by an iterative method. The
restriction on curviness, not primarily on
discrepancy from an a priori, is also an
important difference from Rodgers (2000) and
similar (Maddy and Barnets linearization points,
TGRS, 2007) retrieval methods. The constant term
has some similar role to the a priori. While
optimal estimation analyzes sensitivity
considering the a priori, this method analyses
sensitivity as improvement over a constant mean
estimate. Low sensitivities may still allow for
useful estimation Each view has its benefits, and
the views may be synthesized with further
research. Bayesian methods could be
implemented. To repeat, the new Wood technique
for general linear modeling implements a
comprehensive strategy in which the estimation of
parameters and the degree of smoothing in the
spline can be estimated from the data. Second
derivatives of the spline functions are penalized
in a uniform way within an existing statistical
tradition the penalty be decreased if one allows
a greater chance that random error affects the
parameters Cross-validation compares many
subsets of the model data to estimate the
uncertainties and error-bounds, so that
over-fitting can be avoided incremental
increases in curviness can be seen to have minor
influence on explained variance, and rejected.
The mgcv package employs the popular generalized
cross-validation technique this avoids a minor
pitfall of a naïve use of many subsets.
s(pl)
The mean estimate of sensitivity is high near
the surface, 0.85, but the standard error bands
are broad. In this analysis we used both
tropical and midlatitude sondes, but did not take
into account the presence of clouds. As we are
able to collocate more samples, we expect to
distinguish more effects. As mentioned with the
TOMS technique, the low sensitivity near burst
need not affect error estimates ot the total
strongly these regions may be relatively more
constant in mixing ratio, so that a constant
serves well. More extensive study is required to
make the lower-tropospheric estimates firm.
al
900-700 hPa
700-500 hPa
500-350 hPa
200-0 hPa
1015-900 hPa
350-200 hPa
1015-0 hPa
X. Liu Many-? Optimal Estimation of the
Distribution Tropospheric O3 The X. Liu total
ozone analysis is broken into layers on the
right. (Leftmost) Total ozone column (TOC),
followed by (Proceeding to the right) sensitivity
estimates for a sequence of layers. These are
layers that are significant for the study of smog
and tropospheric ozone, not originating from an
analysis of degrees of freedom for signal. We
composed these layers using the resolving
information in the Liu dataset for each sounding
and our own estimate, from the sondes, for the
same layers. Clearly, analysis for PBL ozone
should not be derived from a 1015-900 hPa layer
estimate. More information for the boundary
layer comes in the estimates for layers up to 500
hPa, or better, thicker layers reaching to
approximately 500 hPa. The 500-350 hPa layer
estimate seems to provide a good empirical
portrayal of the vertical resolution of the
method. The 350-200 hPa region shows the same
complex, problematic behavior as the OMI-MLS
method using OMI/TOMS.
s(pl)
s(pl)
Mid-latitude OMI-MLS
Tropical OMI-MLS
Conclusions We have made an empirical
statistical analysis of the response of two
complete retrieval procedures of OMI
(ultra-violet) ozone column by exploiting
co-variation of ozone at different layers in the
atmosphere and the OMI retrieved products.
Similarly, we have analyzed one derived product
OMI-MLS tropospheric ozone TOR (surface to 200
hPa version) by its correlation with ozonesonde
variations to 200 hPa. Despite the suspected
10 variability of ozonesondes determination of
the ozone column (to balloon burst), the large
number of sondes can provided useful information.
There may be surprisingly good detection of
ozone down to the lowest kilometers, better than
we have expected from some analyses of the
forward model for radiation transfer. There is
some additional support suggesting the precision
OMI TCO to be 12. The use of the generalized
linear model in a form that supports
variable-coefficient models has allowed us to
use regression to estimate response of OMI-based
retrievals, with a set of coefficients smoothly
varying with altitude that multiply narrow
sonde-derived layer ozone partial column amounts.
The statistical technique itself usefully
provides information about the degree of fitting
that appears to be appropriate. The sensitivity
measure differs from Bayesian methods and other
retrievals based on forward radiation models, but
it does not disagree and appears to provide a
useful, separate, empirical view of
sensitivity. There are reasons to be cautious
other analyses have strongly suggested that there
are variations in the typical match-up of sondes
to OMI from station to station, with some
stations particularly at variance. These are
probably more easily attributed to ozonesonde
procedures and sonde properties than they are to
special observing situations affecting the
station location. More detailed and varied
analysis is needed for the effects of clouds.
Additionally, there are reasons to suspect that
variations in sensitivity of the sondes down as
far as even 50 hPa, perhaps affected by pump
degradations or their corrections. However,
variance at these levels is small. For these and
other regions (e.g., the middle troposphere), it
may be that the true variation is sufficiently
low that, in the presence of various errors, the
general linear model regression has little
ability to prefer high sensitivity at those
levels to a constant value. Sensitivity s(pl) at
these levels is nevertheless poorly defined in a
relative sense. These results make more precise
ideas about sensitivity based on situation
analysis (Chatfield et al., 2006) and from
estimates based on unconstrained simple linear
regression (Chatfield et al., Aura Science Mtg.,
2007). UV methods do seem to sense variations
in tropospheric ozone closer to the surface than
we might have expected, although perhaps still
not so well. We do not yet understand when such
sensitivities seem to be high or seem to be low.
UV methods do not appear yet to give much PBL (0
to 1-2.5 km) information for typical clear-sky
significant smog episodes. No better than 6 km
resolution, at the best, appears to be currently
available from the optimal estimation technique.
There is much more to be understood about sondes,
clouds, and retrievals as they actually affect
estimates.
s(pl)
Thanks to Nathaniel Livesey and the whole MLS
team for the MLS products, to the many field
scientists launching ozone sondes, to Jaquelyn
Witte and the WCRP for collating the
information, to Vance Fong and Region-9 US EPA
for stimulating greater interest in making remote
sensing relevant to health and pollution
problems, to Johan de Haan for insight and
useful suggestions, to Simon Wood for
communicating his newest refinement of mgcv,
and to many scientists referenced and
unreferenced for discussions. Supported
primarily by the NASA ACMAP/Aura program.
Schoeberl OMIMLS Tropospheric Ozone Residual
Sensitivities A new form of estimate of
tropospheric ozone residual (TOR) was released by
Mark Schoberl in September, 2008. It was based
on OMI-TOMS Version 3 and with several
improvements including better mapping of MLS
stratospheric ozone. The estimate provides a
partial ozone column for that part of the
troposphere above the equivalent cloud-top
pressure, or equivalently, an appropriate
layer-averaged ozone mixing ratio. Maps of the
quantity must be used with care for the variation
of effective cloud top. We have analyzed the
surface200 hPa version of these recently, and
display them for comment. The large number of
sondes used reflects the reduced need for
screening. (Mid-latitude sondes, above left
tropical sondes, right) Note the very small
intercept estimates, 2 to 2 DU, and high
sensitivities, 0.51.0. The zero-sensitivity
line is off the graph to the left in each case.
Twenty-five tropospheric levels were used to
estimate s(pl). These seem to indicate
relatively good accuracy for the product. We
would like to examine the accuracy of the new
TOR, distinguishing mixing-ratio variation from
effective-cloud-top variation, as resources may
permit. The large variation of estimated
sensitivity for the mid-latitude sondes (left
panel) caused us to seek fits with less curvature
than was justified by generalized
cross-validation. The low-sensitivity estimates
in the upper troposphere for both mid-latitude
and tropical analyses seem to occur near the
climatological levels of upper-troposphere cloud
tops for the respective cases.