SARTA Validation and TuningSat, Jan 1, 2022
1. Purpose, Context and Scope
To summarize the performance of the stand alone fast radiative transfer algorithm (SARTA) by comparing with kCARTA calculations and with various correlated field observations and satellite measurements.
Included: reference bias and standard deviations relative to the fitting profiles, and bias and std. deviations between satellite observations and atmospheric model fields from selected field campaigns, sondes and ERA/ECWMF reanalysis data.
In addition, a summary of the derivation and performance of selected tuning coefficients is discussed.
Statistical comparisons are in general performed using the AIRS L1C version of SARTA, with additional results using the CrIS and/or IASI versions. The CHIRP version is most like the CrIS in medium resolution format and is not directly referenced in this report. In each case the same spectroscopy is used.
2. SARTA Version and Configuration
This work involves the most recent complete build of SARTA at UMBC/ASL, around 2019 with version 2.02.
Validation uses clear sky radiances, however in practice there are limitations on how valid the clear sky assumption might be. Some discussion is included below.
The spectroscopy uses the HITRAN 2016 line and cross-section database. The WV continuum MT-CKD, line mixing <> and non-LTE <>.
2.2. Atmospheric model.
The TOA radiances computed by SARTA and kCARTA use ECMWF or ERA atmospheric fields for water vapour, temperature and ozone, and where available the sonde water and temperature. Where the soundings do not go up to the uppermost level, the AFGL standard model is spliced on top. The surface temperature is from the ECMWF/ERA model or when available from the field campaign data, this is swapped out. The surface emissivity and land fractions are used in the same way as the standard RTP processing at ASL. The cloud fields are also included and 2-slab SARTA computations are routinely computed along side the clear calculations but are not used in the determination of biases.
During this work the atmospheric profiles for CO2, CH4, N2O and CO from ESRL and related data sources were added to account for spatial and temporal variability including seasonality and long term trends. But note that these were not uniformaly applied to all analyses.
2.3. General Comments
Interpretation of bias between observations and calculations as a measure of model accuracy must be made in the context of both the data set being used and the spectral region of interest. For example, in the window regions the surface properties dominate the uncertainty, and so for these spectral regions well characterized ocean views are preferred. In more opaque channels the uncertainty is dominated by the error in the quantity of absorbing species. In all cases (except the most opaque channels with signal from high in the atmosphere) scenes clear of cloud are preferred, however there is the likelihood that some unknown residual contamination remains. Some insight can be gained from using modelled cloud fields.
The magnitude of unaccountable residual uncertainties are likely dominated by skin temperature and emissivity and cloud and is estimated to be of the order 0.1 to 0.2 K in the LW window. The SW window channels uncertainty is about the same for night time view but worse for day-time.
Validation of model accuracy therefore is the combination of results from the varied strategy of direct kCARTA comparisons, large sample comparisons using ECMWF/ERA fields and the radiosonde matched views.
Before reporting detailed results for each of the three approaches, a brief mention is made of the source data and methods.
3. Data Sources
3.1. The NWP data
The ECMWF or ERA atmospheric fields are used in the same way as standard RTP processing, and include profiles of ozone, H2O, temperature, cloud variables, skin temperature, and the surface emissivity. In much of this work the AFGL standard atmospheric profiles for minor gases are used with CO2 adjusted to nominal current surface values. The capbailbity to include contemporary fields of CO2, CH4, N2O and CO has been added, and when used will be clarified in this report.
3.2. Summary of field campaigns.
There are several sources of atmospheric measurement T(z), WV(z) that have been used to compare calculated TOA radiances with spacecraft observations and include: GRUAN and ARM based sondes, Field (research) campaigns and the assimmilated weather service (NWP) analyses ERA/ECMWF. Each have their own characteristics and issues when used for the RTA validation.
3.2.1. Summary of GRUAN.
The GRUAN provides calibrated and quality screened atmospheric soundings from various sites with various types of sondes. The sounding data are provided with or without the best available current algorithmic corrections depending on the data stream chosen. There are two main data streams: the GDP and EDT. GDP refers to the GRUAN data processing with added quality control, the EDT refers to processing to the standard GRUAN data format but only has the manufacturer's supplied corrections. Whereas the GDP stream is preferred, for some situations the EDT stream provides better coverage.
The sites used in this work include: Darwin, Aus (DAR), Graciosa Azores (GRA), Lauder, NZ (LAU), Linberg, D (LIN), Reunion, Fr (REU), Sodanklya, Fin (SOD), Singapore (SNG), Tateno, Japan (TAT), Tenerife, Spain (TEN), Paramaribo (PMO) Where possible a full annual cycle of observations are used, though this is not possible at all sites. See ref: (https://www.gruan.org).
Being rigorously processed the GRUAN soundings are mostly very reliable nevertheless care is taken to identify and eliminate unphysical soundings or those that burst too low.
3.2.2. Summary of ARM.
The ARM include atmospheric soundings and field campaigns from a variety of sources including fixed and mobile sites, and include different sonde types. Some data sets include radiosondes with corrections applied, all include basic QC, but much more care is needed to check for unphysical soundings than the GRUAN.
The mobile (ship) facilities studied include: MAGIC and MARCUS. The fixed locations include: North Slope Alaska (NSA), southern Great Plains (SGP), Tropical West Pacific (three different locations) (TWP), Eastern North Atlantic (ENA), and Gan Airport Maldives (GAN). See ref: (http://www.arm.gov/). See also ref.: (https://doi.org/10.1175/AMSMONOGRAPHS-D-15-0051.1) for a useful summary description of the mobile facilities.
3.2.3. Summary of field campaigns.
- MAGIC (Marine ARM GPCI Investigation of Clouds); Mobile Facility.
Los Angeles, CA to Honolulu, HI. Ocean.
- MARCUS (Shipboard Southern Ocean Campaign); Mobile Facility
Hobart, AUS to Antarctic Coast. Ocean
DYNAMO was a field campaign for the dynamics of the madden Julian Oscillation from 2011-10 to 2012-03, with sondes from various locations including two research vessels, RV Marai and RC Roger-Revelle. See ref: (https://data.eol.ucar.edu/project/DYNAMO).
The initial deployment of the AMF1 took place at Point Reyes, California, between March and September 2005 in support of the Marine Stratus, Radiation, and Drizzle (MASRAD). This dataset was not found to add value to this study.
3.2.4. A note about different radiosondes
Most of the sonde based profiles of T(z) and WV(z) use Viasala RS92, RS41 and RS-11G, but others are available (but not used in the report) and include iMS100, SRS34, M10, and DFM9. The GRUAN PMO site also includes the RS92-SGS Ozone sonde data.
< add some more notes & refs on corrections. ? >
The NWP global fields use the standard RTP processing at ASL and are subset to clear ocean views and restricted to tropical and subtropical latitudes unless otherwise stated. The AIRS FoVs are thus already colocated to the model fields.
Before proceeding with the sonde calculations the sonde data files are screened for bad soundings. In general, the sondes have useful WV information from the surface up to near the tropopause but become inaccurate in this region depending upon condtions. The sonde temperature measurements are generally valid up to the burst height.
The GRUAN sondes that are processed completely and designated -GDP- quality include estimates of the measurement uncertainty. The criteria for using these sonde data are that the sonde burst above (at a lower pressure than) 60 mb, that the humidity and temperature reading are greater than the reported uncertainty and that the relative humidity is greater than 3%, which is an estimate of the lowest valid reading. In practice, the temperature reading is valid up to the burst level, and the humidity reading is valid up to between 60 mb and 100 mb. This is a slightly higher altitude than has been generally used for calibrated ARM sondes in previous SARTA validation studies.
The temperature and WV profiles are extended to the topmost AIRS level by attaching the values from the standard atmospheric profiles. Comparisons of the sonde T(z) and WV(z) to the ECMWF/ERA fields are a useful diagnostic to help interpret the bias obtained between the satellite observations and SARTA calculations.
The Sonde located data sets are identifed for each sonde launch by its location and time, and any AIRS L1C data granules that include the sonde are found. For each granule all fields of view within 1-deg lat/lon of the sonde are selected. When the sonde is at the extreme edge of the granule and the number of nearby FoVs are inadequate the granule is ignored.
Since tropical locations are often not well sampled by AIRS, the nearest preceeding and subsequent sondes are used and interpolated in time to the overpass of AIRS. Otherwise in general the time delay between overpass and sonde launch is kept to within 4 hours unless otherwise stated. The magnitude of the delay was varied in early studies until an optimum was found as a compromise between insufficient samples and bias drift.
For all nearby FoVs two sets of TOA computations are made with SARTA. First with the NWP model fields and then with the sonde fields. Then for each set, the subset of cloud free footprints is determined. The same clear-subsetting algorithm is used as the standard RTP processing.
Histograms of the LW window BT from the original near FoVs and the near and cloud clear subsets are checked.
3.4. Source code
Source code and scripts are at:
/home/chepplew/projects/sarta/validation/ in a git repo
<TBD> except for dependent scripts that are referenced by matlab addpath directives.
< perhaps include example processing pipeline >
4. Comparison with kCARTA
4.1. kCARTA configuration.
< Need Sergio to remind me >
4.2. Summary plots.
The direct comparison of the TOA BT computed for the 49 test and fitting profiles provides a closure test on the fast coefficients and sarta algorithm and is a measure of the limit of accuracy of SARTA. A sample of such a test is shown in figure KA.n. This result is self explanatory.
5. NWP Comparisons
5.1. Contemporary minor gas fields
Before proceeding with direct bias comparisons, we investigate the impact of using contemporary fields of CO2, CH4, N2O and CO for the model atmosphere compared to the AFGL standard atmospheres used by klayers and adjusted to account for nominal growth since the time of those standard profiles.
The CO2 CH4 and N2O fields are from the global monitoring lab. (GML) of the Earth System Research Lab (ESRL) see (http://gml.noaa.gov/), and the CO column is from the MOPITT project, with allowance for observation dates outside the measurement or analysis time frame.
An example of the difference using the standard RTP approach and the enhanced minor gas modelling is shown in figure NWP.n. The data sample is based on 10 days of clear subset RTP for IASI.2 on MetOp-B. Notice that this uses the IASI version of SARTA, where the short-wave CH4 optical depth fast coefficients are not included. < note to add them! >.
5.2. Bias from NWP statistical samples.
6. Sonde Comparison Case Studies
In this section the results of specific sonde sites are discussed.
6.0.1. Tenerife (TEN Spain)
Tenerife is a GRUAN certified site with RS92 sondes periodically available in the years 2008, 2009, 2010, 2016 and 2017, and RS41 from 2018 to 2022 (current date).
Tenerife is situated in the Eastern North Atlantic off the NW coast of Africa which means that it tends to have relatively low incidence of cloud cover, tends to be rather low humidity, has many nearby ocean scenes but can be subject to airborne (mineral) dust and some unresolved cirrus clouds. This attributes makes it more useful for studying the atmospheric window bias and hence for tuning the water continuum absorption model, however, the BT defecit due to WV absorption in the window is similar to the uncertainty due to skin temperature and residual clouds. Example bias plots are shown in figures F.TEN.
Notice that both of the sondes tend to be slightly drier in total and drier in the upper troposphere compared to ECMWF. This is reflected in the Obs:calc bias in the MW band. The mean bias in the LW window is of the order 0.2+/-0.05 K. It is hard to justofy tuning WV continuum model based on this difference. Note that only nominal CO2, O3 and CH4 are included in the RTA.
6.0.2. Tateno (TAT Japan)
Tateno is a GRUAN certified site with with RS92 from 2011 to 2014 and a very few thereafter, and RS11 from 2013 to Jan 2018, and RS41 from 2020 to 2022 (current date). Some are fully processed, others partially processed with calibration.
Tateno is situated north of Tokyo about 20 km from the ocean, so there are some clear ocean views within the specified proximity but the sample size is rather small. There is a strong seasonal cycle with summer months being relatively humid.
Figures F.TAT show the results for subset clear ocean views for two radiosondes RS92 and RS-11G for the years 2013 and 2017. Notice the slight differences in the temperature and water vaopur profiles and how these are reflected in the spectral bias plots. The clear ocean sample size is very small, and of these there appears to be some cloud contamination in the LW window. The suggestion is that both the sondes and the ECMWF have too much water vapour in the UT compared to the observations.
6.0.3. Singapore (SNG)
The Singapore site is GRUAN certified, about 3 miles from ocean with other islands. Available sondes include RS41 from 2017 to current period. Most sondes data are calibrated but don't yet include the GDP uncertainties. The profiles are generally very humid.
Two years of sonde data are matched to AIRS, in 2019 there are 637 valid sonde launches and after subsetting to clear ocean and night views 425 matched FoVs remain. In 2020 the number of launches is 591 with 245 clear match FoVs. A summary of results is plotted in figures F.SNG.
6.0.4. Paramaribo (PMO)
Paramaribo is a GRUAN site currently being certified, near the capital city of Suriname about 8 miles from the Ocean. Soundings used in this study were provided by the Royal Dutch Met Institute (KNMI). There are soundings from 1999 to 2020 using RS92-SGP about once a week. KNMI apply some T and WV corrections, have QC applied but don't have individual reading uncertainty,
Plots to follow
6.0.5. Reunion (REU)
Reunion is a GRUAN site in the Indian Ocean East of Madagacar, very close to the ocean. Plots to follow …
6.0.6. ARM TWP
Tropical Western Pacific ARM soundings …
6.0.7. ARM SGP
Southern Great Plains ARM soundings …
6.0.8. ARM LIN
Lindenberg ARM soundings …
6.0.9. Clear Matched soundings by Irion
There are pre-matched AIRS observations for sondes in the TWP, NSA, SGP and Magic
ARM campaigns. The data files are available in
sub-directories, with the T(z) and WV(z) already incorporated.
The TWP collection includes 199 matched sonde launches from late 2002 to early 2003. The profiles are processed through klayers and sarta to obtain computed spectra for comparison to the observations. A sample of the bias after subsetting for clear ocean views is shown in figures IS.n. Note that the atmospheric model only includes T(z), WV(z) and skin temperature, with the remaining gases using the default AFGL standard atmosphere from klayers. The TWP clear ocean bias subset sample size is only about 30 footprint matchups. The results are not very useful. There is too much uncertainty in the skin temperature and residual cloudiness.
The Magic Campaign includes 295 matched sonde launches from about September 2012 to October 2013. There are originally about 74k matchups reducing to about 24k clear ocean subset. The mean bias and std.dev of the bias is shown in figure IS.n. The results are not very useful, with too much uncertainty in the skin temperature and residual cloudiness.
The Southerm Great Plains (SGP) collection includes 99 matched sonde launches from August 2002 to February 2003. The original set of 31k samples is subset to 8k clear ones. The mean bias and std.dev of the bias is shown in figure IS.n. These are similarly not very useful.
6.0.10. Hannon Original Collection
From work in 2009, collections in:
for Phase1_corr3 there are 220 sondes, 148 semi-clear AIRS FoV matchups w/ AIRIBRAD L1b data (2378 channels). Sonde T(z) WV(z) have corrections applied. Proximity: 1-deg lat/lon, max delay 1.2 hours.
Code: calc_hannon_airs_2009_bias.m child proc: initialize_sonde_airs_bias.m sarta_exec: /airs_l1c_2834_may19_prod_v2
As example shown in figures HO.n the original obs:calc bias from the Hannon file and the new sarta calcs. There appears to be no added value in using those data.
7. Tuning Parameters
7.1. The tuning method
The computation of tuning parameters uses the method developed by Scott and uses the fractional change in radiance computed from the bias of the observed to calculated radiances. The fractional change is then applied to the SARTA optical depths at that point during the SARTA algorithm.
Scripts are found in
7.2. Methods, Results and Discussion of tuning
The CO2 bands in the region 650 to 790 cm-1 and 2300 to 2400 cm-1 are of interest for tuning as they are (at least the LW Q-branch) often used temperature sounding channels. The bias signal is mostly dominated by the CO2 and little other species and so is easy to correlate to CO2 abundance. The complication comes in the form of the accuracy of the actual CO2 abundance up to very high altitudes at the time and place of the samples used. Studies (not shown here but conducted in this task) indicate that the maginitude of the bias is very sensitive to the choice of CO2 profile, and the existing method of adjusting the standard profile to the current global average surface abundance is inadequate. This is the reason for using the ESRL GML CO2 fields that are monthly average profiles on 30 levels at 2.5-deg lat/lon grids. An example of the difference is shown in figure <TBD 2021d267_airs_rtp_clr_ocn_sub_night_upd_co2_ch4_n2o_vs_default>
The change of bias due to using updated CO2 fields in the long-wave Q-branch part of the spectrum is shown in figure <TBD>, with a change of 0.2 K max in the band.
The application of a set of tuning parameters improves the bias around the 720 cm-1 region as shown in figure <TBD> [fig:ngrams] with in this case a worse bias near the 667 cm-1 region. This is expected to be improved with a further iteration of the tuning parameters.
7.2.2. Water Lines
The MW band water lines contain information from water vapor absorption across the troposphere into the lower stratosphere so knowledge of the WV concentration profile throughout this altitude range is required.
Tuning parameters have been evaluated using the bias data from GRUAN PMO 2018 bias calculations, subset to clear ocean views. An example is shown in figure WLT.n notice the improvement in the ECMWF based bias. Notice also the very small bias in the 1050 cm ozone band - which is attrbuted to using the ozone profile from these RS92-SGP sondes, and the 1305 cm-1 methane bias due to using nominal CH4 abundance. However, notice the opposite behaviour of the correction for lines around 1000 cm-1 compared to thos around 1400 cm-1.
7.2.3. Tuning water continuum using the LW window bias
Checking the bias for tuning of the water continuum uses the window channels, and uses nominally clear ocean scenes, where influence of other absorbers are negligible away from proximal absorption lines. The spectral range for correction is nominally from 790 cm-1 to 1020 cm-1. It is debatable to extend this to include the MW band - where weak lines and the transmission in the Ozone band are influenced by the continuum near the surface, and further to the SW window region, 2400 to 2520 cm-1, although this is not included in this study.
The main contributors to uncertainty of the bias from observations, neglecting observational bias include: water vapor continuum absorption error, representation of the surface condition, emissivity and skin temperature, and residual clouds. It has been found that it is necessary to use tropical sites where there is most water vapour in order to have a sufficiently large continuum effect.
The determination of clear scenes is the same as that used in the RTP processing algorithm using criteria based on the uniformity of the scene across groups of neighboring FOVs and on the magnitude of the observation and clear-sky calculation brightness temperature in select window channels.
Using the NWP (ECMWF/ERA) based data sets provides ample sampling, in this case 10 days provides about 386,000 clear ocean observations. The bias can be computed directly from this set. The addition of the cloud parameters from the NWP allow scattering calculations to be made and these can be used to further analyse and subset the sample. By doing this the robustness of the bias can be examined. Although care must be taken not to assume that there are not competing effects acting to reduce the LW window bias. For example, residual unaccounted cloud would reduce the observation relative to the clear calculation.
It is expected that the NWP would place clouds inaccurately, further exacerbated by displacement of the cloud relative to the observation time and location. Nevertheless, over relatively short periods there are regions over the sub-tropical oceans that can remain cloud free. Other regions such as the ITCZ remain mostly cloudy and it is around the cloudy regions that cloud errors are most likely. This is seen the the figure AX. Several studies have been undertaken to find the sensitivity to the bias in the window by selecting different subsets of clear scenes from the large NWP based sets and the bias has been found to vary relatively little. This suggests that residual clouds in clear subsets in general contribute a similar but smaller proportion of the total bias.
Considering the surface condition, the NWP model includes the re-analysis skin temperature, the surface emissivity and roughness <need ref. or summary factoid>. The surface condition for open ocean are better characterised in the models than land conditions, which is useful for bias in the atmospheric window channels. Land based observations can be used for more opaque channels, and in general for the MW water band. It has been suggested to use the night- time short-wave window channels to adjust ocean skin temperatue, but it has been found does not reduce the uncertainty.
Determination of window bias using sonde sites generally does not improve on the findings from the NWP based sets. Tropical sites are better for larger continuum absorption, but most data are over land, of those that have ocean views there is often too much cloud contamination, the number of clear sonde views is negligible to be useless, and near-sonde clear ocean views have been found and results are provided.
Examples of the performance of this method as used in this work are shown in figures WC.n.
8. Other Notes and Conclusions.
9. Other References
10.1. Abbreviations and footnotes.
10.2. Data and Code at ASL