Metrics & Diagnostics
Current Practice
The CESM Ocean Model Working Group (OMWG) has diagnostics packages to assess both spatial and temporal characteristics of ocean model simulations. We perform such assessments routinely for output from both forced ocean and fully-coupled experiments. The use of these packages includes evaluating the validity and climate state of a particular coupled model simulation, assessment of whether new capabilities in any component model, when coupled to the entire coupled model system (i.e., a new model code base), have impacted the ocean, and validating long coupled control simulations. The OMWG also employs many different metrics and diagnostics when implementing new capabilities and parameterizations into the ocean component for their particular assessments. These diagnostics are usually very specific for the particular new model additions and are tested within forced ocean-only or forced ocean – sea-ice simulations. The following table is a summary of the most commonly used metrics and diagnostics that are considered. However, we note that many other metrics and diagnostics for additional fields as well as expert judgement are also used in our assessment of a given model simulation.
Current Metrics and Diagnostics for Coupled Model Assessment
| Metric/Diagnostic Name | Assessment Method | Assessment Criteria | Assessment Type |
|---|---|---|---|
| Sea surface temperature (SST) | Spatial distributions; mean and RMS of global differences from obs | mean near zero; RMS near zero; both comparable to or better than previous versions; small west coast biases; spatial structure; etc. | Climatological mean over 20+ years |
| Sea surface salinity (SSS) | Spatial distributions; mean and RMS of global differences from obs | mean near zero; RMS near zero; both comparable to or better than previous versions; small local biases; spatial structure; etc. | Climatological mean over 20+ years |
| Global and Atlantic meridional overturning circulation (MOC) | Spatial distributions; time series | No collapse; “reasonable” structure; if possible, comparison with the RAPID array | Climatological mean over 20+ years; time series |
| Mixed layer depth | Annual and winter mean spatial distributions compared to obs | Small RMS; spatial structure | Climatological mean over 20+ years |
| Antarctic Circumpolar Current (ACC) transport | Transport at Drake Passage | magnitude close to obs | Time series |
| Nino analysis | Time series; spectral and wavelet analysis | Amplitude and period close to obs | Time series and time series analysis |
| Equatorial undercurrent and thermocline | SST mean across the equatorial Pacific and Atlantic; SST seasonal cycle; temperature along the equatorial, upper-ocean Pacific, longitude slices of temperature with depth; thermocline depth | Structure (including tilt), seasonal cycle, distributions, etc. close to obs | Climatological mean over 20+ years |
| Global and horizontal average temperature and salinity time series | Drift | Consistent with surface fluxes; similar to obs in 20th century simulations | Time series |
| Heat and salt budgets | Ocean heat and salt contents; surface fluxes | Consistency of fluxes across components; conservation of heat and salt | Time series and time averages over given time segments |
| Meridional heat transport | Transport distribution; maximum values | Compare to present-day obs; compare to RAPID array | Climatological mean over 20+ years |
Desired Redesign & Improvements
Our current diagnostics packages and assessment metrics date back to more than 10 years ago. These tools grew out of a few individual people’s own research analysis and plotting codes. As such, they are not really designed to handle continuously growing needs of our expanding modeling community. Incorporation of new diagnostics tools, new analysis methods, new metrics, any changes in the CESM data output structure, etc. are not as straight-forward as we would like. Therefore, we would like to redesign our metrics and diagnostics packages, starting at the upper-most level and considering our anticipated needs. We note that some of the desired diagnostics also require code additions and modifications in the CESM ocean component. However, the path forward will be influenced by our limited resources as well as possible development of a similar package for the MPAS-Ocean model that is being considered as a future CESM component. The following bullet items list only some of the many additional, new assessment diagnostics that need to be developed to meet our future needs.
Desired Metrics and Analysis
- Water mass analysis to assess i) atmosphere-ocean partitioning of carbon and heat on decadal to centennial timescales, ii) global and basin scale distributions of volumetric temperature and salinity census. Among others, we will use water mass transformation function.
- Near surface habitat and its dynamic coupling to the ocean interior.
- Sea surface height analysis, taking advantage of available observations.
- CFC analysis to assess i) ventilation (e.g., Southern Ocean), ii) CFC distributions as a proxy for other relevant tracers.
- Western boundary current structure and associated bias analysis: Gulf Stream and Kuroshio paths, considering SSH, velocity, pattern correlations.
- Ability to plot on isopycnal surfaces.
- Diagnostics to assess high-resolution simulations such as eddy kinetic energy, eddy length scales, etc.
- Add new metrics, incorporating new or updated observational data sets, including hydrography.
- Expand diagnostics to include seasonal cycle and longer term variability for relevant variables.
- Consider creating a summary table for a set of primary metrics as well as a list of potentially large biases that the user should be aware of.