White paper 9 - Benchmarking homogenisation algorithm performance against test cases

The benchmarking homogenisation algorithm performance against test cases white paper is now available for discussion. When making posts please remember to follow the house rules. Please also take time to read the full pdf before commenting and where possible refer to one or more of section titles, pages and line numbers to make it easy to cross-reference your comment with the document.

The recommendations are reproduced below:
• Global pseudo-data with real world characteristics

• GCM or Reanalyses data should be used as source base with real spatial, temporal and climatological characteristics applied to recreate to a reasonable approximation the observational record statistics

• Review of inhomogeneity across the globe finalised via a session at an international conference (link with White Paper 8) to ensure plausibility of discontinuity test cases

• Suite of ~10 discontinuity test cases that are physically based on real world inhomogeneities and orthogonally designed to maximise the number of objective science questions that can be answered

• Benchmarking to rank homogenisation algorithm skill in terms of performance using climatology, variance and trends calculated from homogenous pseudo-data and inhomogenous data (discontinuity test cases applied) (skill assessment to be synchronised with broader efforts discussed in White Paper 10)

• Independent (of any single group of dataset creators) pseudo-data creation, test case creation and benchmarking

• Peer-reviewed publication of benchmarking methodology and pseudo-data with discontinuity test cases but ‘solutions’ (original homogenous pseudo-data) to be withheld