Taylor Diagram and Wasserstein Distance for Model Evaluation

Source abstract

The Taylor diagram is used to evaluate and compare predictive models with observed data and has many applications in climate and environmental sciences. Three statistics of interest--the centered root-mean-squared error, standard deviation, and the product-moment correlation coefficient--are related by the law of cosines; Taylor diagrams leverage this fact to allow visualization of all three statistics in a two-dimensional plot without any loss of information. In this work, we present a novel model evaluation tool--the Wasserstein-Taylor diagram--formed from integrating the Wasserstein distance from optimal transport into a Taylor diagram framework. This tool is built upon the fact that the law of cosines still holds in the Taylor diagram if the centered root-mean-squared error and product-moment correlation coefficient are replaced by the centered 2-Wasserstein distance and quantile correlation coefficient, respectively. The advantage of this Wasserstein-Taylor diagram is that one can use the distribution perspective afforded by the Wasserstein distance to compare observed and predicted data sets of different sizes and even to compare statistical models with only observed data. We further show that the quantile correlation coefficient on empirical measures converges to the quantile correlation coefficient on the sampled measures.