On rank graduation metrics for high–dimensional ordinal data

Source abstract

Evaluating the reliability of machine learning classifications remains a fundamental challenge in Artificial Intelligence (AI), particularly when the target variable is multidimensional. Classification variables can be expressed by means of a categorical scale which, at best, is ordinal. Because ordinal data lack a natural metric structure in their underlying space, most conventional distance measures aimed at assessing the accuracy of machine learning classifications cannot be directly or meaningfully applied. In this paper, we develop a mathematical framework for comparing ordinal data based on a family of Rank Graduation [Formula: see text] metrics. We demonstrate that these metrics can quantify the proportion of variability of the response explained by the predictions, in a similar manner as the predictive [Formula: see text] for continuous response variables. After establishing theoretical connections between the [Formula: see text] family and other prominent metrics in AI, we conduct extensive experiments across diverse datasets and learning tasks to evaluate their empirical performance. The results underscore the versatility, interpretability, and robustness of the [Formula: see text] metrics as a principled foundation for developing trustworthy and SAFE AI systems.