Hierarchical Majority Decomposition for Imbalanced Classification: A Recursive Binary Majority-Class Tree with Persistent Minority Comparison

Source abstract

Class imbalance remains a persistent challenge in supervised classification, particularly when the minority class represents rare but important observations. Common approaches address this problem through undersampling, synthetic oversampling, cost-sensitive learning, or decomposition of complex class distributions. This study introduces Hierarchical Majority Decomposition (HMD), an architecture in which the majority class is recursively partitioned into a binary tree of increasingly localized subclasses while the complete minority class remains intact. A supervised local classifier is associated with each internal majority node and distinguishes the minority class from the node’s two child majority subclasses. During top-down inference, a minority prediction terminates traversal, whereas a majority-subclass prediction simultaneously identifies the current observation as not yet recognized as minority and determines the branch in which the next minority comparison is performed. HMD was evaluated using six binary benchmark datasets with imbalance ratios ranging from approximately 3.2:1 to 42.0:1. An unrestricted decision tree was used as a controlled base learner, and all methods were evaluated using 5-fold stratified cross-validation repeated 10 times. HMD was compared with an untreated decision tree, a class-weighted decision tree, random undersampling, SMOTE, and a root-only flat majority decomposition. Increasing HMD inference depth increased minority recall relative to the untreated decision tree on all six datasets. The strongest effects were observed on Mammography, Climate Model Simulation Crashes, Oil Spill, and Ozone Level Detection. Importantly, the root-only decomposition generally remained close to the untreated decision tree, whereas deeper HMD inference produced substantially larger sensitivity increases, indicating that recursive minority comparison rather than majority decomposition alone is responsible for much of the observed effect. Random undersampling frequently achieved equal or greater minority recall and balanced accuracy, but often at a substantially greater loss of specificity and minority precision. The results demonstrate that a recursively decomposed majority class can be used as an active inference structure in which an unchanged minority class is repeatedly reconsidered against progressively localized majority contexts.