Adaptive Spectral Regularization for Stable Inference Under Multicollinearity

Source abstract

Multicollinearity can make linear regression severely ill‐conditioned, leading to unstable coefficient estimates and unreliable inference. Classical methods such as ridge regression and principal component regression improve numerical stability through global shrinkage or component reduction, but they do not explicitly link the amount of regularization to the conditioning of each eigendirection. This study examines an adaptive spectral regularized estimator (ASRE), formulated as a power‐law eigenvalue parameterization within generalized ridge regression. The penalty assigned to each spectral direction increases as its eigenvalue decreases, allowing weak directions to be stabilized more strongly, whereas dominant components are retained. We derive finite‐sample bias, variance, and risk expressions, establish a variance bound, provide an exact condition for lower directional MSE than classical ridge regression, and show fixed‐ p mean square consistency under standard design regularity conditions. Monte Carlo experiments include contamination, heavy‐tailed and heteroskedastic errors, and a near‐square setting with n = 120 and p = 100. A diabetes progression application evaluates both predictive error and bootstrap‐based inferential stability. ASRE yields the smallest coefficient Beta‐MSE in the near‐square spectral stress test and, in the real‐data analysis, produces the lowest bootstrap coefficient variability and the narrowest confidence intervals while remaining close to the best predictive method. These results support ASRE as a targeted generalized ridge parameterization for applications in which stable estimation and interpretable inference are central objectives.