Comparative Analysis of Machine Learning and Classical Statistical Methods for Predictive Modeling in Small Datasets

Source abstract

Introduction: This paper conducted a comparative study that evaluated classical statistical and modern machine-learning regression models of small sample predictive modeling using the Concrete Compressive Strength (n = 100) data. Methodology: The current research used a powerful nested cross-validation model which used conformal prediction to measure both predictive accuracy and uncertainty calibration. It compared five of them, namely Ordinary Least Squares (OLS), Ridge, Lasso, Support Vector Regression (SVR) and Random Forest, based on performance measures such as Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), Coefficient of Determination (R2) and coverage-width measures of 95 percent prediction intervals. Results: The results indicated that Ridge Regression was most balanced (RMSE = 8.54 MPa; R2 = 0.735), and slightly higher compared to OLS and Lasso, and the confidence issues were narrower and more reliable. The constrained sample regime had greater variance of machine-learning models, as is expected of any small-sample sensitive model. The Friedman 2 -test failed to show statistically significant difference in ranks (p = 0.406), yet the regular hierarchy revealed that regularized linear models performed better in the low-data-density case. Conclusion: The scientific article presented a reproducible open-source system that was used to nest validation, conformal uncertainty estimation, and regularization-based modeling, based on which the methodology of predictive analytics in low-data regimes is clearly presented. Increasingly larger multivariate datasets and Bayesian techniques and explainable AI should be used to enhance interpretability and generalizability, yet future researchers are advised to construct this pipeline.