Applying Machine Learning and SHAP Method to Identify Key Influences on Middle-School Students’ Mathematics Literacy Performance

Source abstract

The PISA 2022 literacy assessment highlights a significant decline in math performance among most OECD countries, with the magnitude of this decline being approximately three times that of the previous round. Remarkably, Hong Kong, Macao, Taipei, Singapore, Japan, and Korea ranked in the top six among all participating countries or economies, with Taipei, Singapore, Japan, and Korea also demonstrating improved performance. Given the widespread concern about the factors influencing secondary-school students’ mathematical literacy, this paper adopts machine learning and the SHapley Additive exPlanations (SHAP) method to analyze 34,968 samples and 151 features from six East Asian education systems within the PISA 2022 dataset, aiming to pinpoint the crucial factors that affect middle-school students’ mathematical literacy. First, the XGBoost model has the highest prediction accuracy for math literacy performance. Second, 15 variables were identified as significant predictors of mathematical literacy across the student population, particularly variables such as mathematics self-efficacy (MATHEFF) and expected occupational status (BSMJ). Third, mathematics self-efficacy was determined to be the most influential factor. Fourth, the factors influencing mathematical literacy vary among individual students, including the key influencing factors, the direction (positive or negative) of their impact, and the extent of this influence. Finally, based on our findings, four recommendations are proffered to enhance the mathematical literacy performance of secondary-school students.