Measuring change in mathematics enjoyment: longitudinal invariance, local dependence of negatively worded items, and complex-sample inference

Source abstract

Introduction Inferences about developmental change require comparable measurement across occasions. We examined one-year change in mathematics enjoyment from Grade 4 to Grade 5 in the TIMSS 2023 Longitudinal Study. Methods Linked files comprised 44,325 students from nine education systems; 43,979 had a positive sampling weight and at least one focal response. Seven positively worded item contents were common to 2023 and 2024, whereas two negatively worded items appeared only in 2023. Within each system, ordinal longitudinal confirmatory factor models incorporated sampling weights, correlated residuals for matching contents, and configural through scalar invariance testing. Separately, design-based uncertainty for latent change was estimated by refitting the scalar model across all 1,604 replicates from 802 paired jackknife zones. Results Weighted WLSMV diagnostics supported scalar invariance in every system (scalar CFI = 0.994–0.997; RMSEA = 0.023–0.053; SRMR = 0.019–0.034; maximum absolute change in CFI across successive steps = 0.00050). A baseline one-factor model combining all nine items fit markedly worse; allowing residual dependence between the two negatively worded items yielded standardized residual correlations of 0.413–0.715. Within-system latent change was negative in all nine systems (−0.553 to −0.117 system-specific Grade 4 SD), with all paired-JRR 95% confidence intervals excluding zero. Estimates retained their sign under every response-completeness and weighting definition. Plausible-value-pooled response models produced the same substantive selection conclusion as a sensitivity model using the mean of the five achievement plausible values. Discussion The content-matched seven-item core supports within-system longitudinal comparison; no cross-country equality of change magnitudes was tested. Local dependence between the negatively worded items may reflect wording direction, shared aversive content, or both.