Optimizing Large‐Scale Mathematical Assessments: Leveraging Hierarchical Attribute Structures and Diagnostic Classification Models for Enhanced Student Diagnostics

Source abstract

Abstract Diagnostic classification models (DCMs) assess students’ mastery of cognitive attributes to provide personalized ability profiles. Retrofitting DCMs to large‐scale mathematics assessments usually relies on inferred Q‐matrices, which can reduce accuracy and diagnostic value. This study evaluated whether constructing items from cognitive models—yielding Q‐matrices directly—and incorporating hierarchical relationships among attributes improve diagnostic outcomes. Responses from 5,336 third‐grade students to a Luxembourgish image‐based, large‐scale standardized mathematics exam were analyzed using multiple DCMs and their hierarchical extensions. Items were constructed based on a Q‐matrix, derived from the curriculum and cognitive models. The hierarchical A‐CDM outperformed other models, classifying students into 60 latent classes with acceptable attribute‐ and test‐level accuracy and more interpretable results than the G‐DINA model. Using cognitive model‐based item generation and Q‐matrices as well as specifying attribute hierarchies enhance the accuracy and interpretability of DCM‐based diagnostics in large‐scale assessments, complementing traditional psychometric approaches by discerning meaningful within‐score differences.