Large Language Models for Test-Enhanced Learning: A Tutorial and Proof of Concept

Source abstract

Practice tests improve retention of studied information and enhance learning and retrieval of new information (test-enhanced learning; TEL). In five preregistered studies (total N = 623)—four online randomized experiments and one correlational classroom study with final-year psychology undergraduates—we examined whether large language models (LLMs; Gemini and ChatGPT) can generate practice tests that promote TEL and whether this use of LLMs can be feasibly incorporated into a classroom setting. LLM-generated multiple-choice tests with fixed feedback did not improve 2-day recall of a TED Talk relative to a control (Studies 1 and 2). In contrast, LLM-generated short-answer tests with personalized feedback significantly improved 7-day recall of an educational video relative to rewatching it (Studies 3 and 4), regardless of whether final test format matched that of the practice test. In the classroom study, participation in fortnightly LLM-generated short-answer tests (with personalized feedback) on lecture content was positively associated with graded assessment performance, and students’ attitudes towards this use of LLMs were generally favorable. Overall, we demonstrate that LLMs can substantially reduce the practical burden of implementing TEL, while maintaining sufficiently good question quality and enabling automated personalized feedback, and that this approach can be feasibly deployed in an undergraduate psychology class.