Auditing Prospective Memory in Frozen Language-Model Systems: Causal Mechanisms and the Limits of Naturalistic Trigger Grounding

Source abstract

Prospective memory in an interactive system is not merely long-term retrieval: a stored intention must become behaviorally relevant when a later event satisfies its condition, while irrelevant near-matches should leave behavior unchanged. We study this problem in frozen language-model systems using the Prospective Conditional Memory Intervention Pilot (PCMIP), an auditable controller that separates persistent condition-payload storage, semantic trigger detection, selective intervention, causal payload controls, and an exact null path. In a prespecified controlled evaluation of 7,200 executions, PCMIP achieved 1.0 triggered-action accuracy, 1.0 correct non-action, zero false triggers, and satisfied all 11 prespecified adequacy criteria. The unchanged trigger did not transfer robustly to a held-out 128-case naturalistic challenge: balanced accuracy was 0.703 and the false-trigger rate was 0.297. Development-only redesigns improved pooled naturalistic performance or restored performance on the frozen synthetic preservation set, but none satisfied the unchanged source-group, semantic-group, action, and preservation criteria. A final factorized analysis achieved perfect action resolution conditional on a valid trigger, localizing the remaining errors to semantic trigger grounding. The results establish controlled system-level feasibility while rejecting robust zero-shot naturalistic transfer under the tested variation, and show why pooled accuracy and preservation alone are insufficient evidence of reliable prospective triggering.