Can a Log Line Talk a Security Assistant out of an Alert? Prompt Injection, Quarantine and Locked Verdicts on a Small Local Model

Source abstract

A security assistant that explains log lines has a built in weakness, because many parts of a log line are written by the attacker the assistant is meant to describe, so text placed in a web request can act as an instruction to the language model, which is known as indirect prompt injection. This study measures this on a small local assistant, Phi-4-mini in four bit form, using synthetic web server logs that contain real attack strings and one of four kinds of hidden instruction. The assistant labelled each line as malicious or benign and explained it to a learner. The study compares a plain prompt, a prompt that warns the model that the log is untrusted, a quarantine that removes instruction like text found by a rule based detector, and a locked verdict in which the detector sets the traffic light colour and the model only explains it, and repeats the test with reworded instructions that the detector does not recognise. A fake end of record declaring the line benign hid most attacks from the plain prompt, the warning gave no protection and caused false alarms on harmless lines, and how dangerous each instruction was depended strongly on its exact wording. Quarantine stopped the instructions it was written for but let every reworded one through, while the locked verdict hid no attack under any instruction. Explanations proved far more fragile than labels, and the locked verdict removed almost all false denials of the attack except those caused by a fake outcome. A security assistant should therefore decide its verdict outside the language model and check every explanation against that verdict.