X-MAG-COS: Composite Open-Set Scoring for Explanation-Aware Multi-Agent Intrusion Detection in 5G-IoT Networks

Source abstract

5G-enabled Internet of Things (IoT) environments require intrusion detection systems that are accurate, lightweight, interpretable, and robust to attacks not observed during training. Existing IDS pipelines commonly assume centralized feature access and treat explainability as a post-hoc reporting mechanism. This paper proposes X-MAG-COS, an explanation-aware multi-agent IDS in which local security agents transmit compact evidence messages instead of raw traffic records or full feature vectors. The main configuration, X-MAG-COS-24B, uses a 24-byte per-flow message containing top-ranked class evidence, top-ranked explanation evidence, and a local anomaly scalar. A coalition classifier uses the message for known-attack classification, while a separate composite open-set head combines message-prototype residuals, owner-agent uncertainty, and anomaly evidence to detect unknown attack families. We evaluate X-MAG-COS on real 5G-NIDD traffic with leave-one-attack-family-out testing across eight attack families and three random seeds. Across 24 trials, X-MAG-COS-24B achieves known-class macro-F1 of 0.9981±0.0005, unknown-attack AUROC of 0.9439±0.0931, unknown-attack recall of 0.8395±0.2794 at approximately 5% false-alarm rate, and a fixed 24-byte communication budget. Additional experiments compare against centralized and distributed baselines, ablate message content and open-set scores, examine communication-size tradeoffs, evaluate noisy-agent robustness, and test category-level transfer on CICIoT2023. The results show that compact explanation-aware communication is effective on 5G-NIDD and robust to explanation and anomaly perturbations, but they also reveal important limitations: class-evidence corruption harms known-class prediction, and DoS/DDoS-style category transfer remains difficult on CICIoT2023. The study supports a two-head design principle for lightweight IDS: known-attack classification and unknown-attack rejection should be optimized separately under explicit communication constraints.