Bayesian Temporal Fusion Transformer Model for Thermal Target Following in Infrared Panoramic Sequences

Source abstract

Thermal target following in infrared panoramic inspection sequences is challenging because small angular targets, intermittent heat distractors, panorama-boundary crossings, and robot rotation produce ambiguous observations. This study proposes a Bayesian Temporal Fusion Transformer (B-TFT) that combines wrapped panoramic descriptors, robot-motion context, gated temporal fusion, and posterior uncertainty within a shared representation for tracking and control. Candidate thermal regions are encoded with angular position, scale, contrast, edge, and background statistics; these observations are fused with velocity, yaw rate, route context, and previous commands. A Bayesian projection head estimates the target-state distribution, while predictive variance reduces command aggressiveness and widens the next-frame search range when evidence is weak. Evaluation used 42.6 h of long-wave infrared data comprising 128,400 annotated panoramic frames and 15 target categories collected in corridor, substation, tunnel, and outdoor night routes. Against DeepSORT-Thermal, ConvLSTM, Kalman-Transformer, and deterministic TFT baselines, B-TFT achieved a 94.7% following success rate and a median center error of 3.8 pixels. Expected calibration error decreased from 0.087 for deterministic TFT to 0.031, and average recovery after target reappearance required 0.76 s. End-to-end onboard inference required 31.4 ms per frame. These results indicate that motion-aware Bayesian temporal fusion can preserve identity under thermal ambiguity and provide a calibrated signal for conservative inspection-robot control.