Interpretable Reinforcement Learning for Fixed Wing Aerial Pursuit with Differentiable Decision Trees and Curriculum Learning

Source abstract

Neural policy networks can achieve effective control in dynamic pursuit tasks, but their decisions are difficult to audit. This study replaces the actor in proximal policy optimization with a full binary differentiable decision tree and uses a two-stage curriculum to stabilize learning. The policy was trained in a JSBSim six degree of freedom fixed wing simulation with a 15-dimensional observation, eight discrete maneuvers, a 0.2 s simulation step, and 100 million interaction steps. The learned tree has five decision levels and 32 leaves, and it is discretized after training into feature-threshold rules and deterministic leaf actions. Across 1,000 evaluation episodes covering ten initial pursuit geometries, the combined tree policy and curriculum achieved a 98.8 percent success rate, compared with 96.2 percent for neural PPO, 90.0 percent for soft actor-critic, 50.7 percent for deep Q-learning, and 0 percent for decision tree policy optimization under the reported configurations. Mean inference time was 0.26 ms per decision for the tree policy and 0.63 ms for neural PPO. In 100 direct matches against neural PPO, the tree policy won 57. The final policy can be inspected as 32 rules, although the present results are limited to one simulator, one discrete action design, and evaluations without reported variability across independent training seeds.