SCOPE-RL: Reinforcement Learning with Executable Step-Level Verification for Mathematical Reasoning

Source abstract

Large language models (LLMs) achieve strong reasoning performance through Chain-of-Thought (CoT), yet their autoregressive generation provides limited supervision over the correctness of intermediate steps. We introduce Step-wise Check Optimization for Process-Enhanced Reinforcement Learning (SCOPE-RL), a reinforcement learning framework that equips reasoning steps with executable mathematical checks. Each step is associated with a typed verification operator, such as symbolic equivalence, substitution, boundary evaluation, or range checking. Candidate checks are certified using reference solutions and controlled counterexamples to ensure both validity and discriminative ability. During reinforcement learning, these checks are executed directly on intermediate reasoning states to provide a process-level reward, which is combined with the final-answer reward under the standard GRPO framework. Experiments across multiple language models and mathematical reasoning benchmarks show that SCOPE-RL consistently improves accuracy over strong RLVR baselines while producing more verifiable and mathematically consistent reasoning trajectories.