Chain-of-Thought ICL for GeoAI: Breaking the Hardness Barrier

Source abstract

Foundation models for geospatial artificial intelligence (GeoAI) exhibit a striking dichotomy: they excel at pixel-wise tasks such as classification and segmentation, yet struggle with instance-level tasks requiring identification of multiple discrete elements. This limitation, termed the ICL-Hard barrier, arises from fundamental computational constraints of constant-depth transformers, which cannot solve problems reducible to sparse parity for more than J* = O(log log n) ≈ 2–4 objects. The barrier manifests across geospatial domains: geodetic source localization, remote sensing object detection, climate extreme event identification, and multi-modal counting tasks all fail when the number of targets exceeds this threshold. This paper introduces a unified theoretical framework demonstrating that chain-of-thought (CoT) prompting provides a universal mechanism to overcome ICL-Hard barriers across all GeoAI domains. We prove that autoregressive token generation amplifies effective computational depth: generating T intermediate tokens increases effective depth from L to L + γT, enabling transformers to escape AC⁰/TC⁰ circuit limitations and solve previously intractable tasks. Our main contributions include: (1) the CoT Depth Amplification Theorem, proving that T tokens provide effective depth Θ(T); (2) tight bounds on token complexity, establishing T = Ω(J) tokens are necessary and T = O(J log J) sufficient for J-element detection; (3) a smooth success probability scaling law P(success) = (1 − e^{−αT/J})^J; (4) the CoT Threshold Shift Theorem, showing the effective threshold increases to J*_{CoT}(T) ≈ J* + T; and (5) equivalence conditions under which CoT matches fine-tuned model performance without parameter updates. We validate the framework across four GeoAI domains—geodesy, remote sensing, climate science, and multi-modal applications—deriving domain-specific token multipliers (κ ∈ [2.5, 4.5]) and practical deployment guidelines. The theory yields eight testable predictions for empirical validation. This work completes a theoretical arc: where previous work established what is hard for in-context learning in GeoAI, we now show how to overcome these barriers through principled application of chain-of-thought reasoning.