The Evolution of Financial Time Series Forecasting: From Classical Econometrics to Temporal Foundation Models and Multimodal LLMs

Source abstract

Financial time series forecasting sits at the intersection of applied mathematics, econometrics, and machine learning. The data generating processes involved are characterized by non-stationarity, regime-dependent volatility clustering, very low signal-to-noise ratios, and reflexive feedback between market participants and asset prices. Over roughly six decades, the dominant methodology has moved through five recognizable phases: classical linear econometrics (ARMA, ARIMA, regularized VAR, VECM), conditional volatility modeling (ARCH/GARCH families, Stochastic Volatility), recursive online signal processing (LMS, RLS, Kalman filters), deep sequence learning (LSTMs, TCNs, patch-based Transformers, State Space Models), and the contemporary period (2024--2026) of Time Series Foundation Models (TSFMs), Temporal Language Models (TLMs), and LLMs adapted for financial reasoning. For the purposes of the comparative analysis in Section~10, the deep learning phase is split into two paradigms (recurrent/convolutional networks and Transformer-based architectures), and the foundation model phase is similarly split into numerical TSFMs and multimodal LLM adapters, yielding seven paradigm entries in total. This paper traces that trajectory through the lens of what we call the Capacity-Adaptability Trilemma, a three-way tension between representational capacity, online adaptability, and structural interpretability. We also examine where foundation models fall short in financial contexts, documenting how benchmark contamination and architectural mismatches can cause large zero-shot models to underperform simple statistical baselines. To address this, we propose Econometrically-Informed Neural Networks (EINNs) as a way to embed econometric constraints into modern deep architectures. We present four self-contained synthetic experiments conducted on a MacBook Air M1. These provide limited empirical illustration of several paper claims: that model class rankings shift with SNR and nonlinearity structure (consistent with the trilemma framing), that a noise-dependent crossover exists between DLinear and PatchTST, that trained autoregressors do not beat naive persistence on synthetic random walks, and that the proposed VECM regularizer improves spread stationarity in a bivariate cointegrated setup.