From Stochastic Parrots to Digital Intelligence: The Evolution of Language Models and Their Cognitive Capabilities

Source abstract

ABSTRACT This review paper traces the evolution of language models from rudimentary statistical approaches to sophisticated neural networks, with a focus on their cognitive capabilities and potential for artificial general intelligence. We examine the “stochastic parrot” hypothesis and evaluate whether contemporary language models demonstrate authentic comprehension and reasoning. In particular, we address the emergence of world representations, planning, and search algorithms in these models, drawing comparisons with human cognition and traditional Artificial Intelligence (AI) paradigms. The paper also explores recent advancements in model architectures, training techniques, and evaluation methods that aim to bridge the gap between pattern recognition and genuine understanding. We conclude by discussing the future directions for research in this field, including the integration of classical AI approaches with neural language models and the potential for developing more robust and interpretable AI systems. This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Deep Learning