Large language models and emergence: a complex systems perspective

Source abstract

Abstract Emergence is a concept in complexity science that describes how many-body systems manifest novel higher-level properties, properties that can be described by replacing high-dimensional mechanisms with lower-dimensional effective variables and theories. This is captured by the idea ‘More is Different’. Intelligence is a consummate emergent property manifesting increasingly efficient—cheaper and faster—uses of emergent capabilities to solve problems. This is captured by the idea ‘Less is More’. In this paper, we first examine claims that Large Language Models exhibit emergent capabilities, reviewing several approaches to quantifying emergence, and secondly ask whether LLMs possess emergent intelligence. This article is part of the theme issue ‘World models in natural and artificial intelligence’.