Georgii, Karelin and Nakajima, Kohei and Soto-Astorga, Enrique F. and Carr, Earnest and James, Mark and Froese, Tom (2025) Indeterminism in Large Language Models: An Unintentional Step Toward Open-Ended Intelligence. [Preprint]
![[img]](https://philsci-archive.pitt.edu/style/images/fileicons/text.png) |
Text (Submitted version (without line numbers))
Indeterminism_in_Large_Language_Models__preprint.pdf - Submitted Version Download (199kB) |
Abstract
Synergy between stochastic noise and deterministic chaos is a canonical route to unpredictable behavior in nonlinear systems. This letter analyzes the origins and consequences of indeterminism that has recently appeared in leading Large Language Models (LLMs), drawing connections to open-endedness, precariousness, artificial life, and the problem of meaning. Computational indeterminism arises in LLMs from a combination of the non-associative nature of floating-point arithmetic and the arbitrary order of execution in large-scale parallel software-hardware systems. This low-level numerical noise is then amplified by the chaotic dynamics of deep neural networks, producing unpredictable macroscopic behavior. We propose that irrepeatable dynamics in computational processes lend them a mortal nature.
Irrepeatability might be recognized as a potential basis for genuinely novel behavior and agentive artificial intelligence and could be explicitly incorporated into system designs.
The presence of beneficial intrinsic unpredictability can then be used to evaluate when artificial computational systems exhibit lifelike autonomy.
| Export/Citation: | EndNote | BibTeX | Dublin Core | ASCII/Text Citation (Chicago) | HTML Citation | OpenURL |
| Social Networking: |
Monthly Views for the past 3 years
Monthly Downloads for the past 3 years
Plum Analytics
Actions (login required)
 |
View Item |
![[feed]](http://philsci-archive.pitt.edu/style/images/feed-icon-32x32.png){kind=icon}