Pietsch, Wolfgang (2026) Nothing New Under the Sun – Large Language Models and Scientific Method. [Preprint]
![[img]](https://philsci-archive.pitt.edu/style/images/fileicons/text.png) |
Text
pietsch-llms_and_scientific_method.pdf Download (847kB) |
Abstract
The fundamental principle of large language models (LLMs) is next-word prediction based on a modeling architecture that is—in comparison to the achievements of LLMs—strikingly simple consisting of just a vast number of vector and matrix operations. For predictions, large language models rely on induction inferring generalized relationships from an enormous number of particulars without any hypothetical assumptions concerning the modeled subject matter itself, i.e. language. Regarding the specific type of induction, I argue that LLMs employ a difference-making logic (or variational induction). I show that central aspects of such variational induction are realized by LLMs. In particular, the training of large language models requires variation in evidence drawing on as much text data from as wide a range of contexts as possible. If sufficient data is available, difference and indifference makers among the input token sequence are identified during training. The resulting large language models essentially consist of aggregations of vast numbers of probabilistic laws, where each law relates Boolean combinations of ordered input tokens with output probability vectors. Linguistic meaning can be extracted, because variational induction allows for distinguishing between spurious and necessary relationships. The hierarchical layer structure of LLMs can be interpreted as a continuous probabilistic generalization of the deterministic binary Boolean logic of conventional variational induction. Specific features of LLMs that go beyond a simple neural network architecture such as token embeddings or self-attention are discussed to determine their role in the context of variational induction. Token embeddings transform the input sequence into a compressed representation capturing semantic relationships so that difference and indifference makers can be found more efficiently. Self-attention allows for analysing long-range difference making dependencies in token sequences, which are prevalent in natural language. In the history of science, the difference making logic employed by large language models has been widely used. Maybe most importantly, it underlies the experimental method, where causal relations are derived by systematically varying individual circumstances to determine their influence on a phenomenon.
| Export/Citation: | EndNote | BibTeX | Dublin Core | ASCII/Text Citation (Chicago) | HTML Citation | OpenURL |
| Social Networking: |
Monthly Views for the past 3 years
Monthly Downloads for the past 3 years
Plum Analytics
Actions (login required)
 |
View Item |
![[feed]](http://philsci-archive.pitt.edu/style/images/feed-icon-32x32.png){kind=icon}