Kelly, Matthew (2026) Beyond Pattern Matching: Representation and the Case for a Middle-Level Theory of Large Language Models. [Preprint]

Text Beyond Pattern Matching_MKelly.pdf Download (363kB)

Abstract

Large language models are routinely described as pattern-matching engines. The description is technically defensible at the level of the training objective, but it is incomplete: it names the optimisation pressure under which these systems are produced without describing the system that pressure produces. This paper argues, using Marr's framework of explanatory levels, that the gap between training objective and resulting system marks a genuine algorithmic level of description that current LLM discourse has not yet adequately occupied. Three competing ontologies—the sequence-model account, the circuit-system account, and the quasi-cognitive account—each fail to characterise this level, in distinct ways. The paper develops the representational consequences of a structural feature of compression-based generation: that statistical structure is preserved under compression while the indexical relations anchoring outputs to particular evidential contexts are not. Large language models accordingly produce structured outputs without inheriting the evidential relations that ordinarily make representation epistemically accountable. Recent findings in mechanistic interpretability—circuits implementing reusable algorithms, superposition, linear representations of abstract properties, and features without human labels— establish that the internal organisation of these systems is substantially richer than the pattern-matching description implies, and that an adequate algorithmic-level account is both required and partially in view. The paper specifies what such an account must explain and argues that the system class large language models belong to is best characterised as one of representational compression: the case for a middle-level theory is therefore not programmatic but mandated by what the evidence already shows.

---

Export/Citation:

EndNote | BibTeX | Dublin Core | ASCII/Text Citation (Chicago) | HTML Citation | OpenURL

Social Networking:

Share |

---

Item Type:	Preprint
Creators:	Creators Email ORCID Kelly, Matthew mattkelly.curtin@gmail.com 0000-0002-7665-6220
Additional Information:	Submitted to Synthese special issue on "Models, Representation, and Computation"
Keywords:	large language models; representation; Marr's levels; algorithmic level; mechanistic interpretability; middle-level theory.
Subjects:	Specific Sciences > Cognitive Science > Computation Specific Sciences > Computation/Information General Issues > Explanation General Issues > Models and Idealization
Depositing User:	Dr Matthew Kelly
Date Deposited:	05 May 2026 13:44
Last Modified:	05 May 2026 13:44
Item ID:	29470
Subjects:	Specific Sciences > Cognitive Science > Computation Specific Sciences > Computation/Information General Issues > Explanation General Issues > Models and Idealization
Date:	4 May 2026
URI:	https://philsci-archive.pitt.edu/id/eprint/29470

Monthly Views for the past 3 years

Monthly Downloads for the past 3 years

Plum Analytics

Actions (login required)

View Item