Goethe to van Gelder: Comments on “Dynamical Systems” Accounts of Cognition

Clark Glymour

Tim van Gelder has my thanks for providing me with references to the work he recommends to us, and for answering a number of questions I posed about his views. I am grateful, but not agreeable. In my reading, his thesis is this: 

Connectionism, in company with every treatment of humans as systems that do unconscious computations to produce cognitive competence, should be replaced a “dynamical systems” approach to explaining human cognition. 

The thesis is far more radical than this simple statement may appear.

1. Underdetermination, connectionism and cognitive physiology.

The great aim of cognitive science is to figure out how humans develop so as to be able to do all of the things we call “cognitive.” It aims to describe the internal mechanics--biological mechanics or otherwise--and relations with the rest of the world that make possible the common features of normal human cognition, and to describe how these mechanisms and relations develop through infancy and childhood. Cognitive psychology aims to solve this problem using the traditional sorts of evidence available to psychologists: behavior under experimentally controlled conditions, errors, self-reports of mental processes, and response times. The most obvious and most important fact about cognitive psychology is that on almost every dimension this aim, and even more specialized pieces of it, are radically underdetermined by this sort of evidence. For every fair way of describing the body of phenomena psychologists can establish, there are an infinity of processes in between that will accomodate them. The mathematical recognition of these limitations has taken several forms, for example in Kelly's "paradox of cognitive psychology," proving that if we are computers then we cannot reliably determine from input/output behavior whether systems are computers; in John Anderson's application of learning theory to force a radical Quinean conclusion; in Townsend and Ashby's study of the difficulties in separating parallel from serial processes; in Duncan Luce's elegant study of the limitations of reliable inferences from response time data, and so on. Psychologists have given three sorts of responses to these and other evidences of underdetermination. One is to ignore alternatives and treat speculation as nearly established fact; another is to try to establish only more modest, but still relevant, claims about mental processes and their development; and a third is to try to connect models of mind, in so far as possible, with biology, in the hope that biology will eventually so constrain mechanism that together with psychological experiments many of the the big questions about how cognition works can be answered.

Connectionism is the oldest and most influential instance of the third strategy. I will describe a vanilla version, but you should keep in mind that the ideas can be modified as evidence and ingenuity permit. The nervous system is regarded as a computational system based on a directed graph whose vertices are identified with nerve cell bodies and whose directed edges with synaptic junctions. The state of the computational system is the states of the several vertices--an assignment of numbers to the vertices, vaguely corresponding to cell voltages--and the "weights" assigned to edges. The state of a vertex representing a cell is a function of the weighted sums of states of its parents, the function roughly approximating how changes in cell potentials depend on inputs. Learning takes place by any of several essentially local algorithms that adjust weights. Memory resides in the weight values. 

Mathematical details aside, this basic picture was fully developed as soon as the synaptic structure of nerve cell connections and the electrical sensitivity of the nervous system were established in the 19th century. In varying levels of detail, including specifications of learning procedures, you can find these ideas in Cajal, James, Pearson, Freud, Exner, just about everywhere near the turn of the century. Most broadly, connectionism was the computational part of the project of investigating the cognitive physiology of the nervous system; the project has not stopped since before the turn of the century, and its results are now too vast to survey. What is newer is the digital computer, more developed biological knowledge, and more developed psychological experimentation. 

One sort of connectionist work describes as completely as possible the nerve connections of very simple animals and simulates them on a computer; in this way, connectionist principles and simulations have been used to help predict features of the neural wiring arrangements in some animals. A second sort of work is characteristic of physiology; it focuses on cellular structures identified both physically and by a functional role, usually for nerve structures an information processing or control role, presupposing a separation of inputs from outputs. Much of the work in neurobiology relevant to cognition is of this kind. Another sort of work tries to identify serial processes and see how they can be implemented in a connectionist system of nerve cells. Some of the most impressive work of this sort is on optical flow, where physical descriptions of serial processes in terms of differential equations are taken to approximate cellular processes that are independently known, and lead to psychological predictions that are established in the laboratory. A fourth sort of work builds networks to simulate normal behavior in using language or in visual perception, destroys random vertices from a region of the network, and shows that the resulting network simulates the behavior--including the learning behavior--of certain brain damaged people. 

The connectionist approach has very incomplete or very inadequate acccounts of human learning and memory. None of the local learning algorithms for networks have any established biological basis, and there is some evidence that learning, or at least some of it, may not be local at all. If so, global processes (e.g., changes in hormonal or gas concentrations) may make connectionist learning easier rather than more difficult. I simply don't know enough about recent research on the biological basis of memory to judge how things are faring there. But these problems arise just where more biology is needed to guide the development of reasonable models. 

The connectionist approach is a sensible response to the underdetermination problem, one that might (and of course might not), say within the next century, lead to major improvements in our understanding of how the brain works the mind. The constraints and opportunities it offers lie in the combination of studies in the biology of neural adaptation, in computational studies of network implementations of mixed serial and parallel processes, and in psychological experimentation that confirms phenomena required by the biology and by the computational hypotheses. One could think that understanding would emerge from biological studies alone, or from biology and psychology without the computational intermediate, and perhaps it would, although judging from the literature computational ideas are virtually indispensable in describing the function of cellular structures in the nervous system, and functional explications are virtually indispensable in recognizing the importance and relevance of cellular processes to cognition.

I think no one with scientific experience can read the papers reviewed by Churchland and Sejnowski, or many other sources, and doubt that it is real science, or that computational--and representational--ideas are essential to it. For many burning research issues--how the hippocampus functions, for example--identifying what inputs and outputs represent is among the central problems. In almost all of this work, an essential assumption is that cognition depends on computable biological processes. And here is where the radical character of van Gelder's thesis begins to come home: Van Gelder's thesis is that the thousands of papers on the computational biology of nervous systems relevant to cognition are scientific junk, pursued under some fundamental metaphysical error. What, ultimately, he takes that error to be we will consider in due time, but make no mistake about the boldness (or, to be less generous, the crankiness) of his claim. What should replace a century's scientific investigation of cogntiive physiology and its computational aspects is "dynamical systems."  O.K. What are they, why are they different from computational theories, and why are they better?

2. Dynamical systems and computation.

Van Gelder offers the following characterization: "dynamical systems are numerical systems, that is, systems whose changing entities are numerical - either numerically measurable quantities (features of the world) or numerical magnitudes." The definition doesn't clearly distinguish the physical system and its features, which one doubts have any "numerical entities," and the mathematical description, which may have something to do with numbers, but I think the idea is clear enough for our purpose if we keep the distinction in mind. So defined, dynamical systems include systems described by differential equations but they also include every machine model of computation ever proposed, including computation over "physical symbol systems," since the latter are discrete and so can be measured by numbers through standard encodings. 

So van Gelder hasn't offered an alternative to computational theories of mind, just a brief description of a vastly more general class of imaginable processes. So, how can dynamical systems serve as an alternative to connectionism or to the general idea that cognition proceeds through computable biological processes? What else might be going on in van Gelder's scheme to make a difference that is both so important and so plausible that computational cognitive physiology should be abandoned? Van Gelder's remarks suggests that the science he wants will be distinguished either (i) by characterizing uncomputable dynamical systems in cognition, or (ii) simply by refusing to talking about computation because it is somehow irrelevant to scientific description, or (iii) by allowing computation and its relevance but denying some other feature characteristic of uses of computation in cognitive science, in particular denying that any stages or computational aspects of a dynamical process correspond in any way relevant to cognition to circumstances outside the physical system that realizes the process, or (iv) by rejecting questions about the internal processes that bring about cognition. The last, I think, is what is really at stake, but I offer some comments on each alternative.

2.1 Difference through uncomputability.

What makes a dynamical system computable or not? There are several answers. Suppose we consider a dynamical system as a function f(w, t), where t is a real variable representing time, w is some n-tuple of numerical quantities, including possibly integers, real or complex numbers, taking values in a space of k-tuples, u, of similar objects. I assume you already know what computability on natural numbers means. Computable complex numbers are defined in terms of computable reals;  a computable real number r can be defined in various ways--as a computable sequence of rationals converging to r with a computable bound on the error at any stage in the sequence; as a number whose digits in some base--say 2--can be computed by an infinitary generalization of a Turing machine (essentially a multi-tape Turing machine that need never stop reading input or printing output), and in other ways. A computable function on a compact region of the reals can be characterized as a function that takes any computable sequence of computable reals from that region to a computable sequence of computable reals. More often in physics we are interested in the computability of functionals that take functions defined on regions of the reals to functions defined on regions of the reals. Computability of such functionals can be defined in terms of a measure of closeness of functions, and different measures are natural in different applications. Alternatively, a computable function can be viewed as any for which there is a corresponding generalization of a Turing machine that computes it. Not all definitions of computability, including these, are equivalent, nor should they be, because computability is relative to representation and, for computation on the reals, to the measure of approximation. For example, in the generalized Turing machine account of computability, multiplication by 3 is not computable in decimal representation of real numbers, but is computable in binary representation. These various accounts of computability are, according to context, fair representations of what is going on in many cases when we use a computer and numerical methods to approximate a function. So we can say: f(w,t) is computable over a time interval (in whichever sense) if for all to in that interval, f(w, to) is computable. And analogously for functionals.

There are uncomputable dynamical systems. In fact every system described by an unbounded linear operator (on the Banach space of functions assigning initial data) admits uncomputable solutions. But whether an operator is unbounded is sensitive to how we describe the system, or technically, to the norm used on the Banach space, which is just a fancy way of saying that it depends on how we measure the "closeness" of functions and thus on how we measure approximation. The wave equation has uncomputable solutions under some measures of closeness, but not under others. While very little is known about the recursion theoretic properties of non-linear systems (or , at least, little was known when my education stopped, about ten years ago), the same general point can be expected. It is important, however, not to let your intuitions rely on popular notions. Chaotic systems, for example, are not necessarily uncomputable. It is important as well not to confuse computability with practical predictability. Not only are there considerations of complexity in computing a function, but the precision required of initial data for "useful" error bounds on the output of a function may increase too rapidly with time.

If functions or families of functions, or functionals are proposed to explain the generation of cognition, we can expect they will be computable in some interesting sense; that may be only because computable systems are those we tend to postulate, or because natural dynamical systems, including people, are mostly computable. It would be very interesting to find a good empirical argument for a cognitive process whose psychological features, under any natural description, constitute an uncomputable dynamical system, but there are no examples in waiting. Although we could certainly postulate an uncomputable system, I'm not the least clear--especially in view of Kelly's theorem--just how we could establish one empirically. I asked van Gelder if he had an example. He didn't. 

2.2 Distinction by silence.

In describing aspects of cognition as dynamical systems, we can avoid computational talk if we choose, and according to van Gelder, we should avoid computational talk in cognitive psychology. The transitions a hypothesis proposes may be computable, but we ought not to describe them as computations, or say how they are computed.

One way of accomodating this injunction is to take on the big ambitions of cognitive psychology, and propound hypotheses to meet them, but avoid all explicitly computational notions in doing so. Consider what good or harm would come from obedience. The prohibition is vague and vast. If an array of cells acts as an "or" gate for some feature, we must not call it an or gate. If an array of cells acts as an adder for some feature, we must not call it an adder. We must not describe any mental process or human biological system as a finite state machine, or a push-down automaton, or....The list is unlimited and rather vague; much of the language of circuit descriptions, for example, is very close to the language of machine operations. Is it to be banned as well? It is clear that what would be lost by van Gelder's prohibition includes a great deal that is valuable: mathematical convenience, functional insight, the capacity for simulation, and the ability to bring to bear a rich body of mathematical results in understanding the cognitive physiology of the nervous system. It is not clear that anything would be gained, certainly nothing toward the fundamental underdetermination problem confronting cognitive psychology. 

Imagine demanding some comparable eliminations. Everything done in real algebra can be done in Euclidean geometry with small extentions (the addition of a unit). Suppose a philosopher came along and said: "You can do without real algebra, and you should." We should rightly regard his injunction as silly. Suppose a philosopher came to biologists and said, "You must not ever use any more functional locutions." The biologists might rightly tell the philosopher that his function was to be quiet and clean some glassware. Hartry Field gave a convincing argument that Newtonian physics could be done without numbers; suppose he had said, as he did not, that physicists should stop using numbers? Is there any good reason as yet to take van Gelder's prohibitions more seriously?

One might think van Gelder's idea is this: although computation may be implicit in dynamical descriptions of cognitive processes, it is an irrelevant feature, something in the way that notation is irrelevant, or the set theoretic types of the mathematical objects is implicit but irrelevant. In this sense, irrelevance doesn't mean alterable without loss; it means, not to the point. But if computational features of cognitive systems are irrelevant to their dynamical descriptions, that would be entirely novel in science; in every application I know of, the computational properties of the dynamical system under consideration are of the first importance. Consider classical physics. If a dynamical system is not a computer for various arrays of initial data and subsequent times, then the differential equations describing the system will not yield a method of predicting states at later times from state descriptions at earlier times, nor a method of explaining regularities of motion, nor a way of explaining the particularities of a system’s state from features of its history; the same is true for any qualitative features of the time development of systems, for example asymptotic properties such as limiting state values. Nor will we be able to provide general algorithms that use the description of the system to explain any details of its behavior. Of course a system can be a computer without our having the means to predict its course of values from our mathematical descriptions of it, but that very fact makes for fundamental and profound research problems; that is the very kind of problem, for example, posed by the classical n body problem for n greater than 2.  If the system is not a computer then the standard methods we have for approximately solving differential equations and other constrained problems, namely numerical approximation techniques, are to no avail, and that fact itself may be a very interesting feature of the system.

It might have turned out that in biological investigations of cellular processes relevant to cognition, computational notions and computational models are possible but beside the point--uninformative, unneeded, unilluminating. The evidence to the contrary seems by now overwhelming. 

I think that van Gelder regards all these losses as costless, exactly because he rejects the entire project of cognitive physiology. 

2.3 Distinction by absence of correspondence

The positions and momenta of the planets constitute a computable dynamical system, but the states of the solar system don't correspond to features of anything outside the solar system itself, or to relational features of any larger system that includes the solar system. One might think that even if people are computable dynamical systems, their internal states don't correspond in any way to actual or possible external circumstances or to relations between the individual and environment. To take seriously that thought would require showing what is wrong with thousands of pieces of research--from evoked response potentials, to activation in the visual cortex, to vestibular signals, to single cell responses, etc. etc.--that seem to establish such correspondences. Van Gelder does none of this, and in fact he does not reject representation, only representation by means of features of biological mechanisms; instead he makes vague proposals for dynamical features--e.g., "attractors" as representations. I think that is not a serious scientific proposal.

3. Natur-philosophie.

One way of abiding by some of Van Gelder’s prohibitions is to adopt a kind of neo-behaviorism, and that is how I first read Van Gelder's proposal. Skinner’s version of behaviorism tried to confine scientific inquiry and conjecture to functional--indeed dynamical--descriptions of how human and animal action depends on the environmental history to which the creature has been exposed. Doubtless van Gelder’s behaviorism would differ considerably from Skinner’s in what it lets in, but van Gelder appears to agree with Skinner in wishing to prohibit any inquiry into the internal mechanisms by which the creature does what it does or thinks what it thinks. But on reflecting further, I believe van Gelder's fundamental proposal is different, and in a way more radical and revanchist.

In several places van Gelder's remarks make clear that his deeper motivation has little or no logical connection with whether or not under some description the brain is a computer. The connections, appropriately enough, are spiritual, not logical. Van Gelder's fundamental concern is for holism, for a science of cognition that does not separate brain, body and environment into pieces that are explained by combining mechanisms internal to the parts with external relations to other systems. It is a "science" that does not have the fundamental underdetermination problem of cognitive psychology, because it does not have the goal of understanding the cognitive mechanisms of the nervous system. Accordingly, the objection to computation is that it requires a mechanistic analysis into parts whose relations constitute the computation. The objection to neural representation is the separation of subject and object into systems that interact as they do because of structures that can be studied separately.  Although many of the conclusions coincide, these were not the motivations of behaviorism. They are closer to the motivations of natur-philosophie and to its last great movement in physics, energeticism. Van Gelder's dynamicism is Goethe's natur-philosophie, Ostwald's energeticism, decked out in new mathematical garb. The precendents do not inspire optimism. Goethe discovered real facts about the world, and energeticist thermodynamics was a great theory in its time and is useful still. In the same way, psychologists who do not inquire into the internal mechanisms by which patterns of behavior they describe are realized may nonetheless provide real scientific information. But the philosophical opposition to scientific explanation in terms of the composition of fundamental and quasi-isolable parts, their relations, and the processes they produce has over and over again proved a scientific graveyard. In van Gelder's censorship of computation we hear an echo of Ostwald's censorship of atoms; Ostwald at the end of the 19th century would have had the scientific community discard the scientific work of a century on the structure of matter; van Gelder at the end of the 20th would have it now discard the scientific work of a century on the cognitive physiology of the nervous system. Wrong as he was, of the two, Ostwald had the more reason.