|
Sherlock Holmes and Probabilistic Induction Soshichi Uchii, Kyoto University, Japan |
 |
|
Paper presented for the
Lunchtime Colloquium, Sept. 24, 1991,
Center for Philosophy of Science, Univ.
of Pittsburgh [This is the text-only version of: http://www.bun.kyoto-u.ac.jp/~suchii/holmes_1.html]
|
|
CONTENTS
1. What does Holmes have to do with philosophy of science?
2. Sherlock Holmes as a logician
3. Key words of Holmes's theory of reasoning
4. Herschel-Mill's theory of induction
5. Whewell's Theory of Induction
6. What is the difference between these methods
(Q0) "From a drop of water," said the writer, "a logician could infer the possibility of an Atlantic or a Niagara without having seen or heard of one or the other. So all life is a great chain, the nature of which is known whenever we are shown a single link of it. Like all other arts, the Science of Deduction and Analysis is one which can only be acquired by long and patient study, nor is life long enough to allow any mortal to attain the highest possible perfection in it. " [A Study in Scarlet, pt. 1, ch. 2]
1. What does Holmes have to do with
philosophy of science?
Today, I am going to talk about Sherlock Holmes and his relationship with philosophy of science. Since my talk is going to be something like background music, hopefully, for your lunch, I have chosen a rather light topic instead of a heavy dose of philosophy of science.
To begin with, let me briefly outline what I am going to say. Of course you know who Sherlock Holmes is, although you may not know in detail what he did and what he said. Everyone knows that he was a very good detective and solved many difficult problems in criminal investigations. Everyone knows that he was an expert of reasoning and observation, so that he could frequently tell, on his first acquaintance with you, who you are and what you do, or where you come from, just by looking at you. [By the way, do you believe that Sherlock Holmes exists? If you believe that, you are a Sherlockian!]
But how many of you know that he was a good logician, namely, a logician according to the standard of the late 19th century? If you already know this and can prove it, then probably there is no need to listen to me any more. For the first thing I want to show is that he was a good logician, and I wish to prove this on the evidence of what he did, and what he said he was doing (of course, as told by Dr. Waston or Sir Arthur Conan Doyle). This can be shown in a rather general way, not getting into any specific details of his reasoning.
Secondly, I wish to show what his method of reasoning was. This is a harder task than the first, because we have to identify the essential features of his method of reasoning. In order to show this, I have not only to examine what he says he is doing, but also to look at the methods of scientific reasoning recommended by several distinguished philosophers of science in the 19th century. I want to examine Holmes's method of reasoning in a historical setting; and this has something to do with the philosophy of science in the 19th century, and hopefully with the philosophy of science today. I will examine whether such methods are similar or dissimilar to Holmes's method. Logicians and philosophers I wish to examine are, John Herschel, John Stuart Mill, William Whewell, Augustus de Morgan, and William Stanley Jevons; however, since we do not have much time, I cannot do justice to all of them.
If I may suggest my conclusion in advance, for those listeners who are impatient, it is this: Sherlock Holmes was distinctly different from Herschel or Mill or Whewell who may be called a classical methodologist; but he was very close to de Morgan or Jevons who were an advocate of new symbolic logic and probabilistic theory of induction. And this makes him closer to logicians in the 20th century (although I am quite sure Holmes didn't know Frege at all).
But what is the point of showing all this? The rise and development of statistical method in the19th century had a great impact on the theories of scientific reasoning, and de Morgan's or Jevons's theory is a newer theory of induction in this century. And such a change of methodology is clearly reflected in the popular stories of Sherlock Holmes, which were written in the late 19th century and early 20th century.
2. Sherlock Holmes as a logician
We can find many words in Holmes stories which may be used as an evidence for showing that he was a logician. First of all, he frequently characterizes himself as a logician. Quotation Q0 is one of such examples. Here, he speaks of what a logician can do, as well as of what a logician should do. It is certainly a typical task of a logician to do "deduction and analysis," and he or she should work hard in order to be a good logician!
And secondly, many of his words clearly show his stance as a logician. The following quotations are typical examples:
(Q1) "Some facts should be suppressed, or at least, a just sense of proportion should be observed in treating them. The only point in the case which deserved mention was the curious analytical reasoning from effects to causes, by which I succeeded in unravelling it." [The Sign of Four, ch.1]
(Q2) "it is not really difficult to construct a series of inferences, each dependent upon its predecessor and each simple in itself. If, after doing so, one simply knocks out all the central inferences and presents one's audience with the starting-point and the conclusion, one may produce a startling, though possibly a meretricious, effect. " [The Adventure of the Dancing Men]
Quotation Q1 shows where his interest is, when he reads a report of criminal investigations; and since he is only interested in the method of reasoning, it is clear that his stance is nothing but a logician's.
Quotation Q2 shows, again, his stance as a logician, because he analyzes a certain effect of logical reasoning; and his analysis and observation touch upon some of the essential characteristics of logical reasoning. Each single step of logical reasoning is so easy, so obvious; but by combining such easy steps two or three, you lose sight of the necessary connection between the premisses and the conclusion. Take an extreme example: before 1931, who could see such a result as Goedel's Theorem may be obtained?
Thus, my hypothesis that Holmes was a good logician seems very promising. But its proof may not be sufficient; so let us continue our examination, getting into more specific features of his reasoning and his opinions about what he is doing.
3. Key words of Holmes's theory of reasoning
There are several key words when Holmes characterizes his own method of reasoning.
Method by elimination, method of exclusion
(Q3) "By the method of exclusion, I had arrived at this result, for no other hypothesis would meet the facts." [A Study in Scarlet, pt. 2, ch. 7]
(Q4) "You will not apply my precept," he said, shaking his head. "How often have I said to you that when you have eliminated the impossible, whatever remains, however improbable, must be the truth?" [The Sign of Four, ch. 6]
[Let me give you a simple example of this method. In the beginning part of The Sign of Four, Holmes surprizes Watson by telling him that Watson went to the post-office in order to send a telegram. His reasoning may be put in the following form:
(1) A v B v C (this is already proved from other sources);
(2) -A (from observational evidence);
(3) -B (from observational evidence);
(4) therefore, C (conclusion).
(Let A, B, C mean, respectively, "Watson went to the post-office in order to send a letter"; or "in order to buy stamps or postcards"; or "in order to send a telegram.")
This seems simple and perfectly all right. But Holmes's eliminative method may not be as simple as this, if we want to take into consideration the link between the three premisses and their evidence, which may be probabilistic. Notice that in (Q4), eliminative method is somehow combined with the consideration of probability or improbability.]
Next, it is interesting to notice that Holmes seldom uses the word "induction," when he speaks of his own method. Instead, he prefers the word "hypothesis."
Hypothesis
(Q5) "I have devised seven separate explanations, each of which would cover the facts as far as we know them. But which of these is correct can only be determined by the fresh information which we shall no doubt find waiting for us." [The Adventure of the Copper Beeches]
(Q6) "Where is he, then?"
"I have already said that he must have gone to King's Pyland or to Mapleton. He is not at King's Pyland. Therefore he is at Mapleton. Let us take that as a working hypothesis and see what it leads us to." [Silver Blaze]
So far, even a layman can understand what Holmes wanted to say. But we need good philosophical knowledge in order to understand the following words:
Analytical reasoning, synthetic reasoning (reasoning backward, reasoning forward)
(Q7) "I have already explained to you that what is out of the common is usually a guide rather than a hindrance. In solving a problem of this sort, the grand thing is to be able to reason backward. That is a very useful accomplishment, and a very easy one, but people do not practise it much. In the everyday affairs of life it is more useful to reason forward, and so the other comes to be neglected. There are fifty who can reason synthetically for one who can reason analytically."
"Most people, if you describe a train of events to them, will tell you what the result would be. They can put those events together in their minds, and argue from them that something will come to pass. There are few people, however, who, if told them a result, would be able to evolve from their own inner consciousness what the steps were which led up to that result. This power is what I mean when I talk of reasoning backward, or analytically." [A Study in Scarlet, pt.2, ch.7]
Comments: As you may well know, Cartesian analysis is a procedure like this: given a problem to be solved, we examine the conditions to be fulfilled, and divide them into simpler conditions which are easier to solve (in Descartes's words, "divide each of the difficulties I was examining into as many parts as possible and as is required to solve them best"). We go backward, so to speak, from the given problem to the simpler and solvable constituents. In the preceding quotation, Holmes explains a similar procedure in terms of cause-effect relations; i.e., given a problem consisting of a number of facts (effects), we go backward in search for their unknown causes. (Presumably, Holmes adopted this way of explanation because this was easier for Dr. Watson to understand!)
[By the way, eliminative method and analysis are closely related. We can show this by means of Jevons's idea of logical alphabets. For example, given three propositions A, B, C, we can form logical alphabets in the following way: for each proposition, there are two possibilities, either affirmation or negation; so let us signify the former by a Capital letter, the latter by a lower case letter. And further, let us understand that juxtaposing two or more letters means a logical conjunction. Then, we can express all the possibilities out of these three propositions by the following eight conjunctions, which are Jevons's logical alphabets in this case:
ABC, ABc, AbC, Abc, aBC, aBc, abC, abc.
And these correspond to Descartes's "as many parts as is required to solve them best." The process of reasoning is essentially eliminative in that, given any information, this information eliminates some of the logical alphabets; and what remains after all premisses are represented, that is the conclusion. This process takes place within the framework of Cartesian analysis.]
We finally come to the most important key word:
Balance of probabilities
(Q8) "Ah,that is good luck. I could only say what was the balance of probability. I did not at all expect to be so accurate."
"But it was not mere guesswork?"
"No, no: I never guess. It is a shocking habit---destructive to the logical faculty. What seems strange to you is only so because you do not follow my train of thought or observe the small facts upon which large inferences may depend." [The Sign of Four, ch. 1]
(Q9) "We are coming now rather into the region of guesswork," said Dr. Mortimer.
"Say, rather, into the region where we balance probabilities and choose the most likely. It is the scientific use of the imagination, but we have always some material basis on which to start our speculation." [The Hound of the Baskervilles, ch. 4]
Scientific use of the imagination
See the last quotation above.
We have to notice that Sherlock Holmes is contrasting his method, which essentially depends on the balance of probabilities, with "mere guesswork," which he despises as destructive to the logical faculty. He is saying that his method is logical and scientific, although it might seem uncertain or unstable to a layman, like Watson or Mortimer.
It should be clear by now, from these examinations of key words, that Sherlock Holmes's method of reasoning has a firmer structure than you might have imagined at first sight. Is there any theory of scientific method which captures all or almost all of these features? Let us next see some of the 19th century methodologists.
4. Herschel-Mill's theory of induction
Herschel, in his Preliminary Discourse on the Study of Natural Philosophy (1830) suggested ten rules of philosophizing. These are rules for discovering and confirming causal relations which can explain the given phenomena (we will see some of these rules later). According to him, the objectives of scientific inquiry are described as follows:
(Q10) The first thing that a philosophic mind considers, when any new phenomenon presents itself, is its explanation, or reference to an immediate producing cause. If that cannot be ascertained, the next is to generalize the phenomenon, and include it, with others analogous to it, in the expression of some law, in the hope that its consideration, in a more advanced state of knowledge, may lead to the discovery of an adequate proximate cause. [sect. 137]
The relation of cause and effect is, according to him, an invariable connection, i.e. the same cause always produces the same effect (unless, of course, prevented by some counteracting cause). Notice that, thus characterized, there is no room for probability in the relation of cause and effect.
Herschel's ten rules are modified and reorganized by Mill as his Five Canons of Induction.
(1) Method of agreement
(2) Method of difference
(3) Joint method of agreement and difference
(4) Method of residues
(5) Method of concomitant variations
The essence of these canons is that they enable us to eliminate hypotheses which are incompatible with given data and the law of universal causation (which says, roughly speaking, that every event has a cause).
Let me explain such a process of elimination by a simple example. Suppose that, after our family dinner, my wife and I began to feel a strong stomach ache but our two daughters were all right. In order to identity the cause of our pain, you may analyze the situation in this way: What did they eat during the dinner? They all had steamed rice, beefsteaks, greenbeans, and a melon for dessert. Since only the parents feel the pain, its cause must be in some conditions which are satisfied by both of them, but not by the children; now the parents like beer, and they had a glass of beer, whereas the daughters didn't; therefore, that beer must be the cause of the stomach ache. In this reasoning, both methods of agreement and of difference are used; and these somehow succeeded in eliminating many factors from the candidates of cause.
Thus you may think that Mill's method is quite close to Holmes's method of elimination. But we have to see whether or not Mill's method can capture all features of Holmesian reasoning.
Mill emphasized the role of induction as a method of (empirical) proof. This is clear from his distinction between Deductive Method and Hypothetical Method (what we call hypothetico-deductive method corresponds to the latter, not the former). The former consists of three steps: induction, deduction, and verification. By means of induction, we ascertain causes or laws involved in a given case; next, we combine these causes and laws and calculate their effects, i.e. we deduce concrete consequences from them; and finally, we verify whether or not such consequences hold in the actual case. Hypothetical Method differs from this, in that the step of induction is absent. We merely assume such causes or laws, deduce particular consequences from them, and try to verify whether or not they hold (notice that, according to Mill, even a logical or mathematical truth must be verified ultimately by referring to experience). Hypothetical Method lacks any proof of causes and laws by means of induction.
Herschel is not as strict as Mill on this point. But it is clear that Herschel also aims at some sort of proof by means of his ten rules; so that he claims that we can obtain certainty in the field of physics. He distinguishes theoretical certainty and practical certainty, and admits that the former can be attained only in such fields as mathematics or geometry. However, he is never in doubt that in many fields of physics we have attained the latter, practical certainty; and he has a strong belief that our knowledge by means of scientific inquiry can be certain in that sense.
Then a question arises: Does Holmes share such a belief in the certainty of scientific knowledge? We will come back to this question after we give a brief look at what Whewell said about induction.
5. Whewell's Theory of Induction
Whewell emphasized the importance of concepts and ideas, as well as of empirical factors, in science. He was a Kantian, not an empiricist like Herschel or Mill. According to him, there are three steps of induction: (1) explication of conceptions, (2) colligation of facts by means of a conception, and (3) verification. (2) is the central idea in Whewell's theory. For example, the concept of elliptical orbit can unite all observational data of Mars into a coherent whole; this Whewell calls "the colligation of facts by means of a concept." Similarly, if we can explain and unite several disconnected facts by means of a hypothesis, this may be regarded as a candidate for such a colligation. Thus, Whewell's idea has a strong affinity to Peirce's notion of abduction. [And we can find many cases of such a colligation in the reasoning of Holmes; e.g., deciphering is a typical example, since we have to find a key pattern in order to decipher; the key pattern colligates, so to speak, disconnected words into a meaningful sentence.]
Although the central idea of Whewell's induction is different from Herschel's or Mill's, Whewell agrees with them that our knowledge of laws of nature can attain certainty (he claims even necessity). For example, if a law or hypothesis for explaining a certain kind of phenomena turns out good for explaining another kind of phenomena, he calls this "the consilience of inductions"; and this is, for Whewell, a clear case of our attaining certainty.
6. What is the difference between these methods
and Holmes's?
Holmes's reasoning by elimination has a strong affinity with Mill's; his reasoning backward, i.e. imagining several hypotheses for explaining the given facts and selecting the best one, also has a strong affinity with Whewell's idea of the colligation of facts. But however strong these affinities may be, there is one essential factor which is present in Holmes's method but absent in Herschel-Mill's or Whewell's method. That is the consideration of probabilities of hypotheses, and of the probabilistic connections between hypotheses and data. Only by taking these into consideration, it can make sense to speak of "the balance of probabilities."
Comments: Unlike Whewell and Mill, who were unsympathetic to statistical or probabilistic methods, Herschell clearly saw the merits of these methods in a well-known article in Edinburgh Review (1850), in which he reviewed Quetelet's work. [Maxwell is said to be influenced by this article, and began his research on the kinetic theory of gases.] However, we can find no such views in his 1830 book on scientific methodology (and we know that he was unsympathetic or hostile to Darwin's theory of natural selection, which essentially depends on a statistical principle). Thus Herschel comes in between one extreme of Whewell and Mill, and the other extreme of de Morgan and Jevons.
7. Jevons's Method of Inverse Probability
Then what is the method which unites all of Holmesian ideas---reasoning by elimination, reasoning backward or analytical reasoning, hypothesis, and the balance of probability? We can find the best example in W. S. Jevons's theory of induction.
(Q11) The study both of Formal Logic and of the Theory of Probabilities has led me to adopt the opinion that there is no such thing as a distinct method of induction as contrasted with deduction, but that induction is simply an inverse employment of deduction. [The Principles of Science, viii]
But what is "an inverse employment of deduction"? This is not a simple hypothetico-deductive method suggested by Herschel-Mill or Whewell. Their methods do not employ, in any essential way, the notion of probability.
What Jevons means is that induction is an inverse application of the theory of probability [op. cit., 240]. And he says that this inverse application depends on the following proposition (this is a special case of the Bayes's Theorem, with an equal prior probability distribution):
(Q12) If an event can be produced by any one of a certain number of different causes, all equally probable a priori, the probabilities of the existence of these causes as inferred from the event, are proportional to the probabilities of the event as derived from these causes. [op. cit., 242-243]
[The point of (Q12) may become easier to understand with some symbols of probability theory. Let P(X, Y) mean the conditional probability of X on Y; and let P(E, C1)=p, P(E, C2)=q, and P(E, C3)=r; then P(C1, E)=p/(p+q+r).]
Here, it is clear that even the relation of cause and effect is regarded as containing a probabilistic factor. Speaking more strictly, what Jevons means must be understood as saying that our knowledge of causal relations must inevitably contain a probabilistic factor. And such views of knowledge containing uncertainty come from his teacher, Augustus de Morgan. And de Morgan in turn was a British advocate of Laplace's method of inverse probability.
Now, getting back to our main point, we can tell what the balance of probability is, according to this method of inverse probability, by saying that the most probable cause of an event which has happened is that which would most probably lead to the event supposing the cause to exist (all possible causes are assumed to be equiprobable).
Jevons's method of inverse probability essentially agrees with Sherlock Holmes's method. (1) It can be regarded as eliminative in the sense that it eliminates all but the most probable hypothesis (conditional on the given event). (2) It is reasoning backward, because we infer the probabilities of hypotheses given the event as data. Or, to put it Jevons's way, it is reasoning the probability of a cause, given the effect and the probabilistic relations between the effect and its possible causes. (3) And finally, it tells us what the balance of probability is. What Holmes means by the 'scientific use of the imagination' is to invent many probable hypotheses (given the initial information), and to select among them according to the balance of probabilities (after testing them by all data obtained by investigation).
Maybe some words of caution are in order here. Although I assert that probabilistic element is essential in Holmes's method, I do not mean Herschel-Mill's method or Whewell's are incompatible with it. Thus, it is perfectly all right, for instance, if Holmes applies Mill's eliminative induction, or Whewell's colligation, first, and then uses his reasoning backward, in terms of probabilities.
As a further confirmation of the previous conclusion, I can quote Holmes's conversation with Watson in chapter 10 of The Sign of Four ; there he speaks of probabilistic or statistical method, and even of an a priori probability of a hypothesis. This is crucial, because without good knowledge of statistical method, he cannot use such a word.
(Q13) "We have no right to take anything for granted," Holmes answered. "It is certainly ten to one that they go downstream, but we cannot be certain. From this point we can see the entrance of the yard, and they can hardly see us. ..... We must stay where we are. See how the folk swarm over yonder in the gaslight."
"They are coming from work in the yard."
"Dirty-looking rascals, but I suppose every one has some little immortal spark concealed about him. You would not think it, to look at them. There is no a priori probability about it. A strange enigma is man!"
(Q14) "Winwood Reade is good upon the subject," said Holmes. "He remarks that, while the individual man is an insoluble puzzle, in the aggregate he becomes a mathematical certainty. You can, for example, never foretell what any one man will do, but you can say with precision what an average number will be up to. Individuals vary, but percentages remain constant. So says the statistician."
The first of these two quotations is one of the hardest passages from Holmes, but I think the point is this: If we assign an extreme value (either zero or one) to any hypothesis as its a priori probability, we cannot learn from our experience. So Holmes is saying in (Q13) that, even to such a hypothesis as supposing an immortal spark in a dirty-looking rascal, we should not assign a priori probability zero. This interpretation is indeed in the spirit of his first remark that "we have no right to take anything for granted."
The second, (Q14), states his agreement with statisticians' observation; and since this is now familiar to us, I don't have to explain it any further.
From all this, we can conclude that Sherlock Holmes knew well the new probabilistic theory of scientific reasoning, and applied it, as his own method, to his criminal investigations. Moreover, since the major advocates of this new theory were also an expert of symbolic logic, we may conclude, in all probability, that Holmes knew Boolean symbolic logic as well. On the total evidence, the balance of probability is that he was a very good logician.
Bibliography
De Morgan, Augustus. (1847) Formal Logic.
Doyle, Arthur Conan. The Penguin Complete Sherlock Holmes, Penguin Books, 1981.
Eco, Umberto and Thomas A Sebeok, eds. (1983) The Sign of Three: Dupin, Holmes, Peirce. Indiana University Press.
Herschel, J. F. W. (1830) A Preliminary Discourse on the Study of Natural Philosophy. Johnson Reprint, New York, 1966.
Herschel, J. F. W. (1850) "Quetlet on Probabilities." Edinburgh Review, 92.
Jevons, William Stanley. (1874) The Principles of Science, Macmillan.
Mill, J. S. (1843) A System of Logic.
Uchii, S. (1988) Sherlock Holmes's Theory of Reasoning (in Japanese), Kodansha.
Whewell, W. (1837) History of the Inductive Sciences, 3 vols.
Whewell, W. (1847) Philosophy of the Inductive Sciences, 2 vols., 2nd ed.
For the web resources, begin with 221B Baker Street and Sherlockian Homepage.
Postscript
The materials presented in this paper are a part of my book on Holmes written in Japanese (Uchii, 1988); my treatment of Holmes's reasoning in this book is more comprehensive.
In reproducing this paper on the web page, I wish to thank Prof. Jerry Massey (then the Director of the Center for Philosophy of Science) for his warm hospitality.
Last modified February 27, 2001. (c) Soshichi Uchii