I need to give some set-up; many readers will be familiar with some or all of this. So you can skip the next few paragraphs if you already know about the base-rate objection to the No-Miracles Argument and the Pessimistic Induction.
I suspect many readers are familiar with the base-rate fallacy; there are plenty of explanations of it around the internet. But just to have a concrete example, let’s consider a classic case of base-rate neglect. We are given information like the following, about a disease and a diagnostic test for this disease:
(1) There is a disease D that, at any given time, 1 in every 1000 members of the population has: Pr(D)=.001.
(2) If someone actually has disease D, then the test always comes back positive: Pr(+|D)=1.
(3) But the test has a false positive rate of 5%. That is, if someone does NOT have D, there is a 5% chance the test still comes back positive: Pr(+|~D)=.05.
Now suppose a patient tests positive. What is the probability that this patient actually has disease D?
Someone commits the base-rate fallacy if they say the probability is fairly high, because they discount or ignore the information about the ‘base rate’ of the disease in the population. Only 1 in 1000 people have the disease. But for every 1000 people who don’t have it, 50 people will test positive. You have to use Bayes’ Theorem to get the exact probability that someone who tests positive has the disease; the probability turns out to be slightly under 2%.
In the context of the No-Miracles and Pessimistic Induction arguments, the objection is that both arguments ignore a relevant base rate. For example, the No-Miracles argument says:
(A) Pr (T is empirically successful | T is approximately true) = 1
(B) Pr (T is empirically successful | ~ (T is approximately true)) <<1. Inequality (B) is supposed to capture the ‘no-miracles intuition’: the probability that a false theory would be empirically successful is so low that it would be a MIRACLE if that theory were empirically successful. Hopefully you can see that (A) corresponds to (2) in the original, medical base-rate fallacy example, and (B) corresponds to (3). Empirical success is analogous to a positive test for the truth of a theory, and the no-miracles intuition is that the false-positive rate is very low (so low that a false positive would be a miracle). The base-rate objection to the No-Miracles argument is just that the No-Miracles argument ignores the base rate of true theories in the population of theories. In other words, in the NMA, there is no analogue of (1) in the original example. Without that information, even a very low false-positive rate cannot license the conclusion that an arbitrary empirically successful theory is probably true. (And furthermore, that base rate is somewhere between extremely difficult and impossible to obtain: what exactly is the probability that an arbitrary theory in the space of all possible theories is approximately true?)
OK, that concludes the set-up. Now I can state my concern: I am not sure the objectors’ demand for the base rate of approximately true theories in the space of all possible theories is legitimate. Why? Think about the original medical example again. There, we are simply GIVEN the base rate, namely (1). But how would one acquire that sort of information, if one did not already have it? Well, you would have to run tests on large numbers of people in the population at large, to determine whether or not they had disease D. These tests need not be fast-diagnosing blood or swab tests; they might involve looking for symptoms more ‘directly,’ but they will still be tests. And this test, which we are using to establish the base rate of D in the population, will still presumably have SOME false positives. (I’m guessing that most diagnostic tests are not perfect.) But if there are some false positives, and we don’t yet know the base rate of the disease in the population, then—if we follow the reasoning of the base-rate objectors to the NMA and the PI—any conclusion we draw about the proportion of the population that has the disease is fallacious, for we have neglected the base rate. But on that reasoning, we can never determine the base rate of a disease (unless we have an absolutely perfect diagnostic test), because of an infinite regress.
In short: if the NMA commits the base-rate fallacy, then any attempt to discover a base rate (when detection tools have false positives) also commits the base-rate fallacy. But presumably, we do sometimes discover base rates (at least approximately) without committing the base-rate fallacy, so by modus tollens, the NMA does not commit the base-rate fallacy.
NMA does not commit the base rate fallacy, because it does not ignore AVAILABLE evidence about the base rate of true theories in the population of theories. In the medical example above, the base rate (1) is available information; under-weighing generates the fallacy. In the scientific realism case, however, the base rate is not available. If we did somehow have the base rate of approximately true theories in the population of all theories (the gods of science revealed it to us, say), then yes, it would be fallacious to ignore or discount that information when drawing conclusions about the approximate truth of a theory from its empirical success, i.e. the NMA would be committing the base-rate fallacy. But unfortunately the gods of science have not revealed that information to us. Not taking into account unavailable information is not a fallacy; in other words, the base-rate fallacy only occurs when one fails to take into account available information.
I am not certain about the above. I definitely want to talk to some more statistically savvy people about this. Any thoughts?
