
1.2 THOMAS BAYES 5
1.2.1 The Doctrine of Chances
The work which preserved Bayes’ place inhistory is“An Essay towards solving a Problem
in the Doctrine of Chances” published posthumously (1763) through the good offices of Bayes’
friend Richard Price. The essay begins as follows:
“Given the number of times inwhich an unknown event has happened and failed: Required the chance that
the probability of its happening inasingle trial lies somewhere between any two degrees of probability that
can be named.”
Bayes’ “unknown event” is what we would today call a Bernoulli trial: a random event with
two possible outcomes, labeled “success” and “failure.” The flipofacoin, an attempt to guess
the correct answer on a multiple choice exam, whether a newly hatched chick survives to
fledging; each is a Bernoulli trial.
The “probability of the event happening inasingle trial” is referred to as the success para-
meter of the Bernoulli trial, and denoted by p. Sometimes, the nature of the trial leads to
a specification of p.Acoin is deemed “fair” if “Heads” isaslikely as “Tails”, so that the
probability of getting “Heads” is p =1/2. On a multiple choice exam with
five options per
question, we might assume that the probability of guessing correctly is p=1/3 if we could rule
out two of the options and were forced to guess among the remaining three. In both cases, the
nature of the event leads to a specification for p.
More frequently, however, there isnobasis for knowing p. Indeed, even ifwehadabasis
for an educated guess, we might wish to put aside whatever prejudices we have, and allow
the data to speak for themselves. Suppose that having observed 10 newly hatched chicks, we
note that only four survive to fledging. Bayes’ essay seeks means to translate such data into
conclusions like “The probability that p is between 0.23 and 0.59 is 0.80.” Given “two degrees
of probability,” say a and b, the problem is to determine Pr(a ≤p≤b), i.e.,
the probability that
p is in the interval from a to b.
For those familiar with such things, it may sound as though Bayes were trying to construct
a confidence interval for p. Doubtless, Bayes’ goal was the same as that of the frequentist
statistician who computes a confidence interval: to quantify uncertainty about an unknown
quantity, using data and probabilities. As we shall see, however, Bayes’ way of accounting for
uncertainty about p isquite different from the frequentist approach, which developed later.
The difference lies in the Bayes’ use of the term “probability.” We illustrate withasimple
example.
The
authors of this text have a deformed New Zealand 50 cent piece; it is convex, rather
than flat, with the Queen’s image on the side that bulges outward (see Fig. 1.1). Consider the
event that the coin lands, when flipped, with the Queen’s image facing upward (“Heads”);
the unknown quantity of interest is p = Pr(Heads).
2
The convexity of the coinraises doubts
about whether p =
1
/
2
.
2. There isanimplicit ceteris paribus in defining the event, as also in Bayes’ essay: individual flips of the coin are replicates,
in the sense that conditions are assumed to be relatively constant with regard to the outcome. For instance, the authors have
observed that p appears to depend on the hardness of the surface on which the coin lands. Speaking entirely in terms of the
coin, and not in terms of a mathematical model, it seems impossible to explicitly define what we mean by “replicates,” and
whatever ambiguity remains in that definition is carried over into our definition of the probability p.
I. PROBABILITY AND INFERENCE