The researchers’ own ‘sceptical’ analysis led them to posterior odds of 167,000 to 1, or a 0.999994 probability that they had found Richard III. This was considered sufficient evidence to justify burying the skeleton with full honours in Leicester Cathedral.
In legal cases, likelihood ratios are typically attached to DNA evidence in which a ‘match’ of some degree is found between the suspect’s DNA and a trace found at the scene of the crime. The two competing hypotheses are that either the suspect left the trace of DNA, or someone else did, so that we can express the likelihood ratio as follows:
The number on the top of this ratio is generally taken to be one, and the number on the bottom is assumed to be the chance that a random person picked from the population would coincidentally provide the match—this is known as the random match probability. Typical likelihood ratios for DNA evidence can be in the millions or billions, although the exact values might be contested, such as when there are complications due to the traces containing a mix of DNA from multiple people.
Individual likelihood ratios are allowed in British courts, but they cannot be multiplied up, as in the case of Richard III, since the process of combining separate pieces of evidence is supposed to be left to the jury.3 The legal system is apparently not yet ready to embrace scientific logic.
Would the Archbishop of Canterbury cheat at poker?
It is a lesser known fact about the renowned economist John Maynard Keynes that he studied probability, and came up with a thought experiment to illustrate the importance of taking into account the initial odds when assessing the implications of evidence. In this exercise, he asked us to imagine playing poker with the Archbishop of Canterbury, who in the first round deals himself a winning royal flush. Should we suspect him of cheating?
The likelihood ratio for this event is
We might assume the numerator is one, while the denominator can be calculated to be 1/72,000, making the likelihood ratio 72,000—using the standards in Table 11.2, this corresponds to ‘very strong’ evidence that the Archbishop is cheating. But should we conclude that he is truly cheating? Bayes’ theorem tells us that our final odds should be based on the product of this likelihood ratio with the initial odds. It seems reasonable to assume that, at least before we started playing, we would place strong odds against the Archbishop cheating, perhaps 1 to 1,000,000, given that he is supposed to be a respectable man of the cloth. So the product of the likelihood ratio and the prior odds ends up being around 72,000/1,000,000, which are odds of around 7/100, corresponding to a probability of 7/107 or 7% that he is a cheat. So we should give him the benefit of the doubt at this stage, whereas we might not be so generous with someone we had just met in the pub. And perhaps we should keep a careful eye on the Archbishop.
Bayesian Statistical Inference
Bayes’ theorem, even if it is not permitted in UK courts, is the scientifically correct way to change our mind on the basis of new evidence. Expected frequencies make Bayesian analysis reasonably straightforward for simple situations that involve only two hypotheses, say about whether someone does or does not have a disease, or has or has not committed an offence. However, things get trickier when we want to apply the same ideas to drawing inferences about unknown quantities that might take on a range of values, such as parameters in statistical models.
The Reverend Thomas Bayes’ original paper in 1763 set out to answer a very basic question of this nature: given something has happened or not happened on a known number of similar occasions, what probability should we give to it happening next time?* For example, if a thumbtack has been flipped 20 times and it has come down point-up 15 times and point-down 5 times, what is the probability of it landing point-down next time? You might by now think the answer obvious: 15/20 = 75%. But this might not be the Reverend’s answer—he might say 16/22 = 73%. How would he come to this?
Bayes used a metaphor of a billiard table* which is hidden from your view. Suppose a white ball is thrown at random on to the table, its position along the table marked with a line, and then the white ball is removed. A number of red balls are then thrown at random on to the table, and you are told only how many lie to the left and how many to the right of the line. Where do you think the line might be, and what should be your probability of the next red ball falling to the left of the line?
For example, say five red balls are thrown, and we are told that two landed to the left, and three to the right of the line left by the white ball, as shown in Figure 11.4(a). Bayes showed that our beliefs about the position of the line should be described by the probability distribution shown in Figure 11.4(b)—the mathematics is quite complex and is given in an endnote.4 The position of the dashed line, which indicates where the white ball landed, is estimated to be 3/7 along the table, which is the mean (expectation) of this distribution.
Figure 11.4
Bayes’ ‘billiard’ table. (a) A white ball is thrown on to the table and the dashed line indicates its resting position. Five red balls are thrown on to the table and land as shown. (b) An observer cannot see the table, and is only told that two red balls landed to the left, and three on the right of the dashed line. The curve represents the observer’s probability distribution for where the white ball landed, superimposed on the billiard table. The curve’s mean is 3/7, which is also the observer’s current probability for the next red ball to land on the left of the line.
This value of 3/7 may seem odd, as the intuitive estimate might be 2/5—the proportion of red balls landing to the left of the line. Instead Bayes showed that in these circumstances we should estimate the position as
This means, for example, that before any red balls are thrown at all, we can estimate the position to be (0 + 1)/(0 + 2) = ½, whereas the intuitive approach might suggest that we could not give any answer since there is not yet any data. Essentially Bayes is making use of the information about how the position of the line has been initially decided, since we know it is picked at random by throwing the white ball. This initial information takes the same role as the prevalence used in breast screening or dope-testing—it is known as prior information and it influences our final conclusions. In fact, since Bayes’ formula adds one to the number of red balls to the left of the line and two to the total number of red balls, we might think of it as being equivalent to having already thrown two ‘imaginary’ red balls, and one having landed each side of the dashed line.
Note that if none of the five balls had landed to the left of the dashed line, we would not have estimated its position as 0/5, but instead as 1/7, which seems much more sensible. The Bayes’ estimate can never be 0 or 1, and is always nearer to ½ than the simple proportion: this is known as shrinkage, in that estimates are always pulled in or shrunk, towards the centre of the initial distribution, in this case ½.
Bayesian analysis uses knowledge about how the position of the dashed line was decided to establish a prior distribution for its position, combines it with evidence from the data known as the likelihood, to give a final conclusion known as the posterior distribution, which expresses all we currently believe about the unknown quantity. So, for example, computer software can calculate that an interval that runs from 0.12 to 0.78 contains 95% of the probability in Figure 11.4(b), and so we can claim with 95% certainty that the line marking the white ball is between these limits. This interval will become steadily narrower as more and more red balls are thrown on to the table and their positions relative to the line announced, until eventually we shall converge on the correct answer.