It is hardly surprising that these two different sources of data can come up with rather different conclusions about trends: for example, the Crime Survey estimated that crime fell by 9% between 2016 and 2017, while the police recorded 13% more offences. Which should we believe? Statisticians have more confidence in the survey, and concerns about the reliability of police-recorded crime data led it to lose its designation as a national statistic in 2014.
When we have all the data, it is straightforward to produce statistics that describe what has been measured. But when we want to use the data to draw broader conclusions about what is going on around us, then the quality of the data becomes paramount, and we need to be alert to the kind of systematic biases that can jeopardize the reliability of any claims.
Whole websites are dedicated to listing the possible biases that can occur in statistical science, from allocation bias (systematic differences in who gets each of two medical treatments being compared) to volunteer bias (people volunteering for studies being systematically different from the general population). Many of these are fairly common sense, although in Chapter 12 we shall see some more subtle ways in which statistics can be done badly. But first we should consider ways of describing our ultimate aim—the target population.
The ‘Bell-Shaped Curve’
A friend in the US has just given birth to a full-term baby weighing 6 lb 7 oz (2.91 kg). She has been told this is below average, and is concerned. Is the weight unusually low?
We have already discussed the concept of a data distribution—the pattern the data makes, sometimes known as the empirical or sample distribution. Next we must tackle the concept of a population distribution—the pattern in the whole group of interest.
Consider an American woman who has just given birth. We might think of her baby as having been drawn, as a sort of sample of only one person, from the entire population of babies recently born to non-Hispanic white women in the US (her race is important, since birth weights are reported for different races). The population distribution is the pattern made by the birth weights of all these babies, which we can obtain from the US National Vital Statistics System’s report on the weights of over a million babies born at full-term in the US in 2013 to non-Hispanic white women—although this is not the entire set of contemporary births, it is such a large sample that we can take it as the population.6 These birth weights are only reported as the numbers in groups spanning 500 g, and are shown in Figure 3.2(a).
The weight of your friend’s baby is indicated as a line at 2,190 grams, and its position in the distribution can be used to assess whether its weight is ‘unusual’. The shape of this distribution is important. Measurements such as weight, income, height, and so on can, at least in principle, be as fine-grained as desired, and so can be considered ‘continuous’ quantities whose population distributions are smooth. The classic example is the ‘bell-shaped curve’, or normal distribution, first explored in detail by Carl Friedrich Gauss in 1809 in the context of measurement errors in astronomy and surveying.* Theory shows that the normal distribution can be expected to occur for phenomena that are driven by large numbers of small influences, for example a complex physical trait that is not influenced by just a few genes. Birth weight, when looked at for a single ethnic group and gestation period, might be considered such a trait, and Figure 3.2(a) shows a normal curve with the same mean and standard deviation as the recorded weights. The smooth normal curve and the histogram are gratifyingly close, and other complex traits such as height and cognitive skills also have approximately normal population distributions. Other, less natural phenomena may have population distributions that are distinctly non-normal and often feature a long right-hand-tail, income being a classic example.
Figure 3.2
(a) The distribution of birth weights of 1,096,277 children of non-Hispanic white women in the US in 2013, born at 39–40 weeks’ gestation, with a normal curve with the same mean and standard deviation as the recording weights in the population. A baby weighing 2,910g is shown as the dashed line. (b) The mean ± 1, 2, 3 standard deviations (SDs) for the normal curve. (c) Percentiles of the normal curve. (d) The proportion of low-birth-weight babies (dark shaded area), and babies less than 2,910g (light shaded area).
The normal distribution is characterized by its mean, or expectation, and its standard deviation, which as we have seen is a measure of spread—the best-fitting curve in Figure 3.2(a) has a mean of 3,480 g (7 lb 11 oz) and a standard deviation of 462 g (1 lb). We see that the measures used to summarize data sets in Chapter 2 can be applied as descriptions of a population too—the difference is that terms such as mean and standard deviation are known as statistics when describing a set of data, and parameters when describing a population. It is an impressive achievement to be able to summarize over 1,000,000 measurements (that is, over a million births) by just these two quantities.
A great advantage of assuming a normal form for a distribution is that many important quantities can be simply obtained from tables or software. For example, Figure 3.2(b) shows the position of the mean and 1, 2 and 3 standard deviations each side of the mean. From the mathematical properties of the normal distribution, we know that roughly 95% of the population will be contained in the interval given by the mean ± two standard deviations, and 99.8% in the central ± three standard deviations. Your friend’s baby is around 1.2 standard deviations below the mean—this is also known as her Z-score, which simply measures how many standard deviations a data-point is from the mean.
The mean and standard deviation can be used as summary descriptions for (most) other distributions, but other measures may also be useful. Figure 3.2 (c) shows selected percentiles calculated from the normal curve: for example the 50th percentile is the median, the point which splits the population in half and which could be said to be the weight of an ‘average’ baby—this is the same as the mean in the case of a symmetric distribution such as the normal curve. The 25th percentile (3,167 g) is the weight under which 25% of babies lie—the 25th and 75th percentiles (3,791 g) are known as the quartiles, and the distance between them (624 g), known as the inter-quartile range, is a measure of the spread of the distribution. Again, these are exactly the same summaries as used in Chapter 2, but here applied to populations rather than samples.
Your friend’s baby lies on the 11th percentile, which means that 11% of full-term babies born to non-Hispanic white women will weigh less—Figure 3.2(d) shows this 11% as a light grey shaded area. Birth-weight percentiles are of practical importance, since your friend’s baby’s weight will be monitored relative to the growth expected for babies on the 11th percentile,* and a drop in the baby’s percentile may be a cause for concern.
For medical rather than statistical reasons, babies below 2,500 g are considered ‘low birth weight’, and those below 1,500 g ‘very low birth weight’. Figure 3.2(d) shows that we would expect 1.7% of babies in this group to be low birth weight—in fact the actual number was 14,170 (1.3%), in close agreement with the prediction from the normal curve. We note that this particular group of full-term births to non-Hispanic white mothers has a very small rate of low birth weights—the overall rate for all births in the US in 2013 was 8%, while the rate in black women was 13%, a notable difference between races.