Representation of data

Choosing a way to present data

Statistics starts with a list of values, the raw data. Before calculating anything, decide how to show the data so that its important features can be seen. Paper 5 expects you to choose a suitable diagram and to say what it shows well and what it hides.

• Stem-and-leaf diagram. Every value is kept, so you can still find the exact median, quartiles and mode, and the diagram also shows the shape of the distribution. It only works for a fairly small data set (up to about 50 values) recorded as numbers with a sensible stem. A back-to-back stem-and-leaf diagram puts two data sets on one stem, which makes them easy to compare.
• Box-and-whisker plot. It shows five numbers only: the lowest value, the lower quartile, the median, the upper quartile and the highest value. It is excellent for comparing two or more data sets side by side, because the medians and the spreads are visible at a glance. Its disadvantage is that the individual values, the mode and the number of values are all lost.
• Histogram. Used for continuous data that has been grouped, especially a large data set. The area of each bar represents the frequency, so classes may have unequal widths. The original values are lost, so any statistic calculated from a histogram is an estimate.
• Cumulative frequency graph. Also for large grouped data sets. It is the best diagram for estimating the median, the quartiles, any percentile and the number of values above, below or between given values.

A good answer to "give one advantage" names a specific feature and ties it to the context, for example "a stem-and-leaf diagram shows every individual mass, so the exact median can be found", rather than "it is clearer".

Stem-and-leaf diagrams

A stem-and-leaf diagram splits each value into a stem (the leading digits) and a leaf (the last digit). The leaves in each row are written in increasing order, and the diagram must have a key, such as $4|7$ means 47 kg. Without a key the diagram is incomplete and loses a mark.

Because the values are in order, the median of $n$ values is the $\frac{n+1}{2}$th value. When $n$ is even this is halfway between the two middle values. The lower quartile $Q_1$ is the median of the lower half of the data and the upper quartile $Q_3$ is the median of the upper half. When $n$ is odd, leave the median itself out of both halves. For example, with 11 values the median is the 6th value, and the quartiles are the 3rd and 9th values. The interquartile range is $Q_3-Q_1$.

In a back-to-back diagram the leaves of the left-hand data set are written outwards from the stem, so they are read from right to left: in the row $852\left|5\right|$ the left-hand values are 52, 55 and 58. Count carefully on that side, since it is the side where most slips happen.

To compare two data sets, always give one comparison of a measure of central tendency (for example the medians) and one of a measure of spread (for example the interquartile ranges), each in the context of the question: "the swimmers' pulse rates are lower on average and less spread out".

Box-and-whisker plots

A box-and-whisker plot (box plot) is drawn against a scale. The box runs from the lower quartile to the upper quartile, a line across the box marks the median, and the whiskers run out to the lowest and highest values. So the box holds the middle half of the data, and each whisker holds a quarter of it.

That fact is useful for estimates: if 80 people are represented, about 20 of them lie above the upper quartile and about 60 lie above the lower quartile.

A question may define an outlier, commonly as a value more than $1.5\times$ the interquartile range below $Q_1$ or above $Q_3$. Use the definition the question gives. An outlier is usually plotted as a separate cross, with the whisker stopping at the most extreme value that is not an outlier.

When you draw a box plot, use a ruler and a labelled linear scale, and plot all five values accurately. Two box plots drawn against the same scale are the standard way to compare two data sets.

Histograms

In a histogram there are no gaps between the bars, because the data is continuous, and the area of each bar is proportional to the frequency. With classes of unequal width the height must therefore be the frequency density:
$$\text{frequency density}=\frac{\text{frequency}}{\text{class width}}$$
Why? A wide class collects values from a longer interval, so its frequency is naturally larger. Dividing by the width gives the frequency per unit, which is what can be compared fairly between classes. If you plotted frequency instead, a wide class would look far more important than it is.

Use the true class boundaries. A class written $20⩽t<30$ has boundaries 20 and 30 and width 10. If lengths are recorded to the nearest centimetre and a class is written 20 to 29, its boundaries are 19.5 and 29.5, but its width is still 10. If ages are given in completed years, the class 20 to 29 means $20⩽\text{age}<30$.

Reading a histogram backwards, frequency = frequency density $\times$ class width. To estimate how many values lie in part of a class, assume the values are spread evenly across the class and take the matching fraction of its frequency. Sometimes the vertical scale is not labelled, and you are told the frequency of one class. Use that class to find how much frequency one unit of area stands for.

Cumulative frequency graphs

A cumulative frequency is a running total: the number of values less than or equal to the upper boundary of each class. To draw the graph, plot each cumulative frequency against the upper class boundary of its class, starting with a point at the lower boundary of the first class with cumulative frequency 0. Join the points with a smooth curve or with straight lines, as the question directs.

Plotting at the upper boundary matters. The cumulative frequency 52 for the class $2<t⩽4$ means 52 values are at most 4, not at most 3, so the point is $(4,52)$. Plotting at mid-values is a common error and costs the mark for the graph.

For $n$ values read from the graph:

• the median at cumulative frequency $\frac{n}{2}$;
• the lower and upper quartiles at $\frac{n}{4}$ and $\frac{3n}{4}$;
• the $k$th percentile at $\frac{k}{100}n$, so the 90th percentile is at $0.9n$;
• the number below a value $x$ by reading up from $x$, and the number above $x$ as $n$ minus that reading;
• the number between two values as the difference of two readings.

Draw the reading lines on the graph in the exam: across from the cumulative frequency, then down to the axis. All of these are estimates, because the graph assumes the values are spread evenly within each class.

Measures of central tendency and variation

A measure of central tendency is a typical value:

• the mean $\bar{x}=\frac{\sum x}{n}$ uses every value, but an extreme value can pull it a long way;
• the median is the middle value, and it is not affected by a few extreme values;
• the mode is the most common value, and it is the only average for non-numerical data.

A measure of variation tells you how spread out the data is:

• the range is highest minus lowest, which is quick to find but depends on two values only, so one extreme value distorts it;
• the interquartile range is $Q_3-Q_1$, the spread of the middle half, so it ignores extreme values;
• the standard deviation measures the typical distance of the values from the mean, and uses every value.

Pair them sensibly: the median with the interquartile range, the mean with the standard deviation. If the data has an extreme value, the median and interquartile range describe it better. If the data is roughly symmetrical with no extreme values, the mean and standard deviation use all the information and are preferred.

Mean and standard deviation from data and from totals

The variance is the mean of the squared deviations from the mean, and the standard deviation is its square root. MF19 gives both forms:
$$\text{standard deviation}=\sqrt{\frac{\sum (x-\bar{x}{)}^2}{n}}=\sqrt{\frac{\sum x^2}{n}-{\bar{x}}^2}$$
The second form is the one to use with totals. It says: variance = (mean of the squares) $-$ (square of the mean). The two forms agree because expanding $\sum (x-\bar{x}{)}^2$ gives $\sum x^2-2\bar{x}\sum x+n{\bar{x}}^2=\sum x^2-n{\bar{x}}^2$.

In Paper 5 you divide by $n$, not by $n-1$. (Dividing by $n-1$ belongs to Paper 6, where it estimates a population variance from a sample.)

For grouped data use the mid-value of each class as $x$ and the frequencies $f$:
$$\bar{x}=\frac{\sum xf}{\sum f},\text{standard deviation}=\sqrt{\frac{\sum x^{2}f}{\sum f}-{\bar{x}}^2}$$
These are estimates, because every value in a class is treated as if it were the mid-value.

On your calculator use statistics mode to check, but show the substitution in the formula: unsupported answers from a calculator earn no marks.

The totals can also be used backwards. If you know $n$, $\bar{x}$ and the standard deviation $s$, then $\sum x=n\bar{x}$ and $\sum x^2=n(s^2+{\bar{x}}^2)$. This is the key step in any question that adds or removes a value, or combines two data sets.

Coded totals and combining two data sets

Data is often coded by subtracting a convenient number $a$ from every value, so the totals are smaller. You are then given $\sum (x-a)$ and $\sum (x-a{)}^2$.

Subtracting $a$ moves every value by the same amount, so it moves the mean by $a$ but does not change the spread at all. That gives:
$$\bar{x}=a+\frac{\sum (x-a)}{n},\text{standard deviation}=\sqrt{\frac{\sum (x-a{)}^2}{n}-(\frac{\sum (x-a)}{n}{)}^2}$$
Notice that the standard deviation is worked out entirely from the coded totals, with no $a$ added back.

To recover the uncoded totals, expand: $\sum (x-a)=\sum x-na$, so $\sum x=\sum (x-a)+na$. And $\sum (x-a{)}^2=\sum x^2-2a\sum x+n{a}^2$, so $\sum x^2=\sum (x-a{)}^2+2a\sum x-n{a}^2$. The $na$ and $n{a}^2$ terms come from adding $a$ or $a^2$ once for each of the $n$ values: forgetting the $n$ is the commonest slip.

Two data sets. To combine two data sets, never average the two means or the two standard deviations. Instead, add the totals: $n=n_1+n_2$, $\sum x=\sum x_1+\sum x_2$ and $\sum x^2=\sum x_1^2+\sum x_2^2$. Then use the formulas on the combined totals. If you are given means and standard deviations instead of totals, first turn each into $\sum x$ and $\sum x^2$ as above.