What you'll learn
- How a sample can provide information about a wider population.
- How representativeness, sample size and sampling variability affect your conclusions.
- How to compare sample statistics such as the mean, median and measures of spread.
- How to write cautious, contextual conclusions without claiming more than the evidence supports.
From populations to samples
In statistics, you are often interested in a very large group. Measuring every member of that group may be too expensive, slow or impractical, so you collect data from a smaller group instead.
Population and sample
A population is the complete set of people or objects you want information about. A sample is the smaller group from which data are actually collected.
For example, if a company wants to investigate the lifetime of all batteries produced by a particular factory, the population is all batteries produced by that factory. A selection of 100 tested batteries would be a sample.
The word population does not necessarily mean people. It could refer to manufactured components, plants, transactions, journeys or measurements.
Parameters and statistics
A numerical value describing a population is called a parameter. A numerical value calculated from a sample is called a statistic.
For example:
- the population mean lifetime is a parameter;
- the sample mean lifetime is a statistic;
- the population proportion of defective batteries is a parameter;
- the sample proportion of defective batteries is a statistic.
The population parameter is usually unknown. You use the sample statistic to estimate it.
Using sample evidence
A sample statistic gives evidence about the corresponding population parameter, but it will not usually be exactly equal to it.
What is an informal inference?
An inference is a conclusion about a population drawn from sample data. An informal inference uses the observed data and your statistical judgement rather than a formal hypothesis test or confidence interval.
You might use a sample to suggest that:
- one population tends to have higher values than another;
- a population value is likely to be near a sample estimate;
- one population appears more variable than another;
- a claimed value does or does not seem consistent with the sample.
Your conclusion must remain cautious because a different sample would probably give slightly different results.
Estimating a population mean
A random sample of 40 train journeys on a route has a mean duration of 52.6 minutes. Use this sample to comment on the mean duration of all journeys on the route.
-
The sample mean of 52.6 minutes is the available estimate of the unknown population mean.
-
Because only 40 journeys were observed, the sample mean is unlikely to equal the population mean exactly. Natural variation between journeys and between possible samples creates uncertainty.
-
A suitable inference is: The sample suggests that the mean duration of all journeys on this route is around 52.6 minutes. It would not be justified to state that every journey takes 52.6 minutes or that the population mean is exactly 52.6 minutes.
Representativeness
A representative sample reflects the important characteristics of the population. For an inference to be reliable, the sample should resemble the population relevant to the investigation.
Suppose you want to estimate the average time students at a college spend on homework. A sample taken only from an after-school revision club may contain unusually committed students, so it may not represent the whole college.
Bias
Bias is a systematic tendency for a sampling method to over-represent or under-represent particular outcomes. A biased sample can produce a misleading estimate even if the sample is large.
Possible sources include:
- selecting from an incomplete sampling frame;
- allowing people to volunteer;
- collecting data from only one location or time;
- asking a leading question;
- receiving responses mainly from people with strong opinions.
Assuming a large sample removes bias
A large biased sample can give a very precise estimate of the wrong quantity. Increasing sample size reduces random sampling variation, but it does not automatically correct a poor sampling method.
Assessing whether a sample is representative
A cinema asks people leaving a 10 p.m. horror film whether the cinema should show more horror films. Of the 200 people questioned, 72% say yes.
-
The population of interest appears to be all cinema customers, but the sample contains only customers who chose to attend a late horror film.
-
These customers are likely to be more interested in horror films than the population as a whole, so horror fans are over-represented.
-
The result provides evidence about customers attending that particular screening, but it is not strong evidence that 72% of all cinema customers want more horror films.
Sampling variability
If you take several samples from the same population, you will not usually obtain identical results. This natural sample-to-sample difference is called sampling variability.
For example, different random samples may have different:
- means;
- medians;
- ranges or interquartile ranges;
- proportions possessing a particular characteristic.
A result based on one sample is therefore uncertain. This does not mean that the sample is useless; it means that your conclusion should acknowledge the uncertainty.
The effect of sample size
Larger samples generally show less sampling variability than smaller samples, provided that the sampling method is appropriate. An unusual individual observation also has less influence on a large sample.
However, sample size is only one part of reliability. You should consider both:
- how the sample was chosen;
- how many observations it contains.
Reliability of an inference
A convincing informal inference usually needs a reasonably large, representative sample. Representativeness protects against bias, while sample size helps reduce sampling variability.
Comparing evidence from different sample sizes
Two random samples are used to estimate the proportion of households that recycle food waste.
- Sample A contains 10 households, of which 7 recycle food waste.
- Sample B contains 500 households, of which 350 recycle food waste.
-
Both sample proportions are 70%, since 7/10=0.77/10=0.77/10=0.7 and 350/500=0.7350/500=0.7350/500=0.7.
-
In Sample A, changing the outcome for one household changes the proportion by 10 percentage points. In Sample B, changing one outcome changes it by only 0.2 percentage points.
-
If both samples were selected appropriately, Sample B gives stronger evidence that the population proportion is around 70% because its result is less sensitive to individual households and is likely to have less sampling variability.
Comparing samples
You may be given samples from two populations and asked what they suggest. Compare both location and spread.
A measure of location describes a typical or central value. Common measures are the mean and median.
A measure of spread describes how variable the data are. Common measures are the range, interquartile range and standard deviation.
When making an inference:
- use the median with the interquartile range;
- use the mean with the standard deviation;
- consider the sample sizes;
- notice outliers, skewness or substantial overlap;
- refer to the populations and the variable in context.
A higher sample mean or median suggests a higher typical population value, but it does not prove that every member of one population has a higher value than every member of the other.
Comparing delivery times
Two random samples of delivery times, measured in minutes, give the following summaries:
- Company A: sample size 80, median 32, interquartile range 8.
- Company B: sample size 75, median 38, interquartile range 15.
-
Compare location: Company A has the lower sample median, 32 minutes compared with 38 minutes. This suggests that Company A typically delivers more quickly.
-
Compare spread: Company A has the smaller interquartile range, 8 minutes compared with 15 minutes. Its middle 50% of delivery times are therefore less variable in the sample.
-
Since both samples are reasonably large and were selected randomly, it is reasonable to infer that Company A's population of delivery times is likely to have a lower typical value and less variability than Company B's.
-
The evidence does not show that every Company A delivery is faster than every Company B delivery. The two distributions could overlap considerably.
Make conclusions contextual
Do not finish with only “A is lower than B”. Name the population and variable: for example, “The sample suggests that Company A has a lower typical delivery time.”
Judging how strong the evidence is
The size of an observed difference matters alongside the amount of variation.
A small difference between sample means is less convincing when the data have a large spread. A large difference, seen in sizeable representative samples with relatively little spread, gives stronger informal evidence of a genuine population difference.
Outliers also matter. A single extreme value can have a noticeable effect on the mean and standard deviation, especially in a small sample. The median and interquartile range are more resistant to extreme values.
Judging a difference between sample means
Samples from two machines give:
- Machine P: mean mass 250.4 g, standard deviation 9.8 g.
- Machine Q: mean mass 251.1 g, standard deviation 10.2 g.
-
The difference between the sample means is only 0.7 g.
-
Both standard deviations are around 10 g, so individual masses vary much more than the difference between the means.
-
Without further evidence, the samples give only weak informal evidence that the machines have different population mean masses. The observed difference could plausibly be due to sampling variability.
Informal does not mean unsupported
Your conclusion must still be based on the sample statistics, graphs, sample sizes and sampling method. Personal expectations about the context are not statistical evidence.
Writing a balanced conclusion
A strong conclusion usually follows this pattern:
- State what the sample shows.
- Extend this cautiously to the population.
- Mention an important limitation where relevant.
Useful phrases include:
- “The sample suggests that…”
- “There is some evidence that…”
- “This may indicate that…”
- “The inference is limited because…”
- “The difference could be due to sampling variability.”
Avoid words such as proves, definitely, always and all unless the evidence genuinely supports them.
In the exam
- Identify the population, the sample and the variable being measured before drawing a conclusion.
- Compare appropriate measures of location and spread, and use the sample sizes or sampling method when judging reliability.
- Write your conclusion in context using cautious language such as “suggests” or “provides evidence”, and do not generalise beyond the population sampled.
- Check for bias, outliers, large variability and overlap before deciding how strong the evidence is.
Check yourself
- Why can two random samples from the same population produce different sample means?
- Why might a large voluntary-response sample still give a misleading inference?
- What features would you compare when deciding whether one population appears to have higher and more variable values than another?
