What you'll learn
- How a population, census, sample and sampling frame are connected.
- How to select samples using random, systematic, stratified, quota and opportunity sampling.
- How to judge whether a sampling method is suitable in context.
- Why different samples from the same population can produce different conclusions.
Populations, censuses and samples
Statistical investigations begin with a question about a group of individuals or items.
Population
The population is the complete set of people or items that you want to draw conclusions about. Each individual person or item is called a sampling unit.
For example, if a college wants to estimate how long its students spend travelling to college, the population is all students at that college. A sampling unit is one student.
A census collects information from every member of the population. A sample collects information from only part of the population.
Census or sample?
A census can give detailed information about the whole population and avoids uncertainty caused by choosing a sample. However, it may be expensive, slow or impractical. It can also become outdated before all the data have been collected.
A sample is usually quicker and cheaper. It may even be essential when testing destroys the item, such as testing the lifetime of a light bulb. However, conclusions based on a sample are uncertain because the chosen sample may not represent the population accurately.
Representativeness
A representative sample reflects the relevant characteristics of the population. Increasing the sample size can reduce natural variation, but it does not automatically remove bias caused by a poor sampling method.
Choosing between a census and a sample
A manufacturer wants to estimate the breaking strength of a batch of 20,000 safety cables. Testing a cable until it breaks destroys it.
- A census would require all 20,000 cables to be broken, leaving none available for sale.
- A sample allows the manufacturer to test a manageable number of cables while preserving most of the batch.
- The sample should be selected from throughout the batch so that it is not concentrated in one production period. A well-chosen sample is therefore the appropriate method.
Sampling frames
Sampling frame
A sampling frame is a list or collection containing the sampling units from which a sample is selected.
A college register could be a sampling frame for students. A list of customer account numbers could be a sampling frame for customers.
A good sampling frame should:
- include every member of the target population;
- include each member only once;
- exclude people or items outside the population;
- be accurate and up to date.
If the sampling frame is incomplete, even a perfectly random selection from it may not represent the target population.
Confusing the population and sampling frame
The population is the whole group you want to study. The sampling frame is the list from which you actually select. These may differ: a telephone directory, for example, does not contain every resident.
Simple random sampling
Simple random sample
A simple random sample of size nnn is selected so that every possible group of nnn population members has an equal chance of being chosen.
A practical method is to assign a different number to each member of the sampling frame and use a random number generator. Repeated numbers and numbers outside the required range are ignored.
Strengths and limitations
Simple random sampling reduces deliberate selection bias and supports valid statistical analysis. However, it requires a complete sampling frame and may be inconvenient when the population is large or geographically scattered.
Selecting a simple random sample
A school has 800 pupils and wants a random sample of 50.
- Number the pupils uniquely from 001 to 800 using the school register as the sampling frame.
- Generate three-digit random numbers and select the pupil attached to each valid number.
- Ignore 000, numbers from 801 to 999 and any repeated number. Continue until 50 different pupils have been selected.
Describe the randomisation precisely
Do not merely write “choose randomly”. State how units are numbered, how random numbers are obtained and how invalid or repeated values are handled.
Systematic sampling
Systematic sample
A systematic sample selects units at regular intervals from an ordered sampling frame, beginning from a random starting position.
If the population size is NNN and the required sample size is nnn, the sampling interval is approximately
k=Nn.k=\frac{N}{n}.k=nN.When kkk is a whole number, choose a random start from the first kkk positions and then select every kkkth unit.
Taking a systematic sample
A company has 1,200 employees and requires a sample of 100.
- Calculate the interval:
- Randomly choose a starting position from 1 to 12. Suppose the random start is 7.
- Select positions 7, 19, 31, 43 and so on, adding 12 each time, until 100 employees have been selected.
Systematic sampling is quick and spreads the sample across the frame. However, it can be biased if the ordering contains a repeating pattern related to the interval.
Periodicity
Periodicity is a repeating pattern in an ordered list. If every twelfth employee works in the same department, selecting every twelfth name could produce a seriously unrepresentative sample.
Stratified sampling
A population often contains recognisable groups, such as year groups, age bands or departments.
Stratified sample
In stratified sampling, the population is divided into non-overlapping groups called strata. A random sample is taken from every stratum, usually in proportion to its size.
If a stratum contains SSS members, the population contains NNN members and the total sample size is nnn, then
number sampled from the stratum=SN×n.\text{number sampled from the stratum}=\frac{S}{N}\times n.number sampled from the stratum=NS×n.The result may need to be rounded. The final rounded allocations must add to the required sample size.
Allocating a stratified sample
A sixth form contains 360 Year 12 students and 240 Year 13 students. A sample of 50 students is required.
- Find the total population:
- Calculate the Year 12 allocation:
- Calculate the Year 13 allocation:
- Use random sampling within each year group to select 30 Year 12 students and 20 Year 13 students.
Stratified sampling guarantees representation from each chosen stratum and can improve precision when the strata differ meaningfully. It requires accurate information about group membership, and it may take longer to organise.
Stopping after the allocation
Calculating how many people are needed from each stratum is not the complete method. The required people should then be chosen randomly within each stratum.
Quota and opportunity sampling
Quota sampling
In quota sampling, the population is divided into categories and an interviewer selects people until a target number, called a quota, has been reached for each category.
Quota sampling is quick, relatively cheap and does not require a complete sampling frame. It can ensure that specified categories are represented.
However, selection within each category is not random. An interviewer may approach people who look friendly, are easy to reach or appear likely to agree. This can create interviewer bias.
Opportunity sampling
An opportunity sample, also called a convenience sample, uses people or items that are readily available.
For example, a student might survey the first 30 people entering the college library. This is fast and easy, but library users at that time may differ from the wider college population.
Random and non-random methods
Simple random, systematic and stratified sampling use random selection when carried out correctly. Quota and opportunity sampling do not, so it is harder to justify generalising their results to the whole population.
Bias and non-response
Bias
Bias is a systematic tendency for a sampling or data-collection method to favour certain outcomes, causing results to differ consistently from the true population pattern.
Possible causes include:
- undercoverage, where some population groups are absent or poorly represented in the sampling frame;
- non-response, where selected individuals do not provide data;
- selecting people at a particular place or time;
- allowing interviewers or participants to choose who takes part;
- wording questions in a leading way.
Non-response can cause bias if the people who respond differ from those who do not. A high response rate is useful, but it does not prove that the sample is unbiased.
Critiquing a sampling method
A gym surveys people leaving its weekday morning exercise classes to estimate the weekly exercise time of all adults in the town.
- The target population is all adults in the town, but the available participants are morning gym users.
- People who do not attend the gym, work during the morning or exercise elsewhere have little or no chance of selection.
- Morning class users are likely to exercise more than a typical adult, so the estimate is likely to be biased upwards.
- A stronger approach would use an up-to-date frame of town residents and take a random sample, possibly stratified by relevant characteristics such as age.
Why different samples give different conclusions
Even when sampling is random and unbiased, different samples usually contain different individuals. Their calculated statistics, such as the sample mean or sample proportion, therefore vary.
This natural variation is called sampling variability. A value calculated from a sample is an estimate of the corresponding population value, not a guaranteed exact answer.
Comparing conclusions from two samples
Two random samples of 20 students are used to estimate support for a proposed timetable change. In sample A, 13 students support it; in sample B, 9 support it.
- The sample A proportion is
so this sample suggests 65% support.
- The sample B proportion is
so this sample suggests 45% support.
- Sample A suggests a majority supports the change, whereas sample B suggests a majority does not. Neither calculation is incorrect: the samples contain different students.
- The difference shows why conclusions should acknowledge sampling variability. A larger, representative sample would generally give a more stable estimate.
Treating variation as proof of bias
Different random samples can give different results without either method being biased. Bias is a systematic problem; sampling variability is the natural result of observing only part of the population.
Selecting and critiquing a method
There is no single best sampling method for every investigation. In context, consider:
- whether a complete and reliable sampling frame exists;
- whether important subgroups need guaranteed representation;
- the time, cost and geographical spread involved;
- whether the ordering of a frame contains a pattern;
- whether selection is genuinely random;
- the likely level and direction of any bias;
- whether non-response could alter the conclusion;
- whether the sample is large enough for a reasonably stable estimate.
Your criticism should be specific. Rather than writing “the sample is biased”, identify who is overrepresented or underrepresented and explain how that could affect the result.
In the exam
- Identify the target population, the sampling frame and the sampling units before judging the method.
- For a stratified sample, use SN×n\frac{S}{N}\times nNS×n, check rounding and state that selection within each stratum is random.
- When criticising a method, name the missing or overrepresented group and explain the likely effect on the conclusion.
- Remember that two well-selected samples may still produce different statistics because of sampling variability.
Check yourself
- What is the difference between a population, a sampling frame and a sample?
- How would you select a systematic sample of 80 people from an ordered list of 1,600?
- Why can two unbiased random samples lead to different conclusions about the same population?
