What you'll learn
- How to choose a sampling technique that suits the population and statistical problem.
- How to compare random, systematic, stratified, quota and opportunity sampling.
- How to identify bias, practical limitations and problems with a sampling frame.
- Why different samples from the same population can produce different conclusions.
Population, census and sample
A statistical investigation begins with a question about a population. The population does not have to be a group of people: it could be all light bulbs made by a factory, all train journeys on a route, or all trees in a forest.
Population and sample
The population is the complete set of individuals or items being studied. A sample is a subset of the population from which data are collected.
A census collects data from every member of the population. This can provide very detailed information, but it may be expensive, slow or impossible. For example, testing the lifetime of every bulb made by a factory would leave no bulbs to sell.
Sampling is usually quicker and cheaper. However, conclusions based on a sample are uncertain because the sample may not perfectly reflect the population.
Representative sample
A representative sample has relevant characteristics in approximately the same proportions as the population. A sample that systematically over-represents or under-represents part of the population is biased.
Choosing between a census and a sample
A college wants to estimate the mean time taken by its 1800 students to travel to college.
- A census would require travel-time data from all 1800 students, which could take considerable time to collect and check.
- Travel time is not being measured using a destructive test, so a census is possible, but it is unlikely to be necessary for an estimate.
- A suitably selected sample would be faster and cheaper. It should include students from different year groups and study programmes because these may be related to travel arrangements.
- Therefore, a representative sample is more practical than a census for this investigation.
Sampling frames and sampling units
Sampling frame
A sampling frame is a list or other record of all the members of the population from which a sample can be selected. Each individual member of the population is called a sampling unit.
For example, a school register could be a sampling frame when the sampling units are students. A good sampling frame should be complete, up to date and free from duplicates.
If the frame excludes part of the population, that part cannot be selected. This creates undercoverage. An outdated customer list, for instance, may exclude recent customers and include people who are no longer customers.
Assuming a list is automatically suitable
A sampling frame can still produce bias if it is incomplete, outdated, contains duplicates or does not match the target population.
Random sampling
A sampling method is random if chance determines which population members are selected. Random methods reduce the researcher's opportunity to favour particular individuals.
Simple random sampling
Simple random sample
In a simple random sample, every possible sample of the required size has an equal chance of being selected.
One method is to number every member of the population and use a random number generator to choose distinct numbers. A complete sampling frame is needed.
Advantages include reduced selection bias and the ability to use probability-based statistical methods. Disadvantages include the need for a suitable sampling frame and the practical difficulty of contacting widely scattered individuals.
Selecting a simple random sample
A school wants a simple random sample of 40 students from a population of 1200 students.
- Assign each student a unique number from 1 to 1200 using the school register as the sampling frame.
- Use a random number generator to produce 40 distinct integers in this range. Repeated numbers must be ignored because the same student should not be selected twice.
- Contact the students corresponding to those numbers. Selection is controlled by chance rather than by staff choosing students who are easy to reach.
Systematic sampling
Systematic sample
In a systematic sample, sampling units are chosen at regular intervals from an ordered sampling frame, after selecting a random starting point.
For a population of size NNN and a required sample of size nnn, the approximate sampling interval is
k=Nn.k=\frac{N}{n}.k=nN.Systematic sampling is straightforward and spreads the sample through the frame. However, it can be biased if the ordering of the frame has a repeating pattern that matches the interval.
Using a systematic sample
A company has 2000 employees on an alphabetical list and wants a sample of 100.
- Calculate the interval:
- Randomly choose a starting position from 1 to 20, such as 7.
- Select employees numbered 7, 27, 47, and so on, adding 20 each time until 100 employees have been selected.
- The sample is distributed across the list, but the company should check that the list's ordering does not contain a relevant repeating pattern.
Stratified sampling
A population can often be divided into meaningful groups, such as year groups, geographical regions or age bands. These groups are called strata.
Stratified sample
In a stratified sample, the population is divided into non-overlapping strata and a random sample is taken from each stratum, usually in proportion to its size.
If a stratum contains SSS members, the population contains NNN members and the total sample size is nnn, its sample allocation is
SN×n.\frac{S}{N}\times n.NS×n.Stratified sampling ensures that all chosen groups are represented. It is especially useful when the groups may respond differently. However, it requires accurate information about every population member's stratum and is more complicated than simple random sampling.
Allocating a stratified sample
A sixth form has 320 Year 12 students and 280 Year 13 students. A stratified sample of 90 students is required.
- The total population is N=320+280=600N=320+280=600N=320+280=600.
- The Year 12 allocation is
- The Year 13 allocation is
- Randomly select 48 Year 12 students and 42 Year 13 students from the relevant registers. This preserves the year-group proportions of the population.
Dealing with rounding
If proportional allocations are not whole numbers, round them while checking that the final allocations still add to the required total sample size.
Non-random sampling
Non-random methods can be useful when there is no sampling frame or when data must be collected quickly. Their main weakness is that selection probabilities are unknown, so bias is harder to control.
Quota sampling
Quota sample
In quota sampling, the population is divided into categories and an interviewer selects people until a fixed quota for each category has been reached.
Quota sampling can make the sample resemble the population in selected characteristics without requiring a full sampling frame. However, the interviewer usually chooses who to approach, creating selection bias. Matching age or sex proportions does not guarantee that the sample is representative in other ways.
Opportunity sampling
Opportunity sample
An opportunity sample, also called a convenience sample, consists of individuals who are readily available.
It is quick and inexpensive, but often strongly biased. For example, surveying people outside a gym about weekly exercise is likely to overestimate exercise levels in the wider town.
Voluntary response sampling
In a voluntary response sample, individuals choose whether to participate, such as by responding to an online poll. People with strong opinions may be more likely to respond, producing self-selection bias.
Critiquing a convenience sample
A council asks shoppers leaving one city-centre supermarket whether local bus services should receive more funding.
- The target population might be all local residents, but the sample contains only people visiting one supermarket at a particular time.
- Residents who shop elsewhere, shop online or cannot reach the city centre are under-represented.
- Shoppers at that location may use buses at a different rate from the wider population, so the method could bias the estimated level of support.
- A better approach would use a random sample from a suitable resident database, possibly stratified by area or age.
Choosing an appropriate technique
There is no single best sampling technique for every investigation. Your choice should depend on:
- the target population and whether it is clearly defined;
- whether a complete sampling frame is available;
- whether important subgroups need guaranteed representation;
- the likely sources of bias;
- the time, cost and ease of contacting selected individuals;
- whether non-response is likely;
- the precision required from the results.
Justify choices in context
A strong justification links the sampling method to the actual investigation. Do not merely say that a method is “better”; explain which source of bias it reduces or which practical limitation it addresses.
A large sample is not automatically representative. Increasing the size of a biased sample usually gives a more precise estimate of the wrong quantity. The method of selection matters as well as sample size.
Different samples can give different conclusions
The value calculated from a sample, such as a sample mean or sample proportion, is called a statistic. A corresponding numerical feature of the whole population is called a parameter.
Different random samples usually contain different individuals, so they produce different statistics. This natural variation is called sampling variability.
Sampling variability
Sampling variability is the variation in a statistic caused by selecting different samples from the same population.
Larger random samples generally have less sampling variability, but they do not remove it completely. They also do not correct bias caused by a poor selection method.
Comparing conclusions from two samples
Two random samples of 50 passengers are asked whether a train service is reliable. In one sample, 31 passengers agree; in the other, 38 agree.
- The first sample proportion is
so 62% agree.
- The second sample proportion is
so 76% agree.
- The samples give noticeably different estimates because they contain different passengers. Neither result proves that the true population proportion equals the sample proportion.
- This difference may be ordinary sampling variability. Before claiming that opinions have changed, you would need to consider sample size, selection method and whether both samples came from the same target population.
Treating a sample result as exact
A sample statistic is an estimate of a population parameter, not the exact population value. Your conclusion should reflect this uncertainty.
Critiquing a sampling method
When evaluating a method, identify a specific issue and explain its likely effect. Useful questions include:
- Does the sampling frame cover the entire target population?
- Did every relevant member have a reasonable chance of selection?
- Are important groups represented?
- Could the location or time of collection affect who is included?
- Could researcher choice, voluntary response or non-response create bias?
- Is the sample large enough to limit sampling variability?
- Can the method realistically be carried out?
In the exam
- Define the target population and check whether the proposed sampling frame matches it.
- Name the sampling method, then describe exactly how selection would be carried out in the given context.
- When critiquing, identify who is over-represented or excluded and explain the likely effect on the conclusion.
- Distinguish bias from sampling variability: bias is systematic, while sampling variability occurs because different samples contain different members.
- Remember that a larger sample usually reduces variability, but it does not repair a biased sampling method.
Check yourself
- When would stratified sampling be more suitable than simple random sampling?
- Why might a systematic sample be biased if the sampling frame contains a repeating pattern?
- How can two well-selected random samples from the same population lead to different conclusions?
