2.5.4 Sample mean as a random variable (A-level only)

What you'll learn

How a sample mean changes when a different random sample is selected.
Why the sample mean $X‾\overline{X}$ can itself be treated as a random variable.
How to find the mean and variance of $X‾\overline{X}$ .
How this idea supports statistical hypothesis testing.

From a population to a sample

A population is the complete set of individuals or observations that you want to investigate. A sample is a smaller collection selected from that population.

Suppose a population measurement is represented by the random variable $X$ , with:

E⁡(X)=μandVar⁡(X)=σ2.\operatorname{E}(X)=\mu \qquad\text{and}\qquad \operatorname{Var}(X)=\sigma^2.

Here, $μ\mu$ is the population mean and $σ2\sigma^2$ is the population variance.

If you take a random sample of size $n$ , its observations can be represented by the random variables

X1,X2,…,Xn.X_1,X_2,\ldots,X_n.

Unless the context says otherwise, these observations are normally assumed to be independent and to have the same distribution as $X$ .

Definition

Independent observations

Observations are independent when knowing the value of one observation does not give you information about the values of the others.

The sample mean

The sample mean is found by adding all the observations in a sample and dividing by the sample size.

Definition

Sample mean

For a random sample $X1,X2,…,XnX_1,X_2,\ldots,X_n$ , the sample mean is

X‾=X1+X2+⋯+Xnn=1n∑i=1nXi.\overline{X}=\frac{X_1+X_2+\cdots+X_n}{n} =\frac{1}{n}\sum_{i=1}^{n}X_i.

If you have already collected a particular sample, its observed mean is often written using lower-case notation, $x‾\overline{x}$ .

For example, if the observed sample is 4, 7, 8 and 9, then

x‾=4+7+8+94=7.\overline{x}=\frac{4+7+8+9}{4}=7.

The value 7 is fixed once that particular sample has been observed. Before selecting the sample, however, you do not know which observations you will obtain.

Why the sample mean is random

Imagine repeatedly taking random samples of the same size from a population. Different samples will usually contain different observations, so they will usually have different sample means.

Therefore, before a sample is selected, its mean is not a known fixed number. It has a range of possible values, each with an associated probability. This means that $X‾\overline{X}$ is a random variable.

Definition

Sample mean as a random variable

$X‾\overline{X}$ is the random variable whose value is the mean of a randomly selected sample of size $n$ . Its probability distribution is called the sampling distribution of the sample mean.

A population distribution leading to repeated random samples and then a narrower sampling distribution of the sample mean

Key Idea

What is random?

The formula for the mean does not change. The value of the mean is random because the observations included in the sample are random.

Example

Finding the possible sample means

A population consists of the values 2, 4 and 6. Two values are selected independently with replacement.

There are nine equally likely ordered samples:
$(2, 2), (2, 4), (2, 6), (4, 2), (4, 4), (4, 6), (6, 2), (6, 4), (6, 6) .$
Calculating the mean of each sample gives the possible values 2, 3, 4, 5 and 6. For example,
$2+42=3and4+62=5.\frac{2+4}{2}=3 \qquad\text{and}\qquad \frac{4+6}{2}=5.$
Counting how often each mean occurs gives
$P⁡(X‾=2)=19,P⁡(X‾=3)=29,P⁡(X‾=4)=39,P⁡(X‾=5)=29,P⁡(X‾=6)=19.\begin{aligned} \operatorname{P}(\overline{X}=2)&=\frac{1}{9},\\ \operatorname{P}(\overline{X}=3)&=\frac{2}{9},\\ \operatorname{P}(\overline{X}=4)&=\frac{3}{9},\\ \operatorname{P}(\overline{X}=5)&=\frac{2}{9},\\ \operatorname{P}(\overline{X}=6)&=\frac{1}{9}. \end{aligned}$
These probabilities form the sampling distribution of $X‾\overline{X}$ . A single selected sample produces one observed value, but $X‾\overline{X}$ describes all the possible sample means before selection.

The expected value of the sample mean

The expected value of $X‾\overline{X}$ is the population mean:

E⁡(X‾)=μ.\operatorname{E}(\overline{X})=\mu.

To see why, use the linearity of expectation:

E⁡(X‾)=E⁡(X1+⋯+Xnn)=1n(E⁡(X1)+⋯+E⁡(Xn))=1n(nμ)=μ.\begin{aligned} \operatorname{E}(\overline{X}) &=\operatorname{E}\left(\frac{X_1+\cdots+X_n}{n}\right)\\ &=\frac{1}{n}\left(\operatorname{E}(X_1)+\cdots+\operatorname{E}(X_n)\right)\\ &=\frac{1}{n}(n\mu)\\ &=\mu. \end{aligned}

This makes the sample mean an unbiased estimator of the population mean.

Definition

Unbiased estimator

An estimator is unbiased when its expected value equals the population parameter being estimated. Since $E⁡(X‾)=μ\operatorname{E}(\overline{X})=\mu$ , the sample mean is an unbiased estimator of $μ\mu$ .

Being unbiased does not mean every sample mean equals $μ\mu$ . It means that, over many repeated samples, the sample means are centred on $μ\mu$ .

The variance of the sample mean

For independent observations, variances add. Therefore,

Var⁡(X‾)=Var⁡(X1+⋯+Xnn)=1n2(Var⁡(X1)+⋯+Var⁡(Xn))=1n2(nσ2)=σ2n.\begin{aligned} \operatorname{Var}(\overline{X}) &=\operatorname{Var}\left(\frac{X_1+\cdots+X_n}{n}\right)\\ &=\frac{1}{n^2}\left(\operatorname{Var}(X_1)+\cdots+\operatorname{Var}(X_n)\right)\\ &=\frac{1}{n^2}(n\sigma^2)\\ &=\frac{\sigma^2}{n}. \end{aligned}

The standard deviation of $X‾\overline{X}$ is therefore

Var⁡(X‾)=σn.\sqrt{\operatorname{Var}(\overline{X})} =\frac{\sigma}{\sqrt{n}}.

This standard deviation is called the standard error of the sample mean.

Key Idea

Effect of sample size

As the sample size increases, the sampling distribution becomes less spread out. Larger samples tend to produce means closer to the population mean.

Example

Comparing the spread of sample means

A population has mean $μ=50\mu=50$ and standard deviation $σ=12\sigma=12$ . Consider independent random samples of sizes 9 and 36.

For samples with $n = 9$ , the standard error is
$σn=129=4.\frac{\sigma}{\sqrt{n}} =\frac{12}{\sqrt{9}} =4.$
For samples with $n = 36$ , the standard error is
$1236=2.\frac{12}{\sqrt{36}} =2.$
Both sampling distributions have expected value 50, but the means from samples of size 36 are less variable. Multiplying the sample size by 4 has divided the standard error by 2.

Common Mistake

Dividing the variance incorrectly

The variance is $σ2n\frac{\sigma^2}{n}$ , while the standard deviation is $σn\frac{\sigma}{\sqrt{n}}$ . Do not use $σn\frac{\sigma}{n}$ as the standard deviation.

Common Mistake

Independence matters

The calculation $Var⁡(X‾)=σ2n\operatorname{Var}(\overline{X})=\frac{\sigma^2}{n}$ assumes independent observations with common variance $σ2\sigma^2$ . Sampling without replacement from a small finite population can make the observations dependent.

Connection to hypothesis testing

In a hypothesis test about a population mean, the null hypothesis proposes a value for $μ\mu$ . You then consider how the random variable $X‾\overline{X}$ should behave if that proposed value is correct.

An observed sample gives a particular value $x‾\overline{x}$ . If this value lies unusually far into a tail of the sampling distribution predicted by the null hypothesis, it provides evidence against the null hypothesis.

Tip

Keep the notation distinct

Use $X‾\overline{X}$ for the random variable representing all possible sample means, and $x‾\overline{x}$ for the numerical mean obtained from the sample actually observed.

Exam technique

In the exam

State that $X‾\overline{X}$ is random because different random samples can produce different sample means.
Use $E⁡(X‾)=μ\operatorname{E}(\overline{X})=\mu$ and, for independent observations, $Var⁡(X‾)=σ2n\operatorname{Var}(\overline{X})=\frac{\sigma^2}{n}$ .
Check whether the question asks for variance or standard deviation: the standard deviation is $σn\frac{\sigma}{\sqrt{n}}$ .
Distinguish the random variable $X‾\overline{X}$ from an observed value $x‾\overline{x}$ .

Self review

Check yourself

Why can $X‾\overline{X}$ be a random variable even though it is calculated using a fixed formula?
A population has standard deviation 15. What is the standard error of the mean for samples of size 25?
What happens to the expected value and variance of $X‾\overline{X}$ when the sample size increases?

What you'll learn

How a sample mean changes when a different random sample is selected.
Why the sample mean $X‾\overline{X}$ can itself be treated as a random variable.
How to find the mean and variance of $X‾\overline{X}$ .
How this idea supports statistical hypothesis testing.

From a population to a sample

A population is the complete set of individuals or observations that you want to investigate. A sample is a smaller collection selected from that population.

Suppose a population measurement is represented by the random variable $X$ , with:

E⁡(X)=μandVar⁡(X)=σ2.\operatorname{E}(X)=\mu \qquad\text{and}\qquad \operatorname{Var}(X)=\sigma^2.

Here, $μ\mu$ is the population mean and $σ2\sigma^2$ is the population variance.

If you take a random sample of size $n$ , its observations can be represented by the random variables

X1,X2,…,Xn.X_1,X_2,\ldots,X_n.

Unless the context says otherwise, these observations are normally assumed to be independent and to have the same distribution as $X$ .

Definition

Independent observations

Observations are independent when knowing the value of one observation does not give you information about the values of the others.

The sample mean

The sample mean is found by adding all the observations in a sample and dividing by the sample size.

Definition

Sample mean

For a random sample $X1,X2,…,XnX_1,X_2,\ldots,X_n$ , the sample mean is

X‾=X1+X2+⋯+Xnn=1n∑i=1nXi.\overline{X}=\frac{X_1+X_2+\cdots+X_n}{n} =\frac{1}{n}\sum_{i=1}^{n}X_i.

If you have already collected a particular sample, its observed mean is often written using lower-case notation, $x‾\overline{x}$ .

For example, if the observed sample is 4, 7, 8 and 9, then

x‾=4+7+8+94=7.\overline{x}=\frac{4+7+8+9}{4}=7.

The value 7 is fixed once that particular sample has been observed. Before selecting the sample, however, you do not know which observations you will obtain.

Why the sample mean is random

Imagine repeatedly taking random samples of the same size from a population. Different samples will usually contain different observations, so they will usually have different sample means.

Definition

Sample mean as a random variable

$X‾\overline{X}$ is the random variable whose value is the mean of a randomly selected sample of size $n$ . Its probability distribution is called the sampling distribution of the sample mean.

A population distribution leading to repeated random samples and then a narrower sampling distribution of the sample mean

Key Idea

What is random?

The formula for the mean does not change. The value of the mean is random because the observations included in the sample are random.

Example

Finding the possible sample means

A population consists of the values 2, 4 and 6. Two values are selected independently with replacement.

There are nine equally likely ordered samples:
$(2, 2), (2, 4), (2, 6), (4, 2), (4, 4), (4, 6), (6, 2), (6, 4), (6, 6) .$
Calculating the mean of each sample gives the possible values 2, 3, 4, 5 and 6. For example,
$2+42=3and4+62=5.\frac{2+4}{2}=3 \qquad\text{and}\qquad \frac{4+6}{2}=5.$
Counting how often each mean occurs gives
$P⁡(X‾=2)=19,P⁡(X‾=3)=29,P⁡(X‾=4)=39,P⁡(X‾=5)=29,P⁡(X‾=6)=19.\begin{aligned} \operatorname{P}(\overline{X}=2)&=\frac{1}{9},\\ \operatorname{P}(\overline{X}=3)&=\frac{2}{9},\\ \operatorname{P}(\overline{X}=4)&=\frac{3}{9},\\ \operatorname{P}(\overline{X}=5)&=\frac{2}{9},\\ \operatorname{P}(\overline{X}=6)&=\frac{1}{9}. \end{aligned}$
These probabilities form the sampling distribution of $X‾\overline{X}$ . A single selected sample produces one observed value, but $X‾\overline{X}$ describes all the possible sample means before selection.

The expected value of the sample mean

The expected value of $X‾\overline{X}$ is the population mean:

E⁡(X‾)=μ.\operatorname{E}(\overline{X})=\mu.

To see why, use the linearity of expectation:

E⁡(X‾)=E⁡(X1+⋯+Xnn)=1n(E⁡(X1)+⋯+E⁡(Xn))=1n(nμ)=μ.\begin{aligned} \operatorname{E}(\overline{X}) &=\operatorname{E}\left(\frac{X_1+\cdots+X_n}{n}\right)\\ &=\frac{1}{n}\left(\operatorname{E}(X_1)+\cdots+\operatorname{E}(X_n)\right)\\ &=\frac{1}{n}(n\mu)\\ &=\mu. \end{aligned}

This makes the sample mean an unbiased estimator of the population mean.

Definition

Unbiased estimator

Being unbiased does not mean every sample mean equals $μ\mu$ . It means that, over many repeated samples, the sample means are centred on $μ\mu$ .

The variance of the sample mean

For independent observations, variances add. Therefore,

Var⁡(X‾)=Var⁡(X1+⋯+Xnn)=1n2(Var⁡(X1)+⋯+Var⁡(Xn))=1n2(nσ2)=σ2n.\begin{aligned} \operatorname{Var}(\overline{X}) &=\operatorname{Var}\left(\frac{X_1+\cdots+X_n}{n}\right)\\ &=\frac{1}{n^2}\left(\operatorname{Var}(X_1)+\cdots+\operatorname{Var}(X_n)\right)\\ &=\frac{1}{n^2}(n\sigma^2)\\ &=\frac{\sigma^2}{n}. \end{aligned}

The standard deviation of $X‾\overline{X}$ is therefore

Var⁡(X‾)=σn.\sqrt{\operatorname{Var}(\overline{X})} =\frac{\sigma}{\sqrt{n}}.

This standard deviation is called the standard error of the sample mean.

Key Idea

Effect of sample size

As the sample size increases, the sampling distribution becomes less spread out. Larger samples tend to produce means closer to the population mean.

Example

Comparing the spread of sample means

A population has mean $μ=50\mu=50$ and standard deviation $σ=12\sigma=12$ . Consider independent random samples of sizes 9 and 36.

For samples with $n = 9$ , the standard error is
$σn=129=4.\frac{\sigma}{\sqrt{n}} =\frac{12}{\sqrt{9}} =4.$
For samples with $n = 36$ , the standard error is
$1236=2.\frac{12}{\sqrt{36}} =2.$
Both sampling distributions have expected value 50, but the means from samples of size 36 are less variable. Multiplying the sample size by 4 has divided the standard error by 2.

Common Mistake

Dividing the variance incorrectly

The variance is $σ2n\frac{\sigma^2}{n}$ , while the standard deviation is $σn\frac{\sigma}{\sqrt{n}}$ . Do not use $σn\frac{\sigma}{n}$ as the standard deviation.

Common Mistake

Independence matters

Connection to hypothesis testing

Tip

Keep the notation distinct

Use $X‾\overline{X}$ for the random variable representing all possible sample means, and $x‾\overline{x}$ for the numerical mean obtained from the sample actually observed.

Exam technique

In the exam

State that $X‾\overline{X}$ is random because different random samples can produce different sample means.
Use $E⁡(X‾)=μ\operatorname{E}(\overline{X})=\mu$ and, for independent observations, $Var⁡(X‾)=σ2n\operatorname{Var}(\overline{X})=\frac{\sigma^2}{n}$ .
Check whether the question asks for variance or standard deviation: the standard deviation is $σn\frac{\sigma}{\sqrt{n}}$ .
Distinguish the random variable $X‾\overline{X}$ from an observed value $x‾\overline{x}$ .

Self review

Check yourself

Why can $X‾\overline{X}$ be a random variable even though it is calculated using a fixed formula?
A population has standard deviation 15. What is the standard error of the mean for samples of size 25?
What happens to the expected value and variance of $X‾\overline{X}$ when the sample size increases?

What you'll learn

From a population to a sample

The sample mean

Why the sample mean is random

The expected value of the sample mean

The variance of the sample mean

Connection to hypothesis testing

Review 2.5.4 Sample mean as a random variable (A-level only) by teaching Genie

2.5.4 Sample mean as a random variable (A-level only)

What you'll learn

From a population to a sample

The sample mean

Why the sample mean is random

The expected value of the sample mean

The variance of the sample mean

Connection to hypothesis testing

Review 2.5.4 Sample mean as a random variable (A-level only) by teaching Genie

2.5.4 Sample mean as a random variable (A-level only)

What you'll learn

From a population to a sample

The sample mean

Why the sample mean is random

The expected value of the sample mean

The variance of the sample mean

Connection to hypothesis testing

Review 2.5.4 Sample mean as a random variable (A-level only) by teaching Genie

2.5.4 Sample mean as a random variable (A-level only) Revision Guide

2.5.4 Sample mean as a random variable (A-level only)

What you'll learn

From a population to a sample

The sample mean

Why the sample mean is random

The expected value of the sample mean

The variance of the sample mean

Connection to hypothesis testing

Review 2.5.4 Sample mean as a random variable (A-level only) by teaching Genie

2.5.4 Sample mean as a random variable (A-level only) Revision Guide