Regression, Correlation and Hypothesis Testing

What you'll learn

How to describe scatter diagrams and interpret correlation.
How to use regression lines sensibly, including gradients and variables.
How to test for correlation using the product moment correlation coefficient.
How logarithms turn some curved models into straight-line models.

The big picture: paired data

Definition

Bivariate data

Bivariate data means paired values collected from the same item, person, day or situation. For example, temperature and daily sales for the same day form one pair of data.

Usually we call the variable on the horizontal axis $x$ and the variable on the vertical axis $y$ .

The explanatory variable is the variable you use to help predict another variable. The response variable is the variable being predicted.

For example, if you are investigating whether warmer weather affects sales, temperature would normally be the explanatory variable, and sales would be the response variable.

Scatter diagrams and correlation

A scatter diagram plots each data pair as a point. It helps you judge whether a relationship looks:

Examples of scatter diagrams showing positive, negative, strong and weak linear correlation.

positive: as one variable increases, the other tends to increase;
negative: as one variable increases, the other tends to decrease;
strong: points lie close to a clear pattern;
weak: points are more spread out.

Definition

Correlation

Correlation describes the strength and direction of a relationship between two variables. In A-Level Statistics, we usually focus on linear correlation, meaning how closely the points follow a straight-line pattern.

Example

Describing a scatter diagram

A coach records the heights and weights of 12 athletes. A scatter diagram shows that taller athletes tend to be heavier, and the points are fairly close to an upward-sloping line.

Put height on the horizontal axis if height is being used to help explain weight.
Put weight on the vertical axis because it is the response variable.
Since the points rise from left to right, describe the relationship as positive correlation.
Since the points are fairly close to a straight-line pattern, say there is fairly strong positive linear correlation.

Common Mistake

Correlation is not causation

A correlation tells you that two variables tend to move together. It does not, by itself, prove that one variable causes the other.

The product moment correlation coefficient

The product moment correlation coefficient, usually called PMCC, gives a numerical measure of linear correlation.

A PMCC scale showing how values of r correspond to direction and strength of linear correlation.

Definition

PMCC

The PMCC, written $r$ , is a number between -1 and 1 calculated from sample data. Values close to 1 show strong positive linear correlation, values close to -1 show strong negative linear correlation, and values close to 0 show little or no linear correlation.

The population correlation coefficient is written $ρ\rho$ . This is important in hypothesis testing: $r$ comes from your sample, while $ρ\rho$ describes the whole population.

Key Idea

Reading $r$

The sign of $r$ gives the direction. The size of $∣ r ∣$ gives the strength. A value near 0 only means “no clear straight-line relationship”; there could still be a curved relationship.

Example

Interpreting a PMCC

An analyst records years of experience and time taken to complete a task for 12 employees. The PMCC is $r = - 0.47$ .

The value is negative, so the relationship is negative.
This suggests that employees with more experience tend to take less time.
Since $∣ r ∣ = 0.47$ , the relationship is not very strong.
A sensible interpretation is: there is weak to moderate negative linear correlation between experience and task time.

Tip

Quick PMCC checks

The PMCC has no units, and swapping the two variables does not change its value. A very high or very low value only measures how linear the relationship is.

Regression lines

A regression line is a straight-line model used to predict one variable from another.

A regression line of y on x with the explanatory variable on the horizontal axis and the response variable on the vertical axis.

Definition

Regression line of $y$ on $x$

The regression line of $y$ on $x$ is the straight line used to predict $y$ from $x$ . It is usually written in the form $y = a + b x$ , where $b$ is the gradient and $a$ is the intercept.

The gradient tells you the predicted change in the response variable for each one-unit increase in the explanatory variable.

The intercept is the predicted value of the response variable when the explanatory variable is 0. This is only meaningful if 0 makes sense in the context.

Example

Interpreting a regression equation

A café owner records daily temperature, $t$ °C, and daily sales, £ $S$ , for 10 summer days. A regression model is

S = 4800 + 135 t

and the PMCC is $r = 0.69$ .

Temperature is the explanatory variable because it is being used to predict sales.
Sales is the response variable because it is being predicted.
The gradient is 135, so the model predicts that sales increase by about £135 for each extra 1°C.
Since $r = 0.69$ , the positive linear correlation is moderately strong, so a linear model is reasonably consistent with the data.
The intercept 4800 means the model predicts sales of £4800 when the temperature is 0°C, but this may not be useful if the data were only collected in summer.

Common Mistake

Extrapolation

Extrapolation means using a model outside the range of the original data. Regression predictions become less reliable when you extrapolate.

Extrapolation uses the regression line beyond the observed range of x-values, where predictions are less reliable.

Hypothesis testing for correlation

A hypothesis test uses sample evidence to decide whether there is enough evidence to reject a starting assumption.

Definition

Hypotheses for correlation

For a PMCC test, the null hypothesis is usually $H0:ρ=0H_0:\rho=0$ , meaning no linear correlation in the population. The alternative hypothesis $H_1$ says what kind of correlation you are testing for.

Use:

$H1:ρ>0H_1:\rho>0$ for a positive correlation test;
$H1:ρ<0H_1:\rho<0$ for a negative correlation test;
$H1:ρ≠0H_1:\rho\neq 0$ for a two-tailed test for any correlation.

A critical value is the boundary value from the PMCC tables. You compare your sample value of $r$ with this value.

Critical regions for one-tailed and two-tailed PMCC hypothesis tests on the r-scale.

One-tailed test

Use a one-tailed test when the question asks for a positive or negative correlation.

Example

Testing for positive correlation

A takeaway owner records temperature and daily sales for 10 days. The PMCC is $r = 0.67$ . Test, at the 5% significance level, whether there is positive correlation.

State the hypotheses:
$H0:ρ=0,H1:ρ>0H_0:\rho=0,\qquad H_1:\rho>0$
There are 10 pairs of data, so $n = 10$ . For a one-tailed 5% test, the critical value is 0.549.
Compare the sample PMCC with the critical value:
$0.67 > 0.549$
Since the sample value is in the critical region, reject $H_0$ .
There is sufficient evidence at the 5% level of positive correlation between temperature and daily sales.

Two-tailed test

Use a two-tailed test when the question asks whether there is a correlation, without specifying positive or negative.

Example

Testing for any correlation

A scientist records humidity and air pressure on 14 days. The PMCC is $r = - 0.55$ . Test, at the 5% level, whether there is correlation.

State the hypotheses:
$H0:ρ=0,H1:ρ≠0H_0:\rho=0,\qquad H_1:\rho\neq 0$
There are 14 pairs of data, so $n = 14$ . For a two-tailed 5% test, the critical value is 0.532.
For a two-tailed test, compare $∣ r ∣$ with the critical value:
$∣ - 0.55∣ = 0.55 > 0.532$
Reject $H_0$ .
There is sufficient evidence at the 5% level of correlation between humidity and air pressure. Since $r$ is negative, the evidence suggests negative correlation.

Common Mistake

Wrong tail

If the wording says “positive correlation” or “negative correlation”, use a one-tailed test. If it just says “correlation”, use a two-tailed test.

Logarithms and non-linear regression models

Sometimes the original scatter diagram is curved, but a logarithmic transformation makes it straight.

A logarithmic transformation can turn a curved relationship in the original variables into a straight-line relationship.

A transformation means replacing a variable with a function of that variable, such as $log_{10}y$ .

Exponential models

An exponential model has the form

y=ab^x

Taking logs gives

log_{10}y=\log_{10}a+x\log_{10}b

So if the regression line for $Y=\log_{10}y$ is

How the intercept and gradient of a straight line in log form become the constants in an exponential model.

Y = A + B x

then

a=10A,b=10Ba=10^A,\qquad b=10^B

Example

Finding an exponential model

The temperature of water, $w$ °C, is recorded $t$ minutes after boiling. Using $Y=\log_{10}w$ , the regression line is

Y = 1.99 - 0.031 t

Find a model of the form $w=ab^t$ .

Compare the line with the exponential log form:
$log_{10}w=\log_{10}a+t\log_{10}b$
Identify the constants:
$log⁡10a=1.99,log⁡10b=−0.031\log_{10}a=1.99,\qquad \log_{10}b=-0.031$
Undo the logarithms:
$a=101.99≈97.7,b=10−0.031≈0.931a=10^{1.99}\approx 97.7,\qquad b=10^{-0.031}\approx 0.931$
The model is:
$w=97.7(0.931)^t$
The constant $a$ is the predicted temperature when $t = 0$ , so here it is about 97.7°C.

Power models

A power model has the form

y=ax^b

Taking logs gives

log_{10}y=\log_{10}a+b\log_{10}x

So if you set $X=\log_{10}x$ and $Y=\log_{10}y$ , a straight-line model supports a power relationship.

A log-log straight line supports a power model, with intercept determining a and gradient determining b.

Example

Converting a log model to a power model

For two variables $x$ and $y$ , let $m=\log_{10}x$ and $n=\log_{10}y$ . The regression line is

n = 0.25 - 0.16 m

Substitute back into the original variables:
$log_{10}y=0.25-0.16\log_{10}x$
Identify the power-model constants:
$log⁡10a=0.25,b=−0.16\log_{10}a=0.25,\qquad b=-0.16$
Undo the logarithm for $a$ :
$a=100.25≈1.78a=10^{0.25}\approx 1.78$
Write the model:
$y=1.78x^{-0.16}$
If the PMCC for the transformed variables is closer to -1 or 1 than the original PMCC, that supports using the transformed model.

Common Mistake

Forgetting the inverse log

If the intercept in a base-10 log equation is $A$ , then the original constant is $10^A$ , not just $A$ .

Exam technique

In the exam

Decide the test type from the wording: “positive” or “negative” means one-tailed; “correlation” means two-tailed.
Write hypotheses using $ρ\rho$ , not $r$ . Use $r$ only for the sample PMCC you compare with the critical value.
Give a conclusion in context using “sufficient evidence” or “insufficient evidence”; do not say the result proves causation.

Self review

Check yourself

Can you choose the correct alternative hypothesis for positive, negative and two-tailed correlation tests?
Can you interpret the gradient of a regression line in context, including the correct units?
Can you convert a line involving $log_{10}y$ into an exponential or power model?

Regression, Correlation and Hypothesis Testing

What you'll learn

How to describe scatter diagrams and interpret correlation.
How to use regression lines sensibly, including gradients and variables.
How to test for correlation using the product moment correlation coefficient.
How logarithms turn some curved models into straight-line models.

The big picture: paired data

Definition

Bivariate data

Bivariate data means paired values collected from the same item, person, day or situation. For example, temperature and daily sales for the same day form one pair of data.

Usually we call the variable on the horizontal axis $x$ and the variable on the vertical axis $y$ .

The explanatory variable is the variable you use to help predict another variable. The response variable is the variable being predicted.

For example, if you are investigating whether warmer weather affects sales, temperature would normally be the explanatory variable, and sales would be the response variable.

Scatter diagrams and correlation

A scatter diagram plots each data pair as a point. It helps you judge whether a relationship looks:

Examples of scatter diagrams showing positive, negative, strong and weak linear correlation.

positive: as one variable increases, the other tends to increase;
negative: as one variable increases, the other tends to decrease;
strong: points lie close to a clear pattern;
weak: points are more spread out.

Definition

Correlation

Example

Describing a scatter diagram

A coach records the heights and weights of 12 athletes. A scatter diagram shows that taller athletes tend to be heavier, and the points are fairly close to an upward-sloping line.

Put height on the horizontal axis if height is being used to help explain weight.
Put weight on the vertical axis because it is the response variable.
Since the points rise from left to right, describe the relationship as positive correlation.
Since the points are fairly close to a straight-line pattern, say there is fairly strong positive linear correlation.

Common Mistake

Correlation is not causation

A correlation tells you that two variables tend to move together. It does not, by itself, prove that one variable causes the other.

The product moment correlation coefficient

The product moment correlation coefficient, usually called PMCC, gives a numerical measure of linear correlation.

A PMCC scale showing how values of r correspond to direction and strength of linear correlation.

Definition

PMCC

The population correlation coefficient is written $ρ\rho$ . This is important in hypothesis testing: $r$ comes from your sample, while $ρ\rho$ describes the whole population.

Key Idea

Reading $r$

The sign of $r$ gives the direction. The size of $∣ r ∣$ gives the strength. A value near 0 only means “no clear straight-line relationship”; there could still be a curved relationship.

Example

Interpreting a PMCC

An analyst records years of experience and time taken to complete a task for 12 employees. The PMCC is $r = - 0.47$ .

The value is negative, so the relationship is negative.
This suggests that employees with more experience tend to take less time.
Since $∣ r ∣ = 0.47$ , the relationship is not very strong.
A sensible interpretation is: there is weak to moderate negative linear correlation between experience and task time.

Tip

Quick PMCC checks

The PMCC has no units, and swapping the two variables does not change its value. A very high or very low value only measures how linear the relationship is.

Regression lines

A regression line is a straight-line model used to predict one variable from another.

A regression line of y on x with the explanatory variable on the horizontal axis and the response variable on the vertical axis.

Definition

Regression line of $y$ on $x$

The regression line of $y$ on $x$ is the straight line used to predict $y$ from $x$ . It is usually written in the form $y = a + b x$ , where $b$ is the gradient and $a$ is the intercept.

The gradient tells you the predicted change in the response variable for each one-unit increase in the explanatory variable.

The intercept is the predicted value of the response variable when the explanatory variable is 0. This is only meaningful if 0 makes sense in the context.

Example

Interpreting a regression equation

A café owner records daily temperature, $t$ °C, and daily sales, £ $S$ , for 10 summer days. A regression model is

S = 4800 + 135 t

and the PMCC is $r = 0.69$ .

Temperature is the explanatory variable because it is being used to predict sales.
Sales is the response variable because it is being predicted.
The gradient is 135, so the model predicts that sales increase by about £135 for each extra 1°C.
Since $r = 0.69$ , the positive linear correlation is moderately strong, so a linear model is reasonably consistent with the data.
The intercept 4800 means the model predicts sales of £4800 when the temperature is 0°C, but this may not be useful if the data were only collected in summer.

Common Mistake

Extrapolation

Extrapolation means using a model outside the range of the original data. Regression predictions become less reliable when you extrapolate.

Extrapolation uses the regression line beyond the observed range of x-values, where predictions are less reliable.