What you'll learn

• What reliability means in psychological research and why it matters.
• The difference between internal reliability and external reliability.
• How to assess reliability using split-half, test-retest and inter-rater methods.
• How to improve reliability in your own Component 2 practical work.

AO1: What is reliability?

In psychology, researchers often measure things that cannot be seen directly, such as anxiety, aggression, memory or wellbeing. These are called constructs: psychological qualities or processes that we infer from behaviour, responses or scores.

A measure is useful only if it gives results that are consistent. If the same questionnaire, observation schedule or test produces wildly different results for no good reason, it is hard to trust the findings.

Reliability is closely linked to the scientific goal of replicability, which means that findings can be repeated using the same method. If a study cannot be repeated consistently, psychologists should be cautious about drawing firm conclusions from it.

Reliability is not the same as validity

A measure can be reliable but not valid. For example, a questionnaire might consistently give the same score, but if the questions do not really measure anxiety, it is not valid.

Internal reliability

For example, if you create a ten-item questionnaire to measure self-esteem, all ten items should be pulling in roughly the same direction. If half the questions seem to measure confidence, while the other half measure popularity, the scale may have poor internal reliability.

Assessing internal reliability: split-half reliability

A common method is to split the questionnaire into odd-numbered and even-numbered items. The researcher then compares participants’ scores on the two halves. If the two sets of scores are strongly positively correlated, the measure has good internal reliability.

A correlation coefficient is a number showing the strength and direction of a relationship between two sets of scores. It is often written as $r$, where values closer to plus one show a stronger positive relationship.

External reliability

External reliability is especially important in observations, experiments and repeated testing. If findings depend too much on the exact observer, day or situation, the research may be unreliable.

Assessing external reliability: test-retest reliability

The researcher gives participants the same measure twice, then compares Time 1 and Time 2 scores. If scores are very similar, the measure has good test-retest reliability.

This is most appropriate when the construct should be fairly stable, such as personality traits or long-term attitudes. It is less useful for states that naturally change quickly, such as mood, tiredness or stress before an exam.

Assessing external reliability: inter-rater reliability

This is especially important in observations and content analysis. For example, if two observers are recording aggression in a playground, they should agree on what counts as “aggression”. If one observer counts teasing as aggression and the other does not, the data will be inconsistent.

Researchers can assess inter-rater reliability using percentage agreement or a correlation. In simple observational coding, percentage agreement is often used:

$$Percentage\; agreement=\frac{number\; of\; agreements}{number\; of\; agreements+number\; of\; disagreements}\times 100$$

AO2: Applying reliability to your own investigation

In Component 2, reliability is often applied to a novel scenario or your own practical work. You should be able to say exactly what could go wrong and how to check or improve it.

If you use a questionnaire

You can improve reliability by:

• using clear, unambiguous questions;
• avoiding double-barrelled questions, such as “I feel anxious and unhappy”;
• using consistent response options, such as the same five-point rating scale throughout;
• assessing split-half reliability if several items measure the same construct.

If you use an observation

You can improve reliability by:

• creating operationalised behavioural categories, meaning categories that define behaviours clearly and objectively;
• making categories mutually exclusive, so one behaviour cannot fit two categories at once;
• training observers using the same coding manual;
• checking inter-rater reliability before analysing the final data.

If you repeat a measure

You can improve reliability by:

• keeping instructions and materials the same each time;
• using the same testing conditions where possible;
• choosing a sensible time gap;
• checking test-retest reliability if the construct should remain stable.

Ways of dealing with reliability issues

A strong answer should go beyond saying “make it reliable”. You need to explain the specific method.

Useful strategies include:

• Standardised procedures: give every participant the same instructions, materials and time limits.
• Pilot studies: run a small trial first to detect confusing questions, unclear categories or timing problems.
• Observer training: practise coding examples until observers apply categories consistently.
• Clear scoring rules: decide in advance how responses will be marked or coded.
• Equipment checks: make sure apparatus, online surveys or recording devices work consistently.
• Reviewing weak items: remove or rewrite questionnaire items that do not match the rest of the scale.

Ethics and reliability

Reliability checks still need to follow the BPS Code of Ethics and Conduct. This includes informed consent, protection from harm, right to withdraw, confidentiality and debriefing.

For example, if you video-record behaviour so that two observers can code it, participants should normally know they are being recorded, recordings should be stored securely, and identities should be protected.

If animal research is involved, additional ethical issues apply, including minimising suffering, using the fewest animals necessary, providing suitable housing and considering alternatives where possible.

AO3: Evaluating reliability

Reliability is a major strength because it increases confidence that findings are not just due to chance, unclear procedures or researcher inconsistency. Reliable methods are easier to replicate, which supports psychology’s scientific status.

However, reliability has limits. A measure can be highly reliable but invalid. Very rigid standardisation may also reduce ecological validity if the situation becomes artificial. Inter-rater reliability can be high because categories are simple, but this may miss subtle behaviour. Test-retest reliability can be affected by memory, practice effects or genuine change over time.

You can also link reliability to the wider statistics content. Reliability checks often use descriptive comparisons, percentages or correlations. In inferential testing, Eduqas expects you to choose tests using the aim, design and level of measurement: for example, Spearman’s rho for correlations, chi-square for nominal frequency data, Wilcoxon or Mann-Whitney U for ordinal differences, and related or unrelated t-tests for interval differences. The usual significance convention is $p\le 0.05$, but remember: statistical significance is not the same thing as reliability.