Reliability
The extent to which a measure gives the same, consistent result each time it is used.
- Reliability is the consistency of a procedure or measure.
- A reliable measure gives the same result when it is repeated under the same conditions.
- It is about consistency, not about being correct.
- Think of reliability as repeatability: same procedure, same result.
- A bathroom scale that always reads 2 kg heavy is reliable, yet not accurate.
The Two Types of Reliability
Inter-observer reliability
How far two or more observers watching the same behaviour agree on what they record; close agreement means the observation is more reliable.
- Inter-rater reliability
- The consistency and agreement between two people who judge or score the same responses.
- Inter-observer reliability
- The consistency and agreement between two people who observe the same behaviours.
- When the two people closely agree, the measure is treated as more reliable.
- Two markers scoring the same essays and giving nearly the same marks shows inter-rater reliability.
- Two observers tallying the same playground behaviours and matching closely shows inter-observer reliability.
Reliability and Validity
- Do not confuse reliability with validity; reliability is consistency, validity is accuracy.
- Do not assume a consistent measure is measuring the right thing.
Spotting Reliability in a Study
- Ask whether the measure would give the same result if repeated; that is reliability, not accuracy.
- Ask whether two raters or observers would record the same thing.
- Name the exact type: inter-rater reliability for scoring responses, inter-observer reliability for observing behaviour.
- What is reliability?
- What is inter-rater reliability?
- What is inter-observer reliability?
- Why is a scale that always reads 2 kg heavy still reliable?