What bias is
Bias
Anything in the sample, the researcher, the recording or the wording of a study that pushes the findings in one direction every time, rather than producing random error.
- Bias is anything that pushes research away from the truth in one set direction.
- Bias shifts every result the same way, so collecting more data does not remove it.
- A random mistake cancels itself out across many participants, which is what separates it from bias.
- The six sources are gender bias, cultural bias, age bias, experimenter bias, observer bias and bias in questioning.
- The first three concern who was studied, and the last three concern how the study was run.
- Bias tilts every result the same way, which is why a larger sample makes a biased study more confident rather than more accurate.
- Three of the six sources are about who was in the sample and three are about how the study was run, so the remedy is different in each case.
- Bias is rarely deliberate, and a warmer tone of voice on one condition is enough to move the findings.
Gender, cultural and age bias
Sampling bias
When the sample over-represents some groups and leaves others out, so it does not reflect the wider population the study wants to describe.
- Gender bias.
- One gender is over-represented in the sample, or findings from one gender are applied to everybody.
- It also sits in the materials, where a question assumes a boy plays football or a girl does the tidying.
- Effect: a conclusion is presented as being about people when it only holds for one gender, so population validity falls.
- Recruiting similar numbers of each gender and reporting their results separately reduces it.
- Cultural bias.
- The sample, or the idea being tested, comes from one culture and the findings are then treated as true everywhere.
- It also sits in the materials, where a task or a question only makes sense inside one way of living.
- Effect: behaviour that is ordinary in another culture gets written up as unusual, which is how research ends up describing difference as deficit.
- Testing the same idea in more than one culture, and checking the materials make sense to each group, reduces it.
- Age bias.
- Only one age group is studied, most often students, and the results are assumed to hold at every age.
- It also appears when materials written for adults are read differently by a younger participant.
- Effect: a finding about memory taken from 18-year-olds may describe nobody in their seventies, so generalisability is limited.
- Sampling a range of ages, or writing the conclusion about the age group actually tested, reduces it.
- A conformity study run on male American undergraduates carries gender, cultural and age bias at once.
- Its conclusion is written about people in general, so it claims more than its sample can support.
- A conclusion written about male American undergraduates would match the sample that was actually tested.
Experimenter bias
Researcher bias
When a researcher's own expectations or wishes shape how they run a study, record it or interpret it, pushing the findings towards the result they expected.
- Experimenter bias is the person running the study wanting a particular result, which then shapes how they run it, record it or interpret it.
- It can be as small as a warmer tone of voice when reading the instructions for the condition they favour.
- It enters the instructions, the way each condition is delivered, and the choice of which results to write up.
- Effect: the findings drift towards the hypothesis, so the theory looks better supported than the evidence allows.
- It is also called researcher bias, and it is rarely deliberate, because a researcher can tilt a study without meaning to.
- Underneath it sits confirmation bias, where the evidence that fits the prediction is the evidence that gets noticed.
- Standardised instructions, and having someone with no stake in the outcome score the data, reduce it.
- Nothing in a set of results shows on its own that bias was at work, which is why the procedure is reported in enough detail for somebody else to check it.
- Setting out the standardised instructions word for word lets a reader judge whether both conditions really were delivered the same way.
Observer bias
Observer bias
When an observer records what they expect or want to see rather than what actually happens, so their expectations distort the data.
- Observer bias is recording the behaviour that was expected rather than the behaviour that happened.
- It enters at the moment of recording, when a borderline behaviour lands in the hoped-for column.
- Two observers holding different expectations can log the same event in different columns.
- Effect: the tallies end up matching the observer's expectations, so the data confirms an idea it never tested.
- Tightly defined behavioural categories and a check on inter-rater reliability reduce it.
- One researcher often runs the study and records it, so both experimenter bias and observer bias can appear in the same study.
- A check on inter-rater reliability is what turns a worry about observer bias into something testable, since two observers who agree are unlikely to be recording their own expectations.
- Observer bias is hardest to spot in the observer's own data, because a tally looks just as tidy whether or not it matches what happened.
Bias in questioning
Leading question
A question worded in a way that suggests a particular answer, which can reshape what a person recalls.
- Bias in questioning is the wording or the order of a question pushing the person towards a particular answer.
- A leading question does it openly, so "Don't you agree that exams are far too stressful?" tells the reader which answer is wanted.
- "How stressful do you find exams, from 1 to 5?" asks the same thing and leaves the answer open.
- A double-barrelled item asks two things at once, so a single tick cannot mean anything clear.
- Order matters, because a question about crime in the news changes how someone rates their own street a page later.
- Single words shift answers too, and swapping "hit" for "smashed" changes the account someone gives of the same event.
- Effect: the study measures what the question suggested rather than what the person thought, so validity drops.
- Neutral wording, one idea per question and a balanced set of options reduce it.
- Piloting the questions on a few people shows up any item that reads as a hint, which can then be reworded.
- Biased: "How much do you hate revising for maths and English?"
- It assumes the person hates it, and it asks about two subjects in one item.
- Fair: "How much do you enjoy revising for maths, from 1 (not at all) to 5 (a great deal)?"
Telling the look-alikes apart
- Observer bias is the observer recording the wrong thing, so the error sits on the researcher's side.
- The observer effect is the participant behaving differently because they know they are watched, so it sits on the participant's side.
- Demand characteristics are participants guessing what the study is about and then playing along with it.
- Social desirability is participants giving the answer that makes them look good.
- Observer bias is a source of bias, and the other three are threats to validity that come from the participant.
- Observer bias is the researcher seeing what they expect, and the observer effect is participants behaving differently because they are watched.
- A biased sample is not a small flaw, because it decides who the findings can fairly describe.
Turning this into marks
- Name the source precisely, such as gender bias or bias in questioning, because "the study was biased" does not secure the knowledge mark.
- Tie the bias to the study in front of you by quoting its sample, its wording or its observer, because a bias named on its own scores little.
- Keep the source and its effect as two separate points, one naming the bias and one saying which way it tilts the findings.
- The mark most often lost is answering with the observer effect when the question is about observer bias.
- What is bias in research, and how does it differ from a random error?
- Outline one way a study could show gender bias.
- How do cultural bias and age bias limit what a study can conclude?
- What is the difference between experimenter bias and observer bias?
- Name two ways of reducing bias in questioning.