What you'll learn
- What a pilot study is and where it fits into the research process.
- The main aims of piloting in psychological research.
- How pilot studies improve validity, reliability, practicality and ethics.
- How to apply pilot studies to scenario questions in AQA A-Level Psychology.
Starting point: why researchers do not just “begin”
In psychology, researchers often study complex human behaviour: memory, obedience, attachment, stress, mental health, social influence and more. A method that looks clear on paper may work badly with real participants.
For example, participants might misunderstand instructions, a questionnaire item might be ambiguous, a task might take too long, or a procedure might cause more anxiety than expected.
A pilot study helps researchers spot these problems before the full investigation begins.
What is a pilot study?
Pilot study
A pilot study is a small-scale trial run of a planned investigation, carried out before the main study to check whether the procedure, materials and practical arrangements work effectively.
A pilot study usually uses a small number of participants who are similar to the intended target population. It follows the planned method as closely as possible, but its purpose is not usually to test the main hypothesis fully.
Instead, the aim is to find out whether the research design needs improving before the researcher commits time, money and participants to the full investigation.
Core idea
A pilot study is not “the real study but smaller”. It is a rehearsal used to detect problems and improve the main investigation.
The basic sequence looks like this: design the study, test it on a small scale, identify problems, make changes, and then run the main investigation.
Key terms you need first
Main study
The main study is the full investigation that produces the data used to test the researcher’s hypothesis.
Procedure
The procedure is the step-by-step set of actions carried out by the researcher and participants. This includes instructions, tasks, timings, conditions and debriefing.
Materials
Materials are the things used in the study, such as questionnaires, interview schedules, memory tests, consent forms, stimulus pictures, video clips or rating scales.
Standardisation
Standardisation means keeping the procedure the same for all participants, so that differences in results are more likely to be due to the variable being studied rather than accidental differences in how the study was run.
Validity and reliability
Validity refers to whether a study measures what it claims to measure. Reliability refers to whether a measure or procedure is consistent.
Pilot studies can improve both.
The main aims of piloting
1. Checking instructions and materials
One major aim is to check whether participants understand what they are being asked to do.
This matters because unclear instructions can damage validity. If participants respond incorrectly because they misunderstood the task, the researcher may end up measuring confusion rather than the intended behaviour.
Piloting can reveal problems such as:
- technical words participants do not understand
- questions that are too vague
- response options that do not cover likely answers
- instructions that lead participants towards a particular answer
- rating scales that are too narrow or too confusing
For example, a questionnaire asking “Do you often feel stressed?” is vague. “Often” could mean daily to one person and monthly to another. A pilot might show that participants interpret the question differently, so the researcher could revise it to “How many days in the last week did you feel stressed?”
Improving a questionnaire item
A researcher is piloting a questionnaire about exam anxiety. One item says: “Do you revise a lot?”
- The researcher compares participants’ feedback and notices that some think “a lot” means two hours a day, while others think it means revising most evenings.
- This shows the item lacks clarity, so responses may not be comparable across participants.
- The researcher revises the item to: “On a typical school day, how many minutes do you spend revising for exams?”
- The revised item produces more specific data, which should improve validity because it measures revision time more directly.
2. Checking timing and practical arrangements
A pilot study can show how long the investigation actually takes.
This is useful because researchers may underestimate the time needed for consent, instructions, the task itself, breaks, debriefing or dealing with participant questions.
Piloting can help researchers decide:
- whether the study fits into the available time
- whether participants become tired or bored
- whether equipment works properly
- whether the order of tasks is sensible
- whether enough researchers are needed to run the procedure
This is especially important in school, clinical or laboratory settings where time and resources may be limited.
3. Checking whether the measure produces usable data
A measure is the tool or method used to collect data, such as a questionnaire score, reaction time, observation tally or interview response.
A pilot can show whether the measure produces the right level of measurement:
- Nominal data: categories, such as “passed” or “failed”.
- Ordinal data: ranked or ordered scores, such as a rating from 1 to 5.
- Interval data: numerical data with equal intervals, such as many psychological test scores.
This matters because the planned statistical test depends on the type of data and the research design. For instance, a correlation using two co-variables might use Spearman’s rho for ordinal data or Pearson’s r for interval data, usually tested against a critical value at a significance level such as p<0.05p < 0.05p<0.05.
A pilot study is not normally large enough to provide a strong statistical conclusion, but it can check whether the researcher is collecting the kind of data needed for the planned analysis.
Using pilot data as proof
Do not write that a pilot study “proves” whether the hypothesis is correct. A pilot is mainly used to check and improve the method, not to draw firm conclusions about the research question.
4. Checking standardisation and controls
Pilot studies help researchers see whether the procedure can be kept consistent.
A control is a way of reducing unwanted influences from variables other than the independent variable. These unwanted influences are called extraneous variables.
For example, if one participant receives more encouragement than another, this could affect performance. A pilot might reveal that different researchers are giving slightly different explanations, so the team could introduce a standardised script.
This improves reliability because each participant experiences the same procedure. It also improves internal validity because differences in results are less likely to be caused by accidental variations in the method.
5. Checking ethical safeguards
Ethics matter in pilot studies too. Participants in a pilot are still real participants, so the researcher must consider:
- informed consent: participants should know what they are agreeing to, unless justified deception is used
- deception: participants should not be misled unless necessary and ethically approved
- right to withdraw: participants should be able to leave the study without penalty
- protection from harm: researchers must avoid unnecessary distress or risk
- confidentiality: personal data should be protected
- debriefing: participants should be told the true aims and given support if needed
A pilot can reveal unexpected ethical issues. For example, a stress questionnaire might make some participants uncomfortable, or a memory task might embarrass participants who perform poorly. The researcher could then adjust the procedure, add clearer warnings, strengthen the right to withdraw, or improve the debrief.
Ethics link
If you are asked why a pilot study is useful, mention both method and ethics: it can improve the quality of the data and reduce the risk of harm to participants.
What happens after a pilot study?
After the pilot, the researcher reviews what happened. This may include participant feedback, researcher observations, timing notes, missing data, technical problems and any ethical concerns.
The researcher then decides what to change before the main study.
Possible changes include:
- rewriting instructions
- removing confusing questionnaire items
- changing the order of tasks
- shortening the procedure
- adding a practice trial
- improving the consent form or debrief
- training researchers to use a standardised script
- changing the sampling method
- choosing a more suitable statistical test
If the changes are major, the researcher may run another pilot to check that the new version works.
Do not automatically include pilot participants
Pilot participants are usually not included in the main study because they may have practised the task, guessed the aims, or experienced an earlier version of the method. Including them could bias the results.
Applying piloting to a scenario
In AQA questions, you may be given a research scenario and asked why a pilot study would be useful. The strongest answers apply the purpose of piloting to the specific study.
Do not just write: “It checks if the study works.” Say what exactly it checks and why that matters.
Applying a pilot to a memory experiment
A psychologist designs a study similar to Loftus and Palmer’s research into eyewitness testimony, using video clips of car accidents and follow-up questions about speed.
- The psychologist pilots the video clips with a small similar sample and finds that one clip is much darker than the others, making the accident hard to see.
- This creates a validity problem because later speed estimates may reflect poor visibility rather than memory distortion.
- The researcher replaces the unclear clip or improves visibility so that participants can see the event more equally.
- The pilot also checks whether questions such as “How fast was the car going?” are understood consistently, reducing ambiguity in the main study.
Notice how the answer links the pilot to the specific issue: visibility, question wording and validity.
AO3: strengths of pilot studies
They improve validity
Pilot studies can improve validity by ensuring the study measures what it intends to measure. If confusing instructions, ambiguous questions or irrelevant distractions are removed, the data are more meaningful.
For example, if a researcher wants to measure anxiety but the questionnaire items are really measuring general tiredness, a pilot may reveal this through participant feedback.
They improve reliability and replicability
A pilot can help create a standardised procedure. This makes the study more reliable because each participant is treated in the same way.
It also improves replicability, meaning another researcher could repeat the study using the same procedure. Replicability is important in psychology because scientific findings should not depend on hidden or accidental features of one researcher’s method.
They save time and resources
A small pilot may prevent a much larger failure. It is better to discover that a questionnaire is confusing after testing it on ten people than after collecting data from two hundred.
This is a practical strength because psychological research often requires access to participants, rooms, technology, trained researchers and ethical approval.
They support ethical research
Piloting can reveal unexpected distress, embarrassment, fatigue or misunderstanding. This helps researchers protect participants from harm in the main study.
This is especially important in sensitive topics such as mental health, trauma, addiction, aggression or obedience.
AO3: limitations of pilot studies
A small pilot may miss problems
Because pilot studies use small samples, they may not reveal every issue. A problem that affects only some groups might not appear in the pilot at all.
For example, a pilot using only psychology students may not show that the instructions are difficult for younger participants or people unfamiliar with psychological terminology.
Pilot participants may not represent the target population
A target population is the group the researcher wants to generalise findings to. If the pilot sample is not similar to the target population, the pilot may give misleading reassurance.
For instance, piloting a questionnaire on confident university students may not reveal problems that would occur when using it with anxious GCSE students.
Piloting can encourage over-refinement
Sometimes researchers may make a study so controlled and polished that it becomes artificial. This could reduce ecological validity, which is the extent to which findings apply to real-life settings.
For example, simplifying a social interaction task too much may make it easier to standardise but less realistic.
Pilot findings are not usually conclusive
A pilot study is not normally designed to provide a strong test of the hypothesis. The sample is usually too small, and the focus is on feasibility rather than statistical significance.
So, a pilot may suggest a trend, but the researcher should be cautious about treating that trend as a real effect.
Balanced evaluation
Pilot studies are valuable because they improve research quality before the main study, but they cannot guarantee that the final investigation will be valid, ethical or problem-free.
How to write about pilot studies in exam answers
For short-answer questions, a good structure is:
- Define the pilot study as a small-scale trial run.
- Identify what it checks in the specific scenario.
- Explain how this improves the final study.
For longer answers, add evaluation. You could discuss validity, reliability, ethics, practicality, sample representativeness and the limitation that pilot findings may not generalise.
In the exam
- Link piloting to a specific aim: checking instructions, timing, materials, ethics, controls, data recording or planned analysis.
- Apply your point to the research scenario rather than giving a generic sentence.
- Add impact: explain whether the pilot improves validity, reliability, standardisation, ethics or practicality.
Check yourself
- What is the difference between a pilot study and the main study?
- Give two specific problems a pilot study might identify before data collection begins.
- Why might a pilot study improve validity but still not prove the hypothesis?
