What a questionnaire is
Questionnaire
A self-report method that collects data through a set of written questions, which can be given to many people at once.
- A questionnaire collects data through a set of written questions that the participant fills in themselves.
- It is a self-report method, so the data are what people say about themselves.
- Because the questions are written rather than spoken, the same form can be handed to a whole year group in one lesson.
- Every participant meets identical wording in identical order, which is standardisation.
- The items used decide the data collected, whether open questions, closed questions or rating scales.
- What a questionnaire measures is decided when the items are written, so a badly worded question cannot be repaired once the forms are back.
- The written wording is identical for everybody, which is what lets another researcher hand out the same form and check whether the results hold.
- A questionnaire reaches a whole year group in one lesson, and what comes back from each person is only what the form asked for.
Open questions
Open question
A question that lets people answer in their own words with no fixed options, producing qualitative data.
- An open question lets people answer in their own words, with no options to pick from.
- That gives qualitative data, detailed about how someone thinks or feels.
- Strength: an answer can raise a reason the researcher never thought to offer as an option, so validity is higher.
- Weakness: the answers are slow to analyse, and two researchers can summarise the same answer differently, which lowers reliability.
- An open question suits an aim about why something happens or how it feels, such as why revision makes pupils anxious.
- An open question on a form usually draws a shorter answer than the same question asked out loud, because writing takes more effort than talking.
- A couple of open items at the end of a form of closed questions is common, since it catches anything the fixed options missed.
Closed questions
- A closed question gives a fixed set of options, so the participant ticks rather than writes.
- That gives quantitative data that can be totalled, tallied into a frequency table and expressed as a percentage.
- Strength: everyone picks from the same list, so answers are quick to compare and reliability is high.
- Weakness: when none of the options fits, the person ticks the nearest one anyway, so the answer no longer measures what they think.
- A closed question suits an aim about how many or how often, such as how many pupils revise on a school night.
- The options have to cover every possible answer without overlapping, so 0 to 1, 2 to 3 and 4 or more works while 0 to 2 and 2 to 4 does not.
- A closed question about how often something happens needs a time frame attached, because one person reads "often" as daily and another as weekly.
Rating scales
Rating scale
A type of closed question that asks people to rate something on a fixed run of numbers, such as 1 to 5, so the answers come back as quantitative data.
- A rating scale is a closed question that asks for a rating on a fixed run of numbers, most often 1 to 5.
- Both ends carry a label, such as 1 for "not at all" and 5 for "very much", so everyone reads the numbers the same way.
- The number of points stays the same on every item, so ratings can be compared across questions.
- Adding the ratings and dividing by the number of participants gives a mean rating, reported as a figure out of 5.
- Because each rating is a number, it can also be plotted against a second measurement on a scatter diagram.
- Strength: it turns a strength of feeling into a number, so a mean rating can be worked out and one group compared with another.
- Weakness: the number leaves out the reason, so a 2 from one person and a 2 from another may mean different things.
- Weakness: many people avoid the two extreme ends and bunch in the middle, which flattens the differences the item was measuring.
- Weakness: an unlabelled run from 1 to 5 leaves each person to decide what a 3 means, so the mean rating is hard to interpret.
- A scale with three points flattens differences that a scale with five points would show.
- A rating scale suits an aim about strength of feeling, such as how strongly someone agrees with a statement.
- A screen-time survey is given to a Year 11 class.
- Closed: "How many hours a day are you on your phone? 0 to 1, 2 to 3, 4 or more."
- Open: "How does being on your phone affect your revision?"
- Rating scale: "How distracting is your phone when you revise, from 1 (not at all) to 5 (very distracting)?"
- One survey often carries all three item types, because each one answers a different kind of aim.
Strengths and weaknesses of questionnaires
Social desirability bias
The tendency for people to answer in a way that makes them look good, rather than giving an honest response.
- Strength: they are quick and cheap, so a large sample can be reached in one go, which helps generalisability.
- Strength: the written wording is standardised, so another researcher can run it again and check whether the results hold, giving high external reliability.
- Strength: a private form can be filled in without saying an awkward answer out loud to anyone.
- Weakness: a misread question cannot be explained by anyone, so the answer measures something the researcher did not intend, which lowers validity.
- Weakness: social desirability bias pushes people towards the answer that makes them look good.
- Weakness: many forms are never returned, and the people who do reply may differ from those who do not, which weakens population validity.
- Weakness: when the items pull in different directions the questionnaire is not consistent within itself, which is low internal reliability.
- A questionnaire suits an aim that needs the same information from a large number of people quickly.
- A rating scale is a closed question and it produces numbers, which is what separates it from an open question.
- Fixed written wording is what makes a questionnaire repeatable, so a flat statement that questionnaires are unreliable misses where their reliability comes from.
Writing questionnaire items
- Keep each question about one thing, so nobody has to answer two questions with a single tick.
- Cut any leading wording that hints at a particular answer, so "Don't you agree exams are unfair?" becomes "How fair do you think exams are?"
- Match the item type to the data the aim needs before writing any questions.
- Try the form on a few people first, then reword anything they misread.
- Promise confidentiality on the front of the form, which reduces social desirability bias and meets the British Psychological Society's guidance.
Turning this into marks
- When an item type and a reason are both asked for, keep them as two separate points, one naming the type and one naming the data it produces.
- Write the item out in full, including both scale labels, because "I would use a rating scale" does not secure the knowledge mark.
- Give a figure with the scale it came from, so a mean rating reads 4.2 out of 5.
- The mark most often lost is calling a rating scale an open question, when it offers a fixed run of numbers to choose from.
- What is a questionnaire, and how does it differ from an interview?
- What type of data does an open question give, and what type does a closed question give?
- What is a rating scale, and what is one strength of using one?
- Why does a low response rate weaken a questionnaire's findings?
- Rewrite "Don't you agree exams are unfair?" as a fair question.