Two different questions about a study

1. Reliability is consistency, so it asks whether the study would give you the same result again if it were run the same way.

1. Validity is aim, so it asks whether the study measured the thing you actually meant to measure rather than something else that moved at the same time.
2. The two come apart, because a badly aimed measure can be perfectly consistent and still be measuring the wrong thing every single time.

Applied to sampling methods

1. A stratified sample supports validity because every subgroup is present in proportion, so the findings really are about the population you named.
2. A volunteer or opportunity sample weakens validity, because you have measured the keen or the nearby rather than the population.
3. Reliability is a different question, and random and stratified sampling are written-down procedures another researcher can follow and end up with a similar sample, so a repeat is more likely to give a similar result.
4. Opportunity sampling depends on who happened to be there that lunchtime, so a repeat can produce a completely different group and a completely different figure.

Applied to experimental designs

1. Independent measures protects validity from order effects, because nobody does the task twice and so practice and tiredness cannot build up.
   1. It threatens validity through participant variables, because the two conditions are different people and a difference between those people can be read as an effect of the independent variable.
2. Repeated measures protects validity against participant variables, because each person is compared with themselves and cannot simply be out-scored by a stronger group.
   1. It threatens validity through order effects, because an improvement caused by practice gets counted as an effect of the independent variable when it is nothing of the kind.
3. Matched pairs protects validity on both counts, because pairing people on a variable that matters, such as a memory score, evens out the groups without anyone doing the task twice.
   1. It threatens validity because you can only match on the variables you thought of, so any you missed still differ between the groups.
4. Turning to reliability, a repeated measures study is the easiest to repeat, because one modest group of people can be run through the same conditions again.
5. An independent measures study sits in between, because it needs a fresh set of people for every condition, but nothing has to be paired up before the study can start.
6. A matched pairs study is the hardest, because the whole matching exercise has to be done again before the conditions can even start.

Applied to quantitative and qualitative methods

1. Quantitative methods produce numbers, such as a score out of twenty or a count of how often a behaviour happened.
   1. Reliability is usually high, because a fixed set of closed questions is applied in the same way to everybody and gives the same answer when repeated.
   2. Validity can suffer, because squeezing a complicated feeling into a number from one to five may not capture what the person actually meant.
2. Qualitative methods produce words, such as what somebody said in an interview or wrote in an open question.
   1. Validity is usually high, because people describe their experience in their own words and can raise things the researcher never thought to ask about.
   2. Reliability is harder to secure, because an open conversation goes differently every time and two researchers can read the same answers in two different ways.

Turning this into marks

1. Name the feature of the study and say what it does to consistency or to aim.
2. Write it out in full, such as the sample being 20 volunteers so the findings may only describe unusually keen pupils.