Types of Reliability & Validity

Question 1

What is reliability?

Answer 1

Reliability is a measure of consistency, if a study is repeated using the same method, design and measurements and the same results are obtained- the results are said to be reliable.

Question 2

What are the two types of reliability?

Answer 2

Internal reliability: concerns the extent to which something is consistent with itself. For example, a set of scales should measure the same weight between 50 and 100 grams as between 150 and 200 grams.

External reliability: concerns the extent to which a test measures consistently over time.

Question 3

What are the two ways in which a researcher can assess the reliability of their research?

Answer 3

Test-Retest Reliability
Inter-Observer Reliability

Question 4

What is Test-Retest Reliability?

Answer 4

It involves administering the same test or questionnaire to the same person (or people) on different occasions.

If the test or questionnaire is reliable then the results obtained should be the same, or at least very similar, each time they are administered.

Question 5

What is Inter-Observer Reliability?

Answer 5

The extent to which two or more observers are observing and recording behaviour in a consistent way; particularly useful in situations where there is a risk of subjectivity.

Question 6

What is validity?

Answer 6

Concerns accuracy, the degree to which something measures what it claims to and the extent to which findings can be generalised beyond research settings.

Question 7

What is Internal (within) validity?

Answer 7

A measure of whether results obtained are solely affected by the changes in the variable being manipulated (by the independent variable) in a cause-and-effect relationship. A threat to internal validity is demand characteristics.

Question 8

What is External (outside) validity?

Answer 8

External (outside) validity: a measure of whether data can be generalised to other situations outside of the research environment.

Question 9

What are some examples of threats to validity?

Answer 9

Extraneous Variable

Confounding Variable

Demand Characteristics

Investigator Effects

Leading Questions

Question 10

What is Ecological Validity?

Answer 10

A type of external validity, refers to the extent to which psychologists can apply their findings to other settings; predominantly to everyday life.

A lack of ecological validity is typically a point made when discussing weaknesses of lab-based experiments, due to the artificial and contrived settings within a lab (people do not behave naturally) - therefore is problematic when generalising findings to a more natural situation.

Question 11

What is Temporal Validity?

Answer 11

Type of external validity,
Temporal validity refers to the extent to which research findings can be applied across time to different contexts.

Question 12

What is Population validity?

Answer 12

Type of external validity, Population validity describes how well a sample used in a research study can be generalised to the wider population as a whole.

Question 13

What is Face Validity?

Answer 13

A form of internal validity focusing on the extent to which a questionnaire, etc., appears (on the face of it) to be measuring what it claims to.

Question 14

What is Concurrent Validity?

Answer 14

Type of internal validity, this is where the performance of the test in question is compared to a test that is already recognised and trusted within the same field.

Question 15

How would someone improve reliability within a questionnaire?

Answer 15

Comparing two sets of data should produce a correlation of +0.8 or more to be reliable.

A questionnaire that produces lower than this may need to take out some of the questions (open questions) and replace with closed or fixed choice questions. Or they may need rewriting so that respondents can interpret them correctly.

Question 16

How would someone improve the validity within a questionnaire?

Answer 16

Include a lie scale to check honesty and for effects of social desirability bias. One way that researchers can do this is by having two items that are asking the same thing.

Ensure all questionnaires are submitted anonymously.

Question 17

How would someone improve the reliability within an interview?

Answer 17

To ensure reliability you need to use the same interviewer each time, this will also help to reduce researcher bias – if this isn’t possible then interviewers need to be trained.

Structured interviews usually have higher reliability as there is more control over questions.

Question 18

How would someone improve the validity within an interview?

Answer 18

Use direct quotes to show what they have found

Triangulation – use of a number of different sources as evidence e.g. diaries, interviews with family etc

Question 19

How would someone improve the reliability within an experiment?

Answer 19

Lab experiments have higher reliability than field experiments as they have lots of control over the variables.

All conditions/instructions must be kept exactly the same for all participants in the study to ensure standardisation.

Question 20

How would someone improve the validity within an experiment?

Answer 20

Use of a control group so we can see the effect of the IV on the DV.

Standardised procedures to limit impact of investigator effects and participant reactivity (demand characteristics).

Use of single blind and double blind procedures.

Question 21

How would someone improve the reliability within an observation?

Answer 21

Reliability can be increased by operationalising the behavioural categories used e.g. aggression – punching, kicking. This helps to
ensure that observations are objective rather than subjective to interpretation.

Categories should not overlap, e.g. snogging, kissing

Question 22

How would someone improve the validity within an observation?

Answer 22

Narrow behavioural categories

Covert instead of overt