STUDY POWER AND SAMPLE SIZE - LEARNING OUTCOMES

Question 1

Why is it important to consider sample size?

Answer 1

We need to get an idea for what sample size is appropriate and target our resources appropriately - want to figure out the number required for us to see the effect we are looking for without wasting resources.

If our sample size is too large it could be considered a waste of resources - e.g. following up a large group of people will take a lot of resources compared to a small group.

If we have too few people in our sample size we may fail to detect an important effect. The estimates of effect may be too imprecise (wide confidence intervals).

We want to choose a sample size such that if the new drug truly is substantially better, we would be fairly certain of getting a significant result.

Question 2

If our confidence interval spans 0 then what can we conclude about the difference between two groups?

Answer 2

If our confidence interval spans 0 we can not conclude that there is any real difference because a value for difference of 0 is a plausible value.

Question 3

We want to choose a sample size such that if the new drug truly is substantially better, we would be fairly certain of getting a significant result. How do we ensure that we do this?

Answer 3

By performing sample size calculations.

Question 4

If the aim of your study is to obtain a prevalence estimate (or other estimate) and 95% CI then what approach would we take for estimating sample size?

Answer 4

The precision approach

Question 5

What information would we need to provide for the precision approach of study size estimation?

Answer 5

1. A rough guess of prevalence (or s.d if continuous)

2. An idea of how precise or narrow you want the confidence interval

Question 6

Why factors might increase the estimated sample size in our precision approach calculations? i.e. what would lead to us needing more people in a sample?

Answer 6

We may need more people in the sample when:

• The prevalence is closer to 0.5% (50%), or if continuous, the sd is larger
• We want a narrower confidence interval

Question 7

If the study aim is to carry out a statistical test to compare 2 groups then what method would we use to calculate sample size?

Answer 7

We would use the power approach if we want a sample size estimate for a study which looks at differences between groups.

Question 8

What information would we need to provide in order to carry out the power approach?

Answer 8

If the study aim is to carry out a statistical test to compare 2 groups we would use the power approach. We still need some pilot data either based on previous experimental results or previous survey data.

For this we need:

1. A rough guess of prevalence, i.e. % with outcome in baseline group
   or if continuous outcome, mean and s.d. in baseline group
   or is case-control, % exposed amongst controls
2. The minimum difference / effect you want to be able to detect, i.e. what is clinically important? Are we interested in an absolute difference between our two groups or are we looking at a difference in the risk ratio or odds ratio? Remember a risk ratio is only applicable to cohort or cross-sectional studies whereas an odds ratio can be applied to case-control studies etc.
3. We need to think about the probability of getting a false positive result (type I error - the probability of rejecting the null hypothesis when there is no true difference). This is the same of the significance level of your study and so is normally set at 5%.
4. Set the probability of type II error - the probability of failing to reject the null hypothesis when there is a true effect.

1 minus the probability of type II error is the power of the study.

The power is the probability of detecting an effect as significant if it really exists (usually 80-90%).

Question 9

What is meant by the power of a study?

Answer 9

1 minus the probability of type II error is the power of the study.

The power is the probability of detecting an effect as significant if it really exists (usually 80-90%).

Question 10

What does a study power of 90% mean?

Answer 10

A study power of 90% means there is a 10% chance of obtaining a type II error in the results - i.e. failing to reject the null hypothesis when there is a true effect.

We are 90% certain that we will detect the effect.

Question 11

When would we have to increase the sample size when carrying out a study in which you want to compare two groups?

Answer 11

We would need more people in the sample when:

• We want to detect a smaller difference between the groups
• We want a smaller significance level
• We want greater power
• The population prevalence is closer to 0.5 or for continuous outcome, the variance is larger

Question 12

Some general points.

Answer 12

1. Calculations are only rough estimates
2. We should try different scenarios - e.g. what would happen if we decided we wanted to look for a smaller effect, how would we have to increase our recruitment and is that feasible?
3. When thinking about recruitment we should always increase our sample size to allow for non-response, drop-outs etc.
4. Matching (case-control study) can increase power.
5. We can do calculations on unequal sized groups (more difficult if computing by hand).

Question 13

What method can increase power in case-control studies?

Answer 13

Matching

Question 14

SPSS won’t do power calculations for you - list some other packages that will.

Answer 14

• EpiInfo

- SamplePower

Question 15

What should we consider with regards to the results in a research paper considering what we know about power and sample size?

Answer 15

• A non significant result may not always mean that there is no true effect. The sample size may simple be too small to show any effect.
• The research paper should always give details of power / sample size calculation in the methods section. However, this is not always the case.