Critical appraisal Flashcards Preview

MFSRH part 2 > Critical appraisal > Flashcards

Flashcards in Critical appraisal Deck (134)
Loading flashcards...
1
Q

What is meany by the Number Needed to Treat (NNT)

A

The NNT measures the number of patients who need to receive a treatment to prevent one additional bad outcome

2
Q

equation for NNT

A

In studies it is calculated by 1 / (EER - CER)

i.e. NNT = 1 / ARR

3
Q

what is meant by relative risk

A

RR = Probability of an event when exposed/Probability of event in control group
Also called risk ratio.

measure of proportionate increase in exposed group

RR = EER / CER (= (a / a+c) / (b/b+d) )

4
Q

what is meant by absolute risk?

A

Absolute risk is likelihood of developing a disease over a time period

5
Q

what is meant by hazard ratio

A

A hazard ratio considers your absolute risk to be 1.
If something you do / take doesn’t change your risk, then the hazard ratio is 1.

If something you do lowers your risk by 30% compared to not doing it then that action makes your hazard ratio 0.70 = the risk is 70% of what it was (30% lower).
If something you do triples your risk, then your hazard ratio is 3.0 (risk is 3 times greater than it was before).

6
Q

Experimental event rate calculation

A

EER = a/ (a+c)

a = exposed and diseased
c = exposed and not diseased

A measure of how often a particular event occurs within the experimental group of a trial.

7
Q

what is the Absolute risk reduction

A

ARR (RD) = CER - EER
i.e. (a/a+c) / (b/b+d)
absolute risk reduction is the same as risk difference. Comes from subtracting the risk in one group from the risk in the other group.

High risk patients show a bigger risk difference than low risk patients. Patients who are unlikely to experience the outcome will need more patients to be treated for one outcome to be avoided, and the risk difference is smaller.

8
Q

what does it mean if the relative risk is 1

A

no risk difference

no causation

9
Q

what does it mean if the relative risk is > 1

A

Increased risk amongst those exposed

i.e. it is a risk factor

10
Q

what does it mean if the relative risk is < 1

A

decreased risk amongst those exposed

i.e. it is a protective factor

11
Q

what is relative risk reduction

A

Relative risk reduction is the proportional reduction in risk between active and control participants in a trial.

If the relative risk reduction is 10%, the risk in the active treatment group will be 90% of that in the control group.

12
Q

what makes the NNT larger?

A

Less effective treatments

i.e. prevent fewer events, so more patients need to be treated to prevent a single event.

13
Q

what makes NNT smaller?

A

The NNT gets smaller for longer durations of treatment.

Or for more effective treatments

14
Q

Which will appear more impressive of the relative risk or the absolute risk?

A

The relative risk will always be a larger percentage figure than absolute risk, so relative risk seems more impressive.

15
Q

What impact does the framing of research results have

A

The framing of results is the way that results are presented and this can influences doctors’ attitudes about a treatment.

16
Q

what is EER

A
EER = experimental event rate (incidence in group receiving intervention)
EER = a/ (a+b)
17
Q

what is CER

A
CER = control event rate (incidence measured outcome in control)
CER = c / c+d
18
Q

What is meant by odds ratio

A

Odds Ratio is a method of comparing how likely events are between 2 groups.
odds of event occurring in one group divided by the odds of event occurring in the other group.
OR = a x d / b x c

19
Q

calculation for odds of a positive outcome in exposed patients

A

odds f a positive outcome in exposed patients = a/b

e.g. odds of rolling a 1 on a dice = 1/5 = 0.2

20
Q

what is a Student’s t-Test

A

Students T-test compares two means (averages) and tells you if they are different from each other.
And how significant the differences are.

21
Q

what is Chi Squared test

A

Chi Squared test is a

Nonparametric test used to compare numerical or categorical data sets.

22
Q

what is categorical variable

A

ategorical variables represent types of data which may be divided into groups.
Examples of categorical variables are race, sex, age group, and educational level.

23
Q

what is meant by Nonparametric data

A

Does not fit a normal distribution.

Nonparametric statistics uses data that is often ordinal, meaning it does not rely on numbers, but rather on a ranking or order of sorts.
e.g. a survey conveying consumer preferences ranging from like to dislike would be considered ordinal data.

24
Q

what is a numerical / quantitative (Non-categorical) variable

A

Numerical (Non-categorical) variables are usually referred to as range variables and are expressed as numbers within the set of the Reals, usually varying from negative infinity to positive infinity

25
Q

what statistical tests can be done with data that is normally distributed

A

Students T test

ANOVA (analysis of variance)

26
Q

what statistical tests can be done with data that is not normally distributed

A

nonparametric tests - which dont assume anything about the population parameters.

chi-square,
Fisher’s exact test
Mann-Whitney test.

27
Q

what is a regression analysis used for

A

A regression equation is used in stats to find out what relationship, if any, exists between sets of data

Regression analysis is also used to understand which among the independent variables are related to the dependent variable.

In restricted circumstances a regression analysis can be used to infer causal relationships between the independent and dependent variables.

28
Q

what does simple linear regression result in

A

A linear relationship between one dependent variable and one independent variables = a straight line of best fit
Like an average of where all the points line up.

In linear regression, the regression line is a perfectly straight line

29
Q

what does polynomial regression result in

A

A polynomial regression results in a curved line of best fit .
polynomial regression is considered to be a special case of multiple linear regression

30
Q

what does multiple linear regression result in

A

Also known as multiple regression
A statistical technique to model the linear relationship between the 2 pr exploratory (independent) variables and the response (dependent) variable.

linear relationship between one dependent variable 2 or more independent variables.

31
Q

What are Rank Sum tests?

A

Wilcoxan and Mann Whitney U tests are both rank sum tests.

These are Nonparametric tests

Does not require the assumption of normal distributions.

32
Q

What are Bradford Hills criteria and what are they for?

A

The criteria for causation

  1. Temporality - time sequence
  2. plausability
  3. consistency
  4. strength of association
  5. dose - response gradient
  6. reversibility
  7. specificity
  8. coherence
  9. analogy
33
Q

points to look at when appraising a case control study?

A
  • focused aim / issue
  • Is the method appropriate for the question
  • Recruitment - case definition, cases representative of population, system for case selection, are cases incident or prevalent, time frame of stuy relevant to disease?, power calculation
  • contol selection - representative and matched?
  • Exposure accurately measured? recall bias, subjective, is the measure validated
  • blinding
  • confounders
  • results
  • strenght of association, OR, P-value, CI
34
Q

what is a consort flow diagram

A

a flow diagram to provide a broad overview of how the trial was conducted
Concise method to report research method and protocol
Shows progress through phases of an RCT of each trial arm

35
Q

Advantage of case control studies

A
cheap
Quicker than cohort
requires smaller sample size
not prone to loss to follow up
study rare outcomes
36
Q

Define positive predictive value

A

how likely someone who tests positve actually has the disease
= true positive
PPV = A/A+B
(where A is screen +ve has disease and B is screen positive no disease)

37
Q

Define negative predictive value

A

How likely someone who tests negative actually does not have the diease
= true negative
PPV = D/C+D
(where D is screen -ve no disease and C is screen -ve has disease)

38
Q

what is the attributable risk

A

Same as the risk difference.

difference between incidence rates in the exposed and non exposed groups

39
Q

calculation for control event rate (CER)

A

CER = b / b + d

b = not exposed and diseased
c = not exposed and not diseased
40
Q

calculation for estimated population exposure

A

a + c / a + b + c + d

41
Q

calculation for (population attributable risk percent) PAR%

A

PAR% is calculated by
dividing the population attributable risk (PAR) by the incidence in the total population
then multiplying the product by 100 to obtain a percentage

42
Q

How is relative risk different to attributable risk?

A

Relative risk helps determine whether and how strongly a precursor is associated with a particular outcome.

Attributable risk helps determine how much of an
outcome may be attributable to a particular risk factor (i.e. an estimate of the excess risk) in a
population exposed to that factor.

43
Q

what is a population attributable risk?

A

The proportion of all cases of an

outcome in the total population that could be attributed to the exposure to the risk factor.

44
Q

what factors help distinguish cause from association?

A
strength of association (relative risk) 
time sequence - cause MUST precede outcome
distribution - geographical
gradient - dose and duration correlation with outcome
consistency
specificity
biologically plausable
experimentally reproducable
precentative trials
45
Q

what is the absolute risk?

A

Number of events (or disease) / total number in the group or population

46
Q

uses of descriptive studies

A

guidance on association
demonstrate a broad geographical difference
determine relative frequency of dieases or factors
service planning
evaluating service effectiveness

47
Q

types of sampling methods for observational studies

A

simple random sampling - e.g. random number generator
systematic sampling - seclect person at regular intervals
stratified sample - propabbility of patient selection varies by a pre-determined characteristic to ensure a subgroup is represented
cluster sample - groups are the sampling units, randomly selected
continuous sampling
convenience sampling

48
Q

sources of bias in sampling

A

deviation from sampling methodology
omission of people who are hard to identify as belonging to the population
large numbers declined to participate
self selection or volunteer patients
excessive exclusion criteria
Berkson bias - selection taken from a sub-population who do not reflect the whole population (e.g hospital patients)

49
Q

What does CONSORT stand for?

A

Consolidated Standards Of Reporting Trials

50
Q

what is CONSORT used for

A

Consolidated Standards Of Reporting Trials

A statement introducing a set of recommendations to improve RCT reporting

Protocol to identify problems from conducting RCTs and assist with reporting them fully and clearly

51
Q

what are the 4 stages of an RCT listed in a CONSORT flow diagram?

A
  1. Enrollment
  2. Intervention allocation
  3. Follow - up
  4. Analysis of data / results
52
Q

The first part of a CONSORT flow diagram is ‘enrollment’

what 3 things should this include?

A

Enrollment - include
1 - Number assessed for eligibility
2 - Number excluded and why - not meeting inclusion criteria / decline / other reasons
3 - Number randomised (study population)

53
Q

The second part of a CONSORT flow diagram is ‘allocation’

what should this include?

A

Allocation includes

  • number of patients allocated to intervention
  • number of patients in each arm who receive the intervention
  • number in each arm who do not receive the intervention
54
Q

The third part of a CONSORT flow diagram is ‘follow-up’

what should this include?

A

Follow-up should include - for each arm

  • number lost to follow up
  • reasons for loss to follow up
  • number discontinuing the intervention
  • reasons for discontinuing
55
Q

The fourth part of a CONSORT flow diagram is ‘analysis’

what should this include?

A

Analysis includes - for each arm

  • number analysed
  • number excluded from analysis
  • reasons for exclusion from analysis
56
Q

Disadvantages of a retrospective case control study

A
Retrospective
recall bias + reduced reliability 
incomplete records
previous records not standardised
Difficult to prove causality
Difficult to select appropriate controls
Cannot calculate incidence within a population
57
Q

Why is an intention to treat analysis usually done?

What is the benefit of this?

A

subjects are analysed according to the treatment allocation irrespective of whether is was recieved

This is because treatment allocation was random.
Any changes to treatment were not random.
ITT avoids breaking randomisatin

Therefore not doing an ITT analysis introduces a potential for confounding.

ITT analysis demonstrates the potential effect of the treatment protocol rather than the specific treatment.

58
Q

What is a ‘rate’ in relation to studies

A

The relative frequency of an event per unit of time

The calendar time period must be the same for the numerator and denominator

often expressed as per 1000 per year

59
Q

What is PICO

A

Population / patients
Intervention
Control / comparator
Outcome

60
Q

In PICO what should be considered in relation to ‘P’

A

Population / patients

  • age
  • gender
  • ethnicity
  • diagnosis
61
Q

In PICO what should be considered in relation to ‘I’

A

Intervention

- the treatment or exposure

62
Q

In PICO what should be considered in relation to ‘C’

A

Control / comparison

  • control / non exposed
  • placebo
  • current best treatment
63
Q

In PICO what should be considered in relation to ‘O’

A

Outcome - must be measurable

  • pain score
  • survival time
  • cure / remission
  • shorter hospital stay
64
Q

Points to look at when appraising an RCT

A
Aims and primary outcome - clarity
Recruitment and selection 
Randomisation 
Allocation concealment
Controls and matching 
Confounders
Exclusion criteria
Blinding
Drop out rates - why
sample size - power calculation 
results - p-values, CI
ITT analysis or per-protocol
65
Q

what is the risk of an under powered study

A

A small sample size leads to an under-powered study.

This increases the risk of making a type 1 error
= where it is believed that a differnce exists where no such difference does exist

66
Q

What is a type 1 error

A

where researchers believe an effect is demonstrated when really there is no effect

I.e to incorrectly reject the null hypothesis

67
Q

Methods of correcting for multiple statistical analyses

A

Bonferroni - P-value / number of tests

Benjamini-Hochberg procedure - involves ranked p-values from smallest to largest and allocating the smallest rank to one, rank 2 to the next and so on. Then compare each to the B-H critical value

68
Q

Sources of error

A
  1. Subject Variation = differences in the observed factor
    • e.g. variation in BP on different occasions
    • impact of motivation, recall etc
  2. Observer variation
    • intra-observer variation
    • inter-observer variation
    • bias due to awareness of the hypothesis
    • inconsistent procedure or question wording
    • lack of skill / training / interest
  3. Technical Limitations
    • Test doesnt measure what it is intended to
    • method intrinsically unreliable
    • defective instruments
    • incorrect calibration
    • test repetition and replication
69
Q

Advantages of prospective cohort studies

A

Prospective = better able to standardise methods and reduce error
Can calculate incidence in exposed and non-exposed groups
Can distinguish cause from association
Can study several outcomes from one hazard

70
Q

disadvantages of prospective cohort studies

A

Long duration
Large number of subjects
Expensive
May have higher drop out rate / loss to follow up
cannot be certain of true causation - this requires experimentation

71
Q

Why is it usually preferable for a study to recruit incident rather than prevalent cases?

A

Incident cases are newly diagnosed and have less risk of confounders
Prevalent cases are patients with already established disease - therefore they represent the survivors of an earlier cohort of incident cases.
Prevelent cases are less likely to represent everyone who is diagnosed as it doesn’t include those who get better or die.

72
Q

management of confounding variables in a case-control study

A

matching of controls
exclusion criteria
subset analysis based on confounding variables
multivariate analysis technique e.g. logistic regression

73
Q

Calculation of Rate %

A

(number of cases / population at risk) x 100

74
Q

Hierarchy of evidence

A
Meta analysis / Systematic review
RCT
Non randomised CT
Cohort study
Case control study
Cross sectional survey
Case series
Case report
Editorials, expert oppinions
Animal research
75
Q

Define sensitivity

A

the ability of the test to correctly identify those patients with the disease.
= patient does have the disease and is positive on the test

A/ a+c

76
Q

define specificity

A

the ability of the test to correctly identify those patients without the disease.
= patient does not have the disease and is negative on the test

77
Q

What does the positive predictive value depend upon?

A

PPV and NPV are dependent on the population being tested

Are influenced by the prevalence of the disease

78
Q

Define likelihood ratio

A

How much more likely is it that a patient who tests positive has the disease compared with one who tests negative.

likelihood that a given test result would be expected in a patient with the target disorder compared to the likelihood that that same result would be expected in a patient without the target disorder.

79
Q

5 steps of EBM

A

1) form a question re a clinical problem
2) evidence
3) critical appraisal
4) application
5) implementation and monitoring

80
Q

Items to consider if appraising a website

A
Is there a named author?
Author qualifications?
Funding / supporting of site listed?
Any conflicts of interest recorded?
Is the site hosted by a reputable organisation - e.g. .org / .gov / .edu
Any peer review process
Is the coverage full and unbiased
does the spelling, grammar and information appear accurate?
81
Q

What does the term ‘MeSH search’ mean?

A

Medical Subject Headings search - uses pre-defined keywords

82
Q

Types of observational descriptive studies

A

Survey / questionaire
Qualitative survey
Case series
Case report

83
Q

Types of observational analytical studies

A

Case control study

Cohort study

84
Q

Types of experimental studies

A
Randomised control trial
Controlled trial
Uncontrolled trial
Cross over trial
N of 1 trial
85
Q

What is an ecological study

A

A study on a population or community

86
Q

What is a pragmatic trial?

A

Takes place in a real life environment therefore may be more reflective of everyday practce

87
Q

what is an economic analysis

A

Assess cost and/or utilities of intervention

Helps to prioritize services

88
Q

What is a systematic review

A

A group of studies which are grouped and appraised together to answer a hypothesis

89
Q

What is a meta-analysis

A

A statistical procedure for combining the numerical data of separate `studies to answer a clinical question

90
Q

What is internal validity

A

To what extent does the study measure what it set out to measure.
What is the evidence that the intervention caused the outcome
Did the study limit and control confounding variables
Robust research design
Sufficient power
Correct statistical analysis

91
Q

Factors affecting internal validity

A
Attrition / mortality (drop outs) 
maturation - respondants grow out of the condition / grow experinenced
instrumentation
regression towards the mean
selection
contamination
history
per-protocol analysis
92
Q

What is external validity

A

The extent to which results from the study can be generalised to the wider population
Can the results be applied to real patients in real life settings
Can the results be generalised to other locations and times

93
Q

Factors affecting external validity

A

Sample population representative of wider population?
Sampling method
Does the test / measurement affect behavior or outcome
does the test environment mirror real life
is the test stimuli similar to real life
do the elicited behaviours represent real life behaviours

94
Q

Define efficacy vs effectiveness

A

Efficacy = the impact of interventions under optimal trial conditions

Effectiveness = the impact of the interventions under clinical or real life circumstances

95
Q

Considerations regarding the journal when appraising evidence

A

Is it peer reviewed - independently, non-editorial staff
Is peer review double-blind?
Impact factor - based on number of citations

96
Q

Disadvantages of the peer review system?

A

delay in publication
author may be known to the reviewer - therefore reduces objectivity
revolutionary / unpopular opinions may face opposition within peer review
doesnt guarantee errors will not appear

97
Q

How is a journal impact factor calculated?

A

Ranking of journals by measuring the frequency of the average article cited per year
Done by reuters
Comparison of journal performance

98
Q

What is the Indelfinger rule?

A

Neither original research
or any of its pictures or tables
may be published in more than one outlet

99
Q

Types of conflict of interest

A

Factors which may influence professional judgements
Financial - grants / honoraria
Professional - membership to an organisation
Personal

100
Q

criteria of authorship

A

Any of the following

  • Substantial contribution to conception
  • or design of the work
  • or acquisition / analysis / interpretation of data for the work
101
Q

Risk of multiple subgroup analysis

A

‘data dredging’
Increased risk of Type 1 Error
more analyses = more likely that a significant result result will be found by chance
- unless the p-value is corrected

102
Q

Risk associated with adding subgroup analyses after the data is collected

A

Introduces bias
increases risk of making a type 1 error
research method and power may be sub-optimal to answer additional questions

103
Q

Difference between bias and confounding

A

Bias leads to wrong results - the data is incorrect

Confounding leads to the wrong conclusion - but the results are correct

104
Q

What causes bias

A

Mistakes in the research techniques
The results generated are therefore wrong
Bias is not a real-life association or relationship but is introduced by the research process

105
Q

What causes confounding?

A

A real-life relationship that exists between exposure and outcome independent of the research process.
NOT a mistake
Studies have the correct result but the wrong conclusion may be reached if confounders are not recognized

106
Q

2 common types of bias

A

Selection bias

Observation bias

107
Q

What is selection bias

A

Problem with recruitment and allocation
Sample not representative of target population
includes - sampling bias and response bias

108
Q

What is observation bias

A

Problem with the way data is gathered

Results influenced by expectations of research team or subjects

109
Q

Types of selection bias

A
Sampling bias
Response bias
Berkson / admission rate bias
Diagnostic purity bias
Neyman bias / incidence-prevalence bias
Membership bias
Historical control bias
110
Q

What is a Berkson Bias

A

Admission rate bias
e.g. sample taken from inpatient setting but the hospital cases do not represent the severity of the condition in the general population

111
Q

What is a diagnostic purity bias?

A

Excessive exclusion criteria
Comorbidity is excluded from the sample to such an extent that the sample doesnt reflect the complexities of the general population

112
Q

What is a Neyman bias?

A

Incidence-prevalance bias
Occurs if prevalence of a condition doesnt reflect its incidence
Often affects rapidly progresssive conditions
Gap between onset of the condition and selection of sample means some individuals with the condition are no longer available due to death
Therefore sample = milder form - but generalised to whole population

113
Q

What is a membership bias

A

When membership to a group is used to recruit subjects - members may not be representative of the population due to motivation etc

114
Q

What is historical control bias

A

Subjects and controls are selected across time
Changes in definition, exposures, diseases and treatment can mean that the subjects and controls cannot validly be compared

115
Q

What is a response bias?

A

type of selection bias
Individuals volunteer for the study but differ in some way from the general population
e.g volunteers are motivated
can arise from advertising study e.g. radio / papers

116
Q

Types of observation bias

A

Interviewer / ascertainment bias
recall bias
response bias
hawthorne effect

117
Q

What is Interviewer / ascertainment bias

A

Researcher is not blinded to the subjects status in the trial which affects their approach to recording results

118
Q

What is recall bias

A

Subjects selevtively recall details from the past

Esp in case control studies or surveys

119
Q

What is response bias

A

Can occur in any study where the patient is asked questions

Patients may answer in the way they think the researcher wants to hear - rather than their true feelings ./ beliefs

120
Q

what is the Hawthorne effect

A

Subjects alter their behaviour when they are aware they they are being observed in a study

121
Q

What is performance bias?

A

Systematic differences exist between groups in the care provided
or in exposure to factors other than the intervention of interest
Reduced by
- blinding
- randomization
- standardization of care protocols

122
Q

What is attrition bias

A

When the numbers of individuals dropping out of the studies differ significantly between the study arms
Incomplete outcome data
Those at the end may not be representative of those who started and the target population

123
Q

What is a qualitative survey?

A

Gathering opinions of a group of people.
Emphasis on subjective meaning and experience
Used to study complex issues

124
Q

What is a case report

A

The experience of a single person described
Usually anecdotal
Usually cannot be replicated
Can be used to generate a hypothesis

125
Q

What is a case series?

A

The experiences of a group of people is described

Useful for studying rare diseases

126
Q

Can you critically appraise a case study or case series?

A

Not really
But consider
- Did the study address a clearly focused question / issue?
- Is the researcher’s perspective described / taken into account?
- Is the patients perspective described / taken into account?
- Are the conclusions drawn justified by the case?
- Are the findings of the study relevant to other settings?

127
Q

What is an observational analytical study

A

Reports similarities and differences observed between 2 or more groups

  • case control study
  • cohort study
128
Q

What is a case control study

A

Type of observational analytical study
Retrospective
People who have the outcome variable are compared to people who do not - to determine risk factors they have been exposed to in the past

129
Q

Advantages of a case control study

A

Cheap
Easier than other studies
few subjects required
suitable for studying rare outcomes / those with a long time lag between exposure and outcome

130
Q

Disadvantages of a case control study

A

Rely on recall or health records = source of bias
Unsuitable for rare exposures as this would require a v large sample
can be difficult to select a control group - need to be similar to the cases group

131
Q

What is a cohort study

A

Type of observational analytical study
Prospective
Group of people followed to see how development of n outcomes differs between those with an exposure and those without

132
Q

Advantages of a cohort study

A

Useful for rare exposures
can assess multiple outcomes
can assess temporal relationships between risk factors and outcomes
can directly estimate outcome and incidence rates

133
Q

Disadvantages of a cohort study

A

May take a long time from exposure to outcome
Expensive
Problems with dropu outs / loss to follow up
unsuitable for rare outcomes as would need a v. large sample

134
Q

How is a ‘retrospective cohort study’ conducted?

A

Using a pre-existing cohort
Use a historical group of patients who had an exposure and contact them now to see if they have developed the outcome of interest - avoids need to wait