Definition of Uncertainty?
Probability that they will happen and will not happen … intrinsic to economic evaluation (impossible to remove)
How does uncertainty arise?
- methodological disagreement (amongst analysts and commentators) when it comes to comparing results of studies that have employed different methods
- data requirements of the study in terms of the estimated resources and health outcomes
- the need to extrapolate (generalise) over time/region
- patients do not act the same way as consumers
Definition of Parameter Uncertainty?
Uncertainty about the true numerical values of the parameters used as inputs’ in an economic evaluation model
Definition of Methodological Uncertainty?
Model Structure uncertainty = concerns how the elements of the model are fitted together
Modelling Process uncertainty = results from decisions made by the analyst
Definition of Sensitivity Analysis?
A set of techniques that seek to analyse how sensitive results are to uncertainty … how sensitive results are to changes in a model
What is Sensitivity Analysis?
Involves systematically examining the influence of uncertainties in the variables and the assumptions employed, on the estimated results
- varying parameter estimates across a range … seeing how this impacts on the results
What are the two types of models?
- DETERMINISTIC MODEL: a model where discrete changes are made to the model … The parameters are deterministic (i.e. they have one value within an analysis)
- e.g. one-way or multi-way sensitivity analyses - STOCHASTIC MODEL: a model with a probabilistic sensitivity analysis … The parameters are stochastic (i.e. there is uncertainty about their value within an analysis)
- Based on distributions
- All of parameters in model have to have distribution attached to them … decide which distribution attached to the parameters … find suitable standard error … put them into the model
Definition of One-Way Sensitivity Analysis?
LOOKING AT HOW SENSITIVE RESULTS ARE TO CHANGES IN ONE PARAMETER … impact of changing one parameter on the results
- how robust are the results to changes in one of the variables?
How does One-Way Sensitivity Analysis work?
Choose a variable → change that variable → look at the results
Which parameters do we change in One-Way Sensitivity Analysis?
Only relevant or very uncertain parameters are usually explored … those parameters which could have an impact on the results
The alternative values can be:
- Highest to lowest if a range of estimates is available
- Confidence intervals of the parameter (if reported)
- Plausible range: e.g 20% higher / lower
- Informed by expert opinion
Advantages of One-Way Sensitivity Analysis?
May be helpful to isolate important model parameters – gives insight into factors influencing the results
Can provide a validity check to assess what happens when particular variables take extreme values
Simple to understand and undertake
- “One-way sensitivity analysis has its greatest value in developing and reviewing a model” (Drummond et al., 2015)
Disadvantages of One-Way Sensitivity Analysis?
Likely to underestimate uncertainty … only changing one parameter
Doesn’t take account correlation between the other parameters in the model
Cannot indicate what parameter contributes the most to uncertainty
Definition of Multi-Way Sensitivity Analysis?
LOOKING AT HOW SENSITIVE RESULTS ARE TO CHANGES IN MULTIPLE PARAMETERS … impact of changing multiple parameters on the results
ADDING CORRELATION BETWEEN PARAMETERS
How does Multi-Way Sensitivity Analysis work?
Can look at combinations of parameters
- changes to a combination of parameters might make results sensitive
Which parameters do we change in Multi-Way Sensitivity Analysis?
E.g. setting all parameter estimates at their highest or lowest bounds simultaneously giving the most pessimistic or optimistic scenarios
Advantages of Multi-Way Sensitivity Analysis?
Can look at combinations of parameters … more likely to have an effect on results
Disadvantages of Multi-Way Sensitivity Analysis?
Difficult to interpret correctly … require judgement about the joint probability of two parameters taking values greater than their threshold values simultaneously
What are the limitations to One-Way and Multi-Way Sensitivity Analysis?
- Selection bias … choice of parameters is discretionary
- Interpretation arbitrary … no guidelines as to what degree of variation in results is acceptable (Drummond et al, 2005)
- Impossible or time consuming to use effectively in large/complex models
Definition of Threshold Sensitivity Analysis?
Values that parameters would need to take for cost and effects to change sufficiently to alter the decision about which alternative offered the highest expected net benefits
When is Threshold Sensitivity Analysis useful?
For variables such as cost or effectiveness of a drug
- Up or down to where can take the price to keep it cost-effective
Definition of Extreme Sensitivity Analysis?
Each parameter is set at extreme (but plausible) upper and lower bounds … the difference in costs and effect over this range is recorded
How do we judge the quality of Sensitivity Analysis?
- Identify the uncertain parameters
- Specify the plausible range … review the literature, get expert opinion, use CI around the mean
- When reading literature, identify if an appropriate form of sensitivity analysis was applied
Definition of Probabilistic Sensitivity Analysis?
Changes all parameters in the model at the same time to account for correlation between model parameters
STOCHASTIC MODEL: The parameters are stochastic (i.e. there is uncertainty about their value within an analysis)
What do all the parameters in a Probabilistic Sensitivity Analysis have?
All of parameters in model have to have PROBABILITY DISTRIBUTION attached to them
