Interim analyses are widely used in clinical trials. They offer companies the opportunity to stop studies early in cases where it looks as if the primary objective will not be achieved (often referred to as ‘stopping for futility’).
This can reduce the overall cost of the trial, allows companies to focus resources on other projects, and also has an ethical benefit, since fewer patients are given a treatment that is not effective. They also offer the opportunity to stop studies early when there is clear evidence that the primary objective has already been met (often referred to as ‘stopping for efficacy’ or ‘stopping for success’). This can again reduce the cost of the trial, and speed up the time it takes to develop the compound, which benefits both the company and ultimately patients, since new medicines become available more quickly.
Interim Analysis in the Bayesian Framework
The Bayesian framework lends itself quite naturally to application in interim analysis in clinical trials. The basic structure of the Bayesian approach is as follows:
i) I have a belief about the likely magnitude of effect of the compound, and I am able to express how confident I am in that belief (Prior belief)
ii) I gather some data to explore what the likely magnitude of effect of the compound might be (Likelihood)
iii) Based on the data I have collected, I update my belief about the likely magnitude of effect of the compound (Posterior belief).
Once we have posterior belief (step iii) we can used this as prior belief (step i) for the next study, or stage of the study, and go through the steps again. Hence when applying Bayesian approaches to the area of interim analysis, we simply go round this cycle. The first time round, our prior belief may be that of ignorance (i.e. we do not have any belief about the likely magnitude of the treatment effect) or it may come from previous studies with the compound. We then collect some data, up to the first interim analysis, from which we derive our first posterior belief. This becomes our prior belief for the next cycle, we gather some more data, conduct another analysis combining our new prior belief and our new data, and update our posterior belief. And so on and so forth, until we have enough posterior belief to conclude that either our compound is effective, or that collecting more data would be unlikely to result in success (i.e. futile). At each interim analysis, there is also the option to make adaptations to the clinical study design based on our belief at that stage.
How should a clinical development team decide whether they have gathered enough evidence to stop the trial? There are two simple approaches that are straightforward to use.
The first is to calculate, at the interim analysis, the posterior probability that the true difference between the test compound and the control is greater than the target effect. This can answer the questions ‘what is the probability, given the data we have so far, that our test compound is efficacious?’ and ‘what is the probability, given the data we have so far, that our test compound delivers the effect that we need?’. If these probabilities are sufficiently high (or low) then the study can be stopped and no further data need to be collected.
A second approach calculates, at the interim analysis, the predictive probability of achieving a successful result at the end of the study. This can be particularly helpful if statistical criteria for determining success/failure of the study have been clearly defined. The predictive probability answers the question ‘What is the probability, given the data that I have gathered so far, and the planned additional number of subjects to be recruited, that at the end of the study, I will meet my criteria for success?’. If this predictive probability at the interim is low, then the study can be stopped for futility, since there is enough evidence to show that it is unlikely the study will be successful if it is continued through to the end.
With both approaches, it should be noted that there is a risk that a wrong decision could be made at the interim. This could result in the study being stopped for futility when actually in truth the test compound is effective. Alternatively the study could be stopped early for success, but in truth the test compound is not effective, leading to failure later in development. It is therefore important before the study begins both to agree what the planned stopping thresholds will be (i.e. the amount of evidence, in terms of posterior or predictive probability, that is required for a ‘stop’ decision), and to assess up front the risks of making wrong decisions. This can be done using simulations to show how often the selected decision rules will lead to ‘stop’/’continue’ decisions for different possible values of the true treatment effect.
Concluding Thoughts
The concepts of posterior probability and predictive probability are intuitive for making interim decisions about continuation or early stopping of a study. Simulations of proposed designs can help assess the performance of different decision rules and assist in the determination of the sample size and timing of the interim analysis. Using these methods, and Bayesian adaptive designs in clinical drug development can result in efficient studies that make the best use of resources whilst ensuring good chances of success.
A more detailed article available by clicking the button below.
Related Blog Posts:
