Quanticate has released several white papers around Clinical Study Design in the area of Bayesian Statistics and a focus on Phase 1 studies. Our statistical consultancy team was pleased to receive feedback and questions from our clients and piers on these papers which we would like to share with you all below.
The first whitepaper to be addressed is ‘Bayesian Study Design: The Pragmatic Solution for Phase II Clinical Development’
1. Where does the prior belief come from?
If this is the first clinical study of the new product, it may be that there is no evidence on which to base a prior belief. There are a couple of options here. One is to assume a ‘non-informative’ prior, which assumes all possible values of treatment effect are equally likely, i.e. prior ignorance. Alternatively, a prior elicitation exercise could be undertaken to establish a prior distribution for the treatment effect. This may involve designing a questionnaire for key opinion leaders to complete, to give their opinion on how they think the treatment will perform, based on the pre-clinical information and their experience of the disease. This can be incorporated with general historical information on success of trials in the disease at this phase of development, and a prior distribution can be constructed.
In cases where a non-informative prior is assumed for the test treatment effect, it may still be possible to construct a prior distribution for the control treatment (placebo or active control) which can reduce the sample size required in the control group. One way to do this is through a meta-analysis of historical trials of the disease which used the same control.
2. Where does the target effect size come from?
The target effect size is the effect that the sponsor feels would make the test treatment worthwhile developing further through to registration and the market. Typically, it is determined by a number of factors. These include the effect demonstrated by the current gold standard treatment for the disease under study, commercial factors such as input from Key Opinion Leaders as to what effect size would make the treatment attractive to them to use in clinical practice, the competitor environment and factors internal to the sponsor such as other projects in the development pipeline. The sponsor should consider all these factors and determine, prior to designing the new study, what the target effect should be.
3. How are Bayesian designs and methods viewed by regulatory agencies?
Regulators are keen to encourage the use of Bayesian methodology in exploratory studies and early phase trials, particularly for studies such as dose-finding designs. Currently, the regulators are more wary of accepting Bayesian methods in pivotal, confirmatory trials and are unlikely to accept Bayesian results as a primary analysis in such studies.
The second white paper to be addressed is ‘Bayesian Study Design: Using Interim Analyses to Improve Efficiency in Drug Development’
1. In a Bayesian adaptive design, is there a penalty to pay for including interim analyses?
In a traditional design, including interim analysis usually means that an adjustment has to be made to the significance level (alpha) to allow for the fact that there are now two (or more) opportunities to declare success. With more hypothesis tests, the chance of a false positive result increases unless this adjustment is made. The adjustment will lead to an increase in overall sample size, should the study continue for the whole planned duration.
In a Bayesian adaptive design, there are no traditional hypothesis tests, and the concept of alpha adjustment is foreign to this way of thinking. However, there is no such thing as a ‘free lunch’ and the concerns over making multiple decisions still apply. Each time an interim analysis is performed and decision criteria are evaluated, it is possible that the wrong decision could be made. When designing a Bayesian study with interim analyses, it is important to explore at the design stage the impact on the overall operating characteristics (the chances of making right and wrong decisions) of including interim analyses. It is likely that, in order to maintain satisfactory design characteristics (appropriate levels of risk of a positive and negative outcome overall) that the sample size may need to be increased when interim analyses are included, unless the impact can be shown to be minimal (such as the case study in the paper – compare table 1 and table 2).
2. What logistical planning is required when including an interim analysis?
Interim analyses, Bayesian or otherwise, should be planned at the design stage and included in the study protocol. The interim analysis should be performed by statisticians and programmers who are independent from the study team/sponsor, and the results reviewed by an Independent Data Monitoring Committee. This is so that the study blind is not compromised whilst the study is still being conducted. Considerations need to be made regarding potential biases that could affect the data in the remaining part of the study once interim results are announced, and plans should be made to minimize this. A charter should be prepared giving details of how the Independent Data Monitoring Committee will operate, including roles and responsibilities of the committee members, format, frequency and documentation of meetings, processes for breaking the blind, conducting the analyses and making the results available to the committee, and communication and confidentiality of results.
The final white paper to be addressed is ‘Flexible Phase I Study Designs: Expediting Early Clinical Drug Development’
1. Does using combination, flexible protocols lead to delays in regulatory approval?
With First-in-human designs there is often pressure on the sponsor to get the study started as soon as possible. Some sponsors are nervous about over-complicating the study design in case there is a delay in obtaining regulatory approval (and/or ethics approval). However, if the design is clearly described in the protocol (perhaps using a graphic to illustrate the flow of the study and different options) and as long as there is clarity over when data will be reviewed in-stream, to ensure the safety of subjects at all times (with stopping rules clearly specified), there is no reason why there should be delays.
2. Is it possible to report data from some parts of the study whilst other parts are ongoing What if the study is terminated before all cohorts have completed?
When a study has several cohorts, there may be a need to access results from the first cohorts when later cohorts are still in progress. For example, data from the single ascending dose cohorts may be needed for inclusion in a new protocol, IND submission, or other purpose whilst the repeat dose cohorts are ongoing. Alternatively, the study may be terminated before all cohorts have completed (e.g. on safety grounds) but the data from the completed cohorts are still of interest.
The good news is that, because each cohort is typically a set of independent subjects, once any cohort has completed (all visits including follow-up), there is no risk to the integrity of the rest of the study by unblinding and reporting the data from the completed cohorts. Each cohort is essentially a ‘mini-study’ in its own right. When each mini-study is complete you can report it and use the data. It is purely for convenience and time saving that several mini-studies have been grouped under one protocol. There may be value in keeping some cohorts together (e.g. if you have two interlocking cohorts for single ascending dose, it would be sensible to wait until both have completed before you unblind and report the data) but otherwise you can report data upon completion of each cohort. However, this must be transparent in the protocol – it is important to make allowance up front for early unblinding and reporting on a by-cohort basis.
