Biotech Values

[iwfal]

While most designers of these trials will tell you there really is no alpha spend, in my experience that is never completely accurate. Whether the threshold HR goes down the more looks you get or the implied p-value goes down, there is some sort of penalty for multiple looks.

In general, however, the adaptive designs allow more looks at a smaller alpha penalty than a traditional design with multiple interim looks.

While I realize that you will never believe me, nonetheless it should be stated that adaptive trial designs do not save alpha spend except to the extent that:

a) The regulatory agency allows one trial to substitute for 2. And, of course, by statute this is possible even without adaptive trials.

b) The adaptive trial is Baysean in nature - but of course the regulatory agencies do not accept Baysean trials.

Nor do the large number of formal papers I've seen (or the book I have) claim saved alpha. In fact they talk extensively about making sure they are mathematically precise and correct for their Type I error calculations- math buzz words to say no alpha free lunch.

That said - I fully agree that some statisticians use english in such a way as to make it sound like alpha is saved. E.g. the YMI trial. But in fact that is a perfect example since when I laboriously converted HR endpoints for YMI into alphas (for an earlier conversation on this topic) it was exactly the same as a total allotted alpha of 0.05 (or less - I no longer remember).

So, if not an alpha savings, why use an adaptive trial design? Because:

a) They are cool and funner (for a statistician). Never underestimate the desire of technical people to do something that isn't actually in the customer's interest just because it is more fun/interesting

b) They allow you to effectively run two trials (or more) and call it one - i.e. not have the shut down and start up processes for each trial. And you cannot complain about the PR of having just one trial. But the penalty is that they are predefined so unexpected post hocs cannot be incorporated in the adaptive plan. E.g. If efficacy seen is X, then increase enrollment to Z. Or, for a different trial, if one of three subgroups does substantially better then stop enrolling the other two subgroups for remainder of trial (and p value will be for only good subgroup - but with p value penalty for having looked). The alpha calcs can get mindbogglingly complicated for this stuff precisely to avoid forgetting to compensate for an alpha spend.

PS I would lump adaptive trials into the same complexity bucket as Cox Regression. Complicated with lots of knobs that have to be adequately pre-specified. But if the FDA allows one they ought to allow the other.