Replies to post #119883 on Biotech Values

[DewDiligence]

…it's inaccurate to suggest a SPA on an adaptive design is less meaningful than a SPA on a traditional design.

I stand by the statement in my prior post—there are simply more things that can go wrong (in the opinion of FDA reviewers) to invalidate an SPA in an adaptive trial than in a conventional one. When there are sufficient data on this to make a meaningful comparison, I’m pretty sure my contention will be borne out.

Having an SPA is always better than not having one, of course.

[iwfal]

While most designers of these trials will tell you there really is no alpha spend, in my experience that is never completely accurate. Whether the threshold HR goes down the more looks you get or the implied p-value goes down, there is some sort of penalty for multiple looks.

In general, however, the adaptive designs allow more looks at a smaller alpha penalty than a traditional design with multiple interim looks.

While I realize that you will never believe me, nonetheless it should be stated that adaptive trial designs do not save alpha spend except to the extent that:

a) The regulatory agency allows one trial to substitute for 2. And, of course, by statute this is possible even without adaptive trials.

b) The adaptive trial is Baysean in nature - but of course the regulatory agencies do not accept Baysean trials.

Nor do the large number of formal papers I've seen (or the book I have) claim saved alpha. In fact they talk extensively about making sure they are mathematically precise and correct for their Type I error calculations- math buzz words to say no alpha free lunch.

That said - I fully agree that some statisticians use english in such a way as to make it sound like alpha is saved. E.g. the YMI trial. But in fact that is a perfect example since when I laboriously converted HR endpoints for YMI into alphas (for an earlier conversation on this topic) it was exactly the same as a total allotted alpha of 0.05 (or less - I no longer remember).

So, if not an alpha savings, why use an adaptive trial design? Because:

a) They are cool and funner (for a statistician). Never underestimate the desire of technical people to do something that isn't actually in the customer's interest just because it is more fun/interesting

b) They allow you to effectively run two trials (or more) and call it one - i.e. not have the shut down and start up processes for each trial. And you cannot complain about the PR of having just one trial. But the penalty is that they are predefined so unexpected post hocs cannot be incorporated in the adaptive plan. E.g. If efficacy seen is X, then increase enrollment to Z. Or, for a different trial, if one of three subgroups does substantially better then stop enrolling the other two subgroups for remainder of trial (and p value will be for only good subgroup - but with p value penalty for having looked). The alpha calcs can get mindbogglingly complicated for this stuff precisely to avoid forgetting to compensate for an alpha spend.

PS I would lump adaptive trials into the same complexity bucket as Cox Regression. Complicated with lots of knobs that have to be adequately pre-specified. But if the FDA allows one they ought to allow the other.

[DFRAI]

BSR_David

The chief worry on the ONCY trial is not the adaptive design. It is the fact they entered a combination Phase III with zero randomized experience with the combination in this disease and very few patients at all treated with this combination in this disease stage.

I dont quite understand your statement above. Randomized experience - does one need experience to administer the same drug to different patients?

I think its a question of how comfortable you are that your drug works and it seemed to them that it may work best in this type of cancer (maybe too simplistic)

Thanks for the adaptive explanation