Replies to post #120021 on Biotech Values

[iwfal]

inferentially seamless adaptive designs do not require alpha spend, but operationally seamless sometime do.

They ALL require adjustments/design to "preserve integrity of the type 1 error" (or "alpha adjustments" or ... ) unless they are Bayesian (i.e. bring in aprioris). See, for instance:

http://www.biopharmnet.com/doc/2008_01_10_webinar.pdf

Note that, just as I predicted for David, the above cite, which is about seamless trials, talks of preserving the validity of type 1 error. And it was the first google link I looked at on seamless adaptive.

That said, it is a little like a mathematical game of 3 Card Monte - it is very difficult to track the alpha through the incredible complexity of an adaptive trial. But you'll find all papers on adaptive trial designs talk of maintaining the integrity of the type 1 error (or the equivalent).

If you still believe otherwise point me to a trial design which you believe gets something for nothing and I'll dissect it for you (but it may take several weeks - as I said the math is an unbelievable bear).

THe FDA was weary but did approve after reviewing thoroughly the simulation plans. Simulation plans and adaptive designs plans are however all based on simulation. And very frequently the real word trial does not mimic the pre-specificed simulation that the FDA agreed upon.

As I noted in my response to David the problem with adaptive trials is that you can't implement an unanticipated post hoc which you could do if it were two separate trials. E.g. 'the real world does not mimic the pre-specified simulation that the FDA agreed upon'

PS I suspect (but cannot confirm) that the GNVC trial that failed last year was an adaptive trial - and their final alpha was fairly low as a result.

[iwfal]

Adaptive trials - when is an alpha send not an alpha (i.e. why adaptive trials are like 3 card monte with the disappearing alpha - or why you have to look really carefully when someone talks of alpha magic).

Take the case of a trial where:

1) They want to test 4 different drug protocols against one another in part a

2) After enrolling and randomizing 100 patients into those 4 arms (non placebo) they then measure PFS and

3) for part b they take the best protocol forward to be testing in 200 patients randomized 1:1 for placebo vs the best protocol.

4) In the final analysis they want to use not just the 200 patients - but also the 25 patients that were in the carried forward protocol in part a. I.e. it ends up being 125 patients in the treatment arm and 100 patients in the placebo arm.


Note that there is no p value comparison against an assigned alpha at the end of part a (because there was no placebo group), but because you looked, you acted and you want to use those patients in the final analysis the final alpha will not be 0.05. I.e. there was an alpha spend even through there was no 'alpha' 'used' at the 'interim'.

And more complicated adaptive trials get a lot odder/messier.