Replies to post #444247 on Anavex Life Sciences Corp (AVXL)

[frrol]

Your understanding of drug trial SAPs is poor.

[Anshu2]

Most trails have only one primary endpoint (i.e, what is measured and what statistic/measure is used to compare; the latter is mostly the "delta from baseline to trails-end). In P2b/P3 case, CM never disclosed how the two primary co-endpoints Cog and ADL endpoints will be compared, but at CTAD came up with ORs as the way to do it (which he later changed to LSMs recently).

P2b/P3 was unusual -- in that, it had two primary endpoints. That's actually putting unnecessary burden on yourself -- since you need to pass both with p < 0.05 -- so I fail to see the point of having two primary endpoints.

However, it seems statistically reasonable (but I have never seen before -- so, not sure if FDA will agree) to consider the trail a success -- if any one of the two co-primary endpoints passes with p < 0.025. This is what AVXL did recently with delta/LSMs as the comparison measure.

Point: Irrespective of SAPs, the primary endpoint was known. How it should be compared is also pre-specified (and, in almost all cases, is disclosed to the public). That is what defines true success in a pivotal trial. Anything else -- is ad hoc analysis and puts trial success into a question mark (but doesn't necessarily mean an NDA rejection).

[Joseph_K]

Steady, that's just not how it works:

Doesn't matter where the Odds Ratio ranks in the SAP, if such a ranking exists.

I know this from past conversations with my biostatistician friend. A SAP's first-listed statistical analysis for an endpoint is given much more weight by the FDA (and I presume the EMA) than those that are lower in the hierarchy. They may pay attention to the lower-listed tests if there is lots of other evidence in the trial that inclines toward approval of the drug, but there is a very strong preference that the first-listed test be passed. She's also said there is no way that difference of the means would not be the first-listed test for all three of the endpoints for the types of endpoints in the P2a/3.

(Starting here it's all me without my friend's input.) If a trial sponsor pre-specifies various statistical analyses, it creates a problem called multiplicity. Choosing one of various ways to analyze data ad-hoc is one of several means of doing what's called p-hacking. This problem of multiplicity is controlled in part by putting the onus on the sponsor to first-list the analysis it wants the FDA to depend on. If the sponsor can choose among different analyses, it has a much better chance of finding a significant p-value; but the other analyses are actually showing the trial failed on that endpoint.

Here are a couple quotes to help give a sense of the problem:

[T]here are many different ways to implement multiple imputation, such as including different variables in the imputation model and imputing under different statistical models. Therefore, this approach to pre-specification is ineffective, as it still allows investigators to analyse the data in many different ways before deciding on a final approach. This issue of ‘incomplete’ pre-specification, where methods are pre-specified to some extent but the specification still allows for some degree of p-hacking, is common in clinical trials.

https://bmcmedicine.biomedcentral.com/articles/10.1186/s12916-020-01706-7

The second discrepancy, termed an ‘addition’, occurred when the original analysis plan gave the investigators flexibility to subjectively choose the final analysis method after seeing trial data. This could occur if the original analysis plan (i) contained insufficient information about the proposed analysis or (ii) allowed the investigators to subjectively choose between multiple different potential analyses.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7257229/

The football analogy doesn't work. I can't come up with a good analogy, but this is closer: I say I have a great way to pick stocks and I list ten that will go up over the next month, and then I want you to confirm my method by looking at only the winners.