First, I noticed something interesting from the PR yesterday. They said the following under the PD Clinical Data:
"2.) Changes in 4 out of the 8 secondary clinical outcome measures (UPDRS ADL, Schwab & England, Hoehn & Yahr and MOCA) at week 2 (visit 6) were statistically significant at the one-tailed 10% level."
A one-tailed study would only be measuring one direction of the data and would greatly inflate the data numbers. Of course BCLI and CUR are doing two-tailed studies, measuring both directions of the effect of the treatment. Presenting one-tailed data to the FDA would almost certainly result in the FDA saying "great now do it again in a two-tailed study".
Here is a good paper on the difference between one-tailed and two-tailed studies:
www.ats.ucla.edu/stat/mult_pkg/faq/general/tail_tests.htm
(excerpt)
"When is a one-tailed test NOT appropriate?
Choosing a one-tailed test for the sole purpose of attaining significance is not appropriate. Choosing a one-tailed test after running a two-tailed test that failed to reject the null hypothesis is not appropriate, no matter how "close" to significant the two-tailed test was. Using statistical tests inappropriately can lead to invalid results that are not replicable and highly questionable--a steep price to pay for a significance star in your results table!"
Genervon simply wanted to prove that there is a positive effect in one direction from their treatment. After checking clinicaltrials.gov, I confirmed this as it is indeed just a "pilot trial":
"GM604 Phase 2a Randomized Double-blind Placebo Controlled Pilot Trial in Amyotrophic Lateral Disease (ALS)(GALS)"
(Description of the trial)
"1. This pilot trial is designed to test proof of principle, i.e. determine if a 2-week IV bolus treatment with this agent can (1) change ALS protein expression (target biomarkers and efficacy biomarkers) after treatment (2) have preliminary effects measures of ALS disease clinical progression."
What is a pilot trial?:
m.circ.ahajournals.org/content/119/13/1694.extract
"Pilot trials are exploratory studies limited in size and scope that give insight into the actions, efficacy, and safety of a drug or device, but cannot provide definitive support for specific mechanists or therapeutic claims."
So to make a long story short, Genervon's trial seems to be a one-tailed pilot study that is only measuring one direction and cannot really be used to fairly and accurately measure statistical significance or make therapeutic claims. Of course that is fine as it is just a pilot study, but I think it is a little misleading in the way they touted the results.
Would love to hear any feedback or corrections if I am wrong in my analysis of this.
