The PARS Disease:
Very sincere and knowledgeable longs are just a symptom of the PARS disease - not the root cause. The real problem was that Pharmos had plausible PII data, which showed a trend for the primary end point, and was statistically significant for ICP, an important prognostic indicator. But then the PIII showed that the drug actually had absolutely no effect on ICP, or the primary end point either.
How do you protect yourself from this kind of result, where the PII data looks good but turns out to be a stastical fluke? This should hardly ever happen, but instead it's very common in biotech. So far I've been invested in 3 late stage trials that showed absolutely no effect for the drug being tested: An ISIS antisense cancer drug (PIII), Cortex's MCI drug CX-516 (PII), and Pharmos' Dexanabinol (PIII).
My explanation is survivorship bias: Take 1000 compounds and put them through a PI. Just by chance 100 will show efficacy at a p value of 0.1. Take those 100 compounds and run them through a PII. Just by chance, 10 will show efficacy at a p value of 0.1. Then take those 10 remaining compounds and put them through a PIII and try to prove efficacy at a p value of 0.05. Almost all of them will fail.
How can you protect yourself against survivorship bias?
