Biotech Values

[iwfal]

In a sense, the result here is driven by the 4 control patients... if by chance they're a tightly knit group of 4 with very similar (but somewhat atypical) outcomes (ie. low st. dev.), then they could be the driving reason for the statistical significance.

Finally figured out what bothered me about this sentence. It is incorrect in the sense that it matters not at all whether the 4 are 'tightly knit' since, given that they were chosen randomly then 95% of the time they would be no more tightly knit than the remaining 12. I realize this is counter-intuitive, but you'll just have to trust me (unless you try the intuition building suggestion below). And the odds of you picking 4 patients who happened to be skewed from the remaining 12 is 0.05 (or whatever the p value was).

You are making the same mistake as biomaven made - that the distribution matters. It doesn't - any possible distribution is enveloped in the p value calculation. I would strongly recommend to everyone that they build a spreadsheet and play with some of the simpler non-parametric tests - e.g. it is possible to gin up an MC spreadsheet on the statistics used for ORR p value calculation in less than an hour. And no matter what distribution you assume for responses (bimodal, tri-modal, heavily skewed, flat, ...) as long as the drug is a null then the p value will get less than p=0.05 only about 5% of the time in an MC run of thousands. {Note that if the FDA allowed distribution assumptive calculations then for ORR (or HCV viral levels or ...) everyone would probably do a difference of the means test. But instead they do a responder test because it is non-parametric.}