Drug 1: Another NSAID that shows pain relief N=2000 , p=.05
Drug 2: Improves OS (HR=1.8) in stage 3 NSLC N=100, p=.06
Which one should be on market?
The issue is whether the randomization principle holds up.
For the larger trial, randomization likely ensures that there won't be any serious imbalance in any prognostic factors including possibly hidden ones (ie, ones we haven't thought of yet).
For the smaller trial, you would need to worry about imbalances. But that is why it is important to do a variety of sensitivity analysis as the FDA statisticians always do. But remember that this is clinical data and often there are also other sources of data outside of the immediate trial that can provide corroborative evidence.
So, if we can assure that no imbalance could influence the p value computation, then the smaller trial would be a better gauge of the drug benefit than the larger one. The difference between .05 and .06 is just noise.
