Replies to post #19453 on Biotech Values

[iwfal]

I consider “noise” to be the bias caused when a trial is as balanced in all respects as a trial of the given size can be in practice, but still not perfectly balanced in a mathematical sense.

I would change the definition somewhat. Noise is that which, even in a perfectly(!!!!) balanced trial, causes the treatment to seem smaller in efficacy than it really is. The Chi Square example is a good one to illustrate this. Very close to perfectly balanced and yet it obscured the size of the treatment effect regardless of trial size.

Using this definition (which may be different from yours), “noise” for a trial is optimally balanced in a practical sense ought to decrease as the trial size increases.

I agree completely that the chance of an important imbalance goes down substantially as trial size increases. But this isn't about imbalance. Even in a perfectly balanced trial the noise has an impact on measured efficacy and p value. See Chi Square example. Really, the Chi Square example is very illustrative.

[iwfal]

Explanation of the Chi Square example:

You have 100 patients in each arm. In the placebo arm 30 'get better' just due to the attention that they get in the trial. In a perfectly ballanced trial by definition 30 of the patients in the treated arm will do the same. In addition the treatment actually helps 10 people. Assuming that this treatment effect is uncorrelated with the placebo responders that means that in a perfectly ballanced trial (30 placebo responders in each arm) you would get 30 responders in the placebo arm and about 40 in the treated arm (really should be nearer 37, but that would just make the regression effect larger).

Now assume that you find that a certain psychological profile predicts a significant proportion (2/3) of placebo response so you regress this out. Thus you are left with 10 responses in the placebo arm and 20 in the treated. The treatment effect is unmasked.