The trial design didn't suck. It was small number of subjects. The smaller the number of subjects increases the potential variability of the results.
Had there been say several hundred subjects and an even split between placebo and dosed subjects the probability of an unequal representation of mild and moderate Rett subjects in either the drug arm or the placebo arm would be reduced.
Having a larger number of subjects with a rare disease like Rett makes getting the required number of subjects in the trial more of a problem. It also costs more money and it takes longer to get the trail run. So trial design is always a question of tradeoffs.
The best trial would be to enroll every Rett girl on the planet. Then there is absolutely no question about how well the drug works. Of course that is not possible or desirable.
So a smaller segment of the overall population of Rett girls is enrolled in the trial. Now we have to use statistics to see how well the sample of the Rett population represents the overall Rett population. If the sample is very large then the sample is less likely to not be representative of the overall population.
Lets take the extreme case and just use 1 subject for the test. How do you know if that one person represents the overall population? It is unlikely that one person represents the the wide range of Rett impact on every Rett girl. So now we need to have more subjects in the trial to get a more representative sample of the overall Rett population. Here is where sample statistics come into play. The larger the sample the smaller the odds are that the people selected for the trial don't represent the over all population.
This is really what the p value is about. The p value indicates that if you took say 100 different samples from the overall Rett population that for a p <0.05 95 times out of 100 the results you get from the trial represent the actual drug effect and not some random bias in the trial by selecting only people that don't respond to the drug.
A placebo group helps to test the selection bias and other types of biases that can show up in a trial by hopefully having a placebo group that matches the drug group in as many ways as possible. The theory is that the two groups are equal and so the differences between them represent the drugs real effect. In most cases that is how it works.
However, just like the problem with selecting subjects for the trial, selecting subjects for the placebo group can have the same issues of the placebo group not being representative of the over all population and more importantly not exactly matching drug group. If the two two groups are not equal then that introduces a bias into the statistical analysis and you might not get true results.
Subjects are randomly assigned to either the drug group or the placebo group which hopefully eliminates any potential bias. But it is a random process which means it is not always going to be bias free.
Just like flipping a coin. Over many flips the coin will be about even number of heads and tails. But in a small number of flips you might get 10 or more consecutive heads in a row throwing off the appearance of a fair coin.
One of the more interesting examples of this was in a stat course I took. The instructor broke the class into two groups. One group flipped a coin 100 times and wrote the results on one board. Then on another board he had the students make up what they thought 100 flips would look like and he left the room saying that when he came back he could tell which group made up the number and which group actually flipped the coin.
He came back to the room and immediately pointed to the real coin group. When he was asked how he knew which was which, he pointed to the variability of the real coin flip. There were many more strings of head or tails in a row than in the made up group.
Statistics is never having to say you're certain.
Success is the best revenge.
AI
