AVII: "Doktornolittle You might consider reviewing some (all?) of these videos
EMA workshop on multiplicity issues in clinical trials
-----------------------------------------------------------
A few minutes into the first lecture I get the gist of the issue. That is a broader issue than what I brought up in response.
In fact, early on the speaker states that setting up multiple efficacy goals in advance avoids the problem, and it is not clear to me that even fully addresses the issue.
If you don't setup the multiple goals in advance, then that certainly introduces a major problem if you, for example, search scads of subgroups for an efficacy win after the fact. That is related to my example of being able to chose when efficacy is high in a running average, if you had a running average for efficacy, even for the whole trial. Being able to select when the noise was in your favor.
But this multiplicity issue is broader. Not just deliberate selection of statisitical outliers, after the fact, but even setting up "Many Chances to Win" in advance, increases the odds of one of them winning by chance. Which is where the lecturer started on page 1.
Clearly very important stuff. I feel lucky to understand the gist, but I'm anticipating this getting very confusing as it goes deeper.
I have a masters in DSP, though it is ancient and unused. DSP splits between unrelated analytic and stochastic realms. I kicked butt in the analytic realm, but the stochastic realm never fully jelled. There was no prep class available, so I had to start at the graduate level in stochastics, which I didn't want to do, and which did not work out for me. But I do get the gist of this multiplicity issue. Fortunately, it is intuitive to some degree. Not that even the simple example of two 50% population subgroups showing the same level of efficacy is entirely clear yet. Probably comes down to an apriori vs not issue. Apriori what...?
Part of the answer must be that if you setup multiple ways to win in advance that this must also be multiple ways to fail, the entire trial. No... can't quite be that. I know they are going to be talking about alpha spend for this... but.... back to the lecture.
