For those who are interested: By how much does 80 exceed 60 etc, my current thoughts on RI and the 80% IA.
Many of us have been scratching our heads about what to expect from RI and the 80% interim analysis (IA). I recently looked at some relevant issues.
When any trial measures a single quantity, such as the RI primary endpoint, it forms an evolving estimate of the true value that becomes more reliable and precise as the trial progresses. At any point, the estimate is about equally likely to be higher or lower than the true value. The odds of the estimate being significantly low are less than 50% (say about 40% for the sake of discussion).
When an early stop depends on sufficiently high estimates of multiple endpoints, the odds change dramatically. With three endpoints, for example, the odds of the estimate of at least one being significantly low go from 40% for a single endpoint to about 80% (100 x (1 - .6^3) = 78.4). That is, from probably no problem, to very likely a problem. We don't know details of the RI early stop criteria, but there are multiple secondary endpoints (SEs) involved. They could include the number of hard events vs soft, performance for key subgroups, etc. The criteria are unknown and loosely defined, but I think my illustration here is generally relevant.
So, we are dependent on the passage of time to tighten up all the estimates so they meet their requirements with no stragglers. Which raises the question: is 80% enough greater than 60% to make much difference? A relevant issue is the startup period with reduced drug benefit due to the delay for V to reach full risk reduction. This is an unknown quantity. We do know some V benefits are fairly quick (blood thinning is fast, and the Cherry study reported measurable cap thickening within a few months), but some JELIS graphs seemed to show arms separating after a couple years or so, although roughness in the placebo arm render the timing less than conclusive IMO. The most relevant JELIS subgroup had too few subjects for smooth arms, and therefore inconclusive on the length and impact of the startup period IMO. Not having a reliable measurement of the length and impact of the startup period does not mean it's not real and significant. It has the effect of slowing the demonstration of efficacy.
The longer the trial progresses, the smaller the fraction of patient-years that were part of the startup period. Additionally, subject recruitment standards were tightened partway through, in a way expected to show more benefit for later recruits, and their fraction of the total patient years increases with time. So, the effective value of the added time from 60% IA to 80% IA is more than just the 33% by which 80 exceeds 60, because the more recently accrued patient years are better on average.
To roughly quantify that, let's assume for the moment that the value of a subject is 50% during the first year, 75% during the second year, and 100% after that. Weighting the patient-years accordingly and running the numbers, I find that at the 80% event, the adjusted patient-years were 35.9% more than at the 60% IA, when the increase would only have been 33% without the startup adjustment. If I additionally increase by 10% the value of patient-years from subjects recruited after the criteria were tightened, that further increases the 80% IA patient year to 36.5% more than the 60% IA. Stronger adjustments (33% and 66% startup values in the first two years and 20% premium for later recruits) yielded 37.7% increased adjusted patient years at 80% IA vs 60% IA, which is still only moderately more than the unadjusted 33% by which 80 exceeds 60. Even the extreme adjustments of 0 value assumed for the first two years and a 30% premium for patients recruited after Q2 of 2013, only give the 80% IA 40.5% more adjusted patient years than the 60% IA (vs 33% with no adjustments).
Since the modeled effects of startup periods and tightened recruitment standards are moderate at most under a wide range of assumptions, I am currently leaning more toward the SEs as probably the dominant factors influencing the continuation at 60%. Note that this does not preclude high PE efficacy. These results also helped restrain my estimated odds of stop for efficacy at the 80% IA, which I currently put somewhere around 40%-55%. (No guarantee, insufficient info for a precise estimate.) There are plenty of reasons to be optimistic about the PE efficacy. It's very possible we just barely missed stopping at 60%. Or not. One subgroup could be less than stellar, or there could have been too many "soft" events. I remain convinced the final outcome will be very successful, with strong PE efficacy, and interim stop at 80% remains a very real possibility IMO. As always, no guarantees.
