NorthWest Biotherapeutics Inc (NWBO)

[FeMike]

Busting the Bear Thesis

Gather 'round ladies and gentlemen, boys and girls. Today, we're taking a look at data manipulation and getting a lesson on 'numbers'. This class is free and may be reposted without worrying about journal embargoes.

Okay, we've all heard the trial is an utter failure when viewed through the lens of the bear/short argument. The crux of the claims they are making is based on simple number manipulation. It is quite easy to grab these numbers, interpolate them in a manner that suits your needs, and print out a poorly assembled STAT article with nothing to back your claims. So let's step through these claims one by one and see where the arguments don't quite add up.

Bear Argument #1: The Original Endpoint of PFS Failed
I cannot believe I even have to speak to this one, as it has been absolutely beaten into the ground and disproven time and time again. But, yet again it popped up in the short report and so, yet again, it must be shot down. I'm going to keep this short and sweet; if you want an explanation on progression, progression free survival, or pseudoprogression (PsPD) do some googling.

When DCVax-L works, i.e. instigates a response, the response is inflammation of the tumor. This is a signature of immunotherapy. From the NYAS presentation:

PsPD is an even bigger issue with immune cell therapies:
vaccine-induced infiltration of immune cells

For much of the duration of the trial, this inflammation was branded as progression and when it was seen in a patient another tally went into the 'non-PFS' group. For this reason, the trial was required to 'fail' on the endpoint of PFS, as at the time of the trial the response of DCVax-L was labeled as 'progression'. If we could post-hoc take the PsPD group out of the 'progressed' group and add it to the PFS group, DCVax-L would undoubtedly have reduced progression and likely passed the PFS endpoint. Alas, this cannot be done, so PsPD is doomed to stay mislabeled as progression and thus there is no scenario in which DCVax could work and not fail the original primary. That bears are saying that 'the placebo did better' is actually a good thing - if somehow the trial had shown that the treatment arm actually reduced PFS, it would only indicate that the vaccine is not getting the response that is intended.

Bear Argument #2: The Endpoints Were Changed Post Hoc
Well, this one can be short and sweet. TLDR: They Weren't
The argument here is two fold. One bear claim is that, in general, the regulatory agencies just aren't going to accept a change to the SAP. This is just false, in its entirety. ICH Guidance very clearly states:

"The statistical analysis plan may be written as a separate document to be completed after finalising the protocol. In this document, a more technical and detailed elaboration of the principal features stated in the protocol may be included. The plan may include detailed procedures for executing the statistical analysis of the primary and secondary variables and other data. The plan should be reviewed and possibly updated as a result of the blind review of the data (see 7.1 for definition) and should be finalised before breaking the blind. Formal records should be kept of when the statistical analysis plan was finalised as well as when the blind was subsequently broken.
If the blind review suggests changes to the principal features stated in the protocol, these should be documented in a protocol amendment. Otherwise, it will suffice to update the statistical analysis plan with the considerations suggested from the blind review.

What part of that statement makes anyone think that the SAP cannot be updated before breaking the blind? And, given the very logical and demonstrable reason for changing the primary AND the control, it is likely these changes are acceptable.
The second prong of this argument is that the changes were made post-hoc, i.e. after unblinding. This is definitely a no-no. Luckily for us, they weren't. The changes were originally discovered pre-data lock, so that is obviously before the blind was lifted. In addition, this FACT was reiterated during the NYAS presentation of results:

This SAP and its Endpoints were pre-specified
and submitted to regulators before unblinding.

Bear Argument #3: The RA's Will Not Allow the ECA
Alright, alright, this argument probably has the most teeth of them all. I'm not saying they're sharp teeth, but teeth nonetheless. ECA's are a new monster. Yes, recent guidance has shown that the FDA (and other RA's) are beginning to consider ECA's as a viable and useful tool for trial design/analysis. And there are examples of them doing just this. However, 'beginning to' is the key phrase here, and we simply don't have a solid foundation of trials that used ECA's on which to base the assumption that the ECA will be acceptable. For this reason, it is critical that identification of the ECA group be made with the utmost integrity. By using an independent firm to identify these comparators and assure a lack of bias, the trial accomplished this. Extensive sensitivity analyses were done, removing any comparators with even slight indications of compromising prognostic factors. By looking at the patient demographic charts on slides 18-21 of the NYAS presentation, it is clear that the control arm matches the trial demographics almost indecipherably. Also, it is clear that there was no cherry picking or post hoc data mining as the new ECA for nGBM has an mOS of 16.5 months compared to an historical mOS of <15 months. Yes, use of ECA's is pretty virgin territory, but if it is going to be allowed on any trial, it will be this one.
Bear Argument #4: The Placebo Outperformed the Treatment
Woowee, now we're getting into it! I think this is the bear argument's favorite case to make, and so it's my favorite too. And it is an argument that was made quite well, it had me scratching my head for a bit, so I imagine it is largely responsible for the doubt in the market right now. Here is where the numbers game is really being manipulated.
The argument here is that, when you combine the unknown variables of the 2018 blinded review with the known variables of the 2022 reveal, you can calculate that the control arm must have performed better than the treatment arm. Try to stick with me, I'm going to walk through these numbers a couple times as simply as I can. At first glance, this holds weight. If the blinded results showed a 3 year overall survival of ~28% of the entire population and the unblinded results show that the treatment group had an overall survival of 25%, then logically you must conclude that the reason the blinded 28% was brought down to 25% is because a higher performing group, the 'placebo', was taken out of the results. If I have only 25 lb jugs on a scale, in order to bring the average weight of the jugs up to 28 lbs I must add several jugs weighing greater than 28 lbs. There is really no other way to look at it, right? Well, hold on just a minute. This is where I find out what kind of math is being done and, luckily, the students showed me their work!

From a random bear argument I found:

Now, if blended OS at 3 years post surgery was 28.2%, and treatment was around 25%, then control had to be about 34%.

Okay, so here we have an estimated 36 month survival rate of the placebo group of 34%! Wow. This particular individual even made a joke about this, saying maybe they should be running a trial on whatever placebo this is. Haha. It's funny, because it's ridiculous. A placebo is not going to boost survival that much, obviously. This was my first tip that something isn't adding up. So let's back calculate where these numbers came from.

2018 Results: n = 331
- At ~28% survival that means ~92.68 patients were still alive.
2022 Results: treatment arm n = 232
- At ~25% survival, that means ~58 patients were still alive
Now, back calculate the 2018 results with the 58 patients in the treatment arm still alive and you get ~34 patients on placebo still alive. Subtract the treatment arm (n=232) from the 2018 trial n (331) and you get a total control arm of n=99. Can anybody guess what 34 alive placebo patients divided by 99 total placebo patients equals? Yup, the alleged 34% survival rate of the bear thesis placebo group.

So, we have confirmed the math that bear thesis uses to arrive at the conclusion of "the placebo group performed better than the treatment group" argument. And the math adds up. Unfortunately, the numbers used in the math are not based on what actually happened in the trial. Here's where it all unwinds. These numbers assume that the 2018 data looked at the original trial setup of a ~2:1 treatment:control randomization. However, the reality is that the control arm had already crossed over and 90% of patients in the 2018 review (that's n=298; which, I think is actually just a bit lower but I'm using the round 90% number let's not get hung up) had actually received DCVax L. Not the bear argument treatment n of 232. So, the 'placebo arm' that 'performed better' than the nGBM treatment arm was actually mostly comprised of the new rBGM treatment group that had indeed received DCVax-L. Their magic placebo that they recommended we run a trial on is actually...DCVAX! Wonderful! There were also outliers included, such as control group with IDH mutants that did not need to cross over, that also would have beefed up the control OS by not making it into the treatment arm. Either way, the increased proportion of patients that received DCVax-L much more reasonably explains this slight quirk in the data than saying 34% of the placebo group survived 36 months. And the rGBM patients didn't even receive the vaccine until much much later, so it can be presumed the rGBM survival curve is actually even more impressive than these results show. So, thanks bears, you just confirmed that DCVax-L is extending survival!

That's all the time we have today, folks. Cheers!