Replies to post #799453 on NorthWest Biotherapeutics Inc (NWBO)

[seekinganswers]

You go ahead and believe that. LOL--

https://www.sciencedirect.com/science/article/pii/S0035378723009190?via%3Dihub

3. A problematic built-in crossover
Crossover generally describes patients in the control arm receiving the experimental therapy upon progression. When approaching trial interpretation, there are situations where crossover is recommended, while in other cases, it may pose difficulties [7].
Comparative trials can be divided between two categories. One category is trials testing the optimal sequencing of a molecule: in those, the therapy has already proven to be beneficial (for instance in later lines of therapy) and is investigated in earlier lines (including neo or adjuvant settings). In those trials, crossover is important because the question is whether earlier treatment is better than later.
Another category, in which the DCVax-L trial falls, is trials testing the fundamental efficacy of a new therapy. In those studies, before the trial is run, it is unknown if the treatment may yield clinical benefit or not. In those situations, crossover should be avoided since it can prevent an accurate evaluation of survival results [7].
Consider the scenario when no survival difference can be demonstrated between arms in cases of problematic crossover. One of three explanations may be similarly plausible:
•
the experimental treatment has true efficacy, but the survival benefit was masked by the crossover;
•
the new treatment was detrimental, and a survival decrement was masked by the crossover;
•
a neutral effect of the experimental arm can also remain uncaptured.
In the DCVax-L trial, the crossover should not have been permitted. Because 90% of control patients crossed over to the experimental therapy, survival could not be reliably analyzed, in the admission of the authors themselves.
4. Removing PFS as the primary endpoint: no satisfactory explanation
The authors state that they removed PFS as the primary endpoint because estimates were unreliable due to pseudoprogression, which could occur in patients receiving the vaccine product. The modified Macdonald criteria, which cannot distinguish between true progression and pseudoprogression, were used. However, pseudoprogression had already been described before the study began, [8] and the RANO working group updated the criteria in 2010 to address the issue [9]. Considering that 91.5% of patients were enrolled between 2012 and 2015, accounting for pseudoprogression in the PFS assessment could have been done with an amendment before most participants were enrolled. Lastly, one could ask why it took almost 13 years after the first enrolled patient to decide PFS was no longer a valid endpoint in this trial, which would be a major deviation from the original design. We are concerned that changes in endpoints were made because PFS was ultimately negative.
5. Limitations of external or synthetic controls
Randomization is certainly one of the major scientific advances of the 20th century [10]. In biomedicine, where most interventions may have modest effect size, [11] examples are numerous where historical or observational data supported practices that were later reversed by properly conducted randomized trials [12]. Randomization allows for three key methodological strengths: balancing known and unknown confounders, set a time-zero, and avoids multiple testing analyses [13]. Observational data, historical control, or external controls, because they lack those strengths, are susceptible to fundamental biases that cannot be fully corrected after the fact. Last, contrary to some may suggest, the use of synthetic or external control arms may not actually lead to faster or cheaper clinical research [14].
6. Selection bias and confounders
An increase in the extent of surgical resection has been associated with longer survival in patients with glioblastoma [15]. It is important to note that the DCVax-L trial excluded patients who only underwent a biopsy as their initial surgical management, thus selecting for patients possibly bearing a better prognosis. In contrast, in trials retained for external controls in the newly diagnosed setting, 4 out of 5 trials, constituting 97% of external control patients, could include patients undergoing biopsy as their initial surgery. These patients likely had more advanced disease or were not deemed fit enough to undergo optimal surgery. In other words, due to a key inclusion criteria, patients included in the DCVax-L trial were likely to have a more favorable prognosis than patients selected in external control arms.
The vaccine trial specifically excluded patients with early progression, i.e., patients with known adverse prognostic features. However, 2 out of the 5 trials used for external controls in the newly diagnosed setting did not explicitly exclude those patients. The 2 trials represent more than half of the patients (52.7%) assembled for the survival external comparison. This pitfall was noted by the authors, which conducted a sensitivity analysis excluding those 2 trials. However, key data are missing from this sensitivity analysis: did the confidence interval cross one, what was the P-value?
To evaluate whether data can be used as an external comparator, several “fit-for-purpose” factors are considered, including the appropriateness of the comparator [16]. However, we found that at least 2 of the trials selected for external controls did not meet this criterion, which further undermines the reliability of the survival result.
7. Analytic flexibility: nearly two decades of changes in trial design
Randomized-controlled trials typically mandate a predetermined hypothesis, endpoints, and statistical analysis to avoid the possibility of data manipulation. This ensures that the data is not sliced multiple times, which could lead to the increased likelihood of discovering a “significant” outcome purely by chance.
Here, a timeline spanning nearly two decades in the changes made available in https://clinicaltrials.gov/reveals the risk of p-hacking [17]. An in-depth examination of the history report shows 104 versions of study records, dating back to 2005, with the latest update in 2022. It is worth noting that the trial, initially designed as a phase 2 randomized trial, was modified to a phase 3 trial in 2012, after 4 years of accrual, apparently due to an increase in the number of enrolled patients. However, it was not until 2020 (a change recorded in May 2022), that external controls and modifications of endpoints were introduced, several years after the last patient was enrolled.
8. The need for better reporting and data sharing
We previously mentioned how omission of critical information can impede proper appraisal of trials [18]. In the present study report, beyond methodological issues already raised, inconsistent statements are present. The abstract mentions a “nonrandomized” trial and later present results with groups “from randomization” which is confusing. The study is referred to as a phase 3 trial, probably because it was designed to be a randomized controlled phase 3 trial at some point. However, this design feature was ultimately abandoned, and to our knowledge, there are no other records of a nonrandomized externally controlled study being referred to as a phase 3 trial in biomedicine. Finally, one could note that the investigators of the DCVax-L trial requested individual patient data (IPD) for the external controls utilized in their study, while in contrast, their data sharing statement briefly mentions: “data available: no”. Better reporting and data sharing practices are essential to enable transparent appraisal of trials and enhance shared decision-making with patients.
9. Conclusion
This dendritic vaccine trial has several limitations. These limitations include changing the primary endpoint (OS instead of PFS), creating a new study population (recurrent glioblastoma), conducting unplanned analyses, using external controls in a design originally intended to be randomized, all changes occurring years after the trial finished enrollment. The selected external controls likely included patients with less favorable outcomes, which opposes the “fit-for-purpose” criteria usually applied in selecting external controls. Therefore, the purported survival benefit from the vaccine is unreliable. The accumulation of limitations, along with multiple changes made over nearly two decades; further hamper the reliability of the reported results. Without data sharing, those concerns cannot be alleviated. Glioblastoma is the most common primary brain tumor, and there is no valid reason to stray from the gold standard of a randomized-controlled trial with overall survival as the primary endpoint in order to change clinical practice. Dendritic cell vaccination is a very promising approach for GBM, the neuro-oncology medical community can only regret that due to the described DCVax-L trial weaknesses, we cannot draw any conclusions on potential efficacy.

[exwannabe]

We will see how the final reads. The draft had:

25. Appropriate study design and analysis methods must be pre-specified in the protocol (see next section). Potential sources of bias should be identified and discussed in the protocol, along with details of how these will be explored and addressed.

26. It is important that the protocol is finalised before enrolment begins e.g. the design and analysis methods should be sufficiently specified in the protocol and there should be no amendments planned to fill in important missing details. Ideally the protocol should encompass all plans for the trial including the use of a pre-specified RWD ECA.

And on other issues:

. A RWD ECA comprises patient level data collected outside of a clinical study which will be used as a control or part of a control arm to estimate the comparative efficacy and safety of an intervention being studied in a clinical trial.

Yes, patient level data, not summary data. There is a sound reason why everything we have seen on the subject from any RA says this.

A trial with an ECA is not the preferred clinical trial design as a fully powered randomised controlled trial (RCT) should be used if possible.

Obvuiously it is possible as they completed an RCT, they just did not like the results.

27. Without sufficient pre-specification there could be issues related to multiplicity. For example, if a trial is analysed without an external control, and after the analysis an external control is added, this may inflate type I error if it is not clear which was the primary analysis. Therefore, it must be clear what the pre-specified primary analysis for the trial is.

I guess performing an efficacy IA then changing to an ECA would be a concern.