Biotech Values

[AlpineBV_Miller]

Two thoughts...

I must have missed it in that article, but where does he say Genasense should be "approved"?

Second, the FDA did make serious math errors in the melanoma application. Here's the situation...

Patients on the melanoma trial were randomized to 5 (I think, the number is not important for the discussion below) days of Genasense followed by DTIC versus DTIC alone. Three weeks after the DTIC dose, they came in for their scans and the next dose.

If you're sharp, you'll immediately recognize a problem. Here it is spelled out...

Genasense Arm:
Cycle 1: Baseline scan Day 0. Dose G days 0-4, DTIC day 4
Cycle 2: Scan day 25. Dose G days 25-29, DTIC day 29
Cycle 3: Scan day 50. Dose G days 50-54, DTIC day 54.
...

Control Arm:
Cycle 1: Baseline scan Day 0. Dose DTIC Day 0
Cycle 2: Scan day 21. Dose DTIC Day 21.
Cycle 3: Scan day 42. Dose DTIC Day 42.
...

Let's say both patients progress at scan #3. The data for the G patient will show progression at day 50, the control patient will show progression at day 42. That's an 8-day progression advantage. Now that may actually be correct, but because the design of the trial specified the next visit was 21 days after DTIC (instead of 21 days after the last scan) there is a built in progression bias.

In fact, the differential was not this large because of typical patient shifting and dosing delays. This is just an illustration of the fundamental design flaw* in this study. The potential impact of this flaw on the PFS endpoint was discovered by a consultant under contract to a hedge fund short Genta and communicated to the FDA staff.

There was no established biostatistical procedure to correct for this differential follow-up in late 2003/2004. The FDA statistician came across a message board posting by one of the senior coders for the statistical package they use to analyze trials. The coder had made a suggestion on an unrelated question from a user about a specific course of analysis. The FDA statistical reviewer followed the same path.

The FDA statistical reviewer performed the calculation based on the concept outlined in the message board poast and utilizing a range of assumptions as to follow-up differentials. In the initial calculation, most of the assumptions rendered the PFS benefit not statistically significant. These were printed in the briefing documents.

In Genta's review of the FDA's briefing documents, they detected a math error in the calculations. The FDA issued an errata correction based upon their own recalculation. Now only about half the the assumptions were not statistically significant. The panel members were not provided this correction until either the night before the panel meeting or the morning of the panel meeting, depending on who you believe.

The errata also had math errors. Depending on the story, the presentation made by the FDA statistical reviewer at the panel meeting was either a third correction or the same correction in the errata with additional typographical errors. In any case, the FDA's presented data were still wrong.

Genta presented the correct equations in their presentation. While the PFS benefit shrank to clinical insignificance in some situations, none of the scenarios (or none but the most outlandish differential assumption, I don't quite remember) failed to reach statistical significance.

The FDA has published a guidance document since that panel meeting warning sponsors to take special care to make sure progression follow-up is matched.

The Genasense melanoma NDA had many more flaws in it than this. They simply cannot get around the fact the survival benefit failed on the ITT analysis, rendering the subgroup analyses moot. At the time, however, the survival analysis was thought to be simply immature. All melanoma drugs had been approved on the basis of ORR, which the panel voted that Genasense had met. The PFS was voted down, primarily based upon the flawed statistical analysis presented by the FDA.

In hindsight, I'm not certain that panel would have voted for Genasense even if they voted 'yes' on the ORR and PFS endpoints. Two voting statisticians and no significant melanoma expertise voting on the panel makes me believe the NDA was going down regardless.

The math mistake, the use of a blog post as the statistical justification for the differential calculation, and having no signficant melanoma expertise on the panel all add up to at least the impression of an unfair panel hearing.



* The "funniest" thing about this is the original SAP proposed by Genta to the FDA had each visit occuring 21 days after the last visit. The FDA requested a change to 21 days after the DTIC dose so there would be a constant break between doses. The FDA was worried that a 17 day break in the Genasense arm versus a 21 day break in the control would bias the efficacy results. Genta agreed since they were worried the shorter break would result in more side effects on the Genasense arm. Ironically, the FDA essentially hung Genasense based upon the trial design alteration they requested.