on solanezumab estimated effect size. backcaluculated from one of thee AD KOLs
Comments on News and Primary Papers
Comment by: Gary Cutter, Lon Schneider, ARF Advisor (Disclosure)
Submitted 10 September 2012 | Permalink Posted 10 September 2012
Quantitatively Assessing Lilly’s Solanezumab Trials Disclosure: More Than Meets the Eye
Reports on Alzforum and in the mainstream media revealed that expert opinions on the solanezumab outcomes varied widely, ranging from dismissive to expectant, and from "the drug is dead" to "there is life yet for the amyloid-ß hypothesis." Various predictions and interpretations were made, despite Lilly’s sparse press statements and suggestion to wait until the data presentations in October and discussions with the FDA.
Lilly is bound by the Sarbanes-Oxley Act to disclose all information that may affect an investor's assessment of the company’s prospects; some questioned whether the press release—carefully parsed as it was—met this requirement. We examined whether more information was provided than seemed apparent, and whether a less impressionistic and a more quantitative assessment of solanezumab’s clinical effects could be made. We found that solanezumab could be exerting a small, but potentially clinically meaningful cognitive effect in mild AD patients.
Lilly stated that the two trials included more than 2,050 patients (the planned sample size for each was 1,000). About two-thirds of each sample was mild AD, MMSE 20-26, and one-third was moderate AD, MMSE 16-19. Neither trial was significant on the ADAS-cog, but the two trials pooled were significant. The mild AD subgroups of EXPEDITION1 and 2 showed significance and a “trend” for solanezumab, respectively. The pooled mild AD groups showed significance. Neither the moderate AD subgroups nor the pooled moderate AD group showed significance.
We used standard formulae to convert p values and sample sizes to z scores and standardized effect sizes (ES); that is, z = ES/([sqrt (2/N]), assuming that the standard deviation of the change in ADAS-cog and sample size N are about the same in each group. We then considered “best-case” and “worst-case” scenarios based on Lilly’s stated outcomes, and tabled the results below.
EXPEDITION1 EXPEDITION2 Pooled
Mild AD Sig*, N = 666 “Trend,” N = 666 Sig*, N = 1332
“Best-case” P = .001, ES = 0.26 P = .06, ES = 0.15 P = .003, ES = 0.20
“Worst-case” P = .049, ES = 0.15 P = .15, ES = 0.11 P = .02, ES = 0.13
Moderate AD NS, N = 360 NS, N = 360 NS, N = 720
“Best-case” P = .80, ES = -0.14 P = .54 , ES = 0.06 P =.37, ES = -0.04
“Worst-case” P = .98, ES = 0.00 P = .98, ES = 0.00 P = .97, ES = 0.00
Total NS, N = 1026 NS, N = 1026 Sig*, N = 2052
“Best-case” P = .06, ES = 0.12 P = .06, ES = 0.12 P =.008, ES = 0.12
“Worst-case” P = .11, ES = 0.10 P = .24 , ES = 0.07 P = .049, ES = 0.09
We considered the “best-case” scenario for the ADAS-cog for the mild AD subgroups as p = .001 for trial 1 and p = .06 for trial 2, allowing for the strongest “statistical trend.” Calculated ESs for this scenario are ES = 0.26 and ES = 0.15, for trial 1 and 2, respectively; and for the pooled mild AD group, it is ES = 0.20. For the ES to be 0.26 in trial 1, however, the ES for the moderate AD group must trend negative at -0.14 in order to ensure that the trial overall has a non-significant p value = .06. The ES for the moderate AD group in trial 2 must be 0.06 in order for trial 2 also to be non-significant at the strongest trend, p = .06. (The pooled ES for the moderate subgroups would be ES = -0.04). Finally, the best case when the two non-significant trials are pooled is p = .008 and ES = 0.12.
The “worst-case” scenario assumed p = .049 and p = .15 for mild AD in each trial, i.e., the highest p values for significance and trend, yielding a pooled ES = 0.13 for mild AD. The p value for the combined trials could be as high as p = .049 with an ES = 0.09, or as low as p = .01 with an ES = .12, with the moderate AD effect sizes ranging from 0.00 or 0.10.
In sum, the best- and worst-case effect size scenarios for the pooled mild AD subgroups are 0.20 and 0.13, and for the pooled trials, 0.12 and 0.09.
Drug-placebo differences in ADAS-cog points can be estimated by multiplying the ES by the SD of ADAS-cog change. Thus, an ES = 0.20 can represent from 1.2 to 1.8 points' difference if the SD of the placebo group is from 6 to 9 over 18 months, as it is in other trials. Similarly, the slope reduction—viewed by some as an indicator for “disease modification”—may range from 13 percent to 30 percent if the mean ADAS-cog worsening in the placebo group ranges (also) from 6 to 9 over 18 months.
It is possible that solanezumab—if its statistical effect is robust and an MRI biomarker is properly supportive—could gain marketing approval for mild AD on the basis of these trials alone, as the FDA has stated it would accept a single cognitive outcome criterion and a single trial (rather than two) under certain conditions. This, however, seems to us to be very unlikely. More likely, a confirming trial needs to be done, and would involve about 1,000 mild AD patients to avoid unwanted outcomes due to play of chance. Of course, it is quite possible that both trials were unlucky in randomly estimating the true effect size and the truth lies elsewhere. On the plus side, adverse events don’t appear to be issues.
These best- and worst-case estimates—requiring certain assumptions—provide information on why Lilly apparently is not discontinuing solanezumab’s development. These estimates can be used as a framework on which to compare the fuller presentations of the outcomes of both the solanezumab and the bapineuzumab trials.
View all comments by Gary Cutter
View all comments by Lon Schneider
