Miracle Cure, or Murky Research? Tuesday, October 25, 2005 By Wendy McElroy
Herceptin, a therapeutic drug for breast cancer, was trumpeted across the news last week after a glowing report appeared in the New England Journal of Medicine.
Forbes called it a "wonder drug"; the London Times declared Herceptin "stunning"; CNN heralded the drug as "perhaps the most powerful cancer medicine in a decade," which "can halve the risk of relapse" in many cases.
I hope the reports are accurate.
But another body of research has gone comparatively unnoticed.
"Why Most Published Research Findings Are False" by John P. A. Ioannidis -- an epidemiologist at the University of Ioannina (Greece) -- presents convincing evidence that an alarmingly high number of scientific "findings" are eventually proven false.
Dr. Ioannina first published his controversial claim in the Journal of the American Medical Association (JAMA, July 2005). JAMA's report studied "all original clinical research studies published in three major general clinical journals or high-impact-factor specialty journals in 1990-2003," each of which had been "cited more than 1000 times" in subsequent literature.
In short, Ioannina focused on original research published by influential journals and widely accepted as accurate by other scientists. (These studies are the cream-of-the-cream and can be expected to return the highest accuracy rate within medical research in general.)
Their data impact the health and, sometimes, the life of patients.
Of 49 studies, 45 "claimed that the intervention [examined] was effective," which means they changed the day-to-day decisions of medical care.
Fourteen of those 45 studies (approx. 32 percent) were subsequently refuted. Twenty (44 percent) were replicated or validated. Eleven (24 percent) remain unchallenged and, so neither validated nor refuted.
One refuted study concerned the safety of hormone-replacement therapy for women…in case you wondered why that therapy was deemed safe one minute and risky the next. Fortunately for women, the media focused on the medical establishment's about-face on hormone replacement. But the refutation of other studies hasn't received similar publicity.
Nor has Ioannina's claim that fully 50 percent of medical research is wrong, with approximately the same chance of accuracy as flipping a coin.
Ioannina's conclusion is speculative but, given that so many prestigious studies have been contradicted, it is not wildly improbable.
Register for free to join our community of investors and share your ideas. You will also get access to streaming quotes, interactive charts, trades, portfolio, live options flow and more tools.