The short answer is program-survival bias. Since a huge number of drugs and indications are tested in phase-2, by sheer chance many drugs with no true efficacy will appear to show efficacy according to conventional statistical metrics.
Because only the drugs that appear to show efficacy in phase-2 will be advanced to phase-3 by their sponsors, the efficacy shown in phase-2 by the drugs that do advance will in the aggregate be a high-bias estimate of the true efficacy of these drugs. It follows that a large proportion of them will ultimately fail in phase-3.
Please see #msg-7959424 for a study of this effect in one development area (chemotherapy).