Paul: Could it be they're having enough trouble with cache defects that they had to add enough to be able to disable some and still have the 1Meg?
I see 3 possibilities:
1) Opteron's L2 cache seems performans remarkably better than AthlonXP's. Ace's Hardware notes an increase in bandwidth of 30% over AthlonXP's L2 cache (link below). TecChannel graphs show a similar picture. As for latencies, TecChannel measures an L2 latency of 16 clock cycles, down from 20 on the AthlonXP. L1 cache is still 3 clock cycles. It wouldn't be unreasonable to assume that this increase in performance came at a cost.
2) The AthlonXP-style cache was having problems scaling in terms of frequency, so a change was necessary (increase in cell size or whatever). This could fit with the persistant rumours that the P4's L2 cache is run at half speed, double wide with each half accessed on alternating clocks. The only downside of that approach would (as far as I can tell) be a latency penalty of 1 cycle.