Advanced Micro Devices Inc (AMD)

[UpNDown]

chipguy, re Opteron hop distance

The average number of hops is meaningless, it is the maximum
number of hops in any system that is important. The Opteron relies on a broadcast coherency
system. That means any processor that wants to read memory has to broadcast its intent to
every other processor in the system and then wait to see if one of them has the target line
cached. Hop, hop, hop, hop.

The memory access setup for the processor that holds the requested line (assuming it's not in cache) can be started as soon as that processor has received the cache probe. So the setup can overlap completion of the probe sequence. As this setup is slower than the probe sequence, the completion of the probe sequence is free. The important point for latency is how many hops to the processor holding (or hosting) the cache line and how many hops back.