Advanced Micro Devices Inc (AMD)

[wbmw]

YB, Re: The listed IBM ASIC one-hop latency is listed as 2 microseconds, i.e. 20 times slower. I think the article has some mistakes here, because it is known that in Opteron HT world each hop adds just 25 ns of latency.

Is it possible that this 2 ms latency is the average distance latency for the whole 10,000 cpu computer? It 27x16x24 mesh, so the average distance might be in the 20 hop range. That will match the mumbers with reality.

First, the ASIC will be manufactured at IBM, but it will be designed by Cray. Second, you cannot compare it to the latency in a 4-way Opteron system, because it's an interconnect that is responsible for keeping track of the cache states of multiple processors and processor nodes, which makes the state machine orders of magnitude larger. That's why the latency is 20 times slower.

Also, the idea of a 3D mesh is to limit the number of hops required to go from one end of the system to the other. A better design might involve a 4 dimensional cube or greater, but the design complexity goes through the roof. A 3D cube might be a good balance, and if the worst case latency from one end of the cube to the other is only (27+16+24) 67 hops, or 134us, that's not bad, considering the size of the structure. It will scale far better than 10,000 CPUs clustered via ethernet or other external cables, that's for sure.