Intel Corp (INTC)

[jhalada]

Alan,

2 sockets ( 2 requests (one per socket) 2 answers (one per socket ) total 4 messages
4 sockets (4 requests, 12 answers (3 per request X 4 requests)) total 16 messages
8 sockets (8 requests, 56 answers (7 per request X 8 requests) total 64 messages.

You have to also consider the #of connections (you can consider it a unit of bandwidth) as well. There is one additional complication, which is number of hops, that works to saturate these connections more.

Also, IMO, it is better to think in terms of "traffic" rather than requests and answers, because what you call request, meaning a requesting processor sends a message to every receiving processor, the amount of traffic for requests is the same as for the answers, and since we are dealing in orders of magnitude, you can just think of one way traffic.

For a 2 way system, you have 2 units of traffic, and 1 connection. (2 / 1 = 2)

In a 4 way system (with a cross connect), you have 14 units of trafic, 5 connections (units of bandwidth) (14 / 5 = 2.8)

In an 8 way system, there are 100 units of traffic and 10 units of bandwidth (100 / 10 = 10)

So, it does jump. But the increasing number of connections reduces the rate of growth. For example, to go from 2 way to 8 way system you go from ratio of 2 to 10, or 5x increase, rather than your example where you went from 4 to 64, or 16x increase.

Further, you need to consider how saturated the connections become with these coherency requests. If it is 1% in 2 way system, going to 5%, we don't really have a problem. If it goes from 10% to 50%, it may start to become a bottleneck, since the HT bus needs to respond with data from distant processor request as well.

If you consider the possibility of adding 4th link than the data ratios become:

2 socket system: 2 / 1 = 2
4 socket system: 12 / 6 = 2
8 socket system: 84 / 14 = 6

Meaning 4 socket system would have equal coherence traffic as 2 way system (no penalty), 8 socket system would have only 3x coherence traffic vs. 2 way system.

Anyway, it would be helpful if we had some 8 socket data which would speak to the ability of the current HT implementation to handle this much data. It also gets to be more important for the OS to be more NUMA aware so data can be stored in the local memory rather than in the memory of a remote CPU, which would generate even more HT traffic.

True, and it also depends greatly on types of app. So the answer is not really simple. Not as simple that HT coherency strategy is inadequate and runs out of steam.

Also, note that just because Elmer said something does not make it wrong

Not at all, but you have to consider that he perfected the art of FUD, and clearly states it (in his handle), so you can't really get angry, or suspect any kind of deception.

When it comes to AMD, when AMD does not do something, Elmer says AMD can't do it - until proven otherwise, by which time he moves on to the next subject of what AMD can't do.

Well, the answer to the question why AMD does not do something does not automatically mean it can. It can be that AMD has not been presented with (or has not won) the opportunity to prove that it can.

Joe