This might be a daft question, but has anyone actually tried organising nodes in a 3D torus format? What are the downsides of doing this? (You get the bonus of more evenly distributed link lengths, but fitting a 3D structure onto a chip...)
eourcs
@Cake The ASICs that compose the Anton supercomputer are arranged in a 3D torus. I'd be surprised if there were any single chip examples with that topology for the reasons you mentioned.
ask
Can the 2D torus be viewed as a sphere where every node has 4 neighbors?
nemo
The latency here is still O(sqrt(N)), right?
pdp
@nemo: Yes!
Abandon
@nemo But the max latency is reduced to sqrt(N)/2 rather than sqrt(N) in the mesh style.
thunder
In this case, since the links that connect the bottoms and the tops of each column and the left most and right most of each row are much longer than other links, the latency on these links might also be longer. Therefore, I think the max latency can still be reduced but might be larger than sqrt(N)/2.
paramecinm
@Abandon For N=16 case like the picture in slides, the max latency will be (0, 0) visiting (2, 2) which is 4 = sqrt(N) not sqrt(N)/2.
Abandon
@paramecinm Oh, I see! Thank you! So the answer is that the max latency is reduced from 2*sqrt(N) to sqrt(N). right?
shreeduh
Laying out such networks on chips are pretty difficult, and due to number of links crossing each other, multi-layer routing is needed, increasing organizational complexity and link latency.
This might be a daft question, but has anyone actually tried organising nodes in a 3D torus format? What are the downsides of doing this? (You get the bonus of more evenly distributed link lengths, but fitting a 3D structure onto a chip...)
@Cake The ASICs that compose the Anton supercomputer are arranged in a 3D torus. I'd be surprised if there were any single chip examples with that topology for the reasons you mentioned.
Can the 2D torus be viewed as a sphere where every node has 4 neighbors?
The latency here is still O(sqrt(N)), right?
@nemo: Yes!
@nemo But the max latency is reduced to sqrt(N)/2 rather than sqrt(N) in the mesh style.
In this case, since the links that connect the bottoms and the tops of each column and the left most and right most of each row are much longer than other links, the latency on these links might also be longer. Therefore, I think the max latency can still be reduced but might be larger than sqrt(N)/2.
@Abandon For N=16 case like the picture in slides, the max latency will be (0, 0) visiting (2, 2) which is 4 = sqrt(N) not sqrt(N)/2.
@paramecinm Oh, I see! Thank you! So the answer is that the max latency is reduced from 2*sqrt(N) to sqrt(N). right?
Laying out such networks on chips are pretty difficult, and due to number of links crossing each other, multi-layer routing is needed, increasing organizational complexity and link latency.