Clusters of machines clearly seems like a downside because of its difficulty to manage and the cost of partitioning as mentioned above. Isn't there also a heavy communication cost? Like in assignment 4 where communication across 2 nodes would cost more, wouldn't communication of masses of data across different clusters during program runtime result in major slowdown?
gogogo
In cases where the graph must be distributed over several clusters, what frameworks/strategies are there to manage graph components in a simple way?
Split_Personality_Computer
@amolakn I imagine the idea is that the massive data would be stored per cluster machine in such a way that it never has to be transported. All data would be computation would be done locally so that the only info being passed between machines is just the end result.
Clusters of machines clearly seems like a downside because of its difficulty to manage and the cost of partitioning as mentioned above. Isn't there also a heavy communication cost? Like in assignment 4 where communication across 2 nodes would cost more, wouldn't communication of masses of data across different clusters during program runtime result in major slowdown?
In cases where the graph must be distributed over several clusters, what frameworks/strategies are there to manage graph components in a simple way?
@amolakn I imagine the idea is that the massive data would be stored per cluster machine in such a way that it never has to be transported. All data would be computation would be done locally so that the only info being passed between machines is just the end result.