What is the overhead (both in terms of time to sort and extra storage needed to keep track of shard boundaries) of actually preprocessing a graph (given to the system by an external application, saved in an arbitrary order) into this state? This seems to me a serial operation and the sorting might be expensive?
Allerrors
Sorting can be done by in memory merge sort or map/reduce so if the graph analysis will do lots of computation on the graph, the overhead is relative small.
What is the overhead (both in terms of time to sort and extra storage needed to keep track of shard boundaries) of actually preprocessing a graph (given to the system by an external application, saved in an arbitrary order) into this state? This seems to me a serial operation and the sorting might be expensive?
Sorting can be done by in memory merge sort or map/reduce so if the graph analysis will do lots of computation on the graph, the overhead is relative small.