Question: Why is it important for GraphLab to know the working set of a GraphLab vertex program?
sanchuah
I think the reason is that the operation abstraction of GraphLab is defined only related to the adjacent neighbors of a node.
landuo
I think one of the reason is concurrency and the other reason is for memory locality.
kayvonf
Said differently, what optimizations are possible if the system has this extra information?
jazzbass
@kayvonf This extra information allows GraphLab to:
Identify locality
Identify dependencies
With this extra information GraphLab can do a better job at the time of partitioning the graph (try to make communication as local as possible), to maximize parallelism (run in parallel all vertex programs that share no dependencies), and scheduling (perhaps improving temporal locality by scheduling nodes that have similar neighborhoods close in time).
Question: Why is it important for GraphLab to know the working set of a GraphLab vertex program?
I think the reason is that the operation abstraction of GraphLab is defined only related to the adjacent neighbors of a node.
I think one of the reason is concurrency and the other reason is for memory locality.
Said differently, what optimizations are possible if the system has this extra information?
@kayvonf This extra information allows GraphLab to:
With this extra information GraphLab can do a better job at the time of partitioning the graph (try to make communication as local as possible), to maximize parallelism (run in parallel all vertex programs that share no dependencies), and scheduling (perhaps improving temporal locality by scheduling nodes that have similar neighborhoods close in time).