Worker Nodes and parameter servers are separate in the graph, but they are not necessary to be on different machines. It is interesting to notice that the workers are computation bounded while the parameter servers are network IO bounded. The resources of a machine may be better utilized if it has both workers and parameter servers. They could live in different processes, containers or virtual machines on one physical machine.
mario
You can pair up nodes in order to represent that pairs of nodes can be effectively considered as one node by their computations.
Worker Nodes and parameter servers are separate in the graph, but they are not necessary to be on different machines. It is interesting to notice that the workers are computation bounded while the parameter servers are network IO bounded. The resources of a machine may be better utilized if it has both workers and parameter servers. They could live in different processes, containers or virtual machines on one physical machine.
You can pair up nodes in order to represent that pairs of nodes can be effectively considered as one node by their computations.