Can you elaborate on Remote Memory? Is that faster or slower than disk?
kayvonf
Here's an example: Imagine a supercomputer like Blacklight that is comprised of 256 blade servers each with 16 cores and 128GB of RAM (4096 total cores). The machine is a shared address space machine, so the hardware supports the ability for any processor to access any memory address across all the blades (e.g., a program load instruction can trigger a cache line sized data transfer from a DRAM on blade 0 into the cache of a processor on blade 255). You can probably imagine that the latency of the load is highly correlated to how far apart on the supercomputers interconnect the requested and responding blades are.
And no, it will still be far lower latency than local magnetic disk.
Olorin
@kayvonf When you were talking about this in lecture, I first pictured "network hops" as meaning over, say, the Internet (and if I'm not mistaken, you may have mentioned this explicitly?). For example, if Google or Facebook has some data you request in another data center than the one that's handling your HTTP connection, would we consider that a remote memory access? (Could we also consider it a communication cache miss?)
kayvonf
@Olorin, network hops can also refer to hops in the interconnection network in a single machine.
I would certainly call your example remote "data" access, but I wouldn't refer to it as a communication miss. A communication miss is generated by interaction between two processors (see this conversation about communication misses). We really need to get into the details of cache coherence to really understand what I mean by communication miss.
Can you elaborate on Remote Memory? Is that faster or slower than disk?
Here's an example: Imagine a supercomputer like Blacklight that is comprised of 256 blade servers each with 16 cores and 128GB of RAM (4096 total cores). The machine is a shared address space machine, so the hardware supports the ability for any processor to access any memory address across all the blades (e.g., a program load instruction can trigger a cache line sized data transfer from a DRAM on blade 0 into the cache of a processor on blade 255). You can probably imagine that the latency of the load is highly correlated to how far apart on the supercomputers interconnect the requested and responding blades are.
And no, it will still be far lower latency than local magnetic disk.
@kayvonf When you were talking about this in lecture, I first pictured "network hops" as meaning over, say, the Internet (and if I'm not mistaken, you may have mentioned this explicitly?). For example, if Google or Facebook has some data you request in another data center than the one that's handling your HTTP connection, would we consider that a remote memory access? (Could we also consider it a communication cache miss?)
@Olorin, network hops can also refer to hops in the interconnection network in a single machine.
I would certainly call your example remote "data" access, but I wouldn't refer to it as a communication miss. A communication miss is generated by interaction between two processors (see this conversation about communication misses). We really need to get into the details of cache coherence to really understand what I mean by communication miss.