Consider the complexity of a GPU memory controller. A modern NVIDIA GPU is simultaneously executing hundreds of warps, all of which can have outstanding memory requests. (And each thread in the warp might be accessing different memory addresses!)