We want to restructure our program in such a manner that we consume whatever intermediate data we generate.Storing that data to memory and loading it back to perform computations on it just incurs the extra overhead of waiting on memory when we could just use the data we produced immediately and avoid the overhead. This is especially true in case our system is bandwidth-bound!
We want to restructure our program in such a manner that we consume whatever intermediate data we generate.Storing that data to memory and loading it back to perform computations on it just incurs the extra overhead of waiting on memory when we could just use the data we produced immediately and avoid the overhead. This is especially true in case our system is bandwidth-bound!