Real world applications can be quite different, which means the scale can vary a lot. This causes the main difficulty --- we need to deal with different corner cases efficiently while keep common case unaffected.
pdp
What would make it more challenging is to have communication requirements between processors in a heterogeneous system - communication between processors through MPI, between gpu and cpu within a processor, etc.
Metalbird
Heterogeneous applications almost always have computational requirements between CPU and GPU, at least in the hybrid CPU GPU case. Even in the simplest case if you have data on the CPU you have to transfer it to the GPU to perform calculations, and then transfer the results back to the CPU after calculations finish.
In a lot of applications from what I have seen, the main difficulty with hybrid CPU, GPU computing has to do with minimizing the effect of data transfer between the processing units. So ya it's a challenging problem that may be getting better with newer versions of CUDA and newer GPUs, but it still requires some thought on the side of the programmer.
Real world applications can be quite different, which means the scale can vary a lot. This causes the main difficulty --- we need to deal with different corner cases efficiently while keep common case unaffected.
What would make it more challenging is to have communication requirements between processors in a heterogeneous system - communication between processors through MPI, between gpu and cpu within a processor, etc.
Heterogeneous applications almost always have computational requirements between CPU and GPU, at least in the hybrid CPU GPU case. Even in the simplest case if you have data on the CPU you have to transfer it to the GPU to perform calculations, and then transfer the results back to the CPU after calculations finish.
In a lot of applications from what I have seen, the main difficulty with hybrid CPU, GPU computing has to do with minimizing the effect of data transfer between the processing units. So ya it's a challenging problem that may be getting better with newer versions of CUDA and newer GPUs, but it still requires some thought on the side of the programmer.