What does "unlike an ISPC gang, the warp concept does not exist in the programming model" mean?
CSGuy
It's alluding to the fact that CUDA doesn't enforce the use of warps, it just so happens that NVIDIA GPUs use them. Whereas ISPC is designed specifically to make use of SIMD instructions in its "gang of instances", and so it is a part of ISPC's programming model.
thunder
As stated in slide 58, warps are GPU implementation detail but we do not apply this concept when we program CUDA, so the concept does not exist in the programming model. However, for ISPC gang, we explicitly define the gang of instances when programming and thus it's in the programming model.
boba
From the perspective of a CUDA programmer, warps happen to be an implementation detail of NVIDIA GPUs, not a model of CUDA programs.
Levy
All threads in one block run at the same time: this is not exactly true. The number of execution contexts in one SM core is greater than the number of instruction fetcher, so if the number of warps is greater than the simultaneous running warps but less than the number of warps in one SM core, they will be executed batch by batch.
What does "unlike an ISPC gang, the warp concept does not exist in the programming model" mean?
It's alluding to the fact that CUDA doesn't enforce the use of warps, it just so happens that NVIDIA GPUs use them. Whereas ISPC is designed specifically to make use of SIMD instructions in its "gang of instances", and so it is a part of ISPC's programming model.
As stated in slide 58, warps are GPU implementation detail but we do not apply this concept when we program CUDA, so the concept does not exist in the programming model. However, for ISPC gang, we explicitly define the gang of instances when programming and thus it's in the programming model.
From the perspective of a CUDA programmer, warps happen to be an implementation detail of NVIDIA GPUs, not a model of CUDA programs.
All threads in one block run at the same time: this is not exactly true. The number of execution contexts in one SM core is greater than the number of instruction fetcher, so if the number of warps is greater than the simultaneous running warps but less than the number of warps in one SM core, they will be executed batch by batch.