This slide says that threads in a thread block can run as both "SPMD shared address space programming" or "SIMD". Aren't they different?
bwf
SPMD and SIMD are different, however you can use both techniques for achieving parallelism at the same time. SIMD deals with the manner in which instructions are executed(vector intrinsics) whereas SPMD deals with where the program is being executed. You can have a program that is both SIMD and SPMD.
arjunh
@andrewwuan They are different, as @bwf pointed out, but a GPU uses both of them in different ways; on on the block level and the other on the intra-block level.
In a GPU, the threads running in a block can be seen as an example of SPMD (each thread is executing a function on a different input and each thread must finish execution before the block can be scheduled off a SM unit). Within a block, threads are organized into warps, which are executed in SIMD style.
WARNING: Confusing abstraction with implementation is occurring here by students! ;-) CUDA presents the programmer with an SPMD programming model (a logical abstraction) that modern GPUs implement efficiently via SIMD execution.
This slide says that threads in a thread block can run as both "SPMD shared address space programming" or "SIMD". Aren't they different?
SPMD and SIMD are different, however you can use both techniques for achieving parallelism at the same time. SIMD deals with the manner in which instructions are executed(vector intrinsics) whereas SPMD deals with where the program is being executed. You can have a program that is both SIMD and SPMD.
@andrewwuan They are different, as @bwf pointed out, but a GPU uses both of them in different ways; on on the block level and the other on the intra-block level.
In a GPU, the threads running in a block can be seen as an example of SPMD (each thread is executing a function on a different input and each thread must finish execution before the block can be scheduled off a SM unit). Within a block, threads are organized into warps, which are executed in SIMD style.
I found some super useful NVIDIA slides on CUDA programming.
WARNING: Confusing abstraction with implementation is occurring here by students! ;-) CUDA presents the programmer with an SPMD programming model (a logical abstraction) that modern GPUs implement efficiently via SIMD execution.