When a block is scheduled, does it run till every thread completes its work or can a block be preempted by another block of threads?
afa4
To add to @gryffolyon's question: Is a warp preemptable? That is to say, if threads in a warp are stalled on some operation does the scheduler schedule another warp from the same thread block to hide latency?
arjunh
A block can't be preempted; it must execute to completion before it can be swapped off the SM. Multiple blocks can be executed on the same SM (8 cores per multi-processor).
On the other hand, warps can be preempted. In fact, __syncthreads is an example of preemption. Essentially, you are blocking a warp of thread from proceeding until all other warps within the block have finished executing up to the same point.
xSherlock
Interesting to note is that the fact that warps getting preempted seems to be critical to maximize throughput, because at any point in time many warps could be blocked on memory, while others can continue executing. This means that memory will utilize the maximum amount of bandwidth, while keeping other warps utilizing ALUs.
kayvonf
@afa4: Yes, the SMX core will switch to another warp to run. However, any runnable warp present on the core is fair game, it need not be a warp from the same thread block. Remember, the GPU scheduler will pack an SMX core with as many thread blocks as it can.
When a block is scheduled, does it run till every thread completes its work or can a block be preempted by another block of threads?
To add to @gryffolyon's question: Is a warp preemptable? That is to say, if threads in a warp are stalled on some operation does the scheduler schedule another warp from the same thread block to hide latency?
A block can't be preempted; it must execute to completion before it can be swapped off the SM. Multiple blocks can be executed on the same SM (8 cores per multi-processor).
On the other hand, warps can be preempted. In fact,
__syncthreadsis an example of preemption. Essentially, you are blocking a warp of thread from proceeding until all other warps within the block have finished executing up to the same point.Interesting to note is that the fact that warps getting preempted seems to be critical to maximize throughput, because at any point in time many warps could be blocked on memory, while others can continue executing. This means that memory will utilize the maximum amount of bandwidth, while keeping other warps utilizing ALUs.
@afa4: Yes, the SMX core will switch to another warp to run. However, any runnable warp present on the core is fair game, it need not be a warp from the same thread block. Remember, the GPU scheduler will pack an SMX core with as many thread blocks as it can.
See the visualization beginning here.