Previous | Next --- Slide 49 of 79
Back to Lecture Thumbnails

Going through important terminology here:

Threads: Runs the execution of a kernel at a particular index

Warps: Consist of a number of threads (usually 32) which share the same instruction stream

Thread blocks: Thread blocks have access to the same shared memory. Number of threads in a thread block should be a multiple of the size of a single warp

Grids: A collection of blocks. There is no synchronization between these blocks. An entire grid is processed by a single GPU chip


I think an important point to note is that, there is a certain amount of shared memory available. In real life programming, the amount of shared memory is limited, which means that sometimes you have to think about decomposing code that would normally access a large amount of memory into smaller independent chunks.