Parallelism is limited by the barriers as all threads must wait on the slowest thread in order to proceed computing.
pdp
There is a sync abstraction in ISPC as well for task synchronization as explained here.
CSGuy
In what situation would we want num_threads for a barrier to be less than the total number of worker threads? It seems like there would be some pretty nasty races if one tried it. Is num_threads there just as a convenience to inform the code how many threads there are?
nate
@CSGuy Yeah that's a really good question. The only thing I can think of is that there may be barriers within conditionals. For example, if you are having all the odd threadIds do one task and all the evens do another (two independent tasks), you could have a barrier() call in each conditional with half the number of worker threads to not have to wait for computation that isn't a dependency.
POTUS
One thing I'm confused about is whether any problems will arise with barriers when there are more threads than execution units? Is this a possible scenario?
apr
@POTUS, I don't think any problems will arise if there are more threads than execution units. In the examples above, we are dealing with SPMD, which means that the registers / other context required is stored in each vector lane. Barriers will then come into play in order to sync threads that are scheduled on different cores / different 'gangs' to use a previously used term. If there is more data parallelism than can fit in a 'gang', then execution units are used to allow for fast hardware context switching in order to not stall any particular gang.
Now, when if there are more tasks (to refer to tasks that are dividing the processor's time resource) than execution contexts, then on at least one core, there will be a few tasks that are software context switched into. The difference between using hardware context switching and software context switching is not exposed to the user, and hence there won't be any problem in this case.
themj
What are the differences between barriers and synchronizing the threads? They both essentially force the program to wait until all of the threads have reached that point.
There is an application of barrier in openGL. In openGL there is something called Sync Object. Here is a link:
https://www.khronos.org/opengl/wiki/Sync_Object
Parallelism is limited by the barriers as all threads must wait on the slowest thread in order to proceed computing.
There is a sync abstraction in ISPC as well for task synchronization as explained here.
In what situation would we want num_threads for a barrier to be less than the total number of worker threads? It seems like there would be some pretty nasty races if one tried it. Is num_threads there just as a convenience to inform the code how many threads there are?
@CSGuy Yeah that's a really good question. The only thing I can think of is that there may be barriers within conditionals. For example, if you are having all the odd threadIds do one task and all the evens do another (two independent tasks), you could have a barrier() call in each conditional with half the number of worker threads to not have to wait for computation that isn't a dependency.
One thing I'm confused about is whether any problems will arise with barriers when there are more threads than execution units? Is this a possible scenario?
@POTUS, I don't think any problems will arise if there are more threads than execution units. In the examples above, we are dealing with SPMD, which means that the registers / other context required is stored in each vector lane. Barriers will then come into play in order to sync threads that are scheduled on different cores / different 'gangs' to use a previously used term. If there is more data parallelism than can fit in a 'gang', then execution units are used to allow for fast hardware context switching in order to not stall any particular gang.
Now, when if there are more tasks (to refer to tasks that are dividing the processor's time resource) than execution contexts, then on at least one core, there will be a few tasks that are software context switched into. The difference between using hardware context switching and software context switching is not exposed to the user, and hence there won't be any problem in this case.
What are the differences between barriers and synchronizing the threads? They both essentially force the program to wait until all of the threads have reached that point.