We can know in following slides that blocked assignment have less data access and communication. But does the sentence "answer: it depends on the system" mean that there will be some situations that Interleaved Assignment can perform better than Blocked Assignment?
anonymous
@momoda I think one situation in which interleaved assignment might be better would be if the computation takes longer than the data access. If the data access time is no longer relevant there are certain computations (not necessarily in this question) where interleaved assignment is better, to ensure that work is equally distributed over the threads.
jrgallag
@momoda Another example of a situation where interleaved assignment would be better is if there are some regions of data that are much more expensive to calculate than others. For example, imagine if the points on the lower half of the grid cost twice as much to compute, and we didn't need to communicate. With blocked assignment, thread 1 would do much less work than thread 4, but with interleaved all threads do about the same amount.
Funky9000
In my opinion, interleave assignment may divide the work more evenly among workers and lead to better performance. However, the cost of managing workers have to be considered as well.
As for blocked assignment, while there are fewer tasks to perform,the cost of managing workers is lower. So it really depends on the system which it is implemented for.
Elmur_Fudd
As we saw in the Mandelbrot problem, interleaving can massively improve the division of work.
I think we talk about communication in an abstract sense: the communication can be implemented in different ways such as message passing or using a shared memory address space. If the implementation of the communication is very fast, then the higher cost of communication won't overcome the benefits of having a better distribution of work. (e.g. all of the threads are on the same core and the data fits all into that core's cache).
We can know in following slides that blocked assignment have less data access and communication. But does the sentence "answer: it depends on the system" mean that there will be some situations that Interleaved Assignment can perform better than Blocked Assignment?
@momoda I think one situation in which interleaved assignment might be better would be if the computation takes longer than the data access. If the data access time is no longer relevant there are certain computations (not necessarily in this question) where interleaved assignment is better, to ensure that work is equally distributed over the threads.
@momoda Another example of a situation where interleaved assignment would be better is if there are some regions of data that are much more expensive to calculate than others. For example, imagine if the points on the lower half of the grid cost twice as much to compute, and we didn't need to communicate. With blocked assignment, thread 1 would do much less work than thread 4, but with interleaved all threads do about the same amount.
In my opinion, interleave assignment may divide the work more evenly among workers and lead to better performance. However, the cost of managing workers have to be considered as well. As for blocked assignment, while there are fewer tasks to perform,the cost of managing workers is lower. So it really depends on the system which it is implemented for.
As we saw in the Mandelbrot problem, interleaving can massively improve the division of work.
I think we talk about communication in an abstract sense: the communication can be implemented in different ways such as message passing or using a shared memory address space. If the implementation of the communication is very fast, then the higher cost of communication won't overcome the benefits of having a better distribution of work. (e.g. all of the threads are on the same core and the data fits all into that core's cache).