If higher arithmetic intensity is required to efficiently utilize, modern parallel processors, then is it right to assume that cases with extremely low arithmetic intensity (specifically, algorithms which cannot be optimized to improve arithmetic intensity) are usually better off running on a serial processor?

@bAAn, I believe there are other ways to improve the performance. The problem with low arithmetic intense program is that it is bandwidth limited. With multiple processor on same machine doesn't help since they are sharing the same memory. However, if we could split data to multiple machines then bandwidth limitation is reduced.

So, instead using one serial processor, may be use multiple small processors, each having separate memory, is better.

This model seems doesn't take consideration of not all communications are the same. For example, given a distributed system, we need to take care that communicate across data warehouse is order of magnitude higher than access from local memory.

And a back-of-envelop analysis of communication cost at different level could be found here.

http://highscalability.com/blog/2011/1/26/google-pro-tip-use-back-of-the-envelope-calculations-to-choo.html