even though throughput and latency are ways to measure the speed of a computation, there is a bit of an opposing force between increasing throughput and decreasing latency.
to minimize latency, you would not want to use more than one thread per processor, since if there are multiple threads, the processor might switch to another one, (and might not switch back for a while). obviously, this decreases the amount throughput
if you're increasing throughput with SIMD intrinsics, you also increase the latency, because the time it takes to finish processing an element is the longest out of all 8 elements you are processing at the same time.
username
Remembering the freeway example, throughput is the width of the freeway and latency is the speed limit. Thus if our program only has one thing running (one car on the freeway) increasing the bandwidth (width of the freeway) won't really make sense; we would want to decrease the latency (increase the speed limit). However if we had a lot of processes running in parallel (there are a tonne of cars on the freeway), we would be better off (usually) increasing the bandwidth (making the freeway wider would reduce traffic, increasing the speed limit would help though would not be as effective if there simply isn't enough room on the freeway for the cars)
even though throughput and latency are ways to measure the speed of a computation, there is a bit of an opposing force between increasing throughput and decreasing latency.
to minimize latency, you would not want to use more than one thread per processor, since if there are multiple threads, the processor might switch to another one, (and might not switch back for a while). obviously, this decreases the amount throughput
if you're increasing throughput with SIMD intrinsics, you also increase the latency, because the time it takes to finish processing an element is the longest out of all 8 elements you are processing at the same time.
Remembering the freeway example, throughput is the width of the freeway and latency is the speed limit. Thus if our program only has one thing running (one car on the freeway) increasing the bandwidth (width of the freeway) won't really make sense; we would want to decrease the latency (increase the speed limit). However if we had a lot of processes running in parallel (there are a tonne of cars on the freeway), we would be better off (usually) increasing the bandwidth (making the freeway wider would reduce traffic, increasing the speed limit would help though would not be as effective if there simply isn't enough room on the freeway for the cars)