On a small scale, it is more a question of latency - that is why we created concurrent threads is proxylab. We did not want a small request to be blocked by a large one that came earlier. We wanted to be able to serve the small request quickly (keep the latency low) even if a large request came first.
On a large scale, throughput matters also, for example if you serving millions of requests at once.
muchanon
For the user, it is a question of latency. Users will not use a site if every request takes a long time to be handled. For the owner of the server, it is both. They want to keep latency down because it needs to be low for the user, and they need throughput high because they want to be able to service many users.
bamboo
I believe threads helped with both latency (as you could get the data faster as a user) and throughput(from both perspectives - user and server, server as explained in muchanon's comment and user because imagine a user making multiple requests and all being served concurrently).
And the I think the caching part mainly helped with latency, as it reduced the response time for a request and saved a round trip to the server. I think it was not very helpful with throughput as it was shared and had contention (for writing).
rsvaidya
It is not a question of throughput. Assume there are 2 requests, one small and the other large. Even if the larger one gets through first keeping the small one waiting, the processor is not idle and is busy processing the bigger request.
tkli
I think latency should be the priority for the site. As long as the site is consistently able to respond to requests with low latency, everyone should be happy. The problem arises when a lack of throughput harms latency i.e. we are bottlenecked. So, while throughput may be important to keep in mind, it is only because it harms the ultimate goal of latency.
On a small scale, it is more a question of latency - that is why we created concurrent threads is proxylab. We did not want a small request to be blocked by a large one that came earlier. We wanted to be able to serve the small request quickly (keep the latency low) even if a large request came first.
On a large scale, throughput matters also, for example if you serving millions of requests at once.
For the user, it is a question of latency. Users will not use a site if every request takes a long time to be handled. For the owner of the server, it is both. They want to keep latency down because it needs to be low for the user, and they need throughput high because they want to be able to service many users.
I believe threads helped with both latency (as you could get the data faster as a user) and throughput(from both perspectives - user and server, server as explained in muchanon's comment and user because imagine a user making multiple requests and all being served concurrently).
And the I think the caching part mainly helped with latency, as it reduced the response time for a request and saved a round trip to the server. I think it was not very helpful with throughput as it was shared and had contention (for writing).
It is not a question of throughput. Assume there are 2 requests, one small and the other large. Even if the larger one gets through first keeping the small one waiting, the processor is not idle and is busy processing the bigger request.
I think latency should be the priority for the site. As long as the site is consistently able to respond to requests with low latency, everyone should be happy. The problem arises when a lack of throughput harms latency i.e. we are bottlenecked. So, while throughput may be important to keep in mind, it is only because it harms the ultimate goal of latency.