The idea of autoscaling is can be described as Load Balancing -- making workload distribution more efficient. It helps cloud admins not waste money (i.e. not have too many cloud instances running idley), and have enough resources (ensure a website does not slow down due to too many requests).
holard
Does the time required to add servers to a worker pool cause problems w.r.t sudden spikes in traffic? I.e. by the time the new worker is up and running it may be too late already.
harlenVII
I think we need to consider that. The provision time is not negligible. So maybe the rule of scaling needs to be a little conservative?
khans
What happens to the servers being rented out when Amazon has a burst of traffic? Does it reduce how many servers it rents out around Christmas?
mak
@holard Time required to add new servers to the worker pool can impact the quality of service. Therefore a typical implementation of perf monitor will need to make predictions about the no. of required servers in near future based on the current trend in load change.
For example, if a server starts with 4 worker and perf monitor observes that the CPU utilization of these workers exceed certain threshold, let's say 80% for a period of time. Then, it can increase the no. of workers from 4 to let's say 8 (not 5). Cpu utilization threshold, new no. of servers, etc. are tuneable parameters and which involve trade-off between quality of service and operational cost.
jedi
@khans, AWS is no longer a subset of Amazon (retail), but rather a first-class citizen. In fact, Amazon is probably one of the largest users of AWS services. Most of the time, the promise of "the cloud" is that across a large number of customers, demand is normalized (think the law of large numbers). If it's Valentine's day, everyone is busy Netflix-and-chilling (or whatever you kids do these days), so Coursera can scale down the number of autograding servers they have running.
For "special" days like Black Friday / holidays, where demand is correlated, such that it does not average out, the way it seems to work is:
1. Amazon retail does capacity planning and estimates the number of servers it will need in Q4 and places an order with AWS to reserve this capacity. Other large customers (Netflix), reserve capacity in the same way. Pre-provisioning resources is cheaper for applications on the cloud.
2. AWS now receives a large number of reservations for resources in Q4 and uses this to estimate whether it needs to buy more servers to deal with the load in any given period.
[Source: adding large amounts of beer to an AWS engineer]
For EC2, I believe this thing that automatically adds or removes web servers is called an Autoscaler -- Here is a description of one on Amazon's site.
The idea of autoscaling is can be described as Load Balancing -- making workload distribution more efficient. It helps cloud admins not waste money (i.e. not have too many cloud instances running idley), and have enough resources (ensure a website does not slow down due to too many requests).
Does the time required to add servers to a worker pool cause problems w.r.t sudden spikes in traffic? I.e. by the time the new worker is up and running it may be too late already.
I think we need to consider that. The provision time is not negligible. So maybe the rule of scaling needs to be a little conservative?
What happens to the servers being rented out when Amazon has a burst of traffic? Does it reduce how many servers it rents out around Christmas?
@holard Time required to add new servers to the worker pool can impact the quality of service. Therefore a typical implementation of perf monitor will need to make predictions about the no. of required servers in near future based on the current trend in load change.
For example, if a server starts with 4 worker and perf monitor observes that the CPU utilization of these workers exceed certain threshold, let's say 80% for a period of time. Then, it can increase the no. of workers from 4 to let's say 8 (not 5). Cpu utilization threshold, new no. of servers, etc. are tuneable parameters and which involve trade-off between quality of service and operational cost.
@khans, AWS is no longer a subset of Amazon (retail), but rather a first-class citizen. In fact, Amazon is probably one of the largest users of AWS services. Most of the time, the promise of "the cloud" is that across a large number of customers, demand is normalized (think the law of large numbers). If it's Valentine's day, everyone is busy Netflix-and-chilling (or whatever you kids do these days), so Coursera can scale down the number of autograding servers they have running.
For "special" days like Black Friday / holidays, where demand is correlated, such that it does not average out, the way it seems to work is: 1. Amazon retail does capacity planning and estimates the number of servers it will need in Q4 and places an order with AWS to reserve this capacity. Other large customers (Netflix), reserve capacity in the same way. Pre-provisioning resources is cheaper for applications on the cloud. 2. AWS now receives a large number of reservations for resources in Q4 and uses this to estimate whether it needs to buy more servers to deal with the load in any given period.
[Source: adding large amounts of beer to an AWS engineer]