Megam: Cloud Buzz

Amazon Web Services is out with another offering: Spot Instance. This falls under the EC2 umbrella with new and innovative gambling for instances methodology. No, not gambling, but its more like studying the fluctuations in spotprices and making the right bid.

Amazon decides the spotprices on the the basis of some kind of algorithm which take sthe availability of resources into consideration. But, all you need to do is

• analyze the flexibility of your needs
• choose the region and number of instances that you are interested in
• study the history of the spot prices for that region
• place a stop instance request
• place a bid or the maximum you are willing to pay
  

Now what?

Wait!

Well, assume that you have placed bid of $.04/hr and the stop price at that moment is $.045/hr, your instances will come up only, and only if, the spot price falls below your bid. That is, it should $.039/hr or lower.

The catch is: if the spot price moves above, mind you above, your bid; the instance will be terminated immediately. That is if the spot price is $.04/hr your instance will still continue to run, depending on the availability.

Bid or Spot?

Now this is fairly interesting, at least for me. If your instances do come up, satisfying the above criteria, you would pay the spot price. Yes, you heard it right, the spot price and not the bid price. So, this is where you will end up saving.

Apart from that, when your instance is terminated by amazon dues spot price fluctuation, you would not be paying for the partial hour. But, if you decide to terminate the instance from some reason, you will end up paying for the partial hour.

Bright Spots

Spotprice history is a good trend watch mechanism. Analyzing spotprice history will give details on the fluctuating trends of spotprices. This will help you decide and place bid. There is an api to spotprice history.

Bids could be onetime or persistent. Onetime bids will vanish the moment your spot instance, for the bid, is run and terminated. While persistent bids are persistent :)

Launch Groups, is great mechanism of making sure that a group of instances will launch together only, otherwise none will be launched, subject to request bids for those instances. The other way to group instances will be to launch all of them in the same availability zone.

Tight Spots!

Spot instance are very volatile and hence you need take care in using them for specific use cases only. Like those that are not bound by time, like flexi processing of scientific data. For example, you have already a cluster of instances running. But intermittent boost are always welcome, within given a budget, spot instances are the way to go.

Being highly volatile, there is need to take care of persistence and your applications should be modeled along those lines, so they(the applications) use AWS’s persistent capabilities, like EBS and S3 to maintain persistence.

There is no notification when the instances comeup, so you need to take care of that yourself. And the same goes for, when the instances are terminated. These alerting mechanism need to be built into your ami, so when they launch you are intimated.

My Spot

Spot instances are a very innovative mechanism to harness compute within budgets, but that needs to be taken with a pinch of salt and imagination. Good trend analyzing mechanism should evolve to harness these Spot Instances.

Overall, I believe this is a very helpful service for scientific needs.