I recently noticed a surprising lack of new features being announced for AWS spot market. This disappoints me, as I have had high hopes for this groundbreaking idea since it was initially launched. Assuming spot price is set only through supply and demand, I think the following 3 suggestions could be of interest.
Let’s start with the key problem for spot market customers today - their biggest problem right now is managing anomalous price spikes. I emphasize that I use the word "manage" instead of "avoid" because I am focusing on general case, not on any specific strategy. And in general case, one can’t categorically demand that absolutely all spikes be avoided.
In order to be able to manage the spikes, customers must be able to see them coming (with some confidence level). Right now we can only rely on pricing charts and our ability to extrapolate past spike patterns into the future. This is extremely unreliable and error-prone. And besides, there is no guarantee that past patterns will still be valid in the future. Instead, customers need a better way.
Suggestion #1: AWS should publish real-time indicator of current available supply.
It could be an absolute number (“N spot instance slots remaining”) or it could be a range (“the number of spot instance slots currently remaining is between N and M”).
The idea here is that the lower these numbers are, the more likely spot price is about to go up. Nothing is guaranteed of course - supply could be decreasing without any significant price changes. But in theory every spike is preceded by a drop in available supply (while not every drop necessarily leads a spike).
Next. Once we get a more or less good idea when spikes could be coming, we need to know how large they could potentially get. And this leads to the next suggestion.
Suggestion #2: AWS should publish the top of its order book in real-time.
Essentially it’s a list of N highest bids, along with corresponding sizes of each bid. This will kill two birds with one stone. On one hand, there will be a clear reference point how high spot price could get in extreme situation if a spike were to occur right now. On the other hand, it will establish an upper-side price anchor that could reduce irresponsible bidding, which in its turn could reduce severity of the spikes - both I expect are going to be good news for customers.
For a provider, it’s important to avoid publishing entire order book though - as I showed before, provider benefits from diversity of bids, especially at the low end.
Suggestion #3: Allow in-place modification of a running spot instance's bid.
Right now, even if you know when a spike is coming and you can estimate how high it could be, there is nothing you as customer can do except terminate your instance or let it run - you can’t do anything to survive a spike if it’s going to exceed your instance’s price. This severely limits your options and makes proper management of price spikes pointless.
I realize that these could be far-reaching suggestions. Coupled with AWS’ preference for secrecy, they could even be nearly impossible. But without these (or anything similar), spot market will remain very difficult to bid on intelligently - in its current form, it’s just too random.