Last week I came across a blog post titled NVGRE Musings. It’s got some great links to posts about two recently introduced proposals - VXLAN and NVGRE. But what drew my attention was the following thought from the first paragraph:
Supporting an L2 service is important for virtualized servers, which need to be able to move from one physical server to another without changing their IP address or interrupting the services they provide.
I don’t know if this is what network vendors are hearing from their big customers, but I strongly believe it's not the right answer to failover needs in the long term. No matter how hard I am looking, I don't see failover within the application as the network layer's problem to solve. The fact that they can doesn’t mean they should. My experience, as well as my understanding of current state of affairs in the biggest web-based tech ops organizations, lead me to a conclusion that the best place to handle application-level failover is application software itself.
Since application lives on top of the network layer, network is not in a good position to provide custom tailored failover solutions - all it can do is generic functionality that must remain transparent to the app. The biggest selling point of this approach is customers don’t need to re-architect their applications. There is a mounting body of evidence, however, that application does benefit from at least some modernization when it’s being moved to a newer operating environment (this was the apporach taken by Netflix who picked parts of the old app that they liked and discarded parts that they found lacking after years of running them in their datacenter environment or that were found unfit for their new IaaS environment).
George Reese said it best:
[...] New apps -> app is responsible for availability; Old apps -> infrastructure is responsible for availability
With this said, I do see a part of the network stack where the need for a new standard is the greatest. I am talking about DNS service name resolution.
Some parts are being addressed by Google in their efforts around including a part of end-user’s IP address as a part of a resolution request so that a more meaningful geo-distribution can be done by authoritative DNS servers. But I think it should not stop here.
Right now when a client requests a list of IP addresses corresponding to a given hostname, it simply gets a list of IPs (which in general case technically is not even ordered). Instead, the response could include a lot more meta information: “here is a list of IP addresses corresponding to the service you requested - try them in this order; if all of them fail to respond with such and such timeout, here is a backup list; and finally here is a token for your request - if you fail to get any response from any of the IPs listed within such and such timeout, retry this DNS query.”
With expanded retry and failover capabilities in host name resolution protocol on the client, it will become much easier to build hyper-distributed highly available services and applications - and that’s what I think the network industry should be focusing on.