Category: cloud-computing

How Enterprise IT Gave Rise to Cloud

Have you ever noticed that enterprise IT organizations often times have (sometimes numerous) project managers but never have product managers? Did you know that this fact is directly responsible for the rise of cloud computing? Read on if you want to learn more. Enterprise IT is an interconnected set of...

Read more...

Response To Simon Wardley: Innovation in Interface Implementations

You probably heard about the most recent episode of “AWS API in Openstack” saga. If you haven’t, head over to Nati Shalom’s blog to read one of the best recaps I have seen. My personal position in this discussion is very simple. I am not saying Openstack should fully adopt...

Read more...

What "Software Defined" Actually Means

There seems to be a pretty widespread belief held by many IT practitioners that “software defined” stands for something that can be dynamically configurable or something that offers all or most of its administration functions via API. While you won’t hear an argument from me against the fact that these...

Read more...

Applying 5 Whys to Amazon EC2 Outage

Earlier this week AWS published a post-mortem report about their last week’s outage - http://aws.amazon.com/message/67457/. Of several impairments and service disruptions caused by the outage, an hour-long unavailability of us-east-1 control plane is in my opinion the most important. Let’s apply 5 whys analysis to this impact. All answers below...

Read more...

How AWS Could Improve Spot Market

I recently noticed a surprising lack of new features being announced for AWS spot market. This disappoints me, as I have had high hopes for this groundbreaking idea since it was initially launched. Assuming spot price is set only through supply and demand, I think the following 3 suggestions could...

Read more...

Amazon Web Services and Innovator's Dilemma

AWS rolled out yet another service last week - Simple Workflow Service (SWF). I haven’t had a chance to kick the tires yet but I liked what I saw in the docs. But then on Twitter I started noticing people expressing concern about how this new service stacks up against...

Read more...

On Amazon EC2 Spot Price Spikes

Last week I came across an interesting post on Amazon EC2 spot price spikes published on GigaOm. In the comments, in response to a question from a reader, the author stated that “I don’t think anyone ever expected that the market would behave like this.” I have been interested in...

Read more...

VXLAN and NVGRE - Not a Long Term Answer

Last week I came across a blog post titled NVGRE Musings. It’s got some great links to posts about two recently introduced proposals - VXLAN and NVGRE. But what drew my attention was the following thought from the first paragraph: Supporting an L2 service is important for virtualized servers, which...

Read more...

Amazon EC2 Spot Instances - A Flop?

When Amazon Web Services launched EC2 spot instances in December 2009, I was very excited about the beginnings of potential revolution in how computing resources could be priced, bought and sold. I have followed this unprecedented phenomenon with great interest, blogging my thoughts along the way. But today, over 1.5...

Read more...

Network: From Hardware Past To Software Future

At this year’s GigaOm Structure conference, there was a single event that attracted my interest the most - network virtualization panel (I didn’t attend the conference, I was only following along over the Internet). It wasn’t just because it involved OpenFlow. I think there is a bigger trend at play...

Read more...

IaaS vs PaaS

For a very long time, I had regarded platform-as-a-service (PaaS) as a catch-all bucket for everything cloudy that was not software delivered over Internet on demand (SaaS) or infrastructure (IaaS). Over the past several months however, with announcement of new players in PaaS space such as CloudFoundry and OpenShift, I...

Read more...

On Importance of Planning for Failure

As you probably heard, on May 21 of this year US East region of Amazon EC2 cloud experienced a severe outage. The event received considerable coverage around the blogosphere - you can find the most comprehensive collection of links on the topic at highscalability.com. The guidelines of design for failure...

Read more...

Netflix The Rising Star of Cloud

Every disruptive change has its pioneers. Someone must be the first to think of it, someone must be the first to launch it, someone else could be the first to use it at scale. While most observers agree that Amazon Web Services are an undisputed pioneer of cloud computing on...

Read more...

On Nuances of Jevons Paradox in Cloud Computing

I am not a formally trained economist. I am a software engineer, and economics (and economics in IT in particular) is merely a hobby of mine. I have now seen many mentions of Jevons paradox in the context of cloud computing. Initially, when I first learned about it, it sounded...

Read more...

Cloud As Application Data Exchange Point

Among numerous technical components that comprise an infrastructure-as-a-service cloud, there is one that usually draws most criticism and causes most annoyance. I am talking about the network. Too flat, too inflexible, too slow, too unpredictable - cloud network has been accused of being each and every one of these. And...

Read more...

My Doubts About Idea Behind SpotCloud.com

This is part 5 of my series on pricing in the cloud. The moment a second provider joined Amazon Web Services in offering IaaS cloud computing service, people in the industry started talking about a cloud computing marketplace where providers would be able to list available resources and customers would...

Read more...

Basics of IaaS Spot Pricing

This is part 4 of my pricing in the cloud series. Exactly one year ago - on Monday, December 14, 2009 - Amazon Web Services launched spot pricing as a new feature of Amazon EC2. In a nutshell, spot pricing is a dynamic pricing scheme for EC2 instances. At each...

Read more...

Unexpected Similarities in EC2 Spot Price History Between Regions

If you have been following my blog for a while, you may recall a post from earlier this year where I looked at some basic statistical properties of time series behind Amazon EC2 spot price history. Because EC2 regions are completely separate and independent, you’d think spot price history for...

Read more...

Connecting to Eucalyptus Walrus (S3) with Boto and socket.gaierror

If you are trying to perform any bucket operations on Eucalyptus Walrus (S3) service from boto and are getting “socket.gaierror: [Errno -2] Name or service not known”, it’s because you did not specify the proper calling_format when creating your S3Connection object. Default calling format is SubdomainCallingFormat which will attempt to...

Read more...

Run on the Cloud

This is part 3 of my series on pricing in the cloud. Do you think IaaS cloud prices are due for another reduction or will they remain at current levels for now? Regardless of which answer you or I would give to this question, ours is just a guess -...

Read more...

Dealing with Noisy Neighbors in the Cloud

This is part 2 of my series dedicated to pricing in the cloud. As I mentioned in the past, pricing is one of the most important aspects of cloud computing offerings. Up until now, however, I have been talking about pricing only from perspective of selling the services. This post...

Read more...

Pricing in the Cloud

I started writing this post, but realized that it was going to be way too long. So I decided to split it into a series. Here goes part 1 of my series on Pricing in the Cloud. Happy Birthday, Amazon EC2! You are 4 years old now - I wonder...

Read more...

Extending EC2 API - ec2-describe-ipaddress-ranges

Do you remember how we used to programmatically consume services on the web before proliferation of APIs? That’s right - scraping! And do you know what prevents us from using this technique now, when some piece of data you need for your application, is not available via API? That’s right...

Read more...

Parallelize Your EC2 API Calls with Python, Boto and Threading

I started a small new project on Github - http://github.com/somic/ec2-multiregion. It includes several small tools that facilitate EC2 API operations that involve multiple regions at the same time. If I were to query each endpoint one after another, I quickly discovered it would take too long. Therefore, I created a...

Read more...

Russell's Paradox and Cloud Computing

I am sure you’ve heard of Bertrand Russell’s paradox and one of its more widely known versions - Barber paradox. But let me rephrase the Wikipedia article: Suppose there is a town with just one public IaaS cloud provider, and that every business in the town runs their own IT:...

Read more...

CohesiveFT VPN-Cubed as Networking Fabric of the Intercloud

This post is about stuff I work on at my current job. I do not speak for my employer on this blog though, therefore please consider thoughts and opinions below as strictly my own, not necessarily endorsed or approved by CohesiveFT. It has been about 6 months since I last...

Read more...

Probing Ports in Remote Security Groups in EC2

This is the third part of my series on Amazon EC2 security groups. In part 1, I described how security groups are possibly the most underappreciated feature in EC2. In part 2, I described a UDP hole punching technique, which led to some interesting conclusions. On several occasions, when troubleshooting...

Read more...

IaaS, Hype and Marginal Cost

Theo Schlossnagle published a great piece titled The cloud is great. Stop the hype. As a technologist, I totally agree that hype is what’s killing it. In fact, on several occasions I mentioned to my co-workers that often I get the feeling that cloud computing is already a bubble that...

Read more...

Workloads in Cloud Computing

In computer science, according to Wikipedia, abstraction is a “mechanism to reduce and factor out details so that one can focus on a few concepts at a time.” When you hear about abstraction in the context of virtualization-based IaaS cloud computing, the most well known abstraction is computing resources themselves...

Read more...

Digging into EC2 Spot Price History

In December 2009, Amazon Web Services team introduced yet another innovation - spot pricing for EC2 instances. Several sites were created shortly to track spot price history by creating price charts. But price charts are relatively boring - juicy meat is in the dynamics hidden inside series of numbers which...

Read more...

Cloud Overlay Networks Demystified - Holiday Edition

As most of you probably know, I work at CohesiveFT where I focus on VPN-Cubed product. In short, it’s a solution to build overlay networks in third-party clouds. Overlay networks in this case are based on redundant encrypted point-to-point connections from your regular servers to your VPN-Cubed servers called “managers”...

Read more...

My Thoughts After CloudCamp Boston 2009

This past Thursday I had a chance to attend CloudCamp Boston that took place in Microsoft research center in Cambridge, as a representative of CohesiveFT. The event was very well attended, and I was able to meet a lot of smart interesting people working in cloud computing space. The lightning...

Read more...

Costs vs Agility as Drivers for Cloud Computing

I have recently noticed that costs were no longer always touted as the main driver for cloud computing - some have been advocating agility as the primary reason (for example, see here). It’s one thing when this theme gets mentioned in a talk at a technology conference where a company...

Read more...

Punching UDP Holes in Amazon EC2

Disclaimer 1: Despite its possibly ominous name, this is NOT a network vulnerability or an attack that could lead to unauthorized access. UDP hole punching requires cooperation between two hosts, and hence can't be easily used as an attack by itself (in other words, in order to run it, you...

Read more...

Standalone Web Front Door a Must in EC2?

Most of you have probably heard about a recent outage at BitBucket. In a nutshell, their systems hosted at AWS came under a UDP flood DDoS attack, which led to significantly increased traffic, which led to saturation of their local network interface, which led to their being unable to connect...

Read more...

Security Groups - Most Underappreciated Feature of Amazon EC2

Having been developing software to run on Amazon EC2 for over a year now, I find security groups to be among its least understood and appreciated features. Basic Usage In short, EC2 security group (SG) is a set of ACCEPT firewall rules for incoming packets that can apply to TCP,...

Read more...

On Cloud Lock-In

I left this comment on today's post by Randy Bias titled VMWare vs Amazon... ROUND ONE... FIGHT!: Functionality is more important, imho. As a hypothetical example, say there exists an EC2-like cloud where security groups span all regions (in EC2, as we all know, security groups are confined to a...

Read more...

Shiny Cloud APIs - Necessary But Not Sufficient

In the stream of non-stop cloud computing chatter that was surrounding VMWorld 2009 that wrapped up last week, I noticed a pattern - folks were paying disproportionate amount of attention to API, API portability and API standardization, as opposed to actual technology concepts and constructs that are going to power...

Read more...

Evaluating Cloud Computing from Buy vs Rent Perspective

What is driving people, projects and organizations to adopt cloud computing? There is no single answer. Everyone’s situation is different, and everyone assigns different weights to different factors. But what is common in “to cloud or not to cloud” decision making is that fundamentally it’s like buy vs rent in...

Read more...

Developer's Attempt to Define Cloud Computing

I have been closely following cloud computing for many months now. As a developer, I get often frustrated by lack of clear and widely accepted definition of what cloud computing actually is. This is a problem, because without a definition, every imaginable operation performed over the Internet all of a...

Read more...

How Long Ago Was This EC2 Instance Started?

By accident, today I discovered an easy way how to determine how long ago your EC2 instance was started. Note that uptime shows time since last reboot, so it’s not what we get here. Here is a bash implementation.

Read more...

Incentives and Cloud Computing Interoperability

To succeed, cloud interoperability must drive down costs for cloud computing vendors, both established and aspiring. This is how interop has been achieved throughout the history - look at car industry, railroads (selecting rail gauge), etc. Or check out Wikipedia article on Standardization in general. Indeed, for something to be...

Read more...

My Comment on Open Federated Clouds

I left the following comment at CloudAve yesterday, on a post titled Open Federated Clouds And Sun's Cloud Announcement. Interesting. Looks to me it all depends on how you look at different clouds - as infrastructure providers or as software platforms. The former case is roughly similar to buying Internet...

Read more...

CohesiveFT Launches VPN-Cubed For Amazon EC2

Update: CohesiveFT now also offers IPsec connectivity to VPN-Cubed running inside Amazon EC2. Read more. Today CohesiveFT team officially launches VPN-Cubed for Amazon EC2, a product that has been in beta for several weeks now. Check out the announcement on Elastic Server blog, which talks about both Pay and Free...

Read more...

APIs Foreva, or What If Cloud Computing Were a Bubble

I recently came across a fascinating book - Pop! Why Bubbles Are Great For The Economy, by Daniel Gross, author and Moneybox columnist for Slate. In it, he focuses on positive consequences of several bubbles in US economic history (namely telegraph, railroads, financial New Deal, Internet, and real state) and...

Read more...

Identification Friend or Foe (IFF) in IaaS Clouds

I was recently building a distributed system which will run in Amazon EC2 cloud. It consisted of several instances of the same AMI that were going to communicate with each other using private IP addresses assigned by EC2. One interesting scenario popped up in my head. What if, after initial...

Read more...

Technical Overview of CohesiveFT VPN-Cubed

A technical post on VPN-Cubed that I contributed several thoughts to, is now up on CohesiveFT Elastic Server blog.

Read more...

When is a Cloud Not a Cloud

As “cloud computing” is gaining on its rivals in the race for the hottest IT buzzword du jour, I started noticing that many products that only yesterday were marketed as “hosting,” “grid” or “cluster” became “clouds” virtually overnight. I realized something the other day. For something to call itself a...

Read more...

Let's Prove GigaOM Wrong On Enterprises + Clouds

On Tuesday, GigaOM published 10 Reasons Enterprises Aren’t Ready to Trust the Cloud. I personally think the title is somewhat misleading. It would have been more appropriately named “10 Reasons Enterprises Aren’t Ready to Take Their Entire In-House IT Operations to the Cloud.” The difference is huge. Enterprises can totally...

Read more...

EC2 Instances Always Boot to Runlevel 4

It appears that Amazon EC2 instances always boot to runlevel 4, no matter what runlevel you set as default in your /etc/inittab. I found a very old reference to this fact in forums (see here), which explains how it happens (XEN does it) but doesn’t explain why. And yes, by...

Read more...

EC2 for Web Hosting

Lack of persistent disk out of the box and IP addresses that are not preserved between reboots are indeed the 2 primary issues that make hosting public web sites on Amazon EC2 “tricky” (even though not impossible). Instead of addressing the problem head on, have you considered multisourcing your infrastructure?...

Read more...