It does. Most of my IT-focused colleagues would agree, even if they very much enjoy their job and the toys with which they get to play. The problem is that server infrastructure has just become too convoluted and interdependent, which means that failures can have catastrophic ripple effects, that security is extremely difficult to manage, and that application deployment can be excruciatingly slow. I am thoroughly convinced that the best way for most companies to run their IT applications, both internal and customer-facing, is on some sort of PaaS running in a public cloud.
Did I just say public? Yes, public. Because private clouds are a gutless compromise between security, cost and scalability and because private clouds allow for an unhealthy dose of laziness when it comes to security. Public clouds force developers to use encryption and authentication. Private clouds are just a different name for the same data center that’s been around for the last fifteen years. Choosing a hosted platform (not a hosted infrastructure) forces application developers to be flexible about underlying hardware and operating systems, yielding code that will run in a variety of environments instead of only one tightly-architected stack. That means freedom from vendors and easy bursting for excess capacity. It means EngineYard, Node.js, AppFog, Cloud Foundry, etc. (I’d personally avoid “raw” EC2 and similar solutions, simply because I’d much rather have someone else managing an operating system for me.)
Clouds over Echo Summit, South Lake Tahoe
When I first started developing web-based applications, they had to run them on a very carefully-managed cluster of BSD-UNIX machines. I had to take care of backups, failover, monitoring, patching, upgrades, hard disk failures, firewalls and network connectivity. The servers ran on-site because bandwidth was precious. Very few people had the expertise to troubleshoot the stack when things went wrong, and if I went too long without poking around on my own servers, I would sometimes forget how it was all cobbled together.
I eventually left the application development world and moved into technical sales, where I helped customers buy super-sophisticated Ethernet and Fibre Channel switches. When network virtualization companies started popping up I went to work for one of them, thinking that network virtualization was a great idea to abstract away a layer of the data center, making life simpler. After all, abstraction is a fundamental part of computer science, isn’t it?
I agree that server and network virtualization are necessary steps for enterprises to move into the cloud. But why should they move into the cloud in the first place? There’s a great infographic on Mashable that points out only 17% of companies move into the cloud for cost reasons. The real driver is often mobility – access to information from any device – or “elasticity” – the ability to start an application for a small number of users, and scale it up as the customer base grows.
The problem is that most enterprise applications don’t really support mobility. And even fewer support true elasticity. Scale-out requires a fundamental change in architecture from the legacy scale-up technology that IT departments have become so adept at managing. Scale-out often means a much looser coupling between data and logic. Scale-out relies on generic IP connectivity, not a finely-tuned Fibre Channel SAN with carefully-provisioned LUNs, and certainly not a statically-provisioned network laden with hundred-line ACLs and QoS maps.
On the other hand, a true cloud application makes some assumptions and some sacrifices, and at the end of the day it pulls a lot of logic out of the traditional IT department and into the application or platform’s code. Networking, for example: since a cloud is inherently shared, network access must be presumed to be insecure. Rather than relying on the IT department’s network team to “lock down” a server’s VLAN, ACL, firewall rules (which is a fool’s errand to begin with), a cloud application must expose itself to network requests only after appropriate authentication. No cleartext, unauthenticated database connections, for instance, which are still shockingly prevalent in enterprise-grade applications.
Image by Andrew McKaskill
There exist today a bewildering list of open-source and commercial packages upon which to build a “cloud” application. We’ve all heard of Ruby on Rails, OpenStack, Hadoop, HBase, Cassandra, Amazon EC2, EBS. As a developer, when I look at my options for hosting a web application I find it rather daunting. As an MBA, I realize that very few of those options really deliver on the promise of the cloud – the cloud that we see in billboard ads: “resilient, elastic, secure, cheap, mobile”.
Obviously it’s possible to build a great cloud application on top of EC2; from Netflix to Smugmug, there are many success stories. But Netflix and Smugmug, just like Google, Facebook, LinkedIn and many others, have no business outside the cloud. They have armies of developers who work tirelessly to capture the relevant bits of network, storage, backup and security in their own proprietary code. What about the rest of us? How would a regional transit authority, for example, build the back-end for an iPhone application that lets passengers ride the train with nothing more than their mobile phone? How about an ad agency working on a social media project with a quick turnaround? A developer in a dorm room with a great idea?
The recent meteoric rise in popularity of “apps” has been driven by the fact that they encapsulate just the right amount of “business logic” in a cheap and easily-consumed medium. The value of an application is in its customer-facing logic – most of the back-end logic is irrelevant (as long as it works). VLAN tagging, LUN provisioning, RAID configuration, operating system patches and host security are just there to support the application logic, right? Somehow we have become so focused on the infrastructure that the real objective is lost in the confusion.
Infrastructure shouldn’t be taken for granted, but it also shouldn’t dictate how an application is designed, developed and deployed. With offerings today like those from Engine Yard, Joyent and all the others, there’s very little in between a great idea and paying customers. As a developer, a sales guy and a marketer, I’d say that’s a pretty great thing.
