Rational Survivability

This a post that's bigger than a thought on Twitter but almost doesn't deserve a blog, but for some reason, I just felt the need to write it down.  This may be one of those "well, duh" sorts of posts, but I can't quite verbalize what is tickling my noggin here.

As far as I can tell, the juicy bits stem from the intersection of cloud cost models, cloud adopter profile by company size/maturity and the concept of the consumerization of IT.

I think 😉

This thought was spawned by a couple of interesting blog posts:

1. James Urquhart's blog titled "The Enterprise barrier-to-exit in cloud computing" and "What is the value of IT convenience" which led me to…
2. Billy Marshall from rPath and his blog titled "The Virtual Machine Tsunami."

These blogs are about different things entirely but come full circle around to the same point.

James first shed some interesting light on the business taxonomy, the sorts of IT use cases and classes of applications and operations that drive businesses and their IT operations to the cloud, distinguishing between what can be described as the economically-driven early adopters of the cloud in SMB's versus mature larger enterprises in his discussion with George Reese from O'Reilly via Twitter:

George and I were coming at the problem from two different angles. George was talking about many SMB organizations, which really can't justify the cost of building their own IT infrastructure, but have been faced with a choice of doing just that, turning to (expensive and often rigid) managed hosting, or putting a server in a colo space somewhere (and maintaining that server). Not very happy choices.

Enter the cloud. Now these same businesses can simply grab capacity on demand, start and stop billing at their leisure and get real world class power, virtualization and networking infrastructure without having to put an ounce of thought into it. Yeah, it costs more than simply running a server would cost, but when you add the infrastructure/managed hosting fees/colo leases, cloud almost always looks like the better deal.

I, on the other hand, was thinking of medium to large enterprises which already own significant data center infrastructure, and already have sunk costs in power, cooling and assorted infrastructure. When looking at this class of business, these sunk costs must be added to server acquisition and operation costs when rationalizing against the costs of gaining the same services from the cloud. In this case, these investments often tip the balance, and it becomes much cheaper to use existing infrastructure (though with some automation) to deliver fixed capacity loads. As I discussed recently, the cloud generally only gets interesting for loads that are not running 24X7.
…

This existing investment in infrastructure therefore acts almost as a "barrier-to-exit" for these enterprises when considering moving to the cloud. It seems to me highly ironic, and perhaps somewhat unique, that certain aspects of the cloud computing market will be blazed not by organizations with multiple data centers and thousands upon thousands of servers, but by the little mom-and-pop shop that used to own a couple of servers in a colo somewhere that finally shut them down and turned to Amazon. How cool is that

That's a really interesting differentiation that hasn't been made as much as it should, quite honestly.  In the marketing madness that has ensued, you get the feeling that everyone, including large enterprises, are rushing willy-nilly to the cloud and outsourcing the majority of their compute loads, not the cloudbursting overflow.

Billy Marshall's post offers some profound points including one that highlights the oft-reported and oft-harder-to-prove concept of VM sprawl and the so-called "frictionless" model of IT, but with a decidedly cloud perspective. 

What was really interesting was the little incandescent bulb that began to glow when I read the following after reading James' post:

I've yet to figure out if the notion of frictionless capacity is a good thing or not if your ability to capacity plan is outpaced by a consumption model and a capacity yield that can just continue to climb without constraint.  At what point does the crossover between cost savings from infrastructure that bounded costs by resource constraints of physical servers become eclipsed by runaway use?

I guess I'll have to wait to see his bill 😉

Back to James' post, he references an interchange on Twitter with George Reese (whose post on "20 Rules for Amazon Cloud Security" I am waiting to fully comment on) in which George commented:

"IT is a barrier to getting things done for most businesses; the Cloud reduces or eliminates that barrier."

…which is basically the same thing Billy said in a Nick Carr kind of way.  The key question here is for whom?  As it relates to the SMB, I'd agree with this statement, but the thing that really sunk it was that statement just doesn't yet jive for larger enterprises.  In James' second post, he drives this home:

I think these examples demonstrate an important decision point for IT organizations, especially during these times of financial strife. What is the value of IT convenience? When is it wise to choose to pay more dollars (or euros, or yen, or whatever) to gain some level of simplicity or focus or comfort? In the case of virtualization, is it always wise to leverage positive economic changes to expand service coverage? In the case of cloud computing, is it always wise to accept relatively high price points per CPU hour over managing your own cheaper compute loads?

Is the cloud about convenience or true business value?  Is any opportunity to eliminate a barrier — whether that barrier actually acts as a logical check and balance within the system — simply enough to drive business to the cloud?

I know the side-stepping IT bit has been spoken about ad nauseum within the context of cloud; namely when describing agility, flexibility, and economics, but it never really occurred to me that the cloud — much in the way you might talk about an iPhone — is now being marketed itself as another instantiation of the democratization, commoditization and consumerization of IT — almost as an application — and not just a means to an end.

I think the thing that was interesting to me in looking at this issue from two perspectives is that differentiation between the SMB and the larger enterprise and their respective "how, what and why" cloud use cases are very much different.  That's probably old news to most, but I usually don't think about the SMB in my daily goings-on.

Just like the iPhone and its adoption for "business use," the larger enterprise is exercising discretion in what's being dumped onto the cloud with a more measured approach due, in part, to managing risk and existing sunk costs, while the SMB is running to embrace it it at full speed, not necessarily realizing the hidden costs.

/Hoff

Update: Ooops.  I forgot to announce that I'm once again putting on my Devil's Advocacy cap. It fits nicely and the contrasting color makes my eyes pop.;)

It should be noted that obviously I recognize that dedicated
hardware offers performance and scale capabilities that in many cases
are difficult (if not impossible) to replicate in virtualized software
instantiations of the same functionality. 

However, despite spending the best part of two years raising
awareness as to the issues surrounding scalability, resiliency,
performance, etc. of security software solutions in virtualized
environments via my Four Horsemen of the Virtualization Security Apocalypse presentation, perception is different
than reality and many network capabilities will simply consolidate into the virtualization platforms until the next big swing of the punctuated equlibrium.

This is another classic example of "best of breed" versus "good enough" and in many cases this debate becomes a corner-case argument of speeds and
feeds and the context/location of the network topology you're talking
about. There's simply no way to sprinkle enough specialized hardware around to get the pervasive autonomics across the entire fabric/cloud without a huge chunk of it existing in the underlying virtualization platform or underlying network infrastructure.

THIS is the real scaling problem that software can address (by penetration) that specialized hardware cannot.

There will always be a need for dedicated hardware for specific needs, and if you have an infrastructure service issue that requires massive hardware to support traffic loads until the sophistication and technology within the virtualization layer catches up, by all means use it!  In fact, just today after writing this piece Joyent announced they use f5 BigIP's to power their IaaS cloud service…

In the longer term, however, application delivery control (ADC) will ultimately become a feature of the virtual networking stack provided by software as part of a larger provisioning/governance/autonomics challenge provided by the virtualization layer.  If you're going to get as close to this new atomic unit of measurement in the VM, you're going to have to decide where the network ends and the virtualization layer begins…across every cloud you expect to host your apps and those they may transit.

I've been reading Lori McVittie's f5 DevCentral blog for quite some time.  She and Greg Ness have been feeding off one another's commentary in their discussion on "Infrastructure 2.0" and the unique set of challenges that the dynamic nature of virtualization and cloud computing place on "the network" and the corresponding service layers that tie applications and infrastructure together.

The interesting thing to me is that why I do not disagree that that the infrastructure must adapt to the liquidity, agility and flexibility enabled by virtualization and become more instrumented as to the things running atop it, much of the functionality Greg and Lori allude to will ultimately become a function of the virtualization and cloud layers themselves*.

One of the more interesting memes is the one Lori summarized this morning in her post titled "Managing Virtual Infrastructure Requires an Application Centric Approach," wherein the she lays the case for the needs of infrastructure becoming "application" centric based upon the "highly dynamic" nature of virtualized and cloud computing environments:

…and yet the applications themselves, despite how integrated they may be, suffer from the same horizontal management problem as the network today does.  So I'm not so sure about the finality of the "it's all about the application" because we haven't even solved the "virtual infrastructure management" issues yet.

Bridging the gap between where we are today and the infrastructure 2.0/application-centric focus of tomorrow is illustrated nicely by Billy Marshall from rPath in his post titled "The Virtual Machine Tsunami," in which he describes how we're really still stuck being VM-centric as the unit measure of application management:

So we still have all the problems of managing the applications atomically, but I think there's some general agreement between these two depictions.

However, where it gets interesting is where Lori essentially paints the case that "the network" today is unable to properly provide for the delivery of applications:

This is where I start to squint a little because Lori's really taking the notion of "application intelligence" and painting what amounts to a router/switch in an appliction delivery controller as a "platform" as she attempts to drive wedge between an ADC and "the network."

Besides the fact that "the network" is also rapidly evolving to adapt to this more loosely-coupled model and the virtualization layer, the traditional networking functions and the infrastructure service layers are becoming more integrated and aware thanks to the homgenizing effect of the hypervisor, I'll ask the question I asked Lori on Twitter this morning:

Why won't this ADC functionality simply show up in the hypervisor?  If you ask me, that's exactly the goal.  vCloud, anyone?  Amazon EC2?  Azure?

If we take the example of Cisco and VMware, the coupled vision of the networking and virtualization 800 lb gorillas is exactly the same as she pens above; but it goes further because it addresses the end-to-end orchestration of infrastructure across the network, compute and storage fabrics.

So, why do we need yet another layer of network routers/switches called "application delivery controllers" as opposed to having this capability baked into the virtualization layer or ultimately the network itself?

So, can you really make the case for deploying more "application-centric" routers/switches (which is what an application delivery controller is) regardless of how aware it may be?

/Hoff

It's Thanksgiving here in the U.S., so in between baking, roasting and watching Risk Astley rickroll millions in the Macy's Thanksgiving Day Parade, I had a thought about how the utility and agility of the cloud computing models such as Amazon AWS (EC2/S3) and the pricing models that go along with them can actually pose a very nasty risk to those who use the cloud to provide service.

That thought — in between vigorous whisking during cranberry sauce construction — got me noodling about how the pay-as-you-go model could be used for nefarious means.

Specifically, this usage-based model potentially enables $evil_person who knows that a service is cloud-based to manipulate service usage billing in orders of magnitude that could be disguised easily as legitimate use of the service but drive costs to unmanageable levels. 

If you take Amazon's AWS usage-based pricing model (check out the cost calculator here,) one might envision that instead of worrying about a lack of resources, the elasticity of the cloud could actually provide a surplus of compute, network and storage utility that could be just as bad as a deficit.

Instead of worrying about Distributed Denial of Service (DDos) attacks from botnets and the like, imagine having to worry about delicately balancing forecasted need with capabilities like Cloudbursting to deal with a botnet designed to make seemingly legitimate requests for service to generate an economic denial of sustainability (EDoS) — where the dyamicism of the infrastructure allows scaling of service beyond the economic means of the vendor to pay their cloud-based service bills.

Imagine the shock of realizing that the outsourcing to the cloud to reduce CapEx and move to an OpEx model just meant that while availability, confidentiality and integrity of your service and assets are solid, your sustainability and survivability are threatened.

I know there exists the ability to control instance sprawl and constrain costs, but imagine if this is managed improperly or inexactly because we can't distinguish between legitimate and targeted resource consumption from an "attack."

If you're in the business of ensuring availability of service for large events on the web for a timed event, are you really going to limit service when you think it might drive revenues?

I think this is where service governors will have to get much more
intelligent regarding how services are being consumed and how that
affects the transaction supply chain and embed logic that takes into
account financial risk when scaling to ensure EDoS doesn't kill you. 

I can't say that I haven't had similar concerns when dealing with scalability and capacity planning in hosted or dedicated co-location environments, but generally storage and compute were not billable service options I had to worry about, only network bandwidth.

Back to the mashed potatoes.

/Hoff

James Urquhart posted a summary a week or so ago of what he described as the "Big 4" players in Cloud Computing.  It was a slightly humorous pass at describing their approaches and offerings:

James provided quite a bit more (serious) detail in the text below his table which I present to you here, tarted up with a column I've added and James left off titled "Security." 

It's written in the same spirit as James' original, so feel free to take this with an equally well-provisioned grain of NaCl.  I'll be adding my own perfunctory comments with a little more detail shortly: The point here is that the quantification of what "security" means in the cloud is as abstracted and varied as the platforms that provide the service.  We're essentially being asked to take for granted and trust that the underlying mechanicals are sound and secure while not knowing where or what they are.

We don't do that with our physically-tethered operating systems today, so why should we do so with virtualization platform hypervisors and the infrastructure "data center operating systems" of the cloud?  The transparency provided by dedicated infrastructure is being obscured by virtualization and the fog of the cloud.  It's a squeezing the balloon problem.

And so far as the argument goes toward suggesting that this is no different than what we deal with n terms of SaaS today, the difference between what we might define as legacy SaaS and "cloud" is that generally it's someone elses' apps and your data in the former (ye olde ASP model.) 

In the case of the "cloud," it could be a mixture of applications and data, some of which you own, some you don't and some you're simply not even aware of, perhaps running in part on your infrastructure and someone elses'.

It should be noted also that not all cloud providers (excluding those above) even own and operate the platforms they provide you service on…they, in turn, could be utilizing shared infrastructure to provide you service, so cross-pollination of service provisioning could affect portability, reliability and security.

That is why the Big4 above stand up their own multi-billion dollar data centers; they keep the architecture proprietary so you don't have to; lots of little clouds everywhere.

/Hoff

P.S. If you're involved with platform security from any of the providers above, do contact me because I'm going to be expounding upon the security "layers" of each of these providers in as much detail as I have here shortly.  I'd suggest you might be interested in assuring it's as complete and accurate as possible 😉

"Cloud Providers Are Better At Securing Your Data Than You Are…"

To some, this is a contentious point while to others it seems entirely logical.

I must tell you that I've witnessed this very assertion as it has been raised more times in the last few days than I can count.

Before I get into any more juicy bits regarding this topic, I wonder if you wouldn't mind popping over and reading a blog post I wrote in August, 2007 titled "On-Demand SaaS Vendors Able to Secure Assets Better than Customers?" 

Come to think of it, you can read the follow-on post to that one which clearly indicated my point when Salesforce.com and Monster.com (you know, those so-called "Cloud" providers) were breached.

Forgot about those breaches, did you?  Oh, that must have been because they were SaaS providers and not Cloud providers at the time.  Gotcha.

As you read these posts, first do so within the context of what we've come to know as software as a service (SaaS.)  Then kindly re-read it and substitute 'SaaS' with 'Cloud,' won't you?

Thanks.

I have more, but I'll wait till you're done.

/Hoff

I haven't been whipped into this much of a frenzy since Hormel changed the labels on the SPAM cans in Hawaii.

PDP (of gnucitizen fame) masterfully stitched together a collection of mixed metaphors, generalizations, reductions to the ridiculous and pejoratives to produce his opus magnum on cloud computing (in)security titled "The Cloud Is Not That Insecure."

Oh.

Since I have spent the better part of my security career building large "cloud-like" services and the products that help, at a minimum, to secure them, I feel at least slightly qualified to dispute many of his points, the bulk of which are really focused on purely technology-driven mechanical analogies and platforms rather than items such as the operational, trust, political, jurisdictional, regulatory, organizational and economical issues that really go toward the "security" (or lack thereof) of "cloud-based" service.

Speaking of which, PDP's definition of the cloud is about as abstract as you can get:

Well, I'm glad we cleared that up.

At any rate, it's a seriously humorous read that would have me taking apart many of his contradictory assumptions and assertions were it not for the fact that I have actual work to do.  So, in the issue of time, I'll offer up his conclusion and you can go back and read the rest:

So, there you have it.  Those of you who "know what you are doing" are otay and thanks to security by obscurity due to a lack of trust, cloud computing is secure.  That's not confusing at all…

This probably won't end well, but…

Sigh.

/Hoff

I had a  briefing with Patrick Kerpan and Craig Heimark of CohesiveFT last week in response to some comments that Craig had left on my blog post regarding PCI compliance in the Cloud, here. 

I was intrigued, albeit skeptically, with how CohesiveFT was positioning the use of VPNs within the cloud and differentiating their solution from the very well established IPSec and SSL VPN capabilities we all know and love.  What's so special about tunnels across the Intertubes?  Been there, done that, bought the T-Shirt, right?

So I asked the questions…

I have to say that unlike many other companies rushing to associate their brand by rubber cementing the word "cloud" and "security" onto their product names and marketing materials, CohesiveFT spent considerable time upfront describing what they do not do so as to effectively establish what they are and what they do.  I'll let one of their slides do the talking:

Okay, so they're not a cloud, but they provide cloud and virtualization services…and VPN-Cubed provides some layer of security along with their products and services…check. But…

Digging a little deeper, I still sought to understand why I couldn't just stand up an IPSec tunnel from my corporate datacenter to one or more cloud providers where my assets and data were hosted.  I asked for two things: (1) A couple of sentences summarizing their elevator pitch for VPN-Cubed and (2) a visual representation of what this might look like.

Here's what I got as an answer to question (1):

Here are two great visuals to address question (2):

 

So the differences between a typical VPN and VPN-Cubed comes down to being able to securely extend your "internal clouds infrastructure" in your datacenters (gasp! I said it) in a highly-available manner to include your externally hosted assets which in turn run on infrastructure you don't own.  You can't stand up an ASA or Neoteris box on a network you can't get to.  The VPN-Cubed Managers are VM's/VA's that run as a complement to your virtual servers/machines hosted by your choice of one or multiple cloud providers.

They become the highly-available, multiprotocol aribters of access via standardized IPSec protocols but do so in a way that addresses the dynamic capabilities of the cloud which includes service governance, load, and "cloudbursting" failover between clouds — in a transparent manner.

Essentially you get secure access to your critical assets utilizing an infrastructure independent solution, extending the VLAN segmentation, isolation and security capabilities your providers may put in place while also controlling your address spaces within the cloudspaces encompassing your VM's "behind" the VPN Managers.

VPN-Cubed is really a prime example of the collision space of dynamic application/service delivery, virtualization, security and cloud capacity governance.  It speaks a lot to re-perimeterization that I've been yapping about for quite some time and hints at Greg Ness' Infrastructure 2.0 meme.

Currently VPN-Cubed is bundled as a package which includes both product and services and supports the majority of market-leading virtualization formats, operating systems and cloud providers such as Amazon EC2, Flexiscale, GoGrid, Mosso, etc.

It's a very slick offering.

/Hoff

Mogull commented yesterday on my post regarding TCG's IF-MAP and remarked that in discussing cloud security and security models, the majority of folks, myself included, were focusing on the network:

Chris’s posting, and most of the ones I’ve seen, are heavily focused
on network security concepts as they relate to the cloud. But if we
look at cloud computing at the macro level, there are additional layers
which are just as critical (in no particular order):

• Network: The usual network security controls.
• Service: Security around the exposed APIs and services.
• User: Authentication- which in the cloud world, needs to
  move to more adaptive authentication, rather than our current static
  username/password model.
• Transaction: Security controls around individual transactions- via transaction authentication, adaptive authorization, and other approaches.
• Data: Information-centric security controls for cloud
  based data. How’s that for buzzword bingo? Okay, this actually includes
  security controls for the back-end data, distributed data, and any
  content exchanged with the user.

I'd say that's a reasonable assertion and a valid set of additional "layers."  There also not especially unique and as such, I think Rich is himself a little disoriented by the fog of the cloud because as you'll read, the same could be said of any networked technology.

The reason we start with the network and usually find ourselves back where we started in this discussion is because the other stuff Rich mentions is just too damned hard, costs too much, is difficult to sell, isn't standardized, is generally platform dependent and is really intrusive.  See this post (Security Will Not End Up In the Network) as an example.

Need proof of how good ideas like this get mangled?  How about Web 2.0 or SOA which is for lack of a better description, exactly what RIch described in his model above; loosely coupled functional components of a modular architecture. 

We haven't even gotten close to having this solved internally on our firewalled enterprise LANs so it's somewhat apparent why it might appear to be missing in conversations regarding "the cloud."  It shouldn't be, but it is. 

It should be noted, however that there is a ton of work, solutions and initiatives that exist and continue to be worked on these topics, it's just not a priority as is evidenced by how people exercise their wallets.

And finally:

Down the road we’ll dig into these in more detail, but any time we
start distributing services and functionality over an open public
network with no inherent security controls, we need to focus on the
design issues and reduce design flaws as early as possible. We can’t
just look at this as a network problem- our authentication,
authorization, information, and service (layer 7) controls are likely
even more important.

I believe we call this thing of which he speaks, "the Internet."  I think we're about 20 years late. 😉

/Hoff

In my previous post titled "Cloud Computing: Invented By Criminals, Secured By ???" I described the need for a new security model, methodology and set of technologies in the virtualized and cloud computing realms built to deal with the dynamic and distributed nature of evolving computing:

This
basically means that we should distribute the sampling, detection and
prevention functions across the entire networked ecosystem, not just to
dedicated security appliances; each of the end nodes should communicate
using a standard signaling and telemetry protocol so that common
threat, vulnerability and effective disposition can be communicated up
and downstream to one another and one or more management facilities.

Greg Ness from Infoblox reminded me in the comments of that post of something I was very excited about when it
became news at InterOp this last April: the Trusted Computing Group's (TCG) extension to the Trusted Network Connect (TNC) architecture called IF-MAP.

IF-MAP is a standardized real-time publish/subscribe/search mechanism which utilizies a client/server, XML-based SOAP protocol to provide information about network security objects and events including their state and activity:

While the TNC was initially designed to support NAC solutions, extending the capabilities to any security product to subscribe to a common telemetry and information exchange/integration protocol is a fantastic idea.

I'm really interested in how many vendors outside of the NAC space are including IF-MAP in their roadmaps. While IF-MAP has potential in convential non-virtualized infrastructure, I see a tremendous need for it in our move to Infrastructure 2.0 with virtualization and Cloud Computing. 

Integrating, for example, IF-MAP with VM-Introspection capabilities (in VMsafe, XenAccess, etc.) would be fantastic as you could tie the control planes of the hypervisors, management infrastructure, and provisioning/governance engines with that of security and compliance in near-time.

You can read more about the TCG's TNC IF-MAP specification here.

/Hoff