Rational Survivability

A few weeks ago I saw some RT’s/@’s on Twitter referencing John Flowers and that name brought back some memories.

Today I sent a tweet to John asking him if I remembered correctly that he was at SANS in New Orleans in 1999 when he was still at Hiverworld.

He responded back confirming he was, indeed, at SANS ’99.  I remarked that this was where I first met many of today’s big names in security: Ed Skoudis, Ron Gula, Marty Roesch, Stephen Northcutt, Chris Klaus, JD Glaser, Greg Hoglund, and Bruce Schneier.

John responded back:

I couldn’t agree more.  That was an absolutely amazing time. I was on my second security startup (NodeWarrior Networks,) times were booming and this generation of the security industry as we know it was being given birth to.

I remember many awesome things from that week:

• Sitting in “Intrusion Detection Shadow Style” with Stephen Northcut and Judy Novak for something like 8 hours going cross-eyed reading tcpdump packet traces and getting every question Stephen asked wrong. Well, some of them, anyway 😉
• Asking Ron Gula’s wife something about Dragon and her looking back at me like I was a total n00b
• Asking Ron Gula the same question and having him confirm that I was, in fact, a complete tool
• Staying up all night drinking, writing code in Perl and doing dangerous things on other people’s networks
• Participating in my first CTF
• Almost getting arrested for B&E as I tried to rig the CTF contest by attempting to steal/clone/pwn/replace the HDD in the target machine. The funniest part of that was almost pulling it off (stealing the removable drive) but electrocuting myself in the process — which is what alerted my presence to the security guard.
• Interrupting Lance Spitzner’s talk by stringing a poster behind him that said “www.lancespitznerismyhero.com” (a domain I registered during the event.)
• Watching Bruce Schneier scream at the book store guy because they, incredulously, did not stock “Practical Cryptography“
• Sitting down with Ed Skoudis (who was with SAIC at the time, I believe,) looking at one another and wondering just what the hell we were going to do with our careers in security
• Spending $14,000 (I shit you not, it was the Internet BOOM time, remember) by hitting 6 of the best restaurants in New Orleans with a party of hax0rs and working the charge department at American Express into a frenzy (not to mention actually using the line from Pretty Woman: “we’re going to spend obscene amounts of money here” in order to get in…)
• Burning the roof of my mouth by not heeding the warnings of the waitress at Cafe Dumonde, biting into a beignet which cauterized my mouth as I simultaneously tried to extinguish the pain with scalding hot Chicory coffee.

I came back from that week knowing with every molecule in my body that even though I’d been “doing” security for 5 years already, it was exactly what I wanted to for the rest of my life.

I have Stephen Northcut to thank for that.  I haven’t been to a SANS since 1999 (don’t ask me why) but I am so excited about going back in August in DC (SANS What Works In Virtualization and Cloud Computing Summit) and giving a keynote at the event.

It’s been a long time.  Too long.

/Hoff

My good friend Amrit Williams (@amrittsering) from BigFix (congrats on the IBM acquisition!) has an awesome Podcast titled “Beyond the Perimeter.”

He was nice enough to invite me to record episode 93 titled “Is Trust the Real Barrier To Cloud Computing?” (ultimately points you to an iTunes subscription.)

We spoke for almost an hour on all sorts of great discussion points related to Cloud Computing, specifically focusing on Trust (which I define in context as Security, Compliance, Control, Reliability and Privacy.)

We also spoke about the Cloud Security Alliance, CloudAudit and the HacKid conference — three things I am very passionate about.

Thanks Amrit, great conversation as usual.

/Hoff

Related articles by Zemanta

• IBM secures BigFix for network visibility and compliance (deals.venturebeat.com)
• CLOUDINOMICON: Idempotent Infrastructure, Survivable Systems & Bringing Sexy Back to Information Centricity (rationalsurvivability.com)
• Security: In the Cloud, For the Cloud & By the Cloud… (rationalsurvivability.com)
• IBM to Acquire BigFix – Hallelujah! Can I Get a Witness?! (techbuddha.wordpress.com)
• Introducing The HacKid Conference – Hacking, Networking, Security, Self-Defense, Gaming & Technology for Kids & Their Parents (rationalsurvivability.com)

I don’t have time to write a big blog post and quite frankly, I don’t need to. Not on this topic.

I do, however, feel that it’s important to bring back into consciousness how very important open source security solutions are to us — at least those of us who actually expect to make an impact in our organizations and work toward making a dent in our security problem pile.

Why do open source solutions matter so much in our approach to dealing with securing the things that matter most to us?

It comes down to things we already know but are often paralyzed to do anything about:

1. The threat curve and innovation of attacker outpaces that of the defender by orders of magnitudes (duh)
2. Disruptive technology and innovation dramatically impacts the operational, threat and risk modeling we have to deal with (duh duh)
3. The security industry is not in the business of solving security problems that don’t have a profit motive/margin attached to it (ugh)

We can’t do much about #1 and #2 except be early adopters, by agile/dynamic and plan for change. I’ve written about this many times and built and entire series of talks presentations (Security and Disruptive Innovation) that Rich Mogull and I have taken to updating over the last few years.

We can do something about #3 and we can do it by continuing to invest in the development, deployment, support, and perhaps even the eventual commercialization of open source security solutions.

To be clear, it’s not that commercialization is required for success, but often it just indicates it’s become mainstream and valued and money *can* be made.)

When you look at the motivation most open source project creators bring a solution to market, it’s because the solution generally is not commercially available, it solves an immediate need and it’s contributed to by a community. These are all fantastic reasons to use, support, extend and contribute back to the open source movement — even if you don’t code, you can help by improving the roadmaps of these projects by making suggestions and promoting their use.

Open source security solutions deliver and they deliver quickly because the roadmaps and feature integration occur in an agile, meritocratic and vetted manner than often times lacks polish but delivers immediate value — especially given their cost.

We’re stuck in a loop (or a Hamster Sine Wave of Pain) because the problems we really need to solve are not developed by the companies that are in the best position to develop them in a timely manner. Why? Because when these emerging solutions are evaluated, they live or die by one thing: TAM (total addressable market.)

If there’s no big $$$ attached and someone can’t make the case within an organization that this is a strategic (read: revenue generating) big bet, the big companies wait for a small innovative startup to develop technology (or an open source tool,) see if it lives long enough for the market demand to drive revenues and then buy them…or sometimes develop a competitive solution.

Classical crossing the chasm/Moore stuff.

The problem here is that this cycle is broken horribly and we see perfectly awesome solutions die on the vine. Sometimes they come back to life years later cyclically when the pain gets big enough (and there’s money to be made) or the “market” of products and companies consolidate, commoditize and ultimately becomes a feature.

I’ve got hundreds of examples I can give of this phenomenon — and I bet you do, too.

That’s not to say we don’t have open-source-derived success stories (Snort, Metasploit, ClamAV, Nessus, OSSec, etc.) but we just don’t have enough of them. Further, there are disruptions such as virtualization and cloud computing that fundamentally change the game that we can harness in conjunction with open source solutions that can accelerate the delivery and velocity of solutions because of how impacting the platform shift can be.

I’ve also got dozens of awesome ideas that could/would fundamentally solve many attendant issues we have in security — but the timing, economics, culture, politics and readiness/appetite for adoption aren’t there commercially…but they can be via open source.

I’m going to start a series which identifies and highlights solutions that are either available as kernel-nugget technology or past-life approaches that I think can and should be taken on as open source projects that could fundamentally help our cause as a community.

Maybe someone can code/create open source solutions out of them that can help us all.  We should encourage this behavior.

We need it more than ever now.

/Hoff

Related articles by Zemanta

• The Security Hamster Sine Wave Of Pain: Public Cloud & The Return To Host-Based Protection… (rationalsurvivability.com)
• New Commercially-Supported Open-Source Network Sensors: nPulse Technologies and Partners Deliver the High-Performance Dragonfly FlowMeter (prnewswire.com)
• Intelligence Services Using Open Source (brighthub.com)
• Security: In the Cloud, For the Cloud & By the Cloud… (rationalsurvivability.com)
• The Hypervisor Platform Shuffle: Pushing The Networking & Security Envelope (rationalsurvivability.com)

I’m hurrying to polish up the next in my series of virtualization and cloud computing security presentations which I’m going to give at this year’s Black Hat conference in Las Vegas on July 29th.  I’m speaking from 10-11am on day two up next to folks like Jeremiah Grossman, Moxie Marlinspike, Ivan Ristic, Haroon Meer…quite the “power hour” as someone said on the Twitter.

At any rate, I started the series a couple of years ago with the following progression:

1. The Four Horsemen of the Virtualization Security Apocalypse
2. The Frogs Who Desired a King: A Virtualization & Cloud Computing Fable Set To Interpretative Dance
3. Cloudifornication: Indiscriminate Information Intercourse Involving Internet Infrastructure

I proudly present numero quatro:

CLOUDINOMICON: Idempotent Infrastructure, Survivable Systems & Bringing Sexy Back to Information Centricity

Mass-market, low-cost, commodity infrastructure-as-a-Service Cloud Computing providers abstract away compute, network and storage and deliver hyper-scaleable capabilities.

This “abstraction distraction” has brought us to the point where the sanctity and security of the applications and information transiting them are dependent upon security models and expertise rooted in survivable distributed systems, at layers where many security professionals have no visibility.

The fundamental re-architecture of the infostructure, metastructure and infrastructure constructs in this new world forces us back to the design elements of building survivable systems focusing on information centricity — protecting the stuff that matters most in the first place.

The problem is that we’re unprepared for what this means and most practitioners and vendors focused on the walled garden, perimeterized models of typical DMZ architecture are at a loss as to how to apply security in a disintermediated and distributed sets of automated, loosely-coupled resources.

We’re going to cover the most salient points relating to how IaaS Cloud architecture shifts how, where and who architects, deploys and manages security in this “new world order” and what your options are in making sustainable security design decisions.

It’s progressing nicely.  Hope to see you there (and at Defcon)

This is a revisitation of a blog I wrote last year: Incomplete Thought: Cloud Security IS Host-Based…At The Moment

I use my ‘Security Hamster Sine Wave of Pain” to illustrate the cyclical nature of security investment and deployment models over time and how disruptive innovation and technology impacts the flip-flop across the horizon of choice.

To wit: most mass-market Public Cloud providers such as Amazon Web Services rely on highly-abstracted and limited exposure of networking capabilities.  This means that most traditional network-based security solutions are impractical or non-deployable in these environments.

Network-based virtual appliances which expect generally to be deployed in-line with the assets they protect are at a disadvantage given their topological dependency.

So what we see are security solution providers simply re-marketing their network-based solutions as host-based solutions instead…or confusing things with Barney announcements.

Take a press release today from SourceFire:

Snort and Sourcefire Vulnerability Research Team(TM) (VRT) rules are now available through the Amazon Elastic Compute Cloud (Amazon EC2) in the form of an Amazon Machine Image (AMI), enabling customers to proactively monitor network activity for malicious behavior and provide automated responses.

Leveraging Snort installed on the AMI, customers of Amazon Web Services can further secure their most critical cloud-based applications with Sourcefire’s leading protection. Snort and Sourcefire(R) VRT rules are also listed in the Amazon Web Services Solution Partner Directory, so that users can easily ensure that their AMI includes the latest updates.

As far as I can tell, this means you can install a ‘virtual appliance’ of Snort/Sourcefire as a standalone AMI, but there’s no real description on how one might actually implement it in an environment that isn’t topologically-friendly to this sort of network-based implementation constraint.*

Since you can’t easily “steer traffic” through an IPS in the model of AWS, can’t leverage promiscuous mode or taps, what does this packaging implementation actually mean?  Also, if  one has a few hundred AMI’s which contain applications spread out across multiple availability zones/regions, how does a solution like this scale (from both a performance or management perspective?)

I’ve spoken/written about this many times:

Where Are the Network Virtual Appliances? Hobbled By the Virtual Network, That’s Where… and

Dear Public Cloud Providers: Please Make Your Networking Capabilities Suck Less. Kthxbye

Ultimately, expect that Public Cloud will force the return to host-based HIDS/HIPS deployments — the return to agent-based security models.  This poses just as many operational challenges as those I allude to above.  We *must* have better ways of tying together network and host-based security solutions in these Public Cloud environments that make sense from an operational, cost, and security perspective.

/Hoff

Related articles by Zemanta

• Sourcefire Expands Real-Time Application Awareness, Extending Leadership of Intelligent Cybersecurity Solutions (eon.businesswire.com)
• Securing the public cloud (news.cnet.com)
• Amazon Security Groups for EC2 (kinlane.com)
• You Can’t Secure The Cloud… (rationalsurvivability.com)
• Security: In the Cloud, For the Cloud & By the Cloud… (rationalsurvivability.com)
• The Hypervisor Platform Shuffle: Pushing The Networking & Security Envelope (rationalsurvivability.com)
• Incomplete Thought: The DevOps Disconnect (rationalsurvivability.com)
• Dear SaaS Vendors: If Cloud Is The Way Forward & Companies Shouldn’t Spend $ On Privately-Operated Infrastructure, When Are You Moving Yours To Amazon Web Services? (rationalsurvivability.com)
• The Four Horsemen Of the Virtualization (and Cloud) Security Apocalypse… (rationalsurvivability.com)

• Incomplete Thought: The DevOps Disconnect (rationalsurvivability.com)
• Dear SaaS Vendors: If Cloud Is The Way Forward & Companies Shouldn’t Spend $ On Privately-Operated Infrastructure, When Are You Moving Yours To Amazon Web Services? (rationalsurvivability.com)
• The Four Horsemen Of the Virtualization (and Cloud) Security Apocalypse… (rationalsurvivability.com)

* I “spoke” with Marty Roesch on the Twitter and he filled in the gaps associated with how this version of Snort works – there’s a host-based packet capture element with a “network” redirect to a stand-alone AMI:

@Beaker AWS->Snort implementation is IDS-only at the moment, uses software packet tap off customer app instance, not topology-dependent

and…

they install our soft-tap on their AMI and send the traffic to our AMI for inspection/detection/reporting.

It will be interesting to see how performance nets out using this redirect model.

Every day I get asked to discuss how Cloud Computing impacts security architecture and what enterprise security teams should do when considering “Cloud.”

These discussions generally lend themselves to a bifurcated set of perspectives depending upon whether we’re discussing Public or Private Cloud Computing.

This is unfortunate.

From a security perspective, focusing the discussion primarily on the deployment model instead of thinking holistically about the “how, why, where, and who” of Cloud, often means that we’re tethered to outdated methodologies because it’s where our comfort zones are.

When we’re discussing Public Cloud, the security teams are starting to understand that the choice of compensating controls and how they deploy and manage them require operational, economic and architectural changes.  They are also coming to terms with the changes to application architectures as it relates to distributed computing and SOA-like implementation.  It’s uncomfortable and it’s a slow-slog forward (for lots of good reasons,) but good questions are asked when considering the security, privacy and compliance impacts of Public Cloud and what can/should be done about them and how things need to change.

When discussing Private Cloud, however, even when a “clean slate design” is proposed, the same teams tend to try to fall back to what they know and preserve the classical n-tier application architecture separated by physical or virtual compensating controls — the classical split-subnet DMZ or perimeterized model of “inside” vs “outside.” They can do this given the direct operational control exposed by highly-virtualized infrastructure.  Sometimes they’re also forced into this given compliance and audit requirements. The issue here is that this discussion centers around molding cloud into the shape of the existing enterprise models and design patterns.

This is an issue; trying to simultaneously secure these diametrically-opposed architectural implementations yields cost inefficiencies, security disparity, policy violations, introduces operational risk and generally means that  the ball doesn’t get moved forward in protecting the things that matter most.

Public Cloud Computing is a good thing for the security machine; it forces us to (again) come face-to-face with the ugliness of the problems of securing the things that matter most — our information. Private Cloud Computing — when improperly viewed from the perspective of simply preserving the status quo — can often cause stagnation and introduce roadblocks.  We’ve got to move beyond this.

Public Cloud speaks to the needs (and delivers on) agility, flexibility, mobility and efficiency. These are things that traditional enterprise security are often not well aligned with.  Trying to fit “Cloud” into neat and tidy DMZ “boxes” doesn’t work.  Cloud requires revisiting our choices for security. We should take advantage of it, not try and squash it.

/Hoff

Related articles by Zemanta

• The Four Horsemen Of the Virtualization (and Cloud) Security Apocalypse… (rationalsurvivability.com)
• Virtualization & Cloud Don’t Offer An *Information* Security Renaissance… (rationalsurvivability.com)
• The Hypervisor Platform Shuffle: Pushing The Networking & Security Envelope (rationalsurvivability.com)
• Security: In the Cloud, For the Cloud & By the Cloud… (rationalsurvivability.com)
• Incomplete Thought: “The Cloud in the Enterprise: Big Switch or Little Niche?” (rationalsurvivability.com)
• Incomplete Thought: The DevOps Disconnect (rationalsurvivability.com)
• You Can’t Secure The Cloud… (rationalsurvivability.com)
• Friday Cloud Poetry: “On the Bullshit That is False Cloud” (rationalsurvivability.com)

I was inspired to write this given the latest round of marketing being tended to by Amazon Web Services in their renewed campaign to convince Enterprises CIO’s that their server-hugging IT teams are luddites and interested in nothing more than boat anchoring the success of their companies to some desperate need to buy legacy kit.

The “public-all-or-nothing” approach being hammered by AWS simply ignores the reality that the very customers they hope to woo face on a daily basis and instead seeks to rub their noses in the idealism that we should all simply trust that public, mass-market, one-size-fits-all Clouds are ready for critical, compliance-shackled, and heavily regulated applications today.

Werner, this one’s for you…

—

If the language of Cloud
Were something to parse
You’d find that some constructs
are rooted in farce

Dogmatic pursuits
of cloud terms that are pure
Yields terms of endearment
some profound, some demure

“Private Cloud is a false cloud!”
Werner peddles his schpiel
That’s to be expected
given where he gets his next meal

Cloud’s not about exclusion
There’s no right or no wrong
It’s not a crusade
OR a kumbayah song

Public or private
Inside or out,
Serving the business
is what cloud’s all about

If you make this religious,
Telling people to choose
All you’ll accomplish
is how fast you’ll lose

Say what you’re good at
What value you add
Not that differing approaches
Are inherently bad

Be evangelistic for sure
Promote Public Cloud’s virtue
And don’t be afraid
Private Cloud’s not out to hurt you

The reality is
No matter how you try and avoid it
Private cloud will add value,
No, you haven’t destroyed it

The value prop’s clear
On where each model works best
The market will sort out
where the laurels will rest

Public Cloud is fantastic
Private Cloudies agree
Hybrid models will win
Just wait and you’ll see

Related articles by Zemanta

• Survey: IT wants public and private cloud services (infoworld.com)
• Amazon experts launch private-cloud start-up (news.cnet.com)
• Structure 2010: Is There a Hybrid Cloud in Your Future? (gigaom.com)
• Incomplete Thought: “The Cloud in the Enterprise: Big Switch or Little Niche?” (rationalsurvivability.com)

Hot on the tail of the announcement that VMware and Novell are entering into a deeper “strategic partnership” in order to deliver and support SUSE Linux Enterprise Server (SLES) for VMware vSphere environments, was an interesting blog post from Stu (@vinternals) titled “Enter the Appliance.”

Now, before we get to Stu’s post, let’s look at the language from the press release (the emphasis is mine):

VMware and Novell today announced an expansion to their strategic partnership with an original equipment manufacturer (OEM) agreement through which VMware will distribute and support the SUSE® Linux Enterprise Server operating system. Under the agreement, VMware also intends to standardize its virtual appliance-based product offerings on SUSE Linux Enterprise Server.

…

Customers who want to deploy SUSE Linux Enterprise Server for VMware® in VMware vSphere™ virtual machines will be entitled to receive a subscription to SUSE Linux Enterprise Server that includes patches and updates as part of their newly purchased qualifying VMware vSphere license and Support and Subscription. Under this agreement, VMware and its extensive network of solution provider partners will also be able to offer customers the option to purchase technical support for SUSE Linux Enterprise Server delivered directly by VMware for a seamless support experience. This expanded relationship between VMware and Novell benefits customers by reducing the cost and complexity of deploying and maintaining an enterprise operating system with VMware solutions.

As a result of this expanded collaboration, both companies intend to provide customers the ability to port their SUSE Linux-based workloads across clouds.  Such portability will deliver choice and flexibility for VMware vSphere customers and is a significant step forward in delivering the benefits of seamless cloud computing.

Several VMware products are already distributed and deployed as virtual appliances. A virtual appliance is a pre-configured virtual machine that packages an operating system and application into a self-contained unit that is easy to deploy, manage and maintain. Standardizing virtual appliance-based VMware products on SUSE Linux Enterprise Server for VMware® will further simplify the deployment and ongoing management of these solutions, shortening the path to ROI.

What I read here is that VMware virtual appliances — those VMware products packaged as virtual appliances distributed by VMware — will utilized SLES as the underlying operating system of choice. I don’t see language or the inference that other virtual appliance ISVs will be required to do so

To that point, Stu’s blog post said:

VMware will be adopting SUSE Linux Enterprise Server, SLES, as the single platform for their virtual appliances.

I’ve ranted in the past about the problem with virtual appliances. Everything from the lack of a standard Linux platform even within a single vendor (let alone amongst multiple vendors), to the additional overhead such a model of software distribution would place upon software vendors, to the security needs of the Enterprise around patch response times etc. And today, every single one of those arguments has been nullified in one fell swoop. Hallelujah, someone was listening after all!

So far, so good. Seems pretty much in-line with what VMware said.

Here’s the interesting assertion Stu makes that inspired my commentary:

If you’re a software vendor looking to adopt the virtual appliance model to distribute your wares then I have some advice for you – if you’re not using SLES for the base of your appliance, start doing so. Now. This partnership will mean doors that were previously closed to virtual appliances will now be opened, but not to any old virtual appliance – it will need to be built on an Enterprise grade distro. And SLES is most certainly that.

Chris Wolf, Stu and I had a bit of banter on Twitter regarding this announcement wherein I suggested there’s a blurring of the lines and a conflation of messaging as well as a very unique perspective that’s not being discussed.

Specifically, I don’t see where it was implied that ISV’s would be forced to adopt SLES as their OS of choice for virtual appliances.  I’m not suggesting it’s not compelling to do so for the support and distribution reasons stated above, but I suggest that the notion that “…doors that were previously closed to virtual appliances” from the perspective of support and uniformity of disto will also have and equal and opposite effect caused by a longer development lifecycle for many vendors.

Especially networking and security ISVs looking to move their products into a virtual appliance offering.

I summed up many of the issues associated with virtual security and networking appliances in my Four Horsemen of the Virtualization Security Apocalypse presentation, and given how the definition and capabilities of “the network” are (d)evolving (depending upon how you view abstraction) you might also find Where Are the Network Virtual Appliances? Hobbled By the Virtual Network, That’s Where… an interesting read:

What does this mean?  It means that ultimately to ensure their own survival, virtualization and cloud providers will depend less upon virtual appliances and add more of the basic connectivity AND security capabilities into the VMMs themselves as its the only way to guarantee performance, scalability, resilience and satisfy the security requirements of customers. There will be new generations of protocols, APIs and control planes that will emerge to provide for this capability, but this will drive the same old integration battles we’re supposed to be absolved from with virtualization and Cloud.

Tell me that’s not what’s happening *right* now.

Unlike most user-facing or service-delivery applications that are not topology sensitive (that is, they simply expect to be able to speak to “the network” without knowing anything about it,) network and security ISVs do very interesting things with drivers and kernel-space code in order to deal with topology, where they sit in the stack, and how they improve performance and stability that are extremely dependent upon direct access to hardware or at the very least, customer drivers or extended/hacked kernels.

One of the reasons you see a slow trickle of network and security virtual appliances is because of these bespoke OS builds and what virtualization has done to how these services are delivered, scaled and deal with resilience.  We’ve already seen the challenge of ISVs having to re-write code to fit the VMsafe fast/slow-path driver model.

You can imagine the consternation involved if what Stu alluded to is actually required — that you must build your virtual appliances on a specific OS.  It’s going to slow down innovation and delivery of solutions if the ISV does not (for any number of valid reasons) use SLES.  This is also one of the downsides of a JEOS approach.

Stu’s warnings about compliance to SLES development notwithstanding, this puts ISVs in a delicate position — one they’ve faced before but is now exacerbated by virtualization and Cloud.  Security vendors generally minimize and harden OS stacks to fit their “application” and then tune the environment accordingly.  We’re already introducing new monocultures and uniformity in attack surfaces with hypervisors.  Are we going to do the same with the operating systems that power the virtual appliances/virtual machines that run atop them — especially those designed to protect these very systems?

Diversity is a good thing — at least when it comes to your networking and security infrastructure.  While I happen to work for a networking vendor, we all recognize that uniformity brings huge benefits as well as the potential for nasty concerns.  If you want an example, check out how a simple software error affected tens of millions of users of WordPress (WordPress and the dark side of multitenancy.) While we’re talking about a different layer in the stack, the issue is the same.

I totally grok the standardization argument for the cost control, support and manageability reasons Stu stated but I am also fearful of the extreme levels of lock-in and monoculture this approach can take.

/Hoff

Related articles by Zemanta

• VMware, Novell partner on Linux, virtualization (infoworld.com)
• Novell: The Microsoft-Taxed Fog Computing Solution (techrights.org)
• Where Are the Network Virtual Appliances? Hobbled By the Virtual Network, That’s Where… (rationalsurvivability.com)
• Virtualization & Cloud Don’t Offer An *Information* Security Renaissance… (rationalsurvivability.com)
• The Four Horsemen Of the Virtualization (and Cloud) Security Apocalypse… (rationalsurvivability.com)
• Security: In the Cloud, For the Cloud & By the Cloud… (rationalsurvivability.com)
• Incomplete Thought: The Other Side Of Cloud – Where The (Wild) Infrastructure Things Are… (rationalsurvivability.com)
• Virtual Networking/Nexus 1000v Virtual Switch Blogger Roundtable/WebEx Logistics – March 2nd. (rationalsurvivability.com)
• Patching the (Hypervisor) Platform: How Do You Manage Risk? (rationalsurvivability.com)
• The Hypervisor Platform Shuffle: Pushing The Networking & Security Envelope (rationalsurvivability.com)

DevOps — what it means and how it applies — is a fascinating topic that inspires all sorts of interesting reactions from people, polarized by their interpretation of what this term really means.

At CloudCamp Denver, adjacent to Gluecon, Aaron Peterson of OpsCode gave a lightning talk titled: “Operations as Code.”  I’ve seen this presentation on-line before, but listened intently as Aaron presented.  You can see John Willis’ version on Slideshare here.  Adrian Cole (@adrianfcole) of jClouds fame (and now Opscode) and I had an awesome hour-long discussion afterwards that was the genesis for this post.

“Operations as Code” (I’ve seen it described also as “Infrastructure as Code”) is really a fantastically sexy and intriguing phrase.  When boiled down, what I extract is that the DevOps “movement” is less about developers becoming operators, but rather the notion that developers can be part of the process whereby they help enable operations/operators to repeatably and with discipline, automate processes that are otherwise manual and prone to error.

[Ed: great feedback from Andrew Shafer: “DevOps isn’t so much about developers helping operations, it’s about operational concerns becoming more and more programmable, and operators becoming more and more comfortable and capable with that.  Further, John Allspaw (@allspaw) added some great commentary below – talking about DevOps really being about tools + culture + communication. Adam Jacobs from Opscode *really* banged out a great set of observations in the comments also. All good perspective.]

Automate, automate, automate.

While I find the message of DevOps totally agreeable, it’s the messaging that causes me concern, not because of the groups it includes, but those that it leaves out.  I find that the most provocative elements of the DevOps “manifesto” (sorry) are almost religious in nature.  That’s to be expected as most great ideas are.

In many presentations promoting DevOps, developers are shown to have evolved in practice and methodology, but operators (of all kinds) are described as being stuck in the dark ages. DevOps evangelists paint a picture that compares and contrasts the Agile-based, reusable componentized, source-controlled, team-based and integrated approach of “evolved” development environments with that of loosely-scripted, poorly-automated, inefficient, individually-contributed, undisciplined, non-source-controlled operations.

You can see how this might be just a tad off-putting to some people.

In Aaron’s presentation, the most interesting concept to me is the definition of “infrastructure.” Take the example to the right, wherein various “infrastructure” roles are described.  What should be evident is that to many — especially those in enterprise (virtualized or otherwise) or non-Cloud environments — is that these software-only components represent only a fraction of what makes up “infrastructure.”

The loadbalancer role, as an example makes total sense if you’re using HAproxy or Zeus ZXTM. What happens if it’s an F5 or Cisco appliance?

What about the routers, switches, firewalls, IDS/IPS, WAFs, SSL engines, storage, XML parsers, etc. that make up the underpinning of the typical datacenter?  The majority of these elements — as most of them exist today — do not present consistent interfaces for automation/integration. Most of them utilize proprietary/closed API’s for management that makes automation cumbersome if not impossible across a large environment.

Many will react to that statement by suggesting that this is why Cloud Computing is the great equalizer — that by abstracting the “complexity” of these components into a more “simplified” set of software resources versus hardware, it solves this problem and without the hardware-centric focus of infrastructure and the operations mess that revolves around it today, we’re free to focus on “building the business versus running the business.”

I’d agree.  The problem is that these are two different worlds.  The mass-market IaaS/PaaS providers who provide abstracted representations of infrastructure are still the corner-cases when compared to the majority of service providers who are entering the Cloud space specifically focused on serving the enterprise, and the enterprise — even those that are heavily virtualized — still very dependent upon hardware.

This is where the DevOps messaging miss comes — at least as it’s described today. DevOps is really targeted (today) toward the software-homogeneity of public, mass-market Cloud environments (such as Amazon Web Services) where infrastructure can be defined as abstract component, software-only roles, not the complex mish-mash of hardware-focused IT of the enterprise as it currently stands. This may be plainly obvious to some, but the messaging of DevOps is obscuring the message which is unfortunate.

DevOps is promoted today as a target operational end-state without explicitly defining that the requirements for success really do depend upon the level of abstraction in the environment; it’s really focused on public Cloud Computing.  In and of itself, that’s not a bad thing at all, but it’s a “marketing” miss when it comes to engaging with a huge audience who wants and needs to get the DevOps religion.

You can preach to the choir all day long, but that’s not going to move the needle.

My biggest gripe with the DevOps messaging is with the name itself. If you expect to truly automate “infrastructure as code,” we really should call it NetSecDevOps. Leaving the network and security teams — and the infrastructure they represent — out of the loop until they are either subsumed by software (won’t happen) or get religion (probable but a long-haul exercise) is counter-productive.

Take security, for example. By design, 95% of security technology/solutions are — by design — not easily automatable or are built to require human interaction given their mission and lack of intelligence/correlation with other tools.  How do you automate around that?  It’s really here that the statement I’ve made that “security doesn’t scale” is apropos. Believe me, I’m not making excuses for the security industry, nor am I suggesting this is how it ought to be, but it is how it currently exists.

Of course we’re seeing the next generation of datacenters in the enterprise become more abstract. With virtualization and cloud-like capabilities being delivered with automated provisioning, orchestration and governance by design for BOTH hardware and software and the vision of private/public cloud integration baked into enterprise architecture, we’re actually on a path where DevOps — at its core — makes total sense.

I only wish that (NetSec)DevOps evangelists — and companies such as Opscode — would  address this divide up-front and start to reach out to the enterprise world to help make DevOps a goal that these teams aspire to rather than something to rub their noses in.  Further, we need a way for the community to contribute things like Chef recipes that allow for flow-through role definition support for hardware-based solutions that do have exposed management interfaces (Ed: Adrian referred to these in a tweet as ‘device’ recipes)

/Hoff

Related articles by Zemanta

• 3 Companies That Tackle Complexity in the Cloud (readwriteweb.com)
• Are You Ready for the New Network? (devcentral.f5.com)
• Understanding cloud and devops–part 1 (news.cnet.com)
• The New DevOps Designers: Cloud and The Big Rethink (slideshare.net)
• Incomplete Thought: The Other Side Of Cloud – Where The (Wild) Infrastructure Things Are… (rationalsurvivability.com)
• Devops (slideshare.net)
• Patching the (Hypervisor) Platform: How Do You Manage Risk? (rationalsurvivability.com)
• Incomplete Thought: “The Cloud in the Enterprise: Big Switch or Little Niche?” (rationalsurvivability.com)
• Security: In the Cloud, For the Cloud & By the Cloud… (rationalsurvivability.com)
• Virtualization & Cloud Don’t Offer An *Information* Security Renaissance… (rationalsurvivability.com)
• Rise of devops (slideshare.net)
• Patching the (Hypervisor) Platform: How Do You Manage Risk? (rationalsurvivability.com)
• Incomplete Thought: “The Cloud in the Enterprise: Big Switch or Little Niche?” (rationalsurvivability.com)
• Security: In the Cloud, For the Cloud & By the Cloud… (rationalsurvivability.com)
• Virtualization & Cloud Don’t Offer An *Information* Security Renaissance… (rationalsurvivability.com)
• Rise of devops (slideshare.net)

Just to point out a fact many/most of you may not be aware of, but Amazon Web Services hired (transferred (?) since he was an AWS insider) Stephen Schmidt as their CISO earlier this year.  He has a team that goes along with him, also.

That’s a very, very good thing. I, for one, am very glad to see it. Combine that with folks like Steve Riley and I’m enthusiastic that AWS will make some big leaps when it comes to visibility, transparency and interaction with the security community.

See. Christmas wishes can come true! Thanks, Santa! 😉

You can find more about Mr. Schmidt by checking out his LinkedIn profile.

/Hoff