Rational Survivability

It seems I have fallen victim to a series of misunderstandings these days.

First there was Joanna-Gate and now Simon Crosby, Citrix’s CTO, suggests in a blog entry titled "Chris Hoff & The Mother Of All Misunderstandings" that I’m puffing on the wrong end of my cigars for disagreeing with his position.

I’m a little concerned that Simon’s response to me was issued on what is listed as the "beta" version of Citrix’s official blog.  Perhaps the virtualized version hasn’t made it out of QA yet? 😉

Simon’s response was extremely well crafted to avoid responding to most of my actual points, was contextually oblique at points, and was a fantastic marketing piece for Xen Citrix, but I wish he’d paid more attention to the actual points within my post. 

Further his little quips/comments on his hyperlinks "Who is this guy, anyway?  Think before you type dude, we’re not idiots," etc. didn’t go unnoticed – cute but juvenile)

I am, however, honored that Simon would accord me the high-status of being "…normally fairly clued-in:"

I
reckon that Hoff, who is normally fairly clued-in,  has put the smoking
end of the cigar in his mouth before thinking through this argument.
He’s horribly confused, but as smug as always, so let me clarify what I
said, and what it means.

…but I can assure you that I’ve only ever done that with a cigar once,
and it was for a much better reason than blogging.  If you must know,
it was Kentucky’s finest bourbon.  That is all I’m going to say about
that. 

I’m glad he’s "clarifying" what he said, since I will also.  I seem to have that effect on people.  Must be the accent thing…

The reason for my allergic reaction to Simon’s comments stem from my opinion that it is the responsibility of virtualization platform providers to ensure that their "[virtualized] data center operating system platforms of the future" don’t become the next generation of insecure infrastructure.

Simon sums up his opinion:

In summary an assertion that the virtualization platform vendor has
to fix the sad state of the OS/App world by making it secure is
demanding too much.  It would mean that we have to be experts in every
piece of system software including all of the vulnerabilities of all
OSes and their apps.  In my view the reason the state of security is
poor now is because of the monolithic approaches of traditional OS and
app vendors. 

We will focus manically on our layer, make it
secure, tiny and bulletproof to attack in its own right.  And we will
work closely with experts in security of OSes and Apps to give them an
opportunity to implement guest-level security outside the guest,
through privileged interfaces that themselves are secure.

After 15 years of dealing with this crap, I respectfully suggest that it is not too much to ask and it’s about time we stood up and did.  First  you criticize OS/App. vendors and blame them for the state of security because of their "monolithic approach" and then you go on to propose the exact same thing!

Focusing only on your little patch of grass is short-sighted and it won’t work.  Just like it hasn’t worked in the past.  It’s a disaster waiting to happen, and you’re enabling it. 

I shudder at the potential tunnel vision of virtualization platform providers only focusing on the security of the hypervisor without taking the bigger picture into consideration and expect a piecemeal approach to securing the expanse of the virtualized environment to suffice.

It’s clear you’re making arguments about security from an engineering and code-base perspective that is simply disconnected from the realities of what it means to actually deploy these solutions. 

Virtualization is more than just the hypervisor.  You should know that by now, Simon.  The company that acquired your company knows all about that.  The hypervisor will shortly become a commodity, so in the long term the value brought to bear has to be more than just an ultra-thin layer of code:

…and furthermore, we’re going to deploy many of them:

I wish to make it clear that I hold all virtualization platform vendors to the same level of scrutiny and criticism, not just Citrix. 

I happen to like Xen very much.  I like VMware, also.  I think the latter is more realistic and measured when it comes to addressing the need and approach in recognizing that as a major layer in the infrastructure, there’s more required than to just secure the hypervisor and leave the remaining mess to someone else to solve.

I think Simon’s blog title is apropos, but I think the misunderstanding is his.

It’s important to understand that I’m not suggesting that virtualization platform providers should secure the actual guest operating systems
but they should enable an easier and more effective way of doing so when virtualized.

I mean that the virtualization platform providers should ensure the security of the instantiation of those
guests as "hosted" by the virtualization platform.  In some cases this means leveraging technology present in the virtualization platform to do things that non-virtualized instances cannot. That’s more than just securing the hypervisor.

Securing the hypervisor whilst closing your eyes to the likelihood
that the majority of attacks against it and other guests will come from "guests" within the same system is planting your head in the sand.  That means that there will be a need to ensure that certain behaviors specific to the hosted guests are mitigated to ensure that bad things don’t happen — to the guest or the hypervisor.

Transferring the responsibility to secure the environment to third party security ISV’s in order
to secure the VM’s
and preventing them from compromising one another or the hypervisor is
difficult for me to comprehend, especially when they are playing catch up of what virtualization means within the context of security.

Fundamentally, attempting to mate static and topology-dependent policies to incredibly dynamic and transitive technology delivered by virtualization will simply fail.  Third party security ISV’s will simply require a complete re-tool to even get close to delivering this and will need to provide intimate hooks to allow for this policy/guest affinity to occur in the first place.

I consider the virtualization infrastructure layer as that of an operating system and as such, I would expect that the underpinning mechanicals are as sound and secure as possible while also ensuring that anything running on top of it is as secure as possible, also.

Let’s take Microsoft (with or without Hyper-V) as an example:

Microsoft is fundamentally concerned now with making the OS as
resilient and secure as possible whilst preventing the applications and
interaction with elements riding on top of the OS from doing bad things
to the system as a whole; this isn’t just to protect the OS, but the
assets on it. 

This is really what I’m getting at.  Yes, Microsoft is an OS provider.  Shortly, that OS provider will integrate virtualization directly into the operating system.  That means more, not less, direct integration and security embedded as a function of the virtualization platform.  Citrix, VMware, etc. are all just operating system vendors of a different shape and size.

It’s unclear to me, Simon, whether your arguments are meant to justify a business model, a lack of planning, a crafty plan to perpetuate the security hamster wheel of pain, or all of the above.  It’s clear to me, however, that you’ve not felt the pain of actually having to use the products you suggest should be deployed in order to secure this mess.

I promised myself I wouldn’t turn this into one of those cut/paste blog pong entries, but the following really confused me:

But we are not in the business of specifically securing guests or their
applications, other than through offering a secure virtualization
platform.  Even VMware with VMsafe simply exposes APIs to third party
security vendors, so that customers can choose their preferred security
partner to secure guests.  I think that the VMware Determina
acquisition was very smart, and that hints to me that VMware sees
itself having a greater role in the security of guest OSes, since it
could choose to be in the vulnerability checking business without 3rd
party security vendors, but thus far they are working very openly with
the ecosystem.

So which is it?  You’ve established that Citrix is not in the business of securing guests or applications (you must mean Xen specifically, because somebody at Citrix spent quite a bit of money on this stuff with their other acquisitions) and that you believe it to be a lousy idea, but you think that VMware’s approach through their Determina acquisition as well as the capabilities of VMsafe is "…very smart?"

Simon, you’re the CTO and I’m the security wonk.  If we didn’t disagree, I’d be alarmed.  However, I think you might want to rethink your approach to how you market the security of your platform.

I’ve got a cigar for you anytime you want one.  I’ll let you light it.

/Hoff