Virtualized Infrastructure: It’s All Fun and Games Until Someone Loses An (PC)I…
I just responded to a comment from Iben Rodriguez on one of my virtualization and PCI blog entries from a while back and posted an observation while at the same time managed to make a funny (see the title.)
I wanted to both reflect upon Iben’s comment as well as chuckle a bit.
From what I extracted from his comment, Iben is suggesting that perhaps virtualization should not affect an auditor’s approach or differentiate the audit process from a physical server depending upon the definition of a "server:"
Is an ESX Host a server?
It should be considered similar to the chassis holding a bunch of blade servers.
These have management ports on separate networks, with LDAP authentication over security protocols like ssh and ssl.
And why not treat them as a hybrid device with different network switches, storage controllers, etc?
Vmware has recently removed the word "Server" from after the ESX product name…
It’s not a server, it’s a hypervisor.
It’s not a server, it’s a switch.
By defining what a server is and is not a PCI Audit should be pretty straight forward.
I think this is a messy question and one we ought to continue to address. I need to go and check out my ISACA references to seek guidance on this matter from a, um, "higher" source 😉 I do think that ultimately this is a very subjective issue, to which I responded:
The answers to your questions/suppositions are quite simple:
"It all depends upon the auditor."
Most of the folks I’ve spoken to recently are essentially counting
upon the ignorance of the auditors and the general confusion regarding
terminology and technology to glide by at this point.Server/blade/hypervisor/switch … it’s all fun and games until someone loses a (PC)I… 😉
"As long as I put in place the same host controls I do in a physical
environment and not tell the auditor it’s virtualized, it’s all good
and what they don’t know, won’t hurt me."Sad but true.
I find this practice/observation to be more and more common as the push to virtualize all infrastructure — including externally-facing DMZ’s — starts to become more visible in the compliance and audit spaces.
Whack-a-mole!
/Hoff
The staff in my organization tried this same argument to get out of auditing the ESX "servers". I kept arguing that this was just an install of Linux running an install of something else. Just like my MacBook running Parallels to run Windows XP. They kept arguing otherwise and I kept brining it up. The auditors eventually listened to me. ESX "servers" now have the same audit points as any other server.
I heard a lot of potential "gotchas" in your Black Hat talk related to PCI – especially regarding network segmentation.
One of the big ways to reduce scope of PCI is to segment out all effected assets onto their own VLAN. I daresay a best practice would be to keep this segmentation when said assets are virtualized – no non-PCI assets on that piece of iron.
It seems that the folks architecting these complex virtualized application, server, desktop and network environments still don't have a firm grasp of what would happen if the security of the software/virtualized hardware were to be breached.
I wouldn't be surprised if we see some clarification of PCI to include the various levels of virtualized assets needing to be in compliance and not just the single instance containing in-scope data.
"Most of the folks I've spoken to recently are essentially counting upon the ignorance of the auditors and the general confusion regarding terminology and technology to glide by at this point."
Yeap, I've been in one of those organisations. We had a lot of problems that we couldn't fix due to budget/resource constraints. Auditor ignorance got us past those issues but then bit us in the butt when the auditor's ignorance came through in his assertion that NAT/PAT is a security measure and *EVERY* server should be hidden behind a NAT/PAT device.
For context the only devices outside of NAT/PAT were the web servers which were behind a first tier firewall. Application, database and communications to third parties/etc were all behind other firewalls that also performed NAT/PAT.
From my perspective that topology was inherently better – someone might exploit the web server and access to the first tier and potentially even play piggy in the middle with inbound connections, but they would still have no visibility of the internal network without further effort attacking the rest of the infrastructure.
I for one think the PCI standard is great but is still too vague. The auditors I've seen are merely skilled at reading standards documents, asking questions and ticking boxes. They lacked the technical knowledge to properly audit a server let alone a complex network.
Hoff, ssssshhhh, or they auditors will hear you. We've been doing this for years and they haven't caught on. The first rule of anti-audit club is that you do not talk about anti-audit club!
My own list of fun auditor-befuddling technologies:
SANs
VLANs
Clustering
SAN-to-SAN replication
Virtual tape libraries
Laboratory equipment
RDP over SSL VPN
Too bad we can't get a gag order for items that an auditor could exploit like we can for the subway fare system. =)
Christopher Hoff, over at Rational Security discussed an interesting idea last week…