Rational Survivability

The talk I was scheduled to give at Blackhat in Vegas had that title.  Due to a timing issue, I couldn’t make Vegas.

The summary of CI^6 goes something like this:

What was in is now out.

This metaphor holds true not only as an accurate analysis of what happens to our data with the adoption trends of disruptive technology and innovation in the enterprise, but also parallels the amazing velocity of how our datacenters are being re-perimiterized and quite literally turned inside out thanks to Cloud computing and virtualization.

One of the really interesting things happening with the massive convergence of virtualization and cloud computing is its effect on security models, the corresponding compensating controls and the information they are designed to protect.

Where and how our data is created, processed, accessed, stored, backed up and destroyed in what is sure to become massively overlaid cloud-based services — and by whom and using whose infrastructure — yields significant concerns related to security, privacy, compliance and survivability.

Further, the “stacked turtle” problem becomes more visible as the notion of nested clouds becomes reality: cloud SaaS providers depending on Cloud IaaS providers which rely on Cloud network providers. It’s a house of, well, turtles.

The fragile application layer of infostructure, sitting atop infrastructure and held together with the bailing-wire and bubble gum of outdated metastructure yields unintended information intercourse.

We will show multiple cascading levels of failure associated with relying on cloud-on-cloud infostructure/metastructure/infrastructure including exposing flawed assumptions and untested theories as it relates to security, privacy and confidentiality in the Cloud with some unique attack vectors.

The gist of the talk shows examples of the fragility at each of the largely independent info-/meta-/infra-structure layers and then as a whole.

I spend quite a bit of time on the Metastructure layer:

While I plan to give the talk publicly soon at a venue which I will announce shortly, thematically, the talk’s content is already playing itself out in the real world.  If you need good examples as to what I am talking about, I’ll use the two I focus in on with the presentation: DNS and BGP.

You need only look at the latest set of DDoS attacks on social media sites to see how relevant this continues to be.

Much of what holds the Internet and our Intranets together are based upon protocols and architecture never designed to
scale to the levels they are going to get pushed to with Cloud.  Further, the inherent trust in the models used to frame fair play are equally as kaput.

The canaries in the coal mine are starting to chirp very loudly…

I find that people spend a lot of time criticizing the styles of delivery and presentation around securing the Metastructure layer.

They say there’s nothing new.  They say it’s just a way of seeking attention.

I’d suggest listening to the message regardless of what you think of the messengers.*

Talk amongst yourselves.

/Hoff

*Lori Macvittie has an interesting post highlighting this.

There I was in the middle of a half moon yoga pose when the thought hit…

I was on a Telepresence the other day with @jamesurquhart and a couple of other colleagues and we were discussing the notion of Cloud services and multitenancy again.

I brought up a well-known Cloud provider who serves thousands (if not tens of thousands) of unique customers.  I argued that based upon what I was told by system architects, the service was never really designed with multitenancy in mind.  James argued to the contrary maintaining that he has had numerous discussions with the same architects and was convinced my point was invalid.

This got me thinking as to how, if we were talking to the same architects, we came away with a diametrically opposed understanding.

It should be noted that this vendor does not use server/OS virtualization in their offering and since multitenancy is often (improperly) associated directly with server/OS virtualization, we recognized that this wasn’t our disconnect.

Then it dawned on me (well today, during Yoga.)  I was talking about the notion of application multitenancy and James was talking about the database/datastore aspects of multitenancy!  The front-end versus the back-end versus the entire stack…

So of course from James’ perspective, the architects definitely built the database, schemas and table structures to support isolated, discrete and “secure” multitenancy.

However from my perspective, the application itself — a single application — isn’t “multitenant” insomuch as it is multi-user.  The application provides a common programmatic entry point (however customized in presentation) to a specific dataset to which James was referring.

Aha!  Seems simple and somewhat silly, but it never occurred to me that we were just thinking from different ends of the stack; this time I was top-down and James was bottoms-up.  Funny as James is the app. guy and I am the Infrastructure bobblehead.  Stupid siloed thinking on my part distracted me from what I know is a larger system architecture artifact that is easy to spot if I had only taken the goggles off.

This is important because when we apply Cloud definitions to SaaS providers wherein the required characteristics “require” multitenancy (see my post here,) many if not most SaaS offerings fail to meet the criterion.  If we think along the lines of not just qualifying the ‘application’ but expand ‘software’ in SaaS to more broadly include the entire stack including the database, it passes the sniff test.

I have to tell you that this was, despite my own taxonomy diagrams which point out this very fact, a block in my vision which was causing me angst.

So, remember, when we’re talking about SaaS, just because the application front-end may not smell of multitenancy, the underlying platform and database probably will — especially if it’s going to scale to elastic cloud levels.

Silly little lightbulbs go off in the most interesting of times.

/Hoff

I am working on laying down the vocals over the music,

For the love of all that is audible, don’t say you weren’t warned…

The first couple of verses are recorded for your, um, pleasure here.

Here’s  an overview of Defcon sung to the tune of Nickleback’s “Rockstar:”

I’m through with standing in line

for talks I’ll never get in

Didn’t make the top 3 in CTF again

Seems Defcon hasn’t turned out

quite the way I want it to be

(tell me what you want)

I want a brand new netbook

that runs Ubuntu

a 3G channel no one can hack into

And a 4 socket server big enough

to crack passwords for me

(yeah, so what you need)

I’ll need a credit card with someone else’s limit

And a wallet from a fed with nice badge in it

Gonna join the wall of sheep club

everyone makes fun of me

(Been there done that)

I want a bootable CD full of old hack tools

and a way to bypass pesky firewall rules

Need to tunnel SSH…DNS and RPC

(So how you gonna do it?)

I’m gonna trade this life for fortune and fame

gonna grow long hair and use a hacker name

[CHORUS]

‘Cause we all just wanna be security rockstars

Hacking parking meters,

windows-powered smart cars

The girls ain’t easy but the caffeine’s cheap

We’ll all stay skinny, can’t afford to eat

And we’ll hang out in the coolest bars

moochin off those vendors

and their sales whores

Every good script kiddie

Gonna wind up there

No pretty people

but we just wont care

Hey hey I’ll be a security rockstar

Hey hey I’ll be a security rockstar

Wanna be…great like Mitnick

with no stay in the pen

Hire a PR firm to make me cool again

Sign-a couple autographs

buy my book ‘cos it’s not free

(I’ll have the quesadilla… ha ha)

Piss off Apple fanbois

cause quite a mess

pwn your precious iPhone

with an SMS

Escape from a VM

cos you’ve got crappy entropy

(So how you gonna do it?)

I’m gonna trade this life for fortune and fame

gonna grow long hair and use a hacker name

‘Cause we all just wanna be security rockstars

Hacking parking meters,

windows-powered smart cars

The girls ain’t easy but the caffeine’s cheap

We’ll all stay skinny, can’t afford to eat

And we’ll hang out in the coolest bars

moochin off those vendors

and their sales whores

Every good script kiddie

Gonna wind up there

No pretty people

but we just wont care

Hey hey I’ll be a security rockstar

Hey hey I’ll be a security rockstar

Have a big pool party

with killer bees

a bread makin’ panel

with robots that freeze

lock picking fu

and hacker jeopardy

I’m gonna write those sploits

that offend the censors

Gonna pop those boxes

like a Pez dispenser

Get washed-up hackers

rewriting my tools for free

I’m gonna dress my ass

in the black shirt fashion

Donate to the EFF

and promote stack smashin’

Gonna date a sysadmin

blow my money on a brand new Wii

(So how you gonna do it?)

I’m gonna trade this life for fortune and fame

gonna grow long hair and use a hacker name

‘Cause we all just wanna be security rockstars

Hacking parking meters,

windows-powered smart cars

The girls ain’t easy but the caffeine’s cheap

We’ll all stay skinny, can’t afford to eat

And we’ll hang out in the coolest bars

moochin off those vendors

and their sales whores

Every good script kiddie

Gonna wind up there

No pretty people

but we just wont care

Hey hey I’ll be a security rockstar

Hey hey I’ll be a security rockstar

I’m gonna give your mama

quite a fright

when I steal her account

on that Facebook site

If Satan’s on her friend’s list

Jesus really ought to be

You’ve got

“Clobber the Cloud”

Chicks pillow fighting

and even the odd

TV celebrity sighting

Korean spies in disguise

get your bail money for free

Fake ATM’s in the lobby

stealin’ your cash

suicidal cab drivers

who think it’s cool to crash

haxors getting pwned

posting your twitter feeds

I’m gonna trade this life for fortune and fame

gonna grow long hair and use a hacker name

‘Cause we all just wanna be security rockstars

Hacking parking meters,

windows-powered smart cars

The girls ain’t easy but the caffeine’s cheap

We’ll all stay skinny, can’t afford to eat

And we’ll hang out in the coolest bars

moochin off those vendors

and their sales whores

Every good script kiddie

Gonna wind up there

No pretty people

but we just wont care

Hey hey I’ll be a security rockstar

Hey hey I’ll be a security rockstar

(I’ve done this once before, but if it was good once…)

The CISO on trial for his condemnation of Cloud:

Jessep: You want answers about securing the Cloud?
Kaffee : I think I’m entitled to them.
Jessep: You want answers?
Kaffee: I want the truth!
Jessep: You can’t handle the truth! Son, we live in a world that has firewalls. And those firewalls have to be guarded by men with rules. Who’s gonna do it? You? You, Lt. Weinberg? I have a greater responsibility than you can possibly fathom. You crow for Cloud and weep for my obstruction of Web2.0 and you curse my railing against SOA. You have that luxury. You have the luxury of not knowing what I know: that my wishing for Cloud’s death, while tragic, will probably save breaches. And my existence, while grotesque and incomprehensible to you, saves breaches…You don’t want the truth. Because deep down, in places you don’t talk about at parties, you want me on that Cloud. You need me on that Cloud.
We use words like policy, trust, federation…we use these words as the backbone to a life spent securing something. You use ’em as a punchline. I have neither the time nor the inclination to explain myself to a man who rises and sleeps under the blanket of the very draconian enforcement I provide, then questions the manner in which I provide it! I’d rather you just said thank you and went on your way. Otherwise, I suggest you pick up a firewall console and make an ACL change. Either way, I don’t give a damn what you think you’re entitled to!
Kaffee: Did you order the Cloud dead?
Jessep: (quietly) I did the job you sent me to do.
Kaffee: Did you order the Cloud dead?
Jessep: You’re goddamn right I did!!

/Hoff

I made a comment on Twitter a couple of days ago reacting to how some were positioning McAfee’s purchase of MX Logic as the latter representing a “Cloud Security provider.”

The link above has the article’s author referring to the deal as one focused on the expansion of McAfee’s “Cloud portfolio” whilst all the McAfee quotes refer to it as bolstering their “security-as-a-service” offerings.

I read many articles referring to this deal as “Cloud” in nature and in a fit of frustration I said:

I’m sorry, but MX Logic is not a “Cloud Security Provider”

That caught the eye of Erik Boles (@ErikBoles) who suggested that because MX Logic is a SaaS provider, they are a Cloud provider and have been since their start in 2002.  MX Logic’s website advertises them as a SaaS provider, but not a Cloud provider.  McAfee refers to them as security-as-a-service.  I thought it was pretty clear.  Then Erik kept pushing.  I’m glad he did.

We tussled with this and I made mention of the fact that the notions of SaaS and Cloud are mutually exclusive; certainly you can have a company utilize SaaS as a delivery model for their offering, but certain other deployment model and essential characteristics must be met to be considered “Cloud.”

I referred to NIST’s definitions for Cloud service so as to work through this dissonance.

Erik suggested that MX Logic meets the NIST requirements.  I have my doubts.

However, I had to take a step back and admit that because I didn’t know what MX Logic’s operational and infrastructure blueprints looked like, I may be hasty and presumptuous in my ability to dispute Erik’s claims.

Further, I had to come to terms with the fact that  I may be looking through a lens that is inappropriate, limiting or unfair simply because I’m overwhelmed with the marketing shuffle occurring with so many services being branded as “Cloud.”

I decided to sit back and think a little.

So, here’s the issue as I see it:

I think in exploring NIST’s definitions of Cloud, when assessing a SaaS offering’s characteristics against them, the sorts of services that are less focused on a direct coupling of interactivity between the user and the application in the traditional “desktop” sense, but rather replace what would previously be an on-premise network-based infrastructure function, do not fit well in these buckets.

Examples are things like security services: Email/web content filtering, Anti-Spam, Anti-Virus, etc.

Even though they are packaged as SaaS to allow for administration, they replace what are generally considered as infrastructure service functions traditionally-supplied via on-premises hardware/software solutions. These offerings provide a way for the consumer to manage certain elements of the service while the rest is operationally obscured.

I have to admit that when I strap on the goggles, it “sounds” like Cloud, but there’s a profound difference.

While we’ve traditionally modeled that PaaS and SaaS are built upon the foundations of IaaS, many of the now-branded “Cloud” services don’t rely at all on the oft-compared Amazon EC2-like IaaS model at all and rather than scale elastically with a “self-service” capability that the consumer has any interaction with, instead rely on good old-fashioned capacity planning and load balancing using the scale out model ala Google. They used to be called managed services and now they are Cloud.

So if a SaaS offering meets all the NIST Cloud characteristics, like Google Docs or GMail, where a user directly interacts with the “service” to perform a function that would otherwise be done locally on their desktop, that seems easy for people to understand and qualify as “Cloud,” at least given how everyone talks about SaaS today.  When we talk about those infrastructure-like services offered up as SaaS, not so much — at least not for people like me — even if it can be shown that they meet the NIST requirements.

So perhaps we’ve got this backwards.  Perhaps it’s the SaaS offerings that have nothing to do with replacing infrastructure that should not be considered as Cloud services, especially when you consider that many of them are built on traditional infrastructure models.  Then again, we see other offerings like Pixily and Animoto that are SaaS offerings built DIRECTLY upon IaaS offerings that also meet the NIST definitions.

To stimulate debate, let’s take a well-accepted “Cloud” SaaS offering such as Salesforce.com and look through the lens above.  Is it really a Cloud SaaS offering?  Is multi-tenancy over the Internet enough?  Will those SaaS providers who also have PaaS offerings blur the issue even further, especially those who have evolved from the days before “Cloud” was an available marketing term?  Is this what Larry Ellison was getting at when he asked “What the hell is Cloud Computing?”

Just to add some color to the conversation check out a previous post on the topic titled: Re-branding Managed Services and SaaS For Security In the Cloud…1995 Never Looked So Shiny It will likely show up in the “related-posts” section below this one, anyway.

So I think I’ve closed in on one of the biggest confusing issues surrounding Cloud service branding perception:

If a SaaS offering is not built upon an IaaS/PaaS offering that is itself characteristically qualified as Cloud per definitions like NIST, is it a Cloud SaaS offering or just a SaaS in Cloud’s clothing?

Do we need to adjust the definition or just re-focus the lens?

What say ye?

/Hoff

I’ll leave it up to you to figure who’s who [I’m the one with the ‘good’ accent,] but Craig Balding from Cloudsecurity.org and I have teamed up to host a regularly-scheduled (whatever that means) podcast on Cloud Security.

It’s called…wait for it…

The Cloud Security Podcast.

You can find it, and the show notes of our very first (and dodgy) version right here, homed at libsyn. We’ll stick it on iTunes shortly.

We had issues with drop-out over Skype, so I apologize for the annoyances there.

This (last) week’s coverage focused on:

• What we mean by Cloud Computing?
• Upcoming Cloud Security Events/Talks
• Clouds News: Cloud FUD
• Need to get past the FUD, how can you shape Cloud security today?
• Non security specific Cloud linkage

Please do comment on our performance.

/Hoff & Craig

A very interesting philosophical and market trajectory arms race is quietly ramping while the rest of the world tries to ping together how the Kindle will kill Cloud Computing and how Twitter already has.

As @Jamesurquhart and I spend our time exploring the longer term evolution of Cloud Computing, we end up in orbit around the notion of the Inter-Cloud (or Intercloud, or InterCloud)

Inter-Cloud represents one vision that describes how Clouds of many types will interoperate, federate and provide for workload portability as well as how those that provide these services and those that consume them, will interact.  You can see an interesting summary of these issues here in a fellow colleague’s post titled: “From India to Intercloud”

In the broadest sense, Cloud is being positioned in the long term to allow for true utility.  This means that at a 30,000 foot view, consumers should be able to declare their business and technology requirements for workloads or application needs and TAMO! (then a miracle occurs,) that workload or application presents itself operating somewhere that meets those needs backed up by some form of attestation by the provider. Ultimately, I’d like to see a common way of auditing and validating those attestations.  Apropos for this discussion, I bring up the notion of an API 😉

This all seems like a deceptively simple scenario.  Realistically, it represents a monstrous challenge in execution.  To wit, in Reuven Cohen’s recent write-up (“The Inter-Cloud and the Cloud of Clouds“) he quotes Vint Cerf’s definition of the problem with the issues at hand:

“…each cloud is a system unto itself. There is no way to express the idea of exchanging information between distinct computing clouds because there is no way to express the idea of “another cloud.” Nor is there any way to describe the information that is to be exchanged. Moreover, if the information contained in one computing cloud is protected from access by any but authorized users, there is no way to express how that protection is provided and how information about it should be propagated to another cloud when the data is transferred.“

There’s a giant sucking sound coming from the Cloudosphere…

The market is essentially rotating around three ways of describing a solution to this problem:

1. Consumers of service declare their requirements using some methodology for doing so (either directly to trusted and discrete service providers or) using an intermediary or “service broker.”  In the case of the service broker, it’s their job to take these declarations of service definition (service contracts) and translate them across subscribing service providers who may each have their own proprietary interface.  This is starting to heat up as we already have players emerging in this space and analyst groups are picking up interest (Yankee, Gartner)It would be much better if there were an open and standardized way of ensuring that all providers used the same common interface and way of providing attestation of service contract satisfaction/compliance, which leads to…
   –
2. There’s the notion of the “semantic” exchange of information between Clouds positioned by folks like Sir Tim Berners-Lee (in reference to Cerf’s quote above): “…by semantically linking data, we are able to create “the missing part of the vocabulary needed to interconnect computing clouds. The semantics of data and of the actions one can take on the data, and the vocabulary in which these actions are expressed appear to constitute the beginning of an inter-cloud computing language.” Capitalizing on Berners-Lee’s definition of the Semantic Web wherein “a vision of information that is understandable by computers, so that they can perform more of the tedious work involved in finding, sharing and combining information on the web,” we see how this approach would play well into the service broker model, also.
   –
   
3. We’ve seen a lot of noise around using one or more API’s — open or proprietary — that allow for individual Cloud operation, management, assurance and governance, however nuanced those functions may be.  Open-sourced or not, and even with unifying management interfaces available such as libcloud, each Cloud vendor today sees its capability for management and streamlined operations as its first layer of competitive differentiation and individual API’s — even when abstracted through service brokers — are a way to move offerings forward whilst working toward open standards such as these.
   –

Honestly, my bet is that this arms race will net out such that we’ll end up with some combination of all three.

This isn’t as simple-sounding as it started, especially when we throw in the definitional differences between workload portability and interoperability  as alluded to by all three approaches.

Add packaging elements such as OVF and the problem starts expanding into a very complex multi-dimensional issue very quickly.

Workload portability using common packaging formats (such as OVF) can be leaned upon to show how providers might deal the “lock-in” argument (you can move from my competitor to me,) but true interoperability is the real challenge here.

Reuven said it very well: “...what the world needs is not yet another API to control the finer nuances of a physical or virtual infrastructure but instead a way for that infrastructure to communicate with other clouds around it regardless of what it is. The biggest hurdle to cloud interoperability appears to have very little to do with a willingness for cloud vendors to create open cloud API’s but instead the willingness to provide the ability for these clouds to effectively inter-operate with one another. More simply the capability to work along side other cloud platforms in an open way.”

Here’s how I see Inter-Cloud playing out: In the short term we’ll need the innovators to push with their own API’s, then the service brokers will abstract them on behalf of consumers in the mid-stream and ultimately we will arrive at a common, open and standardized way of solving the problem in the long term with a semantic capability that allows fluidity and agility in a consumer being able to take advantage of the model that works best for their particular needs.

Thoughts?

/Hoff

Please See the follow-on to this post: http://www.rationalsurvivability.com/blog/?p=1276

Update: Wow, did this ever stir up an amazing set of commentary on Twitter. No hash tag, unfortunately, but comments from all angles.  Most of the SecTwits dropped into “fire in the hole” mode, but it’s understandable.  Thank you @rybolov (who was there when I presented this to the gub’mint and @shrdlu who was the voice of, gulp, reason 😉

The Audit, Assertion, Assessment, and Assurance API (A6) (Title credited to @CSOAndy)

It started innocently enough with a post I made on the crushing weight of companies executing “right to audit clauses” in their contracts.  Craig Balding followed that one up with an excellent post of his own.

This lead to Craig’s excellent idea around solving a problem related to not being able to perform network-based vulnerability scans of Cloud-hosted infrastructure due to contractual and technical concerns related to multi-tenancy.  Specifically, Craig lobbied to create an open standard for vulnerability scanning API’s (an example I’ve been using in my talks for quite some time to illustrate challenges in ToS, for example.)  It’s an excellent idea.

So I propose — as I did to a group of concerned government organizations yesterday — that we take this concept a step further, beyond just “vulnerability scanning.”

Let’s solve BOTH of the challenges above with one solution.

Specifically, let’s take the capabilities of something like SCAP and embed a standardized and open API layer into each IaaS, PaaS and SaaS offering (see the API blocks in the diagram below) to provide not only a standardized way of scanning for network vulnerabilities, but also configuration management, asset management, patch remediation, compliance, etc.

Further (HT to @davidoberry who reminded me about my posts on the topic) we could use TCG IF-MAP as a comms. protocol for telemetry.

This way you win two ways: automated audit and security management capability for the customer/consumer and a a streamlined, cost effective, and responsive way of automating the validation of said controls in relation to compliance, SLA and legal requirements for service providers.

Since we just saw a story today titled “Feds May Come Up With Cloud Security Standards” — why not use one they already have in SCAP to suggest we leverage it to get even better bang for the buck from a security perspective.  This concept extends well beyond the Public sector and it doesn’t have to be SCAP, but it seems like a good example.

Of course we would engineer in authentication/authorization to interface via the APIs and then you could essentially get ISV’s who already support things like SCAP, etc. to provide the capability in their offerings — physical or virtual — to enable it.

We’re not reinventing the wheel and we have lots of technology and standardized solutions we can already use to engineer into the stack.

Whaddya thunk?

/Hoff

Related articles by Zemanta

• Extending the Concept: A Security API for Cloud Stacks (rationalsurvivability.com)
• Follow-On: The Audit, Assertion, Assessment, and Assurance API (A6) (rationalsurvivability.com)
• A6 Workgroup On The Way Soon (cloudave.com)
• Cloud Security (slideshare.net)

If you haven’t yet checked out the papers and presentations from Usenix/HotCloud ’09, you definitely should.

Some very interesting stuff.

Here.

/Hoff

The view from the last 2 weeks clearly has been from the short bus squad*.

That is all.

/Hoff

*WARNING: Those who travel by means of the horizontally-challenged horseless carriage may be offended by my analogy.  Those of you suggesting I am being insensitive should know that I pick equally on long buses also.