A Couple Of Follow-Ups On The EDoS (Economic Denial Of Sustainability) Concept…
I wrote about the notion of EDoS (Economic Denial Of Sustainability) back in November. You can find the original blog post here.
The basic premise of the concept was the following:
I had a thought about how the utility and agility of the cloud
computing models such as Amazon AWS (EC2/S3) and the pricing models
that go along with them can actually pose a very nasty risk to those
who use the cloud to provide service.
That
thought got me noodling about how the pay-as-you-go model could
be used for nefarious means.
Specifically, this
usage-based model potentially enables $evil_person who knows that a
service is cloud-based to manipulate service usage billing in orders of
magnitude that could be disguised easily as legitimate use of the
service but drive costs to unmanageable levels.
If you take Amazon's AWS usage-based pricing model (check out the cost calculator here,) one might envision that instead of worrying about a lack of resources, the
elasticity of the cloud could actually provide a surplus of compute,
network and storage utility that could be just as bad as a deficit.
Instead
of worrying about Distributed Denial of Service (DDos) attacks from
botnets and the like, imagine having to worry about delicately
balancing forecasted need with capabilities like Cloudbursting to deal
with a botnet designed to make seemingly legitimate requests for
service to generate an economic denial of sustainability (EDoS) —
where the dyamicism of the infrastructure allows scaling of service
beyond the economic means of the vendor to pay their cloud-based
service bills.
At any rate, here are a couple of interesting related items:
- Wei Yan, a threat researcher for Trend Micro, recently submitted an IEEE journal submission titled "Anti-Virus In-the-Cloud Service: Are We Ready for the Security Evolution?" in which he discusses and interesting concept for cloud-based AV and also cites/references my EDoS concept. Thanks, Wei!
- There is a tangential story making the rounds recently about how researcher Brett O'Connor has managed to harness Amazon's EC2 to harvest/host/seed BitTorrent files.
The relevant quote from the story that relates to EDoS is really about the visibility (or lack thereof) as to how cloud networks in their abstraction are being used and how the costs associated with that use might impact the cloud providers themselves. Remember, the providers have to pay for the infrastructure even if the "consumers" do not:
"This means, says Hobson, that hackers and other interested parties can
simply use a prepaid (and anonymous) debit card to pay the $75 a month
fee to Amazon and harvest BitTorrent applications at high speed with
little or no chance of detection…It's not clear that O'Connor's clever work-out represents anything new
in principle, but it does raise the issue of how cloud computing
providers plan to monitor and manage what their services are being used
for."
It's likely we'll see additional topics that relate to EDoS soon.
UPDATE: Let me try and give a clear example that differentiates EDoS from DDoS in a cloud context, although ultimately the two concepts are related:
DDoS (and DoS for that matter) attacks are blunt force trauma. The goal, regardless of motive, is to overwhelm infrastructure and remove from service a networked target by employing a distributed number of $evil_doers. Example: a botnet is activated to swarm/overwhelm an Internet connected website using an asynchronous attack which makes the site unavailable due to an exhaustion of resources (compute, network or storage.)
EDoS attacks are death by 1000 cuts. EDoS can also utilize distributed $evil_doers as well as single entities, but works by making legitimate web requests at volumes that may appear to be "normal" but are done so to drive compute, network and storage utility billings in a cloud model abnormally high. Example: a botnet is ativated to visit a website whose income results from ecommerce purchases. The requests are all legitimate but the purchases never made. The vendor has to pay the cloud provider for increased elastic use of resources where revenue was never recognized to offset them.
We have anti-DDoS capabilities today with tools that are quite mature. DDoS is generally easy to spot given huge increases in traffic. EDoS attacks are not necessarily easy to detect, because the instrumentation and busines logic is not present in most applications or stacks of applications and infrastructure to provide the correlation between "requests" and " successful transactions." In the example above, increased requests may look like normal activity.
Given the attractiveness of startups and SME/SMB's to the cloud for cost and agility, this presents a problem The SME/SMB customers do not generally invest in this sort of integration, the cloud computing platform providers generally do not have the intelligence and visibility into these applications which they do not own, and typical DDoS tools don't, either.
So DDoS and EDoS ultimately can end with the same outcome: the target whithers and ceases to be able to offer service, but I think that EDoS is something significant that should be discussed and investigated.
/Hoff
It's good that you're writing on these topics, but as I pointed out in a comment on your original posting on this topic, this isn't a new or original concept. You've done a good job of articulating it, but you cannot legitimately claim that you originated this concept, as you attempt to do in the title of this post.
To be clear – the effects you describe have been happening for years, and are not new or unique in any way. Your contribution is to discuss these issues in a clear and articulate way, but this concept and the effects you describe are not original, but rather are quite common, and it's important to realize this isn't a new or rare phenomenon, but rather something that must be dealt with every day.
Roland:
Yeah, got it. I'm an un-original regurgitator of slightly interesting but prior art that strangely appears not to be discussed elsewhere in context or named as such.
Go ahead, Google it.
<sigh>
Can we move past this silliness and address the topic which I find fascinating?
🙁
P.S. I challenge your assertion that this is something "…that must be dealt with every day." You want to debate something, let's debate this.
Click-fraud.
In fact, it's even more basic than that. While there are DDoS attacks which are motivated by ideology, internecine subcultural conflicts, or nihilism, most DDoS today is in fact economically motivated. This has been the case for the last 6-7 years.
One form is DDoS extortion, in which businesses with online revenue streams are DDoSed, and then receive extortion demands to pay 'protection' money. Another is DDoS-for-hire, in which one organization pays for a DDoS against a rival organization in order to drive the victim organization out of business. Either way, the goal of the attackers is to make money by causing the victim and/or his SP(s) economic hardship which he cannot afford to endure.
With regards to DDoS attack methodologies, in some cases, attackers will perform reconaissance and identify Web UI interfaces which can be repeatedly invoked in order to generate a sufficient number of script executions or database queries to DoS the back-end of a multi-tiered site. There is often a significant amplification factor gained by using these methods.
So, at root, DDoS attacks *always* have economic consequences (in opex, if nothing else), both for the victims as well as those affected by collateral damage – and the most common motivation for DDoS attacks is both economic gain for the attackers and economic loss for the victims.
Roland:
Again, you're missing my point. I took a concept and applied it in context to a very specific scenario which I claim is not being thought/talked about. That's what I do.
Many of us have already had a rousing discussion on Twitter regarding all the ancillary derivations of DDoS; click fraud is one of them. I certainly admit (as I have already multiple times) that EDoS is simply an application/derivation of DDoS. If you re-read the post carefully, it's about leveraging the uber-scalabiity of cloud environments to make problem worse.
If it weren't interesting, people wouldn't be cross-linking and discussing it (as they have) or including it in journal submissions (as they have.)
I'm glad the topic generates discussion.
What more is it you'd like me to say; that my "spinning" a topic for the purpose of getting people to think about consequences is somehow intellectually dishonest? The title of this post isn't some attempt at grandstanding or claiming I "invented" anything.
You could apply your nitpicking regarding provenance of an idea to almost EVERY topic in information security and every blogger.
Can me move on now, please?
What I'm saying is that you write about this topic as if it's something which we need to watch out for in the future, when in fact it's happening *right now*, and in fact has been happening for a long time.
EC2 has already been utilized by spammers who paid for the service using stolen credit cards; VPSes have been cracked and used as botnet C&Cs. These are just a few examples.
Yes, I think it's a bit much for you to claim '. . . My EDoS (Economic Denial Of Sustainability) Concept . . ', because it's not a concept you invented; the bad guys were there long before, and the operational security community has been dealing with it ever since. Nor do I think 'EDoS' is a useful term, because most DDoSes are 'EDoSes', after all. In fact, I believe that by coining a new, confusing, & unnecessary acronym and presenting this as a new threat that folks must worry about in the future, rather than something that is common place and happening now, even as I type this comment, that presenting in this manner may in fact be somewhat counterproductive
You're one of the more clueful people involved in the general security community, and it's good that you're discussing these issues and bringing them to a wider audience. At the same time, it's important that people understand that this is not some theoretical concept – it's real, it's happening now, it's been happening for years, and, as you rightly indicate, it's only going to become more widespread, unless/until operators and users of cloud-based services instrument and baseline their systems so that they can proactively detect and mitigate abuse of their systems.
While this topic apparently has been discussed elsewhere, I (for one) had missed it. Equally important is that I find the issue worthy of serious consideration.
To me, the issue becomes one of economics (as always ;-). Smaller companies seem to be some of the early adopters of certain cloud computing services, and these are the same people who will lack the resources to cover a massive, unexpected uptick in utility charges. Try telling a Fortune 500 company that their utility bill is going to spike…they might not like it but it doesn't threaten their economic existence. . What I'm wondering is whether small companies also lack the assets to be interesting targets for $evil_person who wants to extort money; if they cant afford a big utility bill they cant afford much extortion payout.
To me the question is whether the $evil_person can find a sweet spot between someone's willingness to pay extortion funds but inability to pay a big utility bill. Add to that the fact that the extorter has to threaten their target in advance, which presumably allows the target to at least attempt to create a defensive posture (technology, amended contract with their service provider, etc)
I don't know the answer, but firmly believe the economic value of this attack will dictate whether we see it used or not…and also think that the cloud service providers will take contractual action to limit the economic damage it could cause to their business if customers are too worried to buy their service. Certainly worth building a model of the whole system to see where it balances out.
OK, Roland. Uncle.
I haven't the energy to debate this any longer, especially since I've basically agreed with your points regardless of your continued need to intimate I'm just grandstanding.
I agree, this is something we should worry about (now and in the future) which is why I wrote about it in the first place.
It's unfortunate that you find the term and premise not useful, "confusing and unnecessary." I think it makes people look at this problem from a different perspective. No, I *know* it does.
Your conclusion "… it's only going to become more widespread, unless/until operators and users of cloud-based services instrument and baseline their systems so that they can proactively detect and mitigate abuse of their systems" is the reason I brought it up.
I've changed the title from "…my EDoS concept" to "THE EDoS concept" so as not to further offend the throngs of protesters upon whose shoulders I am standing.
😉
/Hoff
I just want to talk a little bit from the aspect of anti-malware vendors.
Right now they are overhauling their current infrastructures, trying to
move huge signature files from desktops to clouds. As the benefits, security software
will consume less desktop resources and only download light-weight signature files.
The vendors hope Anti-virus cloud can cut their ISP bills.
However, extra costs resulting from EDoS, or extended DDoS, will tradeoff the savings.
In my opinion, I think this scenario is a new threat model for AV cloud. Just like I cannot say a five-year-old virus packed with a latest packer, which has bypassed most AV scanners, is still a old malware.
Lou Steinberg – there's no question at all, it's been happening for the last 6-7 years, and is happening as I type this comment. Most DDoS today is either DDoS extortion or anticompetitive DDoS. That is not speculation, that is well-documented fact.
And, no, the attacker doesn't threaten the target in advance – he attacks first, then stops, then threatens (just like in real-life 'protection' rackets, btw).
This isn't hypothetical. It's real and happening *now*, and has been for years.
PS – Hoff, see why I object to this whole 'EDoS' thing and the speculative general tone of your posts on this topic? Lou Steinberg has been led to believe that this is all theoretical, when in fact it's been going on for the better part of a decade.
Thanks for changing the title, btw. One small step at a time. ;>
@Roland:
I'm beginning to find great humor with the surgical precision which which you continue to choose to ignore the scenarios, questions and context posed by myself and the other commentators above and instead just repeat the same thing over and over…
I've conceded a point and even tried to shake you off this bone by changing the title, but I see that this was the wrong tact to take because it's clear you're not interested in looking at the points/positions others are raising here and instead are just going to keep bullying the point until someone submits.
Everyone who has contributed to the comments has interesting and valid points. You've made yours about 20 times. We hear you. I hear you.
I simply don't need to have my opinions muzzled by you, especially since it's my blog.
I disagree with your reduction of my points to nothing more than speculation, but you're welcome to it.
I appreciate the time you're taking to make your point, just not your style. I'd bet you might say the same to me. So at this point, let's just agree to disagree.
Thanks,
/Hoff
@ Wei:
Which IEEE Journal will your paper be published and when? Curious if you've seen the UMich work or dealt with any of the current offerings such as VirusTotal or NoVirusThanks.org?
Andre:
Yes, I have read the "Cloud AV " paper.
It is a nice work. However, I would rather classify it
and other similar systems as sort of malware voting systems.
Sending a whole sample into the cloud is expensive.
BTW, my paper is still under reviewing.
If need the draft, pls drop me a line weiyan@lycos.com
I haven't ignored anything – I've read it all, understood it all. My point is that noe of it's new, much of it is unnecessarily aond confusingly duplicative, and the general thread of the conversation indicates a lack of awareness of the operational security landscape over the last decade or so.
This is *your* weblog – how could *I* muzzle *you*, heh?
Let my try one more time – you're a smart guy; it's good you're discussing these topics; nothing brought up in this thread is new or unique; those of us who've been dealing with this stuff for many years are always glad to see these topics gaining traction in the general security community; in order to gain an appreciation of the actual threat horizon, one must understand the history and context of these types of attacks, the motivations behind them, and the efforts and BCPs intended to help defeat them.
Let's just don't make the mistake of ignoring years of history and experience by somehow convincing ourselves that any of this is new, that somehow these concepts sprang forth fully formed from anyone's forehead in the last couple of months or so. That's all I'm asking.
Thanks for the discussion and for your efforts in this arena.
At least IP transit is charged as 95th percentile. The way you buy time on a cloud today, EDoS is a lot more serious than DDoS.
Although I guess at some point, DDoS attackers evolved and figured out that if they attack in just over 5 minute increments at least 447 times a month, then they could shift that 95th percentile. Actually, no wait – they didn't.
The problem with DDoS/EDoS isn't the additional OPEX from the "service usage per time involved" from the customer to the service provider. The largest issue with DDoS is either that the pipe is filled (an easy issue to solve if using a RFC 1997 BGP community that performs a blackhole on your entire IP space under attack) – or more importantly – that the infrastructure attacked continued normal operation (which it rarely does since SYN attacks or HTTP starvation fill up memory very nicely while also causing high or max CPU and throttling disk I/O).
So besides the current way that Cloud is sold, EDoS isn't that big of a deal since it doesn't take down infrastructure.
I'm going to have to categorically disagree with you about the significance of EDoS.
@Dre:
"So besides the current way that Cloud is sold, EDoS isn't that big of a deal since it doesn't take down infrastructure."
Really? So if my Ops budget is $10K a month for a cloud-based service and under rising (but seemingly normal use due to what is perceived popularity) my bill from my cloud provider jumps to $50K (or more) with no appreciable rise in revenue, it's "…not that big of a deal?"
It doesn't even have to be an "attack." The elasticity of the cloud can get you into trouble just because of a poor job of capacity planning vs. resource management.
We certainly have tools for dealing with upstream-sourced DDoS attacks; many of them are operated by the service providers themselves as a selfish defense against protecting their infrastructure that, in a cloud, your business operates on.
If someone is utilizing a targeted EDoS attack methodology because they know you're on Cloud Provider X, and have no visibility into your application stack if you're using IaaS, how does the provider distinguish between "good" and "bad" traffic since it's not swamping the pipes?
Answer: They don't.
So, you can certainly disagree about the significance of EDoS, but I think that this illustration and others where the apps. are abstracted from the infrastructure, is going to be a major problem until standard API's are put in place across cloud providers (and the underlying infrastructure such as app. delivery platforms) and instrumentation is baked into the apps.
/Hoff
Yeah I guess what I'm trying to say it that if cloud providers change the way that they sell minutes/usage, then it's less of an issue.
Change the economic model.
Just so I want to understand…
You're suggesting that an industry of vendors who mostly ignore security in the first place are going to abandon the utility model of "pay-as-you-go" that is one of the principle operating cornerstones of Cloud Computing in order to solve the problem above (or DDoS in general?)
Uh, no.
😉
I'm just saying there's an easy fix, or at least an easy path to prevent EDoS from becoming profitable or "worth spending time on".
Look at it this way: DDoS extortion and phishing are now zero-sum games. Zero-day based botnets are effective but expensive. There are already so many 6 month old third-party application vulnerabilities out there, and this will likely be true for any cloud – internal/external, private or provider.
By "an industry of vendors who mostly ignore security", what exactly do you mean? Do you suggest that they are providing cloud VMs that are unpatched and insecure-by-default? Are the authn/authz at the guest OS management and hypervisor layers insecure? How does this relate back to EDoS?
Dre>> By "an industry of vendors who mostly ignore security", what exactly do
Dre>> you mean? Do you suggest that they are providing cloud VMs that are
Dre>> unpatched and insecure-by-default? Are the authn/authz at the guest OS
Dre>> management and hypervisor layers insecure? How does this relate back to
Dre>> EDoS?
Seriously? The vendors I am referring to are the platform providers and for the purpose of EDoS examples, IaaS vendors, specifically. PaaS/SaaS vendors generally have tighter control over their infrastructure as is the nature of things.
The VM's you run (or AMI's in Amazon's case) are only as "secure" as you, the customer, make them. The authentication mechanisms? Go check out Craig Balding's write-up on the Amazon Secret Key, for example.
Look, in IaaS, the infrastructure is ABSTRACTED. That's the whole point. They don't have visibility into the appstack. It's just like colocated boxes — your ISP gives you an Ethernet feed. In that case you may get DDoS protection because they want to protect THEIR infrastructure (or charge you more for DoS/DDoS at the NETWORK level for yours) but they are WOEFULLY IGNORANT of the services and apps running on your boxes.
This is the case I'm making: DDoS is NOT EDoS. Ultimately, EDoS leads to DDoS, but not because of brute force.
I'm going to have to dwell on another analogy I guess…
Did that help any?
/hoff
For the Cloud user, EDoS leads to action items N through N in the change management backout plan. For the Cloud provider, EDoS leads to billing on 95th percentile usage or some other economic model change. It will also likely lead to more private clouds, more use of DWDM with SONET BLSR or RPR (as we thought would happen with the fiber glut 7 years ago), and probably tons of metro GbE private interconnect if it actually pans out like you explain. Wait until Level-3 or Qwest starts up their own IaaS.
This whole cloud thing is pretty new to me, but I get it. I just don't know who you are talking about besides Amazon EC2, and yes, I'm aware of their problems (and yes, it's all-inclusive and I agree on every point).
As for the PaaS and SaaS guys, I don't think that they bill like IaaS, right? I'm not sure how Force or GApps works, but in traditional SaaS, they would lose every customer if they started charging the customers for their usage of their own IP transit, CPU/mem/disk usage, and other factors.
With the exception of Akamai, which is easily the most profitable SaaS of all time. I have no idea why people would pay those insane prices.
A really late chime in.. we (SensePost) touched on this briefly in a short blog post back in 2007 [http://www.sensepost.com/blog/1627.html%5D
-sigh- it seems the link gets up-screwed in translation:
http://www.sensepost.com/blog/1627.html
Sweet.
/Hoff