Rational Survivability

Mmm; a very interesting discussion topic.

There’s a further angle, too; while a pentesting team will have a Board-level letter excusing them of any harmful consequences of their actions (and if they haven’t, they have a rather bigger risk appetite than most…), I would expect them to also be under some degree of unwritten advice that, if they’re going against a production environment, attacks which take the environment down or otherwise disrupt operations may cause pain to employees or the wider business as a whole.

By contrast, there’ll be some subset of black-hats whose likely *intention* is to take the business infrastructure down, in such a way that it’ll be either hugely painful or practically (beyond financial reason) impossible to bring it back up.

The question is, what the commisioning organisation’s threat model and risk appetite is, as to where the balance of pentester activities gets set.

This is a dilemma, and it has horns…

I dont think so – I say the stakes are the same.
Of all the red team testing I’ve ever done, I was always afraid of having to deal with the cops.
Just because you have a get out jail free card doesn’t mean the cops wont:

1) knock your teeth out anyway
2) taze you
3) shoot at you
4) arrest you anyway
5) any number of other unspeakable horrors that law enforcement is commonly associated with.

They beat the shit out of you first, THEN they ask you about your paperwork.
Not fun – even if you have a get out of jail free card.

Two thoughts. First, EVERY pentest/red team engagement should have a get out of jail free card. It should be part of the standard contract. Second, even with a get out of jail free card, the pentest/red team still has to operate within the bounds of law. The company that hired you cannot give you permission to attack a third-party vendor and pivot through to the client’s system. Nor can they give you permission to impersonate a police officer to attempt to gain physical access. The company can only guarantee they won’t press charges if they have the option to press charges.

In many instances, it’s already a false sense of aggression simply due to time constraints. Red Teams operate in a given time window, with scope limitations/etc. for much of their operation.

Sure, having a ‘get out of jail free’ card enables more ‘chutzpah’, but in a sense, there isn’t a realistic other choice within the other constraints placed upon them.

Actual adversaries will take their time (as time is on their side in most cases), which enables a much more ‘low and slow’ approach to things.

Interesting thought. An adversary with everything to lose might not attempt certain techniques that may have the potential of exposing them, whereas a red team with nothing to lose could throw everything and the kitchen sink at the target.

The red team test might give you a sense of pure vulnerability, but maybe not real world vulnerability given an attacker with something to lose. So, here’s a question – if you can’t use a red team test to determine real world vulnerability how do you use the results to place a priority around remediating?

I could prolly blather on about this all day but I will try and keep to the question

“does the team dial-up or dial-down the aggressiveness of the approach and execution KNOWING that they won’t be prosecuted, go to jail, etc.?”

To get to the answer of this, or at least my opinion, we need to start off with WHY they are Red Teaming. Now that this is becoming a bit more of an en vogue service there is starting to be a large degree of variance of “WHY and HOW.” for this one… let’s just take Red Teaming as “Modeling an adversarial force and executing the attacks with the expected capabilities of that force.” If we can stay to those definitions we get to some of the answer in a less grey area.

We like to look at testing like a fight.

First off, Its a fight. It is not theory, there isn’t some ” theoretical risk ranking” to how you are going to feel or a mapping to some color wheel that can communicate to you how you “WILL” feel during the event. It IS the event.

Now…. there are a few types of fights/fighters each type breeds a different type of fighter. (mind you… there are anomaly’s in all of these but let’s take it as sweeping generalizations)

Typical playground fight (Vulnerability Assessment):
The adversary is usually around the same size, motivation is similar, they aren’t out to kill ya, and over all they very rarely even know HOW to hurt you. Since the adversary has not had a lot of time to train or lots of experience in fighting the overall risk of total destruction is low. The benefit of it is feeling what it is like to get into a fight. You also take away some things you need to do to either avoid a fight or fight better.

training: Low
benefit: Low
risk: Low
education:low-moderate

True /Bar/Early Adult/Public Fight (Penetration Testing):
The adversarial gap is much larger an unknown. The size and strength difference is an assumption, as is their ability to hurt you. The adversary could be an expert or a first timer and the only time you will realize it is by the time the fight is underway or sometime OVER. The real thing to be concerned about in this type of fight is that the risk of “impact” is substantial. Mature humans with potentially TONS of experience may have the ability to completely destroy someone if the reigns are not pulled tight. The benefit of this fight is that they are usually over a specific threat and the winner can support the desired outcome **getting robbed and overpowering the robber …as the example**

training: Variable based on adversary
benefit: moderate
Risk: Moderate to High
education: moderate * when you are a little bit older is when you start to realize if you can survive it or not.

Professional fighting (Red Teaming)
Now apply that to a pro fighter. Their entire life is devoted to the fight. Their financial viability and lifestyle RELY on it. They have sparring partners, coaches, strength trainers, agility trainers, nutritionists, therapists, and more just to get ready for the fight. When they prepare for a fight, they don’t just fight anyone. They prepare for the fight with a very specific regimen. They are well beyond the days where they need to gain a sense of calm during the event. They prepare for the fight with a sense of purpose and extremely well defined goals

training: HIGH
benefit: HIGH
Risk: Low to Moderate ( these are trained professionals…. although death happens it is VERY rare)
education: HIGH

The reason I had to go through all that is to give a sense that this exercise is not just a ” look at how hard I can beat someone up” as a matter of fact it is almost the complete opposite. It is much more about “how many areas can I test, and how will my adversary test those areas.” Each adversarial group will have a higher level of skill/competency in each of the 3 areas of red teaming (Physical, Social, and Electronic). By a company understanding their adversarial classes and their capabilities in each of those areas… they can determine the level of strength they need the red team to test in each. If we are testing an art museum, we can assume that the most likely adversary will be well equipped in the area of physical attack. Depending on the “type” of art museum…..we may find that the adversary has other skills in social or even electronic….. if we model out who the most likely attackers are. Maybe there is a diamond exhibit going on and we know the groups like the Pink Panthers (http://en.wikipedia.org/wiki/Pink_Panthers) are going after it. They have a particular set of skills that are readily available for research. Now there is no need for an insurance company to model the panthers type of attack because we can see through past compromise that the insurance companies get attacked in a much different manner. All of this is much like our pro fighter… they can watch the tapes… identify the likely attacks and the “surprise” moves the other opponent has. they prepare for the fight they ARE going to get in… not the one they MIGHT get in.

In addition to all of this, there is another component to red teaming…. the blue teamer. On every red teaming engagement we offer a blue teamer to ride along with the internal team. You can get a full picture of where there are breakdowns…. even if the red team does not expose it. The blue teamer also gets to measure how cool under fire people are. They may get lucky stopping an attack and the blue teamers job is to identify whether or not it was a fluke or part of the process. This is much like having a coach and a ref in the ring with you. You walk away with a better idea where/why/how to train while still staying within the comfort/pain level of the fighter. this is a CRUTIAL component and literally doubles the value of testing if done concurrently.

So … what’s the quick answer without all this blathering on and on and on????

“The red team’s job is to adequately scope the potential boundaries between training and fight night, and bring their opponent RIGHT TO THAT LINE but never over it”

A criminal doesn’t care about your safety, if you die as a collateral damage… who cares….. as long as they get what they want they do it. As red teamers we just can’t go that far. Don’t kidnap the CEO…. just show em every bit of Intel and surveillance needed to get to the point right before the bag n tag. Don’t burn down the building just to cause a diversion…. show em how it “would” be done. Don’t sell the data on the black market…. show them how/where /when u could get access. Don’t show them that you can bust a door down….. assess if you DID bust it down… how they would know and what the response would be. It’s a fine line to tow…. but if done right you get to patch the unpatchable…. HUMANS.

Security is a feeling not a static concept of technology. The only patch we get in our feelings is experience. The more we can get the defense team to experience a likely threat…. the more calm, cool and collected they will be the day that threat is real. Find the real Perimeter and just barely go over it.

Hunter S Thompson said it best.. ” The Edge… there is no honest way to explain it because the only people who really know where it is are the ones who have gone over. “

All of these are a false pretense. There are too many limits and controls warranting this to basic QA.

Structure your testing this way:

1) Publicly tell “Anonymous” to [insert egregious insult]
2) Call Russian black market teams and tell them you are impervious
3) Communicate with NSA enemies
4) Wait for either system shutdown or announcement on public news channels of system breach

This is a “real” testing methodology… 😉

Any team worth their salt will execute as if getting caught is a fail. Any consultancy that doesn’t provide a major or full refund if they have to use get out of jail free should be scrutinized.

Todd, I could not disagree more! If you clients constantly fail in every area of testing…. either #1 they shouldn’t be doing that type of test… or #2 You are SO terrible at communicating the resolution that they do not understand how to resolve the problem. This type of work….as well as most other security testing, is about measured results and the ability to show progress and change. That can’t happen if you just “win every time.” It is this kind of smash and grab mentality that provides NO VALUE to a client what so ever. What a Red Team SHOULD be scrutinized for is providing clear, concise, and repeatable results that test a specific set of capabilities which range across social, physical,electronic and converged attack surfaces. Their results should be quantified based on impact to the business and their overall quality should be measured by how much they can educate the company during and after the exercise. Simply saying…. ” we will get in every time” is a waste of time/money and effort….

Test many iterations, some should fail some should succeed. Don’t do red teaming as some self edification of how cool you are…. model the adversary and act JUST as they would. TRAIN your client for the fight. Good trainers can take their students to a place where their own skill is surpassed in areas…. shitty trainers just beat up their students and laugh because they are not good enough.

I certainly don’t want to devalue any of the other great findings in a test. As you say in #1 that they shouldn’t be doing that type of test. Definitely not advocating smash &grab either. Rather, in addition to providing the customer max value for where you succeeded you would internalize how you improve your own process via get out of jail free card and maybe even provide said customer advice on how their successful detection method could be even better. Re:how does this affect what methods are used vs not if you dont have the reprieve…likely those methods that cause permanent damage to the biz to test arent used. Good discussion.

Todd… for sure…. should never BREAK things unless they are willing to simulate that type of test. Back in our sprint days, our DR test as provided by the security team, was to go into the datacenter and start pulling cables and taking servers out of the rack. The DR team had gotten to such a proficient level of execution that they were ready for us to “do our worst” without permanent damage to the facility. I think the Netflix guys do this today ( or so i’m told). There is a bit to be learned from that in Red Teaming. We are essentially a modeled disaster. The level of disaster is often set by the maturity of the program. You can bring them all the way to the edge and say….. but if i did THIS….. it would all fall down. the hope there, is that each time they get a little further. I guess this is also like massage therapy. You cant just dig into it, you have to have a graduated approach. I think a mix of failure and success is needed on all sides for the psychological impact of the test. If its all failure, they dont get to see the good work they put in fixing the things of the past. We joke around in class saying “You are much more a therapist than an operator. You are there to connect them to the feeling, make it real, and show them that there IS a light at the end of the tunnel.”

As for this whole thing with the cops… I have dealt with the police many times on engagements. I have had every interaction from them “helping” me search the facility for the intrusion… to getting my occipital broken. Yet another thing to be planned in the scoping phase of the engagement. Are we testing the ability for local law enforcement to follow a specific procedure set forth by the company? OR are we just seeing if we can outsmart a cop? If it’s the latter, we aren’t staying on target for the client and the police’s ability to follow procedure/policy that is internal is just not our gig….. maybe its time to talk to that department about doing a test =) Best example i have for these is that in a gig we were doing, the client had a specific agreement with the local authority on how to respond to an alarm. It was laid out the minimum response time to show up onsite, the contact chain of command, the onsite radio band to communicate on, coordination, sweep patterns, verification of credentials, and how response should be updated,tracked, and the ultimate resolution closeout. Those were all things we had to incorporate into our test to identify the breakdown in the chain. There are others who just say…” if the alarm goes off.. .the police will come.” In those cases…. we ask ” do you know that for sure… how long does it take .. what do they do…etc” For each one of these scenarios, the red team has an obligation to answer the unknown questions with facts and provide recommendations on how to improve the process if any.

agreed…good discussion we need to have more of.

@Chris Nickerson
Stumbled across this one…can’t resist responding. I haven’t read everyone’s stuff but this one is calling to me…

Sorry Chris, but what a bunch of nonsense. Geez laaaweeeez. This is the type of melodramatic blathering that makes it difficult for folks in our biz to be taken serious by the people we are trying to help.

Playing around with Kali is easy but simulating threats isn’t all that easy and most of us are not doing it well. First, there are no “get out of jail free” cards – if you break the law you go to jail. Two, it is not possible for nearly all of us to simulate the really bad, well-resourced, threats.

To simulate them to even a basic extent, we need to start by understanding our customer and their biggest threats.

To understand threats we need to go beyond simple tools and techniques. We need to do a better job of understanding this broad category and recognize our limitations so we can explain them to our customers.

To understand our customers we need to be able to communicate with them…they are suits and most do not speak Klingon, leet, or silly fight speak. They pay our bills – we need to speak their language not force them to learn ours. They speak biz 101.

The entire reason for our existence is to identify risk so folks and fix/mitigate it. Let’s not forget this critical point — and please retain some level of perspective and understanding when we communicate the risk.

michael… not sure i get your point. What was said above is to understand the adversary and understand what is most important to the company. It looks like what you said as well. the “get out of jail free” idea is about saying that the customer is allowing you to perform the test and isn’t going to prosecute you **ust like when u do a pentest…tehy agree to not lock u up for hacking things. I’d like to understand where you are going with the point but I am kinda lost to what your “suggestions” are to answer the initial question posed or what the complaint in other peoples responses really is. Any way to be a bit more clear on that? Its over great interest to me and many others.

Are we testing the ability for local law enforcement to follow a specific procedure set forth by the company? OR are we just seeing if we can outsmart a cop?