Rational Survivability

[Updated below 6/8/15]

The last ten years of my professional life have been spent working for some amazing companies; architecting and building security solutions that are deployed across the globe in the most demanding circumstances imaginable.  It’s been an incredibly fulfilling, challenging and interesting set of challenges, growth opportunities and victories.

I have met amazing people across the globe and the shared perspective of collaborating and co-creating solutions with such a diverse group of people has blown my mind.

The most rewarding aspect is that I have been extremely lucky to have worked alongside really amazing teams of passionate and dedicated people.  I have learned a lot from all of them and I believe I have made a difference.  Through ups and downs, the consistency of a clear mission and a motivation to fight the good fight with credibility, authenticity and transparency has kept me energized.

At the same time, the nature of society, humanity and the way in which our industry has dramatically shifted has driven me to think about the next stage of my career and how I can apply my passion to solve really thorny problems that have equally dramatic outcomes.

I want to take the lessons I have learned and return back to role where I can apply them to help solve them.

With that as motivation, I will transition from building “fire extinguishers” and return to my roots of “fighting fires.”  I have decided to return back to the Enterprise and apply my skills to solving complex problems at a global scale.

I won’t be talking much about where I am going, but it also won’t be a secret…I just want to focus.

My family and I will be moving from California and look forward to a pace of life outside The Valley.

I will truly miss my Juniper family and the amazing company that I have spent the last 4 years helping grow.

/Hoff

Update 6/8:  I can’t say I’m really surprised, but it’s really easy for people to assume the worst when someone in a leadership spot leaves a company; that stuff is falling apart, that there is some acrimonious scenario that makes it untenable, that the strategy or execution is in peril, etc. 

Let me be pointedly clear: NONE of those things is relevant to why I chose to leave Juniper.  The re-aligned strategy and roadmap for both stand-alone and integrated (into routing, switching and SDN solutions) security has never been more solid.  The execution and leadership under Jonathan Davidson and Rami Rahim as CEO has never been better.  I really do think that what we  put together lets Juniper leverage its strengths and differentiate clearly.  I have no interest (financial) in Juniper that would motivate me to state this, but I simply hate half-truths invented by people too lazy to do actual research or ask for comment.

So why leave?  Because after four years of dedicating myself to helping getting Juniper to a better place, I need a break…I need a change of pace and time to reconnect with my family in a slower pace of life…because I need a different set of challenges (not “easier,” but different) that allows me to put my skills to use in a different way.

That’s it. If you’re an analyst, partner or customer of Juniper, don’t read anything else into my departure.  Please.  You’re doing yourself and Juniper a disservice.

If you have any questions or concerns, please reach out to me directly instead of creating your own version of the truth.

There are three immutable rules that pertain to presentations*, even more so important if one’s presentation is in front of several thousand people, live:

1. Never present with kids
2. Never present with a live demo
3. Never present with animals

Not doing one outa three ain’t bad 🙂

My 2015 RSA Security Conference keynote was very special to me.  Besides violating two of those three rules, it was a deeply important and even more deeply personal experience.

My non-profit, HacKid, was a participating organization in RSA’s amazing partnership with The Tech Museum of Innovation and the Cyber Security Safety Village.

It was only fitting then, that I used the wonderful community outreach of my employer, Juniper Networks, to do something very different, actionable and useful with my time on stage.

I was lucky enough to find a willing co-pilot to help me talk about the next generation of security riffing off the lyrics of The Who’s “My Generation.”  You’ll have to watch the video to see who.

You can watch the video below or find it directly, here.

Below is a blog that I wrote and cross-posted from Juniper’s website:

If cyber security in the last two-and-a-half decades has taught us anything, it is that it’s painfully obvious that the tools, tactics, techniques and procedures employed by adversaries bent on causing us harm both professionally and personally have outpaced our ability to successfully defend ourselves.

While there has been progress and innovation in the use of technology products as the frontline of defense, there’s a very important aspect often overlooked in the totality of the solution space.

The “people” element has always been mentioned as an area of focus for cyber security. But beyond technology, we have not done enough to ensure that security and privacy are not eroded for the sake of convenience, putting people at greater risk.

When we have attempted to address the “people problem,” we have often relied mostly on awareness (theory). In the face of sophisticated methodologies of abuse and attack – and frankly some very basic ones – we expect people to make the right decision in the face of complex and confusing events (practice). This is a disaster – or breach – waiting to happen.

Over the last few years, a number of initiatives and organizations have been established to address this issue in a meaningful way, but with a twist and caveat: Rather than focus on adults – the developers and professionals of today – these initiatives and organizations have focused on reaching out and educating children, who are literally the “next generation” and frontline of protection against cyber-attacks.

One thing we have learned is, when dealing with children, it’s not enough to just raise awareness about proper cyber security hygiene and behavior. It’s not enough to simply tell them what not to do – that’s too overwhelming and overall, it’s the wrong model. We must build creative and constructive means, in which they are able to actively contribute toward a more secure world by helping build a better one.

Children love to learn and are inherently curious, imaginative and don’t come with predisposed limits on how things have to function. They are built to push limits and we should harness that.

Without a doubt, today’s children are far more technically advanced than in the past – even at the young age of three or four-years-old. To expect them not to do something just because we told them not to – especially when it comes to computers and mobile devices being used to learn, entertain and engage – is simply unrealistic.

The sophistication and capability that today’s kids have for grasping complex topics is amazing. Furthermore, just because they’re children, we shouldn’t assume they’re unaware and incapable of understanding deep issues like security and privacy.

When it’s related to them in a meaningful way, light bulbs go on.

As such, the new focus is to educate children on how things work and in some cases how to break them, so they have a better understanding of why not to do something – and how to fix things that are broken. This is the true definition of “hacking,” learning and finding creative solutions to big, hairy problems. The reality is that if you don’t understand how adversaries attack and break things, it is generally much more difficult (if not impossible) to defend yourself, detect or fix what is broken.

When it comes to children, we should complement their natural and inherent friendliness towards learning something new. We should tap into their creative ingenuity and turn it into something good. All of this provides us a tremendous opportunity to not give up privacy and security for the sake of convenience by simply changing the way we integrate security versus bolting it on.

That said, it is likewise important to discuss and establish boundaries and constructs around “hacking” to ensure that activities are governed appropriately with respect to legality, morality and ethicality.

These conversations and guidelines do a lot toward short-circuiting the normal knee-jerk reaction of what it means to introduce children to “hacking.” We have “hackathons” to allow communities of interest to come together to solve large social problems. We have companies that focus on “hacking” and “hackers” to develop innovative new platforms and services. “Hacking” isn’t always a bad thing… and in many cases when we think about the culture and approach needed to secure our systems and create resilient, rugged and secure code, hacking is an appropriate word.

Juniper Networks Supports Code.org

This is why at this year’s RSA conference, Juniper Networks announced an extension of its grant to Code.org, which will enable the development of new high school computer science course, intended to be advanced placement (AP), to allow students to learn about cyber security and secure coding. Code.org is a non-profit dedicated to expanding participation in computer science by making it available in more schools, because every student in every school should have the opportunity to learn computer science. It is committed to the notion that computer science and computer programming should be part of the core curriculum in education, alongside other science, technology, engineering, and mathematics (STEM) courses, such as biology, physics, chemistry and algebra.

Juniper and the Juniper Networks Foundation Fund are proud to make this commitment and expect it to be the first step in making cyber security a fundamental element of learning to code and learning to code securely. We’re starting at the high school level and hope to spur activity by other businesses and organizations to partner with Code.org and ultimately, develop programs that include kids at the middle school and elementary school level as well.

Your support and help doesn’t have to come via financial grants or funding (although that definitely helps). Your time is just as valuable. If you’re able and interested, volunteer to teach at your kids’ school or help Code.org recruit new teachers in your community to teach Code.org’s courses.

Join Juniper Networks, the Juniper Networks Foundation Fund, and Code.org at www.code.org and help develop and establish the true “next generation” of cyber security.

Mike Mimoso over at ThreatPost did an awesome write-up which you an find here.  Bank InfoSecurity also did a nice interview with me on this and other topics.

Thanks for reading…and for finding your way of contributing.

/Hoff (AKA @Beaker)

*With apologies to W.C. Fields (and H/T to my old friend Bob Antia)

If Black Hat and DEF CON are Security Summer Camp, then RSA is the Spring Dance.

I think this is my 16th RSA Conference.  I believe I am now eligible for the AARP discount shortly.

Beyond the massive amount of customer, partner, analyst and press stuff I have to do this year, I have some stuff to talk about:

Attribution is hard.  It’s as much art as it is science.  It’s also very misunderstood.

So, as part of my public service initiative, I created and then unintentionally crowdsourced the most definitive collection of reality-based constructs reflecting the current state of this term of art.

Here you go:

• Faptribution => The process of trying to reach PR climax on naming an adversary before anyone else does
• Pattribution => The art of self-congratulatory back patting that goes along with attributing an actor(s) to a specific campaign or breach.
• Flacktribution => The process of dedicating your next press release to the concept that, had the victim only used $our_software, none of this would have happened. (Per Nick Selby)
• Maptribution => when you really just have no fucking idea and play “pin the tail on the donkey” with a world map. (Per Sam Johnston)
• Craptribution => The collective negative social media and PR feedback associated with Snaptribution (Per Gunter Ollmann)
• Masturbution => When you feel awesome about it, but nobody else gives a flying f$ck (Per Paul Stamp, but ‘betterized’ by me)
• Snaptribution => naming the threat actor so quickly you can’t possibly be right but you are first. Also known as premature faptribution. (Chris Wysopal)

May you go forth with the confidence to assess the quality, scope and impact of any attribution using these more specific definitions.

/Hoff

At the 2015 Kaspersky Security Analyst Summit, I kicked off the event with a keynote titled: “Active Defense and the A.R.T. of W.A.R.”

The A.R.T. of W.A.R. stands for “Active Response Techniques of Weaponization and Resilience.”

You can read about some of what I discussed here.  I will post the presentation shortly and Kaspersky will release the video also.  The video of my talk is here (I am walking out, hoodie up, like I’m in a fight per the show thematic):

While thematically I used the evolution of threat actors, defensive security practices, operations and technology against the backdrop of the evolution of modern mixed martial arts (the theme of the conference,) the main point was really the following:

If we now face threat actors who have access to the TTPs of nation states, but themselves are not, and they are attacking enterprises who do not/cannot utilize these TTPs, and our only current “best practices” references against said actors are framed within the context of “cyberwar,” and only able to be acted upon by representatives of a nation state, it will be impossible for anyone outside of that circle to actively defend our interests, intellectual property and business with an appropriate and contextualized framing of the use of force.

It is extremely easy to take what I just mentioned above and start picking it apart without the very context to which I referenced.

The notion of “Active Defense” is shrouded in interpretive nuance — and usually immediately escalates to the most extreme use case of “hacking back” or “counter-hacking.”  As I laid out in the talk — leaning heavily on the work of Dave Dittrich in this area — there are levels of intrusion as well as levels of response, and the Rubik’s Cube of choices allows for ways or responding that includes more than filing a breach report and re-imaging endpoints.

While the notion of “active” and “passive” are loaded terms without context, I think it’s important that we — as the technical community — be allowed to specifically map those combinations of intrusion and response and propose methodologies against which air cover of legal frameworks and sovereignty can be laid.  Not having this conversation is unacceptable.

Likewise unacceptable is the disingenuous representation that organizations (in the private sector) who specialize in one of the most important areas of discussion here — attribution — magically find all their information by accident on Pastebin.  Intelligence — threat, signals, human, etc. — is a very specialized and delicate practice, but as it stands today, there 4-5 companies who operate in this space with ties to the public sector/DoD/IC and are locked in their own “arms race” to be the first to attribute a name, logo and theme song around every attack.

It’s fair to suggest they operate in spaces along to continuum that others do not.  But these are things we really don’t talk about because it exists in the grey fringe.

Much of that information and sources are proprietary and while we see executive orders and governmental offices being spun up to exchange “threat intelligence,” the reality is that even if we nail attribution, there’s nothing most of us can do about it…and I mean that technologically and operationally.

We have documents such as the Tallin Manual and the Army Cyber Command Field Manual for Electromagnetic Warfare that govern these discussion in their realms — yet in the Enterprise space, we have only things like the CFAA.

This conversation needs to move forward.  It’s difficult, it’s hairy and it’s going to take a concerted effort…but it needs a light shone upon it.

/Hoff

Over the last couple of years, we’ve seen some transformative innovation erupt in networking.

In no particular order OR completeness:

• CLOS architectures and protocols are evolving
• the debate over Ethernet and IP fabrics is driving toward the outcome that we need both
• x86 is finding a home in networking at increasing levels of throughput thanks to things like DPDK and optimized IP stacks
• merchant silicon, FPGA and ASICs are seeing increased investment as the speeds/feeds move from 10 > 40 > 100 Gb/s per NIC
• programmable abstraction and the operational models to support it has been proven at scale
• virtualization and virtualized services are now common place architectural primitives in discussions for NG networking
• Open Source is huge in both orchestration as well as service delivery
• Entirely new network operating systems like that of Cumulus have emerged to challenge incumbents
• SDN, NFV and overlays are starting to see production at-scale adoption beyond PoCs
• automation is starting to take root for everything from provisioning to orchestration to dynamic service insertion and traffic steering

Stir in the profound scale-out requirements of mega-scale web/cloud providers and the creation and adoption of Open Compute Platform compliant network, storage and compute platforms, and there’s a real revolution going on:

The Open Compute Networking Project is creating a set of technologies that are disaggregated and fully open, allowing for rapid innovation in the network space. We aim to facilitate the development of network hardware and software – together with trusted project validation and testing – in a truly open and collaborative community environment.

We’re bringing to networking the guiding principles that OCP has brought to servers & storage, so that we can give end users the ability to forgo traditional closed and proprietary network switches – in favor of a fully open network technology stack. Our initial goal is to develop a top-of-rack (leaf) switch, while future plans target spine switches and other hardware and software solutions in the space.

Now, interestingly, while there are fundamental shifts occurring in the approach to and operations of security — the majority of investment in which is still network-centric — as an industry, we are still used to buying our security solutions as closed appliances or chassis form-factors from vendors with integrated hardware and software.

While vendors offer virtualized versions of their hardware solutions as virtual appliances that can also run on bare metal, they generally have not enjoyed widespread adoption because of the operational challenges involved with the operationally-siloed challenges involved in distinguishing the distribution of security as a service layer across dedicated appliances or across compute fabrics as an overlay.

But let’s just agree that outside of security, software is eating the world…and that at some point, the voracious appetite of developers and consumers will need to be sated as it relates to security.

Much of the value (up to certain watermark levels of performance and latency) of security solutions is delivered via software which when coupled with readily-available hardware platforms such as x86 with programmable merchant silicon, can provide some very interesting and exciting solutions at a much lower cost.

So why then, like what we’ve seen with networking vendors who have released OCP-compliant white-box switching solutions that allow end-users to run whatever software/NOS they desire, have we not seen the same for security?

I think it would be cool to see an OCP white box spec for security and let the security industry innovate on the software to power it.

You?

/Hoff

From the “It can happen to anyone” department…

A couple of days ago, prior to the announcement that hundreds of celebrities’ nudie shots were liberated from their owners and posted to the Web, I customized some Growl notifications on my Mac to provide some additional realtime auditing of some apps I was interested in.  One of the applications I enabled was Dropbox synch messaging so I could monitor some sharing activity.

Ordinarily, these two events would not be related except I was also tracking down a local disk utilization issue that was vexing me as on a day-to-day basis as my local SSD storage would ephemerally increase/decrease by GIGABYTES and I couldn’t figure out why.

So this evening, quite literally as I was reading RSnake’s interesting blog post titled “So your nude selfies were just hacked,” a Growl notification popped up informing me that several new Dropbox files were completing synchronization.

Puzzled because I wasn’t aware of any public shares and/or remote folders I was synching, I checked the Dropbox synch status and saw a number of files that were unfamiliar — and yet the names of the files certainly piqued my interest…they appeared to belong to a very good friend of mine given their titles. o_O

I checked the folder these files were resting in — gigabytes of them — and realized it was a shared folder that I had setup 3 years ago to allow a friend of mine to share a video from one of our infamous Jiu Jitsu smackdown sessions from the RSA Security Conference.  I hadn’t bothered to unshare said folder for years, especially since my cloud storage quota kept increasing while my local storage didn’t.

As I put 1 and 1 together, I realized that for at least a couple of years, Jeremiah (Grossman) had been using this dropbox share folder titled “Dropit” as a repository for file storage, thinking it was HIS!

This is why gigs of storage was appearing/disappearing from my local storage when he added/removed files, but I didn’t  see the synch messages and thus didn’t see the filenames.

I jumped on Twitter and engaged Jer in a DM session (see below) where I was laughing so hard I was crying…he eventually called me and I walked him through what happened.

Once we came to terms of what had happened, how much fun I could have with this, Jer ultimately copied the files off the share and I unshared the Dropbox folder.

We agreed it was important to share this event because like previous issues each of us have had, we’re all about honest disclosure so we (and others) can learn from our mistakes.

The lessons learned?

1. Dropbox doesn’t make it clear whether a folder that’s shared and mounted is yours or someone else’s — they look the same.
2. Ensure you know where your data is synching to!  Services like Dropbox, iCloud, Google Drive, SkyDrive, etc. make it VERY easy to forget where things are actually stored!
3. Check your logs and/or enable things like Growl notifications (on the Mac) to ensure you can see when things are happening
4. Unshare things when you’re done.  Audit these services regularly.
5. Even seasoned security pros can make basic security/privacy mistakes; I shared a folder and didn’t audit it and Jer put stuff in a folder he thought was his.  It wasn’t.
6. Never store nudie pics on a folder you don’t encrypt — and as far as I can tell, Jer didn’t…but I DIDN’T CLICK…HONEST!

Jer and I laughed our asses off, but imagine if this had been confidential information or embarrassing pictures and I wasn’t his friend.

If you use Dropbox or similar services, please pay attention.

I don’t want to see your junk.

/Hoff

P.S. Thanks for being a good sport, Jer.

P.P.S. I about died laughing sending these DMs:

Rich Mogull (Securosis) and I have given  a standing set of talks over the last 5-6 years at the RSA Security Conference that focus on innovation, disruption and ultimately making security practitioners more relevant in the face of all this churn.

We’ve always offered practical peeks of what’s coming and what folks can do to prepare.

This year, we (I should say mostly Rich) built a bunch of Ruby code that leveraged stuff running in Amazon Web Services (and using other Cloud services) to show how security folks with little “coding” capabilities could build and deploy this themselves.

Specifically, this talk was about SecDevOps — using principles that allow for automated and elastic cloud services to do interesting security things that can be leveraged in public and private clouds using Chef and other assorted mechanisms.

I also built a bunch of stuff using the RackSpace Private Cloud stack and Chef, but didn’t have the wherewithal or time to demonstrate it — and doing live demos over a tethered iPad connection to AWS meant that if it sucked, it was Rich’s fault.

You can find the presentation here (it clearly doesn’t include the live demos):

Dueling Banjos – Cloud vs. Enterprise Security: Using Automation and (Sec)DevOps NOW

/Hoff

The insufferable fatigue of imprecise language with respect to “stopping” DDoS attacks caused me to tweet something that my pal @CSOAndy suggested was just as pedantic and wrong as that against which I railed:

I think it's fair to say that you can't "stop" a DDoS attack unless you can dispatch the endpoints used for its bidding. "Weather," perhaps

— Hoff (@Beaker) March 11, 2014

The long and short of Andy’s displeasure with my comment was:

@Beaker in the same way you can’t “stop” a punch by blocking it?

— Andy Ellis (@csoandy) March 11, 2014

to which I responded:

RT @csoandy: @Beaker in the same way you can’t “stop” a punch by blocking it? < You deflect it. You didn't "stop" it OR prevent another…

— Hoff (@Beaker) March 11, 2014

…and then…

@Beaker an attack goes from point A to point B. If you intercept an halt it en route, you have stopped it.

— Andy Ellis (@csoandy) March 11, 2014

My point, ultimately, is that in the context of DDoS mitigation such as offload scrubbing services, unless one renders the attacker(s) from generating traffic, the attack is not “stopped.”  If a scrubbing service redirects traffic and absorbs it, and the attacker continues to send packets, the “attack” continues because the attacker has not been stopped — he/she/they have been redirected.

Now, has the OUTCOME changed?  Absolutely.  Has the intended victim possibly been spared the resultant denial of service?  Quite possibly.  Could there even now possibly be extra “space in the pipe?” Uh huh.

Has the attack “stopped” or ceased?  Nope.  Not until the spice stops flowing.

Nuance?  Pedantry?  Sure.

Wrong?  I don’t think so.

/Hoff

During the 2014 RSA Conference, I participated on a repeating panel with Bret Hartman, CTO of Cisco’s Security Business Unit and Martin Brown from BT.  The first day was moderated by Jon Olstik while the second day, the three of us were left to, um, self-moderate.

It occurred to me that during our very lively (and packed) second day wherein the audience was extremely interactive,  I should boost the challenge I made to the audience on day one by offering a little monetary encouragement in answering a question.

Since the panel was titled “Network Security Smackdown: Which Technologies Will Survive?,” I offered a $20 kicker to anyone who could come up with a legitimate counter example — give me one “network security” technology that has actually gone away in the last 20 years.

<chirp chirp>

Despite Bret trying to pocket the money and many folks trying valiantly to answer, I still have my twenty bucks.

I’ll leave the conclusion as an exercise for the reader.

/Hoff