Rational Survivability

The answer to your question in a word – yes.
Compare IT to cars and networking to the transmission. There are plenty of people who like to drive a manual transmission, but plenty more who just want to stick it into D for Drive and just do that. Behind the scenes, the real magic is done – think how far auto transmissions have come in the last 10 or 20 years. It’s now at the stage where the auto experience is preferred, and even given a veneer of “manualness” via paddles etc.
Yes, the consumption of the network will continue to be dumbed down, but perhaps made better overall – maybe decent security can be woven in to the intelligent fabric that makes up that easy to consume service?
Like the auto industry, there will continue to be extremely talented people building these extremely (mechanically) complex but easy to consume services.

You answered your own question with this: “if one recognizes that ultimately these Public clouds will interconnect to Private clouds, and the two networking models are profoundly differentiated.”

For some orgs, they won’t need networking chops and what is provided to them will be sufficient. For everyone else, they will still need networking skills and knowledge.

With all of the rapid application development tools, programmers are still in demand.

Until networking becomes so automated and fail-proof, networking peeps will be in demand.

Networking expertise will be important for both all providers and large-installation/high-growth customers. When attempting to scale DNS, it becomes increasingly reliant on scaling BGP, and potentially vice-versa. Just like scaling IO and memory. The network must be balanced in order to work right.

This also highly depends on the public cloud customers’ tolerance for latency/jitter, packet loss, and downtime. If you can throw a Fail Whale up and call it a day, then you’re not quite there yet.

I don’t know where this leads, but I am worried that fundamental networking skills are already becoming less common.
I’ll also use an automotive metaphor: an understanding of how points and carburetors work will make you a better mechanic- even if you never work on a car equipped with them. Just as many modern mechanics are lost when the computer won’t tell them what is wrong, I fear we face a similar fate in the realm of the packet ninja.

I’d argue that most sysadmins don’t have much of a clue about networking already and as such there won’t be much change at all on that front for the typical admin. Cloud admins and networking folks will have some new lines to draw and some new skills to learn, but the day to day sysadmin, not so much.

@ Jack: You could say the same thing about “fundamental skills in” change management, IT service delivery/management, Agile/xP/Scrum development processes, etc.

What is important to remember, though, is how much MORE you are getting paid than those folk as a Security Analyst, while having NO processes, NO SLAs for delivery/management, and almost ZERO metrics relevant to the business

There’s an 80/20 kind of rule here. 80% of apps and developers (especially green field apps) don’t need to care about the network using the “flat” EC2 model. The other 20% are the problem legacy apps and specialized tools that will use VPC or private clouds. There are also better tools on the way (e.g. from boundary.com) that will give people more insight into their networks. Most developers using PaaS see a network model made up of HTTP end points. They need to understand the difference between a connection failure and a request timeout, that’s about it.

Networks aren’t going away any time soon. If anything, with the different *aas providers in the mix, they seem to be getting more complicated. I think what we’re going to see is the same problems as before, but with the control boundaries completely redefined. Black box networking like we see today in the cloud will give way to a range of alternatives provided either directly by the provider (i.e. EC2 VPC) or third parties. I predict that a year from now things will look very different. I don’t think they’re going to be Network Stupid, just the modern version of the ’95 era ‘webmaster’. That guy knew everything (what little there really was to know), but overtime complexity required specialization in all the familiar parts of IT, storage, apps, security, networks, etc.

Already, network teams are populated by folks who have limited grasp on fundamental concepts like ephemeral source ports. I’ve encountered self-described network engineers who “manage switching & routing”, yet looking at a tcpdump is like staring onto the matrix.

As these services become more prevalent, there will be more who claim knowledge and titles without basis. However, the need for highly-skilled engineers architecting, implementing and operating those network infrastures will grow. The individuals who fill those jobs will need better skills than most to deal with issues of scale, multi-tenancy, and true high availability.

As a result, a lot of unskilled engineers who consume those network services will increasingly push their problems off on providers and their teams to solve. The issue here is that there will be a lot of wild goose chases.

Driving this harder will be economic conditions that are pushing people into IT careers who wouldn’t have otherwise considered a tech career. There will be droves without passion and aptitude for the craft pursuing only a paycheck.

I think we are already there if my attempts to hire qualified people for Reflex is any indicator. 😉

There seems to be a growing knowledge gap in IT.. Too many generalists out there that are losing the core fundamental knowledge of how things work. Without that core knowledge its very difficult to adapt to new deployment models or abstractions like SDL.

If you think about what some big vendors are doing by declaring everything below the application happens automagically, then we are really in trouble. Its PaaS, running on IaaS which means your aaS is screwed when the magic doesn’t happen and you dont understand why.

I’m already seeing a gap with self-described “network engineers” with whom I regularly interact. Fundamental concepts like ephemeral source ports and TCP window sizes draw blank stares. For many, using Wireshark is like staring into the matrix.

As network complexities are further pushed into some sort of managed service model, we’ll see many who don’t know anything about HTTP besides that “it’s on port 80”. When they encounter problems, they’ll push the troubleshooting effort “to the cloud” (or MSP). The hope being, that the folks who designed and built this SDN for scalability, availability, and multi-tenancy will be able to solve their problems. The knock-on effect is that there will be a great many wild goose chases for the more experienced curators of the SDN.

The economy will increasingly drive unemployed folks into tech careers who might not otherwise have pursued one. Not necessarily bad, though it’s easy to anticipate many non-technologists (lacking passion and aptitude) chasing a steady paycheck.

I have to agree with this article. We migrated from a traditional data center to the public cloud two years ago, and the need for advanced networking skills has diminished within the organization. While there is a plus side to this since we can focus on the product and revenue producing projects, I feel ignorance can lead to an over-dependence on the cloud provider.

We have been in a few situations where networking skills were necessary for troubleshooting advanced problems with our cloud provider, and the only way we got them to listen was because they got the feeling we knew what were were talking about. Also, we were successful in deploying an advanced load balancer solution in the public cloud due to past experiences. I imagine the project would’ve been exponentially more difficult if we were green to advanced networking techniques.

@@reflex_mike

I’m bothered by the stigma attached to “generalist”, probably because I consider myself one. An IT generalist should have sound fundamentals in most areas, and the ability to provide perspective. Now, “generalist” is being used more and more to describe someone with no discernible skills.

And in the onward rush toward increasing granularity of specialty IT disciplines, we lose generalists who are fundamentally sound. We lose individuals who have truly multi-disciplinary skillsets, and can enable the hapless sysadmin to work a problem with the network engineering team. It’s not object-oriented model when it comes to technology teams, where engineers have some published API and everyone floats along happily. But, ultimately, as we push more responsibilities (read: complexities) into some MSP’s SDN/IaaS/whatever offering, we similarly abstract the talent of the individuals managing those systems.

Maybe the sum of what I’m driving at is that we may end up missing the “glue” in terms of talent.

@bamchenry :
@@reflex_mike
I’m bothered by the stigma attached to “generalist”, probably because I consider myself one. An IT generalist should have sound fundamentals in most areas, and the ability to provide perspective. Now, “generalist” is being used more and more to describe someone with no discernible skills.

I understand your point. I was going more towards skills becoming overly general. I think the contradictory position can also be true, its just as dangerous to be overly specialized.

What we are seeing is lots of change in IT. Areas where operational and skill silos could be successful can not be sustained in a the virtual/cloudy data center where there is much interdependence of networking, applications, storage, etc.

I prefer teams with a base level of IT knowledge and then there can be solid IT generalists that are backed by domain experts. Virtualization admins are a good example of a group that needs to be fairly competent in multiple disciplines.

As someone else has mentioned, this only works with continued education and application of that knowledge. Back to @beaker’s question, over abstraction can mean that those with that knowledge are not under our supervision and we are trusting that the people operating behind the curtain are capable and doing the right things. If we are not familiar with how things should operate, how can we expect to question the provider or even notice when they dont?

@reflex_mike

I think we agree. Intuitively, the offload or movement to cloud-based or more agile, managed infrastructures might lead you to think you need fewer IT people, and then only specialists, and treat the network as mere plumbing. But, anecdotally, it becomes apparent that while you may need fewer on-staff IT people, those remaining must be increasingly multi-disciplinary to realize the efficiencies promised by more cloud/virtualized/-aaS infrastructure.

And to Beaker’s original point: we’re already there. There aren’t enough skilled people in networking on most IT staffs, or at the providers. And the commoditization of these services along with other motivating factors will probably work at increasing that gap, leaving the hard work to a passionate, curious few.

@Andrew MacLachlan
Andrew – we’re not talking about the drivers here are we? I think Hoff was referring to the mechanical side of things – the “mechanics”. In this case, your transmission analogy falls apart because manual transmissions are more simple and transparent than automatic – and the mechanics nowadays have much more to worry about, even though the front-end is much simpler. I think this applies to the cloud. I’ll agree with you that you’re going to get a generation of less knowledgeable talent on the user end of things, but on the back-end where things get significantly more complicated to serve that abstraction, the level of wizardry will only increase.

What I think some of the comments above are referring to is a general dilution of talent technical depth as more IT workers flood the market. This is natural though … when you have a constant influx of new IT workers, you can count on the overall ‘knowledge’ level to take a dip. Over the years employers haven’t been investing heavily in their employees technical aptitude, so the collective talent level keeps falling every year when new entrants show up… sadly I don’t see this trend ceasing any time soon.

Excellent observation and thought provoking questions Hoff.
I think what we are seeing here is the rise of a new networking star .. the software networking guru .. the guy/gal with linux and python chops who expertly understands how the software is using the network, and how to best optimize both both ends. This person doesn’t get too concerned with anything beyond the top of rack switch.
As always, market forces will provide the balance. Where you have a “shortage” of certain networking skills you will see skyrocketing salaries — which gets people more interested in learning the trade.

Cheers,
Brad

@Wh1t3Rabbit

Sure, the talent/knowledge will dip as more “newbies” enter the field. But I think what Hoff was driving at is: Are cloud/SDN/IaaS types of services enabling new and existing entrants to the technology field to get by with lesser L2-4 skillsets? And are they doing their organizations a disservice by not honing the skills more?

This is true for any technology, (/me gets on rocker on front porch) kids these days are learning how to use tablets and download stuff before they’re old enough to play video games, but they’re not learning the backend, or programming in the way I did. Additionally setting up kids with their own VM is a fun/easy way for them to get started in a sandbox environment, but without going through the knocks of putting together a computer from scratch they’re missing out on the bigger picture; how all the pieces fit together and interact. Perhaps this will lead to a more fragmented IT pool, backend folks that really know the backend, and UI folks that don’t (and don’t want to know) anything about it.

As for the push button ‘networks’ of AWS, I can see a far more perilous security problem as public and private clouds are mixed, and inexperienced admins just want to ‘make it work’ for their users.

Yes and no…think of memory management in programming languages like C and C++ compared to more recent languages like Java. Manually managing memory in C and even in C++ was “replaced” with automatic memory management in Java. Java was to fix the problems of releasing memory too soon or leaking memory with unused objects. And, there is a whole generation of developers that know nothing about memory management. But the best developers, even those who have only developed in Java, have a deep understanding of how memory is allocated, deallocated and managed. This knowledge creates better solutions: stable, scalable, faster, efficient.

Similarly, I expect an entire bread of Data Center architects that can function perfectly well with limited network knowledge. But the best will have a deep understanding of how it works, why it works the way it does and the ability to control (secure) it.

@@bamchenry I don’t have direct experience with any organization that is ‘100% purely in the public cloud’ so I can’t speak to that. I have lots of experiences with organizations that are hybrid, or some variant and I can tell you they still have some very sharp network folks on staff. Managing complex relationships between private public clouds isn’t as simple as mouse clicks – to not cause a train wreck requires serious knowledge as of today … I don’t personally believe that ‘public cloud’ will be the death of IT networking high-grade talent, no.

My 2 cents. It’s a wedge, and over time the wedge will extend and more people will be able to utilize stuff (stuff being technology, however it looks) without core skills, be they networking or anything else. That’s what democratization and consumerization is all about.

How many exchange admins are there today and how many will there be in one year, two years, five years time. Specific technical skill requirements will reduce at an aggregate level. Yes shit is still complex and yes some folks still need to understand it (to your example, there will be a bunch of players who specifically help orgs solve the difficulties of public/private networking) but should non-core technical skills be required at any sort of mass market level? Hell no…. leave it to those who do that stuff as their core business

ymmv

@Andrew MacLachlan
Yes. Look how far automatic transmissions have come in the last 10 years. They are –almost– as gas efficient as manual, and –almost– as fast. Sounds just like cloud thinking, use this XEN is “almost” as fast, emulating the network card is “almost” as good. Do you want your enterprise to work like a civic or like a nascar? Ia m a a nascar guy myself 🙂

First I would state that the history of technology and civilization itself is the gradual abstraction of low-level technologies to make way for increasingly specialized ones. Simple tools make finer tools, make finer tools, etc. We have long since passed the point where even the brightest human beings can understand the sum of human knowledge in any given field and all its dependencies. Ironically it all works great until some skill goes extinct in the working population before it goes extinct in the market (I’ve worked on systems and languages nearly twice my age made by people who are either dead or retired and never wrote anything down because they never expected their systems to survive as long as they did). I think the baby-boomer demographic problem is going to make things very interesting for a lot of organizations because of this phenomenon.

Ironically security is one place where you can’t easily generalize because a sufficiently skilled, resourceful and motivated adversary will drill down as far as he needs to. Hang out with software reverse engineers, forensics and assembly language people sometime, people with obsolete skills in most areas of computing with the notable exception of security (exploit development and reverse engineering) thanks to reliable and proven layers of abstraction (computer languages). The bad guys have it so the good guys have to, despite the fact that most of the population are barely aware it exists.

It is interesting reading the posts above as the perspective is clear.

I take issue with the sysadmin comment above. While my scope may be more limited than the commenter, I can vouch for a good number that speak and develop network architectures in full integration with network personnel. Be careful of such generalizations.

To the point, as virtualized/cloud environments mature, the core networking stack will necessarily reduce. Software networking (SDN) and abstracture will commoditize once complex tasks. I see core networking heading toward a simplified network bus. However, this is not the case for WAN components.

Commodity configuration items (VLAN’s, Subnets, Firewall Policy, Routes, etc.) should be fairly programmatic and defined through simplified interfaces (and they are within cloud environments). I do see growth in layer 2 spans, VPC, GSLB, and similar components.

The consumer will become less technical – a byproduct of automation. To continue the car analogy, the first drivers of vehicles had to be physically fit, trained, and somewhat a personal mechanic. Through the years, we have reached a point where any “slob” can “grab the keys and go”. The complexity of the vehicle is greater than ever, the engineers more specialized, and the service offerings and variety vastly more broad. Comparitively, the vehicle is more expensive, the mechanic more dependent (and prone to part replacement versus troubleshooting), and the variety confusing the generalized consumer.

@Edward Capriolo

The counter point is that who pays for each? We would all be happy own/use a hyper performance vehicle if someone else provides the capital and operational funds. The key is to develop the proper mix to satisfy passion and logic. Good enough is just that, good enough. Take your NASCAR vehicle home and operate it as a daily driver. Feed it gas, replace its parts, and maintain its operation on your dime. Realize that you can only go 70 (cough, without police intervention). Realize that you get 8 miles to the gallon. Realize that each tire costs $450 dollars… This is similar to hosting providers and UCS. Now, bring home minivan (the horror). Realize that it gets to 70 fast enough. Realize that you can haul the whole family. Realize that you get 26 MPG. Realize that it is ugly and boring.

Now both examples are on opposite ends of the scale to provide the comparison. Returning to reality, you identify your requirements (2 adults, 2 children, budget, driving patterns, etc.), your desires (decent looks, satelite radio, handling), and your opportunities (RX-8, Infinit M, etc.). This drives your acquisition of the appropriate solution.

In regard to networking, I commonly see touted capabilities and maximums, yet when interviewed post deployment, a percentage go unused. My “intelligent” clientelle are beginning to realize it is less about the product and more about the requirement.

So, will you buy a NASCAR when only a Mustang is required?

The bigger problem

There’s already a lack of networking skills in IT. Try to find a web developer who can explain in depth both HTTP and SSL/TLS, much less TCP or IPv6 vs v4. They’ve got too much to worry about up at layer 7 to even bother looking too far down the stack.

Cloud platforms are simply allowing the ignorant to remain ignorant – but this isn’t a bad thing. There will always be a need for networking ninjas, just as there will always be need for web application development gurus. Different disciplines.

Companies love the cloud because it removes the need for the web guys to have to talk to the networking guys, and actually agree on an architecture. Which, if you’ve ever sat in those meetings, is like watching a debate on C-SPAN. Lots of punditry, very little progress.

Not too worried about it. Hardly anyone programs in Assembly language anymore and yet we seem to be doing just fine.

An analogy I would suggest would be large passenger aircrafts and the operational transformation they went through from the introduction of the Boeing 707 in the 70s to todays huge flying machines.On top of material, mechanical and other aviation engineering advancements, modern computer systems, fly-by-wire, systems automation, auto-pilot, flight control evolution, etc. brought us from a piloting/operating cabin crew of 3 (pilot, co-pilot, and flight engineer) on a Boeing 707 carrying 140 passengers in 1975, to just two cabin crew members on a 747-400 carrying 660 passengers and on an Airbus A380-800 carrying 853 passengers.

Guess who was cut out…

Wikipedia describes it like this “The advent of computer technology, reliable software, and a desire by commercial airlines to cut costs by reducing flight deck crew have eliminated the requirement for Flight Engineers on modern airliners. The same general logic has led to the removal of the Flight Engineer position in many modern military aircraft.”

You get the point.

There will always be a need for engineers with deep knowledge of the underlying structures and the ability to design systems and take care of smooth operations. In fact it is likely there will be a growing need for such experts. At the same time there will be reduced ‘cabin crews’ flying the plane or sailing the ships. And this is applicable to commercial as well as government and military operations.

The network ops silo is gone, no-one needs to know how to use switch operating systems unless they work in the sausage factory, but higher level protocols like DNS and TCP still matter. There is a need for people who can debug packet loss. This didn’t change between Datacenter and Cloud, although the need to talk to the network ops folks does mostly go away with cloud.