Alright Kids…It’s a Security Throughput Math Test! Step Right Up!
I’ve got a little quiz for you. I’ve asked this question 30 times over the last week and received an interesting set of answers. One set of numbers represent "real world" numbers, the other is a set of "marketing" numbers.
Here’s the deal:
Take an appliance of your choice (let’s say a security appliance like an IPS) that has 10 x 1Gb/s Ethernet interfaces.
Connect five of those interfaces interfaces to the test rig that generates traffic and connect the remaining five interfaces to the receiver.
Let’s say that you send 5 Gb/s from the sender (Avalanche in the example above) across interfaces 1-5.
The traffic passes from the MAC’s up the stack and through the appliance under test and then out through interfaces 6-10 where the traffic is received by the receiver (Reflector in the example above.)
So you’ve got 5Gb/s of traffic into the DUT and 5Gb/s of traffic out of the DUT with zero% loss.
You’re question is as follows:
Using whatever math you desire (Cisco or otherwise,) what is the throughput of the traffic going through the DUT?
I ask this question because of the recent sets of claims by certain vendors over the last few weeks. Let’s not get into stacking/manipulating the test traffic patterns — I don’t want to cloud the issue.
{Ed: Let me give you some guidance on the two most widely applicable answers to this question that I have received thus far. 85% of those surveyed said that the answer was 5Gb/s while a smaller minority asserts that it’s 10Gb/s) It comes down to how one measures "aggregate" throughput. Please read comments below regarding measurement specifics.
So, what’s your answer? Please feel free to ‘splain your logic. I will comment with my response once comments show up so as not to color the results.
/Hoff
Do you mean bandwidth or throughput? Bandwidth is bps, throughput is pps, and one must know the packet size in order to calculate it. Furthermore, the pps of a given device of this sort tends to vary greatly depending upon the size of the packets and the type of traffic generated.
In many cases, pps can be more important than bps in determing transmission/reception efficiency.
You've hit on the first of what I had hoped someone would ask…the very definition of what I was asking for. This is half the problem.
When we see folks advertising X Gb/s of a function such as IPS, many of these little details go unanswered. Like how they measured the "throughput" in the first place…
I use these definitions:
Bandwidth is the theoretical maximum amount of (data) capacity that is available through a "conductor."
Throughput is the actual observed amount of data that is measured through said conductor under load.
For the sake of argument here, and to stay on track, let's use the maximum ethernet (non jumbo) frame size of 1500 bytes.
The second thing I was hoping someone would ask is "Do you mean half-duplex or full-duplex measurement…?"
So, for the sake of discussion, given what I've said here, what's your answer?
/Hoff
Chris – we are doing some work on this right now. I asked our engineers and they say that in the way we measure it this would be 5Gbps aggregate. If it were bi-directional, we would say it was 10Gbps. Like the other comments say, there are other factors that go into measuring this for IDS/IPS purposes though. I wrote about this a while back with the ISS 15/6 Gbps announcements.
Your definition of throughput isn't the same as that used by most networking folks, which is pps. I would suggest rephrasing your terminology using standard terminology.
That being said, it's 5gb/sec end-to-end, within the framework of your terminology.
But at what packet sizes and what pps? With what sort of traffic mix? Devices which classify and filter traffic based upon layer-7 inspection tend to exhibit widely varying performance based upon these factors.
So, your original question is really meaningless. The correct way to frame the question is, "What is the performance envelope of this device in bps and pps with varying packet sizes and varying types of traffic?"
Note that some will say that pps and bps don't really apply because the device is doing some sort of layer-7 inspection/classification/manipulation which should be expressed as tps. That's a cop-out – they're trying to avoid giving numbers which they fear won't be received well.
Roland, you're absolutely correct. My question *IS* meaningless and I actually designed it to be!
I hate to sound like this is a setup, but I'm going to directly thank you for taking the bait…for taking the time to draw these questions out from under the bed so that I didn't feel I'm the only nut-job on the planet whose desire for an empiricial definition of testing methodology goes unanswered.
By the way, those definitions didn't come from me, they came verbatim from a vendor with whom I engaged in debate on this very topic.
You've exactly pin-pointed the problem I have with the so-called standardized manner in which performance figures are "measured" and published.
We all know that vendors use marketing numbers and usually don't publish performance metrics to any agreed-upon specs, but I've found this sort of rubbish heating up lately with vendors releasing absolutely ridiculous claims to performance that have no bearing in reality.
I'll clarify all this in a follow-on post.
Again, in all sincerity, thanks very much for playing along.
/Chris
can anyone tell me how can i calculate the throughput of an appliance. I have a vendor who claims his ips has 200 Mbps throughput. And is there a way to verify this.
can i use a sniffer & find out the no. of packets flowing on that segment & do a calculation of sorts!!! just worndering