For quite some time we have a new buzz word on the market – programmable Ethernet silicon that makes your network so great that it cannot be ignored.
Usually, it’s setoff between so-called fixed-function ASICs and a vendor product that will cause a tectonic shift in networking.
Programmable Ethernet silicon does not rely on backed-in protocols, it can be configured by the software on the control plane. There is a subtype called runtime programmability – updating the SW while the switch is in operation, with no downtime.
The result is a dramatic reduction in operational complexity, higher network flexibility, and higher network availability (c) Broadcom
TL:DR version – true programmability is a niche.
We have a vendor claiming the world’s first end-user programmable Ethernet switch right now (won’t point fingers, but it’s Barefoot).
Is this statement true?
No.
Without looking too far back in time – just during this decade it was Fulcrum (founded in 2000 and acquired by Intel in 2011, long dead by now) with its Alta family.
Quoting:
The Intel® Ethernet FM6000 series switch provides unprecedented parsing flexibility to support the ever-growing portfolio of networking protocols, making the switch an ideal building block for administrators that want to define and introduce their own protocols and standards.
Then Xpliant (founded in 2011 and acquired by Cavium, later Marvell, already discontinued)
Quoting:
The XPliant® CNX880XX family of Ethernet switches provide unprecedented flexibility in protocol processing without compromising speed. XPliant Packet Architecture (XPA™) allows programming of every element of switch packet processing.
Barefoot happened only in 2013 (ironically, also acquired by Intel just recently).
Quoting:
Tofino is protocol independent because the chip has no awareness of the network protocols it supports. Instead, the P4 program provides the logic for handling all supported protocols. When support for a new protocol is required, the network operator or switch manufacturer simply adds new logic to the P4 program.
Innovium (since 2014)
Quoting:
Innovium delivers the world’s highest performance switch silicon with large buffers, unmatched analytics through fine-grain telemetry, line-rate programmability …
Nephos (a subsidiary of MediaTek, founded in 2016. The real activity has begun much earlier.)
Quoting:
Wire-speed programmable deep packet classification, flexible and recursive match/result-actions, packet generation/modification, fungible match and result memory including real-time instrumentation, packet diagnostics, and state updates
Broadcom couldn’t stay aside from the party and launched the Trident 3:
protecting customers’ switch investments with smart, data plane programmability, Broadcom introduces the Trident 3, its new 3.2Tbps, 25G SerDes fully programmable switch.
Now Broadcom already has the Trident 4 that is even more flexible, programmable, faster, feature-rich, etc.:
All these statements are just about the same. Our ASIC is not very ASIC (a fixed-function IC), and a user can define how does it work.
Further on, it’s easier to rely on quotes by Barefoot – they make more fuss than all other vendors combined.
Tofino delivers programmability with no compromise on performance. (c)
No.
What are the performance metrics for a switch ASIC? Switching bandwidth, latency, and packets per second throughput.
Here is a comparison (all from public sources) for the same-generation 100G products:
| ASIC | Switching capacity | Latency | Packet performance |
|---|---|---|---|
Tomahawk/TH+ | 3.2Tbps | sub-400/450 ns | 3.3Bpps |
Trident 3 | 3.2Tbps | 800 ns | 2Bpps |
Tofino (T10) | 3.2Tbps | from 800 ns | 2.4Bpps |
Xpliant XP80 | 3.2Tbps | 2 usec | 1.2Bpps |
As we can see, there is a price to pay.
Tomahawk/Tomahawk+ are “lightweight” chipsets and latency/Pps champions.
More flexible chipsets have much higher latency and lower packet performance.
Trident 3 FlexXGS programmability is pretty good and closely matched with T10. It’s performance figures too. The miracles division is on permanent vacations, sorry.
Having some insights into the custom SW development, customers are looking for a solution to their needs. It does not matter if the chipset is programmable or not. But they do care about latency.
When Marvell canceled the Xpliant product line, they made a very interesting statement.
Eric Hayes, SVP and general manager of networking at Marvell:
“We found at Xpliant that programmability offers value in niche applications.” As far as niches, he cited network monitoring, packet brokering, load balancing, and prototyping of new technologies.
Considering real-world scenarios – small to medium scale companies will never bother with looking into protocol stacks, writing a new stack of their own to define their network behavior is even less reasonable a task.
The next thing is NOS. For most cases, it won’t use all the chipset features, and there will be only a few SW options that may not run on all supported platforms.
For example, Tomahawk (the usual victim in such comparisons). How fixed is it?
In real life, it’s quite flexible. Does it rely on some “backed-in” forwarding logic? No, it has a set of operations it can do to packets. A developer can implement protocols and logic by writing applications in C language.
For example, it is absolutely possible to build a network monitoring product that will fit 5G telecom needs. Not the fanciest of it, but the biggest challenge will be lack of ASIC resources, not its features. Though for advanced designs some features will be missing too.
That’s where you need more flexible products.
Let’s have a look at the latest generation of 400G Ethernet. The picture is quite interesting.
Quoting:
Tofino 2 and Tofino also remain the world’s only fully programmable switching chips.
At the same time:
That moment things become confused. How can fully programmable switching chipsets be divided into families with different features? It seems that they are not so programmable in real life.
This is the most interesting part. If you just listen to the marketing bullshit, everyone talks big about openness, ease of use, etc.
In truth, among all vendors we mentioned at the beginning of this article, Intel with Alta was the only one open vendor as they allowed distribution of the SDK and guides on the “break the seal” terms.
Others will go through a vetting process before delivering the toolchain.
There is a semi-public OpenNSL option from Broadcom, it depends on your switch HW vendor to deliver. Obvious problems – limited chipset support, not all SDK features are there.
In any case, an ASIC won’t do anything without an app on the control plane telling it what to do. An app builds on using the vendor API.
Now, we are back to Barefoot and its creation, the P4 language (Programming Protocol-Independent Packet Processors).
See that “Your Auto-generated API” line?
A P4 program does not run directly on an ASIC. It goes through a compilation process using Barefoot compiler and dev tools. Try to guess how public they are.
In the end, it’s an API that truly runs the switch. It is vendor-provided and never been open. All attempts in the creation of an open switch API (like SAI) only provide a layer accessible for the public with an underlying vendor blob.
Furthermore, it’s an interesting question – are all the P4 features supported in the hardware? Because only FPGAs are truly reconfigurable, but at the cost of performance and latency. Or check NPUs.
When it comes to running P4 programs on SAI it clear enough because the ASIC capacity is declared by SAI.
But what if it is not? You will find it out later when your program did not work. Surprise.
Another interesting problem addressed by P4Runtime is the incompatibility of applications. If the way you process and change packets are different from common standards, your programmability ends within your network. Even if you do – you cannot be sure that your program survives a new HW platform.
Doesn’t seem like a universal answer to all questions.
Broadcom couldn’t stay aside from the push of an open high-level language striving to become industry standard and joined the struggle with not one, but two options:
Software Development Kit Logical Table (SDKLT)
NPL – Network Programming Language
The NPL is an interesting language that can be an alternative to P4, but the devil is in the details as always. No public document describes how to run a program on real hardware.
Still, programmable Ethernet silicon is a niche product. There are NPB, forensic applications, network monitoring things that benefit from flexible silicone, and that was working long before the new buzzword era.
Thanks to the competition, more vendors will emphasize how programmable their products are. Even if they already were.
Programmability has a cost in latency and packet performance. A fair tradeoff.
There will be an interesting clash between P4 and NPL.
Openness and programmability are not very open and programmable. Especially for a typical end-user.
One cannot just come and write a product using P4 from scratch without a strong background. Even teams with solid developing background struggle to deliver something that works.
The majority of network devices will not rely on programmability. Good old stacks and updates from time to time are more than enough for the real world.
Runtime programmability is not a very big thing – a soft reboot after an update adds nigh to the troubleshooting time and efforts.
To pay with PayPal, select PayPal as your payment method at Checkout. You will be redirected to the PayPal payment page, where you can log in with your PayPal username and password and confirm your payment. This method also allows payments without a PayPal account. You can enter your credit card information and pay safely via PayPal.
Our PayPal account is paypal@netbergtw.com.
When placing the order, select Bank Transfer on the Checkout page, and you will see our bank account information.
After you pay with a bank transfer, please send the bank slip to your account manager for tracking.
| Bank: | E.Sun Commercial Bank, Ltd., Taipei, Taiwan |
| Bank Address: | No. 66-1, Sanchong Rd. Nangang District, 115, Taipei. |
| Account name: | Netberg Ltd. |
| Account number: | 1182441011646 |
| SWIFT: | ESUNTWTP |
