At least you can still make use of that hardware, many companies should take note of this. You can make hardware and software that doesn't die once someone pulls the plug on the other side.
Still, of course it would've been better to have released it sooner.
It’s complicated. The NUCs had a lot of inevitable catastrophic hardware failures, like NIC ports that would break. Their problem is they were not good.
100% of the Broadwell NUCs, AFAIK, eventually have the problem, but many were trashed before they did. These issues persisted for years, such as with 10th and 12th generation NUCs. Search "proxmox nuc NIC not working".
Broadwell is gonna broadwell. I won't ever trust their NICs ever again. They've been way too much pain with them both personally and professionally.
Their firmware is crap quality, and their bugs are just absolutely astoundingly bad.
At one job we literally had to fix their firmware for them after several months of back-and-forth, engineers spotted the absurdly obvious bug in minutes of seeing their code.
NUCs are shifting completely to ASUS who is going to continue working on them and there are some long term commitments (there are industrial variants for example)
They spoke on their podcast (I think it was there?) about ditching Tofino for the next generation of the Oxide computer. So it sounds like the current model will always ship with Tofino, but due to no future product development they won't use it again in a new machine. It sounded like they had just secured a replacement for the future but I can't remember who it was.
We've talked about it a bunch, most recently when talking about Intel after Gelsinger.[0] I went into more detail on Intel's total mishandling of Tofino in my blog entry describing why Gelsinger was the wrong choice to lead Intel in 2021.[1]
As you might imagine, this move from Intel is something that we at Oxide have advocated for strenuously -- and it is a tremendous tribute to the former Tofino team at Intel that this got done. As I hope I made clear in my blog entry: the folks working on Tofino at Intel have been great to work with; they deserved much better than their (former) executive leadership.
We definitely advocated for it (at all stages of our interaction with Tofino -- but especially strenuously after the part was killed), and it does have impact on us in that it was one of the few parts of our stack that we couldn't open (and now we can -- stay tuned there). We remain big believers in P4 and want to see P4 compilers for other switching silicon, so getting an open is a big step in that direction. I would also say that what Intel did here beat our expectations, and includes many aspects of their software that we didn't think they would make available. So even though we will be moving beyond Tofino, getting their software open source is great and we're very supportive -- and if anything, it just sharpens for us what a poor strategic decision it was to kill Tofino.
P4 has more or less gone nowhere. Tofino was a full generation behind and didn’t make sense. P4 was compelling because people thought they’d solve the Elephant flow problem with traffic engineering in P4 but the resources to actually do this at scale never materialized for many reasons.
I’m impressed by the political acumen it takes to get a Corp to release code as OSS. My career has seen at least two chunks of work that would have made great OSS (ie potentially useful outside of the single company) but Incoukd not get past the final hurdle
And they would have been nice CV boosters as well (my real motivation!)
Well. It looks like abandonning software to the community.
From the "P4 workflow" described at https://p4.org/ I see mentions of compiling to x86, but no mention of ARM, and no mention of BPF. So, as someone who discover it, I wonder if this project is still relevant in 2025.
I'm not familiar with P4 but with some quick digging it seems like the compiler does support eBPF as a target. Additionally, I don't think the compiler outputs x86 directly at all, and instead includes targets like DPDK which supports ARM in addition to x86.
It helps that Intel has been contributing to OSS since forever, they have good internal processes established and some orgs develop/contribute to OSS pretty much exclusively.
I think the biggest contributor to these processes was going open source in their graphics drivers.
They went from "we have tons of 3rd party IP in these!" to, "you don't need to download anything, it's in kernel mainline now" in a generation and they're off to the races after that.
Maybe their Ethernet drivers were open before that, I don't remember but, video drivers made them pass a threshold in maturity IMHO.
Intel embraced Linux when they bet big in datacenters in early 2002. The momentum they got from there, with Linux dominating the sector, made them realize how much of a benefit it is to have a free software stack. With Linux they were able to go head to head with the incumbents of the day.
Linux and x86 became unbeatable in the space for 20 years.
They have known how important it is. They won't forget.
The ethernet ones were big for awhile, I can remember when you wanted to go out of your way to get something that worked with the e1000 driver if you wanted reliable, usable gigabit.
I had one piece that made it open source (with blessing of 20 committees), then someone that didn't like me ran it back up the flag poll and pulled it back next day
As far as I can tell, Software Defined Networking (which this is about: "P4 is a domain-specific language for network devices") by now is pretty much a decade-old promise that never materialized. I'd still love to be wrong though!
So, let's take the next paragraph: "Before P4, vendors had total control over the functionality supported in the network (...) controlled the rollout of new features (e.g., VXLAN), and rollouts took years"
Anyone has a pointer to any actually available hardware capable of L2 and L3 packet processing where I could have implemented VXLAN in, say, weeks using P4? Again, as far as I can tell, it's all either killed-off-a-long-time-ago, "contact us" vaporware, or exotic 40/100-Gb-only Top-o-Rack gear, and even for those, there is nary an "add to cart" button in sight...
P4 is used by Cisco Silicon One, Xsight, AMD Pensando, Intel Mount Evans, etc. "Contact us" isn't the same thing as vaporware; the Pensando SSDK and Intel ES2K SDE definitely exist for example. I realize it sucks when things aren't available to hobbyists but it's a mistake to pretend it's fake.
P4 is really only needed in data center networks because slower campus/home networks can usually get away with software processing and their lower prices probably can't support the R&D of a programmable architecture.
I would be curious as well. Every time we try something "software defined" the drawbacks are major, cost goes up, stability goes down and most importantly, bandwidth goes down by a factor 10. The only software defined networking gear we use is OpenBSD, to do some complex routing, but we cannot go above 5GB/s.
Not much to add technically but I am very curious who named it Tofino. Tofino is a city by where I live. It is a beautiful location on the Pacific Ocean on Vancouver Island. A favorite destination for our PM Justin Trudeau. Definitely one of the top 100 beautiful places in the world. When I go there I feel stress leave my body and just enjoy the sounds of the waves crashing and the feeling of the sand on my feet melt any worries I may have. Love it there.
Nice we are basically neighbors you have to drive past my house to get to Tofino. If you are ever passing by always like making new like minded friends.
> This bold move from Intel to open-source the Tofino P4 software is more than just a licensing decision; it’s a call to action for the global developer community.
Nah. 5 years ago this would've been bold. Now it's ridding yourself of the baggage of an almost-dead platform that you're about to make fully dead.
Still appreciate getting the tooling as FOSS rather than just terminating it, but let's not go for delusions here.
To my knowledge: nothing. Intel is exiting the network switch silicon business¹. Broadcom and nVidia are dominating (different parts of) the market; Marvell and Microchip are fighting for the scraps.
The only "cool" player is Microchip, who have been providing full datasheets, register maps, and open sourcing their drivers for years now. But I'm under no illusions they're doing this out of the goodness of their heart, they're doing it because it's one of very few competetive advantages available to them.
(Which is perfectly fine! FOSS drivers are a great competetive advantage! It's not working super well sadly :/ — but part of the problem here is Broadcom's anticompetetive behavior. To my knowledge, any switch OEM producing Broadcom-based gear will get their NDAs and silicon access revoked if they so much as dream about making devices with non-Broadcom silicon.)
¹ Intel has already exited this business some while ago, they only bought Fulcrum Micro to get better NICs basically since every NIC is nowadays also a switch. Tofino was always a "special beast", not quite competing against e.g. Qumran or Trio. Tofino is (was?) better thought of as special purpose FPGA…
> To my knowledge, any switch OEM producing Broadcom-based gear will get their NDAs and silicon access revoked if they so much as dream about making devices with non-Broadcom silicon.
Cisco Meraki did; their low end switches are Marvell and their "high end" switches (MS420, MS425, MS450, MS350, MS355) were all Broadcom based. Were because about a year ago they announced the End of Sale of all Broadcom based switches.
Everything above the low end stuff is now Cisco Catalyst. (Although one can argue everything from Meraki is low end apart from their prices)
> Marvell and Microchip are fighting for the scraps
Realtek also. Lots of smaller L2 managed switches based on the RTL93xx series. [1]
But I am not seriously comparing Realtek to Tofino, that's like comparing Hot wheels to the actual car.
> To my knowledge, any switch OEM producing Broadcom-based gear will get their NDAs and silicon access revoked if they so much as dream about making devices with non-Broadcom silicon.
Nokia use Broadcom silicon for low-end, in-house for the rest.
Unless there are devices that I could buy (48 port 1U style switches) and until this supports integrating with something like switchdev ( https://docs.kernel.org/networking/switchdev.html ) then this is too little too late. I hope I'm wrong though.
does anyone know if someone can make this switchdev friendly?
There have always been Tofino switches available to buy but they were priced for the data center not the home. You can probably find them on eBay now that they're old enough.
Writing a switchdev driver is an enormous amount of work so it only happens when someone (usually the chip vendor) pays for it. Any code open-sourced by Intel probably won't meet Linux kernel standards so it would have to be completely rewritten (like the tg3 driver from the old days).
After having used & automated configuration of "traditional" switching/routing devices for years (Cisco , Arista, Juniper), I can't wait for P4 devices to take over
You know all those random python automation scripts network engineers use to configure devices? Now imagine them actually running on device controlling packet and frame flows. Does that sound like it would be fun to troubleshoot?
The APIs these vendors provide are a joke. A bunch of functionality can only be accessed via scripting interactive CLI commands. Some API endpoints cause short downtime / unexpected behavior (eg: by deleting the routing table and adding all entries back 1 by 1), while the on-device commands do not.
And guess what, the switch may decide to print informational or environmental messages interleaved with the command output, because the commands were meant to be run by a human. Good luck knowing if your state-altering command succeeded when you receive broken JSON.
I ended up regex-removing known environmental messages from command outputs..
Improvement on this is more likely to come from switchdev than from Tofino & P4. (though these don't necessarily contradict each other)
You can already run plain Debian on a Mellanox Spectrum device, treat it like a Linux software router, and by the power of magic your routes get pushed into hardware. (Source: device on my table to my right :D) Microchip's SparX-5 should be similar though I don't have one of those to test.
The only switching/routing vendor with an API worth a damn was Juniper - in fact they were the only vendor who was applying CLI changes to the box via their own API, the Way It Should Be Done™.
These days you are spoilt for choice (and price point) with the likes of Arista, HPe (AOS-CX), Cisco (NX-OS, IOS-XR) and plenty of others entering the space.
Am I mistaken or has Intel pretty much shelved the Tofino switching hardware that supports P4 in the first place?
I seem to recall Oxide having to switch suppliers over this?
Yes, Intel canceled Tofino two years ago.
I'm not complaining but it's weird that they're open sourcing the SDK now. Maybe it's to support Mount Evans.
Releasing dead software that is used only to support dead hardware?
At least you can still make use of that hardware, many companies should take note of this. You can make hardware and software that doesn't die once someone pulls the plug on the other side.
Still, of course it would've been better to have released it sooner.
I think half of Tofino's complexity was in their compiler. So it may inspire new hardware vendors to reuse it in some contexts.
That's fair enough.
This was the post by Bryan Cantrill of Oxide on the Tofino saga with Intel
https://bcantrill.dtrace.org/2024/12/08/why-gelsinger-was-wr...
I also was told tofino was looking EOL. Like NUC, dropped from some C suites KPI set and longterm roadmap.
I'd love to be wrong, this is just what people said.
However, NUC form factor is more lively than ever before. Maybe someone can grab the language and run for something? Miktrotik guys come into my mind.
It’s complicated. The NUCs had a lot of inevitable catastrophic hardware failures, like NIC ports that would break. Their problem is they were not good.
How is a ethernet port breaking "inevitable"?
100% of the Broadwell NUCs, AFAIK, eventually have the problem, but many were trashed before they did. These issues persisted for years, such as with 10th and 12th generation NUCs. Search "proxmox nuc NIC not working".
Broadwell is gonna broadwell. I won't ever trust their NICs ever again. They've been way too much pain with them both personally and professionally.
Their firmware is crap quality, and their bugs are just absolutely astoundingly bad.
At one job we literally had to fix their firmware for them after several months of back-and-forth, engineers spotted the absurdly obvious bug in minutes of seeing their code.
NUC somewhat avoided the google^Wintel graveyard - they sold at least the branding to Asus.
NUCs are shifting completely to ASUS who is going to continue working on them and there are some long term commitments (there are industrial variants for example)
Apropos:
https://github.com/oxidecomputer/tofino
https://www.infoq.com/presentations/tofino-2/
They didnt switch supplier iirc they are still using Tofino since it is is still a capable hardware and they see it being useful for years to come
They spoke on their podcast (I think it was there?) about ditching Tofino for the next generation of the Oxide computer. So it sounds like the current model will always ship with Tofino, but due to no future product development they won't use it again in a new machine. It sounded like they had just secured a replacement for the future but I can't remember who it was.
We've talked about it a bunch, most recently when talking about Intel after Gelsinger.[0] I went into more detail on Intel's total mishandling of Tofino in my blog entry describing why Gelsinger was the wrong choice to lead Intel in 2021.[1]
As you might imagine, this move from Intel is something that we at Oxide have advocated for strenuously -- and it is a tremendous tribute to the former Tofino team at Intel that this got done. As I hope I made clear in my blog entry: the folks working on Tofino at Intel have been great to work with; they deserved much better than their (former) executive leadership.
[0] https://oxide-and-friends.transistor.fm/episodes/intel-after...
[1] https://bcantrill.dtrace.org/2024/12/08/why-gelsinger-was-wr...
Thanks for the context, your respect for the Tofino team certainly came through in the podcast.
Thanks Brian [0] was a great podcast and got me through a large lawn mowing!
Does this move to open source P4 have any impact on Oxide at all, or is it too little too late?
We definitely advocated for it (at all stages of our interaction with Tofino -- but especially strenuously after the part was killed), and it does have impact on us in that it was one of the few parts of our stack that we couldn't open (and now we can -- stay tuned there). We remain big believers in P4 and want to see P4 compilers for other switching silicon, so getting an open is a big step in that direction. I would also say that what Intel did here beat our expectations, and includes many aspects of their software that we didn't think they would make available. So even though we will be moving beyond Tofino, getting their software open source is great and we're very supportive -- and if anything, it just sharpens for us what a poor strategic decision it was to kill Tofino.
P4 has more or less gone nowhere. Tofino was a full generation behind and didn’t make sense. P4 was compelling because people thought they’d solve the Elephant flow problem with traffic engineering in P4 but the resources to actually do this at scale never materialized for many reasons.
ehh scream SDN 5 times. kinda miss the 2010’s now.
cisco silicon one uses p4 fwiw. internal development though, but the language makes sense for what the things are.
do they expose the p4 functionality externally? ive heard this from them but never actually seen the proof - it seems like vaporware
I don't think they would lie but they just don't release the compiler. Maybe they give it to Meta.
I’m impressed by the political acumen it takes to get a Corp to release code as OSS. My career has seen at least two chunks of work that would have made great OSS (ie potentially useful outside of the single company) but Incoukd not get past the final hurdle
And they would have been nice CV boosters as well (my real motivation!)
Well. It looks like abandonning software to the community.
From the "P4 workflow" described at https://p4.org/ I see mentions of compiling to x86, but no mention of ARM, and no mention of BPF. So, as someone who discover it, I wonder if this project is still relevant in 2025.
I'm not familiar with P4 but with some quick digging it seems like the compiler does support eBPF as a target. Additionally, I don't think the compiler outputs x86 directly at all, and instead includes targets like DPDK which supports ARM in addition to x86.
https://github.com/p4lang/p4c
There's a bunch of network operating systems that use P4 (Arista EOS, Cisco NX-OS, IPinfusion OCNOS, Microsoft SONiC, etc)
Is x86 dead for wire powered devices?
Regardless, OSS is probably the best way to get it onto other architectures.
Basically, now you have to document x86. xD
It helps that Intel has been contributing to OSS since forever, they have good internal processes established and some orgs develop/contribute to OSS pretty much exclusively.
I think the biggest contributor to these processes was going open source in their graphics drivers.
They went from "we have tons of 3rd party IP in these!" to, "you don't need to download anything, it's in kernel mainline now" in a generation and they're off to the races after that.
Maybe their Ethernet drivers were open before that, I don't remember but, video drivers made them pass a threshold in maturity IMHO.
Intel embraced Linux when they bet big in datacenters in early 2002. The momentum they got from there, with Linux dominating the sector, made them realize how much of a benefit it is to have a free software stack. With Linux they were able to go head to head with the incumbents of the day.
Linux and x86 became unbeatable in the space for 20 years.
They have known how important it is. They won't forget.
Yes, however, they first made their hardware standard compliant rather than making their driver open source.
The open source part came later, starting with CPU and chipset support, then Ethernet, then GPUs IIRC.
The biggest and sweetest side-effect is Desktop/Personal use Linux support as long as the hardware doesn't do anything janky, or too janky.
Fair enough
The ethernet ones were big for awhile, I can remember when you wanted to go out of your way to get something that worked with the e1000 driver if you wanted reliable, usable gigabit.
> they have good internal processes established
There are a good number of people that would LOL at this statement, myself included.
Maybe they have such processes now, because at one point . . . Well “mistakes were made”.
I had one piece that made it open source (with blessing of 20 committees), then someone that didn't like me ran it back up the flag poll and pulled it back next day
As far as I can tell, Software Defined Networking (which this is about: "P4 is a domain-specific language for network devices") by now is pretty much a decade-old promise that never materialized. I'd still love to be wrong though!
So, let's take the next paragraph: "Before P4, vendors had total control over the functionality supported in the network (...) controlled the rollout of new features (e.g., VXLAN), and rollouts took years"
Anyone has a pointer to any actually available hardware capable of L2 and L3 packet processing where I could have implemented VXLAN in, say, weeks using P4? Again, as far as I can tell, it's all either killed-off-a-long-time-ago, "contact us" vaporware, or exotic 40/100-Gb-only Top-o-Rack gear, and even for those, there is nary an "add to cart" button in sight...
P4 is used by Cisco Silicon One, Xsight, AMD Pensando, Intel Mount Evans, etc. "Contact us" isn't the same thing as vaporware; the Pensando SSDK and Intel ES2K SDE definitely exist for example. I realize it sucks when things aren't available to hobbyists but it's a mistake to pretend it's fake.
P4 is really only needed in data center networks because slower campus/home networks can usually get away with software processing and their lower prices probably can't support the R&D of a programmable architecture.
I would be curious as well. Every time we try something "software defined" the drawbacks are major, cost goes up, stability goes down and most importantly, bandwidth goes down by a factor 10. The only software defined networking gear we use is OpenBSD, to do some complex routing, but we cannot go above 5GB/s.
Seems to be used in the military radio space.
Not much to add technically but I am very curious who named it Tofino. Tofino is a city by where I live. It is a beautiful location on the Pacific Ocean on Vancouver Island. A favorite destination for our PM Justin Trudeau. Definitely one of the top 100 beautiful places in the world. When I go there I feel stress leave my body and just enjoy the sounds of the waves crashing and the feeling of the sand on my feet melt any worries I may have. Love it there.
The founding CEO of Barefoot Networks was big surfer. They used famous surfing spots as the internal names of their ASICs.
Same here! Im living over in Comox and spend a lot of time in Tofino / Ucluelet. Best part of all of Canada in my opinion.
I particularly enjoy the winter trips to SF to show my colleagues pictures of the latest cold water surf adventure.
Nice we are basically neighbors you have to drive past my house to get to Tofino. If you are ever passing by always like making new like minded friends.
> This bold move from Intel to open-source the Tofino P4 software is more than just a licensing decision; it’s a call to action for the global developer community.
Nah. 5 years ago this would've been bold. Now it's ridding yourself of the baggage of an almost-dead platform that you're about to make fully dead.
Still appreciate getting the tooling as FOSS rather than just terminating it, but let's not go for delusions here.
What are they making instead?
To my knowledge: nothing. Intel is exiting the network switch silicon business¹. Broadcom and nVidia are dominating (different parts of) the market; Marvell and Microchip are fighting for the scraps.
The only "cool" player is Microchip, who have been providing full datasheets, register maps, and open sourcing their drivers for years now. But I'm under no illusions they're doing this out of the goodness of their heart, they're doing it because it's one of very few competetive advantages available to them.
(Which is perfectly fine! FOSS drivers are a great competetive advantage! It's not working super well sadly :/ — but part of the problem here is Broadcom's anticompetetive behavior. To my knowledge, any switch OEM producing Broadcom-based gear will get their NDAs and silicon access revoked if they so much as dream about making devices with non-Broadcom silicon.)
¹ Intel has already exited this business some while ago, they only bought Fulcrum Micro to get better NICs basically since every NIC is nowadays also a switch. Tofino was always a "special beast", not quite competing against e.g. Qumran or Trio. Tofino is (was?) better thought of as special purpose FPGA…
> To my knowledge, any switch OEM producing Broadcom-based gear will get their NDAs and silicon access revoked if they so much as dream about making devices with non-Broadcom silicon.
Cisco Meraki did; their low end switches are Marvell and their "high end" switches (MS420, MS425, MS450, MS350, MS355) were all Broadcom based. Were because about a year ago they announced the End of Sale of all Broadcom based switches.
Everything above the low end stuff is now Cisco Catalyst. (Although one can argue everything from Meraki is low end apart from their prices)
> Marvell and Microchip are fighting for the scraps
Realtek also. Lots of smaller L2 managed switches based on the RTL93xx series. [1]
But I am not seriously comparing Realtek to Tofino, that's like comparing Hot wheels to the actual car.
[1] https://svanheule.net/switches/
"Although one can argue everything from Meraki is low end apart from their prices"
This is hilariously accurate.
AMD ended up entering the market to some extent - big powerful software-modifiable NIC chips that can also serve as switches from my understanding.
But like Tofino, it's mostly stuff that is "behind the curtain"at hyperscalers or deep inside closed box switches
> To my knowledge, any switch OEM producing Broadcom-based gear will get their NDAs and silicon access revoked if they so much as dream about making devices with non-Broadcom silicon.
Nokia use Broadcom silicon for low-end, in-house for the rest.
Hopefully they don't axe their NICs. At least Intel provides really good open datasheets for their e800 cards, can't say the same about other vendors.
The absence of anything >200G isn't reassuring there...
Related P4: open-source programming language for high-performance packet switching (98 points, 2016, 19 comments) https://news.ycombinator.com/item?id=11903478
Unless there are devices that I could buy (48 port 1U style switches) and until this supports integrating with something like switchdev ( https://docs.kernel.org/networking/switchdev.html ) then this is too little too late. I hope I'm wrong though.
does anyone know if someone can make this switchdev friendly?
There have always been Tofino switches available to buy but they were priced for the data center not the home. You can probably find them on eBay now that they're old enough.
Writing a switchdev driver is an enormous amount of work so it only happens when someone (usually the chip vendor) pays for it. Any code open-sourced by Intel probably won't meet Linux kernel standards so it would have to be completely rewritten (like the tg3 driver from the old days).
After having used & automated configuration of "traditional" switching/routing devices for years (Cisco , Arista, Juniper), I can't wait for P4 devices to take over
I believe that Arista has (had?) P4 devices in their lineup, but after Intel's sunsetting of that platform, they probably axed them.
You know all those random python automation scripts network engineers use to configure devices? Now imagine them actually running on device controlling packet and frame flows. Does that sound like it would be fun to troubleshoot?
What do you want to use P4 for?
The dream is you never have to rely on buggy vendor software again.
The reality is, you end up with a complex stack with no or homegrown documentation that requires experienced engineers to operate and maintain.
In some environments, that's a perfectly fair trade-off. In most, it isn't.
The APIs these vendors provide are a joke. A bunch of functionality can only be accessed via scripting interactive CLI commands. Some API endpoints cause short downtime / unexpected behavior (eg: by deleting the routing table and adding all entries back 1 by 1), while the on-device commands do not.
And guess what, the switch may decide to print informational or environmental messages interleaved with the command output, because the commands were meant to be run by a human. Good luck knowing if your state-altering command succeeded when you receive broken JSON.
I ended up regex-removing known environmental messages from command outputs..
Improvement on this is more likely to come from switchdev than from Tofino & P4. (though these don't necessarily contradict each other)
You can already run plain Debian on a Mellanox Spectrum device, treat it like a Linux software router, and by the power of magic your routes get pushed into hardware. (Source: device on my table to my right :D) Microchip's SparX-5 should be similar though I don't have one of those to test.
That was certainly the case 15 years ago.
The only switching/routing vendor with an API worth a damn was Juniper - in fact they were the only vendor who was applying CLI changes to the box via their own API, the Way It Should Be Done™.
These days you are spoilt for choice (and price point) with the likes of Arista, HPe (AOS-CX), Cisco (NX-OS, IOS-XR) and plenty of others entering the space.
Vote with your wallet!
This is not Perforce.
[dead]
[dead]
[flagged]