I was trying to build a debian VM over the past weekend acting as a router with Tailscale (subnet routing + exit nodes), Mullvad VPN, and local DNS.
The base router part was easy. The weirdness started the second I combined all three. Once you run Tailscale, a VPN tunnel, and local DNS on the same box, you start running into Linux policy-routing priorities, NAT/masquerade ownership, and DNS path conflicts around port 53. Most of the failures were partial and misleading rather than clean failures.
For example, Tailscale could look perfectly healthy while clients were still broken. I had states where `tailscale ping` worked, but DNS/SSH/HTTP to the router’s Tailscale IP would hang. I also had cases where client internet died the moment Tailscale DNS was pointed at the router, because the resolver path was slightly wrong. And one of the nastier bugs was return traffic for a Tailscale client getting pulled into the VPN routing table instead of going back out the tailnet, so it just “disappeared”
What helped honestly was going back to the basic tooling: switching from AdGuard to plain `dnsmasq`, and from the Mullvad app/CLI path to plain `WireGuard`. That made the whole thing much easier to debug. Tailscale handled the overlay/subnet side, `dnsmasq` handled local DNS, and the VPN became just transport.
This is a great writeup! Perhaps I can put in a plug for the create_ap script which I have been maintaining for many years (http://github.com/dlenski/create_ap).
It's a shell script that allows you to turn any ol' Linux computer into a WiFi router in one quick command-line:
By default, it will setup your WiFi card as an access point (allows WPA2/3, MAC filtering, etc), setup packet forwarding and routing, and run a DHCP and DNS server. It will generally pick sensible defaults, but it's also highly customizable. If your WiFi card supports simultaneous AP and client mode, it will allow that.
Its requirements are extremely minimal: basically just Linux, a compatible wireless card, and a few common configuration packages (hostapd, iw, iproute2, iptables, dnsmasq). No NetworkManager needed.
I used it as my own home Internet gateway for many years, running on an ancient fanless Atom mini-PC.
Because it can quickly setup and teardown WiFi networks on-the-fly, it's also a valuable tool for setting up test networks when reverse-engineering IoT devices. I use it frequently for this purpose (see https://snowpatch.org/posts/i-can-completely-control-your-sm...).
Lots of "just use X" comments but the article is about showing the bare minimum/how easy the core part of routing actually is.
Also, if you have ever used docker or virtual machines with NAT routing (often the default), you've done exactly the same things.
If you have ever enabled the wifi hotspot on an android phone also, you've done pretty much what the article describes on your phone.
All of these use the same Linux kernel features under the hood. In fact there is a good chance this message traversed more than one Linux soft router to get to your screen.
Yeah I find it more interesting to see how it's built from scratch, then I can decide if it's worth doing myself or just using X. I think this is a good software principle in general.
Basically any computer is a router if you're brave enough.
Windows PCs had (have?) that Internet connection sharing feature for a long time. It was really just a checkbox to enable NAT too.
Sometimes I think combining a firewall/router/switch/AP/file server/etc into a device called a "router" really confuses people. Even people who should know better.
It is much, much easier than it used to be. The documentation and videos alone available make something like this a very welcoming learning experience that anyone can complete step by step by pausing a video and replaying it.
Like most things, really. I used to build routers from old PCs, but eventually those tiny appliances caught up with the performance/functionality I need.
You can do a lot of routing on a $70 Mikrotik, although they might not be "easy".
For sure, it's a path and passage towards devices like that.
Everyone has a starting point, starting with soemone has lying around is one thing.. the quicker they can get going the more they can get to leveraging the real power in most devices.
This really takes me back. My first actual 'use' for Linux was making routers out of leftover computers.
The perfect machine back then was a 100MHz Pentium, in a slimline desktop case. At the time, the Pentium III was the current desktop chip, so you'd have a pile of early Pentium-class machines to use. And even a 10mb ISA network card (3Com if possible) would have plenty of power for the internet connections of the day. But 100mb PCI cards were still fairly cheap.
Install two NICs, load your favorite Linux distro, and then follow the IP-Masquerading HOWTO and you've got internet access for the whole apartment building, office, or LAN party.
Eventually I moved on to Linux Firewalls by Robert Ziegler for a base to build on.
After that I started piling other services on, like a spam filter, Squid cache, it was amazing to get so much use out of hardware that was going to just get thrown out.
That takes me back, I had the same trajectory , getting a newspaper’s news room and offices online with a single computer sharing its ISDN connection. Think ours was also a 100mhz gateway 2000 computer or some such.
That snowballed into “we want a website do you know how to do that?” and. Well, no, but it had Apache available and I … figured things out enough to take the skills elsewhere.
Repeated the same trick with a place in Wisconsin, who initially shared a 56k dialup connection with all their dispatchers and were impressed the thing had stayed up for 900 days without even redialing. 90% of their work was done in an on-prem wyse terminal anyway, dialup used to do the job for email or googling an address.
27, 28 years later I’m still dragged in front of them once in a while to ask how they can accomplish something cheaply with Linux, bubble gum, paper clips, or whatever . The times and technology have changed, but not how cheap they are!
I briefly put a Pentium MMX 200MHz system in service a few years back to bridge my parents to their neighbor's WiFi (with consent of course) when their DSL line was down for a few days. I installed a PCI Ethernet and WiFi card, booted into OpenBSD, and amazingly it was fast enough to get them through the downtime. :)
Ha, that's very close to my story as well. I had a 166Mhz Pentium and it was all PCI cards and 100mbit by then. That was essentially the start of my career.
Reminds me of a Pentium Pro router put into a datacenter, two 2GB mirrored scsi drives, two nics, happily running a hardened pfSense, ran with zero issues for the better part of a decade.
It just wouldn't die.
The suspicion was because the electricity going to it cleaner than average, in a datacenter, the normal wear and tear on electronics may have been reduced.
Respect was paid at it's decommissioning to convert it into a vm, knowing it's luck, chances are it would still boot up and keep on running.
> The perfect machine back then was a 100MHz Pentium, in a slimline desktop case. At the time, the Pentium III was the current desktop chip, so you'd have a pile of early Pentium-class machines to use. And even a 10mb ISA network card (3Com if possible) would have plenty of power for the internet connections of the day.
I was doing the same. Router and firewall on old Pentium CPUs. I don't have these machines anymore but I still have HDDs from back then with post-it notes on them saying stuff like: "Linux firewall / HDD 120 GB". For whatever reason my HDDs adapter that can read just about everything doesn't have the correct pin out for those HDDs. Would be a blast if they were to still boot: at some point I'll just buy a compatible adapter and see what I can find on those HDDs. I was very likely also saving some backups there.
But really my best memory was years (I think) before 120 GB HDDs became an affordable thing, in the super early Slackware days, on a dial-up connection: I had a 486 desktop computer and I'd share the Internet connection to a very old laptop (!) using... PLIP. A printer cable and the Parallel Line Internet Protocol. Amazing hack: my brother and I could then both use Netscape at the same time and to us this felt like a glimpse into the future.
I've got one of those N100+10Gbit router devices with a handful of ports. It seems a pretty reasonable device with one of the router distros running on it, but it doesn't seem nearly as efficient as my ucg-fiber/route10 devices, and that wouldn't bother me except that I suspect the packet latency is significantly higher too. Those devices AFAIK have hardware programmable router chips, which means the forwarding is done 100% without the interaction of the main CPU, so there isn't any interrupt/polling/etc delays when a packet arrives, the header gets rewritten, the checksum verified and off it goes.
Anyone actually measured this? I see a lot of bandwidth/etc style tests but few that can show the actual impact of enabling disabling deep packet inspection and a few of the other metrics that I actually care about. Serve the home seems to have gotten some fancy test HW but they don't seem to be running these kinds of tests yet.
You actually don't even need two interfaces on the box if you have a managed switch. It's not too difficult to configure your only interface as an 802.11q trunk port, and then you can use the managed switch as a sort of "interface expander". This is referred to as a "router on a stick" configuration, and it's how my home network is configured. Plus, if it's a PoE managed switch, you can install some cheap enterprise surplus Aruba IAPs around the house for Wi-Fi which is a lot higher quality than a consumer router or a mesh setup.
My home router was an old Thinkpad for a while, but then I switched over to a slightly newer Dell Optiplex that my work was throwing out. The plus side of that is that the i7 is total overkill for routing so I can also have my "router" run some VMs for network services and cut down on the number of boxen in my homelab rack.
I have actually been curious about this: How good can a WiFi mesh get latency-wise, given the right equipment, and how close would a consumer router setup be to that, do you happen to know?
I’ve been using OpnSense/pfsense [0] for years and would highly recommend it. It has a great automatic update experience, config backups, builtin wireguard tunnels and advanced features like packet filtering options via suricata.
When I am doing network management on my weekends, I’m so glad I’m not stuck in the Linux terminal learning about networking internals and can instead just go to a webui and configure my router.
I agree on principal, but I often find that the GUI abstractions don't always map to the linux tooling/terminology/concepts, which often ends with a head bashing against the wall thinking "this is linux, I know it can do it, and I can do it by hand, but what is this GUI trying to conceptualize?!?!"
I was recently introduced to a Barracuda router, and bashed my head against the wall long enough to discover it had an ssh interface, and linux userland, and was able to solve my immediate problem by directly entering the commands to get it to [temporarily] do what I needed. (Of course, using the GUI to reapply settings wiped my manual configuration...)
I've used pfsense, OpenWRT, Barracuda, Verizon's OEM router (Actiontec) and they all represent the same functionality wildly differently.
> I've used pfsense, OpenWRT, Barracuda, Verizon's OEM router (Actiontec) and they all represent the same functionality wildly differently.
Worth noting that pfSense (and OPNsense) are not Linux-based, they're based on BSD, specifically FreeBSD. While it's possible to have standard router OS web UIs that are cross platform, the underlying technology is different, so it's not really a surprise that there will be differences in how the devices running these OSes are configured.
The primary reason I stick to iptables instead of nft is that I already learned iptables decades ago, and some software I interact with still defaults to iptables and/or does not have full support for nft.
Why do you doubt the sanity of people sticking to iptables? What makes nft compelling?
My main reason is that nft applies configs atomically. It also has very good tracing/debugging features for figuring out how and why things aren't working as expected.
That said, I think many distros are shipping `iptables` as the wrapper/compatibility layer over nft now anyways.
Are they? I recently had to learn nftables and they seem to be iptables but with a slightly nicer syntax and without pre-defined chains. But otherwise, nftables directly maps to iptables and neither of them seem similar to pf.
I guess I'm different. I typically want my router/firewall/network services box to Just Work. I've made a career in deep-in-the-weeds system administration and engineering. Having to hunt down man pages, examples, tutorials, etc for the dozen or so fiddly bits make up a modern Linux- (or BSD-) based router was fun the first time, not so much the 10th. Been there, done that, got the t-shirt.
I will concede that the OpnSense UI is far from perfect. I would really like to see a device-centric view that lets me set all the things related to that device from one screen (or possibly one screen with multiple tabs). For example, if I add a Roku device to my network, I want to enter in the MAC address and then be taken to a screen where it will let me set the hostname, pick a static IP address, hand it a specific DNS resolver IP, see all of the traffic going to/from the device, only allow it access to the Internet between during certain hours, etc. All of this currently requires jumping around between multiple disconnected parts of the OpnSense UI.
I feel almost exactly the same as you on the subject. When I was young and starry eyed I built my own router out of a PC running openBSD, all by hand. Nice learning experience, interesting OS, but definitely not maintenance free especially around system updates as back then openBSD packages and sys upgrades required recompiling everything. Now I do the same mini-PC thing as the OP's article but I just put OpnSense on it. Agree the UI can be maddening at times but the thing is rock solid, and has very polished update and upgrade mechanisms. Built-ins/plugins are great - unbound, wireguard, openvpn suricata, backups to git etc. Also I like that it is BSD based, my network experience was learned on Cisco's and Junipers in an ISP setting and Linux networking has always driven me crazy
I'm at a stage where I don't want to be doing network management on my weekends. I have a Ubiquiti router that's pretty good, and for my router I'd like something like TrueNAS for my NAS, a distribution that completely turns the hardware into an appliance I can configure once and forget about.
Pfsense/opnsense would be one option (based on FreeBSD). For Linux there is OpenWRT, which you can either run as an alternative firmware on quite a few consumer routers/access points, or install on a PC or Pi or similar.
Caveat: I have only used OpenWRT on a high end consumer router (GL.inet MT6000) out of those. That works well, anything else is based on reading about people using those options.
For all of those, once you set it up you don't really need to do much except install updates a couple of times per year, or if you want to forward a new port or such.
I recently dumped opnsense because they took a stand against a few things I was trying to do (ex, webUI on wan port IIRC) which make sense at a high level. But I _HATE_ devices that think they know better than me. I was trying to configure it on a _LAN_ such that the identified WAN side was actually my local lan, and I spent an hour hacking it to work and was like "you know if they can't get this shit right i'm out". There are a lot of places in the technology world where someone who thinks they understand my use case makes a decision based on some narrow world view because they can't understand that not everyone trying to use their product is some idiot home user using it for their home network.
Yep, this is the way. You will learn loads using Linux but this is not something you want to go wrong.
I used a lower power Intel Atom mini PC with an additional NIC as a router for years. I tested it and found it could route around 300Mb/s which was plenty.
But then I got gigabit internet. So I bought an Intel 4 port GigE card from eBay and now run OPNSense as a VM. If you get the right Intel card you can pass through ports to VM individually, which is nice for playing (don't know the exact details but look for cards with virtualisation support, mine is an 82575GB I think).
To be fair, my setup still probably has too much to go wrong, due to the VM thing, but I just haven't got round to getting dedicated hardware, and it's worked fine for a couple of years now.
This seems like it might be a good place to ask: does anyone know of a low-cost, readily-available SBC box with built-in dual Ethernet interfaces?
I've been very interested in some of Radxa's boards in the ~$30-70 range, like the E52C [0] and the E20C [1], but they don't have many distributors and seem to have stocking issues [2].
I'm curious about the policy rationale behind banning router imports. If a government were considering legislation like that, what would the primary concern usually be? Given that so much internet traffic is now protected by TLS/SSL and other encryption, why would it still matter if citizens were using routers that might be backdoored?
Is the concern mainly things like botnets and DDoS activity, weak default credentials on network equipment, or compromised business networks where poorly secured routers or attached NAS devices could expose sensitive or proprietary data? In other words, is the concern less about decrypting traffic and more about using the router as a foothold for surveillance, disruption, or access to poorly secured internal systems?
Among policy and security people, the term they bandy about is Advanced Persistent Threat (APT). They're not wrong; there are a number of recent cases, and these are ongoing, and you've heard of some of them: Volt, Flax and Salt Typhoon and Velvet Ant. There are more you haven't heard about, because only the operators know they exist.
These are networks of controlled devices. They're hard to eradicate, as shown by the fact that they haven't been eradicated: they're still active and being used to compromise systems, including defense and intelligence systems, power systems, financial systems, identity systems, etc.
Is banning foreign gear going to fix this? No. Security isn't a product. It is, however, a process, and in a process you take steps. I think this: we (individuals and institutions) enjoy tremendous liberty in the use of communications equipment in the US and most of the West. Taking that for granted is a mistake. If part of keeping this means the US has to spin up a domestic supply of network gear, or carefully modulate where such gear comes from, then lets do that. Otherwise, The Powers That Be will leverage its concerns into far worse steps.
It's everything you mention in the second paragraph, and additionally just the ability to turn them off.
Imagine everyone had their routers disabled simultaneously. I don't know if the cell networks could function with the surge in standard traffic that would happen, and then you've effectively plunged all or part of the country into a communication blackout.
I think "turn it off permanently by bricking it" is almost as bad as "leverage for DDoS".
I worked on Bot Mitigation at Amazon, and we once saw a ton of traffic that was heavily distributed amongst consumer devices world-wide, but surprisingly in the US too. We suspected compromised routers that were using the home page as a health check. There was a lot of investigation I did, and the short realization after talking with the network engineers is that the amount of traffic, and distribution of sources, would be impossible to stop. There merely isn't enough bandwidth in the world to stop so many residential device if it hits a specific target. To be clear, this was coming from less than half of active Amazon customers, not everyone in the US.
Anyway, it wasn't routers, but it was a consumer device, and it wasn't nefarious, it was incompetence (in code), as usual.
>Imagine everyone had their routers disabled simultaneously. I don't know if the cell networks could function with the surge in standard traffic that would happen, and then you've effectively plunged all or part of the country into a communication blackout.
IME cell networks definitely can't cope with a loss of all routers in an area, given how mobile data becomes basically unusable when there's a power outage. That said, "everyone had their routers disabled" is probably not realistic, given that there are plenty of non-chinese router vendors.
If this were really about computer security they would follow California’s example of requiring unique passwords. Maybe make manufacturers liable for not patching known remote exploitable security vulnerabilities. It doesn’t matter if the source of a DDoS is a Huawei box or a Netgear box.
> is the concern less about decrypting traffic and more about using the router as a foothold for surveillance, disruption, or access to poorly secured internal systems?
That should probably be the technical concern. Even if you have traffic protected by TLS, you still typically have enough metadata to cause some problems for users individually, but the assumption that foreign equipment is back-doored by some security service or other is probably safe.
“Just use OPNsense” is great advice for production, but terrible advice for learning.
This article is valuable precisely because it shows how little magic is actually involved in routing.
in /etc/sysctl.d/10_router.conf to slightly reduce overhead when being used primarily as a router. There are many other router related knobs but those I would always set especially if trying to reduce overhead for VoIP/Gaming setups. There are many other knobs I tune such as gro_flush_timeout and napi_defer_hard_irqs, sch_cake tuning, lowat and output limits and hundreds more but those rabbit holes would require a large write-up. My overall goal is to give family members latency, jitter and throughput numbers that improve their quality of life and gaming scores of course.
Such things do not preclude additional tuning on the client and server sides as well but those are even bigger topics.
Hmm I've always had a manually configured low power generic box as router.
But I've never even tried to set up my own access point, I just pay Unifi for that [1]. The software part is doable but I don't want to learn to handle the signal issues.
[1] Switched to Unifi in anger after my first consumer level 5 Ghz wifi needed reboots weekly because it was overheating. Do yourself a favour and get the semi pro stuff, Unifi or others.
I've been running a custom router for about a decade, but I too have haven't tried handling the wifi on my own. It's always been easy to get an external access point and there's a bit of a guarantee that it's done correctly.
I kind of feel like that's cheating though; I've outsourced the hardest part of the project to someone else. Maybe one of these days I'll take an old NUC or something and buy a decent wifi antenna for it and try and do it properly.
[1] Initially pfsense, then OpnSense, then ClearOS, and now some custom firewall rules in NixOS.
So it's been awhile but the best and simplest way I think is use an access point. I don't want my wireless gear doing routing. From a logic stand point they acts as wireless "bridge" to the physical network, and nothing more. DHCP, etc. stay handled in one place for the entire network, back on the physical router.
> While it may seem appealing, I would highly recommend against installing loads of software right on the router, and instead forward traffic to a device in a DMZ or VLAN.
Why not? I use an old gaming PC as a "router" (machine exposed to the WAN), and run dozens of services on it besides the firewall/NAT (iptables). Among others: email, Web server, multiple game servers, and many internal services (DNS, hostapd, loads of Docker containers).
You need more careful firewall rules on any device with IP forwarding enabled, and it can be hard to remember exactly when forwarding, NATint, etc. happen with relation to the incoming/outgoing firewall rules.
E.g. is your pf-based load balancer running its rules before or after the global filtering rules? And if they're running first are they SNATing incoming traffic so the LAN rules allow the traffic through or does it need explicit exceptions for external IPs to traverse to a LAN endpoint?
If you're comfortable with more advanced networking then it's fine to run it all on one box. If you just want to open ports for internal LAN services then that is a very canned and well-supported feature for a gateway firewall.
E.g. see AirSnitch which resulted in large part from mixing too many complex networking rules in single devices.
A classic example is Docker inserting its firewall rules before everything else, causing any "published" ports to be wide open even if a firewall ruleset was configured. They might have fixed this, or doubled down on their design choice. Either way, that kind of complexity can really bite you if you're not careful.
It seems like you weren't really asking, but I'll answer anyway.
It's bad security practice, and opens up your network to attack and/or compromise, you're massively increasing the attack surface, and a compromise of one of those components leaves the attacker sat on your edge router, at which point your entire network is fair game.
Generally speaking you shouldn't expose anything on your edge router / firewall, it's a safety barrier.
You can sit things behind it in a "DMZ" and port-forward and isolate them etc so that there's no packets terminating on the actual edge device itself.m, that lowers the risk of a full network level compromise.
Chances are you might be fine and never have a problem, but it's still recommended against.
It was a genuine question, and while you reiterate the author's point about this being "bad security practice," neither you nor the author explain why this is the case.
I don't believe physical separation really buys you much here. At most, if may reduce downtime if you do indeed get pwned, but I think that you can achieve the same objective through a combination of containers, VMs, and UNIX users. And running multiple, somewhat redundant machines also has obvious downsides such as increased power consumption, increased maintenance burden, additional space and cabling, etc.
I recommend replacing hostapd with an enterprise access point plugged directly into an ethernet port on the router. Most support VLAN tagging based on SSID, so you can still set up different subnets and firewall rules for the different SSIDs.
As much as I love hostapd... the performance using commodity hardware has always sucked for me. I can get 150MB/s over wifi with my proprietary AP!
Maybe someone in this thread has a couple of ideas:
What’s the simplest way to spin up a simple „cattle, not pet“ routing VM? I don’t want to mess with any state, I just want version controllable config files. Ideally, if applying a version fails, it would automatically roll back to the previous state.
OpenWRT seems like it fits my description most closely, but maybe someone here is a fan of something more flashy/modern.
NixOS using https://github.com/thequux/nix-zone-firewall/ worked well for me for many years. I only stopped using it because my poor embedded Linux machine started having issues and it made more sense to go with a Mikrotik than to buy a new device to run as a soft router.
That sounds like you might like VyOS. I found it to be relatively easy to achieve exactly what I wanted, but went back to a GUI as it turned out I wanted a pet and not start a farm.
I recommend Pfsense or OpnSense if your hardware works with a FreeBSD-based thing. They're super easy to set up and don't have many surprises.
After I upgraded to a 10GbE ethernet card in my previous router, my card didn't work correctly with FreeBSD-based stuff anymore. I changed to ClearOS and that was actually comparably easy to Pfsense...maybe even easier? I recommend checking that one out.
While I am a linux advocate for networking in the current day outside of hyper specific CDN use cases (a la netflix)... its pretty common for people to just virtualize opnsense/pfsense to take advantage of linux network drivers. Especially if their actual routing requirements are modest and dont require full use of the hardware.
Beyond getting support for devices completely absent on freebsd, quality of drivers, bugs much more rapidly squashed, and general misc features absent on the bsd side like NBASE-T.
This is the kind of low quality information you see on fanboy forums. There is nothing special about Linux drivers and anyone can go look at them. A lot of hardware uses a HAL and there is a smaller OS adaption therefore most of the code is similar across OSes.
Virtualization means you now have multiple layers of drivers and privileged code in the mix to add and amplify bugs, it can and should work but if you are doing this in the name of stability that is a bit curious.
The reason Netflix can do what they do is they have good relationship with their HW vendors, NVIDIA(Mellanox) and Chelsio. If they were on Linux, they'd need the same level of support.
I don't know enough about this level of IT to rebut this.
I use Linux for my router now because my server is NixOS, so I was able to consolidate my router into my server and turn off a machine (and thus save a little power), and I have so thoroughly drunk the Kool-aid for NixOS that I kind of want to put it everywhere. I run the latest kernel and I update daily, so I think most bugfixes (and hopefully security updates) will manifest quick enough.
OpenWrt has a generic x86 PC build that can also be used to turn basically any random PC into a router, complete with an operating system actually designed and developed for that purpose.
Otoh it would make sense if you could combine it with a home server, then it's just a side process and you actually save power by not having an extra device.
Though you'd still need a switch or two. And a fiber modem which already has a router and a switch built-it. Oops.
OpenWRT is great if it fits your use case. If one has reason to stray from the happy path a disadvantage is that the OpenWRT uses a single binary like Busybox and doesn't use glibc. This is great for embedded/low power machines like the OG WRT54G, but not as optimal for when you have an entire random PC. I don't recall the exact things I was looking for but I moved on to pfSense and didn't look back.
I wonder if consumer routers will end up being built in a trivially-not-a-router configuration - something akin to a pull-tab or turn of the screw that closes a circuit, transforming the device from legally something else into a router after it's purchased.
"this here is a virtual network appliance, so called because it doesn't have any ports on it - wait, why are you taking off that blanking panel? That's illegal!"
There's a famous Polish song "Mój jest ten kawałek podłogi", written in times of Soviet influence, about a man building wall around his home, but later he gets hungry and there's no exit...
Anyone with translate.kagi can find it and translate
Pleasant thing about routers that is is so simple to build one after learning basics of networking and pretty much any OS or distro can act as one. There are obvious choices like OPN\PFSENSE, OpenWRT, DD-WRT, FreshTomato, but literally any PC with a single Ethernet port can act as one. My favorite setup was a laptop running Ubuntu and the whole router setup was in a single netplan file + dnsmasq for DHCP.
Edit: And ofc best cheap device imo is OrangePI R1 LTS and a whatever usb wifi dongle. Came in clutch a few times, such a nice little device.
Been using DD-WRT for years. Current setup is a $50 Dell Optiplex i5 from ebay running x86 DD-WRT. I put an intel 4x 1Gbit NIC in it, and it's been an excellent router for years.
Just ensure the firewall appliance thing you buy has I226 intel chipset not I225
The first two versions of 225 have packet drop issues and it’s unclear to me whether v3 third time lucky fixed it. And getting the stepping info out of aliexpress supplier is hard so 226 is safer
I did this back when, just using a 100mbit NIC express card.
Ran openbsd for a few years like that, the base OS included everything needed. I recall it used 24MB of ram and closer to 30MB if ssh'd in. It was very handy to have a local login when playing with firewall rules.
nftables syntax is pretty tough to read. I wonder why they didn't go for an easier to read DSL. I do understand it's likely super fast to parse though, and has a 1:1 relationship to its struct in the kernel.
I’ll pick nftables over iptables any day, it’s leagues better (granted, it’s not hard). The nftables wiki is great, as the syntax and modules are documented in a single easy to read page.
As an added bonus, you get atomic updates of all chains for free.
Granted, for simple usecases, ufw or firewalld may be simpler though.
You can still use the iptables interface for nftables rules if you'd like, but I think you miss out on things like atomic application of rulesets, ranges, lists, and variables (not shell variables).
Anyone has done mesh WiFi (ideally triband) using off the shelf parts and Linux?
I have an Orbi AX system which works reliably, but now I want to upgrade the radio to WiFi 7 and that means I need to upgrade all the hardware.
Hoping to move to using off the shelf parts so in the future I can just change the radio (ideally bunch of USB sticks).
I understand this is not strictly just the router. I can (and used to have) a router as separate device, but any mesh WiFi right now that I can find need a pricy router that acts as the coordinator, essentially negates the economic benefits.
That's a bigger can of worms than you might expect. Most consumer WiFi chips only barely support AP mode, and I'm not aware of any that can do multiple bands simultaneously. You'd probably need 4 adapters on the repeater for triband. One to connect upstream, one for each downstream band. Three instances of hostapd all configured with the same SSID and auth for each downstream interface.
Then there's the roaming issue. This is largely what the commercial "mesh" systems try to solve: deciding / helping inform when clients should switch APs. There are many solutions and none of them are without issues, including the commercial ones. Here's a starting point: https://openwrt.org/docs/guide-user/network/wifi/roaming
- PCEngines APU (x86, AMD T40E) (my current router/firewall) (discontinued)
I'm also currently using an APU2 as one of my wireless access points (with hostapd).
All of these have been solid machines that have given me zero problems.
The next system I plan to use is going to be a Banana Pi R4 (ARM Cortex A73), it's a solid choice for a simple router/firewall/DNS/DHCP box. It has a built-in 4-port gigabit switch where each interface can be used as normal Linux interfaces, as well as 2 SFP+ ports that are capable of supporting up to 10 gig ethernet.
It's also one of the few systems that offers true hardware offloading for connection tracking, so things like netfilter flowtables don't have to use any main CPU processing.
I'm currently experimenting with a Banana Pi R4 as a Wifi7 access point (running Debian with hostapd), however the current state of the wifi7 module for it (BPI-R4-NIC-BE14) and Linux driver (mt7996e) is still pretty young and a bit buggy (i.e., limiting transmit power to 6 dBm without patching the driver to override it, and there's apparently a lack of RF shielding which can contribute to low SNR on the receiving end). With the proper patches in place it makes a decent Wifi 6 access point. I'm hoping these issues get ironed out in the future and I can use it as a true Wifi7 AP. frank-w is doing outstanding work to help support the open source community with this new hardware.
It's hard to recommend one thing because there are so many options and they all have different trade-offs in terms of initial cost, ease-of-use, reliability, performance, etc.
A year or two back, I was able to get a brand-new fanless Intel N150 with 4x2.5G ports with 16 GB memory for about $150 from AliExpress. I run Proxmox on it, with OpnSense and a couple other things in virtual machines. These days, due to tariffs and the memory shortage, that is more like $440 now, unfortunately. I am kicking myself for not buying two, not so much because of the price increase, but because it would have come in handy multiple times to have a second one on-hand for random experiments.
Given that CPU performance does _not_ tend to be critical for firewall/NAS use cases, if I had to replace it tomorrow, I would go onto eBay and get the highest-spec'd used Dell or HP mini workstation I could find for $120 and plug in a USB3 1gig ethernet dongle for the WAN side.
I am truly sorry. I can't understand the physical networking from the pics or the description... I'm probably just missing something. There is one blue plug going from the laptop to the cisco switch or the pci wifi module? I see a blue plug going to each device. So I'm guessing everything is plugged into the cisco switch?
if you could show all the wiring and label it (according to the table below) i think it would add a lot of value for someone less familiar with these kinds of setups (like me)
Hey, op here, this was almost a decade ago, but I'll try to describe what's going on here. It's kind of a crappy picture.
* WAN connection comes in by coax, into my cheapo cable modem (off screen), and then by Ethernet into the franken-NIC sitting on top of the laptop.
* The NIC on top is a normal PCIe card, but with the bracket missing. The ExpressCard riser [1] is connected by a mini-HDMI cable, the flat black cable, which curves up, around, and back in from the left side into the laptop
* Then, the blue cable on the side of the laptop is a VLAN trunk going into the Cisco switch on port 23/24, outside the picture.
* From there, another port on the switch is setup as an access/untagged port going into one of the LAN ports on the D-Link acting as the access switch
I don't think it was set up here, but at one point I also had a dock under the ThinkPad, with the serial adapter wired up to the switch's console port so I could manage everything by ssh'ing into the router.
Also note that all the cables were hand-crimped because I was too cheap to buy new patch cables at the time.
I was in college, and truly had more time than money back then. it's the kind of doohickey made by only somebody very young, very crazy, or a bit of both. ;)
Routing is pretty easy for most use cases... firewalling an Internet connection, on the other hand, is just about impossible (thanks TLS 1.3) without pretty serious overhead, 3rd party maintained live subscriptions, TLS interception, and a willingness to say "no" to a lot of the shenanigans that modern programs and devices try to pull.
I recommend the free home version of Sophos for the least painful way to do it. Buy a Palo Alto with a full subscription if you are really serious.
Not necessarily. For one, the BSD has, or at least used to have much worse driver support for wireless adapters. With a regular server/desktop Linux distro, it's also easier to run other services on the same device. For example, nothing prevents you from running nginx and hosting a website, or a personal email server.
In this day and age, if that's what you're after, you can just point an AI at the problem and give it shell access, and it'll just do what you describe (Claude code, codex, etc).
Here I was thinking this article would tell me how to turn my unmanaged switches into routers, but no, "anything" actually means "any fully featured general purpose computer with networking".
That's theoretically possible but a bad idea for a managed switch, because they seldom have enough CPU performance or IO between the CPU and switch silicon to provide respectable routing performance. For an unmanaged switch, it's more likely that whatever CPU core is present (if any) doesn't have enough resources to run a real network stack.
I live in the SF Bay Area, and ended up with Sonic Internet, and a 10gbe connection. Routing this with anything off the shelf was going to be "very expensive".
I ended up with an Opnsense box. It's an m920q (i5-8500), with riser card and a dual SFP+ nic in it. All in, it was less than 200 bucks (now it would be closer to three). I ended up with a cheap, Chinese "media converter" (from aliexpress because the same thing on amazon is 3x the price) that just had two SFP+ ports on it. That let me go from an SPF+ copper ethernet module to a DAC and not dump a bunch of heat into the 1L pc.
I have to say that the functionality made it a worth while investment: traffic shaping, wireguard and the like have been mostly a joy. And the documentation for Opnsense made the setup and use (mostly) easy.
I did kind of the opposite, I made my main beefy gaming computer the router, then connected to it a nice wireless AP in bridge mode to serve internet to the rest of the computers. That way I can have a local llm agent manage my network and firewall by simply asking.
We are just scraping the surface here; let's imagine a really easy to use and install bit of router software that includes all kinds of p2p bells and whistles.
The extreme difficulty of setting up networking and routers is (obviously?) a weird endgame result of how companies and safety and capitalism and restriction intersect* and given the relatively insane regulatory ideas we're seeing these days, time for another look at all of this.
*edit, and not, e.g. an inherent property of "networking technology," it does NOT have to be this hard.
Qotom mini PCs are my cheatcode. These little PCs are often available with multiple NICs, and I use one as a wifi bridge/router for my office network. Put Linux or FreeBSD on one and you have a very capable little network-appliance box.
Nobody I know makes routers and more importantly WiFi combo AP in the US except for high end corporate stuff - even the US only cable modem stuff from Comcast are Chinese OEM ???.
SpaceX manufactures network gear (Starlink satellite internet terminals) in the US. Bastrop, Texas, specifically. Those phased array transceivers, with heaters, ethernet, GPS and WiFi are way more difficult to make than the typical indoor WiFi router.
Great writeup. One advantage true routers have e.g. Edgerouter or Mikrotik are dedicated hardware for IP & TCP header processing. Some can offload AES for VPN encryption. This leads to cooler temps, lower power utilization & longer life.
I encourage everyone to run a hardware router. A cheap dedicated wired router can be had for $50. Run PfSense or the vendor firmware . It’s very rewarding. Also a long term investment since routers tend to last for many years while wifi standards are revised every year or so .
A fun project that results in a unique and stylish router is repurposing a Mac Pro Trashcan. They can be picked up for a few hundred dollars, offer dual 1GbE Intel NICs that work natively on Linux, and have plenty of CPU and RAM overhead. Throw OPNsense on there and you’re off to the races.
if fancy a bit more of capability, dockerized opnsense and just play right with your vlans. One cable is enough into your switch...did I said managed... and your opn/telco eth exit.
I think the NanoPi range (https://www.friendlyelec.com/index.php?route=product/categor...) has great hardware for making your own router. They support various flavours of Linux, including OpenWRT (or at least their branded version called FriendlyWRT). I like the NanoPi M5 model as it supports using a NVMe so it can happily run some Docker containers on it. The case is really well made and uses passive cooling, so it's ideal for a router.
Absolutely not. My 8700k is able to saturate a 25Gbps link without breaking a sweat. It could still push more than 4Gbps of encrypted traffic (Wireguard tunnel).
Yes. You can take advantage of Netfilter's flowtable infrastructure and if you have the right hardware (NVIDIA/Mellanox ConnectX-5 or MediaTekMT7621) it will actually offload the processing of these packets to the NIC hardware. This only applies to established connections, however, but that typically accounts for like 95% of the traffic passing through.
Any computer with a single network interface, maybe even an (old) laptop, can be used. Anything x86 from at least the last 10 years is energy efficient and fast enough to route at gigabit speed. If you don't care about energy usage, any x86-based computer from the last 20 years is fast enough.
The magic trick is to use VLANs, which require switches that support VLANs, which can be had for cheap. VLANS also allows you to create separate isolated networks for IoT or other 'less secure' or untrusted devices.
I’ve always made my own routers by using low-power devices running Linux (Debian) with IPtables and now NFtables.
No special router OS or software required.
Highly recommend.
P.S. that single network interface is very likely never a bottleneck because network interfaces are full-duplex. Only when your router is also your file server (not recommended), internet traffic and file server traffic could start to compete with each other.
It only needs one port, but for most simple networks two ports on the router means less configuration.
The "router on a stick" paradigm using VLANs to a share a single physical port is perfectly valid. You're creating a "now you have two problems" scenario in which you need a VLAN-capable switch and have VLAN configuration to make.
I typically like the ISP router on a dedicated router port to make monitoring the physical link and/or cycling the physical link easier.
Unless your ISP is >1Gbps adding a second port to most devices is as easy as adding a USB NIC.
Technically you can route without isolation, but VLANs are definitely a good idea if you’re using a single port.
There are 2.5 Gbps, 5, and even 10 Gbps USB NICs these days, although 10 Gbps ones are pretty expensive and require really recent USB ports.
I agree I want my local network and my WAN port separate, if for no other reasons than so I can use ssh to get into the router from my LAN with the WAN port disabled.
Yes, but some folks are wary of using the same physical port for external and internal traffic. Fears of "VLAN hopping" remain, even if unfounded. Also, you'll hit a performance wall since you are sharing a single gigabit port between external and internal traffic. Obviously may not be an issue for many, but if you have gigabit fiber...
I have gigabit fiber and none of this is an issue.
VLAN hopping is only possible due to misconfiguration. I'd like to be proven otherwise if that's not the case. VLANs are used EVERYWHERE where it matters. And no, the single port is absolutely not a bottleneck because the port is full-duplex.
I agree VLAN hopping is not possible without misconfiguration but it still is a "concern" for some. I also make extensive use of VLANs on my home network.
If you're trying to push close to a gigabit up and down simultaneously that single port will become a bottleneck. I agree for most typical use cases it is not a concern.
That's going to be super rare. If it's just LAN traffic it shouldn't hit your router at all and you won't have the bottleneck issue. The actual cases would need to be quite contrived, like you're backing up your media library at the same time you're updating cod warzone.
Sounds interesting. I always wanted to use a Raspberry PI as router (to have one as backup in case the OpenWRT Linksys goes down), but couldn't wrap my head around properly how to overcome the single network port (I think the usual recommendation is to use an extra USB network card/adapter). Can you elaborate more about this VLAN stuff (you would put your modem, your router, and all your machines on the switch... and in the switch you tell the router connection to double use the connection for WAN and LAN separated via VLANs? And put the modem into the "WAN VLAN" too?)
Ideally the PI also should to what the extra DSL Modem does… but I guess that's where the dram must stop. :D
When you setup your single ethernet port (let's call it eth0) as a VLAN trunk port, you'll get the ability to configure multiple virtual interfaces off of it. How many virtual interfaces you get depends on how many VLANs you want to tag traffic for. For example, if you have 2 VLANS with ids 100 and 200 (100 being your public Internet-facing traffic, and 200 being your LAN traffic), you would then have interfaces eth0.100 and eth0.200 to work with that you can then use in your firewall scripts as if they were two separate, physical interfaces.
This of course means you need a VLAN-aware switch that this single ethernet port can plug into, configured as a VLAN trunk (in Cisco terms) port. You would then want to configure one of the other switch ports as a VLAN access port assigned to VLAN 100 (untagged). This is the port you would plug your cable modem into. Then (in the simplest example) you could assign all the rest of the switch ports to VLAN 200 (untagged), and you would plug all your LAN devices into them.
The TL;DR is to have two vlans on the cable from your switch (called a "trunk"), "lan" and "wan", carrying the respective LAN and WAN networks. Then, on the Pi, create two vlans on the underlying Ethernet interface. Then those two VLAN interfaces can be configured just like the LAN and WAN interfaces of the router. On the switch, you’d dedicate one port to the WAN by adding it to the WAN VLAN without tagging, and the other interfaces do the LAN VLAN, also untagged.
Yea, I would add openwrt x86 provides a decent interface for management. Gave dad a little minicomputer with openwrt when he upgraded his internet. He can change wifi password and such and is happy.
VLANs are fine. Running your whole core over one trunk into a general-purpose box gets dumb fast, because one bad config or L2 loop turns into a host-side debugging session.
Extra NICs move forwarding work into the host, and you pay for that in CPU time. If you care about isolation and wire-speed, buy a cheap managed switch instead of stuffing more NICs into the box.
A CPU from the last 20 years can route traffic at gigabit speed. It's only something to worry about for a Raspberry Pi3 or something similarly 'crippled'.
I think I understand why this is true for plain IP forwarding. There isn’t much to break the cache and the lookups are few and fast.
What’s the cheapest (new) computer that can drive a 1Gb port with NAT? With a busy encrypted (wireguard?) connection?
[I don’t think qos has a lot of use in the domestic environment; sure, someone here does it but I think it’s much less mainstream than the features I already mentioned. ]
Such a device could drive my home. But in a couple of years I suspect I’ll want 2Gb or 10.
In the past I’ve tended to use a device until its crappy power supply failed. So I guess I’m hoping for a >5 year life span/upgrade capacity.
For all I know the answer to my question is one of those passively cooled four port n100 bricks from AliExpress. Anecdata happily accepted.
> What’s the cheapest (new) computer that can drive a 1Gb port with NAT?
What's the cheapest new computer you can find? That will work. If you have PPPoE, you need to be a bit more careful; depending on your OS and NICs, it's possible for inbound traffic to only use one core; low power laptop cpu may not have enough throughput from a single cpu, but my information is a little dated.
I did 1G NAT on a dual core haswell [1] for a long time.
Wireguard adds nothing unless you'd want to terminate it on the router. In which case it adds so very little it's unnoticeable.
About any n100 will do. Question is in their reliability which mostly comes down to power regulation components quality. Not performance.
One of my installs runs on a repurposed old android phone. Which has about 100 times CPU capacity of the router I write this through, and that one being cheap tplink shit still terminates wireguard at link speed which is 100Mbps. You don't need fancy gear for routing. And you don't usually need gigabit uplink because speed is limited way upstream.
But if you want "the right gear and damn the price" go get a Microtik. They are very good.
You'd be shocked to find out how old and weak the CPU in your current router is. Typically they're on par with low end desktop CPUs from 10-15 years ago.
I assume the real router OS is extremely neutered to basically only route traffic and filter inbound with everything else being removed? But yeah I can definitely see that.
Except actual routers don't handle the traffic on the CPU, they have dedicated hardware to actually handle the packets. The CPU basically runs the OS, configures the hardware router, and does housekeeping tasks (e.g. ARP or FDB expirations, NAT cleanup, etc). The only packets that ever reach it are "trap to CPU" situations that don't require acceleration as those are rare or expensive to implement in hardware (e.g. better suited to a CPU). Those usually include management protocols (ICMP, ARP, NDP, STP, etc) or packets with unknown destination (e.g. the first packet to an IP that requires ARP resolution).
That's how you can have multi-Gbps on a router with a 200MHz MIPS CPU. Or Tbps on a router with a quad-core Xeon.
I'm currently running a Debian lite weight server on an old ml100 (onlogic) nuc. It's an old i3, with 16gb ram and no fan. But I have another. Anyone recommend a solid router setup on one of these ancient artifacts? Presently using openwrt on a proper router, though if the nuc is capable, I'd dedicate it thusly.
Seeing an old T60 with an ExpressCard-PCIe bridge used as a router is a great look. It's a solid reminder that even a "trash-picked" 18-year-old machine has way more CPU than you actually need for a home gigabit line.
The mention of the serial console (ttyS0) is the real pro-tip in this guide. If you're running a headless box in a closet, a serial getty is a lifesaver for the moment you inevitably misconfigure a firewall rule and lock yourself out of SSH.
Sticking to a minimal Debian base with nftables is often much cleaner than using OPNsense/pfSense; there's no GUI abstraction layer hiding what's actually happening to your packets.
I'm curious - for power consumption, considering that you can get RaspPi products for so cheaply, is a discarded laptop more or less impactful on your electrical bill than a RaspPi?
Like is the "free" laptop going to cost you more in the long-run then a nice little power-sipping ARM like a Pi5? Or do you need those extra operations-per-second that the more power-hungry x86 CPU gets you?
If you're tech-savvy and building your own router, you can add those advanced aspects in if you want them.
I'd be willing to bet, though, that the overwhelming majority of people who use consumer routers aren't doing anything remotely advanced. A how-to that covers the majority of use cases is valuable even when it excludes advanced use cases.
There are a whole lot of normal people using mesh networking Wi-Fi routers. Honestly, most of the least technical people that I know are all using mesh networks because their houses require it.
Where do you live to consider mesh networking a minority use case? I live in a small city apartment so I don't need one, but everyone I know outside of the city needs at least two nodes to cover their houses.
Home mesh is mostly about having wireless backhaul, and you can certainly do that if you have (preferably) two radios, you just set up one radio as a client to your main AP.
Even if you aren't doing wireless backhaul you just rely on regular client behaviour to transition between APs, can enable 802.11r to improve this.
Enterprise "mesh" typically uses wired backhaul for performance and can help clients roam quicker with a controller (auth, not deciding to roam). Controller can also adjusts radio power so APs aren't talking over each other if they're too close.
I was trying to build a debian VM over the past weekend acting as a router with Tailscale (subnet routing + exit nodes), Mullvad VPN, and local DNS.
The base router part was easy. The weirdness started the second I combined all three. Once you run Tailscale, a VPN tunnel, and local DNS on the same box, you start running into Linux policy-routing priorities, NAT/masquerade ownership, and DNS path conflicts around port 53. Most of the failures were partial and misleading rather than clean failures.
For example, Tailscale could look perfectly healthy while clients were still broken. I had states where `tailscale ping` worked, but DNS/SSH/HTTP to the router’s Tailscale IP would hang. I also had cases where client internet died the moment Tailscale DNS was pointed at the router, because the resolver path was slightly wrong. And one of the nastier bugs was return traffic for a Tailscale client getting pulled into the VPN routing table instead of going back out the tailnet, so it just “disappeared”
What helped honestly was going back to the basic tooling: switching from AdGuard to plain `dnsmasq`, and from the Mullvad app/CLI path to plain `WireGuard`. That made the whole thing much easier to debug. Tailscale handled the overlay/subnet side, `dnsmasq` handled local DNS, and the VPN became just transport.
This is a great writeup! Perhaps I can put in a plug for the create_ap script which I have been maintaining for many years (http://github.com/dlenski/create_ap).
It's a shell script that allows you to turn any ol' Linux computer into a WiFi router in one quick command-line:
By default, it will setup your WiFi card as an access point (allows WPA2/3, MAC filtering, etc), setup packet forwarding and routing, and run a DHCP and DNS server. It will generally pick sensible defaults, but it's also highly customizable. If your WiFi card supports simultaneous AP and client mode, it will allow that.
Its requirements are extremely minimal: basically just Linux, a compatible wireless card, and a few common configuration packages (hostapd, iw, iproute2, iptables, dnsmasq). No NetworkManager needed.
I used it as my own home Internet gateway for many years, running on an ancient fanless Atom mini-PC.
Because it can quickly setup and teardown WiFi networks on-the-fly, it's also a valuable tool for setting up test networks when reverse-engineering IoT devices. I use it frequently for this purpose (see https://snowpatch.org/posts/i-can-completely-control-your-sm...).
Any tips on good wifi chipsets that do not suck in AP mode?
Lots of "just use X" comments but the article is about showing the bare minimum/how easy the core part of routing actually is.
Also, if you have ever used docker or virtual machines with NAT routing (often the default), you've done exactly the same things.
If you have ever enabled the wifi hotspot on an android phone also, you've done pretty much what the article describes on your phone.
All of these use the same Linux kernel features under the hood. In fact there is a good chance this message traversed more than one Linux soft router to get to your screen.
Yeah I find it more interesting to see how it's built from scratch, then I can decide if it's worth doing myself or just using X. I think this is a good software principle in general.
Basically any computer is a router if you're brave enough.
Windows PCs had (have?) that Internet connection sharing feature for a long time. It was really just a checkbox to enable NAT too.
Sometimes I think combining a firewall/router/switch/AP/file server/etc into a device called a "router" really confuses people. Even people who should know better.
It is much, much easier than it used to be. The documentation and videos alone available make something like this a very welcoming learning experience that anyone can complete step by step by pausing a video and replaying it.
Like most things, really. I used to build routers from old PCs, but eventually those tiny appliances caught up with the performance/functionality I need.
You can do a lot of routing on a $70 Mikrotik, although they might not be "easy".
I really want to end up with one of these for at least a few months: https://mikrotik.com/product/rds2216
For sure, it's a path and passage towards devices like that.
Everyone has a starting point, starting with soemone has lying around is one thing.. the quicker they can get going the more they can get to leveraging the real power in most devices.
This really takes me back. My first actual 'use' for Linux was making routers out of leftover computers.
The perfect machine back then was a 100MHz Pentium, in a slimline desktop case. At the time, the Pentium III was the current desktop chip, so you'd have a pile of early Pentium-class machines to use. And even a 10mb ISA network card (3Com if possible) would have plenty of power for the internet connections of the day. But 100mb PCI cards were still fairly cheap.
Install two NICs, load your favorite Linux distro, and then follow the IP-Masquerading HOWTO and you've got internet access for the whole apartment building, office, or LAN party.
Eventually I moved on to Linux Firewalls by Robert Ziegler for a base to build on.
After that I started piling other services on, like a spam filter, Squid cache, it was amazing to get so much use out of hardware that was going to just get thrown out.
That takes me back, I had the same trajectory , getting a newspaper’s news room and offices online with a single computer sharing its ISDN connection. Think ours was also a 100mhz gateway 2000 computer or some such.
That snowballed into “we want a website do you know how to do that?” and. Well, no, but it had Apache available and I … figured things out enough to take the skills elsewhere.
Repeated the same trick with a place in Wisconsin, who initially shared a 56k dialup connection with all their dispatchers and were impressed the thing had stayed up for 900 days without even redialing. 90% of their work was done in an on-prem wyse terminal anyway, dialup used to do the job for email or googling an address.
27, 28 years later I’m still dragged in front of them once in a while to ask how they can accomplish something cheaply with Linux, bubble gum, paper clips, or whatever . The times and technology have changed, but not how cheap they are!
I briefly put a Pentium MMX 200MHz system in service a few years back to bridge my parents to their neighbor's WiFi (with consent of course) when their DSL line was down for a few days. I installed a PCI Ethernet and WiFi card, booted into OpenBSD, and amazingly it was fast enough to get them through the downtime. :)
Someone need to write a new book on Linux router.
The old one is getting really old now, nearly 25 years ago [2].
[1] Book Review: Linux Routers - A Primer for Network Administrators, 2nd Ed:
https://www.linuxjournal.com/article/6314
Inverted case here, my first real use cases for Linux was flashing routers with openwrt and doing fun stuff!
Hell, you could do this with a single NIC if you have a VLAN-aware switch.
Ha, that's very close to my story as well. I had a 166Mhz Pentium and it was all PCI cards and 100mbit by then. That was essentially the start of my career.
Reminds me of a Pentium Pro router put into a datacenter, two 2GB mirrored scsi drives, two nics, happily running a hardened pfSense, ran with zero issues for the better part of a decade.
It just wouldn't die.
The suspicion was because the electricity going to it cleaner than average, in a datacenter, the normal wear and tear on electronics may have been reduced.
Respect was paid at it's decommissioning to convert it into a vm, knowing it's luck, chances are it would still boot up and keep on running.
> The perfect machine back then was a 100MHz Pentium, in a slimline desktop case. At the time, the Pentium III was the current desktop chip, so you'd have a pile of early Pentium-class machines to use. And even a 10mb ISA network card (3Com if possible) would have plenty of power for the internet connections of the day.
I was doing the same. Router and firewall on old Pentium CPUs. I don't have these machines anymore but I still have HDDs from back then with post-it notes on them saying stuff like: "Linux firewall / HDD 120 GB". For whatever reason my HDDs adapter that can read just about everything doesn't have the correct pin out for those HDDs. Would be a blast if they were to still boot: at some point I'll just buy a compatible adapter and see what I can find on those HDDs. I was very likely also saving some backups there.
But really my best memory was years (I think) before 120 GB HDDs became an affordable thing, in the super early Slackware days, on a dial-up connection: I had a 486 desktop computer and I'd share the Internet connection to a very old laptop (!) using... PLIP. A printer cable and the Parallel Line Internet Protocol. Amazing hack: my brother and I could then both use Netscape at the same time and to us this felt like a glimpse into the future.
I've got one of those N100+10Gbit router devices with a handful of ports. It seems a pretty reasonable device with one of the router distros running on it, but it doesn't seem nearly as efficient as my ucg-fiber/route10 devices, and that wouldn't bother me except that I suspect the packet latency is significantly higher too. Those devices AFAIK have hardware programmable router chips, which means the forwarding is done 100% without the interaction of the main CPU, so there isn't any interrupt/polling/etc delays when a packet arrives, the header gets rewritten, the checksum verified and off it goes.
Anyone actually measured this? I see a lot of bandwidth/etc style tests but few that can show the actual impact of enabling disabling deep packet inspection and a few of the other metrics that I actually care about. Serve the home seems to have gotten some fancy test HW but they don't seem to be running these kinds of tests yet.
You actually don't even need two interfaces on the box if you have a managed switch. It's not too difficult to configure your only interface as an 802.11q trunk port, and then you can use the managed switch as a sort of "interface expander". This is referred to as a "router on a stick" configuration, and it's how my home network is configured. Plus, if it's a PoE managed switch, you can install some cheap enterprise surplus Aruba IAPs around the house for Wi-Fi which is a lot higher quality than a consumer router or a mesh setup.
My home router was an old Thinkpad for a while, but then I switched over to a slightly newer Dell Optiplex that my work was throwing out. The plus side of that is that the i7 is total overkill for routing so I can also have my "router" run some VMs for network services and cut down on the number of boxen in my homelab rack.
Alpine is a great distro for this.
I have actually been curious about this: How good can a WiFi mesh get latency-wise, given the right equipment, and how close would a consumer router setup be to that, do you happen to know?
I’ve been using OpnSense/pfsense [0] for years and would highly recommend it. It has a great automatic update experience, config backups, builtin wireguard tunnels and advanced features like packet filtering options via suricata.
When I am doing network management on my weekends, I’m so glad I’m not stuck in the Linux terminal learning about networking internals and can instead just go to a webui and configure my router.
0: https://opnsense.org/
I agree on principal, but I often find that the GUI abstractions don't always map to the linux tooling/terminology/concepts, which often ends with a head bashing against the wall thinking "this is linux, I know it can do it, and I can do it by hand, but what is this GUI trying to conceptualize?!?!"
I was recently introduced to a Barracuda router, and bashed my head against the wall long enough to discover it had an ssh interface, and linux userland, and was able to solve my immediate problem by directly entering the commands to get it to [temporarily] do what I needed. (Of course, using the GUI to reapply settings wiped my manual configuration...)
I've used pfsense, OpenWRT, Barracuda, Verizon's OEM router (Actiontec) and they all represent the same functionality wildly differently.
> I've used pfsense, OpenWRT, Barracuda, Verizon's OEM router (Actiontec) and they all represent the same functionality wildly differently.
Worth noting that pfSense (and OPNsense) are not Linux-based, they're based on BSD, specifically FreeBSD. While it's possible to have standard router OS web UIs that are cross platform, the underlying technology is different, so it's not really a surprise that there will be differences in how the devices running these OSes are configured.
Not much different. FreeBSD's pf is a port of OpenBSD's pf, and nftables are heavily influenced by them.
At this point I rather doubt the sanity of people still sticking to iptables tbh.
So there is approximately one concept of "packet filter done right". UI madness is on UI authors.
The primary reason I stick to iptables instead of nft is that I already learned iptables decades ago, and some software I interact with still defaults to iptables and/or does not have full support for nft.
Why do you doubt the sanity of people sticking to iptables? What makes nft compelling?
My main reason is that nft applies configs atomically. It also has very good tracing/debugging features for figuring out how and why things aren't working as expected.
That said, I think many distros are shipping `iptables` as the wrapper/compatibility layer over nft now anyways.
as somebody that's not a network engineer by day and has barely grokked iptables, could you recommend some resources for learning nftables ?
> nftables are heavily influenced by them
Are they? I recently had to learn nftables and they seem to be iptables but with a slightly nicer syntax and without pre-defined chains. But otherwise, nftables directly maps to iptables and neither of them seem similar to pf.
I guess I'm different. I typically want my router/firewall/network services box to Just Work. I've made a career in deep-in-the-weeds system administration and engineering. Having to hunt down man pages, examples, tutorials, etc for the dozen or so fiddly bits make up a modern Linux- (or BSD-) based router was fun the first time, not so much the 10th. Been there, done that, got the t-shirt.
I will concede that the OpnSense UI is far from perfect. I would really like to see a device-centric view that lets me set all the things related to that device from one screen (or possibly one screen with multiple tabs). For example, if I add a Roku device to my network, I want to enter in the MAC address and then be taken to a screen where it will let me set the hostname, pick a static IP address, hand it a specific DNS resolver IP, see all of the traffic going to/from the device, only allow it access to the Internet between during certain hours, etc. All of this currently requires jumping around between multiple disconnected parts of the OpnSense UI.
I feel almost exactly the same as you on the subject. When I was young and starry eyed I built my own router out of a PC running openBSD, all by hand. Nice learning experience, interesting OS, but definitely not maintenance free especially around system updates as back then openBSD packages and sys upgrades required recompiling everything. Now I do the same mini-PC thing as the OP's article but I just put OpnSense on it. Agree the UI can be maddening at times but the thing is rock solid, and has very polished update and upgrade mechanisms. Built-ins/plugins are great - unbound, wireguard, openvpn suricata, backups to git etc. Also I like that it is BSD based, my network experience was learned on Cisco's and Junipers in an ISP setting and Linux networking has always driven me crazy
I'm at a stage where I don't want to be doing network management on my weekends. I have a Ubiquiti router that's pretty good, and for my router I'd like something like TrueNAS for my NAS, a distribution that completely turns the hardware into an appliance I can configure once and forget about.
Is there something like that?
Pfsense/opnsense would be one option (based on FreeBSD). For Linux there is OpenWRT, which you can either run as an alternative firmware on quite a few consumer routers/access points, or install on a PC or Pi or similar.
Caveat: I have only used OpenWRT on a high end consumer router (GL.inet MT6000) out of those. That works well, anything else is based on reading about people using those options.
For all of those, once you set it up you don't really need to do much except install updates a couple of times per year, or if you want to forward a new port or such.
Nice, thanks! I had an OpenWRT router back in the day, but it had no Web interface. I'll try OPNsense, thanks.
OpenWRT has the LuCI web interface you can add to it (often included in installs) which lets you do pretty much everything.
I recently dumped opnsense because they took a stand against a few things I was trying to do (ex, webUI on wan port IIRC) which make sense at a high level. But I _HATE_ devices that think they know better than me. I was trying to configure it on a _LAN_ such that the identified WAN side was actually my local lan, and I spent an hour hacking it to work and was like "you know if they can't get this shit right i'm out". There are a lot of places in the technology world where someone who thinks they understand my use case makes a decision based on some narrow world view because they can't understand that not everyone trying to use their product is some idiot home user using it for their home network.
Yep, this is the way. You will learn loads using Linux but this is not something you want to go wrong.
I used a lower power Intel Atom mini PC with an additional NIC as a router for years. I tested it and found it could route around 300Mb/s which was plenty.
But then I got gigabit internet. So I bought an Intel 4 port GigE card from eBay and now run OPNSense as a VM. If you get the right Intel card you can pass through ports to VM individually, which is nice for playing (don't know the exact details but look for cards with virtualisation support, mine is an 82575GB I think).
To be fair, my setup still probably has too much to go wrong, due to the VM thing, but I just haven't got round to getting dedicated hardware, and it's worked fine for a couple of years now.
This seems like it might be a good place to ask: does anyone know of a low-cost, readily-available SBC box with built-in dual Ethernet interfaces?
I've been very interested in some of Radxa's boards in the ~$30-70 range, like the E52C [0] and the E20C [1], but they don't have many distributors and seem to have stocking issues [2].
[0] https://radxa.com/products/network-computer/e52c/
[1] https://radxa.com/products/network-computer/e20c/
[2] https://shop.allnetchina.cn/products/radxa-e52c?variant=5034...
Banana Pi makes SBCs with lots of networking ports.
Not sure if this counts as low cost... https://store.minisforum.com/products/minisforum-ms-r1-works...
Quite a bit beefier and pricier than I'm looking for, yes! But thank you anyway.
I'm curious about the policy rationale behind banning router imports. If a government were considering legislation like that, what would the primary concern usually be? Given that so much internet traffic is now protected by TLS/SSL and other encryption, why would it still matter if citizens were using routers that might be backdoored?
Is the concern mainly things like botnets and DDoS activity, weak default credentials on network equipment, or compromised business networks where poorly secured routers or attached NAS devices could expose sensitive or proprietary data? In other words, is the concern less about decrypting traffic and more about using the router as a foothold for surveillance, disruption, or access to poorly secured internal systems?
Among policy and security people, the term they bandy about is Advanced Persistent Threat (APT). They're not wrong; there are a number of recent cases, and these are ongoing, and you've heard of some of them: Volt, Flax and Salt Typhoon and Velvet Ant. There are more you haven't heard about, because only the operators know they exist.
These are networks of controlled devices. They're hard to eradicate, as shown by the fact that they haven't been eradicated: they're still active and being used to compromise systems, including defense and intelligence systems, power systems, financial systems, identity systems, etc.
Is banning foreign gear going to fix this? No. Security isn't a product. It is, however, a process, and in a process you take steps. I think this: we (individuals and institutions) enjoy tremendous liberty in the use of communications equipment in the US and most of the West. Taking that for granted is a mistake. If part of keeping this means the US has to spin up a domestic supply of network gear, or carefully modulate where such gear comes from, then lets do that. Otherwise, The Powers That Be will leverage its concerns into far worse steps.
It's everything you mention in the second paragraph, and additionally just the ability to turn them off.
Imagine everyone had their routers disabled simultaneously. I don't know if the cell networks could function with the surge in standard traffic that would happen, and then you've effectively plunged all or part of the country into a communication blackout.
I think "turn it off permanently by bricking it" is almost as bad as "leverage for DDoS".
I worked on Bot Mitigation at Amazon, and we once saw a ton of traffic that was heavily distributed amongst consumer devices world-wide, but surprisingly in the US too. We suspected compromised routers that were using the home page as a health check. There was a lot of investigation I did, and the short realization after talking with the network engineers is that the amount of traffic, and distribution of sources, would be impossible to stop. There merely isn't enough bandwidth in the world to stop so many residential device if it hits a specific target. To be clear, this was coming from less than half of active Amazon customers, not everyone in the US.
Anyway, it wasn't routers, but it was a consumer device, and it wasn't nefarious, it was incompetence (in code), as usual.
>Imagine everyone had their routers disabled simultaneously. I don't know if the cell networks could function with the surge in standard traffic that would happen, and then you've effectively plunged all or part of the country into a communication blackout.
IME cell networks definitely can't cope with a loss of all routers in an area, given how mobile data becomes basically unusable when there's a power outage. That said, "everyone had their routers disabled" is probably not realistic, given that there are plenty of non-chinese router vendors.
The FCC Chairman is sucking up to the President.
If this were really about computer security they would follow California’s example of requiring unique passwords. Maybe make manufacturers liable for not patching known remote exploitable security vulnerabilities. It doesn’t matter if the source of a DDoS is a Huawei box or a Netgear box.
There are a few reasons
- Access to data (dns/ips, domain names (if not using ESNI), amount of traffic, etc) of sites you are visiting
- Access to the inside of your network where it can attack machines that may not be secure
- DDoS
- The ability to shut down your internet
I'm sure there are more.
> is the concern less about decrypting traffic and more about using the router as a foothold for surveillance, disruption, or access to poorly secured internal systems?
That should probably be the technical concern. Even if you have traffic protected by TLS, you still typically have enough metadata to cause some problems for users individually, but the assumption that foreign equipment is back-doored by some security service or other is probably safe.
The policy rationale is the Trump admin takes bribes to permit router imports. No different than how various companies won tariff exemptions.
That, and like drones, maybe one of his kids starts up a router company which becomes the sole company allowed to sell routers in the US.
“Just use OPNsense” is great advice for production, but terrible advice for learning. This article is valuable precisely because it shows how little magic is actually involved in routing.
SOHO toys don't do routing in a real sense at all
Something I did not see in the article are router specific tuning such as
in /etc/sysctl.d/10_router.conf to slightly reduce overhead when being used primarily as a router. There are many other router related knobs but those I would always set especially if trying to reduce overhead for VoIP/Gaming setups. There are many other knobs I tune such as gro_flush_timeout and napi_defer_hard_irqs, sch_cake tuning, lowat and output limits and hundreds more but those rabbit holes would require a large write-up. My overall goal is to give family members latency, jitter and throughput numbers that improve their quality of life and gaming scores of course.Such things do not preclude additional tuning on the client and server sides as well but those are even bigger topics.
How much does disabling early_demux actually matter? I do this too, but honestly I've never tested what difference it makes.
I've always found fq_codel to be good enough for gaming through my router. But I have a 10gbps uplink.
Hmm I've always had a manually configured low power generic box as router.
But I've never even tried to set up my own access point, I just pay Unifi for that [1]. The software part is doable but I don't want to learn to handle the signal issues.
[1] Switched to Unifi in anger after my first consumer level 5 Ghz wifi needed reboots weekly because it was overheating. Do yourself a favour and get the semi pro stuff, Unifi or others.
I've been running a custom router for about a decade, but I too have haven't tried handling the wifi on my own. It's always been easy to get an external access point and there's a bit of a guarantee that it's done correctly.
I kind of feel like that's cheating though; I've outsourced the hardest part of the project to someone else. Maybe one of these days I'll take an old NUC or something and buy a decent wifi antenna for it and try and do it properly.
[1] Initially pfsense, then OpnSense, then ClearOS, and now some custom firewall rules in NixOS.
So it's been awhile but the best and simplest way I think is use an access point. I don't want my wireless gear doing routing. From a logic stand point they acts as wireless "bridge" to the physical network, and nothing more. DHCP, etc. stay handled in one place for the entire network, back on the physical router.
> While it may seem appealing, I would highly recommend against installing loads of software right on the router, and instead forward traffic to a device in a DMZ or VLAN.
Why not? I use an old gaming PC as a "router" (machine exposed to the WAN), and run dozens of services on it besides the firewall/NAT (iptables). Among others: email, Web server, multiple game servers, and many internal services (DNS, hostapd, loads of Docker containers).
You need more careful firewall rules on any device with IP forwarding enabled, and it can be hard to remember exactly when forwarding, NATint, etc. happen with relation to the incoming/outgoing firewall rules.
E.g. is your pf-based load balancer running its rules before or after the global filtering rules? And if they're running first are they SNATing incoming traffic so the LAN rules allow the traffic through or does it need explicit exceptions for external IPs to traverse to a LAN endpoint?
If you're comfortable with more advanced networking then it's fine to run it all on one box. If you just want to open ports for internal LAN services then that is a very canned and well-supported feature for a gateway firewall.
E.g. see AirSnitch which resulted in large part from mixing too many complex networking rules in single devices.
A classic example is Docker inserting its firewall rules before everything else, causing any "published" ports to be wide open even if a firewall ruleset was configured. They might have fixed this, or doubled down on their design choice. Either way, that kind of complexity can really bite you if you're not careful.
https://docs.docker.com/engine/network/#published-ports
Yes, this is a well-known pitfall. It's important not to bind containers to 0.0.0.0 for this reason, unless they are actually intended to be public.
> Why not?
It seems like you weren't really asking, but I'll answer anyway.
It's bad security practice, and opens up your network to attack and/or compromise, you're massively increasing the attack surface, and a compromise of one of those components leaves the attacker sat on your edge router, at which point your entire network is fair game.
Generally speaking you shouldn't expose anything on your edge router / firewall, it's a safety barrier.
You can sit things behind it in a "DMZ" and port-forward and isolate them etc so that there's no packets terminating on the actual edge device itself.m, that lowers the risk of a full network level compromise.
Chances are you might be fine and never have a problem, but it's still recommended against.
It was a genuine question, and while you reiterate the author's point about this being "bad security practice," neither you nor the author explain why this is the case.
I don't believe physical separation really buys you much here. At most, if may reduce downtime if you do indeed get pwned, but I think that you can achieve the same objective through a combination of containers, VMs, and UNIX users. And running multiple, somewhat redundant machines also has obvious downsides such as increased power consumption, increased maintenance burden, additional space and cabling, etc.
I recommend replacing hostapd with an enterprise access point plugged directly into an ethernet port on the router. Most support VLAN tagging based on SSID, so you can still set up different subnets and firewall rules for the different SSIDs.
As much as I love hostapd... the performance using commodity hardware has always sucked for me. I can get 150MB/s over wifi with my proprietary AP!
Maybe someone in this thread has a couple of ideas:
What’s the simplest way to spin up a simple „cattle, not pet“ routing VM? I don’t want to mess with any state, I just want version controllable config files. Ideally, if applying a version fails, it would automatically roll back to the previous state.
OpenWRT seems like it fits my description most closely, but maybe someone here is a fan of something more flashy/modern.
NixOS using https://github.com/thequux/nix-zone-firewall/ worked well for me for many years. I only stopped using it because my poor embedded Linux machine started having issues and it made more sense to go with a Mikrotik than to buy a new device to run as a soft router.
That sounds like you might like VyOS. I found it to be relatively easy to achieve exactly what I wanted, but went back to a GUI as it turned out I wanted a pet and not start a farm.
> but went back to a GUI as it turned out I wanted a pet and not start a farm.
This made me chuckle, I'm definitely going to quote this the next time our K8S cluster has issues
I recommend Pfsense or OpnSense if your hardware works with a FreeBSD-based thing. They're super easy to set up and don't have many surprises.
After I upgraded to a 10GbE ethernet card in my previous router, my card didn't work correctly with FreeBSD-based stuff anymore. I changed to ClearOS and that was actually comparably easy to Pfsense...maybe even easier? I recommend checking that one out.
While I am a linux advocate for networking in the current day outside of hyper specific CDN use cases (a la netflix)... its pretty common for people to just virtualize opnsense/pfsense to take advantage of linux network drivers. Especially if their actual routing requirements are modest and dont require full use of the hardware.
Beyond getting support for devices completely absent on freebsd, quality of drivers, bugs much more rapidly squashed, and general misc features absent on the bsd side like NBASE-T.
This is the kind of low quality information you see on fanboy forums. There is nothing special about Linux drivers and anyone can go look at them. A lot of hardware uses a HAL and there is a smaller OS adaption therefore most of the code is similar across OSes.
Virtualization means you now have multiple layers of drivers and privileged code in the mix to add and amplify bugs, it can and should work but if you are doing this in the name of stability that is a bit curious.
The reason Netflix can do what they do is they have good relationship with their HW vendors, NVIDIA(Mellanox) and Chelsio. If they were on Linux, they'd need the same level of support.
I don't know enough about this level of IT to rebut this.
I use Linux for my router now because my server is NixOS, so I was able to consolidate my router into my server and turn off a machine (and thus save a little power), and I have so thoroughly drunk the Kool-aid for NixOS that I kind of want to put it everywhere. I run the latest kernel and I update daily, so I think most bugfixes (and hopefully security updates) will manifest quick enough.
OPNsense. I use it on dell optiplex SFF for about 8 years. Was never tempted to use VM for routing, but many do.
Version control is in the GUI, you can adapt for your needs the number of changes you need. automatic config.xml backup also possible.
OpenWrt has a generic x86 PC build that can also be used to turn basically any random PC into a router, complete with an operating system actually designed and developed for that purpose.
And of course probably 1000x the power usage compared to the average off the shelf router that runs off a borderline microcontroller.
good point! I think you can run it on a PI though.
Otoh it would make sense if you could combine it with a home server, then it's just a side process and you actually save power by not having an extra device.
Though you'd still need a switch or two. And a fiber modem which already has a router and a switch built-it. Oops.
OpenWRT is great if it fits your use case. If one has reason to stray from the happy path a disadvantage is that the OpenWRT uses a single binary like Busybox and doesn't use glibc. This is great for embedded/low power machines like the OG WRT54G, but not as optimal for when you have an entire random PC. I don't recall the exact things I was looking for but I moved on to pfSense and didn't look back.
> This is great for embedded/low power machines like the OG WRT54G, but not as optimal for when you have an entire random PC.
There are steps in the middle :)
I'm running OpenWRT on the recent WRT3200ACM and it's going beautifully.
Alpine Linux too.
I wonder if consumer routers will end up being built in a trivially-not-a-router configuration - something akin to a pull-tab or turn of the screw that closes a circuit, transforming the device from legally something else into a router after it's purchased.
What a dumb timeline.
"this here is a virtual network appliance, so called because it doesn't have any ports on it - wait, why are you taking off that blanking panel? That's illegal!"
There's a famous Polish song "Mój jest ten kawałek podłogi", written in times of Soviet influence, about a man building wall around his home, but later he gets hungry and there's no exit...
Anyone with translate.kagi can find it and translate
Pleasant thing about routers that is is so simple to build one after learning basics of networking and pretty much any OS or distro can act as one. There are obvious choices like OPN\PFSENSE, OpenWRT, DD-WRT, FreshTomato, but literally any PC with a single Ethernet port can act as one. My favorite setup was a laptop running Ubuntu and the whole router setup was in a single netplan file + dnsmasq for DHCP.
Edit: And ofc best cheap device imo is OrangePI R1 LTS and a whatever usb wifi dongle. Came in clutch a few times, such a nice little device.
Been using DD-WRT for years. Current setup is a $50 Dell Optiplex i5 from ebay running x86 DD-WRT. I put an intel 4x 1Gbit NIC in it, and it's been an excellent router for years.
Just ensure the firewall appliance thing you buy has I226 intel chipset not I225
The first two versions of 225 have packet drop issues and it’s unclear to me whether v3 third time lucky fixed it. And getting the stepping info out of aliexpress supplier is hard so 226 is safer
Love the "An ExpressCard-PCIe bridge in the ThinkPad’s expansion bay".
Would you have a picture of the ExpressCard laptop connector?
It would be something a bit like this: https://www.ebay.com/itm/115721630079
Before Thunderbolt was common, people attempted to use external GPUs with this sort of expander, but it worked really poorly.
I did this back when, just using a 100mbit NIC express card.
Ran openbsd for a few years like that, the base OS included everything needed. I recall it used 24MB of ram and closer to 30MB if ssh'd in. It was very handy to have a local login when playing with firewall rules.
nftables syntax is pretty tough to read. I wonder why they didn't go for an easier to read DSL. I do understand it's likely super fast to parse though, and has a 1:1 relationship to its struct in the kernel.
I personally stick to iptables. nftables does not seem to be an improvement at all. iptables is terse but logical.
I’ll pick nftables over iptables any day, it’s leagues better (granted, it’s not hard). The nftables wiki is great, as the syntax and modules are documented in a single easy to read page.
As an added bonus, you get atomic updates of all chains for free.
Granted, for simple usecases, ufw or firewalld may be simpler though.
Definitely an upgrade over iptables. I kinda miss ipchains though.
You can still use the iptables interface for nftables rules if you'd like, but I think you miss out on things like atomic application of rulesets, ranges, lists, and variables (not shell variables).
Anyone has done mesh WiFi (ideally triband) using off the shelf parts and Linux?
I have an Orbi AX system which works reliably, but now I want to upgrade the radio to WiFi 7 and that means I need to upgrade all the hardware.
Hoping to move to using off the shelf parts so in the future I can just change the radio (ideally bunch of USB sticks).
I understand this is not strictly just the router. I can (and used to have) a router as separate device, but any mesh WiFi right now that I can find need a pricy router that acts as the coordinator, essentially negates the economic benefits.
That's a bigger can of worms than you might expect. Most consumer WiFi chips only barely support AP mode, and I'm not aware of any that can do multiple bands simultaneously. You'd probably need 4 adapters on the repeater for triband. One to connect upstream, one for each downstream band. Three instances of hostapd all configured with the same SSID and auth for each downstream interface.
Then there's the roaming issue. This is largely what the commercial "mesh" systems try to solve: deciding / helping inform when clients should switch APs. There are many solutions and none of them are without issues, including the commercial ones. Here's a starting point: https://openwrt.org/docs/guide-user/network/wifi/roaming
Super interesting, thanks for sharing
Openwrt guys were cooking up a wifi 7 router I think. Think that’s best bet but Not super close to it though
Can anyone recommend a good, energy-efficient, inexpensive dual-NIC SBC or miniPC? Last time I looked into this there were not many good options.
I've gone through quite a few embedded devices for exactly this use case. So far I've used:
- Soekris net4501 (x86, 486-class CPU) (discontinued)
- PCEngines alix2d3 (x86, AMD Geode LX800) (discontinued)
- PCEngines APU (x86, AMD T40E) (my current router/firewall) (discontinued)
I'm also currently using an APU2 as one of my wireless access points (with hostapd).
All of these have been solid machines that have given me zero problems.
The next system I plan to use is going to be a Banana Pi R4 (ARM Cortex A73), it's a solid choice for a simple router/firewall/DNS/DHCP box. It has a built-in 4-port gigabit switch where each interface can be used as normal Linux interfaces, as well as 2 SFP+ ports that are capable of supporting up to 10 gig ethernet.
It's also one of the few systems that offers true hardware offloading for connection tracking, so things like netfilter flowtables don't have to use any main CPU processing.
I'm currently experimenting with a Banana Pi R4 as a Wifi7 access point (running Debian with hostapd), however the current state of the wifi7 module for it (BPI-R4-NIC-BE14) and Linux driver (mt7996e) is still pretty young and a bit buggy (i.e., limiting transmit power to 6 dBm without patching the driver to override it, and there's apparently a lack of RF shielding which can contribute to low SNR on the receiving end). With the proper patches in place it makes a decent Wifi 6 access point. I'm hoping these issues get ironed out in the future and I can use it as a true Wifi7 AP. frank-w is doing outstanding work to help support the open source community with this new hardware.
It's hard to recommend one thing because there are so many options and they all have different trade-offs in terms of initial cost, ease-of-use, reliability, performance, etc.
A year or two back, I was able to get a brand-new fanless Intel N150 with 4x2.5G ports with 16 GB memory for about $150 from AliExpress. I run Proxmox on it, with OpnSense and a couple other things in virtual machines. These days, due to tariffs and the memory shortage, that is more like $440 now, unfortunately. I am kicking myself for not buying two, not so much because of the price increase, but because it would have come in handy multiple times to have a second one on-hand for random experiments.
Given that CPU performance does _not_ tend to be critical for firewall/NAS use cases, if I had to replace it tomorrow, I would go onto eBay and get the highest-spec'd used Dell or HP mini workstation I could find for $120 and plug in a USB3 1gig ethernet dongle for the WAN side.
A used Sophos XG 115. Has Intel Ethernet interfaces which is preferable for BSD compatibility. 8W idle. I power it off a 802.3af to 12V splitter.
If you want maximum speed a Lenovo Thinkcentre m720q has a desktop Intel CPU and a PCIe slot. You can add a 2x SFP+ NIC and PCIe riser to get 10G.
I am truly sorry. I can't understand the physical networking from the pics or the description... I'm probably just missing something. There is one blue plug going from the laptop to the cisco switch or the pci wifi module? I see a blue plug going to each device. So I'm guessing everything is plugged into the cisco switch?
if you could show all the wiring and label it (according to the table below) i think it would add a lot of value for someone less familiar with these kinds of setups (like me)
Hey, op here, this was almost a decade ago, but I'll try to describe what's going on here. It's kind of a crappy picture.
* WAN connection comes in by coax, into my cheapo cable modem (off screen), and then by Ethernet into the franken-NIC sitting on top of the laptop.
* The NIC on top is a normal PCIe card, but with the bracket missing. The ExpressCard riser [1] is connected by a mini-HDMI cable, the flat black cable, which curves up, around, and back in from the left side into the laptop
* Then, the blue cable on the side of the laptop is a VLAN trunk going into the Cisco switch on port 23/24, outside the picture.
* From there, another port on the switch is setup as an access/untagged port going into one of the LAN ports on the D-Link acting as the access switch
I don't think it was set up here, but at one point I also had a dock under the ThinkPad, with the serial adapter wired up to the switch's console port so I could manage everything by ssh'ing into the router.
[1] https://www.ebay.com/itm/115721630079
Also note that all the cables were hand-crimped because I was too cheap to buy new patch cables at the time.
I was in college, and truly had more time than money back then. it's the kind of doohickey made by only somebody very young, very crazy, or a bit of both. ;)
Routing is pretty easy for most use cases... firewalling an Internet connection, on the other hand, is just about impossible (thanks TLS 1.3) without pretty serious overhead, 3rd party maintained live subscriptions, TLS interception, and a willingness to say "no" to a lot of the shenanigans that modern programs and devices try to pull.
I recommend the free home version of Sophos for the least painful way to do it. Buy a Palo Alto with a full subscription if you are really serious.
When I got started, the NSFnet backbone was a bunch of IBM RS/6000 systems with comms cards. There were no routers.[1]
[1] https://www.rcsri.org/collection/nsfnet-t3/
Try it with this! https://www.dewalt.com/en-us/product/dwp611pk/dewalt-1-14-hp...
Surely something like OPNsense/PFsense would be better for the average user than setting up all the software manually?
Not necessarily. For one, the BSD has, or at least used to have much worse driver support for wireless adapters. With a regular server/desktop Linux distro, it's also easier to run other services on the same device. For example, nothing prevents you from running nginx and hosting a website, or a personal email server.
I appreciated learning what's involved, though.
In this day and age, if that's what you're after, you can just point an AI at the problem and give it shell access, and it'll just do what you describe (Claude code, codex, etc).
Here I was thinking this article would tell me how to turn my unmanaged switches into routers, but no, "anything" actually means "any fully featured general purpose computer with networking".
I suppose if you manage to get OpenWRT or something onto your switch you could use it as a router.
That's theoretically possible but a bad idea for a managed switch, because they seldom have enough CPU performance or IO between the CPU and switch silicon to provide respectable routing performance. For an unmanaged switch, it's more likely that whatever CPU core is present (if any) doesn't have enough resources to run a real network stack.
Anyone know how necessary UPnP is? From what I can tell, this setup does not run UPnP for automatic port forwarding
You'd need to run a daemon like miniupnpd to enable UPnP, however I'd avoid it if you can, as it can be a significant security risk.
Not necessarily at all and a huge security risk.
I live in the SF Bay Area, and ended up with Sonic Internet, and a 10gbe connection. Routing this with anything off the shelf was going to be "very expensive".
I ended up with an Opnsense box. It's an m920q (i5-8500), with riser card and a dual SFP+ nic in it. All in, it was less than 200 bucks (now it would be closer to three). I ended up with a cheap, Chinese "media converter" (from aliexpress because the same thing on amazon is 3x the price) that just had two SFP+ ports on it. That let me go from an SPF+ copper ethernet module to a DAC and not dump a bunch of heat into the 1L pc.
I have to say that the functionality made it a worth while investment: traffic shaping, wireguard and the like have been mostly a joy. And the documentation for Opnsense made the setup and use (mostly) easy.
I did kind of the opposite, I made my main beefy gaming computer the router, then connected to it a nice wireless AP in bridge mode to serve internet to the rest of the computers. That way I can have a local llm agent manage my network and firewall by simply asking.
We are just scraping the surface here; let's imagine a really easy to use and install bit of router software that includes all kinds of p2p bells and whistles.
The extreme difficulty of setting up networking and routers is (obviously?) a weird endgame result of how companies and safety and capitalism and restriction intersect* and given the relatively insane regulatory ideas we're seeing these days, time for another look at all of this.
*edit, and not, e.g. an inherent property of "networking technology," it does NOT have to be this hard.
Qotom mini PCs are my cheatcode. These little PCs are often available with multiple NICs, and I use one as a wifi bridge/router for my office network. Put Linux or FreeBSD on one and you have a very capable little network-appliance box.
Nobody I know makes routers and more importantly WiFi combo AP in the US except for high end corporate stuff - even the US only cable modem stuff from Comcast are Chinese OEM ???.
Some more idiocy from the FCC chair.
SpaceX manufactures network gear (Starlink satellite internet terminals) in the US. Bastrop, Texas, specifically. Those phased array transceivers, with heaters, ethernet, GPS and WiFi are way more difficult to make than the typical indoor WiFi router.
Great writeup. One advantage true routers have e.g. Edgerouter or Mikrotik are dedicated hardware for IP & TCP header processing. Some can offload AES for VPN encryption. This leads to cooler temps, lower power utilization & longer life.
I encourage everyone to run a hardware router. A cheap dedicated wired router can be had for $50. Run PfSense or the vendor firmware . It’s very rewarding. Also a long term investment since routers tend to last for many years while wifi standards are revised every year or so .
A fun project that results in a unique and stylish router is repurposing a Mac Pro Trashcan. They can be picked up for a few hundred dollars, offer dual 1GbE Intel NICs that work natively on Linux, and have plenty of CPU and RAM overhead. Throw OPNsense on there and you’re off to the races.
The idle power usage on those is atrocious compared to say an Intel N100 or an Arm system.
Performance per watt is not ideal on the trash can. But totally doable.
> sudo systemctl enable [email protected]
:-)
Let me guess, ".*@.*\..*"?
if fancy a bit more of capability, dockerized opnsense and just play right with your vlans. One cable is enough into your switch...did I said managed... and your opn/telco eth exit.
I think the NanoPi range (https://www.friendlyelec.com/index.php?route=product/categor...) has great hardware for making your own router. They support various flavours of Linux, including OpenWRT (or at least their branded version called FriendlyWRT). I like the NanoPi M5 model as it supports using a NVMe so it can happily run some Docker containers on it. The case is really well made and uses passive cooling, so it's ideal for a router.
It becomes harder if you try to do it with 10 Gbps. Most CPUs struggle with it without dedicated accelerator chips.
Absolutely not. My 8700k is able to saturate a 25Gbps link without breaking a sweat. It could still push more than 4Gbps of encrypted traffic (Wireguard tunnel).
Im running a Beelink EQ15 + OpnSense to do something very similar.
Does routing on Linux have any hardware acceleration for IP packets?
Yes. You can take advantage of Netfilter's flowtable infrastructure and if you have the right hardware (NVIDIA/Mellanox ConnectX-5 or MediaTekMT7621) it will actually offload the processing of these packets to the NIC hardware. This only applies to established connections, however, but that typically accounts for like 95% of the traffic passing through.
you might as well just use vyos.
A router only really needs one network interface.
Any computer with a single network interface, maybe even an (old) laptop, can be used. Anything x86 from at least the last 10 years is energy efficient and fast enough to route at gigabit speed. If you don't care about energy usage, any x86-based computer from the last 20 years is fast enough.
The magic trick is to use VLANs, which require switches that support VLANs, which can be had for cheap. VLANS also allows you to create separate isolated networks for IoT or other 'less secure' or untrusted devices.
I’ve always made my own routers by using low-power devices running Linux (Debian) with IPtables and now NFtables.
No special router OS or software required.
Highly recommend.
P.S. that single network interface is very likely never a bottleneck because network interfaces are full-duplex. Only when your router is also your file server (not recommended), internet traffic and file server traffic could start to compete with each other.
It only needs one port, but for most simple networks two ports on the router means less configuration.
The "router on a stick" paradigm using VLANs to a share a single physical port is perfectly valid. You're creating a "now you have two problems" scenario in which you need a VLAN-capable switch and have VLAN configuration to make.
I typically like the ISP router on a dedicated router port to make monitoring the physical link and/or cycling the physical link easier.
Unless your ISP is >1Gbps adding a second port to most devices is as easy as adding a USB NIC.
Technically you can route without isolation, but VLANs are definitely a good idea if you’re using a single port.
There are 2.5 Gbps, 5, and even 10 Gbps USB NICs these days, although 10 Gbps ones are pretty expensive and require really recent USB ports.
I agree I want my local network and my WAN port separate, if for no other reasons than so I can use ssh to get into the router from my LAN with the WAN port disabled.
Yes, but some folks are wary of using the same physical port for external and internal traffic. Fears of "VLAN hopping" remain, even if unfounded. Also, you'll hit a performance wall since you are sharing a single gigabit port between external and internal traffic. Obviously may not be an issue for many, but if you have gigabit fiber...
I have gigabit fiber and none of this is an issue.
VLAN hopping is only possible due to misconfiguration. I'd like to be proven otherwise if that's not the case. VLANs are used EVERYWHERE where it matters. And no, the single port is absolutely not a bottleneck because the port is full-duplex.
I agree VLAN hopping is not possible without misconfiguration but it still is a "concern" for some. I also make extensive use of VLANs on my home network.
If you're trying to push close to a gigabit up and down simultaneously that single port will become a bottleneck. I agree for most typical use cases it is not a concern.
The bottleneck exists, but is a non-issue for most home use as most consumer connections are wildly asymmetric, usually biased towards download.
What happens if one node on your network is downloading at 1Gbit and another is uploading at 1Gbit?
Both get 500Mbit.
Bottleneck.
That's going to be super rare. If it's just LAN traffic it shouldn't hit your router at all and you won't have the bottleneck issue. The actual cases would need to be quite contrived, like you're backing up your media library at the same time you're updating cod warzone.
Sounds interesting. I always wanted to use a Raspberry PI as router (to have one as backup in case the OpenWRT Linksys goes down), but couldn't wrap my head around properly how to overcome the single network port (I think the usual recommendation is to use an extra USB network card/adapter). Can you elaborate more about this VLAN stuff (you would put your modem, your router, and all your machines on the switch... and in the switch you tell the router connection to double use the connection for WAN and LAN separated via VLANs? And put the modem into the "WAN VLAN" too?)
Ideally the PI also should to what the extra DSL Modem does… but I guess that's where the dram must stop. :D
When you setup your single ethernet port (let's call it eth0) as a VLAN trunk port, you'll get the ability to configure multiple virtual interfaces off of it. How many virtual interfaces you get depends on how many VLANs you want to tag traffic for. For example, if you have 2 VLANS with ids 100 and 200 (100 being your public Internet-facing traffic, and 200 being your LAN traffic), you would then have interfaces eth0.100 and eth0.200 to work with that you can then use in your firewall scripts as if they were two separate, physical interfaces.
This of course means you need a VLAN-aware switch that this single ethernet port can plug into, configured as a VLAN trunk (in Cisco terms) port. You would then want to configure one of the other switch ports as a VLAN access port assigned to VLAN 100 (untagged). This is the port you would plug your cable modem into. Then (in the simplest example) you could assign all the rest of the switch ports to VLAN 200 (untagged), and you would plug all your LAN devices into them.
The TL;DR is to have two vlans on the cable from your switch (called a "trunk"), "lan" and "wan", carrying the respective LAN and WAN networks. Then, on the Pi, create two vlans on the underlying Ethernet interface. Then those two VLAN interfaces can be configured just like the LAN and WAN interfaces of the router. On the switch, you’d dedicate one port to the WAN by adding it to the WAN VLAN without tagging, and the other interfaces do the LAN VLAN, also untagged.
I've created a blog post about this: https://louwrentius.com/raspberry-pi-as-a-router-using-a-sin...
Yea, I would add openwrt x86 provides a decent interface for management. Gave dad a little minicomputer with openwrt when he upgraded his internet. He can change wifi password and such and is happy.
Pretty sure switches that support VLANs are more expensive than a NIC. I think even a 4 port GigE Intel NIC can be had for less.
But you might want VLANs anyway, so it's an interesting thing to consider.
VLANs are fine. Running your whole core over one trunk into a general-purpose box gets dumb fast, because one bad config or L2 loop turns into a host-side debugging session.
Extra NICs move forwarding work into the host, and you pay for that in CPU time. If you care about isolation and wire-speed, buy a cheap managed switch instead of stuffing more NICs into the box.
Have you noticed significantly slowed network speeds over WiFi?
Not that I know of, why would that happen?
Wouldn’t all traffic be routed through the OS/processor on board?
A CPU from the last 20 years can route traffic at gigabit speed. It's only something to worry about for a Raspberry Pi3 or something similarly 'crippled'.
I think I understand why this is true for plain IP forwarding. There isn’t much to break the cache and the lookups are few and fast.
What’s the cheapest (new) computer that can drive a 1Gb port with NAT? With a busy encrypted (wireguard?) connection?
[I don’t think qos has a lot of use in the domestic environment; sure, someone here does it but I think it’s much less mainstream than the features I already mentioned. ]
Such a device could drive my home. But in a couple of years I suspect I’ll want 2Gb or 10.
In the past I’ve tended to use a device until its crappy power supply failed. So I guess I’m hoping for a >5 year life span/upgrade capacity.
For all I know the answer to my question is one of those passively cooled four port n100 bricks from AliExpress. Anecdata happily accepted.
> What’s the cheapest (new) computer that can drive a 1Gb port with NAT?
What's the cheapest new computer you can find? That will work. If you have PPPoE, you need to be a bit more careful; depending on your OS and NICs, it's possible for inbound traffic to only use one core; low power laptop cpu may not have enough throughput from a single cpu, but my information is a little dated.
I did 1G NAT on a dual core haswell [1] for a long time.
[1] https://www.intel.com/content/www/us/en/products/sku/82723/i...
Wireguard adds nothing unless you'd want to terminate it on the router. In which case it adds so very little it's unnoticeable.
About any n100 will do. Question is in their reliability which mostly comes down to power regulation components quality. Not performance.
One of my installs runs on a repurposed old android phone. Which has about 100 times CPU capacity of the router I write this through, and that one being cheap tplink shit still terminates wireguard at link speed which is 100Mbps. You don't need fancy gear for routing. And you don't usually need gigabit uplink because speed is limited way upstream.
But if you want "the right gear and damn the price" go get a Microtik. They are very good.
You'd be shocked to find out how old and weak the CPU in your current router is. Typically they're on par with low end desktop CPUs from 10-15 years ago.
I assume the real router OS is extremely neutered to basically only route traffic and filter inbound with everything else being removed? But yeah I can definitely see that.
Except actual routers don't handle the traffic on the CPU, they have dedicated hardware to actually handle the packets. The CPU basically runs the OS, configures the hardware router, and does housekeeping tasks (e.g. ARP or FDB expirations, NAT cleanup, etc). The only packets that ever reach it are "trap to CPU" situations that don't require acceleration as those are rare or expensive to implement in hardware (e.g. better suited to a CPU). Those usually include management protocols (ICMP, ARP, NDP, STP, etc) or packets with unknown destination (e.g. the first packet to an IP that requires ARP resolution).
That's how you can have multi-Gbps on a router with a 200MHz MIPS CPU. Or Tbps on a router with a quad-core Xeon.
Wait until you hear about CAPWAP!
I'm currently running a Debian lite weight server on an old ml100 (onlogic) nuc. It's an old i3, with 16gb ram and no fan. But I have another. Anyone recommend a solid router setup on one of these ancient artifacts? Presently using openwrt on a proper router, though if the nuc is capable, I'd dedicate it thusly.
Seeing an old T60 with an ExpressCard-PCIe bridge used as a router is a great look. It's a solid reminder that even a "trash-picked" 18-year-old machine has way more CPU than you actually need for a home gigabit line. The mention of the serial console (ttyS0) is the real pro-tip in this guide. If you're running a headless box in a closet, a serial getty is a lifesaver for the moment you inevitably misconfigure a firewall rule and lock yourself out of SSH. Sticking to a minimal Debian base with nftables is often much cleaner than using OPNsense/pfSense; there's no GUI abstraction layer hiding what's actually happening to your packets.
I'm curious - for power consumption, considering that you can get RaspPi products for so cheaply, is a discarded laptop more or less impactful on your electrical bill than a RaspPi?
Like is the "free" laptop going to cost you more in the long-run then a nice little power-sipping ARM like a Pi5? Or do you need those extra operations-per-second that the more power-hungry x86 CPU gets you?
This will certainly work, but the whole mesh networking and more advanced aspects of a real wifi router won't really be present.
I get by without it, but I can imagine some won't be able to.
If you're tech-savvy and building your own router, you can add those advanced aspects in if you want them.
I'd be willing to bet, though, that the overwhelming majority of people who use consumer routers aren't doing anything remotely advanced. A how-to that covers the majority of use cases is valuable even when it excludes advanced use cases.
There are a whole lot of normal people using mesh networking Wi-Fi routers. Honestly, most of the least technical people that I know are all using mesh networks because their houses require it.
Certainly. But it's still a minority use case.
Perhaps someone else will (or did) write up a how-to for support mesh networking in your homebrew router.
Where do you live to consider mesh networking a minority use case? I live in a small city apartment so I don't need one, but everyone I know outside of the city needs at least two nodes to cover their houses.
Home mesh is mostly about having wireless backhaul, and you can certainly do that if you have (preferably) two radios, you just set up one radio as a client to your main AP.
Even if you aren't doing wireless backhaul you just rely on regular client behaviour to transition between APs, can enable 802.11r to improve this.
Enterprise "mesh" typically uses wired backhaul for performance and can help clients roam quicker with a controller (auth, not deciding to roam). Controller can also adjusts radio power so APs aren't talking over each other if they're too close.
Mesh isn't any magic, just regular wifi.
is this the new age .. how to run doom on it?
tl;dr:
echo 1 > /proc/sys/net/ipv4/ip_forward
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
> you can make a router out of basically anything resembling a computer.
So if anything can be turned into a router will importing anything be banned as well?