1) (Mainly) the huge increase in upstream capacity of residential broadband connections with FTTH. It's not uncommon for homes to have 2gbit/sec up now and certainly 1gbit/sec is fairly commonplace, which is an enormous amount of bandwidth compared to many interconnects. 10, 40 and 100gbit/sec are the most common and a handful of users can totally saturate these.
2) Many more powerful IoT devices that can handle this level of attack outbound. A $1 SoC can easily handle this these days.
3) Less importantly, CGNAT is a growing problem. If you have 10k (say) users on CGNAT that are compromised, it's likely that there's at least 1 on each CGNAT IP. This means you can't just null route compromised IPs as you are effectively null routing the entire ISP.
I think we probably need more government regulation of these IoT devices. For example, having a "hardware" limit of (say) 10mbit/sec or less for all networking unless otherwise required. 99% all of them don't need more than this.
> If you have 10k (say) users on CGNAT that are compromised, it's likely that there's at least 1 on each CGNAT IP. This means you can't just null route compromised IPs as you are effectively null routing the entire ISP.
How about we actually finally roll out IPv6 and bury CGNAT in the graveyard where it belongs?
Suddenly, everybody (ISPs, carriers, end users) can blackhole a compromised IP and/or IP range without affecting non-compromised endpoints.
And DDoS goes poof. And, as a bonus, we get the end to end nature of the internet back again.
From having worked on DDoS mitigation, there's pretty much no difference between CGNAT and IPv6. Block or rate limit an IPv4 address and you might block some legitimate traffic if it's a NAT address. Block a single IPv6 address... And you might discover that the user controls an entire /64 or whatever prefix. So if you're in a situation where you can't filter out attack trafic by stateless signature (which is pretty bad already), you'll probably err on the side of blocking larger prefixes anyway, which potentially affect other users, the same as with CGNAT.
Insofar as it makes a difference for DDoS mitigation, the scarcity of IPv4 is more of a feature than a bug.
(Having also worked on DDoS mitigation services) That "entire /64" is already hell of a lot more granular than a single CG-NAT range serving everyone on an ISP though. Most often in these types of attacks it's a single subnet of a single home connection. You'll need to block more total prefixes, sure, but only because you actually know you're only blocking actively attacking source subnets, not entire ISPs. You'll probably still want something signature based for the detection of what to blackhole though, but it does scale farther in a combo on the same amount of DDoS mitigation hardware.
you can heuristically block ipv6 prefixes on a big enough attack by blocking a prefix once a probabilistic % of nodes under it are themselves blocked, I think it should work fairly well, as long as attacking traffic has a signature.
consider simple counters "ips with non-malicious traffic" and "ips with malicious traffic" to probabilistically identify the cost/benefit of blocking a prefix.
you do need to be able to support huge block lists, but there isn't the same issue as cgnat where many non-malicious users are definitely getting blocked.
Presumably a compromised device can request arbitrarily new ipv6 from the dhcp so the entire block would be compromised. It would be interesting to see if standard dhcp could limit auto leasing to guard reputation of the network
Generally, IPv6 does autoconfiguration (never seen a home router with DHCPv6), so no need to ask for anything. Even for ipv4, I've never seen a home router enforce DHCP (even though it would force the public ip).
But the point stands, you can't selectively punish a single device, you have to cut off the whole block, which may include well-behaved devices.
This DDoS is claimed to be the result of <300,000 compromised routers.
That would be really easy to block if we were on IPv6. And it would be pretty easy to propagate upstream. And you could probabilistically unblock in an automated way and see if a node was still compromised. etc.
> That would be really easy to block -- if we were on IPv6.
Make that: If the service being attacked was on IPv6-only, and the attacker had no way to fall back to IPv4.
As long as we are dual-stack and IPv6 is optional, no attacker is going to be stupid enough to select the stack which has the highest probability of being defeated. Don't be naive.
It'd be far more acceptable to block the CG-NAT IPv4 addresses if you knew that the other non-compromised hosts could utilize their own IPv6 addresses to connect to your service.
Some time ago I decided for our site to not roll out ipv6 due to these concerns. (a couple of million visitors per month) We have meta ads reps constantly encourage us to enable it which also do not sit right with me.
Although I belive fingerprinting is sofisticated enough to work without using ip's so the impact of using ipv6 might not be a meaningful difference.
Is there any money an ISP would make, or save, by sinking money and effort on switching to IPv6? If there's none, why would they act? If there is some, where?
For instance, mobile phone operators, which had to turn ISPs a decade or two ago, had a natural incentive to switch to IPv6, especially as they grew. Would old ISPs make enough from selling some of their IPv4 pools?
They already lease them out. TELUS in Canada traditional old ISP rents large portion of their space to a mostly used for Chinese GFW VPN server provider in LA „Psychz“
The ISPs have to submit plans on how to use their IPs for the public,especially for IPv4, Arnic shouldn't approve this kind of stuff. Unless they lied in their ip block application, in which case they should be revoked their block.
I filled out one of these for Cogent to get a /24. I was being honest but all I had to put was services that requires their own IP. I even listed a few but no where near the 253.
They also never responded back and were like "what about NAT" or "what about host based routing".
Not sure what you filled out, but blocks are handed usually not to end users, but to providers that will sublease the ips to their client. So if you are asking for a block for a couple of your HTTP servers, that's a no. If you rent HTTP servers to, say, local small businesses, then that's a yes.
> How about we actually finally roll out IPv6 and bury CGNAT in the graveyard where it belongs?
That depends on the service you are DDosing actually having an IPv6 presence. And lots of sites really don't.
It doesn't help if you have IPv6 if you need to fallback to IPv4 anyway. And if bot-net authors knows they can hide behind CGNAT, why would they IPv6 enable their bot-load when all sites and services are guaranteed to be reachable bia IPv4 for the next 3 decades?
> 3) Less importantly, CGNAT is a growing problem. If you have 10k (say) users on CGNAT that are compromised, it's likely that there's at least 1 on each CGNAT IP. This means you can't just null route compromised IPs as you are effectively null routing the entire ISP.
Null routing is usually applied to the targets of the attack, not the sources. If one of your IPs is getting attacked, you null route it, so upstream routers drop traffic instead of sending it to you.
Haha that last part is pretty wild. rather than worrying about systemic problems in the entire internet let's just make mandates crippling devices that China, where all these devices are made, will defffinitely 100% listen to. Sure, seems reasonable. Systems that rely on the goodwill of the entire world to function are generally pretty robust, after all.
I'm sorry that you find thinking about second order dynamics annoying, but that's what you have to do if you actually want effective laws. Just making laws doesn't magically fix problems. In many cases it just makes much more exciting problems.
> I think we probably need more government regulation of these IoT devices. For example, having a "hardware" limit of (say) 10mbit/sec or less for all networking unless otherwise required. 99% all of them don't need more than this.
What about DDoSs that come from sideloaded, unofficial, buggy, or poorly written apps? That's what IoT manufacturers will point to, and where most attacks historically come from. They'll point to whether your Mac really needs more than 100mbps.
The government is far more likely to figure it out along EU lines: Signed firmware, occasional reboots, no default passwords, mandatory security updates for a long-term period, all other applicable "common sense" security measures. Signed firmware and the sideloading ID requirements on Android also helps to prevent stalkerware, which is a growing threat far scarier than some occasional sideloaded virus or DDoS attack. Never assume sideloading is consensual.
>What about DDoSs that come from sideloaded, unofficial, buggy, or poorly written apps? That's what IoT manufacturers will point to, and where most attacks historically come from.
any source for this claim? Outside of very specific scenarios which differ significantly for the current botnet market (like manjaro sending too many requests to the aur or an android application embedding an url to a wikipedia image) I cannot remember one occourence of such a bug being versatile enough to create a new whole cybercrime market segment.
>They'll point to whether your Mac really needs more than 100mbps.
it does, because sometimes my computer bursts up to 1gbps for a sustained amount of time, unlike the average iot device that has a predictable communication pattern.
>Signed firmware and the sideloading ID requirements on Android also helps to prevent stalkerware, which is a growing threat far scarier than some occasional sideloaded virus or DDoS attack. Never assume sideloading is consensual.
if someone can unlock your phone, go into the settings, enable installation of apps for an application (ex. a browser), download an apk and install it then they can do quite literally anything, from enabling adb to exfiltrating all your files.
Historically, it was called Windows XP and Vista about 15 years ago (Blaster, Sasser, MyDoom, Stuxnet, Conficker?). Microsoft clamped down, hard, across the board, but everyone outside of Big Tech is still catching up.
Despite Microsoft's efforts, 911 S5 was roughly 19 million Windows PCs in 2024, in news that went mostly under the radar. It spread almost entirely through dangerous "free VPN" apps that people installed all over the place. (Why is sideloading under attack so much lately? 19 million people thought it would make them more secure, and instead it turned their home internet into criminal gateways with police visits. I strongly suspect this incident, and how it spread among well-meaning security-minded people, was the invisible turning point in Big Tech against software freedom lately.)
> if someone can unlock your phone, go into the settings, enable installation of apps for an application (ex. a browser), download an apk and install it then they can do quite literally anything, from enabling adb to exfiltrating all your files.
Which is more important, and a growing threat? Dump all her photos once; or install a disguised app that pretends to be a boring stock app nobody uses, that provides ongoing access for years, with everything in real-time up to the minute? Increasingly it's the latter. She'll never suspect the "Samsung Battery Optimizer" or even realize it came from an APK. No amount of sandboxing and permissions can detect an app with a deliberately false identity.
I think there's some exaggeration as few $1 SoC parts come with 10G Ethernet, and >1G to the home is not common, but pretty much any home router can saturate its own uplink - it would be useless if it couldn't!
Not always the case. Generating traffic can be more computationally intense than routing the traffic. I've done speed tests on a few routers local to it and the results have been less than stellar compared to getting expected results with it just routing traffic (consumer routers). Granted these tests were a few years ago and things have progressed, but how often are people upgrading their routers?
Also, most 1Gbit/s and faster routers have hardware-accelerated packet forwarding, aka "flow offloading", aka "hardware NAT", where forwarded packets mostly don't touch software at all.
Some routers even have internal "CPU" port of packet core with significantly slower line rate than that of external ports'. So traffic that terminates/originates at the router is necessarily quite a bit slower, regardless of possibly extra-beefy processor, and efficient software. Not really a problem since that traffic would normally be limited to UI, software updates, ARP/NDP/DHCP, and occasional first packet of a forwarded network connection.
Define most places? I know i dont get one (uk) and neither does my german friend or texan friend.
I've only ever seen one despite having used 4 different ISPs for gigabit, and that one was special. It was in an apartment i rented in a converted office tower, line was done via a b2b provider then included in the rent.
Nope. less than a percent of a percent. symmetric plans are extra cost and offered primarily to business.
almost all homes have no ability to exceed gigabit. infact almost all new homes dont even have data wiring. people just want their netflix to work on wifi.
This is probably technically true but very misleading. Fiber penetration in the US has been consistently rising for over a decade now and it is not at all uncommon to have either Google Fiber, Fios, or a local fiber provider available to you in a big city. I bet within the next decade most places will have gigabit fiber available.
Does it really matter? The grandparent comment states the bandwidth is becoming even more readily available in the US, while the article itself says the bots were largely hosted by US ISPs, and that's obviously enough bandwidth to already cause global disruptions. But that's just the source of the attack, and who is on the receiving end is another.
I get being too US-centric, but I think it's interesting if the US has the right combination of hosting tons of infected devices and having the bandwidth to use them on a much larger scale compared to other countries and possible implications.
Seems more likely that residential modems will be required to use ISP-provided equipment that has government mandated chips, firmware, etc to filter outbound traffic for DDoS prevention.
This is very challenging, in about one year the biggest recorded DDoS attack has increased from 5 Tbps to almost 30.
Almost all of the DDoS mitigation providers have been struggling for a few weeks because they just don't have enough edge capacity.
And normal hosting companies that are not focused on DDoS mitigation also seem to have had issues, but with less impact to other customers as they'll just blackhole addresses under larger attacks. For example, I've seen all connections to / from some of my services at Hetzner time out way more frequently than usual, and some at OVH too. Then one of my smaller hosting providers got hit with an attack of at least 1 Tbps which saturated a bunch of their transit links.
Cloudflare and maybe a couple of the other enterprise providers (Gcore?) operate at a large enough scale to handle these attacks, but all the smaller ones (who tend to have more affordable rates and more application-specific filters for sensitive applications that can't deal with much leakage) seem to be in quite a bad spot right now. Cloudflare Magic Transit pricing supposedly starts at around $4k / month, and it would really suck if that became the floor for being able to run a non-HTTP service online.
Something like Team Cymru's UTRS service (with Flowspec support) could potentially help to mitigate attacks at the source, but residential ISPs and maybe the T1s would need to join it, and I don't see that happening anytime soon.
I'd rather there be periodic DDoS attacks, than a locked-down highly-regulated internet. Don't forget that infamous Franklin quote, and what Stallman has been warning us about for the past few decades.
I can already see the authoritarians salivating every time something like this happens.
> I can already see the authoritarians salivating every time something like this happens.
Tinfoil hat theory says they do this intentionally so that the users demand stricter access willingly. Always better to have someone think it is their idea
My tinfoil body-suit suggests this is how CDN's came to be. People's websites were hammered and extorted indirectly by friends of insert CDN startup here. The tin-foil body-suit also suggests the end goal was to hover up all the data and sell it to the government and/or the highest bidder.
You can rest assured your ISP can produce the graph. If your graph reaches a certain shape, they'll start shaping your traffic for you to something they prefer all while charging you the normal rate.
> “The problem is, even if those infected IoT devices are rebooted and cleaned up, they will still get re-compromised by something else generally within minutes of being plugged back in.”
In the year 2025, we should understand that such devices are defective. They should become bricks and companies that continue to sell such defective merchandise should fail.
So they get to ship a dangerous device that harms innocent third parties because they cut corners, but we’re supposed to reward them by doing the work to secure the devices they couldn’t be bothered with?
Some people get very emotional about the games that they play and will pay to have them DDoSed because of something or someone they're angry about. Others just love to cause chaos and will happily buy a DDoS attack to screw other people over. They even get to watch the outcome in real-time because of streamers.
This is what I wonder. Must be fascinating to engineer such a massive distributed system, but at some point there’s no added value from another bazillion hosts in the network.
ISPs should be regulated to require alerting and disconnecting users with compromised devices.
Furthermore, device manufacturers should be regulated and held accountable for comprised devices. This also implies forbidding sale of noncompliant devices, which requires regulation of platforms and logistics supply chains to prevent counterfeit and dangerous goods from being sold.
I'd rather the industry standardizes on some sort of guest network and proxy/hub. It could even ship with hardware from ISPs. Separating the network buys you a lot of security, and running everything through a proxy makes it easier to inspect data and creates a standard hook for using abandonware.
Ehhh I can see it. The right attack at the right time could directly or indirectly kill people, and that’s ignoring the fact it can cause economic havoc.
Having the entire internet function on a “pay or be nuked” threshold that could easily get much worse if companies like cloudflare become less ethical (not that they’re saints).
I'm honestly kinda curious why nobody's blocking these IPs from sending data near the source.
Like, I can come up with plenty of possible reasons, and reasons why it could potentially be very bad if ISPs started cracking down on this, but I don't actually know any reasons.
Are any talking about why / why not? It seems like this whole insecure-IoT-device thing would probably dry up pretty quickly if people's internet was cut off when one was detected. They can then turn around and lambast / sue / etc the company that sold it, putting pressure on the source of the problem. Right now there's no reason for sellers to do anything at all to ensure security, afaict.
So... not actually arguing in favor of it, but definitely curious about any stated ISP / core networking system's stated reasons.
> “The outbound and cross-bound DDoS attacks can be just as disruptive as the inbound stuff,” Dobbin said. “We’re now in a situation where ISPs are routinely seeing terabit-per-second plus outbound attacks from their networks that can cause operational problems.”
ISPs are starting to feel the pain, so perhaps in the near future they will do something about it.
This does happen, but it seems to depend on the ISP. In the Netherlands I've seen ISPs block the internet connectivity when they've detected infected devices, sometimes they send a letter before blocking and some ISPs seem to dump your internet connection in a captive portal. In all these cases it's been enough to call the ISP after finding the problem and you're connected again minutes later.
> They can then turn around and lambast / sue / etc the company that sold it, putting pressure on the source of the problem
Or just unplug the culprit. But the key seems to be that the device continues working. Ideally you would just shutdown or disconnect the device. If fridge is infected, the fridge can still fridge, but it no longer has internet privileges.
Any device that participates in a DDOS needs to be recalled by the manufacturer, mandated by law. Make it potentially economically crippling to sell a vulnerable device, and security will be taken very seriously. Frivolous uses of tech, won't be worth the risk.
This just in: every computer manufacturer forced to recall every single computer model they've ever sold because some users use weak passwords.
I can't wait for all of them to switch to IOS-ified devices incapable of installing alternative operating systems or programs, as that would be the inevitable end solution for all these manufacturers if this was implemented.
Maybe that's a good thing; relying on users to choose good passwords is a cop-out. Systems should be safe-by-default. And owners losing their system if it participates in a DDOS, would add to the incentives to stop the nonsense. It persists because perpetrators, and those who unwittingly abet them, feel no consequences.
At that point, you should force the pain on the individual themselves. Why should all of us be handicapped because there's a couple morons that can't set decent passwords and connect their devices directly to the internet?
Even if the device removed the capability for passwords and used key based authentication, connecting it directly to the internet means if there's ever a vulnerability, all that was for naught anyway.
This is the way, there should be no access by default, then on first access the user has to setup their desired authentication details, and if they want passwords, then they get a randomly generated one, not one they choose. There should also be a factory reset button too.
Exactly, and fwiw most manufacturers have moved to this model by now, or using randomly generated passwords printed on the physical device itself, in the case of routers.
The latter is still a security issue, because it means that anyone who had your device before, or was able to photograph the printed password briefly, could still have access to your device. Of course it mitigates a fair bit of the DDoS issue, but is still problematic.
Yes, you generally see this kind of thing start from the pain-feelers and move up the chain to the pain-causers.
So why hasn't that happened? These are clearly damaging to many, and ISPs are apparently doing next to nothing to prevent it, and it has been extremely clear for a while now that it's going to just become a bigger and bigger problem.
Made even easier by almost everyone running their local network off the ISP's hardware. Before they get cut off, have the router take a snapshot of what's using what ports, then go hunting.
Are there ISPs that don't charge customers for the amount of bandwidth they consume? Even "unlimited" has been ruled by courts to not really mean "unlimited", after all.
Most non-mobile ISPs will let you get pretty high on bandwidth usage before they flag you; and since DDoS are almost always relatively low-levels of bandwidth (on the source) it's unlikely you'd get flagged.
Most eyeball networks have a lot of inbound traffic and not very much outbound, but interconnections with other networks are almost always symmetric, so there's a lot of room for excess egress before it causes pain to the ISP.
When I ran a large web site that attracted lots of DDoS, it didn't really seem worthwhile to track down the source and try to contact ISPs. I had done a lot of trying to track and stop people sending phishing mail under our name, and it's simply too much work to write a reasonable abuse report that is unlikely to be followed up on. With email, mostly people seem to accept the Received headers are probably true; with DDoS, you'd be sending them pcaps, and they'd be telling you it's probably spoofed, and unless I've got lots of peering, I'm not going to be able to get captures that are convincing... so just do my best to manage the inbound and call it a day.
A large part of the article is dedicated to this, noting how disruptive it is to other services and customers, and listing a few countermeasures (detection and blocking at the ISP level, detection and blocking at the router level, and educating customers on not buying vulnerable IoT trash).
Not really? At best it's "DDOS prevention sellers are having trouble" and "ISPs say they're doing fine". The vast majority of the article is talking about the various kinds of malware causing this, and how some have been "fixed" by stopping the individuals running it (which clearly doesn't work very well, new ones just fill the void).
Or this:
>“The crying need for effective and universal outbound DDoS attack suppression is something that is really being highlighted by these recent attacks,” Dobbins continued. “A lot of network operators are learning that lesson now, and there’s going to be a period ahead where there’s some scrambling and potential disruption going on.”
Uh. No. That's gross negligence if they are only starting to think about it now - the trend has been clear for over a decade, and the IoT threat has been obvious since day 1 and even blasted over public news for the past few years. Their status is pretty much only one of: incompetent, malicious, or they have had plans but haven't acted on them fast enough or strongly enough for [some reason], and that reason isn't something I've seen. Surprises happen, prevention costs money and time, and there are plenty of reasons why everyone isn't already prepared for everything, so I think "incompetent or malicious" is pretty rare.... but what are those reasons?
This really is a function of two things:
1) (Mainly) the huge increase in upstream capacity of residential broadband connections with FTTH. It's not uncommon for homes to have 2gbit/sec up now and certainly 1gbit/sec is fairly commonplace, which is an enormous amount of bandwidth compared to many interconnects. 10, 40 and 100gbit/sec are the most common and a handful of users can totally saturate these.
2) Many more powerful IoT devices that can handle this level of attack outbound. A $1 SoC can easily handle this these days.
3) Less importantly, CGNAT is a growing problem. If you have 10k (say) users on CGNAT that are compromised, it's likely that there's at least 1 on each CGNAT IP. This means you can't just null route compromised IPs as you are effectively null routing the entire ISP.
I think we probably need more government regulation of these IoT devices. For example, having a "hardware" limit of (say) 10mbit/sec or less for all networking unless otherwise required. 99% all of them don't need more than this.
> If you have 10k (say) users on CGNAT that are compromised, it's likely that there's at least 1 on each CGNAT IP. This means you can't just null route compromised IPs as you are effectively null routing the entire ISP.
How about we actually finally roll out IPv6 and bury CGNAT in the graveyard where it belongs?
Suddenly, everybody (ISPs, carriers, end users) can blackhole a compromised IP and/or IP range without affecting non-compromised endpoints.
And DDoS goes poof. And, as a bonus, we get the end to end nature of the internet back again.
From having worked on DDoS mitigation, there's pretty much no difference between CGNAT and IPv6. Block or rate limit an IPv4 address and you might block some legitimate traffic if it's a NAT address. Block a single IPv6 address... And you might discover that the user controls an entire /64 or whatever prefix. So if you're in a situation where you can't filter out attack trafic by stateless signature (which is pretty bad already), you'll probably err on the side of blocking larger prefixes anyway, which potentially affect other users, the same as with CGNAT.
Insofar as it makes a difference for DDoS mitigation, the scarcity of IPv4 is more of a feature than a bug.
(Having also worked on DDoS mitigation services) That "entire /64" is already hell of a lot more granular than a single CG-NAT range serving everyone on an ISP though. Most often in these types of attacks it's a single subnet of a single home connection. You'll need to block more total prefixes, sure, but only because you actually know you're only blocking actively attacking source subnets, not entire ISPs. You'll probably still want something signature based for the detection of what to blackhole though, but it does scale farther in a combo on the same amount of DDoS mitigation hardware.
you can heuristically block ipv6 prefixes on a big enough attack by blocking a prefix once a probabilistic % of nodes under it are themselves blocked, I think it should work fairly well, as long as attacking traffic has a signature.
consider simple counters "ips with non-malicious traffic" and "ips with malicious traffic" to probabilistically identify the cost/benefit of blocking a prefix.
you do need to be able to support huge block lists, but there isn't the same issue as cgnat where many non-malicious users are definitely getting blocked.
You should block the whole /64, at least. It's often a single host. It's often but not always a single host, that's standardized.
Usually a /64 is a "local network", so in the case of consumer ISPs that's all the devices belonging to a given client, not a single device.
Some ISPs provide multiple /64s, but in the default configuration the router only announces the first /64 to the local network.
Presumably a compromised device can request arbitrarily new ipv6 from the dhcp so the entire block would be compromised. It would be interesting to see if standard dhcp could limit auto leasing to guard reputation of the network
Generally, IPv6 does autoconfiguration (never seen a home router with DHCPv6), so no need to ask for anything. Even for ipv4, I've never seen a home router enforce DHCP (even though it would force the public ip).
But the point stands, you can't selectively punish a single device, you have to cut off the whole block, which may include well-behaved devices.
In mobile networks it's usually a single device.
This DDoS is claimed to be the result of <300,000 compromised routers.
That would be really easy to block if we were on IPv6. And it would be pretty easy to propagate upstream. And you could probabilistically unblock in an automated way and see if a node was still compromised. etc.
> That would be really easy to block -- if we were on IPv6.
Make that: If the service being attacked was on IPv6-only, and the attacker had no way to fall back to IPv4.
As long as we are dual-stack and IPv6 is optional, no attacker is going to be stupid enough to select the stack which has the highest probability of being defeated. Don't be naive.
It'd be far more acceptable to block the CG-NAT IPv4 addresses if you knew that the other non-compromised hosts could utilize their own IPv6 addresses to connect to your service.
Better to rely on ip blocks than on NAT to bundle blocks.
I am a bit split this topic. There is some privacy concerns with using ipv6. https://www.rfc-editor.org/rfc/rfc7721.html#page-6
Some time ago I decided for our site to not roll out ipv6 due to these concerns. (a couple of million visitors per month) We have meta ads reps constantly encourage us to enable it which also do not sit right with me.
Although I belive fingerprinting is sofisticated enough to work without using ip's so the impact of using ipv6 might not be a meaningful difference.
its hilarious that you have privacy concerns while at the same time using meta ads.
I am guessing they're trying to limit the privacy harm to normal channels that the slightly savvy can understand rather than completely eliminate it.
Reportedly this is often incorrectly implemented, where /64 prefix is still a stable static address.
Is there any money an ISP would make, or save, by sinking money and effort on switching to IPv6? If there's none, why would they act? If there is some, where?
For instance, mobile phone operators, which had to turn ISPs a decade or two ago, had a natural incentive to switch to IPv6, especially as they grew. Would old ISPs make enough from selling some of their IPv4 pools?
Presumably they'd lose money when a DDoS originating from their network causes all their ips to get blocked.
less expensive IP space, more efficient hardware, and lower complexity if you can eliminate NAT.
They already lease them out. TELUS in Canada traditional old ISP rents large portion of their space to a mostly used for Chinese GFW VPN server provider in LA „Psychz“
The ISPs have to submit plans on how to use their IPs for the public,especially for IPv4, Arnic shouldn't approve this kind of stuff. Unless they lied in their ip block application, in which case they should be revoked their block.
I filled out one of these for Cogent to get a /24. I was being honest but all I had to put was services that requires their own IP. I even listed a few but no where near the 253.
They also never responded back and were like "what about NAT" or "what about host based routing".
Not sure what you filled out, but blocks are handed usually not to end users, but to providers that will sublease the ips to their client. So if you are asking for a block for a couple of your HTTP servers, that's a no. If you rent HTTP servers to, say, local small businesses, then that's a yes.
> How about we actually finally roll out IPv6 and bury CGNAT in the graveyard where it belongs?
That depends on the service you are DDosing actually having an IPv6 presence. And lots of sites really don't.
It doesn't help if you have IPv6 if you need to fallback to IPv4 anyway. And if bot-net authors knows they can hide behind CGNAT, why would they IPv6 enable their bot-load when all sites and services are guaranteed to be reachable bia IPv4 for the next 3 decades?
(Disclaimer: This comment posted on IPv6)
Isn't it enough that the target of the DDOS only accepts ipv6?
Is it advantageous to be someone who supports IPv6 on a day like today?
> 3) Less importantly, CGNAT is a growing problem. If you have 10k (say) users on CGNAT that are compromised, it's likely that there's at least 1 on each CGNAT IP. This means you can't just null route compromised IPs as you are effectively null routing the entire ISP.
Null routing is usually applied to the targets of the attack, not the sources. If one of your IPs is getting attacked, you null route it, so upstream routers drop traffic instead of sending it to you.
Sorry, late here. You are right. I mean filter the IP in question.
Haha that last part is pretty wild. rather than worrying about systemic problems in the entire internet let's just make mandates crippling devices that China, where all these devices are made, will defffinitely 100% listen to. Sure, seems reasonable. Systems that rely on the goodwill of the entire world to function are generally pretty robust, after all.
If they don’t then the devices are not sold in the United States. It’s quite simple.
Great to know that smuggling hardware into the US has been completely stopped.
If the analysis above is accurate, a few smuggled devices would not be an issue, as long as the zillions of devices sold at Walmart are compliant.
Congratulations on the creation of a thriving new black market in which the main beneficiary is organized crime! What could go wrong?
Do you take issue with the concept of laws or are you just being annoying?
I'm sorry that you find thinking about second order dynamics annoying, but that's what you have to do if you actually want effective laws. Just making laws doesn't magically fix problems. In many cases it just makes much more exciting problems.
> I think we probably need more government regulation of these IoT devices. For example, having a "hardware" limit of (say) 10mbit/sec or less for all networking unless otherwise required. 99% all of them don't need more than this.
What about DDoSs that come from sideloaded, unofficial, buggy, or poorly written apps? That's what IoT manufacturers will point to, and where most attacks historically come from. They'll point to whether your Mac really needs more than 100mbps.
The government is far more likely to figure it out along EU lines: Signed firmware, occasional reboots, no default passwords, mandatory security updates for a long-term period, all other applicable "common sense" security measures. Signed firmware and the sideloading ID requirements on Android also helps to prevent stalkerware, which is a growing threat far scarier than some occasional sideloaded virus or DDoS attack. Never assume sideloading is consensual.
>What about DDoSs that come from sideloaded, unofficial, buggy, or poorly written apps? That's what IoT manufacturers will point to, and where most attacks historically come from.
any source for this claim? Outside of very specific scenarios which differ significantly for the current botnet market (like manjaro sending too many requests to the aur or an android application embedding an url to a wikipedia image) I cannot remember one occourence of such a bug being versatile enough to create a new whole cybercrime market segment.
>They'll point to whether your Mac really needs more than 100mbps.
it does, because sometimes my computer bursts up to 1gbps for a sustained amount of time, unlike the average iot device that has a predictable communication pattern.
>Signed firmware and the sideloading ID requirements on Android also helps to prevent stalkerware, which is a growing threat far scarier than some occasional sideloaded virus or DDoS attack. Never assume sideloading is consensual.
if someone can unlock your phone, go into the settings, enable installation of apps for an application (ex. a browser), download an apk and install it then they can do quite literally anything, from enabling adb to exfiltrating all your files.
Historically, it was called Windows XP and Vista about 15 years ago (Blaster, Sasser, MyDoom, Stuxnet, Conficker?). Microsoft clamped down, hard, across the board, but everyone outside of Big Tech is still catching up.
Despite Microsoft's efforts, 911 S5 was roughly 19 million Windows PCs in 2024, in news that went mostly under the radar. It spread almost entirely through dangerous "free VPN" apps that people installed all over the place. (Why is sideloading under attack so much lately? 19 million people thought it would make them more secure, and instead it turned their home internet into criminal gateways with police visits. I strongly suspect this incident, and how it spread among well-meaning security-minded people, was the invisible turning point in Big Tech against software freedom lately.)
https://www.fbi.gov/investigate/cyber/how-to-identify-and-re...
> if someone can unlock your phone, go into the settings, enable installation of apps for an application (ex. a browser), download an apk and install it then they can do quite literally anything, from enabling adb to exfiltrating all your files.
Which is more important, and a growing threat? Dump all her photos once; or install a disguised app that pretends to be a boring stock app nobody uses, that provides ongoing access for years, with everything in real-time up to the minute? Increasingly it's the latter. She'll never suspect the "Samsung Battery Optimizer" or even realize it came from an APK. No amount of sandboxing and permissions can detect an app with a deliberately false identity.
> Signed firmware and the sideloading ID requirements
Ending the last corner of actually free market in software is quite a cost for something that wouldn't prevent DDoS.
> sideloaded, unofficial, buggy, or poorly written apps? That's what IoT manufacturers will point to, and where most attacks historically come from
Is that actually true? What evidence do we have, vs. vulnerabilities in the OEM software (the more common case)?
> A $1 SoC can easily handle this these days.
Could you elaborate?
I think there's some exaggeration as few $1 SoC parts come with 10G Ethernet, and >1G to the home is not common, but pretty much any home router can saturate its own uplink - it would be useless if it couldn't!
Not always the case. Generating traffic can be more computationally intense than routing the traffic. I've done speed tests on a few routers local to it and the results have been less than stellar compared to getting expected results with it just routing traffic (consumer routers). Granted these tests were a few years ago and things have progressed, but how often are people upgrading their routers?
Correct.
Also, most 1Gbit/s and faster routers have hardware-accelerated packet forwarding, aka "flow offloading", aka "hardware NAT", where forwarded packets mostly don't touch software at all.
Some routers even have internal "CPU" port of packet core with significantly slower line rate than that of external ports'. So traffic that terminates/originates at the router is necessarily quite a bit slower, regardless of possibly extra-beefy processor, and efficient software. Not really a problem since that traffic would normally be limited to UI, software updates, ARP/NDP/DHCP, and occasional first packet of a forwarded network connection.
A Allwinner H616 is Quad-Core ARM and can definitely saturate gigabit ethernet with packet generation.
1gb upload is extraordinarily rare.
It’s not; most places that give you gigabit fiber will give you a symmetric connection.
Define most places? I know i dont get one (uk) and neither does my german friend or texan friend.
I've only ever seen one despite having used 4 different ISPs for gigabit, and that one was special. It was in an apartment i rented in a converted office tower, line was done via a b2b provider then included in the rent.
I'm in the US and most fiber providers (I checked a handful: AT&T, Sonic, Google Fiber, Frontier) all provide symmetric connections.
Aren't most residential fiber deployments PONs which generally do not offer symmetric bandwidth? E.g. 10G-PON has 10G down / 2.5G up.
Depends on country, its not common here.
Yup. Spectrum is Michigan will give you up to 2gbps down but not anything more than 200mbps up
Is Spectrum fiber or DOCSIS? I didn't realize anyone was pushing these kinds of numbers for fiber. What's the point other than screwing the users?
Penny pinching. Afaik asymmetric PON is the cheapest possible network tech at scale.
Nope. less than a percent of a percent. symmetric plans are extra cost and offered primarily to business.
almost all homes have no ability to exceed gigabit. infact almost all new homes dont even have data wiring. people just want their netflix to work on wifi.
I didn't pay anything extra for symmetric.
Most places do not have fiber.
We know. The problem is that the above comment said "extraordinarily rare" which is a very different and incorrect threshold.
But for those that do...symmetric is the norm. The number of fiber connections is only going up.
symmetric is not the norm. the infra costs are not worth it. symmetric is primarily a business offering.
This is probably technically true but very misleading. Fiber penetration in the US has been consistently rising for over a decade now and it is not at all uncommon to have either Google Fiber, Fios, or a local fiber provider available to you in a big city. I bet within the next decade most places will have gigabit fiber available.
There are probably more English speakers using the Internet in India than there are in the USA...Let alone the hundreds of millions elsewhere.
You cant just assume everyone is talking about your country online.
Does it really matter? The grandparent comment states the bandwidth is becoming even more readily available in the US, while the article itself says the bots were largely hosted by US ISPs, and that's obviously enough bandwidth to already cause global disruptions. But that's just the source of the attack, and who is on the receiving end is another.
I get being too US-centric, but I think it's interesting if the US has the right combination of hosting tons of infected devices and having the bandwidth to use them on a much larger scale compared to other countries and possible implications.
You can assume the county when it is in the title.
>DDoS Botnet Aisuru Blankets US ISPs in Record DDoS
The US is a big place. But the world is bigger. The internet works across the whole world.
There's a long way to go before fibre is commonplace across the world.
Seems more likely that residential modems will be required to use ISP-provided equipment that has government mandated chips, firmware, etc to filter outbound traffic for DDoS prevention.
Why should they be required to have hardware in their own network to filter that out when the ISP is obviously receiving all of their traffic anyway?
Sometimes the attack, or amplification, comes from the ISP-provided router and its bargain basement firmware.
This is very challenging, in about one year the biggest recorded DDoS attack has increased from 5 Tbps to almost 30.
Almost all of the DDoS mitigation providers have been struggling for a few weeks because they just don't have enough edge capacity.
And normal hosting companies that are not focused on DDoS mitigation also seem to have had issues, but with less impact to other customers as they'll just blackhole addresses under larger attacks. For example, I've seen all connections to / from some of my services at Hetzner time out way more frequently than usual, and some at OVH too. Then one of my smaller hosting providers got hit with an attack of at least 1 Tbps which saturated a bunch of their transit links.
Cloudflare and maybe a couple of the other enterprise providers (Gcore?) operate at a large enough scale to handle these attacks, but all the smaller ones (who tend to have more affordable rates and more application-specific filters for sensitive applications that can't deal with much leakage) seem to be in quite a bad spot right now. Cloudflare Magic Transit pricing supposedly starts at around $4k / month, and it would really suck if that became the floor for being able to run a non-HTTP service online.
Something like Team Cymru's UTRS service (with Flowspec support) could potentially help to mitigate attacks at the source, but residential ISPs and maybe the T1s would need to join it, and I don't see that happening anytime soon.
> has increased from 5 Tbps to almost 30
That's nearly a pint, or over 2 daL!
I'm surprised that the best response to ddos is not blocking traffic, but just handling it.
I'd rather there be periodic DDoS attacks, than a locked-down highly-regulated internet. Don't forget that infamous Franklin quote, and what Stallman has been warning us about for the past few decades.
I can already see the authoritarians salivating every time something like this happens.
> I can already see the authoritarians salivating every time something like this happens.
Tinfoil hat theory says they do this intentionally so that the users demand stricter access willingly. Always better to have someone think it is their idea
My tinfoil body-suit suggests this is how CDN's came to be. People's websites were hammered and extorted indirectly by friends of insert CDN startup here. The tin-foil body-suit also suggests the end goal was to hover up all the data and sell it to the government and/or the highest bidder.
Are there any practical ways to monitor my home network traffic and detect if any devices on my network are compromised?
Not sure about monitoring, but I always put any device I don’t trust on a jailed LAN/AP.
A lot of home routers will give you a traffic graph - if yours doesn't you can either find one that does or flash/build one.
I currently run opnsense which has an ok graph out of the box, I haven't fiddled with it to see if there's something fancy I could do here.
I also used to use IPFire which was slightly clunkier but had a nicer usage graph.
Your ISP should give you a bandwidth usage meter.
Haha. My ISP barely gives me an Internet connection
You can rest assured your ISP can produce the graph. If your graph reaches a certain shape, they'll start shaping your traffic for you to something they prefer all while charging you the normal rate.
Aren't cisco machines compromised by default by design? I imagine all of them are at this point. I wouldn't worry too much about it.
> “The problem is, even if those infected IoT devices are rebooted and cleaned up, they will still get re-compromised by something else generally within minutes of being plugged back in.”
In the year 2025, we should understand that such devices are defective. They should become bricks and companies that continue to sell such defective merchandise should fail.
Just wait 6 months, and the IoT vendor will go out of business and shut down the cloud servers which will effectively brick the device.
That may brick it for its intended use, but nothing's stopping a botnet from repurposing it.
Good-guy botnet, cutting down on e-waste.
How about some good old tort liability
Absolutely not. They should be patched and fixed.
So they get to ship a dangerous device that harms innocent third parties because they cut corners, but we’re supposed to reward them by doing the work to secure the devices they couldn’t be bothered with?
Patched and fixed by the manufacturer, was surely what GP was suggesting.
Bricking the devices doesn't punish the supplier as much as it punishes the consumers.
Many of the companies that sold these devices have already bitten the dust, so they aren't even around to fix anything.
Big botnet has nothing better to do than DDoS Minecraft servers?
…to sell ddos protection to the minecraft server admins, basically extortion.
what about as an demonstration of their capabilities for someone else?
only kids want to destruct without harvesting information/resource.
there's plenty of adults that are fans of scorched earth just to watch it burn. most of them just don't have the fortitude to actually do it.
Like I said, kids
Same happens to our online game: https://news.ycombinator.com/item?id=28675094. Not sure what it is about online games that attracts DDoSes.
Some people get very emotional about the games that they play and will pay to have them DDoSed because of something or someone they're angry about. Others just love to cause chaos and will happily buy a DDoS attack to screw other people over. They even get to watch the outcome in real-time because of streamers.
This is what I wonder. Must be fascinating to engineer such a massive distributed system, but at some point there’s no added value from another bazillion hosts in the network.
I guess if they go after bigger targets they draw unwanted attention? Seemed odd to me too.
ISPs should be regulated to require alerting and disconnecting users with compromised devices.
Furthermore, device manufacturers should be regulated and held accountable for comprised devices. This also implies forbidding sale of noncompliant devices, which requires regulation of platforms and logistics supply chains to prevent counterfeit and dangerous goods from being sold.
Seems pretty clear that the US needs strict regulation on any device connecting to the internet.
* no default password * * no login if not on the local wifi or wired ethernet *
I'd rather the industry standardizes on some sort of guest network and proxy/hub. It could even ship with hardware from ISPs. Separating the network buys you a lot of security, and running everything through a proxy makes it easier to inspect data and creates a standard hook for using abandonware.
Many manufacturers are already moving there of their own accord. I really don't think we'd need some legislation to fix this problem.
Ehhh I can see it. The right attack at the right time could directly or indirectly kill people, and that’s ignoring the fact it can cause economic havoc.
Having the entire internet function on a “pay or be nuked” threshold that could easily get much worse if companies like cloudflare become less ethical (not that they’re saints).
I'm honestly kinda curious why nobody's blocking these IPs from sending data near the source.
Like, I can come up with plenty of possible reasons, and reasons why it could potentially be very bad if ISPs started cracking down on this, but I don't actually know any reasons.
Are any talking about why / why not? It seems like this whole insecure-IoT-device thing would probably dry up pretty quickly if people's internet was cut off when one was detected. They can then turn around and lambast / sue / etc the company that sold it, putting pressure on the source of the problem. Right now there's no reason for sellers to do anything at all to ensure security, afaict.
So... not actually arguing in favor of it, but definitely curious about any stated ISP / core networking system's stated reasons.
> “The outbound and cross-bound DDoS attacks can be just as disruptive as the inbound stuff,” Dobbin said. “We’re now in a situation where ISPs are routinely seeing terabit-per-second plus outbound attacks from their networks that can cause operational problems.”
ISPs are starting to feel the pain, so perhaps in the near future they will do something about it.
Perhaps, or perhaps not. Maybe if we held them accountable they would?
This does happen, but it seems to depend on the ISP. In the Netherlands I've seen ISPs block the internet connectivity when they've detected infected devices, sometimes they send a letter before blocking and some ISPs seem to dump your internet connection in a captive portal. In all these cases it's been enough to call the ISP after finding the problem and you're connected again minutes later.
> They can then turn around and lambast / sue / etc the company that sold it, putting pressure on the source of the problem
Or just unplug the culprit. But the key seems to be that the device continues working. Ideally you would just shutdown or disconnect the device. If fridge is infected, the fridge can still fridge, but it no longer has internet privileges.
Any device that participates in a DDOS needs to be recalled by the manufacturer, mandated by law. Make it potentially economically crippling to sell a vulnerable device, and security will be taken very seriously. Frivolous uses of tech, won't be worth the risk.
This just in: every computer manufacturer forced to recall every single computer model they've ever sold because some users use weak passwords.
I can't wait for all of them to switch to IOS-ified devices incapable of installing alternative operating systems or programs, as that would be the inevitable end solution for all these manufacturers if this was implemented.
Maybe that's a good thing; relying on users to choose good passwords is a cop-out. Systems should be safe-by-default. And owners losing their system if it participates in a DDOS, would add to the incentives to stop the nonsense. It persists because perpetrators, and those who unwittingly abet them, feel no consequences.
At that point, you should force the pain on the individual themselves. Why should all of us be handicapped because there's a couple morons that can't set decent passwords and connect their devices directly to the internet?
Even if the device removed the capability for passwords and used key based authentication, connecting it directly to the internet means if there's ever a vulnerability, all that was for naught anyway.
This is the way, there should be no access by default, then on first access the user has to setup their desired authentication details, and if they want passwords, then they get a randomly generated one, not one they choose. There should also be a factory reset button too.
Exactly, and fwiw most manufacturers have moved to this model by now, or using randomly generated passwords printed on the physical device itself, in the case of routers.
The latter is still a security issue, because it means that anyone who had your device before, or was able to photograph the printed password briefly, could still have access to your device. Of course it mitigates a fair bit of the DDoS issue, but is still problematic.
Does a weak user password have to provide remote access by default?
There's no economic incentive for YOU (as the proximate ISP) to do anything about it, it would cost money, and cost you customers.
Any idea why they don't fix it?
Yes, you generally see this kind of thing start from the pain-feelers and move up the chain to the pain-causers.
So why hasn't that happened? These are clearly damaging to many, and ISPs are apparently doing next to nothing to prevent it, and it has been extremely clear for a while now that it's going to just become a bigger and bigger problem.
How are you going to get an end customer to track down whatever device of theirs was hacked?
As a power user I don't know any way of even checking if I'm involved in a botnet.
Is there something like that out there? Something that routers could install to monitor and report?
Maybe Pi-hole and look for weird lookups? Home routers wont have anything useful, I can see bandwidth and log NAT etc on my Ubiquiti though.
As the ISP you don't care, you just cut off their connection to fix it. Said user will have to contact a local service to come out and find it.
Made even easier by almost everyone running their local network off the ISP's hardware. Before they get cut off, have the router take a snapshot of what's using what ports, then go hunting.
Are there ISPs that don't charge customers for the amount of bandwidth they consume? Even "unlimited" has been ruled by courts to not really mean "unlimited", after all.
Most in Europe doesn't. Back when I first got 100/100 fiber I uploaded almost 30TB in a month and the only complaint I got was from the torrent site.
Yes, most if not all of them. Is it different in the US?
Most non-mobile ISPs will let you get pretty high on bandwidth usage before they flag you; and since DDoS are almost always relatively low-levels of bandwidth (on the source) it's unlikely you'd get flagged.
Of course there is. If you've got all your internet egress tied up with DDoS attacks from your network it is a big problem.
Most eyeball networks have a lot of inbound traffic and not very much outbound, but interconnections with other networks are almost always symmetric, so there's a lot of room for excess egress before it causes pain to the ISP.
When I ran a large web site that attracted lots of DDoS, it didn't really seem worthwhile to track down the source and try to contact ISPs. I had done a lot of trying to track and stop people sending phishing mail under our name, and it's simply too much work to write a reasonable abuse report that is unlikely to be followed up on. With email, mostly people seem to accept the Received headers are probably true; with DDoS, you'd be sending them pcaps, and they'd be telling you it's probably spoofed, and unless I've got lots of peering, I'm not going to be able to get captures that are convincing... so just do my best to manage the inbound and call it a day.
I think we’re just starting to see attacks that big - which might start some practical mitigations (or they’ll just upgrade transit).
A large part of the article is dedicated to this, noting how disruptive it is to other services and customers, and listing a few countermeasures (detection and blocking at the ISP level, detection and blocking at the router level, and educating customers on not buying vulnerable IoT trash).
Not really? At best it's "DDOS prevention sellers are having trouble" and "ISPs say they're doing fine". The vast majority of the article is talking about the various kinds of malware causing this, and how some have been "fixed" by stopping the individuals running it (which clearly doesn't work very well, new ones just fill the void).
Or this:
>“The crying need for effective and universal outbound DDoS attack suppression is something that is really being highlighted by these recent attacks,” Dobbins continued. “A lot of network operators are learning that lesson now, and there’s going to be a period ahead where there’s some scrambling and potential disruption going on.”
Uh. No. That's gross negligence if they are only starting to think about it now - the trend has been clear for over a decade, and the IoT threat has been obvious since day 1 and even blasted over public news for the past few years. Their status is pretty much only one of: incompetent, malicious, or they have had plans but haven't acted on them fast enough or strongly enough for [some reason], and that reason isn't something I've seen. Surprises happen, prevention costs money and time, and there are plenty of reasons why everyone isn't already prepared for everything, so I think "incompetent or malicious" is pretty rare.... but what are those reasons?
Isn't OVH known for offering relatively good anti-DDoS services? Why did they boot them?
9gigsofram was prolific in the "Minecraft Server era" (2010-2016).
Source: Server Owner's Chat
No evidence, just a wild guess, Aisuru might be tied to DDoS mitigation folks.
Our smart homes are becoming dumb weapons in the wrong hands.
"A spokesperson for Comcast responded, 'Currently our network is not experiencing impacts and we are able to handle the traffic.'"
In other words, attacks coming from our own IP's are not our problem ¯\_(ツ)_/¯
This doesn't seem very loving.
[dead]
[flagged]