Cool article! L4 stuff is always fun to read about :)
Since TCP-in-UDP seems to be involve implementing end-to-end features over UDP, I feel like a comparison to multipath QUIC might be helpful so that we can understand it better.
My impression from reading about HTTP/3 is that QUIC is just kind of better than TCP in performing a lot of end-to-end functions, like recovery, encryption, error correction, duplicate suppression, congestion control, and delivery acknowledgement, and of course multiplexing the connection. (The advantage of TCP seems like its simpler and more mature, but it's honestly not clear to me where TCP wins and I'd be interested in hearing about the situations in which TCP is better than QUIC.)
So if we're addressing middleboxes screwing with MPTCP by tunneling TCP over UDP, isn't this very similar to what multipath QUIC does? The article seems to argue this is a simpler, lower-overhead solution than VPN tunnels, which I agree, that would seem like the wrong tool for the job, but I can't really tell which of multipath QUIC and MPTCP-over-UDP is simpler; they seem really similar to me and I'd appreciate help differentiating them.
When would one prefer MPTCP-over-UDP over multipath QUIC, and vice versa? How do the two differ in functionality, stability, and ease-of-use?
TCP is a protocol with multiple implementations for several core aspects (most importantly congestion control and loss handling), so it's not exactly 1:1 comparable with QUIC, which currently is essentially both a protocol and an implementation.
TCP-in-UDP keeps TCP's battle-tested logic but wraps it to bypass middleboxes—great for legacy systems. QUIC is a clean-slate rewrite, better for new apps. Tradeoff: compatibility vs modern features.
By wrapping TCP in UDP, you essentially lose all benefits of compatibility, though. If neither middleboxes nor your OS are involved, there's not much left to be compatible with.
I know it isn't the intended use case but I wonder how this would perform as a censorship circumvention method. The translation alone may be enough of a change to avoid the pattern matching used by firewalls.
It depends on the firewall. For example, a Palo Alto can match by application and do SSL decryption in where the firewall acts as a MITM. A locked down environment would likely only allow SSL on TCP/443, and block UDP/443 to deny QUIC. An application that breaks RFC would likely be blocked. It’s also common to block unknown TCP/UDP applications.
The article is focused on general ossification of TCP preventing newer improvements from functioning. E.g. "performance enhancing proxies" from the article is not a euphemism for censorship, it's boxes design to do one thing with TCP which ends up breaking something for MPTCP. Any bypassing of censorship by avoiding ossification is completely incidental (and likely some extremely bad censorship if it just allows any protocol it can't filter through).
Might get through if they're incompetent, ie maybe works in your typical enterprise environment. If they're competent they wouldn't allow udp at all, if they do they'll only allow specific things.
90% of IDS, spyware, ADVERTS does not support ipv6... That is why most of (public) internet is not on IPV6
And what is most embarrassing is that truly fully IPV6 capable internet requires less of and lower powered "routers". "routers" will cost single thousands instead of hundred of thousands adn be more capable, speeedier. DDoS mitigations are easier in ipv6 too. And if every customer can have 2^64 IP (or even 2^56) addresses then you do not need "ports" anymore, every service on your server can have their own IP, or even every service customer can have their own ip address how much will that simplify CODE (source of bugs, of latency, of unnecessary payments) and lower energy requirements of login infrastructure ? and debugging ? also just right from bat you can trivially see on upstream router who is initiating DOS... PKI+IPv6 is gift from GODs! If your certificate is not issued for specific ip then "openssl" can drop connection in that instance. Is not that little bit more secure ? faster? less clunky. and with more oversoght for network "manager" ?
I'm a networking guy by profession + a big IPv6 fan as well (yay HN these days! news.ycombinator.com->2606:7100:1:67::26)... but don't hold your breath that IPv6 will lower the price of routers at all, let alone by a factor of 10x. From a hardware perspective there are a few minor gains with changes to the protocol but there are also some losses due to the address and network prefix sizes taking up more expensive TCAM/SRAM.
You still want ports, they actually make networking hardware cheaper overall by moving some of the scaling requirements out of the IP layer and into the transport layer. Imagine needing router which can hold 1,000,000 IPv6<->MAC address bindings just because you have 1,000 clients in your network using new addresses instead of ports! ND code is more complex than the code to bind to ports, but I still like the introduction of ND more than not regardless :).
IPv6 infra will probably never adopt the cert stuff you mention. The protocol is just designed to be able to, it doesn't mandate it. In practice it's almost never done and having everyone do it would likely be harder than getting people off IPv4 has been. On the internet routing side, PKI with BGP doesn't really care about the address format and works fine with IPv4.
For the network manager the 2 biggest changes are 1) All of their client subnets are /64s, no more subnet mask tables. 2) No more NAT, which feeds into the debugging side of things, though some of this is advantage is intentionally lost in a tradeoff for increased privacy via temporary addresses. In the last one it's tempting to tie that back to enormous hardware gains but, in reality, the box at that position of the network needs to statefully track sessions regardless of if it needs to translate them, and that's the majority of the cost.
Around 50% of internet packets are IPv6. It's not the future. It's now the present, and IPv4 is the past.
It's a shame they still have ports in IPv6, but I can see why: imagine having to ARP (IPv6 calls it ND) every connection separately. At least you can just allocate another privacy address if you need more than 65535 concurrent connections to the same destination.
Israeli "network traffic monitoring hw used by law enforcement" was deployed in one European country and after few years of deployment public officials embarrassingly confessed that, device was not capable of monitoring IPv6 at all XD just sayin. people expect things working but even west devices are embarrassingly nonsensically flawed. Huawei / zte etc stealing sourcecode and blatantly copying it is even worse situation because they do not even understand how it should work. XD
Makes total sense to me. But why hasn't this been done a long time ago? Pretty sure people spotted this earlier in TCP and UDP but this was never implemented?
Cool article! L4 stuff is always fun to read about :)
Since TCP-in-UDP seems to be involve implementing end-to-end features over UDP, I feel like a comparison to multipath QUIC might be helpful so that we can understand it better.
My impression from reading about HTTP/3 is that QUIC is just kind of better than TCP in performing a lot of end-to-end functions, like recovery, encryption, error correction, duplicate suppression, congestion control, and delivery acknowledgement, and of course multiplexing the connection. (The advantage of TCP seems like its simpler and more mature, but it's honestly not clear to me where TCP wins and I'd be interested in hearing about the situations in which TCP is better than QUIC.)
So if we're addressing middleboxes screwing with MPTCP by tunneling TCP over UDP, isn't this very similar to what multipath QUIC does? The article seems to argue this is a simpler, lower-overhead solution than VPN tunnels, which I agree, that would seem like the wrong tool for the job, but I can't really tell which of multipath QUIC and MPTCP-over-UDP is simpler; they seem really similar to me and I'd appreciate help differentiating them.
When would one prefer MPTCP-over-UDP over multipath QUIC, and vice versa? How do the two differ in functionality, stability, and ease-of-use?
TCP is a protocol with multiple implementations for several core aspects (most importantly congestion control and loss handling), so it's not exactly 1:1 comparable with QUIC, which currently is essentially both a protocol and an implementation.
TCP-in-UDP keeps TCP's battle-tested logic but wraps it to bypass middleboxes—great for legacy systems. QUIC is a clean-slate rewrite, better for new apps. Tradeoff: compatibility vs modern features.
By wrapping TCP in UDP, you essentially lose all benefits of compatibility, though. If neither middleboxes nor your OS are involved, there's not much left to be compatible with.
I know it isn't the intended use case but I wonder how this would perform as a censorship circumvention method. The translation alone may be enough of a change to avoid the pattern matching used by firewalls.
It depends on the firewall. For example, a Palo Alto can match by application and do SSL decryption in where the firewall acts as a MITM. A locked down environment would likely only allow SSL on TCP/443, and block UDP/443 to deny QUIC. An application that breaks RFC would likely be blocked. It’s also common to block unknown TCP/UDP applications.
But circumventing censorship does seem to be the intended use case, no? At leas that's how I read the article.
The article is focused on general ossification of TCP preventing newer improvements from functioning. E.g. "performance enhancing proxies" from the article is not a euphemism for censorship, it's boxes design to do one thing with TCP which ends up breaking something for MPTCP. Any bypassing of censorship by avoiding ossification is completely incidental (and likely some extremely bad censorship if it just allows any protocol it can't filter through).
I wouldn't expect this to fare any better at evading deep packet inspection than any other current technique.
Might get through if they're incompetent, ie maybe works in your typical enterprise environment. If they're competent they wouldn't allow udp at all, if they do they'll only allow specific things.
90% of IDS, spyware, ADVERTS does not support ipv6... That is why most of (public) internet is not on IPV6
And what is most embarrassing is that truly fully IPV6 capable internet requires less of and lower powered "routers". "routers" will cost single thousands instead of hundred of thousands adn be more capable, speeedier. DDoS mitigations are easier in ipv6 too. And if every customer can have 2^64 IP (or even 2^56) addresses then you do not need "ports" anymore, every service on your server can have their own IP, or even every service customer can have their own ip address how much will that simplify CODE (source of bugs, of latency, of unnecessary payments) and lower energy requirements of login infrastructure ? and debugging ? also just right from bat you can trivially see on upstream router who is initiating DOS... PKI+IPv6 is gift from GODs! If your certificate is not issued for specific ip then "openssl" can drop connection in that instance. Is not that little bit more secure ? faster? less clunky. and with more oversoght for network "manager" ?
I'm a networking guy by profession + a big IPv6 fan as well (yay HN these days! news.ycombinator.com->2606:7100:1:67::26)... but don't hold your breath that IPv6 will lower the price of routers at all, let alone by a factor of 10x. From a hardware perspective there are a few minor gains with changes to the protocol but there are also some losses due to the address and network prefix sizes taking up more expensive TCAM/SRAM.
You still want ports, they actually make networking hardware cheaper overall by moving some of the scaling requirements out of the IP layer and into the transport layer. Imagine needing router which can hold 1,000,000 IPv6<->MAC address bindings just because you have 1,000 clients in your network using new addresses instead of ports! ND code is more complex than the code to bind to ports, but I still like the introduction of ND more than not regardless :).
IPv6 infra will probably never adopt the cert stuff you mention. The protocol is just designed to be able to, it doesn't mandate it. In practice it's almost never done and having everyone do it would likely be harder than getting people off IPv4 has been. On the internet routing side, PKI with BGP doesn't really care about the address format and works fine with IPv4.
For the network manager the 2 biggest changes are 1) All of their client subnets are /64s, no more subnet mask tables. 2) No more NAT, which feeds into the debugging side of things, though some of this is advantage is intentionally lost in a tradeoff for increased privacy via temporary addresses. In the last one it's tempting to tie that back to enormous hardware gains but, in reality, the box at that position of the network needs to statefully track sessions regardless of if it needs to translate them, and that's the majority of the cost.
Around 50% of internet packets are IPv6. It's not the future. It's now the present, and IPv4 is the past.
It's a shame they still have ports in IPv6, but I can see why: imagine having to ARP (IPv6 calls it ND) every connection separately. At least you can just allocate another privacy address if you need more than 65535 concurrent connections to the same destination.
So what are you using? Is there a softrouter using mutual TLS over IPv6 that you can recommend?
Israeli "network traffic monitoring hw used by law enforcement" was deployed in one European country and after few years of deployment public officials embarrassingly confessed that, device was not capable of monitoring IPv6 at all XD just sayin. people expect things working but even west devices are embarrassingly nonsensically flawed. Huawei / zte etc stealing sourcecode and blatantly copying it is even worse situation because they do not even understand how it should work. XD
Makes total sense to me. But why hasn't this been done a long time ago? Pretty sure people spotted this earlier in TCP and UDP but this was never implemented?
Netflix was investigating this a while ago (https://reviews.freebsd.org/D15525 - "TCPOUDP")
I love any packet which has length field right after destination field. makes processing little bit faster.
I can see that using eBPF would help backport to older kernel, but would the long term goal be to add this as a kernel patch?
> This solution relies on eBPF, doesn’t add extra data per packet
so none of the security recomendations such exchanging hmac keys and segment counts?