Very neat visualizations! Really good way to demonstrate the underlying principles. Great article.
I will add one thing: the author makes much of how problematic CPU-intensive tasks are. That's true, but also: if you need to do a CPU-intensive operation, you need to tie up a CPU to do it, there's no getting around it. What you might want to do is break the computation up in to smaller chunks with `.await`s sprinkled in there or dispatch it to a background thread, but at some point you have to pay the cost in CPU time.
What is much more problematic is blocking I/O operations. That also ties up a CPU and blocks the event loop, but for no good reason. All the CPU is going to be doing is sleeping waiting for the network or hard-drives or whatever it is to do it's work, when it could be doing other useful things, processing other events. That kind of thing leads to huge under-utilization of your resources, and it kills your concurrency.
This means you really can't mix async I/O with blocking I/O: if you go async, ALL your I/O operations need to be async. Or you have to do it off the event loop in a background thread or something, but then you're losing all the benefits of using async in the first place.
I went through a phase of writing a lot of little things in async Python which was a lot of fun but I wrote an app that really tied up the event loop and now, at least for web servers, I develop with Flask, deploy to gunicorn and often front with IIS or nginx to serve up images and big files now.
One thing I still like async Python for is programs that have a number of polling tasks going on at a slow cadence, say once a minute or less, where it doesn't really matter to me if a task gets delayed 5 seconds because of another task. In that case writing a number of tasks that await.sleep is easy and fun and works great -- I'll use async io in the tasks if it is easy to do so but if I use blocking I/O and it blocks... no problem.
At work I've been writing web servers in Java (or C#) for a couple of jobs and between threads and the garbage collector I also think it is easy and fun. Sure if I was handling Google-levels of traffic I might think differently, but threads scale pretty well for 99% of the web sites out there. Java gives you all the right primitives to do common tasks: sometimes I code something up with an Executor in 20 minutes and it works right the first time and gets a 14x speedup with 16 cores, somebody else tries Actors in the language of the day and spends a few days on it, never gets more than a 3x speedup and gets a different answer every time. (Threaded programming is easy when you have a rich primitive library you get in trouble when you get it in your head that it is noble to build everything in scratch from a small set of primitives)
Java even has virtual threads that play well with real threads if real threads aren't good enough for you.
but in an async I/O server scenario, the typical architecture is a fixed-size pool of worker threads, which ideally entirely occupy a single core each. therefore, if one of those threads is blocked, that core is wasted.
> Unlike Node.js, Rust's Tokio allows us to spawn a new Task and run futures within it.
Nice article! For future reference in Node.js this would be `worker_threads` or if you want a higher level tokio like API something like https://piscinajs.dev/
This has been possible in Node.js for quite a while
--
Also for blocking I/O CPU bound parallelism I've found Rayon to be a pleasure. It has a few caveats but it leverages the Rust race condition checking well and it's very ergonomic.
I happily mix it with `tokio` in an application that is CPU bound but has to perform some async I/O (due to the SDKs it uses)
It’s never addressed in the article that the calculations running on spawned threads produce deformed sine curves. Is that just a result of how its plotted or do tokio threads have less accurate track of time?
Thats a good observation. I believe the deformation comes from the variable runtime of the sin method. Even with the 100 micros sleep that I've put there, in reality it takes a different amount of time between each data point calculation, making the sine wave a bit wonky. This gets worse as more and more calculations are done in parallel
> In other words, each tick of the underlying clock might not be the same length (e.g. some seconds may be longer than others). An instant may jump forwards or experience time dilation (slow down or speed up), but it will never go backwards
Ugh, too late to edit but I’m way off. It’s just scheduling variability, if a task gets more CPU time, it plots more of them near each other, and if it gets less, they’re farther apart.
(I had originally thought the code used the duration since the start as the input to the sin function, didn’t realize it’s just plotting a pre-allocated range.)
> Alright! We managed to plot two sine waves in the most convoluted method ever!
Those sine waves start to resemble a multi-voice music score. You might get some interesting pitch variability depending on other processes that are running.
This is obviously a troll comment, but it made me curious.
I'm familiar with Plan 9, and I know about the interesting mount namespace shenanigans you can pull off with it, but does it actually have anything that would help with async?
I was kinda hoping for a origin story, but alas this doesn't seem to lead to one, and then I checked an actual physical paper dictionary (I'm an old man, old enough that I bought a dictionary in 1982 apparently) and that does define this phrase but doesn't explain where it came from.
I have used this metaphor my whole life without thinking about why pennies dropping have this significance. So now I'm wondering
My best guess is that it's thinking about mechanisms which operate when given money, a topical example would be the Alzara fortune telling machine in a Rumpus Room of Blue Prince. If we put a coin in the machine, it springs into life and tells us... something. If you won't give it a coin, nothing for you.
The penny drops means sudden understanding or realisation of something or other. The expression usually implies a period of non-comprehension before the penny drops and derives from early penny slot machines in which the coin or coins took quite some time to drop. That is why the expression is sometimes in the form of ‘the penny has finally dropped’. The OED cites the 1940s as the first appearance of the figurative meaning. Penny slot machines first appeared during the mid-19th century and some alternative sources date the current figurative meaning from at least the 1920s, if not earlier.
Sounds like your intuition is correct, and the phrase comes in particular from the delay in the penny takes to drop in these machines. I'd imagine if they were faster probably the phrase wouldn't have developed, so an interesting example of how some engineers somewhere who designed the slow mechanism ended up impacting the lexicon.
Very neat visualizations! Really good way to demonstrate the underlying principles. Great article.
I will add one thing: the author makes much of how problematic CPU-intensive tasks are. That's true, but also: if you need to do a CPU-intensive operation, you need to tie up a CPU to do it, there's no getting around it. What you might want to do is break the computation up in to smaller chunks with `.await`s sprinkled in there or dispatch it to a background thread, but at some point you have to pay the cost in CPU time.
What is much more problematic is blocking I/O operations. That also ties up a CPU and blocks the event loop, but for no good reason. All the CPU is going to be doing is sleeping waiting for the network or hard-drives or whatever it is to do it's work, when it could be doing other useful things, processing other events. That kind of thing leads to huge under-utilization of your resources, and it kills your concurrency.
This means you really can't mix async I/O with blocking I/O: if you go async, ALL your I/O operations need to be async. Or you have to do it off the event loop in a background thread or something, but then you're losing all the benefits of using async in the first place.
I went through a phase of writing a lot of little things in async Python which was a lot of fun but I wrote an app that really tied up the event loop and now, at least for web servers, I develop with Flask, deploy to gunicorn and often front with IIS or nginx to serve up images and big files now.
One thing I still like async Python for is programs that have a number of polling tasks going on at a slow cadence, say once a minute or less, where it doesn't really matter to me if a task gets delayed 5 seconds because of another task. In that case writing a number of tasks that await.sleep is easy and fun and works great -- I'll use async io in the tasks if it is easy to do so but if I use blocking I/O and it blocks... no problem.
At work I've been writing web servers in Java (or C#) for a couple of jobs and between threads and the garbage collector I also think it is easy and fun. Sure if I was handling Google-levels of traffic I might think differently, but threads scale pretty well for 99% of the web sites out there. Java gives you all the right primitives to do common tasks: sometimes I code something up with an Executor in 20 minutes and it works right the first time and gets a 14x speedup with 16 cores, somebody else tries Actors in the language of the day and spends a few days on it, never gets more than a 3x speedup and gets a different answer every time. (Threaded programming is easy when you have a rich primitive library you get in trouble when you get it in your head that it is noble to build everything in scratch from a small set of primitives)
Java even has virtual threads that play well with real threads if real threads aren't good enough for you.
> blocking I/O operations. That also ties up a CPU and blocks the event loop
They do block the event loop (if single-threaded) but they do not "tie up" a CPU (core), the OS will schedule other threads on it.
assuming you have other threads
but in an async I/O server scenario, the typical architecture is a fixed-size pool of worker threads, which ideally entirely occupy a single core each. therefore, if one of those threads is blocked, that core is wasted.
> Unlike Node.js, Rust's Tokio allows us to spawn a new Task and run futures within it.
Nice article! For future reference in Node.js this would be `worker_threads` or if you want a higher level tokio like API something like https://piscinajs.dev/
This has been possible in Node.js for quite a while
--
Also for blocking I/O CPU bound parallelism I've found Rayon to be a pleasure. It has a few caveats but it leverages the Rust race condition checking well and it's very ergonomic.
I happily mix it with `tokio` in an application that is CPU bound but has to perform some async I/O (due to the SDKs it uses)
A tokio task is not a thread
A node worker_thread is not a thread either though :]
Right at the top of the docs for worker_threads:
> The node:worker_threads module enables the use of threads that execute JavaScript in parallel.
Now that it doesn’t say “concurrent”, it says “parallel”. This implies that multiple threads are involved.
Really? The docs make it seems like it is: https://nodejs.org/api/worker_threads.html#worker-threads
> piscinajs
Lmao this wordplay made my day
> Rust race condition checking
Nit: Rust does not prevent race conditions, it prevents data races.
It’s never addressed in the article that the calculations running on spawned threads produce deformed sine curves. Is that just a result of how its plotted or do tokio threads have less accurate track of time?
Thats a good observation. I believe the deformation comes from the variable runtime of the sin method. Even with the 100 micros sleep that I've put there, in reality it takes a different amount of time between each data point calculation, making the sine wave a bit wonky. This gets worse as more and more calculations are done in parallel
It's more likely to be the unaccuracy of Instant in rust. It's guaranteed to be non-decreasing, but not steady: https://doc.rust-lang.org/std/time/struct.Instant.html
> In other words, each tick of the underlying clock might not be the same length (e.g. some seconds may be longer than others). An instant may jump forwards or experience time dilation (slow down or speed up), but it will never go backwards
Ugh, too late to edit but I’m way off. It’s just scheduling variability, if a task gets more CPU time, it plots more of them near each other, and if it gets less, they’re farther apart.
(I had originally thought the code used the duration since the start as the input to the sin function, didn’t realize it’s just plotting a pre-allocated range.)
I think it's just scheduling. The deformation is when it didn't get to run exactly in time because Tokio had to wake up enough other tasks first.
Looks like it's just plotting a fixed number of points at the time the iteration was scheduled. It's not trying to map a smooth curve.
This is not limited only to Rust and I'd be very curious to see a similar comparison across multiple runtimes!
> Alright! We managed to plot two sine waves in the most convoluted method ever!
Those sine waves start to resemble a multi-voice music score. You might get some interesting pitch variability depending on other processes that are running.
The async disaster can be avoided by using Plan9 or Inferno.
Don't misinterpret this as criticism of async-rust. It's required to work on legacy platforms like linux.
This is obviously a troll comment, but it made me curious.
I'm familiar with Plan 9, and I know about the interesting mount namespace shenanigans you can pull off with it, but does it actually have anything that would help with async?
Shared 12 days ago: https://news.ycombinator.com/item?id=43675098
"Visualizing futures runtime like this really makes some pennies drop."
What's it mean to drop pennies?
It's a British idiom. When the penny drops you suddenly understand something.
https://en.wiktionary.org/wiki/the_penny_drops
I was kinda hoping for a origin story, but alas this doesn't seem to lead to one, and then I checked an actual physical paper dictionary (I'm an old man, old enough that I bought a dictionary in 1982 apparently) and that does define this phrase but doesn't explain where it came from.
I have used this metaphor my whole life without thinking about why pennies dropping have this significance. So now I'm wondering
My best guess is that it's thinking about mechanisms which operate when given money, a topical example would be the Alzara fortune telling machine in a Rumpus Room of Blue Prince. If we put a coin in the machine, it springs into life and tells us... something. If you won't give it a coin, nothing for you.
I managed to find this at least:
https://idiomorigins.org/origin/penny-dropsSounds like your intuition is correct, and the phrase comes in particular from the delay in the penny takes to drop in these machines. I'd imagine if they were faster probably the phrase wouldn't have developed, so an interesting example of how some engineers somewhere who designed the slow mechanism ended up impacting the lexicon.