Before we move onto simplifying async files in Rust, it's important to learn in which areas the current implementations need improvement. This might surprise you... 1/9
-
Show this thread
-
Create a random 500 MB file, read its contents using std::fs::File, and then do the same with tokio::fs::File. 2/9 dd if=/dev/urandom of=bigfile bs=1M count=500pic.twitter.com/hSdJVoFtCI
1 reply 0 retweets 7 likesShow this thread -
On my machine, std takes 0.238 sec and tokio takes 120 sec, meaning tokio is 500 times slower! Note that this is not some kind of obscure edge case - we're literally just reading a big file. 3/9
1 reply 0 retweets 8 likesShow this thread -
So why are async files so inefficient? Well... because reasons that are too boring for this twitter thread. The more important question to ask is: How can we be comfortable using async files if they can be 500 times too slow without even realizing it? 4/9
2 replies 0 retweets 6 likesShow this thread -
And performance is not the only pitfall of async files. Here's another - write some bytes into a tokio::fs::File. If you forgot to flush before dropping the file, some buffered data will be lost! In this example, the created file is empty. 5/9pic.twitter.com/PFJEsduJiB
1 reply 0 retweets 7 likesShow this thread -
Perhaps most surprisingly, async-std/tokio files are *always* slower than std files. They don't improve performance at all, quite the opposite - their only utility is in that they move blocking file I/O onto a dedicated thread pool. But you can also do that yourself anyway. 6/9
1 reply 0 retweets 6 likesShow this thread -
If you do want fast async files, consider using rio by
@sadisticsystems instead. That is really the only way! 7/9 https://github.com/spacejam/rio https://docs.rs/rio3 replies 3 retweets 23 likesShow this thread -
My advice is to avoid async files in Rust altogether because they're inefficient and full of pitfalls (unless you use rio). Use them only in situations when you know what you're doing, if ever. Things might get different in the future, but that is what I advise *today*. 8/9
2 replies 0 retweets 11 likesShow this thread -
What about calling spawn_blocking() and doing synchronous file I/O there? Is that an option?
-
-
Replying to @stjepang
indeed it is, and didn't pay attention to that bit, sorry. Just a tiny bit more cognitive load there though. Generally, we've been telling people to always use the async IO API so a bit of unlearning/relearning involved here as well it think..
0 replies 0 retweets 1 likeThanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.