How I want to simplify async filesystem APIs... 1/13
-
-
First, take a look at the filesystem APIs in async-std and tokio: 2/13 https://docs.rs/async-std/1.5.0/async_std/fs/index.html … https://docs.rs/tokio/0.2.13/tokio/fs/index.html …pic.twitter.com/KqHynp6smb
1 reply 0 retweets 1 likeShow this thread -
Both async-std and tokio are reimplementing the whole std::fs in an async manner. But can we invent some kind of async adapter for std::fs instead of replacing it entirely? 3/13
1 reply 0 retweets 1 likeShow this thread -
Consider a function like tokio::fs::read_to_string(). Do we really need it? 4/13 https://docs.rs/tokio/0.2.13/tokio/fs/fn.read_to_string.html …pic.twitter.com/8X85P1j2D5
1 reply 0 retweets 0 likesShow this thread -
Imagine there was a blocking!() macro that offloads a piece of code onto the blocking pool, kind of like spawn_blocking() with a nicer syntax. Then we could simply use std::fs::read_to_string() inside this macro. 5/13pic.twitter.com/GTjnEdHXrq
1 reply 0 retweets 0 likesShow this thread -
That's actually not too bad! If we have the blocking!() macro, we don't really need async versions of any of the standalone functions in the std::fs module. 6/13
2 replies 0 retweets 0 likesShow this thread -
What about the File type, though? In the vast majority of uses, files are only read or only written. Seeking is not a very common operation - in fact, I'd say seeking is a vestigial function that is now superseded by read_at()/write_at() functions. 7/13
2 replies 0 retweets 1 likeShow this thread -
Replying to @stjepang
read_at/write_at are Linux only though. Being able to seek asynchronously is essential for anything requiring random-access reads/writes cross-platform. This is common in for example databases. Calling it "vestigial" doesn't sound quite right. https://doc.rust-lang.org/std/fs/struct.File.html?search=read_at …
1 reply 0 retweets 0 likes -
Replying to @yoshuawuyts
They're not Linux-only - read_at/write_at exist on all Unix systems, including macOS/iOS/Android/etc. And Windows has similar functions called seek_read/seek_write: https://doc.rust-lang.org/std/os/windows/fs/trait.FileExt.html#tymethod.seek_read … https://doc.rust-lang.org/std/os/windows/fs/trait.FileExt.html#tymethod.seek_write …
2 replies 0 retweets 0 likes -
Replying to @stjepang @yoshuawuyts
I feel the concept of file cursors and seeking is an old C-ism that requires locking on the OS level and prohibits parallelism. It makes files stateful and is why File implementations in async-std/tokio are so complicated.
2 replies 0 retweets 1 like
Correct me if I'm wrong, but I believe modern databases have switched to read_at/write_at/seek_read/seek_write and don't really use cursors? Perhaps @sadisticsystems could chime in...
-
-
Replying to @stjepang @yoshuawuyts
Yep. Some less-performant databases only added support for pread/pwrite recently - postgres in 2019, sqlite in 2016. io_uring (the only relevant access method for high-performance IO workloads from now on) accepts an offset for async ops since it makes complexity reasonable
0 replies 0 retweets 5 likesThanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.