So TIL Linux's ext4 fs driver in the default configuration imposes a bogus fsync on every atomic replace via rename.
Conversation
The feature is called auto_da_alloc, and the documented purpose is correct - ensuring that, *if* the rename is seen after async power failure, the data in the new file is also seen. But then it goes and does more...
1
1
...which is wrong. It makes the whole rename syscall sleep until the data is committed, as if it were rename+fsync. Which makes sense if you cared about the data, but then you would have called fsync yourself!
2
1
1
Replying to
So the ultimate desired behaviour would be:
- Once rename() returns, new is replaced with old for the running system (in-memory)
- Some time after rename() is called, new is fsynced then replaces old on disk (if a restart happens)
The in-between time sounds scary to me?
2
No, to do a transaction safely, you need to fsync the file to commit the data to storage, rename it and then fsync the containing directory to commit the transaction. By covering up the problem, ext4 is hurting performance while the bugs in software are still present and serious.
Aha, so you can fsync the containing directory to ensure rename is stable on disk?
1
Yes, that's the way to do it as an atomic transaction where either the previous data or the new data is guaranteed to be intact. Since many developers got it wrong and continue to get it wrong the ext4 developers hard-wired hacks to work around broken code at the expense of perf.
1
Show replies