I think it's terrible that ext4 added this hack because it encourages developers not to fix their code to use actual transactions. There's a pervasive mistake of using fsync on the file after the rename instead of before the rename which is broken and can leave inconsistent data.
Conversation
IIRC the problem was that people were doing fsync/rename backwards and not fixing their code, even after wide-spread reports of data loss.
Presumably because previous filesystems happened to get away with it?
I guess slow is better than lost data?
1
It's not slow to do a properly scoped fdatasync before rename. It's faster than doing a full fsync on the file after rename, which doesn't even guarantee that the rename was synced because doing that requires an fsync on the directory rather than the file anyway.
2
By doing rename before sync, it's possible the rename is committed to storage before data. On ext3, it wasn't possible with data=ordered because it didn't have delayed allocation of blocks, but that was never guaranteed or portable. Doubt developers knew about that guarantee.
2
I disagree with this assessment. It was guaranteed by what my, and I think many others', understanding of what data=ordered meant. Delayed allocation is a hack that licenses the fs driver to violate what data=ordered means.
1
It's only relevant what data=ordered means if the code was specifically written for ext3 rather than being portable across filesystems. It shouldn't matter what data=ordered means for the vast majority of code not written to target a specific filesystem implementation.
2
From my perspective, you're framing it wrong as the code being "written for" anything. The code is just written to POSIX (without sync options) with no concept of power failure existing. Filesystem is providing user-facing (not app-facing) features on top...
1
...to give the user (not the application) a level of data integrity they've selected (and made tradeoffs for) based on whether they deem power failure or kernel crash something that could plausibly happen and whether they deem the data valuable.
1
You can choose to use ext4 without delayed allocation. From an application or library perspective, it's not relevant as portable code implementing transactions correctly still works fine. Relying on the ext3 ordering is not just broken on ext4 with delayed allocation.
2
You're treating this as a [non-]contract between the application and the filesystem, in which case of course it would be "non-portable assumptions". I treat them as having no relationship, and this being a matter of contract between the user and the filesystem.
1
Okay, but if you want transactions in application or library code, fsync/fdatasync are required because both filesystems and hardware delay writes and perform them in batches with the order determined based on efficiency. It's not something specific to ext4 at all.
Using rename doesn't make it a transaction. That's true with ext3 too. The only difference with ext3 was that using fsync in the wrong place provided non-portable guarantees about the order the metadata will be written that many applications were relying upon.
1
If there was no fsync at all, using rename still wasn't a transaction for ext3. Delayed allocation exists for not just ext4 but also XFS, ZFS, Btrfs, HFS+, APFS, NTFS, etc. Strict ext3 fsync guarantees are a big part of why applications didn't use transactions due to bad perf.
1
Show replies
I don't want transactions. This thread is not about transactions or any programming-level contracts. It's about the extent of inconsistency that can be observed in their absence with different options and nasty hacks that hurt performance for related reasons.
1
There's a reason that no other filesystem acts like ext3 with data=ordered. The design forced awful performance for everything, and especially anything using transactions. The performance issues still exist in ext4 but they papered over it and added hacks for broken applications.
2
Show replies