Can definitely save a ton of space used by Git though:
git config --global core.compression 9 && find / -type d -name .git -exec git -C {} gc --aggressive \;
Conversation
Git history gets ridiculously bloated over time. Compacting it all with git gc --aggressive helps a lot. Setting core.compression to 9 gets a little bit more out of the zlib compression. Most size wins come from converting loose objects and smaller packs into unified packs.
1
2
11
I'm chronically low on disk space from working on multiple different trees based on Linux, Chromium and the Android Open Source Project. Stuck with this puny 2TB Samsung 980 Pro until they release the 4TB Samsung 990 Pro next year. Have learned lots of ways to save disk space...
1
3
Can also save tons of space with `cp --reflink` if your filesystem has support (XFS and Btrfs but not ext4). It's essentially fork(...) for files. It uses block-based copy-on-write at the destination. Can copy an identical file over another to deduplicate without sharing writes.
2
2
5
I moved to XFS from ext4 for better parallel NVMe performance and started using reflinks a lot. XFS doesn't have transparent compression support yet but I found it hurt build performance too much and most of my space is used by already compressed Git object/pack data anyways.
1
1
4
1
2
actually reducing the underlying disk space used by git wasn't implemented but would be possible by lazily fetching git history and otherwise maintaining a shallow clone. it would be a very large project but it would be fun to work on
1
1
XFS delayed allocation + delayed logging combined with 128GB of memory and a fast NVMe SSD means that barely anything ends up being I/O bound. End up with huge sequential writes as build outputs as flushed out. Reads mostly hit page cache. SSD stats have 10:1 write:read ratio.
1
2
Generating delta updates for GrapheneOS has parts that are extremely I/O bound unless it's done in tmpfs and 128GB memory is not nearly enough without using fewer jobs than CPU cores. It'd already go OOM without tmpfs with 1 job per thread (32) instead of 1 job per core (16).
this is also where the use case for big corps vs individual devs comes into play; at twitter we offloaded stuff onto remote process executions via the build tool but that's not available to most people + it requires rewriting the entire build for a remote enabled build tool
1
2
I need to do official builds, signing and delta generation locally for security and trust reasons so that's inherently bottlenecked by my local compute resources.
I wanted to make a 64 core Zen 3 Threadripper machine to help with this but Zen 3 Threadripper was never released.
2
3
Show replies
+1, if builds are your concern and you have the resources then tmpfs is definitely the way
2