For anyone who ends up with an "argument list too long" error when trying to build Android: run ulimit -s 9999999 first. The maximum command line size on Linux is a percentage of the stack size ulimit.
4
25
144
Conversation
I'm curious whether this is just an oversight in the build script. I recently ran into the "argument list too long" issue with my shell, and it was because I was using subshells instead of named pipes.
1
2
like android is pretty much the only build system i know where even just *starting* a build makes your system mostly unresponsive for the next 20 or so minutes
1
1
3
AOSP build system is ninja, and that's not my experience with it. It chooses the number of jobs based on the number of hardware threads by default rather than opting in, so if you're comparing to using 1 thread that's relevant. CPU cgroup + SCHED_BATCH helps with this in general.
2
i need to add my specs to that:
7200rpm hard disk, 8GB of DDR3 RAM, 8 core CPU
The ninja part is not a problem at all actually, but when you start a build it has to convert all of the makefiles and bp files into a ninja file which takes *ages*
2
1
some of the linking steps require even more memory but that's unrelated to the build system (aside from the fact that the build system does not execute linking steps serially)
1
I definitely wouldn't want it to serialize anything. I've already lost an enormous amount of time waiting for builds. If you want to use fewer jobs, you do have that option, and with 8GB of memory I don't think you can safely get away with more than 1-2 especially due to LTO.
2
I'm not aware of a build system that would go out of the way to serialize things like linking. The make-based build system didn't do that, and the jack compiler used far more memory than anything in the current builds. Peak memory usage now is from the Clang LTO needed for CFI.
1
ninja actually supports "pools" which, while not perfect makes the LLVM build much more bearable when it links clang and the like
1
Disable CFI/LTO and you'll probably find it uses drastically less memory. On my system, linking Chromium definitely uses far more memory than even the peak AOSP build memory usage, but I'm always building production (non-component) releases of Chromium where CFI/LTO are enabled.
So, what you're saying doesn't match my experience. Chromium build memory usage is definitely way higher, and both Android and Chromium use homegrown ninja generators (blueprint and gn) which is how ninja is intended to be used. If that counts as NIH, it definitely does for both.
1
I prefer working with the declarative blueprint files than gn, since it's far easier for me to analyze them and make changes. Build logic is far more contained to blueprint/soong, and I'd rather work with Go than a terrible build programming language. Still prefer gn over make.