It turns out that I can't even use 4-level pages tables in practice for arm64 due to buggy applications, drivers, etc. It's close to working but there are some showstoppers with hard-wired assumptions. Maybe it can be be worked around but I don't enjoy needing to fix everything.
Conversation
Replying to
Yeah, I was really annoyed by this. I could probably work around the Adreno issue or report it and get them to fix it within a few months but I didn't anticipate the level of breakage there would be from enabling usage of more address space. It's pretty frustrating for me.
1
hardened_malloc reserves dedicated regions for each slab size class and their metadata. It never reuses or obtains more address space by design, for security reasons. There are 4 size classes per doubling in size (github.com/GrapheneOS/har) so there are a lot. Arenas multiply this.
1
1
So if I want to use 8 arenas, that means reserving 8x as much address space due to how it works. If the address space was 56-bit, running out wouldn't be a concern, but 48-bit is not really that much especially with other features also heavily using it like Clang cross-DSO CFI.
1
1
I haven't looked into it yet but some of these many apps I use for testing might have some weird runtime or data structure abusing those bits for storing tags. One thing that really annoyed me is what Go does with the brk heap because I want to have that awful thing disabled.
1
1
You can see from the configuration in github.com/GrapheneOS/har that I'm not currently using arenas on GrapheneOS and size class regions are set to 1GiB of usable space (they end up 2x as large for guards) which isn't very future proof. Half that space gets used by guard slabs too.
1
I want to make it so it makes a random choice between inserting 1 or 2 guard slabs instead of only using 1 guard slab between each slab. That's going to further reduce the usable space. I'm already needing to make many compromises based on address space size. Can't do everything.
1
It actually often boots up with 4-level page tables but causes some weird graphical artifacts / corruption and things start crashing repeatedly. I'm convinced that it's an Adreno bug, although it's possible it's a bug with something Adreno uses rather than it being their fault.
1
1
There are clearly some other more application-specific issues too. Did not expect to encounter any issues with this. I'm too annoyed to even look into it for a while longer. I gave up on it for the time being. Bit of a shock when stuff ends up being so broken like this...
1
1
So I was thinking that I could reserve all that extra address space myself to prevent anything else from using it. I don't think any of the bugs are on the kernel side of things. The issue is there are annoying things not using the system libc and I can't easily deal with them.
Probably worth clearing off the layer of dust and golden retriever hair from my HiKey 960 and checking how well this works on Kirin instead of Snapdragon. That thing gets ridiculously hot and it's probably a fire hazard right now... literally burned my finger moving it before.
1