The enforced gap is a special reserved area rather than an actual normal mapping. Secondary stacks are just a mapping made by the libc pthread_create implementation (or the caller, if they provide their own stack) with a guard region. I improved that in my past libc hardening.
Conversation
In my previous Bionic hardening, I added randomly sized guard regions on both ends similar to how I handle large allocations in the hardened allocator along with randomizing the stack pointer within the initial page to a certain extent up to a limit like 2% of available space.
1
Replying to
That's something I need to look into for HardenedBSD again: random-sized MAP_GUARD stack guard pages for per-thread stacks.
1
Replying to
You can see how it's implemented:
github.com/AndroidHardeni
It's only tricky for aligned allocations (not applicable to stacks) and isn't that complex.
Here's where it chooses the random guard size:
github.com/AndroidHardeni
It's stored alongside size:
github.com/AndroidHardeni
1
1
I meant to link to pages.c rather than memory.c:
github.com/AndroidHardeni
The whole purpose of pages.c / pages.h is to provide a wrapper around memory.c / memory.h implementing support for guard pages in a mostly transparent way. It's not entirely transparent due to realloc.
1
1
Since realloc actually has to be aware of how the guard pages work for the efficient approaches to large allocation shrinking, growth and moving pages to a new location with the mapping already reserved via MREMAP_MAYMOVE|MREMAP_FIXED. Those are just optimized fast paths though.
2
Replying to
That's another thing I need to implement in hbsd: mremap. llvm's Cross-DSO CFI uses it. mremap is specific to linux, though.
1
Replying to
MAP_FIXED_NOREPLACE can be quite useful too. It lets you attempt to map at a hint address without falling back to other addresses. It can be used instead of mremap for in-place expansion, but the most useful part of mremap is the ability to move pages efficiently via page tables.
1
Especially when transparent huge pages are being used, mremap moving multiple gigabytes of memory can be thousands of times faster since it only needs to move around the pages in the page tables. It's good even without transparent huge pages but that feature makes it amazing.
1
I tried out MAP_FIXED_NOREPLACE shortly after it was added and the Linux kernel implementation was actually totally broken so I had to report that and get it fixed. I experienced the same thing with Intel Memory Protection Keys combined with forking, which is *still* broken...
1
I only really wanted to use it to avoid calls to mremap when possible to reduce the performance hit from disabling mremap usage in order to remove it from system call whitelists, since it adds a lot of attack surface. I'd like a simpler alternative to mremap for moving pages.