Every time I want to rearrange virtual space within a process? mremap() 🤮
Conversation
Replying to
mremap is super broken because it returns EINVAL if you try to use it on multiple VMAs but the kernel does not reliably merge compatible VMAs.
I have a simple test case involving switching to PROT_NONE and back where it fails to merge them, and then mremap will return EINVAL...
1
1
Look at how painful it was to write this code to remap with guard pages:
github.com/GrapheneOS/har
MREMAP_DONTUNMAP can theoretically be used but I don't think it can expand the size of the mapping so it will be necessary to use mprotect in a way that may lead to non-merged VMAs.
1
1
It's the reason this in-place growth code was disabled:
github.com/GrapheneOS/har
It isn't particularly useful anyway, because in-place growth rarely works in practice.
There's separate code above that for in-place shrink.
All of this is needed due to the limitation of 1 VMA.
1
1
This is my test case if you're curious:
gist.github.com/thestinger/8a1
I haven't really bothered trying to get this fixed upstream at this point. It doesn't really help me that much since it would be so long before I could assume that the kernel actually works properly.
1
1
In this test case, the issue seems to be that it tracks whether a VMA was ever PROT_WRITE and it won't merge one that was PROT_WRITE at some point with one that wasn't. Maybe that makes sense. I think my actual problem was slightly different than this test case I ended up making.
1
1
I can't clearly remember why exactly github.com/GrapheneOS/har ends up triggering a similar issue. I mean... if you look at that, it doesn't seem like it would hit a similar issue, but somehow it does.
It extends guard to add space then unprotects that amount from the start of it.
1
IIRC, the issue is that the expanded usable area between the guards doesn't get reliably merged into a single VMA. That ends up breaking the MREMAP_MAYMOVE case below since it gets EINVAL when it tries to remap the usable area. It's such a nonsense API. Really hate it.
1
I can't remember why that happens. Maybe it doesn't like to merge a fresh PROT_READ|PROT_WRITE VMA with one that was touched. Maybe I could fix it by touching it, toggling it back to PROT_NONE, then back to PROT_READ|PROT_WRITE. Should just delete it since it barely helps anyway.
1
By the way, I was an early adopter of MAP_FIXED_NOREPLACE in hardened_malloc and discovered that it clobbers partially overlapping mappings:
patchwork.kernel.org/project/linux-
I was also seemingly one of the only people to ever use Memory Protection Keys. Was totally broken after forking.
1
1
mail-archive.com/linux-kernel@v was the MPK issue. It would entirely lose your pkey setup on fork. I had to work around it by disabling memory protection keys in the post-fork handler so it would just stop using the feature until exec. I've removed that workaround now, at least...
MPK is super disappointing now. I some neat use cases for it but the Spectre/Meltdown fixes / mitigations seem to have ruined the performance. It's still cheaper than mprotect but it seems like fast paths were lost. I guess no one uses it and it might go the same way as MPX.
1