I'm heavily focused on smartphones where nearly all the drivers are out-of-tree on launch and slowly trickle into the kernel. Literally a majority of the code is not from kernel.org but will gradually trickle there in the years following the launch of an SoC/device.
Conversation
Embedded and phones are a hot mess. I have been lucky to mostly avoid it.
Android's deep kernel fork didn't help. Hopefully that will continue to improve, but it will take time and effort.
3
Android hasn't had a substantial kernel fork for years. It works with mainline kernels.
SoC vendors take latest LTS branch when developing their SoC, implement all their drivers for it, often with a lot of new/rewritten code rather than it just being a minor port from old code.
1
SoC vendor is the upstream for half of the kernel code used on those devices rather than kernel.org.
When Google says they're moving to upstream first they mean giving up on upstreaming remaining few Android common kernel changes and using inferior hacks or eBPF.
1
They defined a stable ABI for kernel modules and hardware support will be done primarily via out-of-tree modules targeting a stable ABI for an LTS branch. ABI can't be at all stable across LTS versions. They won't convince Linux to do anything like that any time soon.
1
github.com/GrapheneOS/ker (just that, no userspace code was dropped) was replaced with literally thousands of lines of code in userspace across multiple components to dynamically manage eBPF rules. What they mean by upstream first is making about a dozen sacrifices like that.
1
2
A particularly painful example is that they switched from ashmem to memfd despite losing the ability to unpin/pin pages similarly to MEM_RESET and MEM_RESET_UNDO on Linux where you can tell the kernel it can free that memory if there's memory pressure but you can ask for it back.
1
1
This was used by web browsers to implement volatile caching aware of memory pressure on Android as they do on Windows since the kernel takes care of it and purges the caches. No such thing on non-Android Linux and now it will be gone on Android, increasing browser memory use.
1
1
They got close enough to not having any required Android common kernel code for them to decide to make these final dozen sacrifices to be able to directly use kernel.org kernels with backported bug fixes, performance improvements and in some cases actual features.
2
1
Replying to
I think there still some active development for proactive reclaim / pageout? But maybe not the part where you are able to undo that...
1
Replying to
I helped get MADV_FREE landed and that's a form of lazy free designed for use with malloc/free and similar systems. It does result in the weird property of uninitialized malloc memory being able to change from non-zero data to zero data, but that's how GCC/LLVM already handle it.
It doesn't drop the pages once writes happen to them via hardware support for detecting that they became dirty. On hardware without support, it can just map to MADV_DONTNEED. Use case for Android ASHMEM_UNPIN/ASHMEM_PIN or Windows MEM_RESET/MEM_RESET_UNDO is mostly different.
1
You generate data you need, use it, and then ASHMEM_UNPIN/MEM_RESET. Later, when you need it again, you do ASHMEM_PIN/MEM_RESET_UNDO. Between those, the kernel was able to drop the memory if there was memory pressure. Extremely useful for a lot of often large in-memory caches.
1
Show replies