I've really warmed up to 's suggestion of using template instantiation for hot generic functions and an interpreter for cold generic functions. You often want an interpreter for your language anyway, so it's an elegant way to reduce code bloat.
Conversation
Fully polymorphic code is pretty large too in machine code, and is full of indirect branches already. Byte code has the opportunity to be a lot smaller, and an interpreter could be nearly as fast while using up less instruction cache space
1
2
I rely heavily on LTO and dead code elimination in generic code for the Rust I write on microcontrollers (< 100kB ROM, < 10kB RAM). I don't think my code would appreciate embedding a full interpreter just because a few cold code paths couldn't be optimized out LOL.
2
1
Sure, there are definitely places where you don’t want to introduce dependencies on a runtime. Across a full blown OS on a desktop CPU, being able to make better use of memory and cache by sharing code is a big systemic optimization; for a single purpose controller not so much
1
2
Cool! I like your idea/think it's worth trying out. Just want an option to back out of embedding a runtime in the cases where "I know what I'm doing" :P.
2
On modern Android versions, apps start out using a multi-tier JIT compiler with an interpreter as the baseline. It saves the JIT profile information persistently and that's used to perform AOT compilation when the device is idle and during background installation of OS updates.
1
2
Cold code never ends up being compiled to native code, since it's never identified as hot enough to warrant being JIT compiled in memory and the AOT compilation is based on an aggregation of those JIT profiles. The AOT compilation is more aggressive but still skips lots of code.
1
1
So it goes bytecode, then JITted native code, and Android decides _some_ of the JITted code warrants further optimizations that wouldn't be practical during JITting?
1
JIT compilation is constrained by memory usage since it creates dirty pages in memory, along with being limited by time since compilation wastes CPU cycles / battery life. It's willing to compile much more code with AOT compilation in the background based on the JIT profiles.
Replying to
I was confused specifically about this wording:
>The AOT compilation is more aggressive but still skips lots of code.
Skips lots of already-JITted code, skips lots of bytecode, or both?
2
Replying to
I mean that while it knows how to AOT compile the entirety of the code, it will only AOT code that is not dead and not cold, based on the persistent JIT profiles. JIT compiled code is always kept in memory since it's not generated in a way that could be reused in the future.
The background compilation happens when the device is both idle and charging. It tries to do it once a day. OS updates are similarly done in the background and the final step of an OS update is to go through all the installed apps and redo background AOT compilation for them.