Conversation

Jun 26, 2021

💯 > In general, it is my opinion that if your program is crashing on Alpine, it is because your program is dependent on behavior that is not guaranteed to actually exist, which means your program is not actually portable

Quote Tweet

Ariadne Conill

@ariadneconill

Jun 25, 2021

understanding thread stack sizes and how alpine is different ariadne.space/2021/06/25/und

Replying to

also glibc sucks for defaulting to giving every thread 8MiB of stack. if you're wondering why alpine is more memory efficient, this is a HUGE part of it

Replying to

and

Wait, isn't that committed on demand? It doesn't actually eat 8 MiB of physical RAM per thread, does it?

Replying to

and

No, but it can use that much so it's accountable memory and requires backing space in memory or swap when memory accounting is enabled. The main thread stack works differently and doesn't actually map all the space at once. It gradually maps it in as the program uses the memory.

Replying to

It means if you want to have reliable main thread stack space under memory exhaustion, you need to touch all of the main thread stack you're going to need later on up front to get it mapped in to the process. Alternatively, you can just not use the main thread after init.

Replying to

You can gradually read it in blocks based on the configured thread gap size to map it in when your process starts and then optionally add your own guard where you want the limit to be since the kernel doesn't necessarily even have a specific limit configured.

Replying to

If the stack limit is set to unlimited, it allows it to grow to some arbitrary amount until it gets something else. Before they had the proper thread gap feature, this didn't work properly and you could just hit something else and overflow into it with no actual guaranteed guard.

4:14 AM · Jun 26, 2021Twitter Web App

Replying to

It's still pretty fucking weird unless you actually go out of the way to get your main thread stack pages mapped in advance and add a normal guard at the end. I also think that /proc/PID/maps now lies about what's happening to deter stuff from mapping there even though it could.

Replying to

Can't remember all the details off hand. It was a really bad long-term problem with the Linux kernel that should be fully resolved now when combined with -fstack-clash-protection. The problem wasn't as bad for secondary stacks but was unsafe with large frames / VLAs / allocas.