Programming twitter: do you have stories when you used a reference count, and that reference count overflowed?
What happened, how did it get there, how did you find the issue, and how did you fix it?
32
20
126
Conversation
Replying to
The Linux kernel has had a few cases where 32-bit refcounts could overflow if you had enough RAM (starting at around sizeof(void*)*pow(2,32) = 32 GiB). In at least one of those cases, the fix was to just not allow that many references.
1
4
32
in a similar context, once proposed to have dynamically-sized refcounts - make the inline refcount 16 or 32 bits wide, and reserve a value that means "the real value is stored in a global hash table indexed by refcount address", or store a table index in the refcount
1
5
They often don't want to pay the memory cost of having properly sized reference counts or they prevented themselves from doing it by using up a limited amount of space for other things. I think my proposal was quite good and it'd be easy to make it transparent via the same API.
1
1
They didn't seem to like the concept though. I find it to be really horrible to put arbitrary limits on the number of objects where if you can reach that limit you're going to cause horrible problems because it could be the wrong processes getting screwed over by other ones.
The default vm.max_map_count of 65530 is ridiculously low and increasingly inadequate for many use cases. It's very common for raising it to be recommended for server applications.
hardened_malloc defaults to very fine-grained use of guard pages requiring raising the limit too.
2
1
Linux kernel ended up been largely incorrectly written in regards to this with the default low vm.max_map_count as an excuse for not doing it properly. It can be blamed on system administrators or distributions for raising the limit, but the limit is increasingly way too low...