Use garbage collection, they said. Manual memory management is too hard, they said. Garbage collection means you don't need to think about memory, they said.
Now the code uses 14m of actual memory, but my OS needs to provide 400m for those 14m, and I am reading the runtime.
Conversation
Seriously, I would *love* golang if I could have it as not-GC'ed.
7
2
39
Replying to
I am not sure that’s Go fault. With TamaGo, which keeps the Go runtime almost identical but on ARM bare metal and therefore without the OS virtual memory, we don’t see much memory waste. And we have much less memory to spare. Are you sure it isn’t just Linux overcommitting?
1
Replying to
From golang's own accounting: Alloc: 31324248 Sys: 917090656 - 900 megs of system memory for 31 megs of live heap-allocated objects. But I am not sure yet, I am in the process of root-causing what is going on.
1
2
Replying to
Sys is the total bytes of memory obtained from the OS, I think Alloc is what matters. Isn't this just the OS over committing VM to the process?
1
Replying to
Let me root-cause this; we're both guessing into the dark. I suspect this is a combination of heap fragmentation & bad scavenging after short memory usage spikes.
1
Go doesn't use a moving (compacting) GC, so it isn't good at reclaiming memory from small objects.
They've chosen to value low pause times and minimal impact on the latency / throughput of pointer accesses above everything else.
It uses lots of memory and has bad throughput.
1
2
Allocation is far slower than a more mainstream modern GC and they don't compact the heap. In exchange, they avoid the cost of ever needing read barriers, largely avoid the cost of write barriers and still have quite low latency. Allocation of small objects resembles tcmalloc.
1
4
Since it's not a compacting collector, you can largely ignore the garbage collector when considering fragmentation. Look at golang.org/src/runtime/ma. It has the typical compromises of a modern malloc implementation but also needs a whole bunch of free space to defer collections.
Their approach makes sense when latency matters a lot and most of the code avoids heavy allocation. If you have a workload where throughput is all that matters, the design is a bad fit, especially if you need lots of dynamic allocation - not nearly as cheap as a mainstream GC.
1
Mainstream modern garbage collectors are compacting, generational collectors, so they repeatedly compact the heap by copying all of the live objects to a new one. Long-lived objects end up moving into older generation heaps. Go takes another path of trying to use the stack more.
1
Show replies