I'm not surprised to find out that Linux kernel socket load balancing is in a terrible state with no good options. Some background:
blog.cloudflare.com/the-sad-state-
Traditional epoll wakes every thread and they race to accept the connection. EPOLLEXCLUSIVE fixes it but uses LIFO order.
Conversation
LIFO order is terrible for a web server. HTTP connections are generally long-lived and reused for mixed / varying workloads. That's even more true with HTTP/2 where clients are only supposed to make a single connection to each server and multiplex everything over it concurrently.
1
6
In a standard nginx setup on Linux, it uses EPOLLEXCLUSIVE. This gives nearly all the connections to the same worker until it starts getting overloaded. Even then, the most overloaded workers still keep getting the most connections among handling other events. It's pretty awful.
1
3
The alternative that's offered is using reuseport to have the kernel evenly distribute new connections across workers.
It doesn't account for the varying usage of connections. It happily hands out new connections to a worker that's not even idle. It's not as nice as it sounds.
1
1
The kernel developers were gradually improving REUSEPORT in different ways. However, they stopped and allowed some of it to regress. You're supposed to use BPF to give the kernel a load balancing approach specific to the application. Of course, applications don't actually do it.
2
1
2
Replying to
It's mentioned in passing in blog.cloudflare.com/perfect-locali. I found the relevant mailing list information previously but don't have it available.
Of course the fastest performance will be obtained via kernel bypass, but that is *really* annoying.