Conversation

🤣🐧🤡 This would not even be an issue if everyone hadn't jumped on the epoll 🤡🚗

Quote Tweet

May 1, 2021

I'm not surprised to find out that Linux kernel socket load balancing is in a terrible state with no good options. Some background: blog.cloudflare.com/the-sad-state- Traditional epoll wakes every thread and they race to accept the connection. EPOLLEXCLUSIVE fixes it but uses LIFO order.

Show this thread

Replying to

The Linux kernel didn't really provide other viable options. The least bad option not requiring patching the kernel is probably REUSEPORT combined with dealing with the BPF nonsense they expect you to do. You still use epoll but with a socket per worker instead of sharing one.

Replying to

Just all workers blocked in accept() for a single socket works fine. Only one of them wakes, the one that gets the connection.

Replying to

and

I think people often go for epoll-driven code when blocking + threads/processes would handle their scale just as well, with better latency. However, a buffering reverse proxy like nginx is a nearly mandatory component between that sort of blocking code and the open internet.

Replying to

and

The reason why I am looking into this is that our servers are under fairly consistent denial of service attacks. Kernel doing a terrible job with load balancing for EPOLLEXCLUSIVE is resulting in barely using the available cores to spread out overhead of new TLS connections.

12:47 AM · May 2, 2021Twitter Web App