In a standard nginx setup on Linux, it uses EPOLLEXCLUSIVE. This gives nearly all the connections to the same worker until it starts getting overloaded. Even then, the most overloaded workers still keep getting the most connections among handling other events. It's pretty awful.
Conversation
The alternative that's offered is using reuseport to have the kernel evenly distribute new connections across workers.
It doesn't account for the varying usage of connections. It happily hands out new connections to a worker that's not even idle. It's not as nice as it sounds.
1
1
The kernel developers were gradually improving REUSEPORT in different ways. However, they stopped and allowed some of it to regress. You're supposed to use BPF to give the kernel a load balancing approach specific to the application. Of course, applications don't actually do it.
2
1
2
A decent approach for many applications would be if epoll used FIFO instead of LIFO order. It would work like reuseport but only distributing connections to idle workers. It was proposed in 2015 with EPOLLEXCLUSIVE but didn't land. It's apparently what Cloudflare uses themselves.
1
3
marc.info/?l=linux-fsdev was an attempt at reviving it. Cloudflare tends to not be persistent enough to get through the hassle of getting changes landed in the Linux kernel or nginx.
They deal with the initial technical aspect but not the politics / evangelism to get it landed.
1
4
They also tend not to invest resources into making things into a more generic solution suitable beyond their use case. Makes sense for them. They clearly don't see that much value in their changes being upstream. The upstream projects tend not to care much about latency, etc.
2
3
Replying to
They'll either do the work to make it useful enough to go upstream, or they'll eventually die in rebase hell when their patches don't apply any more, and bug aren't found on it because people are running all their tests on the vanilla branches.
1
I get the impression that Cloudflare is increasingly trending towards simply forking nginx and going their own way. They attempted to upstream various things and mostly didn't succeed. Most of them would be very useful to others: dynamic TLS records, async open and other things.
1
1
I'm sure they have a lot of useful changes and nginx modules which they've made no attempt to upstream. Upstream nginx development is glacial despite lots of areas to improve and it doesn't help that it's an open core project with some conflicts of interest involved.
1
1
I think the worst example is that the feature to queue up connections for an upstream when it goes above the configured connection limit is only available in NGINX Plus. There are other examples. NGINX Plus packages third party modules so they lack the incentive to upstream them.
So, for example, Google maintains an nginx brotli project with dynamic and static brotli modules. NGINX Plus provides that for you. It should be upstream by now. It would be easy to make the upstream gzip and particularly gzip_static modules generic. Not really in their interest.
1
There are other attempts to get patches upstream which fail or take ages. It goes slowly. No io_uring yet, TCP_NOTSENT_LOWAT only on FreeBSD, etc. nginx is a lot like Linux where people tend to maintain downstream patches. Cloudflare lacks the persistence to land stuff in either.
1
1
1
Show replies