Hello! On Fri, Apr 30, 2021 at 11:14:48AM +0200, Charlie Kilo wrote:
> correction.. if i count with > ss -l -t |grep http | wc -l > we have around 58340 listening sockets.. at least on that machine.. Since "listen ... reuseport;" implies a separate listening socket for each worker process, that's expected with 50 worker processes and 1200 "listen ... reuseport" in the configuration. > On Fri, Apr 30, 2021 at 8:27 AM Charlie Kilo <krikkit...@gmail.com> wrote: > > > Thanks a lot for the hints so far.. to provide the further info and answer > > the questions.. > > > > getconf ARG_MAX shows 2097152 > > ulimit -s shows 8192 > > setting it to unlimited, doesn't change anything (also not with prlimit) > > wc -c /proc/<pid>/environ shows 1949 > > > > it seems on a regular machine we have around 1200 listening sockets and do > > indeed use "reuseport" Thanks, so all seem to be using the defaults. Looking more closely at Linux limits, it seems that you are hitting the MAX_ARG_STRLEN limit. It limits single argument length (and a single environment variable length), and hardcoded to be 32 pages, that is, almost always 128k. With 128k limit nginx should be able to pass about 20k listening sockets, so with 58340 sockets you are well above the limit. An obvious workaround would be to reduce the amount of listening sockets - either by dropping the "reuseport", or by reducing the the number of listening sockets (normally just one on the wildcard address is enough), or by reducing the number of worker processes. It would be interesting to know why you are using such a large number of listening sockets with reuseport enabled. If the use case is reasonable, we probably should consider implementing some workaround for the 128k limit. -- Maxim Dounin http://mdounin.ru/ _______________________________________________ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx
