Sébastien,
Is it possible that messages go to another log file? These messages go
to the main error log file, defined in the root context.
Another common pitfall is a log level above notice. Try setting error
log to a more verbose one, maybe?
Regards,
Igor.
On 28/03/2024 18:27, Sébastien Rebecchi wrote:
Hi Igor,
Thanks for the answer.
I really got that message 'signal process started' every time i do
'nginx -s reload' and this is the only log line I have, I don't have
the other lines you mentioned. Is there anything to do to enable those
logs?
Sébastien
Le jeu. 28 mars 2024, 16:40, Igor Ippolitov <iippoli...@nginx.com> a
écrit :
Sébastien,
The message about the signal process is only the beginning of the
process.
You are interested in messages like the following:
2024/03/26 13:36:36 [notice] 723#723: signal 1 (SIGHUP) received
from 69064, reconfiguring
2024/03/26 13:36:36 [notice] 723#723: reconfiguring
2024/03/26 13:36:36 [notice] 723#723: using the "epoll" event method
2024/03/26 13:36:36 [notice] 723#723: start worker processes
2024/03/26 13:36:36 [notice] 723#723: start worker process 69065
2024/03/26 13:36:36 [notice] 723#723: start worker process 69066
2024/03/26 13:36:36 [notice] 723#723: start cache manager process
69067
2024/03/26 13:36:36 [notice] 61903#61903: gracefully shutting down
2024/03/26 13:36:36 [notice] 61905#61905: exiting
2024/03/26 13:36:36 [notice] 61903#61903: exiting
2024/03/26 13:36:36 [notice] 61904#61904: gracefully shutting down
2024/03/26 13:36:36 [notice] 61904#61904: exiting
2024/03/26 13:36:36 [notice] 61903#61903: exit
Note the 'gracefully shutting down' and 'exiting' message from
workers. Also the 'start' and 'reconfiguring' messages from the
master process.
There should be a similar sequence somewhere in your logs.
Having these logs may help explaining what happens on a reload.
Kind regards,
Igor.
On 26/03/2024 12:41, Sébastien Rebecchi wrote:
Hi Igor
There is no special logs on the IP_1 (the reloaded one) side,
only 1 log line, which is expected:
--- BEGIN ---
2024/03/26 13:37:55 [notice] 3928855#0: signal process started
--- END ---
I did not configure worker_shutdown_timeout, it is unlimited.
Sébastien.
Le lun. 25 mars 2024 à 17:59, Igor Ippolitov
<iippoli...@nginx.com> a écrit :
Sébastien,
Nginx should keep active connections open and wait for a
request to complete before closing.
A reload starts a new set of workers while old workers wait
for old connections to shut down.
The only exception I'm aware of is having
worker_shutdown_timeout configured: in this case a worker
will wait till this timeout and forcibly close a connection.
Be default there is no timeout.
It would be curious to see error log of nginx at IP_1 (the
reloaded one) while the reload happens. It may explain the
reason for connection resets.
Kind regards,
Igor.
On 25/03/2024 12:31, Sébastien Rebecchi wrote:
Hello
I have an issue with nginx closing prematurely connections
when reload is performed.
I have some nginx servers configured to proxy_pass requests
to an upstream group. This group itself is composed of
several servers which are nginx themselves, and is
configured to use keepalive connections.
When I trigger a reload (-s reload) on an nginx of one of
the servers which is target of the upstream, I see in error
logs of all servers in front that connection was reset by
the nginx which was reloaded.
Here configuration of upstream group (IPs are hidden
replaced by IP_X):
--- BEGIN ---
upstream data_api {
random;
server IP_1:80 max_fails=3 fail_timeout=30s;
server IP_2:80 max_fails=3 fail_timeout=30s;
server IP_3:80 max_fails=3 fail_timeout=30s;
server IP_4:80 max_fails=3 fail_timeout=30s;
server IP_5:80 max_fails=3 fail_timeout=30s;
server IP_6:80 max_fails=3 fail_timeout=30s;
server IP_7:80 max_fails=3 fail_timeout=30s;
server IP_8:80 max_fails=3 fail_timeout=30s;
server IP_9:80 max_fails=3 fail_timeout=30s;
server IP_10:80 max_fails=3 fail_timeout=30s;
keepalive 20;
}
--- END ---
Here configuration of the location using this upstream:
--- BEGIN ---
location / {
proxy_pass http://data_api;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $real_ip;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 2s;
proxy_send_timeout 6s;
proxy_read_timeout 10s;
proxy_next_upstream error timeout http_502 http_504;
}
--- END ---
And here the kind of error messages I get when I reload
nginx of "IP_1":
--- BEGIN ---
2024/03/25 11:24:25 [error] 3758170#0: *1795895162 recv()
failed (104: Connection reset by peer) while reading
response header from upstream, client: CLIENT_IP_HIDDEN,
server: SERVER_HIDDEN, request: "POST
/REQUEST_LOCATION_HIDDEN HTTP/2.0", upstream:
"http://IP_1:80/REQUEST_LOCATION_HIDDEN
<http://IP_1:80/REQUEST_LOCATION_HIDDEN>", host:
"HOST_HIDDEN", referrer: "REFERRER_HIDDEN"
--- END ---
I thought -s reload was doing graceful shutdown of
connections. Is it due to the fact that nginx can not handle
that when using keepalive connections? Is it a bug?
I am using nginx 1.24.0 everywhere, no particular
Thank you for any help.
Sébastien
_______________________________________________
nginx mailing list
nginx@nginx.org
https://mailman.nginx.org/mailman/listinfo/nginx
_______________________________________________
nginx mailing list
nginx@nginx.org
https://mailman.nginx.org/mailman/listinfo/nginx
