When shutting down the system, systemd sends a graceful shutdown message to
rsyslog, at which time is starts flushing the queues to disk, then 30 seconds
later (by default), it rsyslog has not finished shutting down, it sends it a
stop now (kill -9) message that aborts anything that is happening.
It's not uncommon for complex systems to take longer than that to shutdown
gracefully, and the default 30 seconds can be extended in the systemd config
see
https://askubuntu.com/questions/1384304/stop-services-gracefully-on-shutdown-increase-timer
That could be your issue. Please enable impstats (with a short reporting time)
and see what it shows is happening during the shutdown/startup process.
David Lang
On Wed, 26 Mar 2025, Mehmet Avcioglu via rsyslog wrote:
Date: Wed, 26 Mar 2025 15:11:18 +0300
From: Mehmet Avcioglu via rsyslog <rsyslog@lists.adiscon.com>
To: rsyslog@lists.adiscon.com
Cc: Mehmet Avcioglu <mta59...@gmail.com>
Subject: Re: [rsyslog] Log loss with RELP and Disk Assisted Queue
We have a setup where the source rsyslog sends log lines to the destination
rsyslog via RELP. The source has a queue file setup with type LinkedList
for disk assisted queue.
If connection to destination fails, and if after some time, while the
connection to destination is still not available, the source rsyslog is
also restarted, some logs are lost when the destination comes back online.
It was my expectation that the source rsyslog would save its buffer in the
queue file (saveOnShutdown) and continue where it left off when the
destination becomes available.
The source server is also picking up logs from imjournal and it doesn't
even save the log lines in /var/log/maillog. Are my queues getting mixed up?
source config:
module(load="imjournal"
StateFile="imjournal.state"
FileCreateMode="0644"
Ratelimit.Interval="0"
Ratelimit.Burst="0"
PersistStateInterval="100"
)
mail.* -/var/log/maillog
if ($syslogfacility-text == 'mail' or $syslogfacility-text == 'local4')
then {
action(type="omrelp"
target="10.10.10.50"
port="1514"
queue.filename="fwdRule"
queue.maxFileSize="1g"
queue.maxDiskSpace="100g"
queue.saveOnShutdown="on"
queue.type="LinkedList"
queue.checkpointInterval="100"
action.resumeRetryCount="-1"
action.resumeInterval="60"
tls="on"
tls.caCert="/etc/pki/tls/certs/ca.crt"
tls.myCert="/etc/pki/tls/certs/local.crt"
tls.myPrivKey="/etc/pki/tls/private/local.key"
tls.authmode="name"
tls.permittedpeer=["*.domain.net"])
}
destination config:
module(load="imrelp")
input(type="imrelp" port="1514"
tls="on"
tls.caCert="/etc/pki/tls/certs/ca.crt"
tls.myCert="/etc/pki/tls/certs/local.crt"
tls.myPrivKey="/etc/pki/tls/private/local.key"
tls.authMode="name"
tls.permittedpeer=["*.domain.net"]
)
$template DynaFile,"/var/log/%HOSTNAME%.log"
*.* -?DynaFile
Both ends are running 8.2408.0 on Rocky8 Linux.
Does anything look wrong with the above setup?
Thank you
--
Mehmet
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
