I ignore the database logging issue. When you have rate-limiting
issues again, please report, together with the description of what
happens.
If you think this is related to mysql, please address that issue first.
Rainer
El mié, 14 dic 2022 a las 17:48, helices
(<mike+rsys...@mdsresource.net>) escribió:
REF: Rsyslogd/ommysql.so: Not writing to DB intermittently
Rainer asked us to start a new post for the rate-limit issue.
A few of many hundreds of rate-limit errors and lost messages:
2022-12-13T02:23:44.003241-06:00 hermes rsyslogd[2539]:
rsyslogd[internal_messages]: 1792 messages lost due to rate-limiting (500
allowed within 5 seconds)
2022-12-13T02:23:50.001278-06:00 hermes rsyslogd[2539]:
rsyslogd[internal_messages]: 1779 messages lost due to rate-limiting (500
allowed within 5 seconds)
2022-12-13T02:23:56.001273-06:00 hermes rsyslogd[2539]:
rsyslogd[internal_messages]: 1835 messages lost due to rate-limiting (500
allowed within 5 seconds)
2022-12-13T02:24:02.005300-06:00 hermes rsyslogd[2539]:
rsyslogd[internal_messages]: 1768 messages lost due to rate-limiting (500
allowed within 5 seconds)
s
# date; grep -v "^\(#\|\s*$\)" /etc/rsyslog.conf ;date
Wed Dec 14 10:35:41 CST 2022
$DebugFile /var/log/rsyslog.debug
$DebugLevel 2
module(load="imjournal" Ratelimit.Burst="30000"
Ratelimit.Interval="1000" StateFile="imjournal.state")
module(load="imklog")
module(load="immark")
module(load="impstats" interval="600" severity="7")
syslog.=debug /var/log/rsyslog-stats
module(load="imtcp")
input(type="imtcp" port="514")
module(load="imudp")
input(type="imudp" port="514")
module(load="ommysql.so")
global(workDirectory="/var/lib/rsyslog")
authpriv.none;cron.none;*.info;mail.none /var/log/messages
authpriv.* /var/log/secure
cron.* /var/log/cron
*.emerg :omusrmsg:*
ftp.* /var/log/vsftpd.log
local7.* /var/log/boot.log
mail.* /var/log/maillog
uucp,news.crit /var/log/spooler
$ActionName Ftp
$ActionQueueFileName dbFtpQueue # Set file name, also enables disk mode
$ActionQueueSaveOnShutdown on # Save messages to disk on shutdown
$ActionQueueType LinkedList # Use asynchronous processing
$ActionResumeRetryCount -1 # Infinite retries on insert failure
ftp.*
:ommysql:10.199.5.177,vsftplog,hermesvsftplog,_____
$ActionName Sftp
$ActionQueueFileName dbSftpQueue # Set file name, also enables disk
mode
$ActionQueueSaveOnShutdown on # Save messages to disk on shutdown
$ActionQueueType LinkedList # Use asynchronous processing
$ActionResumeRetryCount -1 # Infinite retries on insert failure
authpriv.*
:ommysql:10.199.5.177,sftplogDB,hermesvsftplog,_____
$ActionName Admin
$ActionQueueFileName ZenossQueue # Set file name, also enables disk mode
$ActionQueueSaveOnShutdown on # Save messages to disk on shutdown
$ActionQueueType LinkedList # Use asynchronous processing
$ActionResumeRetryCount -1 # Infinite retries on insert failure
*.* @@10.199.1.160
Wed Dec 14 10:35:41 CST 2022
Rainer asked us to setup a debug log, according to:
https://www.rsyslog.com/doc/master/troubleshooting/howtodebug.html
Initial startup here:
https://pastebin.com/DUgwmPC
No rate-limiting occurred since early yesterday (12/13) morning. This
appears to be associated with the errors and multi-line syslog entries
mentioned in the other post.
The sole intent of the database logging is tracking all incoming remote
file transfer (SFTP) activities. There is a firewall between this host and
the internet. Only "whitelisted" IP addresses can get through, and are to
be inserted into the database.
Apparently, at least one client connects in the early morning hours, and
this unusual SFTP unusual activity results in multi-line syslog entries
that come in very large numbers. One problem is, the multiple line entries
are not written to /var/log/messages, are not inserted into the database,
and rate-limiting obscures all content. Hence, this support request is our
attempt to understand what is happening, after which we can act to correct
these problems.
Interestingly, we are not aware of any missing files from this or any
other file transfer clients.
