Thank you.
[1] rsyslog.conf
# date; grep -v "^\(#\|\s*$\)" /etc/rsyslog.conf ;date
Thu Dec 1 13:19:34 CST 2022
module(load="imjournal" StateFile="imjournal.state")
module(load="imklog")
module(load="immark")
module(load="impstats" interval="600" severity="7")
syslog.=debug /var/log/rsyslog-stats
module(load="imtcp")
input(type="imtcp" port="514")
module(load="imudp")
input(type="imudp" port="514")
module(load="ommysql.so")
global(workDirectory="/var/lib/rsyslog")
authpriv.none;cron.none;*.info;mail.none /var/log/messages
authpriv.* /var/log/secure
cron.* /var/log/cron
*.emerg :omusrmsg:*
ftp.* /var/log/vsftpd.log
local7.* /var/log/boot.log
mail.* /var/log/maillog
uucp,news.crit /var/log/spooler
$ActionName Ftp
$ActionQueueDequeueSlowdown 1000 # How long (in microseconds) dequeueing
should be delayed
$ActionQueueFileName dbFtpQueue # Set file name, also enables disk mode
$ActionQueueSaveOnShutdown on # Save messages to disk on shutdown
$ActionQueueType LinkedList # Use asynchronous processing
$ActionResumeRetryCount -1 # Infinite retries on insert failure
ftp.*
:ommysql:10.199.5.177,vsftplog,hermesvsftplog,VdGYphsZSN
$ActionName Sftp
$ActionQueueDequeueSlowdown 1000 # How long (in microseconds) dequeueing
should be delayed
$ActionQueueFileName dbSftpQueue # Set file name, also enables disk mode
$ActionQueueSaveOnShutdown on # Save messages to disk on shutdown
$ActionQueueType LinkedList # Use asynchronous processing
$ActionResumeRetryCount -1 # Infinite retries on insert failure
authpriv.*
:ommysql:10.199.5.177,sftplogDB,hermesvsftplog,VdGYphsZSN
$ActionName Admin
$ActionQueueDequeueSlowdown 1000 # How long (in microseconds) dequeueing
should be delayed
$ActionQueueFileName ZenossQueue # Set file name, also enables disk mode
$ActionQueueSaveOnShutdown on # Save messages to disk on shutdown
$ActionQueueType LinkedList # Use asynchronous processing
$ActionResumeRetryCount -1 # Infinite retries on insert failure
*.* @@10.199.1.160
Thu Dec 1 13:19:34 CST 2022
[2] How do we "log the message with the template RSYSLOG_DebugFormat to a
file?" How much disk space is needed? This problem appears to have started
recently, and appears to happen once or twice per day, without a common
time.
[3] I didn't notice the rate-limiting until now. It is not uncommon. How
can we avoid losing so many messages?
~ Mike
On Thu, Dec 1, 2022 at 1:05 PM David Lang <da...@lang.hm> wrote:
> please post your full config.
>
> It would also help to log the message with the template
> RSYSLOG_DebugFormat to a
> file and find the log entry that is failing to insert.
>
> my guess is that the quotes in the message are confusing mysql
>
> note that rate limiting is throwing away messages because you are trying
> to
> process them too fast.
>
> David Lang
>
> On Thu, 1 Dec 2022, helices via rsyslog wrote:
>
> > Date: Thu, 1 Dec 2022 10:08:01 -0600
> > From: helices via rsyslog <rsyslog@lists.adiscon.com>
> > To: rsyslog-users <rsyslog@lists.adiscon.com>
> > Cc: helices <mike+rsys...@mdsresource.net>
> > Subject: [rsyslog] Rsyslogd/ommysql.so: Not writing to DB intermittently
> >
> > # date; /bin/yum list rsyslog rsyslog-mysql ;date
> > Thu Dec 1 09:47:18 CST 2022
> > Loaded plugins: fastestmirror
> > Loading mirror speeds from cached hostfile
> > * base: download.cf.centos.org
> > * epel: mirror.genesisadaptive.com
> > * extras: download.cf.centos.org
> > * remi-php56: mirror.pit.teraswitch.com
> > * remi-safe: mirror.pit.teraswitch.com
> > * updates: download.cf.centos.org
> > Installed Packages
> > rsyslog.x86_64 8.2210.0-1.el7
> > @rsyslog_v8
> > rsyslog-mysql.x86_64 8.2210.0-1.el7
> > @rsyslog_v8
> > Thu Dec 1 09:47:19 CST 2022
> >
> >
> > Sample of numerous error messages (/var/log/messages):
> > rsyslogd[17344]: ommysql: db error (1172): Result consisted of more than
> > one row [v8.2210.0]
> > rsyslogd[17344]: The error statement was: insert into SystemEvents
> > (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt,
> > InfoUnitID, SysLogTag) values ('close
> > "/incoming/wood.pgez.scen.11302022.sa.pgp" bytes read 0 written 2603
> > [postauth]', 10, 'hermes', 6, '20221201081257', '20221201081257', 1,
> > 'sshd[19654]:') [v8.2210.0 try https://www.rsyslog.com/e/2218 ]
> > rsyslogd[17344]: rsyslogd[internal_messages]: 215 messages lost due to
> > rate-limiting (500 allowed within 5 seconds)
> > rsyslogd[17344]: action 'Sftp' (module 'ommysql.so') message lost, could
> > not be processed. Check for additional error messages before this one.
> > [v8.2210.0 try https://www.rsyslog.com/e/2218 ]
> >
> >
> > We have been writing all data from Internet file transfers to a Mysql
> table
> > for years. Recently, we began seeing intermittent errors like those
> above.
> >
> > What is happening here?
> >
> > What can we do to fix this problem?
> >
> > Please, advise. Thank you.
> >
> > ~ Mike
> > _______________________________________________
> > rsyslog mailing list
> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
> >
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
