Caveat: I've recently inherited a bunch of ubuntu systems with very little
historical knowledge available to me. The issue described below has occurred
only since the inline ubuntu upgrade was done. However, Ive no vision of
whether this is solely an issue with ubuntu implementation of rsyslog, or it's
a higher version of rsyslog that has this issue across all platforms
Systems: Ubuntu 20 (recent upgrade in-line from ubuntu 18)
Scenario:
While servers were ubuntu 18, clients had a a application that wrote logs into
a non-standard system log /var/log/node/Tlog.log (name obfuscated
[cid:image001.gif@01DA9232.6FE9BBE0] )
Previous rsyslog.conf configurations were in place to capture and include this
log into rsyslog for central syslogging purposes where the log would appear on
the central syslog server in /var/log/external/<client>/node/Tlog-<date>.log
client config:
$ModLoad imfile
...
$InputFileName /var/log/node/Tlog.log
$InputFileTag tserv-stdout
$InputFileStateFile tserv-stdout
$InputFileSeverity info
$InputFileFacility local4
$InputRunFileMonitor
That worked well for years allegedly/apparently.
a few months ago these systems were upgraded to ubuntu 20. It was shortly
afterwards noted that the central logging wasn't working well... logs were
very sporadic if at all.
Then it was discovered that somewhere along the line the ubuntu 20 systems
rsyslog config was needed to be altered (who knew?)
So now the client has
module(
load = "imfile"
pollingInterval = "1"
statefile.directory = "/var/log/node"
)
...
input(
type = "imfile"
tag = "tserv-stdout"
facility = "local4"
severity = "info"
file = "/var/log/node/Tlog.log"
)
Rsyslog tries very hard to not break backwards compatibility, so you should not
have needed to change the config. There have been bugs over the years, but in
general, a config should just keep working.
That seemed to fix matters - logs to Tlog.log on the client were appearing in
the central syslog log as well.
good, that should mean that the new style config is working
then about a week ago it just stopped working. All configs are as above ie the
ones that were working. rsyslogd is running. nothing obvious is in any other
log to suggest issues with rsyslog. Other system logs are updated centrally as
normal. It's just this extraneous log that has stopped "working". Tlog.log on
the client is constantly updated via its app (ie it is not a silent/inactive
log)
other logs from the same systems?
Any thoughts/pointers as to what the proverbial is happening, or how better to
troubleshoot it?
I've chucked a local4.* /var/log/node/Tlog.log entry into rsyslog.conf and
logger -p local4.info logs locally and centrally as expected using that - so
its not that eg networks have decided to block that access etc .
This is a good start. But at this point I am not understanding the problem. You
say that with this config it is logging both locally and centrally as expected,
what isn't working as expected?
and FWIW, both systems (client and rsyslog server) have this version of
rsyslogd
rsyslogd 8.2001.0 (aka 2020.01) compiled with:
PLATFORM: x86_64-pc-linux-gnu
PLATFORM (lsb_release -d):
FEATURE_REGEXP: Yes
GSSAPI Kerberos 5 support: Yes
FEATURE_DEBUG (debug build, slow code): No
32bit Atomic operations supported: Yes
64bit Atomic operations supported: Yes
memory allocator: system default
Runtime Instrumentation (slow code): No
uuid support: Yes
systemd support: Yes
Config file: /etc/rsyslog.conf
PID file: /run/rsyslogd.pid
Number of Bits in RainerScript integers: 64
one thing here is that, as this notes, you are using a version of rsyslog
initially released in Jan 2020, that has some number of changes backported to it
by Canonical, this is not a version provided by the rsyslog project (just based
on it), if you can upgrade to a current version to get the 4+ years of changes
and bugfixes (which do include a complete imfile rewrite amoung other things),
we would be in better shape to help. If you can't do that, we will do what we
can, but are far more limited in what we can do with such an old version.
based on your test, it sounds as if imfile is reading things, but not matching
something else on your central system. can you provide more info about the
config there?
David Lang
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
