We are migrating AWS AL1 servers (EOL) to AWS AL2023, with both instance types
using rsyslog. All servers use remote logging, except for the remote server
receiving the logs. Migration of the servers generating the logs will take some
time, so some servers will remain on AL1 for a bit, while others are ready for
AL2023 migration. Both the AL1 and AL2023 servers need to be configured for
remote logging during the migration period.
The AL1 servers are running rsyslog v5:
$ rsyslogd -v
rsyslogd 5.8.10, compiled with:
FEATURE_REGEXP: Yes
FEATURE_LARGEFILE: No
GSSAPI Kerberos 5 support: Yes
FEATURE_DEBUG (debug build, slow code): No
32bit Atomic operations supported: Yes
64bit Atomic operations supported: Yes
Runtime Instrumentation (slow code): No
The AL2023 servers are running rsyslog v8:
$ rsyslogd -v
rsyslogd 8.2204.0-3.amzn2023.0.4 (aka 2022.04) compiled with:
PLATFORM: x86_64-amazon-linux-gnu
PLATFORM (lsb_release -d):
FEATURE_REGEXP: Yes
GSSAPI Kerberos 5 support: No
FEATURE_DEBUG (debug build, slow code): No
32bit Atomic operations supported: Yes
64bit Atomic operations supported: Yes
memory allocator: system default
Runtime Instrumentation (slow code): No
uuid support: Yes
systemd support: Yes
Config file: /etc/rsyslog.conf
PID file: /var/run/rsyslogd.pid
Number of Bits in RainerScript integers: 64
The remote host for logging has been migrated to AL2023, and logging from AL1
hosts works as expected. The drop-in rsyslog conf file for the remote host
worked unchanged when migrated from AL1 to AL2023, and contains rules:
if re_match($programname, 's[0-9]+_misc') and $syslogseverity == 5 and $msg
startswith ' NBA' \
then /var/log/remotelog/nba/server
& stop
if re_match($programname, 's[0-9]+_misc') and $syslogseverity == 6 and $msg
startswith ' NBA' \
then /var/log/remotelog/nba/connections
& stop
The drop-in rsyslog config file on the local AL1 host is:
local1.none /var/log/messages
$template Server3Template,"%timestamp% %hostname% %syslogtag% %msg%\n"
local1.=notice Server3Template
local1.=info Server3Template
local1.=debug Server3Template
$WorkDirectory /var/lib/rsyslog # where to place spool files
$ActionQueueFileName fwdRule1 # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueType LinkedList # run asynchronously
$ActionResumeRetryCount -1 # infinite retries if host is down
*.* @@10.16.130.0:514
The above configuration was tested with the following commands on the local AL1
host:
$ logger -t s4_misc -p local1.notice "NBA-US from AL1 notice"
$ logger -t s4_misc -p local1.info "NBA-US from AL1 info"
Which results in a single entry in /var/log/remotelog/nba/server on the remote
host for local1.notice:
May 13 19:36:50 ip-172-16-3-0 s4_misc: NBA-US from AL1 notice
And also results in a single entry in /var/log/remotelog/nba/connections on the
remote host for local1.info:
May 13 19:38:54 ip-172-16-3-0 s4_misc: NBA-US from AL1 info
The migrated drop-in file on the local AL2023 host is:
$template Server4Template,"%timestamp% %hostname% %syslogtag% %msg%\n"
local1.* action(type="omfwd"
queue.filename="fwdRule1" # unique name prefix for spool files
queue.maxdiskspace="1g" # 1gb space limit (use as much as possible)
queue.saveonshutdown="on" # save messages to disk on shutdown
queue.type="LinkedList" # run asynchronously
action.resumeRetryCount="-1" # infinite retries if host is down
template="Server4Template"
target="10.16.130.0"
port="514"
protocol="tcp"
)
The above configuration was tested with the following commands on the local
AL2023 host:
$ logger -t s4_misc -p local1.notice "NBA-US from AL2023 notice"
$ logger -t s4_misc -p local1.info "NBA-US from AL2023 info"
Which results in a both entries in /var/log/remotelog/nba/server on the remote
host for local1.notice and for local1.info:
May 13 19:59:48 ip-10-24-7-0 s4_misc[54120]: NBA-US from AL2023 notice
May 13 20:04:02 ip-10-24-7-0 s4_misc[54993]: NBA-US from AL2023 info
and the local1.info entry does not get logged in
/var/log/remotelog/nba/connections for the AL2023 local host, unlike the
logging from AL1. We want the local1.info entries for the AL2023 hosts to go to
the same file as the AL1 hosts.
We tried changing the filter rules on the remote host to use the new syntax:
if re_match($programname, 's[0-9]+_misc') and $syslogseverity == 5 and $msg
startswith ' NBA' then {
action(type="omfile" file="/var/log/remotelog/nba/server")
}
& stop
if re_match($programname, 's[0-9]+_misc') and $syslogseverity == 6 and $msg
startswith ' NBA' the {
action(type="omfile" file="/var/log/remotelog/nba/connections")
}
& stop
The above change made no difference to the remote log output for the local
AL2023 host. After making the above change, no remote log output from the local
AL1 host was saved at all on the remote host (i.e. AL1 remote logging stopped
working).
How can we configure the servers so remote logging for the AL2023 servers works
like the AL1 servers, where local1.notice and local1.info are saved in
different files on the remote AL2023 host?
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
