I will also point out that templates in rsyslog are for output only, they have
no effect at all on parsing input.
David Lang
On Tue, 13 Aug 2024, David Lang wrote:
Date: Tue, 13 Aug 2024 13:42:38 -0700 (PDT)
From: David Lang <da...@lang.hm>
To: "Drumm, Daniel" <daniel.dr...@dob.texas.gov>
Cc: David Lang <da...@lang.hm>, rsyslog-users <rsyslog@lists.adiscon.com>
Subject: RE: [rsyslog] Formatting CEF to log.
Drumm, Daniel wrote:
David -
If you want me to instance a brand-new Ubuntu 22.04 VM in the cloud, get
rsyslog on it, and move the 514/udp Palo flow to it, I can do that.
Conversely, if you have access to a PAN-OS instance or old physical
firewall, page 8-10 of this guide will set the CEF parameters and, if I am
guessing correctly, will cause rsyslogd not to log the messages anywhere
even with a *.* directive.
https://docs.paloaltonetworks.com/content/dam/techdocs/en_US/pdf/cef/pan-os-10-0-cef-configuration-guide.pdf
If, as you state, the behavior of rsyslog is best effort to write to log
even if the fields are misaligned, then I cannot explain the behavior I
see. Per the discussion on tcpdump, I do not believe these UDP packets to
be corrupt or excessively fragmented beyond the MSS. I cannot explain why
the basic *.* is not logging the messages when the CEF formatting is
applied to the firewall. Again, looking at the packets using tcpdump -v, I
know they are arriving, and are about 50% longer than the base log
messages.
it's beyond that, rsyslog does not parse the fields out of the log as it
receives them.
I don't have a pan firewall handy, so please run this test and show the
results.
David Lang
Daniel Drumm, CISSP
Information Security Officer
Texas Department of Banking
512-475-1328
daniel.dr...@dob.texas.gov
-----Original Message-----
From: David Lang <da...@lang.hm>
Sent: Tuesday, August 13, 2024 2:20 PM
To: Drumm, Daniel <daniel.dr...@dob.texas.gov>
Cc: David Lang <da...@lang.hm>; rsyslog-users <rsyslog@lists.adiscon.com>
Subject: RE: [rsyslog] Formatting CEF to log.
Drumm, Daniel wrote:
Is there a rsyslog Template that exists to take the data in this
format and log it? As you state, when rsyslogd gets a malformed
message, or one with fields additional to that it understands or
knows, it will not log it at all, will not write it to disk at all, not
even a partial malformed log.
No, that is the opposite of what I stated.
When Rsyslog recieves a message, it attempts to parse the timestamp,
fromhost, and syslog tag out of it, and everything else is the string
'msg'. If the log is not properly formatted, it may put the wrong things in
these fields, but it will put SOMETHING there (best effort) It also
populates the filed 'rawmsg' with whatever arrived.
it does not care if the msg field is CEF:whatever or 'the quick brown
fox...'
you can then parse fields out of the message if you care to, but the msg
and rawmsg fields will be there no matter what.
If the message is longer than your configured maxmessagelength, you will
get a log message that is truncated at that length (and then, depending on
the config, either throw away the rest of the message, or split it at that
point into a second, malformed message containing the next maxmessagelength
characters that are sent)
but it will not just throw them away because it doesn't understand them.
I have seen the OS IP stack throw away UDP messages when they are extremely
long and get fragmented into multiple packets and some of the packets never
arrive due to congestion problems on the network, but that is an
intermittent problem, you would have some get through and others not (and
it's been a long time since I saw this because networks are now much faster
than they used to be, so it's far less common for a network to be that
congested, but is still possible)
but the fact that you are seeing them via tcpdump would make it unlikely
that this is the problem.
Can you setup a test where the Palo Alto is only sending the traffic
messages to a test machine and setup a very trivial rsyslog config just
writing it to a file? we may end up wanting to get a debug log if it still
fails there.
David Lang
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
