Re: [rsyslog] Formatting CEF to log.

David Lang via rsyslog Mon, 12 Aug 2024 16:15:39 -0700

try logging all traffic with the template RSYSLOG_DebugFormat for a short time(long enough that it should have some of these TRAFFIC messages) and find themessages in there it could be that the messages is not showing up as you wouldexpect


David Lang


On Mon, 12 Aug 2024, Drumm, Daniel wrote:

Date: Mon, 12 Aug 2024 23:11:15 +0000
From: "Drumm, Daniel" <daniel.dr...@dob.texas.gov>
To: David Lang <da...@lang.hm>
Cc: "Drumm, Daniel via rsyslog" <rsyslog@lists.adiscon.com>
Subject: RE: [rsyslog] Formatting CEF to log.

I removed the 'stop' from every single directive, as this is confusing the 
matter.

It looks as such, and yet the TRAFFIC messages do not log anywhere.

--

root@syslog-server-vnic-primary:/etc/rsyslog.d# more 11-paloalto-cef.conf
# Use the template for logs coming from your Palo Alto firewall
if $fromhost-ip == '10.40.0.210' then {
   *.* /var/log/rsyslog/palo-alto-cef.log
}
root@syslog-server-vnic-primary:/etc/rsyslog.d# more 10-paloalto-traffic.conf
# Define a template for Palo Alto TRAFFIC logs
$template PaloAltoTraffic,"/var/log/rsyslog/paloalto/traffic.log"

# Filter and direct TRAFFIC logs to the specified file
if $fromhost-ip == '10.40.0.210' and $msg contains 'TRAFFIC' then {
   action(type="omfile" dynaFile="PaloAltoTraffic")
}
root@syslog-server-vnic-primary:/etc/rsyslog.d# more 10-
10-azuremonitoragent-omfwd.conf  10-paloalto-traffic.conf
root@syslog-server-vnic-primary:/etc/rsyslog.d# more 
10-azuremonitoragent-omfwd.conf
# Azure Monitor Agent configuration: forward logs to azuremonitoragent

template(name="AMA_RSYSLOG_TraditionalForwardFormat" type="string" 
string="<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg%")
# queue.workerThreads sets the maximum worker threads, it will scale back to 0 
if there is no activity
# Forwarding all events through TCP port
*.* action(type="omfwd"
template="AMA_RSYSLOG_TraditionalForwardFormat"
queue.type="LinkedList"
queue.filename="omfwd-azuremonitoragent"
queue.maxFileSize="32m"
action.resumeRetryCount="-1"
action.resumeInterval="5"
action.reportSuspension="on"
action.reportSuspensionContinuation="on"
queue.size="25000"
queue.workerThreads="100"
queue.dequeueBatchSize="2048"
queue.saveonshutdown="on"
target="127.0.0.1" Port="28330" Protocol="tcp")

-----Original Message-----
From: David Lang <da...@lang.hm>
Sent: Monday, August 12, 2024 5:58 PM
To: Drumm, Daniel <daniel.dr...@dob.texas.gov>
Cc: David Lang <da...@lang.hm>; Drumm, Daniel via rsyslog 
<rsyslog@lists.adiscon.com>
Subject: RE: [rsyslog] Formatting CEF to log.

On Mon, 12 Aug 2024, Drumm, Daniel wrote:

Date: Mon, 12 Aug 2024 22:41:21 +0000
From: "Drumm, Daniel" <daniel.dr...@dob.texas.gov>
To: David Lang <da...@lang.hm>
Cc: "Drumm, Daniel via rsyslog" <rsyslog@lists.adiscon.com>
Subject: RE: [rsyslog] Formatting CEF to log.

Here is that file with the -o flag:

root@syslog-server-vnic-primary:/tmp# more rsyslog_combined.conf ##
full conf created by rsyslog version 8.2001.0 at 2024-08-12 22:38:01
##

##### BEGIN CONFIG: /etc/rsyslog.conf
# /etc/rsyslog.conf configuration file for rsyslog # # For more
information install rsyslog-doc and see #
/usr/share/doc/rsyslog-doc/html/configuration/index.html
#
# Default logging rules can be found in /etc/rsyslog.d/50-default.conf

#################
#### MODULES ####
#################
module(load="imudp") # needs to be done just once input(type="imudp"
port="514")

$ModLoad imuxsock # needs to be done just once

# $AllowedSender - specifies which remote systems are allowed to send
syslog messages to rsyslogd # -------------- $AllowedSender UDP,
127.0.0.1, 10.40.0.0/16, 10.30.0.0/23

$SystemLogSocketFlowControl on # enable flow control (use if needed)
$ActionFileEnableSync off

# A template that resambles traditional syslogd file output:
$template TraditionalFormat,"%timegenerated% %HOSTNAME% 
%syslogtag%%msg:::drop-last-lf%\n"
# A more verbose template:
$template 
precise,"%syslogpriority%,%syslogfacility%,%timegenerated::fulltime%,%HOSTNAME%,%syslogtag%,%msg%\n"
# A template that resembles RFC 3164 on-the-wire format:
# (yes, there is NO space betwen syslogtag and msg! that's important!)
$template RFC3164fmt,"<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag%%msg%"
# a template resembling traditional wallmessage format:
$template wallmsg,"\r\n\7Message from syslogd@%HOSTNAME% at %timegenerated% ...\r\n 
%syslogtag%%msg%\n\r"
# The template below emulates winsyslog format, but we need to check
the time # stamps used. It is also a good sampleof the property replacer in 
action.
$template
WinSyslogFmt,"%HOSTNAME%,%timegenerated:1:10:date-rfc3339%,%timegenera
ted:12:19:date-rfc3339%,%timegenerated:1:10:date-rfc3339%,%timegenerat
ed:12:19:date-rfc3339%,%syslogfacility%,%syslo
gpriority%,%syslogtag%%msg%\n"
# A template used for database writing (notice it *is* an actual #
sql-statement):
$template dbFormat,"insert into SystemEvents (Message,
Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt,
InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%,
'%HOSTNAME%',%syslogpr iority%, '%timereported:::date-mysql%',
'%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",sql

# Turn off message reception via local log socket; # local messages
are retrieved through imjournal now.
# $OmitLocalLogging on

###########################
#### GLOBAL DIRECTIVES ####
###########################
$DebugLevel 2
$DebugFile /var/log/rsyslog/rsyslog.debug # # Use traditional
timestamp format.
# To enable high precision timestamps, comment out the following line.
#
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# Filter duplicated messages
$RepeatedMsgReduction on

#
# Set the default permissions for all log files.
#
$FileOwner syslog
$FileGroup syslog
$FileCreateMode 0660
$DirCreateMode 0755
$Umask 0022
$PrivDropToUser syslog
$PrivDropToGroup syslog

#
# Where to place spool and state files # $WorkDirectory
/var/log/rsyslog

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/                 # whole directory (must contain 
the final slash)/etc/rsyslog.d/
##### BEGIN CONFIG: /etc/rsyslog.d/50-default.conf

##### BEGIN CONFIG: /etc/rsyslog.d/21-cloudinit.conf

##### BEGIN CONFIG: /etc/rsyslog.d/20-ufw.conf

##### BEGIN CONFIG: /etc/rsyslog.d/12-azuremonitoragent-omfwd.conf

##### BEGIN CONFIG: /etc/rsyslog.d/11-paloalto-cef.conf

##### BEGIN CONFIG: /etc/rsyslog.d/10-paloalto-traffic.conf

##### BEGIN CONFIG: /etc/rsyslog.d/10-azuremonitoragent-omfwd.conf
# Azure Monitor Agent configuration: forward logs to azuremonitoragent

template(name="AMA_RSYSLOG_TraditionalForwardFormat" type="string"
string="<%PRI%>%TIMESTAMP% %HOSTNAME%
%syslogtag%%msg:::sp-if-no-1st-sp%%msg%")
# queue.workerThreads sets the maximum worker threads, it will scale
back to 0 if there is no activity # Forwarding all events through TCP
port
*.* action(type="omfwd"
template="AMA_RSYSLOG_TraditionalForwardFormat"
queue.type="LinkedList"
queue.filename="omfwd-azuremonitoragent"
queue.maxFileSize="32m"
action.resumeRetryCount="-1"
action.resumeInterval="5"
action.reportSuspension="on"
action.reportSuspensionContinuation="on"
queue.size="25000"
queue.workerThreads="100"
queue.dequeueBatchSize="2048"
queue.saveonshutdown="on"
target="127.0.0.1" Port="28330" Protocol="tcp")

##### END   CONFIG: /etc/rsyslog.d/10-azuremonitoragent-omfwd.conf
# Define a template for Palo Alto TRAFFIC logs $template
PaloAltoTraffic,"/var/log/rsyslog/paloalto/traffic.log"

# Filter and direct TRAFFIC logs to the specified file if $fromhost-ip
== '10.40.0.210' and $msg contains 'TRAFFIC' then {
   action(type="omfile" dynaFile="PaloAltoTraffic")
   stop
}


because you have stop here, your TRAFFIC logs will not be processed past this 
point. That is why they don't appear in the palo-alto-cef.log file below

##### END   CONFIG: /etc/rsyslog.d/10-paloalto-traffic.conf
# Define a template for CEF logs
$template PaloAltoCEF,"/var/log/rsyslog/e%.log"

# Use the template for logs coming from your Palo Alto firewall if
$fromhost-ip == '10.40.0.210' then {
   *.* /var/log/rsyslog/palo-alto-cef.log
   stop
}

##### END   CONFIG: /etc/rsyslog.d/11-paloalto-cef.conf
# Azure Monitor Agent configuration: forward logs to azuremonitoragent

# Azure Monitor Agent configuration: forward logs to azuremonitoragent
template(name="AMA_RSYSLOG_TraditionalForwardFormat" type="string"
string="<%PRI%>%TIMESTAMP% %HOSTNAME%
%syslogtag%%msg:::sp-if-no-1st-sp%%msg%")

*.* action(type="omfwd"
   template="AMA_RSYSLOG_TraditionalForwardFormat"
   queue.type="LinkedList"
   queue.filename="omfwd-azuremonitoragent"
   queue.maxFileSize="32m"
   action.resumeRetryCount="-1"
   action.resumeInterval="30"
   action.reportSuspension="on"
   action.reportSuspensionContinuation="on"
   queue.size="100000"
   queue.discardmark="97500"
   queue.discardseverity="0"
   queue.checkpointInterval="100"
   queue.workerThreads="4"
   queue.timeoutEnqueue="10000"
   queue.timeoutWorkerthreadShutdown="60000"
   queue.saveOnShutdown="on"
   target="127.0.0.1" Port="28330" Protocol="tcp"
)

##### END   CONFIG: /etc/rsyslog.d/12-azuremonitoragent-omfwd.conf
# Log kernel generated UFW log messages to file :msg,contains,"[UFW "
/var/log/ufw.log

# Uncomment the following to stop logging anything that matches the last rule.
# Doing this will stop logging kernel generated UFW log messages to
the file # normally containing kern.* messages (eg, /var/log/kern.log)
#& stop

##### END   CONFIG: /etc/rsyslog.d/20-ufw.conf
# Log cloudinit generated log messages to file :syslogtag, isequal,
"[CLOUDINIT]" /var/log/cloud-init.log

# comment out the following line to allow CLOUDINIT messages through.
# Doing so means you'll also get CLOUDINIT messages in /var/log/syslog
& stop

##### END   CONFIG: /etc/rsyslog.d/21-cloudinit.conf
#  Default rules for rsyslog.
#
#                       For more information see rsyslog.conf(5) and 
/etc/rsyslog.conf

#
# First some standard log files.  Log by facility.
#
auth,authpriv.*                 /var/log/rsyslog/auth.log
cron.*                          /var/log/rsyslog/cron.log
daemon.*                        /var/log/rsyslog/daemon.log
kern.*                          /var/log/rsyslog/kern.log
lpr.*                           /var/log/rsyslog/lpr.log
mail.*                          /var/log/rsyslog/mail.log
user.*                          /var/log/rsyslog/user.log
local0.info                     /var/log/rsyslog/local0_info.log
#*.*                            
/var/log/rsyslog/traditionalfile.log;TraditionalFormat      # log to a file in 
the traditional format
*.*                             /var/log/rsyslog/all-the-stuff.log

#
# Logging for the mail system.  Split it up so that # it is easy to
write scripts to parse these files.
#
# mail.info                     -/var/log/rsyslog/mail.info
# mail.warn                     -/var/log/rsyslog/mail.warn
# mail.err                      /var/log/rsyslog/mail.err

#
# Some "catch-all" log files.
#
*.=debug;auth,authpriv.none;news.none;mail.none /var/log/debug
PreprocFileLineNumber(33)
*.=info;*.=notice;*.=warn;auth,authpriv.none;cron,daemon.none;mail,news.none    
        /var/log/messages
PreprocFileLineNumber(37)

#
# Emergencies are sent to everybody logged in.
#
*.emerg                         :omusrmsg:*

#
# I like to have messages displayed on the console, but only on a
virtual # console I usually leave idle.
#
#daemon,mail.*;#        news.=crit;news.=err;news.=notice;#     
*.=debug;*.=info;#      *.=notice;*.=warn       /dev/tty8

##### END   CONFIG: /etc/rsyslog.d/50-default.conf
                # whole directory (must contain the final slash)

##### END   CONFIG: /etc/rsyslog.conf

-----Original Message-----
From: David Lang <da...@lang.hm>
Sent: Monday, August 12, 2024 5:32 PM
To: Drumm, Daniel <daniel.dr...@dob.texas.gov>
Cc: David Lang <da...@lang.hm>; Drumm, Daniel via rsyslog
<rsyslog@lists.adiscon.com>
Subject: RE: [rsyslog] Formatting CEF to log.

start rsyslog with the command line option -o /path/to/file

that will write the combined config files to that file as rsyslog sees
it. Post that and we can look for other interactions that may cause
some logs to be thrown away first

look especially for othe stop commands earlier than the clause looking
at the fromhost-ip

David Lang

 On Mon, 12 Aug 2024, Drumm, Daniel wrote:

Date: Mon, 12 Aug 2024 22:18:02 +0000
From: "Drumm, Daniel" <daniel.dr...@dob.texas.gov>
To: David Lang <da...@lang.hm>,
    "Drumm, Daniel via rsyslog" <rsyslog@lists.adiscon.com>
Subject: RE: [rsyslog] Formatting CEF to log.

Thank you, I should have corrected that as I iterated.

What I see with this directive:

root@syslog-server-vnic-primary:/etc/rsyslog.d# more
10-paloalto-cef.conf # Define a template for CEF logs $template
PaloAltoCEF,"/var/log/rsyslog/DOB-FW-HA-1.OCI/%programname%.log"

# Use the template for logs coming from your Palo Alto firewall if
$fromhost-ip == '10.40.0.210' then {
   *.* /var/log/rsyslog/palo-alto-cef.log
   stop
}

Is that the log has SYSTEM messages, but none of the Traffic messages:

root@syslog-server-vnic-primary:/var/log/rsyslog# tail -f
palo-alto-cef.log Aug 12 17:07:12 DOB-FW-HA-1.OCI Ashburn FD1
CEF:0|Palo
Alto#012Networks|PAN-OS|11.2.0|url-filtering|SYSTEM|1|rt=Aug 12 2024
22:07:12 GMT#012deviceExternalId=007959000479054 cs3Label=Virtual
System cs3= fname= flexString2Label=Module#012flexString2=general
msg="PAN-DB was upgraded to version 20240812.20332."
externalId=7392297676068399997 cat=upgrade-url-database-success
PanOSDGl1=0#012PanOSDGl2=0 PanOSDGl3=0 PanOSDGl4=0#012PanOSVsysName=
dvchost=DOB-FW-HA-1
PanOSActionFlags=0x0#012anOSTimeGeneratedHighResolution=2024-08-12T17:
07:12.563-05:00 Aug 12 17:08:17 DOB-FW-HA-1.OCI Ashburn FD1
CEF:0|Palo
Alto#012Networks|PAN-OS|11.2.0|url-filtering|SYSTEM|1|rt=Aug 12 2024
22:08:17 GMT#012deviceExternalId=007959000479054 cs3Label=Virtual
System cs3= fname= flexString2Label=Module#012flexString2=general
msg="PAN-DB was upgraded to version 20240812.20333."
externalId=7392297676068399998 cat=upgrade-url-database-success
PanOSDGl1=0#012PanOSDGl2=0 PanOSDGl3=0 PanOSDGl4=0#012PanOSVsysName=
dvchost=DOB-FW-HA-1
PanOSActionFlags=0x0#012anOSTimeGeneratedHighResolution=2024-08-12T17:
08:17.737-05:00 Aug 12 17:08:50 DOB-FW-HA-1.OCI Ashburn FD1
CEF:0|Palo Alto#012Networks|PAN-OS|11.2.0|userid|SYSTEM|1|rt=Aug 12
2024 22:08:49
GMT#012deviceExternalId=007959000479054 cs3Label=Virtual System cs3=
fname= flexString2Label=Module#012flexString2=general msg="gRPC
connection to identity.services-edge.paloaltonetworks.com:443 is
established, 10.40.0.210:46846 -> 34.136.155.117:443 time: 2024-08-12
17:08:50" externalId=7392297676068399999 cat=cuid-conn
PanOSDGl1=0#012PanOSDGl2=0 PanOSDGl3=0 PanOSDGl4=0#012PanOSVsysName=
dvchost=DOB-FW-HA-1
PanOSActionFlags=0x0#012anOSTimeGeneratedHighResolution=2024-08-12T17:
08:50.292-05:00 Aug 12 17:08:54 DOB-FW-HA-1.OCI Ashburn FD1
CEF:0|Palo Alto#012Networks|PAN-OS|11.2.0|userid|SYSTEM|4|rt=Aug 12
2024 22:08:53
GMT#012deviceExternalId=007959000479054 cs3Label=Virtual System cs3=
fname= flexString2Label=Module#012flexString2=general msg="gRPC
connection to identity.services-edge.paloaltonetworks.com:443 is
broken, error: rpc error: code = Unknown desc = [UploadCUID]
[007959000479054] tenantId is empty time: 2024-08-12 17:08:54"
externalId=7392297676068400000 cat=cuid-conn
PanOSDGl1=0#012PanOSDGl2=0 PanOSDGl3=0 PanOSDGl4=0#012PanOSVsysName=
dvchost=DOB-FW-HA-1
PanOSActionFlags=0x0#012anOSTimeGeneratedHighResolution=2024-08-12T17:
08:54.050-05:00

Yet I know for a fact that the TRAFFIC messages are hitting the NIC of the 
syslog server:

root@syslog-server-vnic-primary:/etc/rsyslog.d# tcpdump -v "port 514"
tcpdump: listening on ens3, link-type EN10MB (Ethernet), capture size
262144 bytes
22:16:32.237523 IP (tos 0x0, ttl 63, id 30088, offset 0, flags [+], proto UDP 
(17), length 1500)
   10.40.0.210.40933 > 10.30.0.18.syslog: SYSLOG, length: 1472
       Facility local0 (16), Severity info (6)
       Msg: Aug 12 17:16:36 DOB-FW-HA-1.OCI Ashburn FD1 CEF:0|Palo
Alto Networks|PAN-OS|11.2.0|drop|TRAFFIC|1|rt=Aug 12 2024 22:16:31
GMT\0x0adeviceExternalId=007959000479054 src=162.216.149.73
dst=10.40.1.97
sourceTranslatedAddress=0.0.0.0\0x0adestinationTranslatedAddress=0.0.
0
.0 cs1Label=Rule cs1=Inbound Drop Logging Rule suser= duser=
app=not-applicable\0x0acs3Label=Virtual System cs3=vsys1
cs4Label=Source Zone cs4=Untrust cs5Label=Destination Zone
cs5=Trust\0x0adeviceInboundInterface=ethernet1/2
deviceOutboundInterface= cs6Label=LogProfile cs6=OCI Syslog Server
Profile\0x0acn1Label=SessionID cn1=0 cnt=1 spt=54491 dpt=9408
sourceTranslatedPort=0\0x0adestinationTranslatedPort=0
flexString1Label=Flags flexString1=0x0 proto=tcp
act=drop\0x0aflexNumber1Label=Total bytes flexNumber1=58 in=58 out=0
cn2Label=Packets\0x0acn2=1 PanOSPacketsReceived=0
PanOSPacketsSent=1\0x0astart=Aug 12 2024 22:16:31 GMT
cn3Label=Elapsed time in seconds cn3=0 cs2Label=URL
Category\0x0acs2=any externalId=7

392297676068829896 reason=policy-deny PanOSDGl1=0\0x0aPanOSDGl2=0
PanOSDGl3=0 PanOSDGl4=0\0x0aPanOSVsysName= dvchost=DOB-FW-HA-1
cat=from-policy PanOSActionFlags=0x0\0x0aPanOSSrcUUID= PanOSDstUUID=
PanOSTunnelID=0 PanOSMonitorTag=\0x0aPanOSParentSessionID=0
PanOSParentStartTime= PanOSTunnelType=N/A\0x0aPanOSSCTPAssocID=0
PanOSSCTPChunks=0 PanOSSCTPChunkSent=0\0x0aPanOSSCTPChunksRcv=0
PanOSRuleUUID=d6ca6ff0-71e9-4f09-8e8d-2204deb98205
PanOSHTTP2Con=0\0x0aPanLinkChange=0 PanPolicyID=
PanLinkDetail=\0x0aPanSDWANCluster=
PanSDWANDevice=\0x0aPanSDWANClustype

22:16:32.237613 IP (tos 0x0, ttl 63, id 30089, offset 0, flags [+],
proto UDP (17), length 1500)

So why are they not logging?


-----Original Message-----
From: David Lang <da...@lang.hm>
Sent: Monday, August 12, 2024 3:51 PM
To: Drumm, Daniel via rsyslog <rsyslog@lists.adiscon.com>
Cc: Drumm, Daniel <daniel.dr...@dob.texas.gov>
Subject: Re: [rsyslog] Formatting CEF to log.

On Mon, 12 Aug 2024, Drumm, Daniel via rsyslog wrote:

When I perform a "tcpdump -v" I see that the Palo Alto is sending the rsyslog 
server the correct logs for traffic flows. But the only logs coming from the firewall 
that are logging are the system messages. I had to convert the traffic flow messages 
coming in from the Palo Alto on 514/udp to the CEF (Command Event Format) and I do not 
see them anywhere in logs:

Here is how I know they are arriving to the Ubuntu server:



root@syslog-server-vnic-primary:/var/log/rsyslog/DOB-FW-HA-1.OCI#<mailto:root@syslog-server-vnic-primary:/var/log/rsyslog/DOB-FW-HA-1.OCI#>
 tcpdump -v "port 514"

tcpdump: listening on ens3, link-type EN10MB (Ethernet), capture
size
262144 bytes

20:30:06.780950 IP (tos 0x0, ttl 63, id 25948, offset 0, flags [+],
proto UDP (17), length 1500)

   10.40.0.210.40933 > 10.30.0.18.syslog: SYSLOG, length: 1472

       Facility local0 (16), Severity info (6)

       Msg: Aug 12 15:30:11 DOB-FW-HA-1.OCI Ashburn FD1 CEF:0|Palo
Alto Networks|PAN-OS|11.2.0|drop|TRAFFIC|1|rt=Aug 12 2024 20:30:08
GMT\0x0adeviceExternalId=007959000479054 src=193.163.125.224
dst=10.40.1.98
sourceTranslatedAddress=0.0.0.0\0x0adestinationTranslatedAddress=0.0.
0
.0 cs1Label=Rule cs1=Inbound Drop Logging Rule suser= duser=
app=not-applicable\0x0acs3Label=Virtual System cs3=vsys1
cs4Label=Source Zone cs4=Untrust cs5Label=Destination Zone
cs5=Trust\0x0adeviceInboundInterface=ethernet1/2
deviceOutboundInterface= cs6Label=LogProfile cs6=OCI Syslog Server
Profile\0x0acn1Label=SessionID cn1=0 cnt=1 spt=42495 dpt=2650
sourceTranslatedPort=0\0x0adestinationTranslatedPort=0
flexString1Label=Flags flexString1=0x0 proto=tcp
act=drop\0x0aflexNumber1Label=Total bytes flexNumber1=58 in=58 out=0
cn2Label=Packets\0x0acn2=1 PanOSPacketsReceived=0
PanOSPacketsSent=1\0x0astart=Aug 12 2024 20:30:08 GMT
cn3Label=Elapsed time in seconds cn3=0 cs2Label=URL
Category\0x0acs2=any externalId=

392297676068828287 reason=policy-deny PanOSDGl1=0\0x0aPanOSDGl2=0
PanOSDGl3=0 PanOSDGl4=0\0x0aPanOSVsysName= dvchost=DOB-FW-HA-1
cat=from-policy PanOSActionFlags=0x0\0x0aPanOSSrcUUID= PanOSDstUUID=
PanOSTunnelID=0 PanOSMonitorTag=\0x0aPanOSParentSessionID=0
PanOSParentStartTime= PanOSTunnelType=N/A\0x0aPanOSSCTPAssocID=0
PanOSSCTPChunks=0 PanOSSCTPChunkSent=0\0x0aPanOSSCTPChunksRcv=0
PanOSRuleUUID=d6ca6ff0-71e9-4f09-8e8d-2204deb98205
PanOSHTTP2Con=0\0x0aPanLinkChange=0 PanPolicyID=
PanLinkDetail=\0x0aPanSDWANCluster=
PanSDWANDevice=\0x0aPanSDWANClustyp

20:30:06.780979 IP (tos 0x0, ttl 63, id 25949, offset 0, flags [+],
proto UDP (17), length 1500)

   10.40.0.210.40933 > 10.30.0.18.syslog: SYSLOG, length: 1472

Here is my rsyslog.conf file:

root@syslog-server-vnic-primary:/etc# more rsyslog.conf

# /etc/rsyslog.conf configuration file for rsyslog

#

# For more information install rsyslog-doc and see

# /usr/share/doc/rsyslog-doc/html/configuration/index.html

#

# Default logging rules can be found in
/etc/rsyslog.d/50-default.conf





#################

#### MODULES ####

#################

module(load="imudp") # needs to be done just once

input(type="imudp" port="514")



$ModLoad imuxsock # needs to be done just once



# $AllowedSender - specifies which remote systems are allowed to
send syslog messages to rsyslogd

# --------------

$AllowedSender UDP, 127.0.0.1, 10.40.0.0/16, 10.30.0.0/23


this is why the legacy format is depriciated, you should not mix new
style
input() module() with old style $foo directives



$SystemLogSocketFlowControl on # enable flow control (use if needed)

$ActionFileEnableSync off



# A template that resambles traditional syslogd file output:

$template TraditionalFormat,"%timegenerated% %HOSTNAME% 
%syslogtag%%msg:::drop-last-lf%\n"

# A more verbose template:

$template 
precise,"%syslogpriority%,%syslogfacility%,%timegenerated::fulltime%,%HOSTNAME%,%syslogtag%,%msg%\n"

# A template that resembles RFC 3164 on-the-wire format:

# (yes, there is NO space betwen syslogtag and msg! that's
important!)

$template RFC3164fmt,"<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag%%msg%"

# a template resembling traditional wallmessage format:

$template wallmsg,"\r\n\7Message from syslogd@%HOSTNAME% at %timegenerated% ...\r\n 
%syslogtag%%msg%\n\r"

# The template below emulates winsyslog format, but we need to check
the time

# stamps used. It is also a good sampleof the property replacer in action.

$template
WinSyslogFmt,"%HOSTNAME%,%timegenerated:1:10:date-rfc3339%,%timegene
r
a
ted:12:19:date-rfc3339%,%timegenerated:1:10:date-rfc3339%,%timegener
a t ed:12:19:date-rfc3339%,%syslogfacility%,%syslo

gpriority%,%syslogtag%%msg%\n"

# A template used for database writing (notice it *is* an actual

# sql-statement):

$template dbFormat,"insert into SystemEvents (Message,
Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt,
InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%,
'%HOSTNAME%',%syslogpr

iority%, '%timereported:::date-mysql%',
'%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",sql





# Turn off message reception via local log socket;

# local messages are retrieved through imjournal now.

# $OmitLocalLogging on



###########################

#### GLOBAL DIRECTIVES ####

###########################

$DebugLevel 2

$DebugFile /var/log/rsyslog/rsyslog.debug

#

# Use traditional timestamp format.

# To enable high precision timestamps, comment out the following line.

#

$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat



# Filter duplicated messages

$RepeatedMsgReduction on



#

# Set the default permissions for all log files.

#

$FileOwner syslog

$FileGroup syslog

$FileCreateMode 0660

$DirCreateMode 0755

$Umask 0022

$PrivDropToUser syslog

$PrivDropToGroup syslog



#

# Where to place spool and state files

#

$WorkDirectory /var/log/rsyslog



# Include all config files in /etc/rsyslog.d/

$IncludeConfig /etc/rsyslog.d/                 # whole directory (must contain 
the final slash)



Here is my rsyslog.conf file for Palo Alto directive:

root@syslog-server-vnic-primary:/etc/rsyslog.d#<mailto:root@syslog-s
e r ver-vnic-primary:/etc/rsyslog.d#> more 10-paloalto-cef.conf

# Define a template for CEF logs

$template PaloAltoCEF,"/var/log/rsyslog/DOB-FW-HA-1.OCI/%programname%.log"



# Use the template for logs coming from your Palo Alto firewall

if $fromhost-ip == '10.40.0.210' then {

   *.* ?PaloAltoCEF

   stop

}

Here is my rsyslog.conf file for Catch All directive:

root@syslog-server-vnic-primary:/etc/rsyslog.d#<mailto:root@syslog-s
e r ver-vnic-primary:/etc/rsyslog.d#> more 50-default.conf

#  Default rules for rsyslog.

#

#                       For more information see rsyslog.conf(5) and 
/etc/rsyslog.conf



#

# First some standard log files.  Log by facility.

#

auth,authpriv.*                 /var/log/rsyslog/auth.log

cron.*                          /var/log/rsyslog/cron.log

daemon.*                        /var/log/rsyslog/daemon.log

kern.*                          /var/log/rsyslog/kern.log

lpr.*                           /var/log/rsyslog/lpr.log

mail.*                          /var/log/rsyslog/mail.log

user.*                          /var/log/rsyslog/user.log

local0.info                     /var/log/rsyslog/local0_info.log

#*.*                            
/var/log/rsyslog/traditionalfile.log;TraditionalFormat      # log to a file in 
the traditional format

*.*                             /var/log/rsyslog/all-the-stuff.log



#

# Logging for the mail system.  Split it up so that

# it is easy to write scripts to parse these files.

#

# mail.info                     -/var/log/rsyslog/mail.info

# mail.warn                     -/var/log/rsyslog/mail.warn

# mail.err                      /var/log/rsyslog/mail.err



#

# Some "catch-all" log files.

#

*.=debug;\

       auth,authpriv.none;\

       news.none;mail.none     /var/log/debug

*.=info;*.=notice;*.=warn;\

       auth,authpriv.none;\

       cron,daemon.none;\

       mail,news.none          /var/log/messages



#

# Emergencies are sent to everybody logged in.

#

*.emerg                         :omusrmsg:*



#

# I like to have messages displayed on the console, but only on a
virtual

# console I usually leave idle.

#

#daemon,mail.*;\

#       news.=crit;news.=err;news.=notice;\

#       *.=debug;*.=info;\

#       *.=notice;*.=warn       /dev/tty8



I would think they would go to the all-the-stuff log file, but no. They are not 
being logged anywhere and I do not know why not since the PAN-OS system logs 
are being logged.


well, you have anything arriving from that IP address being written to 
/var/log/rsyslog/DOB-FW-HA-1.OCI/%programname%.log and then you throw away the 
log, so it would never get down to the action to write it to the all-the-stuff 
file.

David Lang



Dan.
_______________________________________________
rsyslog mailing list
https://list/
s.adiscon.net%2Fmailman%2Flistinfo%2Frsyslog&data=05%7C02%7CDaniel.D
r
u
mm%40dob.texas.gov%7Ca5b7d1daeca54918bec408dcbb106d71%7C32a53eeb2759
4
e
f4af2858fd990c7d67%7C1%7C0%7C638590926965494771%7CUnknown%7CTWFpbGZs
b
3
d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
%
7
C60000%7C%7C%7C&sdata=jbqBlxisDwsVCMwjaT90N%2FKimmaD8t4q2sz482zP%2FF
I
%
3D&reserved=0
https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww%2F&data=05%7C02%7CDaniel.Drumm%40dob.texas.gov%7C8f8ea3ccb8764f2df23308dcbb2235d0%7C32a53eeb27594ef4af2858fd990c7d67%7C1%7C0%7C638591002828934531%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=wFHNM6fsEI4vfE8YWBLwe0W4uaH0Up7p%2Fh1vy8Py8Ak%3D&reserved=0.
r%2F&data=05%7C02%7CDaniel.Drumm%40dob.texas.gov%7C41a72e59343143488
7
5e08dcbb1e8a28%7C32a53eeb27594ef4af2858fd990c7d67%7C1%7C0%7C63859098
7
056813333%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMz
I
iLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C40000%7C%7C%7C&sdata=0VDHHDdJX2r3y
i
XUWqNO%2FD%2BEEbpPFWRRAaPIoTcHQ%2Bw%3D&reserved=0
syslog.com%2Fprofessional-services%2F&data=05%7C02%7CDaniel.Drumm%40
d
o
b.texas.gov%7Ca5b7d1daeca54918bec408dcbb106d71%7C32a53eeb27594ef4af2
8
5
8fd990c7d67%7C1%7C0%7C638590926965505555%7CUnknown%7CTWFpbGZsb3d8eyJ
W
I
joiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C6000
0
%
7C%7C%7C&sdata=Y1xISoLoG4QqSzyfShnzV1At6WqTfmphvcwX27vYX%2Bo%3D&rese
r
v
ed=0 What's up with rsyslog? Follow
https://twit/
ter.com%2Frgerhards&data=05%7C02%7CDaniel.Drumm%40dob.texas.gov%7Ca5
b
7
d1daeca54918bec408dcbb106d71%7C32a53eeb27594ef4af2858fd990c7d67%7C1%
7
C
0%7C638590926965512721%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiL
C
J
QIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C60000%7C%7C%7C&sdata=v
V
v
5i4UP326ZNb2mtr1KEfnc4i9ZP58nSzWV5zmzPiw%3D&reserved=0
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] Formatting CEF to log.

Reply via email to