Thank you, I should have corrected that as I iterated.
What I see with this directive:
root@syslog-server-vnic-primary:/etc/rsyslog.d# more 10-paloalto-cef.conf
# Define a template for CEF logs
$template PaloAltoCEF,"/var/log/rsyslog/DOB-FW-HA-1.OCI/%programname%.log"
# Use the template for logs coming from your Palo Alto firewall
if $fromhost-ip == '10.40.0.210' then {
*.* /var/log/rsyslog/palo-alto-cef.log
stop
}
Is that the log has SYSTEM messages, but none of the Traffic messages:
root@syslog-server-vnic-primary:/var/log/rsyslog# tail -f palo-alto-cef.log
Aug 12 17:07:12 DOB-FW-HA-1.OCI Ashburn FD1 CEF:0|Palo
Alto#012Networks|PAN-OS|11.2.0|url-filtering|SYSTEM|1|rt=Aug 12 2024 22:07:12
GMT#012deviceExternalId=007959000479054 cs3Label=Virtual System cs3= fname=
flexString2Label=Module#012flexString2=general msg="PAN-DB was upgraded to
version 20240812.20332." externalId=7392297676068399997
cat=upgrade-url-database-success PanOSDGl1=0#012PanOSDGl2=0 PanOSDGl3=0
PanOSDGl4=0#012PanOSVsysName= dvchost=DOB-FW-HA-1
PanOSActionFlags=0x0#012anOSTimeGeneratedHighResolution=2024-08-12T17:07:12.563-05:00
Aug 12 17:08:17 DOB-FW-HA-1.OCI Ashburn FD1 CEF:0|Palo
Alto#012Networks|PAN-OS|11.2.0|url-filtering|SYSTEM|1|rt=Aug 12 2024 22:08:17
GMT#012deviceExternalId=007959000479054 cs3Label=Virtual System cs3= fname=
flexString2Label=Module#012flexString2=general msg="PAN-DB was upgraded to
version 20240812.20333." externalId=7392297676068399998
cat=upgrade-url-database-success PanOSDGl1=0#012PanOSDGl2=0 PanOSDGl3=0
PanOSDGl4=0#012PanOSVsysName= dvchost=DOB-FW-HA-1
PanOSActionFlags=0x0#012anOSTimeGeneratedHighResolution=2024-08-12T17:08:17.737-05:00
Aug 12 17:08:50 DOB-FW-HA-1.OCI Ashburn FD1 CEF:0|Palo
Alto#012Networks|PAN-OS|11.2.0|userid|SYSTEM|1|rt=Aug 12 2024 22:08:49
GMT#012deviceExternalId=007959000479054 cs3Label=Virtual System cs3= fname=
flexString2Label=Module#012flexString2=general msg="gRPC connection to
identity.services-edge.paloaltonetworks.com:443 is established,
10.40.0.210:46846 -> 34.136.155.117:443 time: 2024-08-12 17:08:50"
externalId=7392297676068399999 cat=cuid-conn PanOSDGl1=0#012PanOSDGl2=0
PanOSDGl3=0 PanOSDGl4=0#012PanOSVsysName= dvchost=DOB-FW-HA-1
PanOSActionFlags=0x0#012anOSTimeGeneratedHighResolution=2024-08-12T17:08:50.292-05:00
Aug 12 17:08:54 DOB-FW-HA-1.OCI Ashburn FD1 CEF:0|Palo
Alto#012Networks|PAN-OS|11.2.0|userid|SYSTEM|4|rt=Aug 12 2024 22:08:53
GMT#012deviceExternalId=007959000479054 cs3Label=Virtual System cs3= fname=
flexString2Label=Module#012flexString2=general msg="gRPC connection to
identity.services-edge.paloaltonetworks.com:443 is broken, error: rpc error:
code = Unknown desc = [UploadCUID] [007959000479054] tenantId is empty time:
2024-08-12 17:08:54" externalId=7392297676068400000 cat=cuid-conn
PanOSDGl1=0#012PanOSDGl2=0 PanOSDGl3=0 PanOSDGl4=0#012PanOSVsysName=
dvchost=DOB-FW-HA-1
PanOSActionFlags=0x0#012anOSTimeGeneratedHighResolution=2024-08-12T17:08:54.050-05:00
Yet I know for a fact that the TRAFFIC messages are hitting the NIC of the
syslog server:
root@syslog-server-vnic-primary:/etc/rsyslog.d# tcpdump -v "port 514"
tcpdump: listening on ens3, link-type EN10MB (Ethernet), capture size 262144
bytes
22:16:32.237523 IP (tos 0x0, ttl 63, id 30088, offset 0, flags [+], proto UDP
(17), length 1500)
10.40.0.210.40933 > 10.30.0.18.syslog: SYSLOG, length: 1472
Facility local0 (16), Severity info (6)
Msg: Aug 12 17:16:36 DOB-FW-HA-1.OCI Ashburn FD1 CEF:0|Palo Alto
Networks|PAN-OS|11.2.0|drop|TRAFFIC|1|rt=Aug 12 2024 22:16:31
GMT\0x0adeviceExternalId=007959000479054 src=162.216.149.73 dst=10.40.1.97
sourceTranslatedAddress=0.0.0.0\0x0adestinationTranslatedAddress=0.0.0.0
cs1Label=Rule cs1=Inbound Drop Logging Rule suser= duser=
app=not-applicable\0x0acs3Label=Virtual System cs3=vsys1 cs4Label=Source Zone
cs4=Untrust cs5Label=Destination Zone
cs5=Trust\0x0adeviceInboundInterface=ethernet1/2 deviceOutboundInterface=
cs6Label=LogProfile cs6=OCI Syslog Server Profile\0x0acn1Label=SessionID cn1=0
cnt=1 spt=54491 dpt=9408 sourceTranslatedPort=0\0x0adestinationTranslatedPort=0
flexString1Label=Flags flexString1=0x0 proto=tcp
act=drop\0x0aflexNumber1Label=Total bytes flexNumber1=58 in=58 out=0
cn2Label=Packets\0x0acn2=1 PanOSPacketsReceived=0
PanOSPacketsSent=1\0x0astart=Aug 12 2024 22:16:31 GMT cn3Label=Elapsed time in
seconds cn3=0 cs2Label=URL Category\0x0acs2=any externalId=73
92297676068829896 reason=policy-deny PanOSDGl1=0\0x0aPanOSDGl2=0 PanOSDGl3=0
PanOSDGl4=0\0x0aPanOSVsysName= dvchost=DOB-FW-HA-1 cat=from-policy
PanOSActionFlags=0x0\0x0aPanOSSrcUUID= PanOSDstUUID= PanOSTunnelID=0
PanOSMonitorTag=\0x0aPanOSParentSessionID=0 PanOSParentStartTime=
PanOSTunnelType=N/A\0x0aPanOSSCTPAssocID=0 PanOSSCTPChunks=0
PanOSSCTPChunkSent=0\0x0aPanOSSCTPChunksRcv=0
PanOSRuleUUID=d6ca6ff0-71e9-4f09-8e8d-2204deb98205
PanOSHTTP2Con=0\0x0aPanLinkChange=0 PanPolicyID=
PanLinkDetail=\0x0aPanSDWANCluster= PanSDWANDevice=\0x0aPanSDWANClustype
22:16:32.237613 IP (tos 0x0, ttl 63, id 30089, offset 0, flags [+], proto UDP
(17), length 1500)
So why are they not logging?
-----Original Message-----
From: David Lang <da...@lang.hm>
Sent: Monday, August 12, 2024 3:51 PM
To: Drumm, Daniel via rsyslog <rsyslog@lists.adiscon.com>
Cc: Drumm, Daniel <daniel.dr...@dob.texas.gov>
Subject: Re: [rsyslog] Formatting CEF to log.
On Mon, 12 Aug 2024, Drumm, Daniel via rsyslog wrote:
> When I perform a "tcpdump -v" I see that the Palo Alto is sending the rsyslog
> server the correct logs for traffic flows. But the only logs coming from the
> firewall that are logging are the system messages. I had to convert the
> traffic flow messages coming in from the Palo Alto on 514/udp to the CEF
> (Command Event Format) and I do not see them anywhere in logs:
>
> Here is how I know they are arriving to the Ubuntu server:
>
>
>
> root@syslog-server-vnic-primary:/var/log/rsyslog/DOB-FW-HA-1.OCI#<mailto:root@syslog-server-vnic-primary:/var/log/rsyslog/DOB-FW-HA-1.OCI#>
> tcpdump -v "port 514"
>
> tcpdump: listening on ens3, link-type EN10MB (Ethernet), capture size
> 262144 bytes
>
> 20:30:06.780950 IP (tos 0x0, ttl 63, id 25948, offset 0, flags [+],
> proto UDP (17), length 1500)
>
> 10.40.0.210.40933 > 10.30.0.18.syslog: SYSLOG, length: 1472
>
> Facility local0 (16), Severity info (6)
>
> Msg: Aug 12 15:30:11 DOB-FW-HA-1.OCI Ashburn FD1 CEF:0|Palo
> Alto Networks|PAN-OS|11.2.0|drop|TRAFFIC|1|rt=Aug 12 2024 20:30:08
> GMT\0x0adeviceExternalId=007959000479054 src=193.163.125.224
> dst=10.40.1.98
> sourceTranslatedAddress=0.0.0.0\0x0adestinationTranslatedAddress=0.0.0
> .0 cs1Label=Rule cs1=Inbound Drop Logging Rule suser= duser=
> app=not-applicable\0x0acs3Label=Virtual System cs3=vsys1
> cs4Label=Source Zone cs4=Untrust cs5Label=Destination Zone
> cs5=Trust\0x0adeviceInboundInterface=ethernet1/2
> deviceOutboundInterface= cs6Label=LogProfile cs6=OCI Syslog Server
> Profile\0x0acn1Label=SessionID cn1=0 cnt=1 spt=42495 dpt=2650
> sourceTranslatedPort=0\0x0adestinationTranslatedPort=0
> flexString1Label=Flags flexString1=0x0 proto=tcp
> act=drop\0x0aflexNumber1Label=Total bytes flexNumber1=58 in=58 out=0
> cn2Label=Packets\0x0acn2=1 PanOSPacketsReceived=0
> PanOSPacketsSent=1\0x0astart=Aug 12 2024 20:30:08 GMT cn3Label=Elapsed
> time in seconds cn3=0 cs2Label=URL Category\0x0acs2=any externalId=
7
> 392297676068828287 reason=policy-deny PanOSDGl1=0\0x0aPanOSDGl2=0
> PanOSDGl3=0 PanOSDGl4=0\0x0aPanOSVsysName= dvchost=DOB-FW-HA-1
> cat=from-policy PanOSActionFlags=0x0\0x0aPanOSSrcUUID= PanOSDstUUID=
> PanOSTunnelID=0 PanOSMonitorTag=\0x0aPanOSParentSessionID=0
> PanOSParentStartTime= PanOSTunnelType=N/A\0x0aPanOSSCTPAssocID=0
> PanOSSCTPChunks=0 PanOSSCTPChunkSent=0\0x0aPanOSSCTPChunksRcv=0
> PanOSRuleUUID=d6ca6ff0-71e9-4f09-8e8d-2204deb98205
> PanOSHTTP2Con=0\0x0aPanLinkChange=0 PanPolicyID=
> PanLinkDetail=\0x0aPanSDWANCluster=
> PanSDWANDevice=\0x0aPanSDWANClustyp
>
> 20:30:06.780979 IP (tos 0x0, ttl 63, id 25949, offset 0, flags [+],
> proto UDP (17), length 1500)
>
> 10.40.0.210.40933 > 10.30.0.18.syslog: SYSLOG, length: 1472
>
> Here is my rsyslog.conf file:
>
> root@syslog-server-vnic-primary:/etc# more rsyslog.conf
>
> # /etc/rsyslog.conf configuration file for rsyslog
>
> #
>
> # For more information install rsyslog-doc and see
>
> # /usr/share/doc/rsyslog-doc/html/configuration/index.html
>
> #
>
> # Default logging rules can be found in /etc/rsyslog.d/50-default.conf
>
>
>
>
>
> #################
>
> #### MODULES ####
>
> #################
>
> module(load="imudp") # needs to be done just once
>
> input(type="imudp" port="514")
>
>
>
> $ModLoad imuxsock # needs to be done just once
>
>
>
> # $AllowedSender - specifies which remote systems are allowed to send
> syslog messages to rsyslogd
>
> # --------------
>
> $AllowedSender UDP, 127.0.0.1, 10.40.0.0/16, 10.30.0.0/23
this is why the legacy format is depriciated, you should not mix new style
input() module() with old style $foo directives
>
>
> $SystemLogSocketFlowControl on # enable flow control (use if needed)
>
> $ActionFileEnableSync off
>
>
>
> # A template that resambles traditional syslogd file output:
>
> $template TraditionalFormat,"%timegenerated% %HOSTNAME%
> %syslogtag%%msg:::drop-last-lf%\n"
>
> # A more verbose template:
>
> $template
> precise,"%syslogpriority%,%syslogfacility%,%timegenerated::fulltime%,%HOSTNAME%,%syslogtag%,%msg%\n"
>
> # A template that resembles RFC 3164 on-the-wire format:
>
> # (yes, there is NO space betwen syslogtag and msg! that's important!)
>
> $template RFC3164fmt,"<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag%%msg%"
>
> # a template resembling traditional wallmessage format:
>
> $template wallmsg,"\r\n\7Message from syslogd@%HOSTNAME% at %timegenerated%
> ...\r\n %syslogtag%%msg%\n\r"
>
> # The template below emulates winsyslog format, but we need to check
> the time
>
> # stamps used. It is also a good sampleof the property replacer in action.
>
> $template
> WinSyslogFmt,"%HOSTNAME%,%timegenerated:1:10:date-rfc3339%,%timegenera
> ted:12:19:date-rfc3339%,%timegenerated:1:10:date-rfc3339%,%timegenerat
> ed:12:19:date-rfc3339%,%syslogfacility%,%syslo
>
> gpriority%,%syslogtag%%msg%\n"
>
> # A template used for database writing (notice it *is* an actual
>
> # sql-statement):
>
> $template dbFormat,"insert into SystemEvents (Message,
> Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt,
> InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%,
> '%HOSTNAME%',%syslogpr
>
> iority%, '%timereported:::date-mysql%',
> '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",sql
>
>
>
>
>
> # Turn off message reception via local log socket;
>
> # local messages are retrieved through imjournal now.
>
> # $OmitLocalLogging on
>
>
>
> ###########################
>
> #### GLOBAL DIRECTIVES ####
>
> ###########################
>
> $DebugLevel 2
>
> $DebugFile /var/log/rsyslog/rsyslog.debug
>
> #
>
> # Use traditional timestamp format.
>
> # To enable high precision timestamps, comment out the following line.
>
> #
>
> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
>
>
>
> # Filter duplicated messages
>
> $RepeatedMsgReduction on
>
>
>
> #
>
> # Set the default permissions for all log files.
>
> #
>
> $FileOwner syslog
>
> $FileGroup syslog
>
> $FileCreateMode 0660
>
> $DirCreateMode 0755
>
> $Umask 0022
>
> $PrivDropToUser syslog
>
> $PrivDropToGroup syslog
>
>
>
> #
>
> # Where to place spool and state files
>
> #
>
> $WorkDirectory /var/log/rsyslog
>
>
>
> # Include all config files in /etc/rsyslog.d/
>
> $IncludeConfig /etc/rsyslog.d/ # whole directory (must
> contain the final slash)
>
>
>
> Here is my rsyslog.conf file for Palo Alto directive:
>
> root@syslog-server-vnic-primary:/etc/rsyslog.d#<mailto:root@syslog-ser
> ver-vnic-primary:/etc/rsyslog.d#> more 10-paloalto-cef.conf
>
> # Define a template for CEF logs
>
> $template PaloAltoCEF,"/var/log/rsyslog/DOB-FW-HA-1.OCI/%programname%.log"
>
>
>
> # Use the template for logs coming from your Palo Alto firewall
>
> if $fromhost-ip == '10.40.0.210' then {
>
> *.* ?PaloAltoCEF
>
> stop
>
> }
>
> Here is my rsyslog.conf file for Catch All directive:
>
> root@syslog-server-vnic-primary:/etc/rsyslog.d#<mailto:root@syslog-ser
> ver-vnic-primary:/etc/rsyslog.d#> more 50-default.conf
>
> # Default rules for rsyslog.
>
> #
>
> # For more information see rsyslog.conf(5) and
> /etc/rsyslog.conf
>
>
>
> #
>
> # First some standard log files. Log by facility.
>
> #
>
> auth,authpriv.* /var/log/rsyslog/auth.log
>
> cron.* /var/log/rsyslog/cron.log
>
> daemon.* /var/log/rsyslog/daemon.log
>
> kern.* /var/log/rsyslog/kern.log
>
> lpr.* /var/log/rsyslog/lpr.log
>
> mail.* /var/log/rsyslog/mail.log
>
> user.* /var/log/rsyslog/user.log
>
> local0.info /var/log/rsyslog/local0_info.log
>
> #*.*
> /var/log/rsyslog/traditionalfile.log;TraditionalFormat # log to a file
> in the traditional format
>
> *.* /var/log/rsyslog/all-the-stuff.log
>
>
>
> #
>
> # Logging for the mail system. Split it up so that
>
> # it is easy to write scripts to parse these files.
>
> #
>
> # mail.info -/var/log/rsyslog/mail.info
>
> # mail.warn -/var/log/rsyslog/mail.warn
>
> # mail.err /var/log/rsyslog/mail.err
>
>
>
> #
>
> # Some "catch-all" log files.
>
> #
>
> *.=debug;\
>
> auth,authpriv.none;\
>
> news.none;mail.none /var/log/debug
>
> *.=info;*.=notice;*.=warn;\
>
> auth,authpriv.none;\
>
> cron,daemon.none;\
>
> mail,news.none /var/log/messages
>
>
>
> #
>
> # Emergencies are sent to everybody logged in.
>
> #
>
> *.emerg :omusrmsg:*
>
>
>
> #
>
> # I like to have messages displayed on the console, but only on a
> virtual
>
> # console I usually leave idle.
>
> #
>
> #daemon,mail.*;\
>
> # news.=crit;news.=err;news.=notice;\
>
> # *.=debug;*.=info;\
>
> # *.=notice;*.=warn /dev/tty8
>
>
>
> I would think they would go to the all-the-stuff log file, but no. They are
> not being logged anywhere and I do not know why not since the PAN-OS system
> logs are being logged.
well, you have anything arriving from that IP address being written to
/var/log/rsyslog/DOB-FW-HA-1.OCI/%programname%.log and then you throw away the
log, so it would never get down to the action to write it to the all-the-stuff
file.
David Lang
>
>
> Dan.
> _______________________________________________
> rsyslog mailing list
> https://list/
> s.adiscon.net%2Fmailman%2Flistinfo%2Frsyslog&data=05%7C02%7CDaniel.Dru
> mm%40dob.texas.gov%7Ca5b7d1daeca54918bec408dcbb106d71%7C32a53eeb27594e
> f4af2858fd990c7d67%7C1%7C0%7C638590926965494771%7CUnknown%7CTWFpbGZsb3
> d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7
> C60000%7C%7C%7C&sdata=jbqBlxisDwsVCMwjaT90N%2FKimmaD8t4q2sz482zP%2FFI%
> 3D&reserved=0
> http://www.r/
> syslog.com%2Fprofessional-services%2F&data=05%7C02%7CDaniel.Drumm%40do
> b.texas.gov%7Ca5b7d1daeca54918bec408dcbb106d71%7C32a53eeb27594ef4af285
> 8fd990c7d67%7C1%7C0%7C638590926965505555%7CUnknown%7CTWFpbGZsb3d8eyJWI
> joiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C60000%
> 7C%7C%7C&sdata=Y1xISoLoG4QqSzyfShnzV1At6WqTfmphvcwX27vYX%2Bo%3D&reserv
> ed=0 What's up with rsyslog? Follow
> https://twit/
> ter.com%2Frgerhards&data=05%7C02%7CDaniel.Drumm%40dob.texas.gov%7Ca5b7
> d1daeca54918bec408dcbb106d71%7C32a53eeb27594ef4af2858fd990c7d67%7C1%7C
> 0%7C638590926965512721%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJ
> QIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C60000%7C%7C%7C&sdata=vVv
> 5i4UP326ZNb2mtr1KEfnc4i9ZP58nSzWV5zmzPiw%3D&reserved=0
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
