Hi David Point taken on the fromhost-ip is the last hop just before "me"(the syslog hosts)
This is what I wanted to do: template(name="forwardlog" type="string" string="<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg% <%fromhost-ip%|%timereported:::date-unixtimestamp%|%$myhostname%|hlu>\n") I was thinking of adding the headers on the LB side, and then use these headers on the rsyslog side to add the source IP to my log message using a template. So what I'm actually is asking. Would rsyslog be able to fetch a X-Forward Header or X-Real-IP header and use that in a template and add it to the message ? So I "just" wanted to pickup the X-Forward header and use that instead of fromhost-ip I will try to look into the corosync/pacemaker and see if this can help me with the challenge. Best regards Jan -----Original Message----- From: David Lang <da...@lang.hm> Sent: 22. august 2023 11:11 To: JPIM (Jan Primdahl Madsen) via rsyslog <rsyslog@lists.adiscon.com> Cc: JPIM (Jan Primdahl Madsen) <j...@aevengroup.com> Subject: Re: [rsyslog] Handling Fromhost-IP on Loadbalanced TCP log messages Caution: This email originates from outside the company. You should carefully examine the sender and any links or actions before clicking. Use the "Report Phishing" button on the ribbon to have Security analyze the email if in doubt. the syslog message is supposed to have the hostname of the sending system in it, you can use that directly if you want. fromhost-ip is specifically bypassing the metadata in the log itself to tell you the IP of the last hop that sent it to you, and as you see, when you run it through a non-transparent load balancer, you get the load balancer instead of the initial sender. where would you set your additional headers? you can set the hostname there at least as easily as you could add something extra (I do like to add extra stuff, I make the body of the message JSON and add a subtree with a company-specific name that's unlikely to match anything in the message, and then make the original message $!msg inside this json one problem with using a load balancer like nginx is that it doesn't understand the syslog protocol, so it can't break the connection to rebalance at a sane point If you are doing the load balancing on the two systems that are receiving the messages, instead of using nginx/keepalived, look at using corosync/pacemaker and use the CLUSTERIP option to share a single VIP across both machines. If you include the senders port number in the balancing, it works well to spread the incoming load pretty evenly Rsyslog also has the rebindinterval option that you can set on the senders to have them disconnect and reconnect periodically (which give a load balancer the chance to rebalance when it reconnects I like to put a pair of load balanced syslog relays on each subnet (sometimes multi-homed as you are doing) to gather all the logs from that subnet, clean them up, parse them, add metadata, and then deliver to the central syslog server (if that central server is in a different datacenter, the RELP protocol for that hop helps avoid log loss that can happen with TCP) David Lang On Tue, 22 Aug 2023, JPIM (Jan Primdahl Madsen) via rsyslog wrote: > Date: Tue, 22 Aug 2023 08:58:02 +0000 > From: "JPIM (Jan Primdahl Madsen) via rsyslog" > <rsyslog@lists.adiscon.com> > To: "rsyslog@lists.adiscon.com" <rsyslog@lists.adiscon.com> > Cc: "JPIM (Jan Primdahl Madsen)" <j...@aevengroup.com> > Subject: [rsyslog] Handling Fromhost-IP on Loadbalanced TCP log > messages > > > Hello All rsyslog users and developers 😊 > > The main problem. > When receiving loadbalanced TCP syslog messages our Loadbalancer IP is the IP > added to the variable $fromhost-ip no matter what we do. This is due to the > fact that the LB is doing TCP routing and becomes the sending IP on the > network layer. The syslog message does not contain the IP of the sending node. > > The Setup: > The setup contains only 2 servers which have 2 subnets (Frontend and a > Backend(For TCP messages)) The servers are using the following > software > > * Nginx (For LB) > * Keepalived (For VIP handling) > * Rsyslog (yes for syslog messages) > > > Question 1: > I was thinking about adding some Proxy Headers to the log message on the LB > (nginx) side and then using that in rsyslog to overwrite the $fronhost-ip. Is > this possible in anyway to have rsyslog to use Proxy Headers like: > > * X-Forwarded-For > * X-Real-IP > > If possible howto do it ? > If not, any other great suggestions to preserve IP address of sending > source in a LB TCP syslog setup is appreciated > > Please be aware, adding more HW to the setup is not an option. > > Best regards > Jan P. Madsen > > This e-mail (including any attachments) is intended for the addressee(s) > stated above only and may contain confidential information protected by law. > You are hereby notified that any unauthorized reading, disclosure, copying or > distribution of this e-mail or use of information contained herein is > strictly prohibited and may violate rights to proprietary information. If you > are not an intended recipient, please return this e-mail to the sender and > delete it immediately hereafter. Thank you. > _______________________________________________ > rsyslog mailing list > https://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites > beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
