I'm using a similat setup but for performance reasons I don't embed the original event in json but instead I glue a delimiter and an additional value at the end of the event. Then in the aggregator I use field() to split them back. One caveat is that you need a character which is really really unlikely to appear in the normal event as a delimiter. Tab is not a very bad choice but there are types of sources which can contain it sometimes.
On 26 May 2022 19:28:52 CEST, Derek Atkins via rsyslog <rsyslog@lists.adiscon.com> wrote: >Thanks, David!! > >Interesting (and pretty cool) concept. In my case I know there will >always only be the 3-level hierarchy (client/forwarder/aggregator), so I'm >not sure I need something that generic, I only need to know the client and >forwarder. Still, I will consider that. > >Silly n00b question: What is the difference between $fromhost-ip (which is >what my current forwarder config is using) and $!fromhost-ip (that you >use)? (The difference being the '!' in there?) > >Thanks, > >-derek > >On Thu, May 26, 2022 1:15 pm, David Lang wrote: >> what I like to do is to format the body of the message as json, I create >> $!msg=$msg and then I create a tree $!trusted and in that I add additional >> metadata, including $!trusted.relay >> >> set $.relay = $!trusted.relay; >> set $!trusted.relay.last = $.relay; >> set $!trusted.relay.host = $hostname; >> set $!trusted.relay.last = $!fromhost-ip; >> set $!trusted.relay.time = $timegenerated; >> >> then in the final aggregator, I have all the info I could want about what >> relays >> the log has gone through, when it was proccessed by each relay, etc. >> >> I also have the sender add additional metadata here as well (if it's >> reading >> from a file , what filename for example) >> >> David Lang >> >> On Thu, 26 May 2022, Derek Atkins via >> rsyslog wrote: >> >>> Date: Thu, 26 May 2022 13:04:00 -0400 >>> From: Derek Atkins via rsyslog <rsyslog@lists.adiscon.com> >>> To: Rainer Gerhards <rgerha...@hq.adiscon.com> >>> Cc: Derek Atkins <de...@ihtfp.com>, rsyslog-users >>> <rsyslog@lists.adiscon.com> >>> Subject: Re: [rsyslog] problems with tls and rsyslog >>> >>> Hi Rainer. >>> >>> Thank you for the reply (even though it's not the answer I was hoping to >>> hear). >>> >>> So I guess the next question is how (or where) to add an identifier for >>> an >>> intermediary. >>> >>> Let's say I have a network that looks like this: >>> >>> [ Client1 ] --\ >>> [ Client2 ] ---+- [ Forwarder1 ] -\ >>> [ Client3 ] --/ \ >>> +-- [ Aggregator ] >>> [ Client4 ] --\ / >>> [ Client5 ] ---+- [ Forwarder2 ] -/ >>> [ Client6 ] --/ >>> >>> >>> When I see messages at the Aggregator I want to know not only what >>> Client >>> it came from, but also what Forwarder it came through. >>> >>> Right now on the forwarders I change the message to include the client >>> IP >>> and Client hostname (using set $!msg), and then send it using an onfwd >>> template (note that I have a intermediary variable for fromhost-ip >>> here): >>> >>> type="string" string="%timegenerated% from:%$fromhost-ip% >>> %syslogseverity-text%%$!msg%\n" >>> >>> At the aggregator I also need to know whether a message came from >>> Forwarder1 or Forwarder2, so I would like to add the Forwarder IP and >>> hostname to the message that goes up to the aggregator. Right now it >>> uses >>> this template for omfile: >>> >>> type="string" string="%timegenerated% %msg%\n" >>> >>> Will $hostname and $fromhost-ip on the aggregator be the hostname and ip >>> of the forwarder? Or the client? >>> >>> What would be the best way to include this extra information in my log >>> entries? >>> >>> Thanks, >>> >>> -derek >>> >>> On Thu, May 26, 2022 12:31 pm, Rainer Gerhards wrote: >>>> unfortunately, this property is not yet available :-( >>>> >>>> Rainer >>>> >>>> El jue, 26 may 2022 a las 13:53, Derek Atkins (<de...@ihtfp.com>) >>>> escribió: >>>>> >>>>> Thanks Rainer, >>>>> >>>>> This is working smashingly! >>>>> >>>>> The next issue I'm trying to solve is how do I add the client >>>>> certificate >>>>> information into the log message? I'd like to add e.g. the client >>>>> certificate subject (or subjectAltName) into my log template (similar >>>>> to >>>>> how you can add the client hostname or fromhost-ip). >>>>> >>>>> Again, I am having issues searching, as any combination of "rsyslog" >>>>> and >>>>> "certificate" seems to bring up documentation on "how to configure >>>>> TLS" >>>>> which, obviously, I already know how to do... >>>>> >>>>> Any help or guidance would be appreciated. >>>>> >>>>> Thanks, >>>>> >>>>> -derek >>>>> >>>>> On Tue, May 17, 2022 4:12 pm, Rainer Gerhards wrote: >>>>> > https://www.rsyslog.com/doc/v8-stable/configuration/modules/imtcp.html >>>>> > >>>>> > https://www.rsyslog.com/doc/v8-stable/configuration/modules/omfwd.html >>>>> > >>>>> > HTH >>>>> > Rainer >>>>> > >>>>> > Sent from phone, thus brief. >>>>> > >>>>> > Derek Atkins <de...@ihtfp.com> schrieb am Di., 17. Mai 2022, 22:01: >>>>> > >>>>> >> Hi, >>>>> >> >>>>> >> Are there docs on how to set this up on a per-input and/or >>>>> per-omfwd >>>>> >> basis? >>>>> >> >>>>> >> All the docs I can find suggest setting the global >>>>> >> DefaultNetstreamDriver* >>>>> >> variables, which in my case are not what I want because I need to >>>>> be >>>>> >> able >>>>> >> to use different keys/certs/CAs for the input/imtcp vs the omfwd >>>>> >> operations. >>>>> >> >>>>> >> I am running 8.2204.1. >>>>> >> >>>>> >> Thanks, >>>>> >> >>>>> >> -derek >>>>> >> >>>>> >> On Mon, April 25, 2022 3:03 am, Rainer Gerhards via rsyslog wrote: >>>>> >> > Yes, it's possible. Worked on that for quite some time last year >>>>> ;-) >>>>> >> > >>>>> >> > Rainer >>>>> >> > >>>>> >> > El lun, 25 abr 2022 a las 7:41, Mariusz Kruk via rsyslog >>>>> >> > (<rsyslog@lists.adiscon.com>) escribió: >>>>> >> >> >>>>> >> >> There were some improvements to TLS handling introduced over >>>>> several >>>>> >> >> versions so you'd have to review the changelog and docs. >>>>> >> >> >>>>> >> >> But from what I see, the omfwd module supports setting separate >>>>> TLS >>>>> >> >> key/cert/cacert per action since 8.2108. >>>>> >> >> >>>>> >> >> The imtcp module also supports setting those on a per-input >>>>> level >>>>> >> since >>>>> >> >> 8.2108. >>>>> >> >> >>>>> >> >> So it should work. >>>>> >> >> >>>>> >> >> It is always a good idea to do a tcpdump and see how the >>>>> handshake >>>>> >> >> progresses and when and where it fails. >>>>> >> >> >>>>> >> >> MK >>>>> >> >> >>>>> >> >> On 24.04.2022 00:35, Shane via rsyslog wrote: >>>>> >> >> > Hi I am trying to get rsyslog to receive store/forward >>>>> messages >>>>> w/ >>>>> >> tls >>>>> >> >> on >>>>> >> >> > both sides. >>>>> >> >> > >>>>> >> >> > client --->tls---> rsyslog --->tls---> remote.something >>>>> >> >> > >>>>> >> >> > I got it set up so i could send to the rsyslog server but then >>>>> i >>>>> >> >> couldn't >>>>> >> >> > add another ca/cert files. My config was using global and >>>>> >> >> defaultnetstream >>>>> >> >> > >>>>> >> >> > I found on rsyslog.com that prior to 8.2202 it couldn't use >>>>> tls >>>>> on >>>>> >> two >>>>> >> >> > different source/dest. I found the cent 7 repo and got >>>>> >> rsyslog-8.2204 >>>>> >> >> > installed. Now nothing works. I think i got the config >>>>> correct >>>>> >> but >>>>> >> >> the >>>>> >> >> > client keeps getting rejected. >>>>> >> >> > >>>>> >> >> > Apr 23 17:13:39 rlog rsyslogd[11417]: GnuTLS handshake retry >>>>> >> returned >>>>> >> >> > error: The TLS connection was non-properly terminated. >>>>> [v8.2204.0 >>>>> >> try >>>>> >> >> > https://www.rsyslog.com/e/2083 ] >>>>> >> >> > Apr 23 17:13:39 rlog rsyslogd[11417]: netstream session >>>>> >> 0x7f6a04013360 >>>>> >> >> from >>>>> >> >> > 192.168.5.22 will be closed due to error [v8.2204.0 try >>>>> >> >> > https://www.rsyslog.com/e/2089 ] >>>>> >> >> > >>>>> >> >> > So then i tried going to the ossl module. Now its even worse. >>>>> My >>>>> >> >> config >>>>> >> >> > is a mess now too. >>>>> >> >> > >>>>> >> >> > Does tls on both sides work? >>>>> >> >> > Do I need the 8.2202+ version? >>>>> >> >> > Do you have an example config? >>>>> >> >> > _______________________________________________ >>>>> >> >> > rsyslog mailing list >>>>> >> >> > https://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> >> >> > http://www.rsyslog.com/professional-services/ >>>>> >> >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>> >> >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED >>>>> by >>>>> a >>>>> >> >> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO >>>>> NOT >>>>> >> POST >>>>> >> >> if you DON'T LIKE THAT. >>>>> >> >> _______________________________________________ >>>>> >> >> rsyslog mailing list >>>>> >> >> https://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> >> >> http://www.rsyslog.com/professional-services/ >>>>> >> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>> >> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by >>>>> a >>>>> >> myriad >>>>> >> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>>>> if >>>>> >> you >>>>> >> >> DON'T LIKE THAT. >>>>> >> > _______________________________________________ >>>>> >> > rsyslog mailing list >>>>> >> > https://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> >> > http://www.rsyslog.com/professional-services/ >>>>> >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>> >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>> >> myriad >>>>> >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>>>> if >>>>> you >>>>> >> > DON'T LIKE THAT. >>>>> >> >>>>> >> >>>>> >> -- >>>>> >> Derek Atkins 617-623-3745 >>>>> >> de...@ihtfp.com www.ihtfp.com >>>>> >> Computer and Internet Security Consultant >>>>> >> >>>>> >> >>>>> > >>>>> >>>>> >>>>> -- >>>>> Derek Atkins 617-623-3745 >>>>> de...@ihtfp.com www.ihtfp.com >>>>> Computer and Internet Security Consultant >>>>> >>>> >>> >>> >>> -- >>> Derek Atkins 617-623-3745 >>> de...@ihtfp.com www.ihtfp.com >>> Computer and Internet Security Consultant >>> >>> _______________________________________________ >>> rsyslog mailing list >>> https://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. > > >-- > Derek Atkins 617-623-3745 > de...@ihtfp.com www.ihtfp.com > Computer and Internet Security Consultant > >_______________________________________________ >rsyslog mailing list >https://lists.adiscon.net/mailman/listinfo/rsyslog >http://www.rsyslog.com/professional-services/ >What's up with rsyslog? Follow https://twitter.com/rgerhards >NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of >sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE >THAT. -- Sent from my Android device with K-9 Mail. Please excuse my brevity. _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
