I'm not saying they don't :-)But json parsing will always be slower than just finding a single character ;-)
Works for me, anyway. On 26.05.2022 19:48, David Lang wrote:
mmjsonparse and mmnormalize have good performance, and they avoid the problem of the unusual character showing up in the message (although they do have a problem if the message gets truncated)David Lang On Thu, 26 May 2022, Mariusz Kruk via rsyslog wrote:Date: Thu, 26 May 2022 19:42:47 +0200 From: Mariusz Kruk via rsyslog <rsyslog@lists.adiscon.com> To: rsyslog-users <rsyslog@lists.adiscon.com> Cc: Mariusz Kruk <k...@epsilon.eu.org> Subject: Re: [rsyslog] problems with tls and rsyslogI'm using a similat setup but for performance reasons I don't embed the original event in json but instead I glue a delimiter and an additional value at the end of the event. Then in the aggregator I use field() to split them back. One caveat is that you need a character which is really really unlikely to appear in the normal event as a delimiter. Tab is not a very bad choice but there are types of sources which can contain it sometimes.On 26 May 2022 19:28:52 CEST, Derek Atkins via rsyslog <rsyslog@lists.adiscon.com> wrote:Thanks, David!! Interesting (and pretty cool) concept. In my case I know there willalways only be the 3-level hierarchy (client/forwarder/aggregator), so I'm not sure I need something that generic, I only need to know the client andforwarder. Still, I will consider that.Silly n00b question: What is the difference between $fromhost-ip (which iswhat my current forwarder config is using) and $!fromhost-ip (that you use)? (The difference being the '!' in there?) Thanks, -derek On Thu, May 26, 2022 1:15 pm, David Lang wrote:what I like to do is to format the body of the message as json, I create $!msg=$msg and then I create a tree $!trusted and in that I add additionalmetadata, including $!trusted.relay set $.relay = $!trusted.relay; set $!trusted.relay.last = $.relay; set $!trusted.relay.host = $hostname; set $!trusted.relay.last = $!fromhost-ip; set $!trusted.relay.time = $timegenerated;then in the final aggregator, I have all the info I could want about whatrelays the log has gone through, when it was proccessed by each relay, etc. I also have the sender add additional metadata here as well (if it's reading from a file , what filename for example) David Lang On Thu, 26 May 2022, Derek Atkins via rsyslog wrote:Date: Thu, 26 May 2022 13:04:00 -0400 From: Derek Atkins via rsyslog <rsyslog@lists.adiscon.com> To: Rainer Gerhards <rgerha...@hq.adiscon.com> Cc: Derek Atkins <de...@ihtfp.com>, rsyslog-users <rsyslog@lists.adiscon.com> Subject: Re: [rsyslog] problems with tls and rsyslog Hi Rainer.Thank you for the reply (even though it's not the answer I was hoping tohear).So I guess the next question is how (or where) to add an identifier foran intermediary. Let's say I have a network that looks like this: [ Client1 ] --\ [ Client2 ] ---+- [ Forwarder1 ] -\ [ Client3 ] --/ \ +-- [ Aggregator ] [ Client4 ] --\ / [ Client5 ] ---+- [ Forwarder2 ] -/ [ Client6 ] --/ When I see messages at the Aggregator I want to know not only what Client it came from, but also what Forwarder it came through.Right now on the forwarders I change the message to include the clientIPand Client hostname (using set $!msg), and then send it using an onfwdtemplate (note that I have a intermediary variable for fromhost-ip here): type="string" string="%timegenerated% from:%$fromhost-ip% %syslogseverity-text%%$!msg%\n" At the aggregator I also need to know whether a message came from Forwarder1 or Forwarder2, so I would like to add the Forwarder IP and hostname to the message that goes up to the aggregator. Right now it uses this template for omfile: type="string" string="%timegenerated% %msg%\n"Will $hostname and $fromhost-ip on the aggregator be the hostname and ipof the forwarder? Or the client?What would be the best way to include this extra information in my logentries? Thanks, -derek On Thu, May 26, 2022 12:31 pm, Rainer Gerhards wrote:unfortunately, this property is not yet available :-( Rainer El jue, 26 may 2022 a las 13:53, Derek Atkins (<de...@ihtfp.com>) escribió:Thanks Rainer, This is working smashingly! The next issue I'm trying to solve is how do I add the client certificate information into the log message? I'd like to add e.g. the clientcertificate subject (or subjectAltName) into my log template (similarto how you can add the client hostname or fromhost-ip).Again, I am having issues searching, as any combination of "rsyslog"and "certificate" seems to bring up documentation on "how to configure TLS" which, obviously, I already know how to do... Any help or guidance would be appreciated. Thanks, -derek On Tue, May 17, 2022 4:12 pm, Rainer Gerhards wrote:> https://www.rsyslog.com/doc/v8-stable/configuration/modules/imtcp.html>> https://www.rsyslog.com/doc/v8-stable/configuration/modules/omfwd.html> > HTH > Rainer > > Sent from phone, thus brief. >> Derek Atkins <de...@ihtfp.com> schrieb am Di., 17. Mai 2022, 22:01:> >> Hi, >> >> Are there docs on how to set this up on a per-input and/or per-omfwd >> basis? >> >> All the docs I can find suggest setting the global >> DefaultNetstreamDriver*>> variables, which in my case are not what I want because I need tobe >> able >> to use different keys/certs/CAs for the input/imtcp vs the omfwd >> operations. >> >> I am running 8.2204.1. >> >> Thanks, >> >> -derek >>>> On Mon, April 25, 2022 3:03 am, Rainer Gerhards via rsyslog wrote: >> > Yes, it's possible. Worked on that for quite some time last year;-) >> > >> > Rainer >> > >> > El lun, 25 abr 2022 a las 7:41, Mariusz Kruk via rsyslog >> > (<rsyslog@lists.adiscon.com>) escribió: >> >> >> >> There were some improvements to TLS handling introduced over several >> >> versions so you'd have to review the changelog and docs. >> >>>> >> But from what I see, the omfwd module supports setting separateTLS >> >> key/cert/cacert per action since 8.2108. >> >> >> >> The imtcp module also supports setting those on a per-input level >> since >> >> 8.2108. >> >> >> >> So it should work. >> >> >> >> It is always a good idea to do a tcpdump and see how the handshake >> >> progresses and when and where it fails. >> >> >> >> MK >> >> >> >> On 24.04.2022 00:35, Shane via rsyslog wrote: >> >> > Hi I am trying to get rsyslog to receive store/forward messages w/ >> tls >> >> on >> >> > both sides. >> >> > >> >> > client --->tls---> rsyslog --->tls---> remote.something >> >> >>> >> > I got it set up so i could send to the rsyslog server but theni >> >> couldn't >> >> > add another ca/cert files. My config was using global and >> >> defaultnetstream >> >> > >> >> > I found on rsyslog.com that prior to 8.2202 it couldn't use tls on >> two >> >> > different source/dest. I found the cent 7 repo and got >> rsyslog-8.2204 >> >> > installed. Now nothing works. I think i got the config correct >> but >> >> the >> >> > client keeps getting rejected. >> >> >>> >> > Apr 23 17:13:39 rlog rsyslogd[11417]: GnuTLS handshake retry>> returned >> >> > error: The TLS connection was non-properly terminated. [v8.2204.0 >> try >> >> > https://www.rsyslog.com/e/2083 ] >> >> > Apr 23 17:13:39 rlog rsyslogd[11417]: netstream session >> 0x7f6a04013360 >> >> from >> >> > 192.168.5.22 will be closed due to error [v8.2204.0 try >> >> > https://www.rsyslog.com/e/2089 ] >> >> >>> >> > So then i tried going to the ossl module. Now its even worse.My >> >> config >> >> > is a mess now too. >> >> > >> >> > Does tls on both sides work? >> >> > Do I need the 8.2202+ version? >> >> > Do you have an example config? >> >> > _______________________________________________ >> >> > rsyslog mailing list >> >> > https://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > http://www.rsyslog.com/professional-services/>> >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVEDby a >> >> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >> POST >> >> if you DON'T LIKE THAT. >> >> _______________________________________________ >> >> rsyslog mailing list >> >> https://lists.adiscon.net/mailman/listinfo/rsyslog >> >> http://www.rsyslog.com/professional-services/ >> >> What's up with rsyslog? Follow https://twitter.com/rgerhards>> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED bya >> myriad>> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POSTif >> you >> >> DON'T LIKE THAT. >> > _______________________________________________ >> > rsyslog mailing list >> > https://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > What's up with rsyslog? Follow https://twitter.com/rgerhards>> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a>> myriad>> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POSTif you >> > DON'T LIKE THAT. >> >> >> -- >> Derek Atkins 617-623-3745 >> de...@ihtfp.com www.ihtfp.com >> Computer and Internet Security Consultant >> >> > -- Derek Atkins 617-623-3745 de...@ihtfp.com www.ihtfp.com Computer and Internet Security Consultant-- Derek Atkins 617-623-3745 de...@ihtfp.com www.ihtfp.com Computer and Internet Security Consultant _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhardsNOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if youDON'T LIKE THAT.-- Derek Atkins 617-623-3745 de...@ihtfp.com www.ihtfp.com Computer and Internet Security Consultant _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhardsNOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.-- Sent from my Android device with K-9 Mail. Please excuse my brevity. _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhardsNOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
