mmjsonparse and mmnormalize have good performance, and they avoid the problem of
the unusual character showing up in the message (although they do have a problem
if the message gets truncated)
David Lang
On Thu, 26 May 2022, Mariusz Kruk via rsyslog
wrote:
Date: Thu, 26 May 2022 19:42:47 +0200
From: Mariusz Kruk via rsyslog <rsyslog@lists.adiscon.com>
To: rsyslog-users <rsyslog@lists.adiscon.com>
Cc: Mariusz Kruk <k...@epsilon.eu.org>
Subject: Re: [rsyslog] problems with tls and rsyslog
I'm using a similat setup but for performance reasons I don't embed the
original event in json but instead I glue a delimiter and an additional value
at the end of the event. Then in the aggregator I use field() to split them
back. One caveat is that you need a character which is really really unlikely
to appear in the normal event as a delimiter. Tab is not a very bad choice but
there are types of sources which can contain it sometimes.
On 26 May 2022 19:28:52 CEST, Derek Atkins via rsyslog
<rsyslog@lists.adiscon.com> wrote:
Thanks, David!!
Interesting (and pretty cool) concept. In my case I know there will
always only be the 3-level hierarchy (client/forwarder/aggregator), so I'm
not sure I need something that generic, I only need to know the client and
forwarder. Still, I will consider that.
Silly n00b question: What is the difference between $fromhost-ip (which is
what my current forwarder config is using) and $!fromhost-ip (that you
use)? (The difference being the '!' in there?)
Thanks,
-derek
On Thu, May 26, 2022 1:15 pm, David Lang wrote:
what I like to do is to format the body of the message as json, I create
$!msg=$msg and then I create a tree $!trusted and in that I add additional
metadata, including $!trusted.relay
set $.relay = $!trusted.relay;
set $!trusted.relay.last = $.relay;
set $!trusted.relay.host = $hostname;
set $!trusted.relay.last = $!fromhost-ip;
set $!trusted.relay.time = $timegenerated;
then in the final aggregator, I have all the info I could want about what
relays
the log has gone through, when it was proccessed by each relay, etc.
I also have the sender add additional metadata here as well (if it's
reading
from a file , what filename for example)
David Lang
On Thu, 26 May 2022, Derek Atkins via
rsyslog wrote:
Date: Thu, 26 May 2022 13:04:00 -0400
From: Derek Atkins via rsyslog <rsyslog@lists.adiscon.com>
To: Rainer Gerhards <rgerha...@hq.adiscon.com>
Cc: Derek Atkins <de...@ihtfp.com>, rsyslog-users
<rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] problems with tls and rsyslog
Hi Rainer.
Thank you for the reply (even though it's not the answer I was hoping to
hear).
So I guess the next question is how (or where) to add an identifier for
an
intermediary.
Let's say I have a network that looks like this:
[ Client1 ] --\
[ Client2 ] ---+- [ Forwarder1 ] -\
[ Client3 ] --/ \
+-- [ Aggregator ]
[ Client4 ] --\ /
[ Client5 ] ---+- [ Forwarder2 ] -/
[ Client6 ] --/
When I see messages at the Aggregator I want to know not only what
Client
it came from, but also what Forwarder it came through.
Right now on the forwarders I change the message to include the client
IP
and Client hostname (using set $!msg), and then send it using an onfwd
template (note that I have a intermediary variable for fromhost-ip
here):
type="string" string="%timegenerated% from:%$fromhost-ip%
%syslogseverity-text%%$!msg%\n"
At the aggregator I also need to know whether a message came from
Forwarder1 or Forwarder2, so I would like to add the Forwarder IP and
hostname to the message that goes up to the aggregator. Right now it
uses
this template for omfile:
type="string" string="%timegenerated% %msg%\n"
Will $hostname and $fromhost-ip on the aggregator be the hostname and ip
of the forwarder? Or the client?
What would be the best way to include this extra information in my log
entries?
Thanks,
-derek
On Thu, May 26, 2022 12:31 pm, Rainer Gerhards wrote:
unfortunately, this property is not yet available :-(
Rainer
El jue, 26 may 2022 a las 13:53, Derek Atkins (<de...@ihtfp.com>)
escribió:
Thanks Rainer,
This is working smashingly!
The next issue I'm trying to solve is how do I add the client
certificate
information into the log message? I'd like to add e.g. the client
certificate subject (or subjectAltName) into my log template (similar
to
how you can add the client hostname or fromhost-ip).
Again, I am having issues searching, as any combination of "rsyslog"
and
"certificate" seems to bring up documentation on "how to configure
TLS"
which, obviously, I already know how to do...
Any help or guidance would be appreciated.
Thanks,
-derek
On Tue, May 17, 2022 4:12 pm, Rainer Gerhards wrote:
> https://www.rsyslog.com/doc/v8-stable/configuration/modules/imtcp.html
>
> https://www.rsyslog.com/doc/v8-stable/configuration/modules/omfwd.html
>
> HTH
> Rainer
>
> Sent from phone, thus brief.
>
> Derek Atkins <de...@ihtfp.com> schrieb am Di., 17. Mai 2022, 22:01:
>
>> Hi,
>>
>> Are there docs on how to set this up on a per-input and/or
per-omfwd
>> basis?
>>
>> All the docs I can find suggest setting the global
>> DefaultNetstreamDriver*
>> variables, which in my case are not what I want because I need to
be
>> able
>> to use different keys/certs/CAs for the input/imtcp vs the omfwd
>> operations.
>>
>> I am running 8.2204.1.
>>
>> Thanks,
>>
>> -derek
>>
>> On Mon, April 25, 2022 3:03 am, Rainer Gerhards via rsyslog wrote:
>> > Yes, it's possible. Worked on that for quite some time last year
;-)
>> >
>> > Rainer
>> >
>> > El lun, 25 abr 2022 a las 7:41, Mariusz Kruk via rsyslog
>> > (<rsyslog@lists.adiscon.com>) escribió:
>> >>
>> >> There were some improvements to TLS handling introduced over
several
>> >> versions so you'd have to review the changelog and docs.
>> >>
>> >> But from what I see, the omfwd module supports setting separate
TLS
>> >> key/cert/cacert per action since 8.2108.
>> >>
>> >> The imtcp module also supports setting those on a per-input
level
>> since
>> >> 8.2108.
>> >>
>> >> So it should work.
>> >>
>> >> It is always a good idea to do a tcpdump and see how the
handshake
>> >> progresses and when and where it fails.
>> >>
>> >> MK
>> >>
>> >> On 24.04.2022 00:35, Shane via rsyslog wrote:
>> >> > Hi I am trying to get rsyslog to receive store/forward
messages
w/
>> tls
>> >> on
>> >> > both sides.
>> >> >
>> >> > client --->tls---> rsyslog --->tls---> remote.something
>> >> >
>> >> > I got it set up so i could send to the rsyslog server but then
i
>> >> couldn't
>> >> > add another ca/cert files. My config was using global and
>> >> defaultnetstream
>> >> >
>> >> > I found on rsyslog.com that prior to 8.2202 it couldn't use
tls
on
>> two
>> >> > different source/dest. I found the cent 7 repo and got
>> rsyslog-8.2204
>> >> > installed. Now nothing works. I think i got the config
correct
>> but
>> >> the
>> >> > client keeps getting rejected.
>> >> >
>> >> > Apr 23 17:13:39 rlog rsyslogd[11417]: GnuTLS handshake retry
>> returned
>> >> > error: The TLS connection was non-properly terminated.
[v8.2204.0
>> try
>> >> > https://www.rsyslog.com/e/2083 ]
>> >> > Apr 23 17:13:39 rlog rsyslogd[11417]: netstream session
>> 0x7f6a04013360
>> >> from
>> >> > 192.168.5.22 will be closed due to error [v8.2204.0 try
>> >> > https://www.rsyslog.com/e/2089 ]
>> >> >
>> >> > So then i tried going to the ossl module. Now its even worse.
My
>> >> config
>> >> > is a mess now too.
>> >> >
>> >> > Does tls on both sides work?
>> >> > Do I need the 8.2202+ version?
>> >> > Do you have an example config?
>> >> > _______________________________________________
>> >> > rsyslog mailing list
>> >> > https://lists.adiscon.net/mailman/listinfo/rsyslog
>> >> > http://www.rsyslog.com/professional-services/
>> >> > What's up with rsyslog? Follow https://twitter.com/rgerhards
>> >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED
by
a
>> >> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO
NOT
>> POST
>> >> if you DON'T LIKE THAT.
>> >> _______________________________________________
>> >> rsyslog mailing list
>> >> https://lists.adiscon.net/mailman/listinfo/rsyslog
>> >> http://www.rsyslog.com/professional-services/
>> >> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by
a
>> myriad
>> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
if
>> you
>> >> DON'T LIKE THAT.
>> > _______________________________________________
>> > rsyslog mailing list
>> > https://lists.adiscon.net/mailman/listinfo/rsyslog
>> > http://www.rsyslog.com/professional-services/
>> > What's up with rsyslog? Follow https://twitter.com/rgerhards
>> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>> myriad
>> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
if
you
>> > DON'T LIKE THAT.
>>
>>
>> --
>> Derek Atkins 617-623-3745
>> de...@ihtfp.com www.ihtfp.com
>> Computer and Internet Security Consultant
>>
>>
>
--
Derek Atkins 617-623-3745
de...@ihtfp.com www.ihtfp.com
Computer and Internet Security Consultant
--
Derek Atkins 617-623-3745
de...@ihtfp.com www.ihtfp.com
Computer and Internet Security Consultant
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
--
Derek Atkins 617-623-3745
de...@ihtfp.com www.ihtfp.com
Computer and Internet Security Consultant
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
