Hi!
If I use omfile then logs contains messages in my desired format but not
with ommongodb. tcpdump shows:
..msg.....{"foo":"bar"}..
Marcin
W dniu 2022-10-05 08:05, Mariusz Kruk via rsyslog napisał(a):
Looks relatively normal.
You can of course try writing to a file with your "i-json" template to
make sure that's what you want to be sent to mongodb.
But then, if your ommongodb action does contain the template="i-json"
parameter, it simply should work.
The only other thing you can do to make sure what's going on over the
wire is of course the tcpdump (unless your traffic is encrypted).
Ot simply looks that it should work - it should not need any more
"processing" as it is.
On 4.10.2022 15:45, Marcin Mirosław wrote:
:)
I had to censore log...
Debug line with all properties:
FROMHOST: 'localhost', fromhost-ip: '127.0.0.1', HOSTNAME: 'linux',
PRI: 174,
syslogtag 'a1', programname: 'x1', APP-NAME: 'x1', PROCID: '-', MSGID:
'-',
TIMESTAMP: 'Oct 4 15:38:53', STRUCTURED-DATA: '[timeQuality
tzKnown="1" isSynced="1" syncAccuracy="415383"]',
msg: '{"foo":"bar","mode":750,"date":"2022-10-04T15:38:53"}'
escaped msg: '{"foo":"bar","mode":750,"date":"2022-10-04T15:38:53"}'
inputname: imtcp rawmsg: '<174>1 2022-10-04T15:38:53.219052+02:00
linux a1 - - [timeQuality tzKnown="1" isSynced="1"
syncAccuracy="415383"] {"foo":"bar
","mode":750,"date":"2022-10-04T15:38:53"}'
$!:
$.:
$/:
W dniu 2022-10-04 09:18, Mariusz Kruk via rsyslog napisał(a):
:-)
OK, I understand that you did
template(name="i-json" type="string" string="%msg%")
action(type="ommongodb" server="..." [...] template="i-json")
right?
This way you should indeed be pushing the %msg% part of the incoming
syslog message.
Try adding
action(type="omfile" file="/tmp/debug.log"
template="RSYSLOG_DebugFormat")
immediately before your ommongodb action to see what exactly your
properties look like. That usually helps finding what's happening
inside your processing pipeline.
On 4.10.2022 09:03, Marcin Mirosław wrote:
I did in the part about connection to mongodb ;)
"It will be used automatically if no other template is specified to
be used"
But I specified template: template(name="ui-json" type="string"
string="%msg%")
so if I understand qutoed docs correctly, all about default, canned
template doesn't apply to my case, am I right?
W dniu 2022-10-04 08:27, Mariusz Kruk napisał(a):
You didn't read the docs, did you? ;-)
https://www.rsyslog.com/doc/v8-stable/configuration/modules/ommongodb.html
<https://www.rsyslog.com/doc/v8-stable/configuration/modules/ommongodb.html>
"Note rsyslog contains a canned default template to write to the
MongoDB. It will be used automatically if no other template is
specified to be used. This template is:
template(name="BSON" type="string" string="\\"sys\\" :
\\"%hostname%\\",
\\"time\\" : \\"%timereported:::rfc3339%\\", \\"time\_rcvd\\" :
\\"%timegenerated:::rfc3339%\\", \\"msg\\" : \\"%msg%\\",
\\"syslog\_fac\\" : \\"%syslogfacility%\\", \\"syslog\_server\\" :
\\"%syslogseverity%\\", \\"syslog\_tag\\" : \\"%syslogtag%\\",
\\"procid\\" : \\"%programname%\\", \\"pid\\" : \\"%procid%\\",
\\"level\\" : \\"%syslogpriority-text%\\"")
This creates the BSON document needed for MongoDB if no template is
specified. The default schema is aligned to CEE and project
lumberjack. As such, the field names are standard lumberjack field
names, and *not* rsyslog property names
<https://www.rsyslog.com/doc/v8-stable/configuration/modules/property_replacer.html>."
On 3.10.2022 22:02, Marcin Mirosław wrote:
W dniu 03.10.2022 o 18:55, Mariusz Kruk via rsyslog pisze:
Don't know about this particular output module but in general
what you want is for rsyslog to parse the message and insert it
as json object.
Meseems that now rsyslog put %msg% as json object. (
ex: msg: '{"foo":"bar"}' }
)
So you need to use parse_json() function on the input string and
then use proper template which will render the json to
appropirate string. I use similar approach (without the parsing
part) to create output json for Splunk's HEC input - the idea is
roughly the same.
On 3.10.2022 18:35, Marcin Mirosław via rsyslog wrote:
Maybe when I show examples from mongo it will be more clear.
> db.log2.find()
[
{ _id: ObjectId("633b0ea6b8f2a532cfa6c64c"), msg:
'{"foo":"bar"}' },
{ _id: ObjectId("633b0eb6b8f2a532cfa6c64d"), foo: 'bar' }
]
First record shows how msg is inserted to mongo by rsyslog.
Second record is what I would like to get.
rsyslog do:
db.log2.insert({msg:'{"foo":"bar"}'})
but I'd like to have:
db.log2.insert({"foo":"bar"})
Marcin
W dniu 03.10.2022 o 17:36, Rainer Gerhards pisze:
I do not fully understand the question (maybe language issue on
my
side), but there is a syntax error:
In a string template, properties must be enclosed in percent
sign. so:
... string="%msg%
HTH
Rainer
El lun, 3 oct 2022 a las 13:18, Marcin Mirosław via rsyslog
(<rsyslog@lists.adiscon.com>
<mailto:rsyslog@lists.adiscon.com>) escribió:
Hello!
Field msg contains complete json with data. I would like to
instert it
to mongodb as is. But now rsyslog inserts it as a value of key
"msg".
So now is:
msg: '{"foo":"bar"}
a i'd like to insert: only:
'{"foo","bar"}'
I tried with template:
template(name="ui-json" type="string" string="%msg")
but it doesn't do what I need. Is it possible to configure it
using
template or this is imposible due to ommnongodb limitation?
Marcin Mirosław
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
<https://lists.adiscon.net/mailman/listinfo/rsyslog>
http://www.rsyslog.com/professional-services/
<http://www.rsyslog.com/professional-services/>
What's up with rsyslog? Follow https://twitter.com/rgerhards
<https://twitter.com/rgerhards>
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED
by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE
and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
<https://lists.adiscon.net/mailman/listinfo/rsyslog>
http://www.rsyslog.com/professional-services/
<http://www.rsyslog.com/professional-services/>
What's up with rsyslog? Follow https://twitter.com/rgerhards
<https://twitter.com/rgerhards>
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by
a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO
NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
<https://lists.adiscon.net/mailman/listinfo/rsyslog>
http://www.rsyslog.com/professional-services/
<http://www.rsyslog.com/professional-services/>
What's up with rsyslog? Follow https://twitter.com/rgerhards
<https://twitter.com/rgerhards>
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT
POST
if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
