Ok gents, here you go This is rsyslog v8.2102…
## Start rsyslog.conf
$AbortOnUncleanConfig on
$RepeatedMsgReduction off
$umask 0000
$CreateDirs on
global(
processInternalMessages="on"
parser.dropTrailingLFOnReception="on"
parser.spaceLFOnReceive="on"
parser.escapeControlCharactersOnReceive="on"
preserveFQDN="on"
net.enableDNS="off"
workDirectory="/home/user/demo"
maxMessageSize="16k"
)
module(load="imudp")
module(load="imptcp")
module(load="imfile")
module(load="mmjsonparse")
module(load="builtin:omfile" Template="RSYSLOG_FileFormat")
template(name="demo_out" type="list") {
constant(value="R: ") property(name="rawmsg") constant(value="\n")
constant(value="M: ") property(name="msg") constant(value="\n")
constant(value="I: ") property(name="inputname") constant(value="\n")
constant(value="S: ") property(name="$!strmsg" format="jsonr")
constant(value="\n")
constant(value="J: ") property(name="$!jsonmsg") constant(value="\n\n")
}
ruleset(name="json_check") {
set $.isjson = parse_json($msg, "\$!parsed");
if ($.isjson == 0) then {
set $!jsonmsg = $msg;
set $!strmsg = "";
} else {
set $!jsonmsg = "null";
set $!strmsg = $msg;
}
}
ruleset(name="to_debug") {
action(
type="omfile"
name="debug_demo"
template="demo_out"
file="/home/user/demo/debug.out"
)
}
ruleset(name="from_Demo") {
call json_check
call to_debug
}
input(type="imudp" name="udp-55141-in" port="55141" ruleset="from_Demo")
input(type="imptcp" name="ptcp-55141-in" port="55141" ruleset="from_Demo")
input(type="imfile"
file="/home/user/practice/*.demolog"
tag="Demo"
facility="auth"
severity="info"
persiststateinterval="0"
readmode="0"
readtimeout="5"
ruleset="from_Demo"
)
## End rsyslog.conf
Then, from the output file, some correct examples...
R: <111>1 2021-06-30T15:05:48.323+00:00 azlapncismlog02 logaudit 22815 test -
test message at 1625065548.325
M: test message at 1625065548.325
I: ptcp-55141-in
S: test message at 1625065548.325
J: null
R: <111>1 2021-06-30T15:05:55.911+00:00 azlapncismlog02 logaudit 25337 test -
test message at 1625065555.912
M: test message at 1625065555.912
I: udp-55141-in
S: test message at 1625065555.912
J: null
R: test message at 1625065662.605
M: test message at 1625065662.605
I: imfile
S: test message at 1625065662.605
J: null
R: a b c d e f
M: a b c d e f
I: imfile
S: a b c d e f
J: null
R: a,b,c,d,e,f
M: a,b,c,d,e,f
I: imfile
S: a,b,c,d,e,f
J: null
R: a|b|c|d|e|f
M: a|b|c|d|e|f
I: imfile
S: a|b|c|d|e|f
J: null
R: <111>1 2021-06-30T15:13:24.190+00:00 azlapncismlog02 logaudit 4266 test -
{"str_field":"pi","num_field":3.14159}
M: {"str_field":"pi","num_field":3.14159}
I: ptcp-55141-in
S:
J: {"str_field":"pi","num_field":3.14159}
Then, from the output file, some incorrect examples...
R: 0,1,2,3,4,5
M: 0,1,2,3,4,5
I: imfile
S:
J: 0,1,2,3,4,5
R: <111>1 2021-06-30T15:13:51.909+00:00 azlapncismlog02 logaudit 4981 test -
"str_field":"pi","num_field":3.14159
M: "str_field":"pi","num_field":3.14159
I: ptcp-55141-in
S:
J: "str_field":"pi","num_field":3.14159
R: <111>1 2021-06-30T15:15:56.392+00:00 azlapncismlog02 logaudit 8469 test -
003491626A431T|Success|2021-04-28 22:15:31|event.type.sybtype|10.10.10.10
M: 003491626A431T|Success|2021-04-28 22:15:31|event.type.sybtype|10.10.10.10
I: ptcp-55141-in
S:
J: 003491626A431T|Success|2021-04-28 22:15:31|event.type.sybtype|10.10.10.10
> On Jun 17, 2021, at 12:03, David Lang <[email protected]> wrote:
>
> what is msg and rawmsg in these two cases?
>
> what you are showing is the output, what we need to look at is the input to
> be parsed.
>
> David Lang
>
> On Thu, 17 Jun 2021, John Chivian wrote:
>
>> Date: Thu, 17 Jun 2021 11:37:14 -0500
>> From: John Chivian <[email protected]>
>> To: David Lang <[email protected]>
>> Cc: John Chivian via rsyslog <[email protected]>
>> Subject: Re: [rsyslog] Odd parse_json Behavior With imfile Input
>> Hi David:
>>
>> I wish I could say that was the case, but the msg object is exactly what it
>> should be. Here’s the output in error...
>>
>> "string": "", "json":
>> 206|domain1\user1|SEC_ACCOUNTING_CONSULTANT|N|2019-10-21
>> 15:46:33.487|domain2\user2|10.224.129.254
>>
>> …and here’s the output with parentheses added…
>>
>> "string": "(206|domain1\\user1|SEC_ACCOUNTING_CONSULTANT|N|2019-10-21
>> 15:46:33.487|domain2\\user2|10.224.129.254)", "json": "null"
>>
>> …produced by…
>>
>> constant(value="\"string\": \"") property(name="$!smsg" format="jsonr")
>> constant(value="\",")
>> constant(value="\"json\": ") property(name="$!jmsg")
>> As you can see the only difference in msg is the added parenthesis, and both
>> represent exactly what msg should be.
>>
>>
>> Regards,
>>
>>
>>
>>> On Jun 17, 2021, at 11:16, David Lang <[email protected]> wrote:
>>>
>>> whenever you have a parsing problem, log the messages with the template
>>> RSYSLOG_DebugFormat
>>>
>>> 90%+ of the time, we find that the contents of the variables are not what
>>> you are expecting, and the rest of the time, seeing the debug output lets
>>> us figure out what is going wrong with the parser
>>>
>>> Also, what version of rsyslog are you running?
>>>
>>> David Lang
>>>
>>> On Thu, 17 Jun 2021, John Chivian via rsyslog wrote:
>>>
>>>> Date: Thu, 17 Jun 2021 11:05:21 -0500
>>>> From: John Chivian via rsyslog <[email protected]>
>>>> To: rsyslog-users <[email protected]>
>>>> Cc: John Chivian <[email protected]>
>>>> Subject: [rsyslog] Odd parse_json Behavior With imfile Input
>>>> I am using the parse_json function to determine whether or not the msg
>>>> element is a JSON object.
>>>>
>>>> set $.isjson = parse_json($msg, "\$!parsed");
>>>>
>>>> if ($.isjson == 0) then {
>>>> set $!jmsg = $msg;
>>>> set $!smsg = "";
>>>> } else {
>>>> set $!jmsg = "null";
>>>> set $!smsg = $msg;
>>>> }
>>>>
>>>>
>>>> For network input this seems to always work, but for file input we seeing
>>>> odd behavior where some strings that are not JSON objects are being
>>>> identified as if they were JSON objects.
>>>>
>>>> Case in point is the msg "a,b,c,d,e,f”. If received over the network
>>>> (imptcp) with a proper syslog header, this msg is correctly identified as
>>>> not being a JSON object, and the value is applied to the $!smsg element in
>>>> the if construct that follows.
>>>>
>>>> However, if that same string is received in file input (imfile), the
>>>> message is incorrectly identified as being a JSON object and the value is
>>>> applied to the $!jmsg element. The same is true if the fields are pipe
>>>> delimited as "a|b|c|d|e|f".
>>>> If I modify the msg to be wrapped in parenthesis, i.e. "(a,b,c,d,e,f)” or
>>>> “(a|b|c|d|e|f)", then the string is identified properly in both cases.
>>>>
>>>> Another thing that may be of significance is that if I add a syslog header
>>>> to each line of the file input, and specify needparse=“on” in the file
>>>> input definition, the issue goes away and the msg element is always
>>>> identified properly.
>>>>
>>>> This is v8.2102. I’m not sure if this is a bug or if I’m missing
>>>> something fundamental about the differences between input types, so any
>>>> advice or guidance is greatly appreciated.
>>>>
>>>> Thanks all!
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com/professional-services/
>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>>>> DON'T LIKE THAT.
>>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
