AS far as I can tell, you haven't provided any information about the user
defined variable fields
but apparently you know better than I do what information I need, so I won't
bother you further.
Good luck.
David Lang
On Tue, 19 Sep 2023, Lennon, Sean (UK) wrote:
David, thanks for your response but providing additional fields will not give
you additional information other than additional fields. I have provided the
form and fit of what is happening. Therefore, I believe what I have provided
is sufficient for this discussion.
-----Original Message-----
From: David Lang <da...@lang.hm>
Sent: 18 September 2023 20:29
To: Lennon, Sean (UK) via rsyslog <rsyslog@lists.adiscon.com>
Cc: Rainer Gerhards <rgerha...@hq.adiscon.com>; Lennon, Sean (UK)
<sean.lenn...@baesystems.com>
Subject: Re: [rsyslog] rsyslog mmextenal logstash json output with escaped
quotations and additional quotations
----------------------------- PHISHING ALERT -----------------------------
This email has been sent from an account outside of the BAE Systems network.
Please treat the email with caution, especially if you are requested to click
on a link or open an attachment.
For further information on how to spot and report a phishing email please access the Global
Intranet, then select <Functions> / <IT>.
------------------------------------------------------------------------------------
we need to see a lot more about what's created, your editing is hiding too much.
go ahead and mask out the contents, but we need to see all the values in the
debug output and their structure (i.e. any json significant characters), change
all the words/numbers to garbage if you want.
Another option would be to contact Adiscon and setup a professional services
contract so that you can have a NDA rather than having to share the content on
a public mailing list for the community to help.
David Lang
On Mon, 18 Sep 2023, Lennon, Sean (UK) via rsyslog wrote:
This email may contain proprietary information of BAE Systems and/or third
parties.
Sorry, but for ‘reasons’ I can only give you a severely edited version, I have
used debug output from mmexternal first and the received message from logstash
second:
1. mexternal debug output – I am satisfied with this.
{ “msg” :
{“messageGroup”:[{“field1”:1,”field2”:2},{“field1”:3,”field2”:4}]}}
2. what logstash receives
“message” => “{ \“msg\” :
{\“messageGroup\”:[{\“field1\”:1,\”field2\”:2},{\“field1\”:3,\”field2\”:4}]}}”
From: Rainer Gerhards <rgerha...@hq.adiscon.com>
Sent: 18 September 2023 15:47
To: Lennon, Sean (UK) <sean.lenn...@baesystems.com>
Cc: rsyslog-users <rsyslog@lists.adiscon.com>
Subject: Re: [rsyslog] rsyslog mmextenal logstash json output with
escaped quotations and additional quotations
PHISHING ALERT
This email has been sent from an account outside of the BAE Systems network.
Please treat the email with caution, especially if you are requested to click
on a link or open an attachment.
For further information on how to spot and report a phishing email please access the Global
Intranet then select <Functions> / <IT>.
If you think this is a phishing email, please report it by using the "Report
Phishing" button in Outlook.
Output the message with RSYSLOG_DebugFormat template. I need to see which data
msg actually has.
Rainer
Sent from phone, thus brief.
Lennon, Sean (UK)
<sean.lenn...@baesystems.com<mailto:sean.lenn...@baesystems.com>> schrieb am
Mo., 18. Sept. 2023, 16:41:
This email may contain proprietary information of BAE Systems and/or third
parties.
Thanks for your response Rainer. I don't think it answers my question, I have
property fields from the Rsyslog message that are fine, they get formatted
correctly, for example 'timereported' or 'syslogseverity-text'. So, the output
json for these and others are correct, it's the msg field that is returned from
my custom code (using mmexternal) that is the problem.
I have created a newer template that is more upto date and looks something
similar to this:
template(name="json-template" type="list" option.jsonf="on") {
property(outname="@timestamp" name="timereported" dataformat="rfc3339"
format="jsonf")
property(outname="message" name="msg" format="jsonf") }
-----Original Message-----
From: Rainer Gerhards
<rgerha...@hq.adiscon.com<mailto:rgerha...@hq.adiscon.com>>
Sent: 18 September 2023 15:26
To: rsyslog-users
<rsyslog@lists.adiscon.com<mailto:rsyslog@lists.adiscon.com>>
Cc: Lennon, Sean (UK)
<sean.lenn...@baesystems.com<mailto:sean.lenn...@baesystems.com>>
Subject: Re: [rsyslog] rsyslog mmextenal logstash json output with
escaped quotations and additional quotations
----------------------------- PHISHING ALERT -----------------------------
This email has been sent from an account outside of the BAE Systems network.
Please treat the email with caution, especially if you are requested to click
on a link or open an attachment.
For further information on how to spot and report a phishing email please access the Global
Intranet, then select <Functions> / <IT>.
----------------------------------------------------------------------
--------------
Does this example from the rsyslog testbench help?
https://github.com/rsyslog/rsyslog/blob/761cb2bc51e3046b242b45994cff11
ff8be3990e/tests/json-nonstring.sh#L4
Rainer
El lun, 18 sept 2023 a las 15:10, Lennon, Sean (UK) via rsyslog
(<rsyslog@lists.adiscon.com<mailto:rsyslog@lists.adiscon.com>>) escribió:
This email may contain proprietary information of BAE Systems and/or third
parties.
This is the one I meant.
-----Original Message-----
From: rsyslog
<rsyslog-boun...@lists.adiscon.com<mailto:rsyslog-bounces@lists.adisc
on.com>> On Behalf Of Lennon, Sean (UK) via rsyslog
Sent: 29 August 2023 17:39
To: rsyslog@lists.adiscon.com<mailto:rsyslog@lists.adiscon.com>
Cc: Lennon, Sean (UK)
<sean.lenn...@baesystems.com<mailto:sean.lenn...@baesystems.com>>
Subject: [rsyslog] rsyslog mmextenal logstash json output with
escaped quotations and additional quotations
----------------------------- PHISHING ALERT -----------------------------
This email has been sent from an account outside of the BAE Systems network.
Please treat the email with caution, especially if you are requested to click
on a link or open an attachment.
For further information on how to spot and report a phishing email please access the Global
Intranet, then select <Functions> / <IT>.
---------------------------------------------------------------------
-
--------------
This email may contain proprietary information of BAE Systems and/or third
parties.
Hi all,
I've encountered an issue with formatting json output to logstash. I'm using
mmexternal to reformat data received from a remote system, the data is project
specific and needs to be massaged into json for use with logstash. The
intention is to create a json message for logstash with the mmexternal output
being part of that message. I'm able to receive this json output at logstash
but the message field (which contains the mmexternal output) is encapsulated
within double quotes and all json fields within have escaped double quotes.
This means that logstash is not able to interpret part of the message. If I
take the raw output of the mmextenal code and send it to a omfile then it looks
perfectly fine.
I have asked a more detailed question, on Stackoverflow:
https://stackoverflow.com/questions/77001549/rsyslog-mmextenal-logsta
s h-json-output-with-escaped-quotations-and-additional-qu
What am I missing?
I appreciate your help.
Sean
********************************************************************
This email and any attachments are confidential to the intended recipient and
may also be privileged. If you are not the intended recipient please delete it
from your system and notify the sender.
You should not copy it or use it for any purpose nor disclose or distribute its
contents to any other person.
********************************************************************
BAE Systems may process information about you that may be subject to
data protection laws. For more information about how we use your
personal information, how we protect your information, our legal
basis for using your information, your rights and who you can
contact, please refer to our Privacy Notice at
www.baesystems.com/en/privacy<http://www.baesystems.com/en/privacy>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is
a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our
control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites
beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE
WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites
beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
