Re: [rsyslog] rsyslog mmextenal logstash json output with escaped quotations and additional quotations

[Lennon, Sean (UK) via rsyslog]

 


This email may contain proprietary information of BAE Systems and/or third 
parties.
 
Thanks for your response Rainer.  I don't think it answers my question, I have 
property fields from the Rsyslog message that are fine, they get formatted 
correctly, for example 'timereported' or 'syslogseverity-text'.  So, the output 
json for these and others are correct, it's the msg field that is returned from 
my custom code (using mmexternal) that is the problem.

I have created a newer template that is more upto date and looks something 
similar to this:

template(name="json-template" type="list" option.jsonf="on") {
        property(outname="@timestamp" name="timereported" dataformat="rfc3339" 
format="jsonf")
        property(outname="message" name="msg" format="jsonf")   
}

-----Original Message-----
From: Rainer Gerhards <rgerha...@hq.adiscon.com> 
Sent: 18 September 2023 15:26
To: rsyslog-users <rsyslog@lists.adiscon.com>
Cc: Lennon, Sean (UK) <sean.lenn...@baesystems.com>
Subject: Re: [rsyslog] rsyslog mmextenal logstash json output with escaped 
quotations and additional quotations

-----------------------------  PHISHING ALERT  ----------------------------- 
This email has been sent from an account outside of the BAE Systems network.

Please treat the email with caution, especially if you are requested to click 
on a link or open an attachment.
For further information on how to spot and report a phishing email please 
access the Global Intranet, then select <Functions> / <IT>.

------------------------------------------------------------------------------------

Does this example from the rsyslog testbench help?

https://github.com/rsyslog/rsyslog/blob/761cb2bc51e3046b242b45994cff11ff8be3990e/tests/json-nonstring.sh#L4

Rainer

El lun, 18 sept 2023 a las 15:10, Lennon, Sean (UK) via rsyslog
(<rsyslog@lists.adiscon.com>) escribió:
>
>
>
>
>
> This email may contain proprietary information of BAE Systems and/or third 
> parties.
>
> This is the one I meant.
>
> -----Original Message-----
> From: rsyslog <rsyslog-boun...@lists.adiscon.com> On Behalf Of Lennon, 
> Sean (UK) via rsyslog
> Sent: 29 August 2023 17:39
> To: rsyslog@lists.adiscon.com
> Cc: Lennon, Sean (UK) <sean.lenn...@baesystems.com>
> Subject: [rsyslog] rsyslog mmextenal logstash json output with escaped 
> quotations and additional quotations
>
> -----------------------------  PHISHING ALERT  ----------------------------- 
> This email has been sent from an account outside of the BAE Systems network.
>
> Please treat the email with caution, especially if you are requested to click 
> on a link or open an attachment.
> For further information on how to spot and report a phishing email please 
> access the Global Intranet, then select <Functions> / <IT>.
>
> ----------------------------------------------------------------------
> --------------
>
> This email may contain proprietary information of BAE Systems and/or third 
> parties.
>
> Hi all,
>
> I've encountered an issue with formatting json output to logstash.  I'm using 
> mmexternal to reformat data received from a remote system, the data is 
> project specific and needs to be massaged into json for use with logstash.  
> The intention is to create a json message for logstash with the mmexternal 
> output being part of that message.  I'm able to receive this json output at 
> logstash but the message field (which contains the mmexternal output) is 
> encapsulated within double quotes and all json fields within have escaped 
> double quotes.  This means that logstash is not able to interpret part of the 
> message.  If I take the raw output of the mmextenal code and send it to a 
> omfile then it looks perfectly fine.
>
> I have asked a more detailed question, on Stackoverflow: 
> https://stackoverflow.com/questions/77001549/rsyslog-mmextenal-logstas
> h-json-output-with-escaped-quotations-and-additional-qu
>
> What am I missing?
>
> I appreciate your help.
>
> Sean
>
> ********************************************************************
> This email and any attachments are confidential to the intended recipient and 
> may also be privileged. If you are not the intended recipient please delete 
> it from your system and notify the sender.
> You should not copy it or use it for any purpose nor disclose or distribute 
> its contents to any other person.
> ********************************************************************
>
> BAE Systems may process information about you that may be subject to 
> data protection laws. For more information about how we use your 
> personal information, how we protect your information, our legal basis 
> for using your information, your rights and who you can contact, 
> please refer to our Privacy Notice at www.baesystems.com/en/privacy 
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This 
> is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our 
> control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE 
> WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites 
> beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.