The needParse option for imfile is how you tell rsyslog to attempt to read syslog header elements out of the imfile content. If it is not used, then everything read from the file is in the “msg” property.
Regards, > On Mar 1, 2023, at 07:23, Mariusz Kruk via rsyslog > <rsyslog@lists.adiscon.com> wrote: > > There is also an option for imfile called needParse > https://www.rsyslog.com/doc/v8-stable/configuration/modules/imfile.html > > Might work, might not - never used it myself. But always worth giving it a > try. > > On 1.03.2023 14:21, David Lang via rsyslog wrote: >> that's what I'm guessing, but you should check what you are getting and >> attempt to turn on the feature to have imfile parse the lines from imfile as >> if they were sent as syslog messages. I say that I don't think '*' is >> allowed, because I think that it's forbidden by the RFC, so rsyslog has to >> guess what to do and I expect that it puts it in the msg field. >> >> you will see this when you log with debugformat. >> >> depending on how successful rsyslog is in parsing the log as a syslog >> message, it's possible that *queries* will be the very beginning of the msg >> field, which will make it very easy to detect and parse., if not, it's more >> work. >> >> mmnormalize is a different beast than regex extraction that grok does, but >> it if FAR more efficient, so worth the time to learn. >> >> If you can fix the thing that's writing these messaes to not put illegal >> characters in them, it will be easier. >> >> David Lang >> >> On Wed, 1 Mar 2023, Tan Mientras via rsyslog wrote: >> >>> After testing what you said, it doesn't seem to exist a property which >>> returns "queries", and I'll be only able to parse it using something like >>> grok. >>> Did I understood right? >>> >>> On Wed, Mar 1, 2023 at 1:55 PM Mariusz Kruk via rsyslog < >>> rsyslog@lists.adiscon.com> wrote: >>> >>>> As my colleague used to say - try and see. Define logging action with >>>> RSYSLOG_DebugFormat template and see what your properties are.* >>>> * >>>> >>>> On 1.03.2023 13:50, Tan Mientras via rsyslog wrote: >>>>> Im not sure I understood properly. >>>>> imfile has a mandatory tag required. but apart from that, the line >>>> contains >>>>> a "static" string "*queries*" >>>>> >>>>> Which *property* would be "*queries*" when processing the line...or is it >>>>> impossible? >>>>> >>>>> 01-Mar-2023 13:20:23.998 *queries*: info: client @0x7fb258b56d80 >>>>> 30.0.30.142#59640 >>>>> (e8333.g.akamaiedge.net): view internal-view: query: >>>> e8333.g.akamaiedge.net IN >>>>> A +E(0)D (192.168.2.254) >>>>> >>>>> On Wed, Mar 1, 2023 at 1:35 PM Mariusz Kruk via rsyslog < >>>>> rsyslog@lists.adiscon.com> wrote: >>>>> >>>>>> You're explicitly telling your imfile to apply the *dns-query* tag. I'd >>>>>> say that this behaviour is expected. $programname is the "static" part >>>>>> of tag. The tag is *dns-query*. So... >>>>>> >>>>>> On 1.03.2023 13:25, Tan Mientras via rsyslog wrote: >>>>>>> Hi. >>>>>>> >>>>>>> Which *property* would be "*queries*" when processing the following >>>> line? >>>>>>> >>>>>>> 01-Mar-2023 13:20:23.998 *queries*: info: client @0x7fb258b56d80 >>>>>>> 30.0.30.142#59640 (e8333.g.akamaiedge.net): view internal-view: query: >>>>>>> e8333.g.akamaiedge.net IN A +E(0)D (192.168.2.254) >>>>>>> >>>>>>> AFAIK, *programname*, but with the following configuration it returns >>>>>>> *dns-query* :( >>>>>>> >>>>>>> module(load="imfile") >>>>>>> template(name="json" type="list" option.json="on") { >>>>>>> constant(value="{") >>>>>>> constant(value="\"@source_timestamp\":\"") >>>>>>> property(name="timereported" dateFormat="rfc3339") >>>>>>> constant(value="\",\"source_message\":\"") >>>>>>> property(name="msg") >>>>>>> constant(value="\",\"source_hostname\":\"") >>>>>>> property(name="hostname") >>>>>>> constant(value="\",\"source_severity\":\"") >>>>>>> property(name="syslogseverity-text") >>>>>>> constant(value="\",\"source_facility\":\"") >>>>>>> property(name="syslogfacility-text") >>>>>>> constant(value="\",\"source_tag\":\"") >>>>>>> property(name="syslogtag") >>>>>>> constant(value="\",\"source_app\":\"") >>>>>>> property(name="*programname*") >>>>>>> constant(value="\",\"source_filename\":\"") >>>>>>> property(name="$.filename") >>>>>>> constant(value="\"}\n") >>>>>>> } >>>>>>> input(type="imfile" file="/var/log/bind/DNSquery.log" addMetadata="on" >>>>>> tag=" >>>>>>> *dns-query*" ruleset="syslog") >>>>>>> ruleset(name="syslog") { >>>>>>> set $.filename = $!metadata!filename; >>>>>>> action(type="omfwd" target="myserver" port="514" >>>> protocol="udp" >>>>>>> template="json") >>>>>>> } >>>>>>> >>>>>>> Thanks a lot for your help >>>>>>> _______________________________________________ >>>>>>> rsyslog mailing list >>>>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>> What's up with rsyslog? Followhttps://twitter.com/rgerhards >>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>> myriad >>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>>> DON'T LIKE THAT. >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> http://www.rsyslog.com/professional-services/ >>>>>> What's up with rsyslog? Followhttps://twitter.com/rgerhards >>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>>> DON'T LIKE THAT. >>>>>> >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> https://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com/professional-services/ >>>>> What's up with rsyslog? Followhttps://twitter.com/rgerhards >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>> DON'T LIKE THAT. >>>> _______________________________________________ >>>> rsyslog mailing list >>>> https://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com/professional-services/ >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>> DON'T LIKE THAT. >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> https://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of >>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >>> LIKE THAT. >>> >> _______________________________________________ >> rsyslog mailing list >> https://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of >> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T >> LIKE THAT. > _______________________________________________ > rsyslog mailing list > https://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
