On Tue, Jul 16, 2013 at 11:17 PM, David Lang <[email protected]> wrote: > On Tue, 16 Jul 2013, Erik Steffl wrote: > > ok, great, will test the local tree ($.name) as soon as available, >> >> once it's avialable, any ideas how long before it gets into packages at >> http://ubuntu.adiscon.com/v7-**devel <http://ubuntu.adiscon.com/v7-devel> >> , >> > > basically the next release. I can't say when that will be. > > I am very hesitant to do a new devel release before my vacation. Especially with late additions there are chances that things break and my folks would really not like me to do that in front of a long vacation. So it's most probably end of August.
Rainer > > sidenote: I just noticed that http://www.rsyslog.com/ubuntu-** >> repository/ <http://www.rsyslog.com/ubuntu-repository/> lists Quantal as >> 13 but Quantal is 12.10 (http://releases.ubuntu.com/) >> >> Current release is Raring Ringtail 13.04. Given that quantal packages >> work on raring (at least according to our testing) it should be easy to add >> raring packages, can somebody add that to the repository? >> > > I think the idea is that if the Quantal packages work, just install those > rather than having the volunteers have to create another build environment > just to build raring packages. > > David Lang > > > > thanks! >> >> erik >> >> On 07/16/2013 01:33 PM, David Lang wrote: >> >>> Rainer will be on vacation out of the country for quite a while >>> (something like a full month), and then when you add the backup that >>> will develop, figure that he won't be able to do much more until >>> September or so. >>> >>> I think that parsing into subtrees is still a valuable thing to do, and >>> that it will probably be implemented eventually. But I also think that >>> these additional trees (local and global variables) is a very useful >>> thing to have. >>> >>> In any case, once it's in place, it's going to be there forever (due to >>> backwards compatibility), so it's safe to plan on this for the long term. >>> >>> David Lang >>> >>> On Tue, 16 Jul 2013, Erik Steffl wrote: >>> >>> sounds good, will test/use it when it's available, >>>> >>>> is this a short term or long term solution? I.e. did you give up on >>>> the parsing into a subtree? Would rather wait for long term solution >>>> (if it's coming in e.g. few weeks) cause I need some workaround right >>>> now anyway and I think whatever it is it will work for short term >>>> (think I will generate the templates with my custom info instead of >>>> using variables). >>>> >>>> erik >>>> >>>> On 07/16/2013 09:12 AM, Rainer Gerhards wrote: >>>> >>>>> Looks like adding "$.name!name!name" for "local variables" seems not >>>>> to be >>>>> overwhelmingly much work. I'll try to fit it in tomorow or Thursday (no >>>>> promise, though). >>>>> Rainer >>>>> >>>>> >>>>> On Tue, Jul 16, 2013 at 10:44 AM, Rainer Gerhards >>>>> <[email protected]>**wrote: >>>>> >>>>> just quickly: will see which of these options I can implement most >>>>>> quickly. Will try to look at it withint this week (but need to make >>>>>> sure >>>>>> the "necessary" things are done first). >>>>>> >>>>>> Rainer >>>>>> >>>>>> >>>>>> On Tue, Jul 16, 2013 at 9:57 AM, David Lang <[email protected]> wrote: >>>>>> >>>>>> On Mon, 15 Jul 2013, Erik Steffl wrote: >>>>>>> >>>>>>> the problem with that is that I have to set variables somewhere >>>>>>> (so >>>>>>> >>>>>>>> that I can use them no matter what, I have two templates but they >>>>>>>> need the >>>>>>>> same info). >>>>>>>> >>>>>>>> Only after that the message is parsed, so it's actually parsing >>>>>>>> of the >>>>>>>> message that would overwrite my info. >>>>>>>> >>>>>>>> >>>>>>> ahh, this is _exactly_ the trusted parameters problem that I >>>>>>> alluded to >>>>>>> earlier. I had suggested having a way to specify a 'protected' and >>>>>>> 'where >>>>>>> to move duplicates' tree when parsing. Rainer is talking about >>>>>>> having the >>>>>>> ability to set the root the the parsed tree to be something other >>>>>>> than $! >>>>>>> ($!parsed! for example, it would need to be configurable, >>>>>>> eventually we >>>>>>> will need the ability to take sub-elements and run them through a >>>>>>> parser to >>>>>>> create additional variables, think a JSON strin embedded in a >>>>>>> name=value >>>>>>> message) >>>>>>> >>>>>>> The proposal to have three sepearate roots >>>>>>> >>>>>>> 1. parsed/generic >>>>>>> >>>>>>> 2. explicitly local (will not show up in $! or all-json) >>>>>>> >>>>>>> 3. explicitly global (will stick around for future messages and not >>>>>>> show >>>>>>> up in $!) >>>>>>> >>>>>>> sounds like a good start. >>>>>>> >>>>>>> By default everything is in one namespace, but we know we need these >>>>>>> other two, and they are different enough to be worth considering >>>>>>> completely >>>>>>> separate. I do like the global filesystem view of things, but if we >>>>>>> do that >>>>>>> we really do need the ability to carve off subtrees (the equivalent >>>>>>> of -x >>>>>>> "don't cross device boundries" in find/tar) >>>>>>> >>>>>>> for most things, I think explicit filters or boundries is good, but >>>>>>> global really will be handled differently than normal variables, and >>>>>>> explcitly local variables are going to be exceptions in lots of >>>>>>> code. So it >>>>>>> seems that this may be sufficient justification to split them off. >>>>>>> >>>>>>> David Lang >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> I currently have two files, one is a simple file that only sets >>>>>>> the >>>>>>> >>>>>>>> variables (generated at deployment/installation time) and the >>>>>>>> other ones is >>>>>>>> a somewhat complex file (stored in git) that has the templates, >>>>>>>> sets up the >>>>>>>> actions, json parsing etc. >>>>>>>> >>>>>>>> file with variables /etc/rsyslog.d/30-my-vars.****conf: >>>>>>>> >>>>>>>> set $!cluster = "something"; >>>>>>>> >>>>>>>> the other file /etc/rsyslog.d/31-my.conf: >>>>>>>> >>>>>>>> template(name="json-parsed-ok" type="list") { >>>>>>>> ... standard header (pri, time, host) >>>>>>>> constant(value="@cee{\"****original-message-json\":\"") >>>>>>>> property(name="$!all-json") >>>>>>>> constant(value=",\"cluster\":\****"") >>>>>>>> property(name="$!cluster") >>>>>>>> constant(value="\"}\n") >>>>>>>> } >>>>>>>> >>>>>>>> template(name="json-parsing-****failed" type="list") { >>>>>>>> ... standard header (pri, time, host) >>>>>>>> constant(value="@cee{\"****original-message-txt\":\"") >>>>>>>> property(name="msg" format="json") >>>>>>>> constant(value=",\"cluster\":\****"") >>>>>>>> property(name="$!cluster") >>>>>>>> constant(value="\"}\n") >>>>>>>> } >>>>>>>> >>>>>>>> if prifilt("local0.*") then { >>>>>>>> action(type="mmjsonparse") >>>>>>>> if $parsesuccess == "OK" then { >>>>>>>> action(template="json-parsed-****ok" type="omfwd" ...) >>>>>>>> } else { >>>>>>>> action(template="json-parsing-****failed" type="omfwd" ...) >>>>>>>> } >>>>>>>> stop >>>>>>>> } >>>>>>>> >>>>>>>> The json-parsing-failed template works fine since $!cluster does >>>>>>>> not >>>>>>>> change property("msg"). The json-parsed-ok does nto work so well >>>>>>>> since >>>>>>>> $!cluster becomes part of json. I could almost use property("msg") >>>>>>>> in >>>>>>>> json-parsed-ok as well but it has @cee at the beginning. >>>>>>>> >>>>>>>> It seems that the only way out of this is to dynamically >>>>>>>> generate the >>>>>>>> templates themselves but I was trying to keep the generated part >>>>>>>> as simple >>>>>>>> as possible and the templates are not very readable even when typed >>>>>>>> directly. >>>>>>>> >>>>>>>> Or maybe use substring of property("msg")? >>>>>>>> >>>>>>>> Is there some other place to store the value of cluster name (in >>>>>>>> reality it's few more items but nothing complex, just strings) >>>>>>>> other than >>>>>>>> $!all-json? >>>>>>>> >>>>>>>> thanks! >>>>>>>> >>>>>>>> erik >>>>>>>> >>>>>>>> ______________________________****_________________ >>>>>>>> rsyslog mailing list >>>>>>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>>>>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>>>>>> > >>>>>>>> >>>>>>>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>>>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>>>>>> > >>>>>>>> >>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>> myriad >>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>>>> you >>>>>>>> DON'T LIKE THAT. >>>>>>>> >>>>>>>> ______________________________****_________________ >>>>>>>> >>>>>>> rsyslog mailing list >>>>>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>>>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>>>>> > >>>>>>> >>>>>>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>>>>> > >>>>>>> >>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>> myriad >>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>>> you >>>>>>> DON'T LIKE THAT. >>>>>>> >>>>>>> >>>>>> >>>>>> ______________________________**_________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>>> http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >>>>> POST if you DON'T LIKE THAT. >>>>> >>>>> >>>> ______________________________**_________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>> http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>>> if you DON'T LIKE THAT. >>>> >>>> ______________________________**_________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>> http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> >> >> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >> http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> >> ______________________________**_________________ > rsyslog mailing list > http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> > http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
