I managed to do some quick writeup. It's far from perfect, but may be useful:
http://blog.gerhards.net/2013/06/rsyslog-performance-main-and-action.html If you have suggestions/questions, we can always extend or add more. Family is now asking me to go off the computer ;) Rainer On Sat, Jun 15, 2013 at 10:00 AM, Rainer Gerhards <[email protected]>wrote: > The effect on action queues is limited. Main q workers are the primary > tool. Explaination in blog later. > > Sent from phone, thus brief. > Am 15.06.2013 09:46 schrieb "David Lang" <[email protected]>: > > after he setup action queues it was only doing one thread per action >> queue, even after the maxthreads for the actionqueue was set to 64. >> >> David Lang >> >> On Sat, 15 Jun 2013, Rainer Gerhards wrote: >> >> Action or main q threads? >>> >>> Sent from phone, thus brief. >>> Am 15.06.2013 09:23 schrieb "David Lang" <[email protected]>: >>> >>> He had set the max threads to 64 and was still only seeing one thread >>>> working. >>>> >>>> In a few of the later messages in the thread, his actionqueue config >>>> parameters are up near the top of the message. >>>> >>>> David Lang >>>> >>>> On Sat, 15 Jun 2013, Rainer Gerhards wrote: >>>> >>>> Requires a longer reply, need to lookup some things myself first - >>>> that's >>>> >>>>> why I didn't throw in comments. >>>>> >>>>> But one very important thing: by default a queue has max 1 worker. This >>>>> needs to be increased if multiple ones are desired. I guess that was >>>>> the >>>>> main point missing. If you create an async action queue, you also get >>>>> one >>>>> more thread. Probably that's why it looks how it looks. I don't know >>>>> the >>>>> parameters out of my head. >>>>> >>>>> Sent from phone, thus brief. >>>>> Am 15.06.2013 03:40 schrieb "David Lang" <[email protected]>: >>>>> >>>>> clarifying some questions that have come up in discussions around >>>>> >>>>>> rulesets, thread, and queues >>>>>> >>>>>> When you tie a ruleset to a specific input, does that still use the >>>>>> same >>>>>> main queue? or will the logs from that input go to a different main >>>>>> queue >>>>>> (so the workers working on that queue will have no locking contention >>>>>> with >>>>>> workers working on logs from a different input), or do they get put >>>>>> on a >>>>>> single main queue. >>>>>> >>>>>> how do output threads and rulesets interact? >>>>>> >>>>>> from what was reported, it looks as if everything goes to one queue, >>>>>> and >>>>>> only if you set action queues per-ruleset will you end up with >>>>>> per-ruleset >>>>>> threads (in this case, actually per-actionqueue threads) >>>>>> >>>>>> Is omfile able to make use of multiple threads if you have a single >>>>>> action >>>>>> (in this case, using dynafiles) >>>>>> >>>>>> >>>>>> note, the documentation for omfile there is a bad entry >>>>>> http://www.rsyslog.com/doc/******omfile.html<http://www.rsyslog.com/doc/****omfile.html> >>>>>> <http://www.**rsyslog.com/doc/**omfile.html<http://www.rsyslog.com/doc/**omfile.html> >>>>>> > >>>>>> <http://www.**rsyslog.com/doc/**omfile.html<http://rsyslog.com/doc/omfile.html> >>>>>> <http://www.**rsyslog.com/doc/omfile.html<http://www.rsyslog.com/doc/omfile.html> >>>>>> > >>>>>> >>>>>>> >>>>>>> >>>>>> * DynaFileCacheSize (not mandatory, default will be used) >>>>>> Defines a template to be used for the output. >>>>>> >>>>>> >>>>>> I think this is the wrong description and it would be good to say what >>>>>> the >>>>>> default is. >>>>>> >>>>>> David Lang >>>>>> ______________________________******_________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/******mailman/listinfo/rsyslog<http://lists.adiscon.net/****mailman/listinfo/rsyslog> >>>>>> <http:**//lists.adiscon.net/**mailman/**listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>>> > >>>>>> <http:**//lists.adiscon.net/**mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/**listinfo/rsyslog> >>>>>> <htt**p://lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>>>> > >>>>>> >>>>>>> >>>>>>> >>>>>>> http://www.rsyslog.com/******professional-services/<http://www.rsyslog.com/****professional-services/> >>>>>> <http://**www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>>> > >>>>>> <http://**www.rsyslog.com/**professional-**services/<http://www.rsyslog.com/professional-**services/> >>>>>> <http:**//www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> >>>>>> > >>>>>> >>>>>>> >>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>> myriad >>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>>> DON'T LIKE THAT. >>>>>> >>>>>> ______________________________****_________________ >>>>>> >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>>> > >>>>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>>> > >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>> myriad >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>> DON'T LIKE THAT. >>>>> >>>>> ______________________________****_________________ >>>>> >>>> rsyslog mailing list >>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>> <http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>> > >>>> http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>> <http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>> > >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>> DON'T LIKE THAT. >>>> >>>> ______________________________**_________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>> http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> >>> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >> http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
