On 07/18/2013 12:36 AM, Rainer Gerhards wrote:
On Thu, Jul 18, 2013 at 9:34 AM, Erik Steffl <[email protected]> wrote:
On 07/17/2013 10:01 PM, Rainer Gerhards wrote:
On Thu, Jul 18, 2013 at 2:43 AM, Erik Steffl <[email protected]> wrote:
On 07/16/2013 09:28 PM, David Lang wrote:
take a look at
http://www.rsyslog.com/doc/****node32.html<http://www.rsyslog.com/doc/**node32.html>
<http://www.**rsyslog.com/doc/node32.html<http://www.rsyslog.com/doc/node32.html>
**and see if it
answers your questions.
since the queue is specified inside action does that mean that each
action needs its own queue?
Yes, and thanks to the new config format this is now clear ;)
E.g. I have two actions that forward messages to remote log aggregator
and
would like them to be assisted by the same disk queue (so that I can
specify one queue.maxdiskspace). Is that possible?
No ... more below
action(
type="omfwd"
template="some template"
queue.filename="/var/log/****myFile"
queue.maxdiskspace="****4294967296"
...
)
...
action(
type="omfwd"
template="a different template"
queue.filename="/var/log/****myFile"
queue.maxdiskspace="****4294967296"
...
)
Would this create one disk based queue with filename /var/log/myFile
that both actions would use or would this just mess things up?
Old version messed things ab, current version rename the second queue file
and emit a warning message. So you'll end up with two seperate queues in
any case.
Side-note: I guess you used omfile just as example, but one never news who
will dig this up via google: it is absolutely counter-productive to use
disk queus with files, as writing the actual output files is much faster
(10 to 20 times) than writing a disk queue. So don't do that. Disk queues
are for potentially slow outputs (databases, remote systems...).
I am a bit confused, I use omfwd in the examples above (left out the
rest of omfwd args for brevity), just added the queue.filename and
queue.maxdiskspace to make sure I understand it corerectly. Do my examples
look correct?
My fault. I just saw /var/log and my mind switched to omfile, ignoring the
facts. Sorry for the confusion. Yes, this sounds right (except that most
folks usually do not put spool files in /var/log, what probably is a good
idea).
is it more common to put them to /var/spool/rsyslog, i.e. were you
expecting queue.filename="/var/spool/rsyslog/myFile"? I see that's the
workdir in default /etc/rsyslog.conf so wasn't sure if that's for
rsyslog internal stuff only or if it's a good idea to put disk queues there,
thanks,
erik
Rainer
and yes, we plan to use the disk based queues so that the messages are
not lost if the remote log aggregator goes down or there is a network
problem or something like that...
Alternatively is there a way to specify total limit for all disk based
queues? (didn't find anything so I guess not but hope never dies :)
No, that would required an immense amount of synchronization with the
current implementation, which makes this infeasable.
Background info: the current queue system is stricly sequential. To do
what
you ask for would require a totally different subsystem, something along
the lines of the OS page file system. Actually, I think such a thing to
have would be a big plus. However, it is a very major task to do, I'd say
3+ month of work. I would need external funding /sponsorship for such a
big
task (at least a larger part of it).
A work-around may be to put things into their own ruleset and create a
ruleset queue. However, this means if one of the actions block, the other
are not executed until that block is solved. May be acceptable based on
the
use case.
Sorry I have no better answer.
ok, I'll just make sure there is enough space for all the queues (only
have two so not a huge deal).
thanks!
erik
Rainer
thanks!
erik
David Lang
On Tue, 16 Jul 2013, Erik Steffl wrote:
Action definition:
action(
type="omfwd"
target="somehost.com"
port="5140"
protocol="tcp"
template="json"
)
How to make this action use disk assisted queue? I did read
http://www.rsyslog.com/doc/****omfwd.html<http://www.rsyslog.com/doc/**omfwd.html>
<http://www.rsyslog.**com/doc/omfwd.html<http://www.rsyslog.com/doc/omfwd.html>
and
http://www.rsyslog.com/doc/****queues.html<http://www.rsyslog.com/doc/**queues.html>
<http://www.**rsyslog.com/doc/queues.html<http://www.rsyslog.com/doc/queues.html>
Section 'Disk-Assisted Memory Queues' says 'If a disk queue name is
defined for in-memory queues (via $<object>QueueFileName), they
automatically become "disk-assisted" (DA)'. However I also see at
http://www.rsyslog.com/doc/****rsyslog_conf_actions.html<http://www.rsyslog.com/doc/**rsyslog_conf_actions.html>
<http**://www.rsyslog.com/doc/**rsyslog_conf_actions.html<http://www.rsyslog.com/doc/rsyslog_conf_actions.html>>that
legacy
format does not affect RainerScript actions (which is what I use).
Or is it going to automatically store it in a directory specified by
this: $WorkDirectory /var/spool/rsyslog and I don't have to worry
about it?
thanks!
erik
______________________________****_________________
rsyslog mailing list
http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog>
<http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>
http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/>
<http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/>
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
if you DON'T LIKE THAT.
______________________________****_________________
rsyslog mailing list
http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog>
<http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>
http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/>
<http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/>
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
______________________________****_________________
rsyslog mailing list
http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog>
<http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>
http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/>
<http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/>
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
______________________________**_________________
rsyslog mailing list
http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>
http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/>
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
______________________________**_________________
rsyslog mailing list
http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>
http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/>
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
