On Wed, 12 Jun 2013, Roberto Giordani wrote:
Hello David,
your optimization is the right way, Ican use only 1 if statement for
client!!!
The last step is something like this that doesn't work for the reasons below
if $fromhost-ip == "10.10.1.7" then {
set $!path="PROD/APP";
set $!server="client1";
set $!owner="robert";
}
if $fromhost-ip == "10.10.1.8" then {
set $!path="PROD/APP";
set $!server="client2";
set $!owner="robert";
}
if $fromhost-ip == "10.10.2.10" then {
set $!path="PROD/WEB";
set $!server="client7";
set $!owner="tom";
}
if $fromhost-ip == "10.10.2.11" then {
set $!path="PROD/WEB";
set $!server="client8";
set $!owner="tom";
}
template(name="filename" type="string"
stringB="/rsyslog-data/%$!path%/%$!server%/%syslogtag%.log")
local6.debug action(type="omfile" DirOwner="%$!owner%" DirCreateMode="0750"
FileCreateMode="0444" DynaFile="filename" )
-----------------------------------------------------------------------
As you can see I need to use the OWNER variable to assign the right owner to
the output file.
Unfortunately:
1) I can't insert TEMPLATE into IF
Correct, but you should not need to, by defining the path variable and then
defining the template later it should work.
2) I can't use the OWNER variable into ACTION
This sounds like it's a bug, we'll need to have Rainer take a look at that.
You told me about output ruleset but reading documentation it seems it
doesn't useful for my purpose.
well, they would possibly be useful to short-circuit the long series of if
statements.
even without resolving the owner you could do:
template(name="filename" type="string"
string="/rsyslog-data/%$!path%/%$!server%/%syslogtag%.log")
if $fromhost-ip == "10.10.1.7" then {
set $!path="PROD/APP";
set $!server="client1";
local6.debug action(type="omfile" DirOwner="robert" DirCreateMode="0750"
FileCreateMode="0444" DynaFile="filename" )
stop
}
if $fromhost-ip == "10.10.1.8" then {
set $!path="PROD/APP";
set $!server="client2";
set $!owner="robert";
local6.debug action(type="omfile" DirOwner="robert" DirCreateMode="0750"
FileCreateMode="0444" DynaFile="filename" )
stop
}
if $fromhost-ip == "10.10.2.10" then {
set $!path="PROD/WEB";
set $!server="client7";
local6.debug action(type="omfile" DirOwner="tom" DirCreateMode="0750"
FileCreateMode="0444" DynaFile="filename" )
stop
}
if $fromhost-ip == "10.10.2.11" then {
set $!path="PROD/WEB";
set $!server="client8";
local6.debug action(type="omfile" DirOwner="tom" DirCreateMode="0750"
FileCreateMode="0444" DynaFile="filename" )
stop
}
you could replace the repeated local6.debug statements with call(ruleset_robert)
or call(ruleset_tom)
that would look cleaner, but not be any faster than this configuration.
This configuration will be a bit faster than your original in that logs from
client1 only go through one fromhost-ip test, the 'stop' causes rsyslog to skip
all the remaining tests.
David Lang
Resolving "owner" variable I can do a stress test.
Thanks.
Roberto.
On 06/11/2013 10:12 PM, David Lang wrote:
On Tue, 11 Jun 2013, David Lang wrote:
On Tue, 11 Jun 2013, Roberto Giordani wrote:
Hello David,
here there are the first actions for each client conf used by rsyslog
server
if $fromhost-ip == "10.10.1.7" and $syslogfacility-text == "local6"
and $syslogseverity-text == "debug" and $syslogtag ==
"PROD_server_01_Server" then action(type="omfile" DirOwner="robert"
DirCreateMode="0750" FileCreateMode="0444"
File="/rsyslog-data/client01/PROD_server_01_Server.log")
if $fromhost-ip == "10.10.1.7" and $syslogfacility-text == "local6"
and $syslogseverity-text == "debug" and $syslogtag ==
"PROD_SRV_server_01A_R" then action(type="omfile" DirOwner="robert"
DirCreateMode="0750" FileCreateMode="0444"
File="/rsyslog-data/client01/PROD_SRV_server_01_R.log")
if $fromhost-ip == "10.10.1.7" and $syslogfacility-text == "local6"
and $syslogseverity-text == "debug" and $syslogtag ==
"PROD_SRV_server_01B_R" then action(type="omfile" DirOwner="robert"
DirCreateMode="0750" FileCreateMode="0444"
File="/rsyslog-data/client01/PROD_SRV_server_01_R.log")
if $fromhost-ip == "10.10.1.7" and $syslogfacility-text == "local6"
and $syslogseverity-text == "debug" and $syslogtag ==
"PROD_SRV_server_01A_S" then action(type="omfile" DirOwner="robert"
DirCreateMode="0750" FileCreateMode="0444"
File="/rsyslog-data/client01/PROD_SRV_server_01_S.log")
if $fromhost-ip == "10.10.1.7" and $syslogfacility-text == "local6"
and $syslogseverity-text == "debug" and $syslogtag ==
"PROD_SRV_server_01B_S" then action(type="omfile" DirOwner="robert"
DirCreateMode="0750" FileCreateMode="0444"
File="/rsyslog-data/client01/PROD_SRV_server_01_S.log")
.
.
.
.
------------------------------------------
if $fromhost-ip == "10.10.1.8" and $syslogfacility-text == "local6"
and $syslogseverity-text == "debug" and $syslogtag ==
"PROD_server_02_Server" then action(type="omfile" DirOwner="robert"
DirCreateMode="0750" FileCreateMode="0444"
File="/rsyslog-data/client02/PROD_server_02_Server.log")
if $fromhost-ip == "10.10.1.8" and $syslogfacility-text == "local6"
and $syslogseverity-text == "debug" and $syslogtag ==
"PROD_SRV_server_02A_R" then action(type="omfile" DirOwner="robert"
DirCreateMode="0750" FileCreateMode="0444"
File="/rsyslog-data/client02/PROD_SRV_server_02_R.log")
if $fromhost-ip == "10.10.1.8" and $syslogfacility-text == "local6"
and $syslogseverity-text == "debug" and $syslogtag ==
"PROD_SRV_server_02B_R" then action(type="omfile" DirOwner="robert"
DirCreateMode="0750" FileCreateMode="0444"
File="/rsyslog-data/client02/PROD_SRV_server_02_R.log")
if $fromhost-ip == "10.10.1.8" and $syslogfacility-text == "local6"
and $syslogseverity-text == "debug" and $syslogtag ==
"PROD_SRV_server_02A_S" then action(type="omfile" DirOwner="robert"
DirCreateMode="0750" FileCreateMode="0444"
File="/rsyslog-data/client02/PROD_SRV_server_02_S.log")
if $fromhost-ip == "10.10.1.8" and $syslogfacility-text == "local6"
and $syslogseverity-text == "debug" and $syslogtag ==
"PROD_SRV_server_02B_S" then action(type="omfile" DirOwner="robert"
DirCreateMode="0750" FileCreateMode="0444"
File="/rsyslog-data/client02/PROD_SRV_server_02_S.log")
.
.
.
.
-----------------------------------------------
if $fromhost-ip == "10.10.1.30" and $syslogfacility-text == "local6"
and $syslogseverity-text == "debug" and $syslogtag ==
"PROD_server_23_Server" then action(type="omfile" DirOwner="robert"
DirCreateMode="0750" FileCreateMode="0444"
File="/rsyslog-data/client23/PROD_server_23_Server.log")
if $fromhost-ip == "10.10.1.30" and $syslogfacility-text == "local6"
and $syslogseverity-text == "debug" and $syslogtag ==
"PROD_SRV_server_23A_R" then action(type="omfile" DirOwner="robert"
DirCreateMode="0750" FileCreateMode="0444"
File="/rsyslog-data/client23/PROD_SRV_server_23_R.log")
if $fromhost-ip == "10.10.1.30" and $syslogfacility-text == "local6"
and $syslogseverity-text == "debug" and $syslogtag ==
"PROD_SRV_server_23B_R" then action(type="omfile" DirOwner="robert"
DirCreateMode="0750" FileCreateMode="0444"
File="/rsyslog-data/client23/PROD_SRV_server_23_R.log")
if $fromhost-ip == "10.10.1.30" and $syslogfacility-text == "local6"
and $syslogseverity-text == "debug" and $syslogtag ==
"PROD_SRV_server_23A_S" then action(type="omfile" DirOwner="robert"
DirCreateMode="0750" FileCreateMode="0444"
File="/rsyslog-data/client23/PROD_SRV_server_23_S.log")
if $fromhost-ip == "10.10.1.30" and $syslogfacility-text == "local6"
and $syslogseverity-text == "debug" and $syslogtag ==
"PROD_SRV_server_23B_S" then action(type="omfile" DirOwner="robert"
DirCreateMode="0750" FileCreateMode="0444"
File="/rsyslog-data/client23/PROD_SRV_server_23_S.log")
.
.
.
.
and so on to 480 actions
On your benchmark test which is the best ratio between #messages/s
and #actions?
it's not that there is a best ratio, it's just that there is overhead
to actions, overhead to if statements, etc
As you have it written, you have very complex conditions for each action
I think you would be better off re-writing your ruleset something
along these lines
if $fromhost='10.10.1.7' then set $!client='client01'
if $fromhost='10.10.1.8' then set $!client='client02'
if $fromhost='10.10.1.9' then set $!client='client03'
if $fromhost='10.10.1.10' then set $!client='client04'
.
.
.
template(name='filename' type="string"
stringB="/rsyslog-data/%$!client%/%syslogtag%.log"
local6.debug action(type="omfile" DirOwner="robert"
DirCreateMode="0750" FileCreateMode="0444" DynaFile="filename" )
now, this is not going to give the exact same results (your ruleset
won't log anything if the syslogtag is not one of the known ones,
while mine will), but if that's a problem, a second set of if
statements to set another variable would satisfy that.
It may end up making sense to create a ruleset "output" that contains
the action line and then change the if statements to
if $fromhost='10.10.1.7' then {
set $!client='client01'
call output
stop
}
this would short circuit the tests, but add the overhead of a ruleset
call. I don't know if 27 rules is enough to justify this.
What you really want is the table lookup function, but that hasn't
been funded yet :-(
it would reduce your config to
set $!client = lookup("clienttable", "%fromhost-ip")
set $!tag = lookup("tagtable", "%syslogtag")
template(name='filename' type="string"
stringB="/rsyslog-data/%$!client%/%$!tag%.log"
local6.debug action(type="omfile" DirOwner="robert"
DirCreateMode="0750" FileCreateMode="0444" DynaFile="filename" )
One other optimization that I would look at is to use arrays. this is
documented at http://www.rsyslog.com/tag/dynafile/
This solves the problem of odd syslogtag values.
As far as the names go, I would suggest that you consider doing some
name resolution games.
If you setup a special DNS server (or hosts file) to map the IPaddresses
to the client01 names that you want, allow rsyslog to do the DNS lookups
on the incoming logs (with TCP this shouldn't be too bad as rsyslog
should only have to do this once per connection), and then you could use
the template:
template(name='filename' type="string"
string="/rsyslog-data/%fromhost%/%syslogtag%.log"
and completely eliminate the if statements to define the client name.
DNS is something to avoid, but not at the cost of a ton of if statements
in your filter.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
