Forum: CFEngine Help
Subject: Reducing filesystem access
Author: sauer
Link to topic: https://cfengine.com/forum/read.php?3,24186,24186#msg-24186
So, I'm trying to better optimize my CFEngine policy, as it's getting to the
point where it takes a noticable chunk of time on the system upon which it's
running. The systems don't exist to run CFEngine, they exist to do something
else. :)
One of the things I'm noticing while trying to tune different things is that
one of my countlinesmatching variable assignments is being evaluated a lot more
than I'd expect. I have a common bundle (g) which contains this:
vars:
any::
"passwd_merge" string => "/etc/passwd.merge";
"passwd_merge_max" int => "1000";
agent&has_passwd_merge::
"passwd_merge_cnt" int => countlinesmatching( ".*", "$(passwd_merge)" );
classes:
any::
"has_passwd_merge" expression => fileexists( "$(g.passwd_merge)" );
has_passwd_merge::
"passwd_merge_ok" expression => islessthan( "$(passwd_merge_cnt)",
"$(passwd_merge_max)" );
For some background on why this part of the policy exists: The point is to
identify cases where that file has more than 1000 entries so it can skip
processing it in that case. There's a defect in cf-agent where it runs out of
hash buckets in the associative array code when I have more than around 1K
entries; it definitely dies at > 3K and is ok at < 1K. The agent class is in
there because having this in a common bundle also seemingly contributes to a
memory leak in both the monitor daemon and the server daemon.
I'd expect the file to be opened and counted either once or three times as
cf-agent iterates through the policy. Maybe six times if cf-promises feels
compelled to play too. When I run cf-agent in verbose mode, it prints each
matching line preceeded by countlinesmatching. Ultimately, when I grep
countlinesmatching out of the verbose output, it appears that the file is
having its contents counted 18 times. This bundle is only listed once in the
bundlesequence, and the variable is only referenced by the class within the
same bundle. Even if the islessthan usage was causing the file's contents to
be recalculated, I can't figure out how I'm getting to 18. Any insights as to
why the number of output lines might be exactly 18x the number of lines in the
file?
_______________________________________________
Help-cfengine mailing list
Help-cfengine@cfengine.org
https://cfengine.org/mailman/listinfo/help-cfengine
