Forum: Cfengine Help
Subject: Re: cf_lock.db gets bigger and bigger
Author: Lars.O
Link to topic: https://cfengine.com/forum/read.php?3,19875,20099#msg-20099
Thanks for the db_dump command, very useful :-)
I have now managed to isolate promises which causes cf_lock.db to grow
indefinitely. And as you say Mark, this is a case of promises that changes. I'm
pretty sure I've seen it also with non changing promises, but this is the first
one I found which survives a fresh restart. What I do to test is that I stop
cf-execd, delete /var/cfengine/outputs/* and /var/cfengine/state/*, reboot the
server, restart cf-execd and monitor cf_lock.db.
I use the following promises.cf and script which it calls (sorry for the long
entry):
#################### promises.cf #######################
##########################################################
body common control {
#bundlesequence => { "executor" };
bundlesequence => { "executor", "outputs" }; }
##########################################################
body executor control {
exec_command => "/var/cfengine/bin/cf-agent";
splaytime => "0";
schedule => { "Min00", "Min02", "Min04", "Min06", "Min08",
"Min10", "Min12", "Min14", "Min16", "Min18",
"Min20", "Min22", "Min24", "Min26", "Min28",
"Min30", "Min32", "Min34", "Min36", "Min38",
"Min40", "Min42", "Min44", "Min46", "Min48",
"Min50", "Min52", "Min54", "Min56", "Min58" }; }
##########################################################
bundle agent executor {
processes:
"/var/cfengine/bin/cf-execd"
comment => "Start cf-execd if no such process running (less
than 1 found).",
process_count => no_of_procs ("0", "0", "start_cfexecd" );
"/var/cfengine/bin/cf-execd"
comment => "Kill all cf-execd and restart one if more than
one running (more than 1 found).",
process_count => no_of_procs ("2", "99999", "kill_cfexecd" );
commands:
"/usr/bin/at -f /var/cfengine/CfengineData/HostInfo/dbout.sh now + 1
minute",
comment => "Keep track of database size.";
start_cfexecd::
"/var/cfengine/bin/cf-execd";
kill_cfexecd::
"/usr/bin/pkill -f /var/cfengine/bin/cf-execd";
"/var/cfengine/bin/cf-execd"; }
##########################################################
body process_count no_of_procs (no1, no2, class) {
match_range => irange ( "$(no1)", "$(no2)" );
in_range_define => { "$(class)" }; }
##########################################################
bundle agent outputs {
files:
"/var/cfengine/outputs/.*"
# action => immediate,
delete => tidy_1,
file_select => tidy_2; }
##########################################################
# body action immediate {
# ifelapsed => "0";
# action_policy => "fix"; }
##########################################################
body delete tidy_1 {
rmdirs => "false"; }
##########################################################
body file_select tidy_2 {
leaf_name => { "cf_.*" };
mtime => irange ( ago ( 0, 0, 0, 0, 10, 0 ), "9999999999" );
# mtime => irange ( ago ( 0, 0, 0, 0, 10, 0 ), now );
file_result => "leaf_name.!mtime"; }
#file_result => "leaf_name.(!mtime)"; }
#file_result => "leaf_name!mtime"; }
################## End of promises.cf ##########################
############# /var/cfengine/CfengineData/HostInfo/dbout.sh ##########
#!/bin/sh
# This script is used only for keeping track of changes in database size.
WhoAmI=`hostname | cut -d"." -f1`
LogFile=/var/cfengine/CfengineData/HostInfo/${WhoAmI}.dbout
Date=`date +"%Y-%m-%d %H:%M.%S"`
if [ ! -f ${LogFile} ]; then OldValue=0;
else
OldValue=`tail -1 ${LogFile} | awk '{print $3}'`
fi
if [ -x /var/cfengine/runtime/bdb/bin/db_dump ]; then
NewValue=`/var/cfengine/runtime/bdb/bin/db_dump -p
/var/cfengine/state/cf_lock.db | wc -l`
Comment="lines"
else
NewValue=`ls -l /var/cfengine/state/cf_lock.* | awk '{print $5}'`
Comment="bytes"
fi
if [ ${NewValue} -ne ${OldValue} ]; then
echo ${Date} ${NewValue} ${Comment} >> ${LogFile}
if [ "${Comment}" = "lines" ]; then
echo ${Date} >> ${LogFile}.contents
/var/cfengine/runtime/bdb/bin/db_dump -p /var/cfengine/state/cf_lock.db
>> ${LogFile}.contents
echo "" >> ${LogFile}.contents
fi
fi
############# End of /var/cfengine/CfengineData/HostInfo/dbout.sh
###################
After the first three runs, db_dump shows the following:
2011-01-13 10:33.01
VERSION=3
format=print
type=btree
db_pagesize=4096
HEADER=END
last.exec_apoptosis.processes.signals.process_select.process_owner.ifelapsed.process_count.-hplnx004.proc__var_cfengine_bin_cf_execd_norestart_3085_MD5=f791afa9801f41f79d0565eb9c16b53a\00
x\1e\00\00\83\c6.M
last.executor.commands.comment.-hplnx004._usr_bin_at__f__var_cfengine_CfengineData_HostInfo_dbout_sh_now___1_minute_3857_MD5=ec06d844689977c14ffdae8e478c66ae\00
\cf\1e\00\00\bf\c6.M
last.executor.processes.comment.process_count.match_range.in_range_define.-hplnx004.proc__var_cfengine_bin_cf_execd_norestart_3579_MD5=735ab2063a2672d0bfbae52e75e01b97\00
\cf\1e\00\00\bf\c6.M
last.executor.processes.comment.process_count.match_range.in_range_define.-hplnx004.proc__var_cfengine_bin_cf_execd_norestart_3579_MD5=db98d36c14cca69388c4ac6770479af7\00
\cf\1e\00\00\bf\c6.M
last.internal_bundle.exec_cfengine.handle.-hplnx004.the_executor_agent_6177_MD5=1c510be2939d47ba8c35625ffe9b3ced\00
x\1e\00\00\83\c6.M
last.outputs.files.delete.rmdirs.file_select.leaf_name.mtime.-hplnx004._var_cfengine_outputs_cf_hplnx004_mo_sw_ericsson_se__1294911167_DAY_MON_13_10_32_47_2011__6bcb90_3782_MD5=cde8ebf3cae689fe58c412b6f0efda1f\00
\cf\1e\00\00\c0\c6.M
lock.internal_bundle.exec_cfengine.handle.-hplnx004.the_executor_agent_6177_MD5=1c510be2939d47ba8c35625ffe9b3ced\00
x\1e\00\00\83\c6.M
DATA=END
2011-01-13 10:35.01
VERSION=3
format=print
type=btree
db_pagesize=4096
HEADER=END
last.exec_apoptosis.processes.signals.process_select.process_owner.ifelapsed.process_count.-hplnx004.proc__var_cfengine_bin_cf_execd_norestart_3085_MD5=f791afa9801f41f79d0565eb9c16b53a\00
x\1e\00\00\83\c6.M
last.executor.commands.comment.-hplnx004._usr_bin_at__f__var_cfengine_CfengineData_HostInfo_dbout_sh_now___1_minute_3857_MD5=ec06d844689977c14ffdae8e478c66ae\00
&\1f\00\007\c7.M
last.executor.processes.comment.process_count.match_range.in_range_define.-hplnx004.proc__var_cfengine_bin_cf_execd_norestart_3579_MD5=735ab2063a2672d0bfbae52e75e01b97\00
&\1f\00\007\c7.M
last.executor.processes.comment.process_count.match_range.in_range_define.-hplnx004.proc__var_cfengine_bin_cf_execd_norestart_3579_MD5=db98d36c14cca69388c4ac6770479af7\00
&\1f\00\007\c7.M
last.internal_bundle.exec_cfengine.handle.-hplnx004.the_executor_agent_6177_MD5=1c510be2939d47ba8c35625ffe9b3ced\00
x\1e\00\00\83\c6.M
last.outputs.files.delete.rmdirs.file_select.leaf_name.mtime.-hplnx004._var_cfengine_outputs_cf_hplnx004_mo_sw_ericsson_se__1294911167_DAY_MON_13_10_32_47_2011__6bcb90_3782_MD5=cde8ebf3cae689fe58c412b6f0efda1f\00
&\1f\00\008\c7.M
last.outputs.files.delete.rmdirs.file_select.leaf_name.mtime.-hplnx004._var_cfengine_outputs_cf_hplnx004_mo_sw_ericsson_se__1294911287_DAY_MON_13_10_34_47_2011__6bcb90_1490_MD5=01c969dfd00c8b0fd2670a3b42f0f504\00
&\1f\00\008\c7.M
lock.internal_bundle.exec_cfengine.handle.-hplnx004.the_executor_agent_6177_MD5=1c510be2939d47ba8c35625ffe9b3ced\00
x\1e\00\00\83\c6.M
DATA=END
2011-01-13 10:37.01
VERSION=3
format=print
type=btree
db_pagesize=4096
HEADER=END
last.exec_apoptosis.processes.signals.process_select.process_owner.ifelapsed.process_count.-hplnx004.proc__var_cfengine_bin_cf_execd_norestart_3085_MD5=f791afa9801f41f79d0565eb9c16b53a\00
x\1e\00\00\83\c6.M
last.executor.commands.comment.-hplnx004._usr_bin_at__f__var_cfengine_CfengineData_HostInfo_dbout_sh_now___1_minute_3857_MD5=ec06d844689977c14ffdae8e478c66ae\00
\81 \00\00\af\c7.M
last.executor.processes.comment.process_count.match_range.in_range_define.-hplnx004.proc__var_cfengine_bin_cf_execd_norestart_3579_MD5=735ab2063a2672d0bfbae52e75e01b97\00
\81 \00\00\af\c7.M
last.executor.processes.comment.process_count.match_range.in_range_define.-hplnx004.proc__var_cfengine_bin_cf_execd_norestart_3579_MD5=db98d36c14cca69388c4ac6770479af7\00
\81 \00\00\af\c7.M
last.internal_bundle.exec_cfengine.handle.-hplnx004.the_executor_agent_6177_MD5=1c510be2939d47ba8c35625ffe9b3ced\00
x\1e\00\00\83\c6.M
last.outputs.files.delete.rmdirs.file_select.leaf_name.mtime.-hplnx004._var_cfengine_outputs_cf_hplnx004_mo_sw_ericsson_se__1294911167_DAY_MON_13_10_32_47_2011__6bcb90_3782_MD5=cde8ebf3cae689fe58c412b6f0efda1f\00
\81 \00\00\b0\c7.M
last.outputs.files.delete.rmdirs.file_select.leaf_name.mtime.-hplnx004._var_cfengine_outputs_cf_hplnx004_mo_sw_ericsson_se__1294911287_DAY_MON_13_10_34_47_2011__6bcb90_1490_MD5=01c969dfd00c8b0fd2670a3b42f0f504\00
\81 \00\00\b0\c7.M
last.outputs.files.delete.rmdirs.file_select.leaf_name.mtime.-hplnx004._var_cfengine_outputs_cf_hplnx004_mo_sw_ericsson_se__1294911407_DAY_MON_13_10_36_47_2011__6bcb90_4699_MD5=5ffe358412f240bed22d2329fd7cb41c\00
\81 \00\00\b0\c7.M
lock.internal_bundle.exec_cfengine.handle.-hplnx004.the_executor_agent_6177_MD5=1c510be2939d47ba8c35625ffe9b3ced\00
x\1e\00\00\83\c6.M
DATA=END
As you can see there are two lines added for every new file in .../outputs
every time Cfengine runs. And these are never deleted.
Also long after ther files have been deleted, the entries in cf_lock.db are
still there.
_______________________________________________
Help-cfengine mailing list
Help-cfengine@cfengine.org
https://cfengine.org/mailman/listinfo/help-cfengine
