Forum: CFEngine Help
Subject: Corruption or wrong type during copy
Author: lauwersw
Link to topic: https://cfengine.com/forum/read.php?3,23316,23316#msg-23316
Hi,
We're using cfengine over some VPN links to remote sites, located at almost the
other side of the world. Cfengine fits the job perfectly, but we sometimes face
problems during remote copies. The VPN's can be really bad at times (I've
struggled to get a file larger than 2 MB copied in one go at certain times of
the day), but in theory CfEngine shouldn't have any problem dealing with that,
which is one of the reasons we've selected it.
The situation has improved a bit after adding things like compare => "digest"
and verify => "true", but still it sometimes breaks rather critical files. This
is some sample code exhibiting the problem:
# Update from remote server
bundle agent update
{
files:
"$(sys.workdir)/inputs"
perms => u_p("600"),
copy_from => update_copy("$(g.masterfiles)","$(g.cfserver)"),
move_obstructions => "true",
depth_search => update_depth_search,
action => uimmediate;
}
body perms u_p(p)
{
mode => "$(p)";
}
body copy_from update_copy(from,server)
{
source => "$(from)";
servers => { "${server}" };
compare => "digest";
copy_backup => "false";
trustkey => "true";
verify => "true";
}
body depth_search update_depth_search
{
depth => "inf";
exclude_dirs => { "\.svn" };
}
body action uimmediate
{
ifelapsed => "1";
}
This is some output of one of the copied directories after a few weeks:
# ll
total 124
drwx------ 2 root root 4096 May 6 13:06 10.150.101.20
drwx------ 2 root root 4096 Jun 24 12:57 10.150.102.20
-rw------- 1 root root 0 Jul 29 13:58 10.150.102.20.cfnew
drwx------ 2 root root 4096 Aug 10 15:43 10.150.103.20
-rw------- 1 root root 45 Aug 4 08:32 10.150.103.20.cf-moved
drwx------ 2 root root 4096 Aug 29 18:56 10.150.104.20
-rw------- 1 root root 4 Jul 12 14:39 10.150.104.20.cf-moved
drwx------ 2 root root 4096 Aug 10 15:44 10.150.105.20
-rw------- 1 root root 3 Jul 12 12:58 10.150.105.20.cf-moved
drwx------ 2 root root 4096 Jun 24 13:00 10.150.106.20
drwx------ 2 root root 4096 Jun 24 12:59 10.150.107.20
drwx------ 2 root root 4096 Aug 25 14:40 10.150.194.20
drwx------ 2 root root 4096 Jun 9 16:32 10.150.195.20
drwx------ 2 root root 4096 Aug 12 18:52 10.150.196.20
-rw------- 1 root root 56 Aug 12 18:48 10.150.196.20.cf-moved
drwx------ 2 root root 4096 Apr 1 08:50 10.150.197.20
drwx------ 2 root root 4096 Sep 2 11:08 10.150.198.20
drwx------ 2 root root 4096 May 17 18:26
10.150.198.20_1314800430_Wed_Aug_31_16_22_33_2011_.cfsaved.dir
drwx------ 2 root root 4096 Aug 10 15:43 10.150.199.20
-rw------- 1 root root 8 Aug 3 18:58 10.150.199.20.cf-moved
drwx------ 2 root root 4096 Jul 1 12:41 10.50.101.21
drwx------ 2 root root 4096 Aug 12 11:42 10.50.101.23
drwx------ 2 root root 4096 Jun 17 22:29 10.50.102.21
lrwxrwxrwx 1 root root 74 Sep 2 11:02 10.50.102.22 -> ./ 2 493 0 500 100
4096 1314953771 1308916201 1308916680 0 8389013 2 64768
drwx------ 2 root root 4096 Jun 24 12:57
10.50.102.22_1314954052_Fri_Sep__2_11_02_29_2011_.cfsaved.dir
drwx------ 2 root root 4096 Aug 10 15:44 10.50.102.23
-rw------- 1 root root 0 Aug 3 18:51 10.50.102.23.cf-moved
drwx------ 2 root root 4096 Jun 17 22:30 10.50.103.21
drwx------ 2 root root 4096 Jun 24 12:58 10.50.103.22
drwx------ 2 root root 4096 Jun 17 22:29 10.50.103.23
drwx------ 2 root root 4096 Jun 24 12:57 10.50.103.24
drwx------ 2 root root 4096 Jun 17 22:30 10.50.103.25
drwx------ 3 root root 4096 Jun 21 07:08 10.50.111.20
# cat 10.150.103.20.cf-moved
t 75OK: 2 493 0 500 100 4096 1312439418 1#
# cat 10.150.104.20.cf-moved
t 75# cat 10.150.105.20.cf-moved
t 7# cat 10.150.196.20.cf-moved
t 75OK: 2 493 0 500 100 4096 1313167505 1308916201 1#
So most of the directories which end up as files are moved out of the way
automatically, but I'm really a bit worried about why they end up as files in
the first place. And in some cases you get a symlink pointing to nowhere, like
10.50.102.22. This prevents us really from using cfengine for many more tasks.
I added the contents of some of these directories-as-files above, maybe that
can be helpful in determining the cause of the problem. This example is from
just one server, the contents are very similar at other servers. And it really
must be related to the bad VPN links, because I've never seen any problems with
our local servers.
I'm willing to put some effort in collecting more detailed debug information,
if that would help.
_______________________________________________
Help-cfengine mailing list
Help-cfengine@cfengine.org
https://cfengine.org/mailman/listinfo/help-cfengine
