Hello,
I did some searching through the rsync archives at mail-archive.com
and didn't find anything less than 2 years old. I am hoping there have
been some developments since then, or that I can look towards debian for
clues to this problem.
I am trying to rsync some pretty large directories. These
directories have about 100-200 users in them. Sometimes we get this on a
single users home directory. Normally this works pretty well, but on
occasion we get repeatable errors:
# rsync -av --numeric-ids
...snip...
ERROR: out of memory in make_file [sender]
rsync: writefd_unbuffered failed to write 4 bytes [sender]: Broken pipe (32)
rsync: connection unexpectedly closed (1960976 bytes received so far)
[sender]
rsync error: error allocating core memory buffers (code 22) at io.c(548)
[sender=3.0.0cvs]
We're running this on a system which is in live use a file server. It is
copying from an NFS mount to an AOE block device. The current output is
it iterating through a Maildir. There is plenty of free memory available
on the server as far as I can tell:
bottom:~# free
total used free shared buffers cached
Mem: 3090232 3068536 21696 0 164 2289396
-/+ buffers/cache: 778976 2311256
Swap: 0 0 0
No limits:
bottom:~# ulimit
unlimited
Linux bottom 2.6.17.14-c1 #1 SMP Mon Nov 6 12:35:31 EST 2006 x86_64
GNU/Linux
It is Debian testing/unstable. We can reproduce this problem on the
packaged debian version (2.6.7) and the current CVS checkout
(2007-04-02, around 11am Pacific time.) There is another current rsync
process running on the server using 171m VIRT, 164m RES, and 728 SHR. It
is the sender and receiver.
I was wondering if anyone could shed some light on this? Is it still an
issue of "break it up into smaller hunks?" Any insight is appreciated. I
can provide any debugging information you need on the system, tuning,
kernel build information, etc, that you may need.
Thank you in advance,
Kelly
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
