On Thu, Feb 12, 2004 at 10:28:31PM -0500, Ray Lischner wrote:
> On Thursday 12 February 2004 06:38 pm, jw schultz wrote:
>
> > You have run out of memory.
>
> According to the FAQ-o-matic, rsync needs about 100 bytes/file. I am
> copying about 1,000,000 files, so that comes out to roughly 100MB. The
> system has 192MB of physical RAM. Of course, the kernel and all the
> other daemons and what-not need memory, but there should still be
> enough for rsync.
>
> I was successful when copying fewer files at a time. When I copy 300,000
> files, rsync's RSS is 50MB after building the file list, and while it
> is copying, the RSS rises to about 75MB.
>
> I conclude that rsync actually needs as much as 250 bytes/file. This
> explains why 192MB is insufficient when I want to copy 1M files.
That 100 bytes/file is per file usage for the purpose of
scaling. Subject to variability according to actual
filename lengths and system wordsize, and does not account
for the additional 72bytes/file that were needed if -H was
in play.
Don't confuse per-file with flat overhead and data
requirements for one-file-at-a-time transfers that get
reused every file.
Try the patch and let us know. That should allow you to
function. CVS (which does not have that patch) requires
about 20 bytes less per-file.
--
________________________________________________________________
J.W. Schultz Pegasystems Technologies
email address: [EMAIL PROTECTED]
Remember Cernan and Schmitt
--
To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
