RE: memory requirements was RE: out of memory in build_hash_table

2002-04-24 Thread David Bolen
Granzow, Doug (NCI) [[EMAIL PROTECTED]] writes: > From what I've observed by running top while rsync is running, its memory > usage appears to grow gradually, not exponentially. The exponential portion of the growth is up front when rsync gathers the file listing (it starts with room for 1000 fi

Re: memory requirements was RE: out of memory in build_hash_table

2002-04-24 Thread Eric Whiting
"Granzow, Doug (NCI)" wrote: > > From what I've observed by running top while rsync is running, its memory > usage appears to grow gradually, not exponentially. A rsync may take > several hours to complete. (I have one running now that started over four > hours ago. The filesystem contains 236

RE: memory requirements was RE: out of memory in build_hash_table

2002-04-24 Thread Granzow, Doug (NCI)
Well, for reference, here's what I'm running: The mirror is running Solaris 8, the source is running Solaris 7. Both are running the same version of rsync: rsync version 2.5.4 protocol version 26 Copyright (C) 1996-2002 by Andrew Tridgell and others Capabilities: 64-

RE: memory requirements was RE: out of memory in build_hash_table

2002-04-22 Thread David Bolen
Granzow, Doug (NCI) [[EMAIL PROTECTED]] writes: > Hmm... I have a filesystem that contains 3,098,119 files. That's > 3,098,119 * 56 bytes or 173,494,664 bytes (about 165 MB). Allowing > for the exponential resizing we end up with space for 4,096,000 > files * 56 bytes = 218 MB. But 'top' tells

memory requirements was RE: out of memory in build_hash_table

2002-04-22 Thread Granzow, Doug (NCI)
>The key growth factor for the file construction is going to be >per-file information that's about 56 bytes (last I looked) per file. >However, the kicker is that the file storage block gets resized >exponentially as you have more files. So for 385,000 files it'll >actually be a 512,000 file bloc

RE: out of memory in build_hash_table

2002-04-19 Thread David Bolen
Eric Echter [[EMAIL PROTECTED]] writes: > I also checked the /etc/security/limits.conf file and everything in > the file is commented out. Are there default limits if there are no > actual settings in this file that may be causing problems? Your > assumption about the memory limit on processes

Re: out of memory in build_hash_table

2002-04-19 Thread Eric Echter
gt; actually be a 512,000 file block of about 30MB. (So yeah, I suppose > an ~50 byte file chunk in memory growing as a power of 2 might average > out close to 100 bytes/file as an estimate :-)) > > > ERROR: out of memory in build_hash_table > > rsync error: error allocati

RE: out of memory in build_hash_table

2002-04-19 Thread David Bolen
00 file block of about 30MB. (So yeah, I suppose an ~50 byte file chunk in memory growing as a power of 2 might average out close to 100 bytes/file as an estimate :-)) > ERROR: out of memory in build_hash_table > rsync error: error allocating core memory buffers (code 22) at util.c(232) Se

out of memory in build_hash_table

2002-04-19 Thread Eric Echter
eceiving a couple of error messages (both refer to the same problem as far as I know): ERROR: out of memory in build_hash_table rsync error: error allocating core memory buffers (code 22) at util.c(232) The following are the commands being used from a bash script (not the whole script of course):