On Sat, Feb 12, 2005 at 07:10:55PM -0500, Alec Berryman <[EMAIL PROTECTED]>
wrote:
> Marc A. Lehmann on 2005-02-12 19:23:55 +0100:
>
> > Hmm, nothing in the man page claims it's copying that much into
> > memory, or that it needs that much memory. It does refer to "900MB
> > history buffer",
>
> For the history buffer to be effective,
BTW, the bug report should either (preferably) be closed or tagged wishlist.
> > not show that it tries to allocate that much memory, all it dos is
> > mmap the file, but the same could be done by read()ing it).
>
> As mentioned, the history buffer needs to be in memory; I don't see
> the advantage in read()ing it instead of mmap()ing it, since the end
> result is that up to 900MB of the file is in memory at the same time.
Well, the advantage would be that it would run :)
Anyway, I looked at the source, and what it does is roughly this:
- hash by linearly reading through the file, if a possible match
is found, look back and compare, if not, don't look back.
Anyways, it's clear that some work would be involved, so the only request I
have would be to document that (more clearly). There is a difference between
an algorithm that has an effective history buffer of 900MB and an algorithm
that needs 900MB of RAM to me :)
> It doesn't *really* need 900MB of RAM + swap, as I read the code; take
> a look at rzip.c:581 if you're so inclined. The minimum history
> buffer size is 100MB, and each additional level of compression adds
> another 100MB of memory. In your original example, you used `rzip
> -9`. I am under the impression that this is how bzip2 works.
Oh, I assumed that would be the bzip2 compression level. Again, this could be
mentioned in the manpage.
> The man page could be a little clearer. If you would like to submit a
> diff, that would be great; otherwise, I'll take care of it as time
> permits.
I'll put it on my todo list, dont' wait for me, though, I am pretty busy. But
if a diff arrives and you haven't patched it yet, feel free.
Thanks for the explanation and insights!
--
The choice of a
-----==- _GNU_
----==-- _ generation Marc Lehmann
---==---(_)__ __ ____ __ [EMAIL PROTECTED]
--==---/ / _ \/ // /\ \/ / http://schmorp.de/
-=====/_/_//_/\_,_/ /_/\_\ XX11-RIPE
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
