tags 548987 + upstream pending
thanks
On Wed, Sep 30, 2009 at 01:31:06PM +0200, Rune Kock wrote:
> On Wed, Sep 30, 2009 at 11:10, Olly Betts <[email protected]> wrote:
> > I think this is probably the cause of upstream #358:
> >
> > http://trac.xapian.org/ticket/358
>
> Yes, #358 is probably two bugs, this one and a memory leak.
> "evoisard" has omindex using 360 MB, which seems excessive, but
> shouldn't be a problem on his 1 GB machine.
Indexing large documents is fairly memory hungry, so I don't think this is
a leak, just the C++ STL hording memory, probably plus some heap fragmentation.
The reporter never responded to my request for further investigation so it's
hard to be totally sure, but there's only one place in omindex which explicitly
allocates memory dynamically, and that is only called once.
> >> Debian bug 404528 decribes a similar bug in another package, and
> >> suggests using _SC_PHYS_PAGES instead of _SC_AVPHYS_PAGES.
> >
> > _SC_PHYS_PAGES isn't ideal as other processes might be using that
> > memory, but _SC_AVPHYS_PAGES clearly isn't suitable, and since this is
> > mostly a catch for filters spiralling out of control, it looks like
> > _SC_AVPHYS_PAGES is probably the best option, perhaps with a lower ratio
> > than 7/8.
>
> I agree. Or maybe even just a fixed limit of, say, 50 MB.
Some document filters will need more than 50MB to extract a large document.
We really just want to ensure that an out of control filter doesn't cause
problems.
I've committed a fix to upstream SVN for this.
Cheers,
Olly
--
To UNSUBSCRIBE, email to [email protected]
with a subject of "unsubscribe". Trouble? Contact [email protected]
