Not sure if this is the case, but you said "searchers", so might be it - you can (and should) reuse searchers for multiple/concurrent queries. IndexSearcher is thread-safe, so no need to have a different searcher for each query. Keep using this searcher until you decide to open a new searcher - actually until you 'warmed up' the new (updated) searcher, then switch to using the new searcher and close the old one.
- Doron Paul Waite <[EMAIL PROTECTED]> wrote on 18/10/2006 18:22:30: > Some excellent feedback guys - thanks heaps. > > > On my OOM issue, I think Hoss has nailed it here: > > > That said: if you are seeing OOM errors when you sort by a field (but > > not when you use the docId ordering, or sort by score) then it sounds > > like you are keeping refrences to IndexReaders arround after you've > > stoped using them -- the FieldCache is kept in a WeakHashMap keyed off of > > hte IndexReader, so it should get garbage collected as soon sa you let go > > of it. Another possibility is that you are sorting on too many fields > > for it to be able to build the FieldCache for all of them in the RAM you > > have available. > > > I'm using a piece of code written by Peter Halacsy which implements a > SearcherCache class. When we do a search we request a searcher, and this > class looks after giving us one. > > It checks whether the index has been updated since the most recent Searcher > was created. If so it creates a new one. > > At the same time it 'retires' outdated Searchers, once they have no queries > busy with them. > > Looking at that code, if the system gets busy indexing new stuff, and doing > complex searches this is all rather open-ended as to the potential number > of fresh Searchers being created, each with the overhead of building its > FieldCache for the first time. No wonder I'm having problems as the archive > has grown! Looking at it in this light, my OOM's all seem to come just > after a bout of articles have been indexed, and querying is being done > simultaneously, so it does fit. > > I guess a solution is probably to cap this process with a maximum number > of active Searchers, meaning potentially some queries might be fobbed off > with slightly out of date versions of the index, in extremis, but it would > right itself once everything settles down again. > > Obviously the index partitioning would probably make this a non-issue, but > it seems better to sort the basic problem out anyway, and make it 100% > stable. > > Thanks Hoss! > > Cheers, > Paul. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
