On Wednesday 07 January 2009 07:25:17 John Wang wrote: > Hi: > > The default buffer size (for docid,score etc) is 32 in TermScorer. > > We have a large index with some terms to have very dense doc sets. By > increasing the buffer size we see very dramatic performance improvements. > > With our index (may not be typical), here are some numbers with buffer > size w.r.t. performance in our query (a large OR query): > > Buffer-size improvement > 2042 - 22.0 % > 4084 - 39.1 % > 8172 - 51.1 % > > I understand this may not be suitable for every application, so do you > think it makes sense to make this buffer size configurable? >
Ideally the TermScorer buffer size could be set to a size depending on the query structure, but there is no facility for this yet. For OR queries larger buffers help, but not for AND queries. See also LUCENE-430 on reducing buffer sizes for the underlying TermDocs for very sparse doc sets. Regards, Paul Elschot
