Hi, a few comments on quickly looking at the code... It's sort of strange, inside the Weight.scorer() method, to go and build an IndexSearcher and run a whole new search, if the cache entry is missing. Could you instead just do a top-level search, which then populates the cache per-segment?
Also, FixedBitSet is better here than OpenBitSet: it should be a bit faster, because OpenBitSet is "elastic". When you use the core cache key, it does not change for a SegmentReader that was reopened with new deletes; this is the whole point of the core cache key. So, if deletes changed for a segment and you reopened, your cache entry will reuse whatever was created the first time for that segment. If deletes matter, then it's usually best to look at liveDocs "live" and fold them in, instead of regenerating the whole cache entry. Mike McCandless http://blog.mikemccandless.com On Thu, Nov 7, 2013 at 8:04 AM, Ravikumar Govindarajan <ravikumar.govindara...@gmail.com> wrote: > Thanks Mike. Can you help me out with one more question? > > I have a sample impl as below, where I am adding a ReaderClosedListener to > purge the BitSet. > > When using NRT with applyAllDeletes, old-reader will get closed and > new-reader will open. In such a case, will the below impl-cache also be > purged and re-built? > > I also saw that FieldCache uses a CoreClosedListener, instead of > ReaderClosedListener and I need such a functionality. It will be great to > maintain the BitSet cache at the cost of taking extra hit for testing > deletes. > > @Override > > public Scorer scorer(AtomicReaderContext context, boolean scoreDocsInOrder, > boolean topScorer, Bits acceptDocs) { > > Object key = context.getReader().getCoreCacheKey(); > > OpenBitSet bitSet = cacheMap.get(key); > > if (bitSet == null) { > > reader.addReaderClosedListener(new ReaderClosedListener() { > > @Override > > public void onClose(IndexReader reader) { > > Object key = reader.getCoreCacheKey(); > > cacheMap.remove(key); > > } > > }); > > final OpenBitSet bs = new OpenBitSet(reader.maxDoc()); > > //Add the empty bit-set first > > cacheMap.put(key, bs); > > IndexSearcher searcher = new IndexSearcher(reader); > > //Do a search and populate the bitset > > return bs; > > } > > //Proceed with scoring logic > > } > > -- > > Ravi > > > On Thu, Nov 7, 2013 at 4:28 PM, Michael McCandless < > luc...@mikemccandless.com> wrote: > >> You need to call .getCoreCacheKey() on each of the sub-readers >> (returned by IndexReader.leaves()), to play well with NRT. >> >> Typically you'd do so in a context that already sees each leaf, like a >> custom Filter or a Collector. >> >> Mike McCandless >> >> http://blog.mikemccandless.com >> >> >> On Thu, Nov 7, 2013 at 1:33 AM, Ravikumar Govindarajan >> <ravikumar.govindara...@gmail.com> wrote: >> > I am trying to cache a BitSet by attaching to >> IndexReader.addCloseListener, >> > using the getCoreCacheKey() >> > >> > But, I find that getCoreCacheKey() returns the IndexReader object itself >> as >> > the key. >> > >> > Whenever the IndexReader re-opens via NRT because of deletes, will it >> mean >> > that my cache will be purged, because a new IndexReader is opened? >> > >> > Are there ways to avoid this purging? >> > >> > -- >> > Ravi >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
