Re: NRT and Caching based on IndexReader

2010-05-17 Thread Shay Banon
Right, make sense. On Tue, May 18, 2010 at 4:23 AM, Yonik Seeley wrote: > On Mon, May 17, 2010 at 9:14 PM, Shay Banon wrote: > > Oh, and one more thing. Deleted docs is a sub case, with NRT, most people > > will almost always add docs as well... . So it is still not really usable > > for field c

Re: NRT and Caching based on IndexReader

2010-05-17 Thread Yonik Seeley
On Mon, May 17, 2010 at 9:14 PM, Shay Banon wrote: > Oh, and one more thing. Deleted docs is a sub case, with NRT, most people > will almost always add docs as well... . So it is still not really usable > for field cache, right? FieldCache should be fine for the general cases - the same entry wil

Re: NRT and Caching based on IndexReader

2010-05-17 Thread Yonik Seeley
On Mon, May 17, 2010 at 9:12 PM, Shay Banon wrote: > Just saw that you opened a case for that. I think that its important in your > test case to also test for object identity, not just equals. This is because > the IndexReader (or the FieldCacheKey) are used as keys in weak hash maps, > which uses

Re: NRT and Caching based on IndexReader

2010-05-17 Thread Shay Banon
Oh, and one more thing. Deleted docs is a sub case, with NRT, most people will almost always add docs as well... . So it is still not really usable for field cache, right? On Tue, May 18, 2010 at 4:12 AM, Shay Banon wrote: > Just saw that you opened a case for that. I think that its important in

Re: NRT and Caching based on IndexReader

2010-05-17 Thread Shay Banon
Just saw that you opened a case for that. I think that its important in your test case to also test for object identity, not just equals. This is because the IndexReader (or the FieldCacheKey) are used as keys in weak hash maps, which uses identity (==) equality for keys. If FieldCacheKey is suppo

Re: NRT and Caching based on IndexReader

2010-05-17 Thread Yonik Seeley
On Mon, May 17, 2010 at 9:00 PM, Shay Banon wrote: > Great, so I am not imagining things this late into the night ... ;), not so > great, since using NRT with field cache (like sorting) or caching filters, > or anything that caches based on IndexReader not really an option. This > makes NRT very p

Re: NRT and Caching based on IndexReader

2010-05-17 Thread Shay Banon
Great, so I am not imagining things this late into the night ... ;), not so great, since using NRT with field cache (like sorting) or caching filters, or anything that caches based on IndexReader not really an option. This makes NRT very problematic to use in a real application. -shay.banon On Tu

Re: NRT and Caching based on IndexReader

2010-05-17 Thread Yonik Seeley
Yep, confirmed what you are seeing. I'll check into it and open an issue. -Yonik http://www.lucidimagination.com On Mon, May 17, 2010 at 5:54 PM, Shay Banon wrote: > Yea, I noticed that ;). Even so, I think that with NRT, even the lower level > readers are cloned, meaning that you always get a

Re: NRT and Caching based on IndexReader

2010-05-17 Thread Shay Banon
Yea, I noticed that ;). Even so, I think that with NRT, even the lower level readers are cloned, meaning that you always get a new instance... . Here is a sample program that tests this behavior, am I doing something wrong? By the way, if what I say is correct, it affects field cache as well p

Re: NRT and Caching based on IndexReader

2010-05-17 Thread Yonik Seeley
On Mon, May 17, 2010 at 5:00 PM, Shay Banon wrote: >   I wanted to verify if my understanding is correct. Assuming that I use > NRT, and refresh, say, every 1 second, caching based on IndexReader, such is > what is used in the CachingWrapperFilter is basically useless No, it's fine. Searching in

NRT and Caching based on IndexReader

2010-05-17 Thread Shay Banon
Hi, I wanted to verify if my understanding is correct. Assuming that I use NRT, and refresh, say, every 1 second, caching based on IndexReader, such is what is used in the CachingWrapperFilter is basically useless, since, even if there is an open sub reader, it gets clones meaning there is a ne