RE: Poor performance "race condition" in FieldSortedHitQueue

2006-10-23 Thread Oliver Hutchison
Kalpesh, Are you using sorting? If you are, then the patch attached to LUCENE-651 may help. It fixes a race condition that exists in the initialization of the FieldCache (which is used to accelerate sorting). Cheers, Ollie > -Original Message- > From: kalpesh patel [mailto:[EMAIL PROTEC

RE: index architectures

2006-10-17 Thread Oliver Hutchison
I can certainly vouch for the benefits of partitioning, we've seen a very big improvement in searcher refresh times (our main pain point) since we implemented such an architecture. Our application has 1000's of indexes, ranging in size from a few meg up several gigabytes, updates occur very freque

RE: Poor performance "race condition" in FieldSortedHitQueue

2006-08-13 Thread Oliver Hutchison
I've created an issue (LUCENE-651) and attached a patch. Hopefully this will help you guys with whatever approach you end up using to solve this. Thanks, Oliver - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional command

RE: NPE when sorting on a field that is missing from a doc

2006-08-13 Thread Oliver Hutchison
> i havne't seen it mentioned before ... i'm guessing it is > specific to the "explicit Locale" String comparator. I've created an issue (LUCENE-650) with patch to fix this. Thanks, Oliver - To unsubscribe, e-mail: [EMAIL PR

NPE when sorting on a field that is missing from a doc

2006-08-10 Thread Oliver Hutchison
Hi all, we have recently noticed that doing a locale sensitive sort on a field that is missing from some docs causes an NPE inside the call to Collator#compare at FieldSortedHitQueue line 320 (Lucene 2.0 src): static ScoreDocComparator comparatorStringLocale (final IndexReader reader, final Stri

RE: Poor performance "race condition" in FieldSortedHitQueue

2006-08-10 Thread Oliver Hutchison
> [EMAIL PROTECTED] wrote on 09/08/2006 20:32:20: > > Heh... interfaces strike again. > > > > Well then since we *know* that no one has their own implementation > > (because they would not have been able to register it), we > should be > > able to safely upgrade the interface to a class (anyone

RE: Poor performance "race condition" in FieldSortedHitQueue

2006-08-09 Thread Oliver Hutchison
> Ah, right... I browsed your code a bit too fast. It looks fine. Great. > > On a related note it would be great if there was a way to plug a > > custom FieldCache implementation into Lucene, given there is a > > FieldCache interface it's a shame there's no way to > actually provide > > an

RE: Poor performance "race condition" in FieldSortedHitQueue

2006-08-09 Thread Oliver Hutchison
way to actually provide an alternative implementation. Since there seems to be a decent level of interest in this I'll try to put together a patch and raise an issue over the next few days. Cheers, Oliver > On 8/9/06, Oliver Hutchison <[EMAIL PROTECTED]> wrote: > > Otis, Doron, tha

RE: Poor performance "race condition" in FieldSortedHitQueue

2006-08-09 Thread Oliver Hutchison
Otis, Doron, thanks for the feedback. First up I'd just like to say that I totally agree with Doron on this - any attempt to fix this issue needs to be done using as fine grain synchronization as is possible or you'd just be introducing a new bottle neck. It terms of the level of granularity, t

RE: Poor performance "race condition" in FieldSortedHitQueue

2006-08-08 Thread Oliver Hutchison
> The nature of the field cache itself means that the first > sort on a particular field can take a long, long time. > Synchronization won't really help that much. I think you may be misunderstanding my description (probably because it was not clear enough :). The issue is not that the first se