Re: LUCENE-831 (complete cache overhaul) -> mem use

2008-11-15 Thread Britske
Thanks Mark , this gets me moving. I will look into it somewhere soon. Geert-Jan markrmiller wrote: > > Like I said, its pretty easy to add this, but its also going to suck. > Kind of exposes the fact that its missing the right extensibility at the > moment. Things are still a bit ugly ov

Re: LUCENE-831 (complete cache overhaul) -> mem use

2008-11-15 Thread Mark Miller
Like I said, its pretty easy to add this, but its also going to suck. Kind of exposes the fact that its missing the right extensibility at the moment. Things are still a bit ugly overall. Your going to need new CacheKeys for the data types you want to support. A CacheKey builds and provides a

Re: LUCENE-831 (complete cache overhaul) -> mem use

2008-11-15 Thread Britske
That ArrayObject suggestion makes sense to me. It amost seemed to be as if you were referring as this option (or at least the interfaces needed to implement this) were already available as 1 out of 2 options available in 831? Could you give me a hint at were I have to be looking to extend what y

Re: LUCENE-831 (complete cache overhaul) -> mem use

2008-11-15 Thread Britske
But isn't this pluggability / visibility issues resolved with using lucene-831? Or did you roll-your-own before 831 was available? PabloS wrote: > > I have the same problem with cache and too many sorted fields, and had to > implement a big workaround to be able to plug my own cache implementa

Re: LUCENE-831 (complete cache overhaul) -> mem use

2008-11-14 Thread Mark Miller
Its hard to predict the future of LUCENE-831. I would bet that it will end up in Lucene at some point in one form or another, but its hard to say if that form will be whats in the available patches (I'm a contrib committer so I won't have any real say in that, so take that prediction with a gra

Re: LUCENE-831 (complete cache overhaul) -> mem use

2008-11-14 Thread Pablo Saavedra
I have the same problem with cache and too many sorted fields, and had to implement a big workaround to be able to plug my own cache implementation in lucene 2.3.2. What I'd really like to see in the new cache implementation is easier pluggability and extension of the lucene classes, which is curre

LUCENE-831 (complete cache overhaul) -> mem use

2008-11-14 Thread Britske
Hi, I recently saw activity on LUCENE-831 (Complete overhaul of FieldCache API/Implementation) which I have interest in. I posted previously on this with my concern that given the current default cache I sometimes get OOM-errors because I have a lot of fields which are sorted on, which ultimate