Re: In memory MultiSearcher

2007-05-22 Thread Erick Erickson
You're right, I am suggesting that you use the Lucene caching and see if it is adequate. Mind you, I have no clue whether your application will be well served by this or not, I've just seen too many examples of folks (includeing me) jumping into a solution to a problem that doesn't exist to be ab

Re: In memory MultiSearcher

2007-05-22 Thread Peter W.
Hoss, My Lucene scaling strategy involves creating numerous indexes, so I was looking for a way to read them in together for quickness. For those interested, your suggestion of using a single IndexSearcher on a MultiReader works well by itself. Or, you can still place in memory like this: Inde

Re: In memory MultiSearcher

2007-05-22 Thread Peter W .
Erick, Thanks for the reply, this is a web application. If you want to serve image files in a scalable fashion on the Internet you make Apache serve them from memory, not the filesystem. For databases, some sites use a distributed object memory caching system such as memcached. I was hoping th

Re: In memory MultiSearcher

2007-05-22 Thread Chris Hostetter
: I'd *strongly* recommend, if you haven't, just using the regular : FSDirectories rather than RAMDirectories and only getting : complex if that's too slow... ...and if you are "Multi Searching" over a bunch of local directories anyway, then use a single INdexSearcher on a MultiReader instead ...

Re: In memory MultiSearcher

2007-05-21 Thread Erick Erickson
that's too slow... I ask because I am searching FS-based indexes that are 4G with no problem. The index *was* 8G and had only a 10% performance hit. Best Erick On 5/21/07, Peter W. <[EMAIL PROTECTED]> wrote: Hello, I have been using a large, in memory MultiSearcher that is reac

In memory MultiSearcher

2007-05-21 Thread Peter W.
Hello, I have been using a large, in memory MultiSearcher that is reaching the limits of my hardware RAM with this code: try { IndexSearcher[] searcher_a= { new IndexSearcher(new RAMDirectory(index_one_path)), new IndexSearcher(new