You're right, I am suggesting that you use the Lucene caching and see if it is adequate.
Mind you, I have no clue whether your application will be well served by this or not, I've just seen too many examples of folks (includeing me) jumping into a solution to a problem that doesn't exist to be able to refrain from asking "Do you *know* you need to get fancy?" <G>... FWIW, a couple of things to watch out for. The searchers need to process a few queries before the caches are built up, so if you are doing load testing, be aware that the first few queries aren't representative. Second, you'll probably need to build at least a simple load tester (actually, I'm probably sure some exist off the shelf, but we've been doing things in-house so far) to fire off a bunch of threads that make requests and measure response. I'd hate for my off-the-cuff advice to cause your app to tip over the first time you opened it up to the public <G>.... Best Erick On 5/22/07, Peter W. <[EMAIL PROTECTED]> wrote:
Erick, Thanks for the reply, this is a web application. If you want to serve image files in a scalable fashion on the Internet you make Apache serve them from memory, not the filesystem. For databases, some sites use a distributed object memory caching system such as memcached. I was hoping the idea translates to Lucene and was trying to overcome reading multiple indexes from attached disks into main memory on one machine first, then across a balanced farm. I think what you are saying is use FSDirectories and the regular built-in Lucene caching provided instead. Let's give that a try in the servlet init() method! Regards, Peter W. On May 21, 2007, at 2:46 PM, Erick Erickson wrote: > Why are you doing this in the first place? Do you actually have > evidence that the default Lucene behavior (caching, etc) is inadequate > for your needs? > > I'd *strongly* recommend, if you haven't, just using the regular > FSDirectories rather than RAMDirectories and only getting > complex if that's too slow... > > I ask because I am searching FS-based indexes that are 4G with > no problem. The index *was* 8G and had only a 10% performance hit. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
