Re: Refreshing RAMDirectory

2007-12-13 Thread Michael McCandless
Ruslan Sivak wrote: Michael McCandless wrote: Ruslan Sivak wrote: Michael McCandless wrote: Ruslan Sivak wrote: Since my app would be multithreaded, there could be multiple threads accessing the reader, while i'm reloading it. This means that if I close the reader, and another thread

Re: Refreshing RAMDirectory

2007-12-13 Thread Ruslan Sivak
Michael McCandless wrote: Ruslan Sivak wrote: Michael McCandless wrote: Ruslan Sivak wrote: Since my app would be multithreaded, there could be multiple threads accessing the reader, while i'm reloading it. This means that if I close the reader, and another thread is using it, it might g

Re: Refreshing RAMDirectory

2007-12-13 Thread Michael McCandless
Ruslan Sivak wrote: Michael McCandless wrote: Ruslan Sivak wrote: Since my app would be multithreaded, there could be multiple threads accessing the reader, while i'm reloading it. This means that if I close the reader, and another thread is using it, it might get an exception. The n

Re: Refreshing RAMDirectory

2007-12-12 Thread Ruslan Sivak
Michael McCandless wrote: Ruslan Sivak wrote: This seems to be problematic though. There are other things that depend on the reader that is not so obvious. For example, IndexReader reader=getReader(); IndexSearcher searcher=new IndexSearcher(reader); Hits hits=searcher.search(query); searc

Re: Refreshing RAMDirectory

2007-12-12 Thread Michael McCandless
Ruslan Sivak wrote: This seems to be problematic though. There are other things that depend on the reader that is not so obvious. For example, IndexReader reader=getReader(); IndexSearcher searcher=new IndexSearcher(reader); Hits hits=searcher.search(query); searcher.close(); reader.close(

Re: Refreshing RAMDirectory

2007-12-12 Thread Ruslan Sivak
This seems to be problematic though. There are other things that depend on the reader that is not so obvious. For example, IndexReader reader=getReader(); IndexSearcher searcher=new IndexSearcher(reader); Hits hits=searcher.search(query); searcher.close(); reader.close(); Iterator i=hits.itera

Re: Refreshing RAMDirectory

2007-12-12 Thread Michael McCandless
You need to keep a reader open so long as you plan to use any of its methods from any thread. The reader does close exactly when you ask it to (when you call reader.close()). You should not have to "open a new reader for every method call" -- you only need to open a new reader (and in y

Re: Refreshing RAMDirectory

2007-12-12 Thread Ruslan Sivak
Thank you to everyone for your comments. I didn't realize that readers need to be kept open and won't close exactly when you ask them too. I have restructured my code to keep the RamDirectory cached, and to open a new reader for every method call. This seems to be working fine. Russ Erick

Re: Refreshing RAMDirectory

2007-12-12 Thread Erick Erickson
Even if you could tell a reader is closed, you'd wind up with unmaintainable code. I envision you have a bunch of places where you'd do something like if (reader.isClosed()) { reader = create a new reader. } But practically, you'd be opening a new reader someplace, closing it someplace else,

Re: Refreshing RAMDirectory

2007-12-12 Thread Michael McCandless
Ruslan Sivak wrote: Michael McCandless wrote: Ruslan Sivak wrote: I have an index of about 10mb. Since it's so small, I would like to keep it loaded in memory, and reload it about every minute or so, assuming that it has changed on disk. I have the following code, which works, except

Re: Refreshing RAMDirectory

2007-12-11 Thread Ruslan Sivak
Michael McCandless wrote: Ruslan Sivak wrote: I have an index of about 10mb. Since it's so small, I would like to keep it loaded in memory, and reload it about every minute or so, assuming that it has changed on disk. I have the following code, which works, except it doesn't reload the cha

Re: Refreshing RAMDirectory

2007-12-11 Thread Michael McCandless
Ruslan Sivak wrote: I have an index of about 10mb. Since it's so small, I would like to keep it loaded in memory, and reload it about every minute or so, assuming that it has changed on disk. I have the following code, which works, except it doesn't reload the changes. protected String

Re: Refreshing RAMDirectory

2007-12-11 Thread Ruslan Sivak
The on-disk index gets updated. Something like this: The second indexDoc function is what does the actual indexing, but this should have the relevant content. public void indexDoc(int userId) throws ClassNotFoundException, SQLException, CorruptIndexException, IOException { IndexWri

Re: Refreshing RAMDirectory

2007-12-11 Thread Erick Erickson
I can't speak to the errors, but how is the index being updated? An indexwriter buffers changes and periodically flushes them out to disk. So the writer may not have flushed your data, depending upon how it's written. Best Erick On Dec 11, 2007 5:37 PM, Ruslan Sivak <[EMAIL PROTECTED]> wrote: >

Refreshing RAMDirectory

2007-12-11 Thread Ruslan Sivak
I have an index of about 10mb. Since it's so small, I would like to keep it loaded in memory, and reload it about every minute or so, assuming that it has changed on disk. I have the following code, which works, except it doesn't reload the changes. protected String indexName; protected Ind