Re: FileNotFoundException during indexing

2008-08-03 Thread Michael McCandless
Wojtek212 wrote: You were right I had 2 IndexWriters. I've checked again and it turned out I had 2 IndexManagers loaded by 2 different classloaders, so even if stored it in static Map, it didn't help. Phew! That's tricky (two different classloaders). Good sleuthing Anyway thanks for

Re: FileNotFoundException during indexing

2008-08-03 Thread Wojtek212
Hi Mike, You were right I had 2 IndexWriters. I've checked again and it turned out I had 2 IndexManagers loaded by 2 different classloaders, so even if stored it in static Map, it didn't help. Anyway thanks for help. But I have last question. Is it correct if I use IndexSearcher during wrking In

Re: FileNotFoundException during indexing

2008-08-01 Thread Michael McCandless
From this log I can see you do in fact have two IndexWriters open at the same time (see how IW 6 and IW 42 have intermingled log lines right before the exception). Are you sure you're not still unlocking the index? Without unlocking the index, and if you're using either Simple or NativeF

Re: FileNotFoundException during indexing

2008-08-01 Thread Wojtek212
Here is Lucene log with first exceptions that occured (FSDirectory with NativeFSLockFactory). IFD [Thread-79]: setInfoStream [EMAIL PROTECTED] IW 4 [Thread-79]: setInfoStream: dir=org.apache.lucene.store.FSDirectory@/tmp/content/3615.0-3618.0 autoCommit=true [EMAIL PROTECTED] [EMAIL PROTECTED] ra

Re: FileNotFoundException during indexing

2008-08-01 Thread Wojtek212
The strange thing is that when I use only FSDirectory with SimpleFSLockFactory I don't see any exception (or I couldn't reproduce the problem). FSDirectory with NativeFSLockFactory doesn't work as well as my implementation of Directory and Lock (based on java.nio). Hmmm, I don't see the reason of

Re: FileNotFoundException during indexing

2008-08-01 Thread Michael McCandless
Hmmm OK. I would stick with the NativeFSLockFactory, and never call IndexReader.unlock. Can you call IndexWriter.setInfoStream, and then post the resulting log? It may provide clues of what's happening. Also, if you can narrow this to a small test case that shows the exception, that'd

Re: FileNotFoundException during indexing

2008-08-01 Thread Wojtek212
I've checked unlock ant it is not called until exception occurs. BTW, I' ve tried to use FSDirectorectory with NativeFSLockFactory and I didn't get LockObtainFailedException. I removed also this part making unlocking (IndexReader.unlock). The exception is: Exception in thread "Thread-95" org.apa

Re: FileNotFoundException during indexing

2008-08-01 Thread Michael McCandless
Another option is to switch to native locks (dir.setLockFactory(new NativeFSLockFactory()), at which point you will never have to call IndexReader.unLock because native locks are always properly released by the OS when the JVM exits/crashes. If on switching to native locks, and removing t

Re: FileNotFoundException during indexing

2008-08-01 Thread Michael McCandless
Wojtek212 wrote: Hi Mike, I'm sharing one instance of IndexManager across all threads and as I've noticed only this one is used during indexing. OK, maybe triple check this -- because that's the only way in your code I can see 2 IWs being live at once. I'm unlocking before every inde

Re: FileNotFoundException during indexing

2008-08-01 Thread Wojtek212
Hi Mike, I'm sharing one instance of IndexManager across all threads and as I've noticed only this one is used during indexing. I'm unlocking before every indexing operation to make sure it would be possible. When IndexWriter is closed I assume it releases the lock and finishes its work. Does Ind

Re: FileNotFoundException during indexing

2008-07-31 Thread Michael McCandless
Are you only creating one instance of IndexManager and then sharing that instance across all threads? Can you put some logging/printing where you call IndexReader.unLock, to see how often that's happening? That method is dangerous because if you unlock a still-active IndexWriter it leads