Re: Problems with IndexWriter#commit() on Linux

Yes. Mike On Wed, Feb 10, 2010 at 6:36 AM, Naama Kraus wrote: > Do you mean by calling > > IndexWriter#*setInfoStream*(PrintStream > > infoStream) > > ? > > Naama > > > On Mon, Feb 8, 2010 at 3:22 PM, Michael McCandless < > luc...@

Re: Problems with IndexWriter#commit() on Linux

Do you mean by calling IndexWriter#*setInfoStream*(PrintStream infoStream) ? Naama On Mon, Feb 8, 2010 at 3:22 PM, Michael McCandless < luc...@mikemccandless.com> wrote: > Hmmm... I think that means you're using the default data

Re: Problems with IndexWriter#commit() on Linux

Hmmm... I think that means you're using the default data mode (ordered), which should properly preserve writes if the OS or machine crashes. And actually I was wrong before -- even if the mount had data=writeback, since you are "only" kill -9ing the process (not crashing the machine), the data mou

Re: Problems with IndexWriter#commit() on Linux

Here is what I get with mount -l /dev/mapper/lvm--raid-lvm0 on /data3 type ext3 (rw) [] Is there anything else to get more details of the mount options ? On Mon, Feb 8, 2010 at 10:57 AM, Michael McCandless < luc...@mikemccandless.com> wrote: > Thanks for sharing... > > Software RAID should be pe

Re: Problems with IndexWriter#commit() on Linux

Thanks for sharing... Software RAID should be perfectly fine for Lucene, in general, unless the mount is configured to ignore fsync (I think the "data=writeback" mount option for ext3 does so on Linux). Can you check the mount options on your RAID filesystem? Mike On Mon, Feb 8, 2010 at 2:09 AM

Re: Problems with IndexWriter#commit() on Linux

Hi All, I am back to this one after some while. It appears the file system I was using resides on software RAID disks. I ran the same code on the same Linux machine, but on another file system residing on SCSI disks. I didn't observe the problem there. Both file systems are ext3. So I am guessing

Re: Problems with IndexWriter#commit() on Linux

Thanks all for the hints, I'll get back to my code and do some additional checks. Naama On Thu, Jan 7, 2010 at 6:57 PM, Michael McCandless < luc...@mikemccandless.com> wrote: > kill -9 is harsh, but, perfectly fine from Lucene's standpoint. > Likewise if the OS or JVM crashes, power is suddenly l

Re: Problems with IndexWriter#commit() on Linux

kill -9 is harsh, but, perfectly fine from Lucene's standpoint. Likewise if the OS or JVM crashes, power is suddenly lost, the index will just fallback to the last successful commit. What will cause corruption is if you have bit errors happening somewhere in the machine... or if two writers are ac

Re: Problems with IndexWriter#commit() on Linux

Can you show us the code where you commit? And how do you kill your process? Kill -9 is...er...harsh Yeah, I'm wondering whether the index file size *stays* changed after you kill you process. If it keeps its growing on every run (after you kill your process multiple times), then I'd suspect

Re: Problems with IndexWriter#commit() on Linux

Thanks dor the input. 1. While the process is running, I do see the index files growing on disk and the time stamps changing. Should I see a change in size right after killing the process, is that what you mean ? 2. Yes, same directory is being used for indexing and search. 3. Didn't try Luke, goo

Re: Problems with IndexWriter#commit() on Linux

Several questions: 1> are the index files larger after you kill your process? Or have the timestamps changed? 2> are you absolutely sure that your indexer, when you add documents, is pointing at the same directory your search is pointing to? 3> Have you gotten a copy of Luke and exami