Thanks Uwe and Simon and for your detailed answers. I'm reviewing my code to find where I'm using a closed index.
Roberto On Nov 22, 2011, at 11:53 AM, Uwe Schindler wrote: > Hi, > > The JDK issues you are talking about are also fixed in 1.6.9_29 (not only > 1.7.0u1). So your crash is not related to any of the "famous java6/7 bugs" > (in addition, the vint bug does *not* crash your JVM at all it just corrupts > your index by returning wrong values - and this one is definitely fixed > since Lucene 3.1). > > But there is another chance you can crash your JVM: You are on a 64 bit > platform (Linux), so the default directory implementation is MMapDirectory. > Lucene uses a hack to be able to unmap mapped files from virtual address > space. This is not allowed by the JVM itsself, but makes unmapping dependent > on garbage collector, which is a problem for Lucene. By default > MMapDirectory unmaps the files after closing the IndexInputs. MMapDirectory > is not synchronized at all, so when another thread or even the same thread > tries to access the IndexInput after unmapping it will access an unmapped > address and will SIGSEGV. > > If your code would be correct this cannot happen, but it looks like you are > using an already closed IndexReader/IndexWriter to access the index. Before > Lucene 3.5 (will come out soon), missing checks in IndexReader will make it > possible that an already closed IndexReader with all its closed (and > unmapped) IndexInputs tries to access index data and segfaults. > > In 3.5 we added additional safety checks to prevent this illegal access, but > its not 100% (as synchronization is missing for performance). I would review > the code and check that nothing accesses closed index. > > A simple check to see if this is your issue would be to use NIOFSDirectory > (slower on Linux) instead of MMapDirectory. If it does not crash and > possibly throws AlreadyClosedExceptions in 3.5, the bug is that you are > accessing closed indexes. > > Uwe > > ----- > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > >> -----Original Message----- >> From: Roberto Fonti [mailto:roberto.fo...@gmail.com] >> Sent: Tuesday, November 22, 2011 8:03 PM >> To: java-user@lucene.apache.org >> Subject: JVM Crash org.apache.lucene.store.DataInput.readVInt()I >> >> My JVM (1.6.0_29) keeps crashing on intensive use when indexing documents >> with Lucene. I get: >> >> >> # >> # A fatal error has been detected by the Java Runtime Environment: >> # >> # SIGSEGV (0xb) at pc=0x00002b6b196d767c, pid=26417, tid=1183217984 # # >> JRE version: 6.0_29-b11 # Java VM: Java HotSpot(TM) 64-Bit Server VM > (20.4- >> b02 mixed mode linux-amd64 compressed oops) # Problematic frame: >> # J org.apache.lucene.store.DataInput.readVInt()I >> # >> # If you would like to submit a bug report, please visit: >> # http://java.sun.com/webapps/bugreport/crash.jsp >> # >> >> Environment: >> >> JDK: 1.6u29 (same issue with 1.6_02) Lucene Version 3.4.0 >> >> vm_info: Java HotSpot(TM) 64-Bit Server VM (20.4-b02) for linux-amd64 JRE >> (1.6.0_29-b11), built on Oct 3 2011 01:19:20 by "java_re" with gcc 3.2.2 > (SuSE >> Linux) >> >> OS:CentOS release 5.0 (Final) >> >> jvm_args: -Dcatalina.home=/var/local/tomcat-8081 - >> Dcatalina.base=/var/local/tomcat-8081 -Djava.io.tmpdir=/var/tmp - >> Dfile.encoding=UTF-8 -Xmx1024M -XX:MaxPermSize=96m >> >> It seems to be a jdk issue that was fixed in jdk 1.7, but other issues > where >> introduced. https://issues.apache.org/jira/browse/LUCENE-3335 "Java 7 >> contains a fix to the readVInt issue since 1.6.0_21 (approx, LUCENE-2975)" >> >> So, how can I fix this issue using JDK 1.6? Should I upgrade to jdk 1.7? >> >> >> >> Thanks, >> Roberto > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
