Hi Grant > Seems like something is closing your InputStreamReader out from under you. > Is there a concurrency issue, perhaps?
I'm coming to the same conclusion - there must be >1 threads accessing this index at the same time. Better go figure it out ... :-) Thanks again - Chris Chris Bamford Senior Development Engineer Scalix chris.bamf...@scalix.com Tel: +44 (0)1344 381814 www.scalix.com ----- Original Message ----- From: Grant Ingersoll <gsing...@apache.org> Sent: Thu, 3/9/2009 2:59pm To: java-user@lucene.apache.org Subject: Re: New "Stream closed" exception with Java 6 On Sep 2, 2009, at 7:45 AM, Chris Bamford wrote: > Hi Grant, > > I have now followed Daniel's advice and catch the exception with: > > try { > indexWriter.addDocument(doc); What does your Document/Field creation code look like? In other words, how do you construct doc? Seems like something is closing your InputStreamReader out from under you. Is there a concurrency issue, perhaps? > } catch (CorruptIndexException ex) { > throw new IndexerException ("CorruptIndexException on doc: " > + doc.toString(), ex); > } catch (IOException ex) { > throw new IndexerException ("*IOException* on doc: " + > doc.toString(), ex); > } > > The result now appears as: > > 2009-09-02 10:14:24,014 ERROR [QueueManager] > [BatchUpdater.processMods:183] Failed to add content document(s) for > user 0bb00000daaa2864-9.0.61.271 > com.scalix.index.api.IndexerException: *IOException* on doc: > Document<indexed,tokenized<return-path:christian.sho...@scalix.com> > indexed,tokenized<received:from scalixmail.scalix.com (localhost > [127.0.0.1]) > by scalixmail.scalix.com (8.13.6/8.13.6/SuSE Linux 0.8) with > ESMTP id m0F9545p008012; > Tue, 15 Jan 2008 04:05:05 -0500> > indexed,tokenized<received:from [10.13.1.51] (scalixmail.scalix.com > [172.16.0.9]) > by scalixmail.scalix.com (Scalix SMTP Relay 11.3.0.11339) > via ESMTP; Tue, 15 Jan 2008 04:05:05 -0500 (EST)> indexed<CREATE- > DATE:20080115090228> indexed,tokenized<date:Tue Jan 15 04:02:28 EST > 2008> indexed,tokenized<from:"Christian Sholtz" <christian.sho...@scalix.com > >> indexed,tokenized<to:SIN - Robert Smith <Robert.Smi > t...@sin.co.at>> indexed,tokenized<cc:Patrick Lauria > <patr...@scalix.com>> indexed<message-id: > 478c76a4.2090...@scalix.com> indexed<in-reply- > to:h000007b0031e9f3.1200380510.scalix.sin.co...@mhs> > indexed<references:h000007b0031e9f3.1200380510.scalix.sin.co...@mhs> > indexed,tokenized<subject: > Re: AW: Lead Landwirtschaftskammer Kärnten -> Smith> > indexed,tokenized<autoforwarded:FALSE> > indexed,tokenized<importance:Normal> > indexed,tokenized<priority:Normal> > indexed,tokenized<sensitivity:Normal> indexed,tokenized<x-msmail- > priority:Medium> indexed,tokenized<x-priority:3> indexed<x-scalix- > autoreplied:FALSE> indexed,tokenized<user-agent:Thunderbird 2.0.0.9 > (Windows/20071031)> indexed,tokenized<x-spam-status:No, score=-1.3 > required=5.0 tests=AWL,BAYES_00, > SUBJECT_ENCODED_TWICE autolearn=no version=3.1.8> > indexed,tokenized<x-spam-checker-version:SpamAssassin 3.1.8 > (2007-02-13) on > scalixmail.scalix.com> indexed,tokenized<x-mime- > autoconverted:from 8bit to quoted-printable by scalixmail.scalix.com > id m0F9545p00801 > 2> indexed,tokenized<mime-version:1.0> indexed<SIZE-KBYTES: > 0000000006> indexed<DELIVERY-DATE:20080115090509> > indexed,tokenized<x-scalix-internaldate:Tue Jan 15 04:05:09 EST > 2008> indexed<x-scalix-has-attach:FALSE> indexed,tokenized<content- > type:text/plain; > charset="ISO-8859-1"; > format="flowed"> indexed,tokenized<content-transfer- > encoding:quoted-printable> indexed,tokenized<content- > disposition:inline> indexed<CONTENT-TYPE:text/plain> > indexed,tokenized<BODY:java.io.inputstreamrea...@75b940> indexed<x- > scalix-class:IPM.Note> indexed<MSG-TYPE:email> stored/ > uncompressed,indexed<INDEXID:90010ac-4682aaad-478c7745-2ee4ab>> > at com.scalix.index.manager.BatchUpdater.performContentAdds > (BatchUpdater.java:394) > at com.scalix.index.manager.BatchUpdater.processMods > (BatchUpdater.java:179) > at com.scalix.index.manager.QueueManager.processQueue > (QueueManager.java:134) > at com.scalix.index.manager.QueueManager.run > (QueueManager.java:81) > at java.lang.Thread.run(Unknown Source) > Caused by: java.io.IOException: Stream closed > at sun.nio.cs.StreamDecoder.ensureOpen(Unknown Source) > at sun.nio.cs.StreamDecoder.read(Unknown Source) > at java.io.InputStreamReader.read(Unknown Source) > at > org.apache.lucene.analysis.standard.StandardTokenizerImpl.zzRefill > (StandardTokenizerImpl.java:443) > at > org.apache.lucene.analysis.standard.StandardTokenizerImpl.getNextToken > (StandardTokenizerImpl.java:629) > at org.apache.lucene.analysis.standard.StandardTokenizer.next > (StandardTokenizer.java:140) > at org.apache.lucene.analysis.standard.StandardFilter.next > (StandardFilter.java:43) > at org.apache.lucene.analysis.LowerCaseFilter.next > (LowerCaseFilter.java:34) > at org.apache.lucene.analysis.snowball.SnowballFilter.next > (SnowballFilter.java:64) > at org.apache.lucene.index.DocInverterPerField.processFields > (DocInverterPerField.java:134) > at > org.apache.lucene.index.DocFieldConsumersPerField.processFields > (DocFieldConsumersPerField.java:36) > at > org.apache.lucene.index.DocFieldProcessorPerThread.processDocument > (DocFieldProcessorPerThread.java:234) > at org.apache.lucene.index.DocumentsWriter.updateDocument > (DocumentsWriter.java:765) > at org.apache.lucene.index.DocumentsWriter.addDocument > (DocumentsWriter.java:743) > at org.apache.lucene.index.IndexWriter.addDocument > (IndexWriter.java:1902) > at org.apache.lucene.index.IndexWriter.addDocument > (IndexWriter.java:1880) > at com.scalix.index.manager.BatchUpdater.performContentAdds > (BatchUpdater.java:390) > ... 4 more > > Does this help? > > Thanks, > > - Chris > > > > ----- Original Message ----- > From: Daniel Shane <sha...@lexum.umontreal.ca> > Sent: Tue, 1/9/2009 10:52pm > To: java-user@lucene.apache.org > Subject: Re: New "Stream closed" exception with Java 6 > > I think you should do this instead (it will print the exception > message > *and* the stack trace instead of only the message) : > > throw new IndexerException ("CorruptIndexException on doc: " + > doc.toString(), ex); > > Daniel Shane > > Chris Bamford wrote: >> Hi Grant, >> >> >>>> I think you code there needs to show the underlying exception, >>>> too, so >>>> we can see that stack trace. >>>> >> >> Ummm... isn't this code already doing that? What am I missing? >> >> try { >> >> indexWriter.addDocument(doc); >> >> } catch (CorruptIndexException ex) { >> >> throw new IndexerException ("CorruptIndexException >> on doc: " + doc.toString() + >> >> " - " + ex.toString()); >> >> } catch (IOException ex) { >> >> throw new IndexerException ("IOException on doc: " + >> doc.toString() + >> >> " - " + ex.toString()); >> >> } >> >> Thanks, >> >> - Chris >> >> Chris Bamford >> Senior Development Engineer >> Scalix >> chris.bamf...@scalix.com >> Tel: +44 (0)1344 381814 >> www.scalix.com >> >> >> >> ----- Original Message ----- >> From: Grant Ingersoll <gsing...@apache.org> >> Sent: Sat, 29/8/2009 12:00pm >> To: java-user@lucene.apache.org >> Subject: Re: New "Stream closed" exception with Java 6 >> >> I think you code there needs to show the underlying exception, too, >> so >> we can see that stack trace. >> >> -Grant >> >> On Aug 28, 2009, at 11:47 AM, Chris Bamford wrote: >> >> >>> Hi Grant >>> >>> Please see attached. >>> >>> Thanks, >>> >>> - Chris >>> >>> >>> ----- Original Message ----- >>> From: Grant Ingersoll <gsing...@apache.org> >>> Sent: Fri, 28/8/2009 1:26pm >>> To: java-user@lucene.apache.org >>> Subject: Re: New "Stream closed" exception with Java 6 >>> >>> Are there any log messages around that spot? Did something happen >>> to >>> the underlying Reader you are using? Can you share that little >>> bit of >>> indexing code? >>> >>> On Aug 27, 2009, at 10:11 AM, Chris Bamford wrote: >>> >>> >>>> Hi, >>>> >>>> Since moving our app to Java 6 and Tomcat 6, we have started >>>> getting >>>> occasional exceptions of the form: >>>> >>>> java.io.IOException: Stream closed >>>> at sun.nio.cs.StreamDecoder.ensureOpen(Unknown Source) >>>> at sun.nio.cs.StreamDecoder.read(Unknown Source) >>>> at java.io.InputStreamReader.read(Unknown Source) >>>> at >>>> org >>>> .apache >>>> .lucene >>>> .analysis >>>> .standard.StandardTokenizerImpl.zzRefill >>>> (StandardTokenizerImpl.java: >>>> 443) >>>> at >>>> org >>>> .apache >>>> .lucene >>>> .analysis >>>> .standard >>>> .StandardTokenizerImpl.getNextToken(StandardTokenizerImpl.java:629) >>>> at >>>> org >>>> .apache >>>> .lucene >>>> .analysis.standard.StandardTokenizer.next(StandardTokenizer.java: >>>> 140) >>>> at >>>> org >>>> .apache >>>> .lucene.analysis.standard.StandardFilter.next(StandardFilter.java: >>>> 43) >>>> at >>>> org.apache.lucene.analysis.LowerCaseFilter.next >>>> (LowerCaseFilter.java: >>>> 34) >>>> at >>>> org >>>> .apache >>>> .lucene.analysis.snowball.SnowballFilter.next(SnowballFilter.java: >>>> 64) >>>> at >>>> org >>>> .apache >>>> .lucene >>>> .index.DocInverterPerField.processFields(DocInverterPerField.java: >>>> 134) >>>> at >>>> org >>>> .apache >>>> .lucene >>>> .index >>>> .DocFieldConsumersPerField >>>> .processFields(DocFieldConsumersPerField.java:36) >>>> at >>>> org >>>> .apache >>>> .lucene >>>> .index >>>> .DocFieldProcessorPerThread >>>> .processDocument(DocFieldProcessorPerThread.java:234) >>>> at >>>> org >>>> .apache >>>> .lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java: >>>> 765) >>>> at >>>> org >>>> .apache >>>> .lucene.index.DocumentsWriter.addDocument(DocumentsWriter.java:743) >>>> at >>>> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java: >>>> 1902) >>>> at >>>> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java: >>>> 1880) >>>> at >>>> com >>>> .scalix >>>> .index.manager.BatchUpdater.performContentAdds(BatchUpdater.java: >>>> 386) >>>> at >>>> com.scalix.index.manager.BatchUpdater.processMods >>>> (BatchUpdater.java: >>>> 176) >>>> at >>>> com.scalix.index.manager.QueueManager.processQueue >>>> (QueueManager.java: >>>> 134) >>>> at com.scalix.index.manager.QueueManager.run(QueueManager.java:81) >>>> at java.lang.Thread.run(Unknown Source) >>>> >>>> We are using Lucene 2.4. Any ideas what is happening? >>>> >>>> Thanks! >>>> >>>> - Chris >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >>>> For additional commands, e-mail: java-user-h...@lucene.apache.org >>>> >>>> >>> -------------------------- >>> Grant Ingersoll >>> http://www.lucidimagination.com/ >>> >>> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) >>> using Solr/Lucene: >>> http://www.lucidimagination.com/search >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: java-user-h...@lucene.apache.org >>> >>> <lucene_msg.txt> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: java-user-h...@lucene.apache.org >>> >> >> -------------------------- >> Grant Ingersoll >> http://www.lucidimagination.com/ >> >> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) >> using Solr/Lucene: >> http://www.lucidimagination.com/search >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > -------------------------- Grant Ingersoll http://www.lucidimagination.com/ Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene: http://www.lucidimagination.com/search --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org