You have to call updateDocument with the unique key of the document to update. The unique key must be a separate, indexed, not necessarily stored key. addDocument just adds a new instance of the document to the index, it cannot determine if it’s a duplicate.
----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -----Original Message----- > From: gudiseashok [mailto:gudise.as...@gmail.com] > Sent: Tuesday, October 01, 2013 2:01 AM > To: java-user@lucene.apache.org > Subject: Rendexing problem: Indexing folder size is keep on growing for > same remote folder > > Hi > > I am reading log files a remote folder and keeping them in local folder and > then I am indexing as I shown below, how ever I am not saving the whole > content as input stream, I am splitting them with Grok regex and saving as > below strings. > > I am repeating this "copy and indexing" process for every 30 minutes (as a > cron job), and my index folder size is getting doubled on completion each > batch. I am using Lucene 4.4 version, after looking at the addDocument > implementation in IndexWriter, I assume that it is calling update though I > call > addDocument with (CREATE_OR_APPEND). > > Kindly suggest right approach if I am not doing in correct way, my > requirement is to update that log folder content for every 30 minutes and re- > indexing the content. > > Thanks for taking time to read this, and please see my code configuration > snippet below... > > <Code> > //Adding document > document.add(new StringField("className", logsVO.getClassName(), > Field.Store.YES)); document.add(new StringField("logLevel", > logsVO.getLogLevel(), Field.Store.NO)); document.add(new > TextField("logMessage", logsVO.getLogMessage(), Field.Store.YES)); > document.add(new StringField("messageType", > logsVO.getMessageType().toString(), Field.Store.NO)); document.add(new > LongField("timeStamp", logsVO.getTimeStamp().getTime(), > Field.Store.YES)); IndexWriter writer = luceneUtil.getIndexWriter(); > writer.addDocument(document); > > //addDocument is calling IndexWriter's (API class) below mentioned > method, > public void addDocument(Iterable<? extends IndexableField> doc) throws > IOException { > addDocument(doc, analyzer); > } > > //Writer creation approach... > > Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_44); > IndexWriterConfig iwc = new IndexWriterConfig(Version.LUCENE_44, > analyzer); > > iwc.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND); > this.writer = new IndexWriter(dir, iwc); /// > > > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Rendexing-problem-Indexing-folder- > size-is-keep-on-growing-for-same-remote-folder-tp4092835.html > Sent from the Lucene - Java Users mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
