Re: Chinese Japanese Korean Indexing issue Version 2.4

2009-09-10 Thread asitag
error. Encountered unkown character. We tried setting the string > encoding to UTF 8 but it does not help. > > Can anyone please help. Any pointers will be highly appreciated. > > Thanks > -- View this message in context: http://www.nabble.com/Chinese-Japanese-Korean-Indexing-i

Chinese Japanese Korean Indexing issue Version 2.4

2009-09-10 Thread asitag
pointers will be highly appreciated. Thanks -- View this message in context: http://www.nabble.com/Chinese-Japanese-Korean-Indexing-issue-Version-2.4-tp25388003p25388003.html Sent from the Lucene - Java Users mailing list archive at Nabble.com

Re: Bizarre indexing issue where thousands of files get created

2009-08-27 Thread Micah Jaffe
Unfortunately, there's no concise summarization of the code to post. As a follow-up to the situation, after throwing more RAM at the problem (from 2GB to 10GB of process space) we no longer see OOM errors and also no longer see the IndexWriter create thousands of files; the OOM error did ap

Re: Bizarre indexing issue where thousands of files get created

2009-08-18 Thread Jason Rutherglen
Micah, If you can post some of your code, it may be easier to identify the problem you're experiencing. -J On Tue, Aug 18, 2009 at 9:55 AM, Micah Jaffe wrote: > Hi, thanks for the response!  The (custom) searchers that are falling out of > cache are indeed calling close on their IndexReader in f

Re: Bizarre indexing issue where thousands of files get created

2009-08-18 Thread Micah Jaffe
Hi, thanks for the response! The (custom) searchers that are falling out of cache are indeed calling close on their IndexReader in finalize(); they are not calling close on themselves as that appears to be a no-op when creating an IndexSearcher with a reader. The searchers are just extend

Re: Bizarre indexing issue where thousands of files get created

2009-08-18 Thread Michael McCandless
Are you .close()ing your IndexReaders when they fall out of the MRU cache? Seems like there are two problems... 1) why are you hitting OOMEs? Seems likely you're just doing too much at once can you ask the JRE to get you a heap dump when it hits OOME? 2) Why is IndexWriter creating zillions o

Bizarre indexing issue where thousands of files get created

2009-08-17 Thread Micah Jaffe
The Problem: periodically we see thousands of files get created from an IndexWriter in a Java process in a very short period of time. Since we started trying to track this, we saw an index go from ~25 files to over 200K files in about a half hour. The Context: a hand-rolled, all-in-one Luc

Re: indexing issue

2008-12-14 Thread Michael Stoppelman
On Sat, Nov 29, 2008 at 11:11 AM, Yonik Seeley wrote: > On Sat, Nov 29, 2008 at 12:45 PM, Michael Stoppelman > wrote: > > Hi all, > > > > I've got an indexing issue I think other folks might be interested in > > hearing about and I wanted to get feedback befo

Re: indexing issue

2008-11-29 Thread Yonik Seeley
On Sat, Nov 29, 2008 at 12:45 PM, Michael Stoppelman <[EMAIL PROTECTED]> wrote: > Hi all, > > I've got an indexing issue I think other folks might be interested in > hearing about and I wanted to get feedback before I went ahead and > implemented a new method. > &

indexing issue

2008-11-29 Thread Michael Stoppelman
Hi all, I've got an indexing issue I think other folks might be interested in hearing about and I wanted to get feedback before I went ahead and implemented a new method. Currently, the way we update indices is by sending individual delete/add document requests to all our search