2.1-dev memory leak?

2006-11-29 Thread Otis Gospodnetic
Hi, Is anyone running Lucene trunk/HEAD version in a serious production system? Anyone noticed any memory leaks? I'm asking because I recently bravely went from 1.9.1 to 2.1-dev (trunk from about a week ago) and all of a sudden my application that was previosly consuming about 1.5GB (-Xmx1500

Re: Cached Hits / closing IndexSearcher after add/delete w/IndexModifier

2006-11-29 Thread Yonik Seeley
On 11/29/06, Mark Miller <[EMAIL PROTECTED]> wrote: If you haven't seen the following option you might want to check it out. It uses reference counting to keep track of writers/searchers/etc(I think solr does as well). Right, Solr uses reference counting... relying on GC will get out out-of-f

Re: Caused by: java.io.IOException: The handle is invalid

2006-11-29 Thread Michael McCandless
Michael McCandless wrote: Van Nguyen wrote: I'm running this on Windows 2003 server (NTFS). The Java VM version is 1.5.0_06. This exception is not consistent, but it is not intermittent either. It does not throw it at any particular point while rebuilding the index, but it will throw this exc

Re: BUG ? - lucene multisearcher / sorting

2006-11-29 Thread Daniel Naber
On Tuesday 28 November 2006 23:09, Kai R. Emde wrote: > we have one problem with the sort routine. We use the multisearcher > function over severall index. Does that also happens when you're not using MultiSearcher? Could you post a small test case that demonstrates this problem? To my knowledge

Re: Cached Hits / closing IndexSearcher after add/delete w/IndexModifier

2006-11-29 Thread Mark Miller
If you haven't seen the following option you might want to check it out. It uses reference counting to keep track of writers/searchers/etc(I think solr does as well). Has worked great for everything I have thrown at it other than need some tweaks to its multisearcher support. Also is a good bas

Re: Cached Hits / closing IndexSearcher after add/delete w/IndexModifier

2006-11-29 Thread Yonik Seeley
On 11/29/06, Yonik Seeley <[EMAIL PROTECTED]> wrote: On 11/29/06, Eric Brown <[EMAIL PROTECTED]> wrote: > > IndexModifier currently has very low performance with mixed adds > > and deletes. > > You can keep the same one over the lifetime of the app though. > > Assuming I call flush() after every

Re: Cached Hits / closing IndexSearcher after add/delete w/IndexModifier

2006-11-29 Thread Yonik Seeley
On 11/29/06, Eric Brown <[EMAIL PROTECTED]> wrote: > IndexModifier currently has very low performance with mixed adds > and deletes. > You can keep the same one over the lifetime of the app though. Assuming I call flush() after every operation (they won't be that frequent), if I don't call close

Re: Cached Hits / closing IndexSearcher after add/delete w/IndexModifier

2006-11-29 Thread Eric Brown
Hi Yonik, On Nov 29, 2006, at 11:16 AM, Yonik Seeley wrote: On 11/29/06, Eric Brown <[EMAIL PROTECTED]> wrote: [snip] Also, should I keep one IndexModifier open for the life of my service/ application or should I open and close it when I get new requests to add or remove documents? (I don't

Re: Cached Hits / closing IndexSearcher after add/delete w/IndexModifier

2006-11-29 Thread Yonik Seeley
On 11/29/06, Eric Brown <[EMAIL PROTECTED]> wrote: I'm using lucene as a backend for my webservices that provide add, remove and search operations. When I add or remove documents via IndexModifier, I believe I'm supposed to close the IndexSearcher I use for query requests. However, I cache Hits a

RE: Syns2Index utility: version of Lucene and Java

2006-11-29 Thread Risov, Maria
Chris, Thanks so much both for the pointer and the sanity check! Oddly enough, I read the same Syns2Index piece of documentation as you suggested very early in the search process. What I did get wrong was the actual download package, not realizing that *.java and *.class files would not be packag

Cached Hits / closing IndexSearcher after add/delete w/IndexModifier

2006-11-29 Thread Eric Brown
I'm using lucene as a backend for my webservices that provide add, remove and search operations. When I add or remove documents via IndexModifier, I believe I'm supposed to close the IndexSearcher I use for query requests. However, I cache Hits and I believe the javadocs indicate closing an

How important is closing an IndexSearcher/IndexModifier on application exit?

2006-11-29 Thread Eric Brown
I'm the only application accessing my index. Is there any chance the lucene index might remain in a locked state if I don't call close() my IndexSearcher or IndexModifier on applicaiton exit? I'm several layers down in a webapp, so using Runtime.addShutdownHook () is unreliable if my webapp

Re: Writing and searching same time

2006-11-29 Thread Simon Willnauer
On 11/29/06, Java Programmer <[EMAIL PROTECTED]> wrote: Hello, I have trouble with writing and searching on lucene index same time, all I did so far is making a class which has 2 methods: private String indexLocation; public void addDocument(int id,String title, String body) throws IOException{

[ANN]VTD-XML 1.8

2006-11-29 Thread Jimmy Zhang
Version 1.8 of VTD-XML is now released. The new features are: ·XMLModifier is a easy to use class that takes advantage of the incremental update capability offered by VTD-XML ·XPath built-in functions are now almost complete ·This release added encoding support for iso-8

Writing and searching same time

2006-11-29 Thread Java Programmer
Hello, I have trouble with writing and searching on lucene index same time, all I did so far is making a class which has 2 methods: private String indexLocation; public void addDocument(int id,String title, String body) throws IOException{ IndexWriter indexWriter = new Ind