Re: Upgrade to 3.6 OR wait for 4.0

2012-07-09 Thread Ganesh
Thanks for the reply. Any idea how much time it would take to go for 4.0 stable release? I want to go for v4.0 but i have to use only the stable version. Regards Ganesh - Original Message - From: "Shai Erera" To: Sent: Tuesday, July 10, 2012 10:50 AM Subject: Re: Upgrade to 3.6 OR

Re: Upgrade to 3.6 OR wait for 4.0

2012-07-09 Thread Shai Erera
Hi Ganesh I recently upgraded my code to 3.6, and yesterday finished part of my upgrades to 4.0-ALPHA. Upgrading from 3.0.3 to 3.6 is relatively easy as all API should be backwards compatible. But I think there were some API breaks, and back-compat issues. Therefore, if I were you, I'd first upgr

Upgrade to 3.6 OR wait for 4.0

2012-07-09 Thread Ganesh
Hello all, I am currently using v3.0.3 and planning to upgrade to v3.6. Shall i go ahead with the upgrade OR wait for 4.0? Regards Ganesh - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional command

Direct memory footprint of NIOFSDirectory

2012-07-09 Thread Vitaly Funstein
Hello, I have recently run into the situation when there was not a sufficient amount of direct memory available for IndexWriter to work. This was essentially caused by the embedding application making heavy use of JVM's direct memory buffers and not leaving enough headroom for NIOFSDirectory to op

Re: Spatial Search

2012-07-09 Thread Amir
Thanks heaps for the quick reply David... Computing the distance on the client is not an issue, however I was wondering as you are already computing the distances, there is any caching mechanism that I could retrieve the computed values. I looked into CachedDistanceValueSource class etc, but appar

Re: index.merge.scheduler exception - java.io.IOException: Input/output error

2012-07-09 Thread T Vinod Gupta
thanks this was really helpful to understand whats going on.. i got these for 2 of my indexes - WARNING: 29 broken segments (containing 385395 documents) detected WARNING: would write new segments file, and 385395 documents would be lost, if -fix were specified WARNING: 29 broken segments (contai

Re: Problems with hundreds of BLOCKED threads.

2012-07-09 Thread Leon Rosenberg
Hello Simon, thank you for the answer and the hint! Leon On Sat, Jul 7, 2012 at 1:29 PM, Simon Willnauer wrote: > On Fri, Jul 6, 2012 at 9:28 PM, Leon Rosenberg > wrote: >> Hello, >> >> we have a small internet shop which uses lucene for product search. >> With increasing traffic we have contin

Re: Problems with hundreds of BLOCKED threads.

2012-07-09 Thread Simon Willnauer
here is a snippet from RAMDirectory JavaDocs: * Warning: This class is not intended to work with huge * indexes. Everything beyond several hundred megabytes will waste * resources (GC cycles), because it uses an internal buffer size * of 1024 bytes, producing millions of {@code byte[1024]} arr

Re: Problems with hundreds of BLOCKED threads.

2012-07-09 Thread Simon Willnauer
On Fri, Jul 6, 2012 at 9:28 PM, Leon Rosenberg wrote: > Hello, > > we have a small internet shop which uses lucene for product search. > With increasing traffic we have continuos problem with literaly > hundreds of threads being BLOCKED in lucene code: > > here is an example taken with jstack on p

Re: Spatial Search

2012-07-09 Thread David Smiley (@MITRE.org)
Amir, The geo query's score happens to be the distance but I don't expect that to remain so. I plan for it to be 1/distance which is a better relevancy value -- a "score" is supposed to be about relevancy after all. If you want to get the distance for a search result, I recommend calculating it

Re: index.merge.scheduler exception - java.io.IOException: Input/output error

2012-07-09 Thread Erick Erickson
no, you can't delete those files, and you can't regenerate just those files, all the various segment files are necessary and intertwined... Consider using the CheckIndex facility, see: http://solr.pl/en/2011/01/17/checkindex-for-the-rescue/ note, the CheckIndex class is contained in the lucene co

Trust in new flexible StandardQueryParser?

2012-07-09 Thread karsten-solr
Hi folks, we really like the flexible QueryParser. We already used it for a Verity-Syntax Parser and it worked well. But now we want to use the Standard-Lucene-Syntax with some additions. Can we trust in StandardQueryParser or should(must) we use the classic QueryParser? For project safety we

Re: index.merge.scheduler exception - java.io.IOException: Input/output error

2012-07-09 Thread T Vinod Gupta
this is on local file system on amazon ec2 host. the file system was fine until a week ago when the outage happened and there were probably some system glitches. i have seen this issue since then.. sometimes regular commands like less or ls hang for many seconds even though there is no cpu/memory p

Re: index.merge.scheduler exception - java.io.IOException: Input/output error

2012-07-09 Thread Ian Lea
Is this on a local or remote file system? Is the file system itself OK? Is something else messing with your lucene index at the same time? -- Ian. On Sun, Jul 8, 2012 at 8:58 PM, T Vinod Gupta wrote: > Hi, > My log files are showing the below exceptions almost at twice a minute > frequency.

Re: about some seacher(I'm new hand, thank you for help)

2012-07-09 Thread Ian Lea
You don't know how to split the string containing the data you want to index?? String s = "2012-07-06 11:11:43some message"; String timestamp = s.substring(0, 19); String content = s.substring(19).trim(); is one way. -- Ian. On Mon, Jul 9, 2012 at 3:55 AM, sam wro

Re: Spatial Search

2012-07-09 Thread amir
Hi David, I'm implementing a project using your RecursivePrefixTreeStrategy and it works fine when I'm only querying based on the geo field as I can sort and can get the distance from the score of the results. However I couldn't figure out how to sort and get the distance when I combine the geoque