Re: Lucene 1.9.1 - How to determine from which machine the hit comes?

2006-04-04 Thread Chris Hostetter
: How to find out which searcher a hit comes from with a MultiSearcher? look at MultiSearcher.subSearcher and MultiSearcher.subDoc. -Hoss - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL P

Re: What is the largest index(s) size lucene can support

2006-04-04 Thread Chris Hostetter
: Or if to be more specific, can Lucene support BILLIONS of documents : (across multiple indexes), and search all of them at once using the : Parallel searcher? I believe the theoretical limit in the number of documents is Integer.MAX_VALUE -- regardless of how you partition the index, your Multi

Custom hit score

2006-04-04 Thread Fisheye
I want to have a simple hit score for every document where the query has been found. E.g. if the query word was found 3 times in a document, this doc should have 100% score, next document with 2 times should have 90% and so on... Normal hit score used by Lucene seems to be strange so I only want

QueryParser error

2006-04-04 Thread miki sun
Hi there Who can tell me why I got the the queryParser error for the following query: Error in parse query :The light of the body is the eye: if therefore thine eye be single, thy whole body shall be full of light. But if thine eye be evil, thy whole body shall be full of darkness. If therefor

QueryParser error

2006-04-04 Thread miki sun
Hi there Who can tell me why I got the the queryParser error for the following query: Error in parse query :The light of the body is the eye: if therefore thine eye be single, thy whole body shall be full of light. But if thine eye be evil, thy whole body shall be full of darkness. If therefor

Re: QueryParser error

2006-04-04 Thread Michael Barry
You need to escape the colons. miki sun wrote: Hi there Who can tell me why I got the the queryParser error for the following query: Error in parse query :The light of the body is the eye: if therefore thine eye be single, thy whole body shall be full of light. But if thine eye be evil, th

Re: QueryParser error

2006-04-04 Thread miki sun
Still got error even without colon: Error in parse query :The light of the body is the eye if therefore thine eye be single, thy whole body shall be full of light. But if thine eye be evil, thy whole body shall be full of darkness. If therefore the light that is in thee be darkness, how great

Re: QueryParser error

2006-04-04 Thread Michael Barry
Exclamation point too ... check out the "Escaping Special Characters" section in the QueryParser syntax guide (http://lucene.apache.org/java/docs/queryparsersyntax.html) miki sun wrote: Still got error even without colon: Error in parse query :The light of the body is the eye if therefore th

Re: QueryParser error

2006-04-04 Thread Erik Hatcher
Miki hasn't posted a complete stack trace, and if the NPE is coming from QueryParser it would likely give stack trace information pinpointing precisely where the issue is. My hunch is some other code is obfuscating the real issue. A complete stack trace would be much more helpful.

Re: QueryParser error + solution

2006-04-04 Thread miki sun
Thanks Erik and Michael! I copied some code from demo.SearchFiles.java, I do not have a more clearer tracing message. Now it works. But do you have a better way than this: //escaping special chars for query parser for (int i = 0; i < qr.length(); i ++) { if( (qr.charAt(i) == '+') ||(qr.charAt

highlighting - fuzzy search

2006-04-04 Thread Fisheye
Is it possible to get back a highlighted text "snippet" when using fuzzy search? I mean where does lucene stores the similar words to the search query? If I know where these words are, I can use one of these words to highlight. thx Simon Dietschi -- View this message in context: http://www.nabb

Re: highlighting - fuzzy search

2006-04-04 Thread Erik Hatcher
On Apr 4, 2006, at 8:30 AM, Fisheye wrote: Is it possible to get back a highlighted text "snippet" when using fuzzy search? I mean where does lucene stores the similar words to the search query? If I know where these words are, I can use one of these words to highlight. You mean using a

Re: QueryParser error + solution

2006-04-04 Thread Erik Hatcher
If you're going to escape all the special characters used as operators in the query expression, then what is the point of using QueryParser? Seems like you could just tokenize the text yourself and build a Query instance via API to support your needs. Do you need to leverage any special

Re: highlighting - fuzzy search

2006-04-04 Thread Fisheye
Ok, thanks Erik. So probably my code may explain it: --- public void searchQuery(String q, float rel, String indexDir){ String excerpt = "";

Re: highlighting - fuzzy search

2006-04-04 Thread Erik Hatcher
So, just like I said call Query.rewrite() and pass the returned Query to the Highlighter, not the original FuzzyQuery. I believe the javadocs for Highlighter even mention this? Or at least its an FAQ that hopefully is on the wiki or easily findable somehow. Erik On Apr 4, 2

Re: highlighting - fuzzy search

2006-04-04 Thread Fisheye
ok, thank Erik, now it works :-) Probably, do you know if there is a possibility to get the similar words generated by the algorithm when doing fuzzy search? Cheers Simon Dietschi -- View this message in context: http://www.nabble.com/highlighting---fuzzy-search-t1392775.html#a3746483 Sent fro

problem Indexing path of files

2006-04-04 Thread pepone pepone
I trying to index the paths of a tree like structure in lucene My docs have info like this Node1 name:a path:/a Node2 name:b path:/a/b Node3 name:c path:/a/b/c when search for path:/a the 3 Nodes match I indexing this info using the StandardAnalyzer any ideas to solve this

RE: problem Indexing path of files

2006-04-04 Thread jwang
Use the PerFieldAnalyzerWrapper and set your path (and probably name) to KeywordAnalyzer. Reserve whatever analyzer you have for the actual contents/meta data of the file. Do a search on PerFieldAnalyzerWrapper in this ML for examples. Jeff Wang diCarta, Inc. -Original Message- From: pe

Re: highlighting - fuzzy search

2006-04-04 Thread Erik Hatcher
On Apr 4, 2006, at 11:23 AM, Fisheye wrote: Probably, do you know if there is a possibility to get the similar words generated by the algorithm when doing fuzzy search? Well, a roundabout way is to simply create a FuzzyQuery, rewrite it, cast it to a BooleanQuery and use the BooleanQuery A

Re: highlighting - fuzzy search

2006-04-04 Thread Daniel Noll
Erik Hatcher wrote: On Apr 4, 2006, at 11:23 AM, Fisheye wrote: Probably, do you know if there is a possibility to get the similar words generated by the algorithm when doing fuzzy search? Well, a roundabout way is to simply create a FuzzyQuery, rewrite it, cast it to a BooleanQuery and use

Re: Re[2]: OutOfMemory with search(Query, Sort)

2006-04-04 Thread Chris Hostetter
: > sort by filePath field which can be 100 bytes at average meaning 400M : > RAM for the cache : : Well, it's probably not quite that bad... : : For string sorting, a FieldCache.StringIndex is used. : It contains a sorted String[num_unique_terms_in_field], and an int[maxDoc] : So if 10 documents

Re: QueryParser error + solution

2006-04-04 Thread Daniel Noll
miki sun wrote: Thanks Erik and Michael! I copied some code from demo.SearchFiles.java, I do not have a more clearer tracing message. Now it works. But do you have a better way than this: [snip] Something like this? String str = "Really bad query string: lots of evil stuff!"; str = Qu

Re[4]: OutOfMemory with search(Query, Sort)

2006-04-04 Thread Artem Vasiliev
Hello Hoss, Thanks for your answer, you're right, filepathes are pretty much unique. Anyway I don't want this total-field-cache-loading situation occur in any circumstances - it's too expensive. My app usually crawls while user searches are performed. Crawl involves additions and deletions so Inde

Re: highlighting - fuzzy search

2006-04-04 Thread Chris Hostetter
: > Well, a roundabout way is to simply create a FuzzyQuery, rewrite it, : > cast it to a BooleanQuery and use the BooleanQuery API to extract the : > TermQuery objects and the Term within the TermQuery has what you're ... : We take an approach somewhere down the middle... ... :

Re: Custom hit score

2006-04-04 Thread Chris Hostetter
: I want to have a simple hit score for every document where the query has been : found. E.g. if the query word was found 3 times in a document, this doc : should have 100% score, next document with 2 times should have 90% and so you haven't defined the scoring mechanism you want to use very clea

Re[4]: OutOfMemory with search(Query, Sort)

2006-04-04 Thread Artem Vasiliev
>>I tried to >> sort by filePath field which can be 100 bytes at average meaning 400M >> RAM for the cache YS> For string sorting, a FieldCache.StringIndex is used. YS> It contains a sorted String[num_unique_terms_in_field], and an int[maxDoc] YS> So if 10 documents share a large string field valu