Re: Increase performance using Pool of IndexSearchers?

2007-02-07 Thread Mohammad Norouzi
Hi would you tell how we can create a searcher pool. I get interested in your idea. thanks On 2/7/07, spamsucks <[EMAIL PROTECTED]> wrote: I currently am sharing my IndexSearcher across multiple requests as a static object. While I understand that the IndexSearcher is threadsafe, this does no

Re: Is there any way to optimize existing unoptimized index?

2007-02-07 Thread Patrick Turcotte
Hi, From my own experience, yes, you can. You just have to reopen your index with a new IndexWriter (making sure you don't use a constructor that will recreate it; from the javadoc, stay clear of those with a boolean argument). Then call optimize() then close() on your writer, it will be optim

Is there any way to optimize existing unoptimized index?

2007-02-07 Thread maureen tanuwidjaja
Hi, May I also ask wheter there is a way to use writer.optimize() without indexing the files from the beginning? It took me about 17 hrs to finish building an unoptimized index(finish when I call IndexWriter.close() ).I just wonder wheter this existing index could be optimized...

Re: How to not tokenize HTML tag from input string

2007-02-07 Thread Erick Erickson
Sure, just don't index the html tags in the first place. Of course that means you need to parse the document first. Here's a parser that was mentioned on the thread a while ago http://sourceforge.net/projects/mozillaparser There may very well be others Depending on how sophisticated you

exception is hit while optimizing index

2007-02-07 Thread maureen tanuwidjaja
Hi , I would like to know about optimizing index... The exception is hit due to disk full while optimizing the index and hence,the index has not been closed yet. Is the unclosed index dangerous?Can i perform searching in such index correctly?Is the index built robust yet? than

How to not tokenize HTML tag from input string

2007-02-07 Thread Joe Tang
My work is to index keywords with a document. In my case, the document is made up with HTML tags which i don't want to index them. For example: Input Document: You are welcome Testing text Expected Keywords: keywords:You keywords:are keywords:welcome keywords:Testing keywords:text Is there

Re: How to return results with null values?

2007-02-07 Thread Chris Hostetter
: A newbie here, but I would think that fields will null value will slow : down the search as much as fields with a specific non-conflicting keyword. : There is still a field to be looked at, even if it does not match. i believe you are correct ... it doesn't matter wether the Field value is nul

Re: Re : Re: Question concerning Analyzers

2007-02-07 Thread Erick Erickson
Then the analyzer you're using when parsing the query is stripping them. It must be different than the one you use when indexing somehow. At least that's the only explanation I can imagine Perhaps, somehow, you are using a default analyzer when you parse a query? Or you aren't specifying the

Re : Re: Question concerning Analyzers

2007-02-07 Thread Xavier To
Thanks Erik and Erick, I guess my question was rather unclear, but you guys answered it all the same : it is impossible for an analyzer to index something and having the same analyzer ignore the thing indexed during a search. If it makes everything clearer, during indexation, numbers are inde

FW: Question concerning Analyzers

2007-02-07 Thread Damien McCarthy
"Can an analyzer index something but ignore it during a search" It sounds like you might be using a different analyzer to parse your search query than you used when you indexed the data. Perhaps you could print out the toString() of the query you pass to the IndexSearcher, as this would show you i

Re: Question concerning Analyzers

2007-02-07 Thread Erik Hatcher
There is no requirement that you use the same analyzer to search as you used to index. So, yes, you could certainly index things and ignore them during a search. Erik On Feb 7, 2007, at 2:10 PM, Xavier To wrote: Hi, me again I'm still stuck with my search engine, but something p

Re: Question concerning Analyzers

2007-02-07 Thread Erick Erickson
Well, at the very worst, you can use a different analyzer at search time than you do at index time. I can imagine you create your own analyzer by subclassing, say, StandardAnalyzer and use your custom analyzer at search time Otherwise, you can't have a stock analyzer do one thing when indexin

Re: Increase performance using Pool of IndexSearchers?

2007-02-07 Thread Simon Willnauer
On 2/7/07, spamsucks <[EMAIL PROTECTED]> wrote: I currently am sharing my IndexSearcher across multiple requests as a static object. static or not a single / shared searcher is a good idea. While I understand that the IndexSearcher is threadsafe, this does not necessarily mean that it's multi

Question concerning Analyzers

2007-02-07 Thread Xavier To
Hi, me again I'm still stuck with my search engine, but something popped in my head : Can an analyzer index something but ignore it during a search ? I'm asking this because now that I've been searching for an answer, I've come to think that I should redo the whole search engine, but I don't wa

Increase performance using Pool of IndexSearchers?

2007-02-07 Thread spamsucks
I currently am sharing my IndexSearcher across multiple requests as a static object. While I understand that the IndexSearcher is threadsafe, this does not necessarily mean that it's multi-threaded capable. I was wondering if there is any thread blocking going on between requests? Would it

Re: How to return results with null values?

2007-02-07 Thread Don . Reese
A newbie here, but I would think that fields will null value will slow down the search as much as fields with a specific non-conflicting keyword. There is still a field to be looked at, even if it does not match. Beyond that I will have to leave it to others with more background with the tool.

Re: How to return results with null values?

2007-02-07 Thread poeta simbolista
Thanks for the suggestions. Well, the thing is, some entries have a value on that facet, and some others not. It may happen that all have, all have not, and any combination within those. Regarding the thing of adding a field with a specific non-conflicting keyword, wouldn't it affect the performa

Re: Counting term frequency without using Explanation

2007-02-07 Thread csahat
Hi Erick, Thanks for your help. I found it now . Maybe I will donate an article / FAQ regarding this... since many newbies like me asking for things like this. On 2/7/07, Erick Erickson <[EMAIL PROTECTED]> wrote: Before you go too far down this path, please consider what a "hit" is. It's

Re: Counting term frequency without using Explanation

2007-02-07 Thread Erick Erickson
Before you go too far down this path, please consider what a "hit" is. It's more complicated than you think . If all you want to do is count up the number of times any term appears in the document, it's not too hard. You should be able to use a termenum/termdocs process to count them. TermDocs s

Counting term frequency without using Explanation

2007-02-07 Thread csahat
Hi all, I'm so sorry if this question already answered before in this list, but I already search the list, and I couldn't find the answer. This is what I want to do : When the user type in the query, for example "WebSphere Java", Lucene will show not only the score, but showing the term cou

Re: An arguable bug in Lucene 1.9.1

2007-02-07 Thread Russ
Sorry, didn't mean for this to go to the list... Russ Sent wirelessly via BlackBerry from T-Mobile. -Original Message- From: "Russ" <[EMAIL PROTECTED]> Date: Wed, 7 Feb 2007 13:56:41 To:java-user@lucene.apache.org Subject: Re: An arguable bug in Lucene 1.9.1 You're not Gary Lee that

Re: An arguable bug in Lucene 1.9.1

2007-02-07 Thread Russ
You're not Gary Lee that went to poly, are you? Sent wirelessly via BlackBerry from T-Mobile. -Original Message- From: [EMAIL PROTECTED] Date: Mon, 5 Feb 2007 19:38:03 To: Subject: RE: An arguable bug in Lucene 1.9.1 I am seeing this issue as well with the exact same stack trace using

How to return results with null values?

2007-02-07 Thread poeta simbolista
HI I would like to make a query and take out those with null values in a specific field. How to? Thanks -- View this message in context: http://www.nabble.com/How-to-return-results-with-null-values--tf3186799.html#a8844779 Sent from the Lucene - Java Users mailing list archive at Nabble.com. -

Re: How to return results with null values?

2007-02-07 Thread Don . Reese
Sorry, under the weather today. Meant to say, don't put fields with null values into the index. You do not have to have a field for the document if that field has no value. -- Don Reese BIPT PO Box 20159 5971 Cattleridge Blvd Ste 101 Sarasota, FL 34276 [EMAIL PROTECTED] 941-358-8004 x139 [E

Re: How to return results with null values?

2007-02-07 Thread Otis Gospodnetic
Can't do that, I'm afraid. When indexing, use some unique value to represent the no value (e.g. __system__null__value__ - just made this up, pick whatever, just make sure it doesn't conflict with a real value and that it's not altered by the tokenizer). Otis - Original Message From:

Re: How to return results with null values?

2007-02-07 Thread Don . Reese
My guess is that if the field has a null empty, don't put the field in the index. poeta simbolista <[EMAIL PROTECTED]> 02/07/2007 07:50 AM Please respond to java-user@lucene.apache.org To java-user@lucene.apache.org cc Subject How to return results with null values? HI I would like

Re: deviations

2007-02-07 Thread poeta simbolista
Thank you very much. Actually, I said size as an example, as i could have said any other field: the thing has to work for any numerical field. That means, I cannot special-case the problem. Cheers Grant Ingersoll-6 wrote: > > Have a look at the FieldSelector stuff, assuming you are using trunk