Vector space implemantion

Hello all, I'm trying to implement a vector space model using lucene. I need to have a file (or on memory) with TF/IDF weight of each term in each document. (in fact that is a matrix with documents presented as vectors, in which the elements of each vector is the TF weight ...) Please Please h

Re: Lucene searching across documents

Hello all, I'm trying to implement a vector space model using lucene. I need to have a file (or on memory) with TF/IDF weight of each term in each document. (in fact that is a matrix with documents presented as vectors, in which the elements of each vector is the TF weight ...) Please Please

Re: Wordnet indexing error

Hi, The simplest thing to do is to grab the latest Lucene and the latest jar for that Wordnet (syns2index) code. That should work for you (that UnIndexed method is an old method that doesn't exist any more). Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original

Re: Suggestive Search

I test the lucene spellchecker and it doesn't support chinese spell checker, how can i achieve this goal as google does? 2009/4/9 Karl Wettin > If you use prefix grams only then you'll get a forward-only suggestion > scheme. I've seen several implementation that use that and it works quite > wel

Re: Query any data

first you should change your querypaser to accept wildcard query by calling method of QueryParser setAllowLeadingWildcard then you can query like this: fieldname:* 2009/4/9 Tim Williams > On Wed, Apr 8, 2009 at 11:45 AM, addman wrote: > > > > Hi, > > Is it possible to create a quer

Re: How can I change that lucene use by default the AND operator between terms ???

call method of QueryParser setDefaultOperator 2009/4/8 Uwe Schindler > The query parser has a option to change that. After creating the query > parser, just set the corresponding option before parsing the query. > > Uwe > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://ww

Re: How Can I make an analyzer that ignore the numbers o the texts ???

Steven A Rowe wrote: Hi Ariel, As Koji mentioned, https://issues.apache.org/jira/browse/SOLR-448 contains a NumberFilter. It filters out tokens that successfully parse as Doubles. I'm not sure, since the examples you gave seem to use "," as the decimal character, how this interacts with the

Re: Lucene help with query

If you omit norms when indexing the name field, you'll get same score back. Koji The Seer wrote: Hello, I have 5 lucene documents name: Apple name: Apple martini name: Apple drink name: Apple sweet drink I am using lucene default similarity and standard analyzer . When I am searching for

Wordnet indexing error

Hi All, We're using Lucene 2.3.2 on Windows. When we try to generate index for WordNet2.0 using Syns2Index class, while indexing, the following error is thrown: Java.lang.NoSuchMethodError: org.apache.lucene.document.Field.UnIndexed(Ljava/lang/String;Ljava/lang/ String;)Lorg/apache/lucene/documen

Re: Suggestive Search

If you use prefix grams only then you'll get a forward-only suggestion scheme. I've seen several implementation that use that and it works quite well. harry potter: ^ha, ^har, ^harr, ^harry, ^harry p, ^harry po.. harry houdini: ^ha, ^har, ^harr, ^harry, ^harry h, ^harry ho.. I prefere the tr

Re: Lucene help with query

Any ideas? John Seer wrote: > > Hello, > > I have 5 lucene documents > > name: Apple > name: Apple martini > name: Apple drink > name: Apple sweet drink > > I am using lucene default similarity and standard analyzer . > > When I am searching for apple I am getting all 4 documents with th

Re: Lucene 1.4.3: Error when creating Searcher

Yes, no locking is done anymore (as of 2.1) when opening IndexSearchers. So, it's fine. Though... if they are within a single JRE, it's best to open a single IndexSearcher and share. Mike On Wed, Apr 8, 2009 at 5:14 PM, Zhang, Lisheng wrote: > Hi, > > So it is always OK in 2.4 when multiple ja

RE: Lucene 1.4.3: Error when creating Searcher

Hi, So it is always OK in 2.4 when multiple java processes try to create IndexerSearcher at the same time? Just want to make sure. I think upgrading should be the best option. Thanks very much for helps, Lisheng -Original Message- From: Michael McCandless [mailto:luc...@mikemccandless

Lucene help with query

Hello, I have 5 lucene documents name: Apple name: Apple martini name: Apple drink name: Apple sweet drink I am using lucene default similarity and standard analyzer . When I am searching for apple I am getting all 4 documents with the same score back. If I use hits the score is 1.0 if I use

Re: Lucene 1.4.3: Error when creating Searcher

Ahh yes right. If multiple IndexSearchers are trying to open at once, they each try to acquire the commit lock and can thus starve one another. The simplest workaround is to just keep retrying opening the IndexSearcher. Though if you accidentally get an orphan'd commit lock in the directory (eg

RE: Lucene 1.4.3: Error when creating Searcher

Hi, Client said they did not index, all they do is searching (create Searcher objects), I looked at 1.4.3 and think this issue can happen in: private static IndexReader open(final Directory directory, final boolean closeDirectory) syncronized(directory) {...} if calls are coming from differe

Re: Lucene 1.4.3: Error when creating Searcher

Likely your exception happened because a reader was trying to open just as a writer was committing, twice in a row. Do you commit (flush or close) frequently from your writer? As of 2.1, Lucene no longer uses a commit locks -- commits are now lockless, so you won't hit this on upgrading to 2.4.

RE: Lucene 1.4.3: Error when creating Searcher

Hi, Sorry that my initial message is not clear, I read lucene source code (both 1.4.3 and 2.4.0), and understood more. The problem is that when using lucene 1.4.3 sometimes when searching, we got the error: Lock obtain timed out: Lock@/tmp/lucene-ba94511756a2670adeac03a50532c63c-commit.lock

Re: Suggestive Search

I use TermEnum for this sort of "browsing" on untokenized, unstored fields e.g. TermEnum terms=reader.terms(new Term("mybrowsefld","harr")). -Gary Matt Schraeder wrote: Corerct me if I'm wrong, but I don't think n-grams is really what I'm looking for here. I'm not looking for a spellchecker or

Re: Suggestive Search

Corerct me if I'm wrong, but I don't think n-grams is really what I'm looking for here. I'm not looking for a spellchecker or phrase checker style suggestive search, but only based on the exact phrases the user is currently typing. Since Lucene uses term-based searching, I'm not sure how to have

RE: How Can I make an analyzer that ignore the numbers o the texts ???

Hi Ariel, As Koji mentioned, https://issues.apache.org/jira/browse/SOLR-448 contains a NumberFilter. It filters out tokens that successfully parse as Doubles. I'm not sure, since the examples you gave seem to use "," as the decimal character, how this interacts with the Locale. (Koji, I don'

Re: How Can I make an analyzer that ignore the numbers o the texts ???

Ariel wrote: Hi everybody: I would want to know how Can I make an analyzer that ignore the numbers o the texts like the stop words are ignored ??? For example that the terms : 3.8, 100, 4.15, 4,33 don't be added to the index. How can I do that ??? Regards Ariel There is a patch for filter

Re: How Can I make an analyzer that ignore the numbers o the texts ???

You can define your own STOP_LIST and pass it in as a constructor to most analyzers. For example from the Lucene Javadocs: StandardAnalyzer public *StandardAnalyzer*(String [] stopWords) Builds an analyzer with the given stop

Re: Suggestive Search

For this you probably want to use ngrams. Wether or not this is something that fits in your current index is hard to say. My guess is that you want to create a new index with one document per unique phrase. You might also want to try to load this index in an InstantiatedIndex, that could sp

How Can I make an analyzer that ignore the numbers o the texts ???

Hi everybody: I would want to know how Can I make an analyzer that ignore the numbers o the texts like the stop words are ignored ??? For example that the terms : 3.8, 100, 4.15, 4,33 don't be added to the index. How can I do that ??? Regards Ariel

RE: Lucene searching across documents

Hi Dan, My guess, though you didn't directly say so, is that you're representing each sentence/"line" as a separate Lucene document. To directly answer your question about whether inter-document relations (like database joins) are queryable in Lucene, I don't think so, other than performing mu

Lucene 1.4.3: Error when creating Searcher

Hi, We are using lucene 1.4.3, sometimes when two threads try to search, one thread got error when creating MultiSearcher: Lock obtain timed out: Lock@/tmp/lucene-ba94511756a2670adeac03a50532c63c-commit.lock I read lucene FAQ and searched previous discussions, it seems that this error should be

Re: Query any data

On Wed, Apr 8, 2009 at 11:45 AM, addman wrote: > > Hi, > Is it possible to create a query to search a field for any value? I just > need to know if the optional field contain any data at all. google for: lucene field existence There's no way built in, one strategy[1] is to have a 'meta field

Query any data

Hi, Is it possible to create a query to search a field for any value? I just need to know if the optional field contain any data at all. -- View this message in context: http://www.nabble.com/Query-any-data-tp22953431p22953431.html Sent from the Lucene - Java Users mailing list archive at Na

Re: How to customize score according to field value?

Hi, Yes, the CustomScoreQuery.customScore() can meet the requirement I described. Thank you all! On Tue, Apr 7, 2009 at 9:01 PM, Tim Williams wrote: > On Tue, Apr 7, 2009 at 3:08 AM, Jinming Zhang > wrote: > > Hi, > > > > I have the following situation which needs to customize the final score

RE: How can I change that lucene use by default the AND operator between terms ???

The query parser has a option to change that. After creating the query parser, just set the corresponding option before parsing the query. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Ariel [mailt

How can I change that lucene use by default the AND operator between terms ???

When I do a search using lucene internally lucene use by default the OR operator between terms, How can I change that lucene use by default the AND operator between terms ??? Regards Ariel

Lucene searching across documents

So I have a requirement where I have a directory filled with xml files. I wrote a parser to parse these files, and index all of the xml attributes and properties into documents. An example of one of these documents is below. I'm parsing sentences into words, and tagging the sentences based on certa

Suggestive Search

I want to add a suggestive search similar to google's to autocomplete search phrases as the user types. It doesn't have to be very elaborate and for the most part will just involve searching single fields. How can I perform a search to be able to fill in autocomplete text? For instance, if I s

Re: test

Can you provide more details? EG a full exception? What was the app doing (indexing, searching, both)? Mike On Wed, Apr 8, 2009 at 2:40 AM, Antony Joseph wrote: > Hi, > > In a long running process Lucene get crashed in my application, Is there any > way to diagnose or how can I  turn on debug