Scoring similarity by the position of the terms

2012-03-22 Thread Thomas Rewig
Hi, a few years ago, I tested this tutorial: http://sujitpal.blogspot.de/2010/10/custom-scoring-with-lucene-payloads.html Here the search are based on scoring the similarity only by the positions of the terms. For this you have to write your own TokenFilter, Analyser, PayloadFunction and Simi

Re: TermQuery - ExactMatching, Lucene 3.1.0 vs. 3.3.0, special character behavior

2011-07-19 Thread Thomas Rewig
ttp://www.thetaphi.de eMail: u...@thetaphi.de -----Original Message- From: Thomas Rewig [mailto:tre...@mufin.com] Sent: Monday, July 18, 2011 12:06 PM To: java-user@lucene.apache.org Subject: Re: TermQuery - ExactMatching, Lucene 3.1.0 vs. 3.3.0, special character behavior Hi Ian, yes the sc

Re: TermQuery - ExactMatching, Lucene 3.1.0 vs. 3.3.0, special character behavior

2011-07-18 Thread Thomas Rewig
n. On Fri, Jul 15, 2011 at 3:01 PM, Thomas Rewig wrote: Hello, there is a index with a lot of docs, 2 of them are: doc1: 1.Field=idITSVopfOLB=ITS---f0-- Value= 192 2.Field=name ITSVopfOLB=ITS0-- Value= queen doc2: 1.Field=idITSVopfOLB=ITS---

TermQuery - ExactMatching, Lucene 3.1.0 vs. 3.3.0, special character behavior

2011-07-15 Thread Thomas Rewig
Hello, there is a index with a lot of docs, 2 of them are: doc1: 1.Field=idITSVopfOLB=ITS---f0-- Value= 192 2.Field=name ITSVopfOLB=ITS0-- Value= queen doc2: 1.Field=idITSVopfOLB=ITS---f0-- Value= 701492 2.Field=name ITSVopfOLB=ITS0-- V

name matching / mapping

2011-07-06 Thread Thomas Rewig
Hello, until now, we use a stupid %like% SQL query script to assign the following terms for Id/Item mapping in different id-spaces: john wayne == john wayne wayne, john == john wayne I can imagine that Lucene offers much more possibilities for this assignment. Maybe with Lucene is also pos

Check Numeric Fields

2011-03-11 Thread Thomas Rewig
Hello, I use a index with a numeric field: /doc.add( new Field(...));/ /doc.add( new Field(...));/ /doc.add(new NumericField(fieldName,Field.Store.YES,true)/ / .setIntValue(intFieldValue));/ /indexWriter.addDocument(doc);/

Re: Deleted File Handles - Index Writer

2010-11-19 Thread Thomas Rewig
Hi. If I set writer.setUseCompoundFile(false); than I see a short time the 35 handles and than it drops to zero. Hmm what's happening during that short time? Do you have a reader open? Yes after a commit has been initiated, a short time a reader is still open and that's ok. This behavior is

Re: Deleted File Handles - Index Writer

2010-11-18 Thread Thomas Rewig
help. Thomas I've found a case, only with compound file, where IndexWriter holds open a SegmentReader on the pre-compound-file files... I'm working on a test case& fix. Mike On Fri, Nov 12, 2010 at 5:49 AM, Thomas Rewig wrote: Hello, I use the searcherManager for LiveIndexin

Deleted File Handles - Index Writer

2010-11-12 Thread Thomas Rewig
Hello, I use the searcherManager for LiveIndexing. With watch -n 60 "lsof | grep indexname | grep deleted | wc -l" I see the number of deleted file handles. The number of handles fluctuates during the indexing. 0 -> 35 -> 53 -> 135 -> 40 -> 85 ... Uwe said that this is expected because seg

Re: File Handle Leaks During Lucene 3.0.2 Merge

2010-11-10 Thread Thomas Rewig
Hello, please excuse that I hijack this old thread but I have the same problem with the deleted file handles, so I think this is the right place for. I also integrated the searchManager in our Code and see the file handles fluctuate up and down. At first glance the situation seems stable but

IndexSearcher - open file handles by deleted files

2010-05-26 Thread Thomas Rewig
Hello, I use Lucene 2.9.1 with two indices, which alternate each day. One is live, the other is erased and renewed with the latest data. The problem is that the index files will be deleted, but the file handles are still available. If the program (JBOSS) is not restarted for some time, the dis

Re: Problems with ItemBasedRecommender with Lucene

2009-09-17 Thread Thomas Rewig
Oh, I overlooked the simplest way to do that. You're right, tokens are the key to this problem. It works pretty well. It would be perfect if I use payloads. I read your advice http://www.lucidimagination.com/blog/category/payloads/. You store the payloads with your PayLoadAnalyzer in this way:

Problems with ItemBasedRecommender with Lucene

2009-09-16 Thread Thomas Rewig
Hello, I build a "real time ItemBasedRecommender" based on a users history and a (sparse) item similarity matrix with lucene. Some time ago Ted Dunning recommended me this approach at the mahout mailing list to create a ItemBasedRecommender: "It is actually very easy to do. The output of the