Lucene and TREC

2015-04-29 Thread Ralf Bierig
Hi all, I am using Lucene and would like to interface to the TREC_EVAL tool so that, by generating my search results in the TREC_EVAL format. I can use the tool to calculate evaluation measures. I came across the lucene-benchmark package (http://maven-repository.com/artifact/org.apache.lucen

Analyzer: Access to document?

2015-02-04 Thread Ralf Bierig
Hi all, an Analyzer has access to content on a per-field level by overwriting this method: protected TokenStreamComponents createComponents(String fieldName, Reader reader); Is it possible to get to the document? I want to collect the text content from the entire document within my analyze

Re: Payload and Similarity Function: Always same value

2014-10-30 Thread Ralf Bierig
imple variable aliasing issue that I don't see at a glance ... -Mike On 10/30/14 2:03 PM, Ralf Bierig wrote: I want to implement a Lucene Indexer/Searcher that uses the new Payload feature to add meta information to tokens. I specifically add weights (i.e. 0-100) to conceptual tags in order t

Re: Payload and Similarity Function: Always same value

2014-10-30 Thread Ralf Bierig
imple variable aliasing issue that I don't see at a glance ... -Mike On 10/30/14 2:03 PM, Ralf Bierig wrote: I want to implement a Lucene Indexer/Searcher that uses the new Payload feature to add meta information to tokens. I specifically add weights (i.e. 0-100) to conceptual tags in order t

Payload and Similarity Function: Always same value

2014-10-30 Thread Ralf Bierig
I want to implement a Lucene Indexer/Searcher that uses the new Payload feature to add meta information to tokens. I specifically add weights (i.e. 0-100) to conceptual tags in order to use them to overwrite the standard Lucene TF-IDF weighting. I am puzzled by the behaviour of this and I belie

Re: MyAnalyzer and Lucene version <= 4.9.1

2014-10-28 Thread Ralf Bierig
://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Ralf Bierig [mailto:ralf.bie...@gmail.com] Sent: Tuesday, October 28, 2014 3:07 PM To: Lucene User List Subject: MyAnalyzer and Lucene version <= 4.9.1 How to write an own Analyzer in Lucene <= 4.9.1? Here my code, somehow the

MyAnalyzer and Lucene version <= 4.9.1

2014-10-28 Thread Ralf Bierig
How to write an own Analyzer in Lucene <= 4.9.1? Here my code, somehow the method tokenStream is not final and cannot be extended any more. How is one supposed to extend it? --- code --- class PayloadAnalyzer extends Analyzer { private PayloadEncoder encoder; PayloadAnalyzer(PayloadEn

Re: Indexing Weighted Tags per Document

2014-10-28 Thread Ralf Bierig
t you also can achieve score normalisation with keywords and amongst tags, and you can also filter results by probability. (3) There potentially is also a solution using child documents and block join, but I may be mistaken, haven't explored this a lot.. On 27 Oct 2014 16:10, "Ralf Bier

Indexing Weighted Tags per Document

2014-10-27 Thread Ralf Bierig
I want to index documents together with a list of tags (usually between 10-30) that represent meta information about this document. Normally, i would create an extra field "tag" store every tag, by its name, inside that field and create my 10-30 fields that and adding it to the document before

Weighted tags for document instances (at index time)

2014-10-27 Thread Ralf Bierig
I want to index documents together with a list of tags (usually between 10-30) that represent meta information about this document. Normally, i would create an extra field "tag" store every tag, by its name, inside that field and create my 10-30 fields that and adding it to the document before

How to modify the Lucene 4 index?

2013-09-17 Thread Ralf Bierig
Hi all, is there any good documentation of how to change and modify the index of Lucene version 4 other than what is already on the website? Blogs, papers, reports etc. or just a report on experience in some form --- anything would be good. Based on an early-stage project, I would like to ge