Hello,
I'm trying to implement a weighed synonyms analyzer using
SynonymGraphFilter and a custom payload analyzer.
My synonyms file looks like:
obama,president^0.9,barry^0.5
speaks,greets^0.8,congratulates^0.9,informs
media,press,journal,tv^0.8
I want user searching for "obama greets press" to find documents containing
"president speaks to media".
I want to keep the original "phrase" at the heighest score, and synonyms to
score less.
The problem is that my Synonyms Filter Could not undestand weights although
I passed the matching analyzer in the SynonymGraphFilter constructor.
Here is my createComponents method:
protected TokenStreamComponents createComponents(String fieldName) {
final KeywordTokenizer src = new KeywordTokenizer();
TokenStream tok = new LowerCaseFilter(src);
try {
tok = new SynonymGraphFilter(tok,
loadSynonyms(new FilesystemResourceLoader(Paths.get("/wsliat/lucene-res/")),
SolrSynonymParser.class.getName(), false, new PayloadAwareAnalyzer()),
true);
} catch (Exception e) {
e.printStackTrace();
}
tok = new DelimitedPayloadTokenFilter(tok, '^', this.weightEncoder);
return new TokenStreamComponents(src, tok) {
@Override
protected void setReader(final Reader reader) {
super.setReader(reader);
}
};
}
My PayloadAwareAnalyzer looks like this:
public class PayloadAwareAnalyzer extends Analyzer {
private PayloadEncoder weightEncoder;
public PayloadAwareAnalyzer(PayloadEncoder weightEncoder) {
this.weightEncoder = weightEncoder;
}
public PayloadAwareAnalyzer() {
super();
this.weightEncoder = new FloatEncoder();
}
@Override
protected TokenStreamComponents createComponents(String fieldName) {
final KeywordTokenizer src = new KeywordTokenizer();
TokenStream tok = new DelimitedPayloadTokenFilter(src, '^',
this.weightEncoder);
return new TokenStreamComponents(src, tok) {
@Override
protected void setReader(final Reader reader) {
super.setReader(reader);
}
};
}
}
I also added the according custom similarity for the searcher.
Any help would be much appreciated.
Thanks.
