[
https://issues.apache.org/jira/browse/SOLR-2628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13058241#comment-13058241
]
Dawid Weiss commented on SOLR-2628:
-----------------------------------
I've talked about it a little bit with Bernd and indeed, it seems possible to
reduce the size of in-memory data structures by an order of magnitude (or even
two orders of magnitude, we shall see). I'm on vacation for the next week and
on a business trip for another one after that, but I'll be on it once I come
back home.
> use of FST for SynonymsFilterFactory and synonyms.txt
> -----------------------------------------------------
>
> Key: SOLR-2628
> URL: https://issues.apache.org/jira/browse/SOLR-2628
> Project: Solr
> Issue Type: New Feature
> Components: Schema and Analysis
> Affects Versions: 3.4, 4.0
> Environment: Linux
> Reporter: Bernd Fehling
> Assignee: Dawid Weiss
> Priority: Minor
> Labels: suggestion
>
> Currently the SynonymsFilterFactory builds up a memory based SynonymsMap.
> This can generate huge maps because of the permutations for synonyms.
> Now where FST (finite state transducer) is introduced to lucene this could
> also be used for synonyms.
> A tool can compile the synoynms.txt file to a binary automaton file which can
> then be used
> with SynoynmsFilterFactory.
> Advantage:
> - faster start of solr, no need to generate SynonymsMap
> - faster lookup
> - memory saving
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
