[
https://issues.apache.org/jira/browse/LUCENE-5437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14000274#comment-14000274
]
Ahmet Arslan commented on LUCENE-5437:
--------------------------------------
ASCIIFoldingFilter's preserveOriginal=true breaks wildcard queries, since it is
a MultiTermAwareComponent.
{code:xml}
<fieldType name="text_ascii_preserve" class="solr.TextField"
positionIncrementGap="100">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.ASCIIFoldingFilterFactory" preserveOriginal="true"/>
</analyzer>
</fieldType>
{code}
With the above type, {{q=manu:Belkı*}} yields:
{code}
"error": {
"msg": "analyzer returned too many terms for multiTerm term: Belkı",
"code": 400
}
{code}
I think preserveOriginal is dangerous for token filters that implement
MultiTermAwareComponent.
What is the preferred action here? Document this limitation/behavior? Or
consider this as a bug and open a jira?
> ASCIIFoldingFilter that emits both unfolded and folded tokens
> -------------------------------------------------------------
>
> Key: LUCENE-5437
> URL: https://issues.apache.org/jira/browse/LUCENE-5437
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/analysis
> Affects Versions: 4.7, 5.0
> Reporter: Nik Everett
> Assignee: Simon Willnauer
> Priority: Minor
> Fix For: 4.7, 5.0
>
> Attachments: LUCENE-5437.patch
>
>
> I've found myself wanting an ASCIIFoldingFilter that emits both the folded
> tokens and the original, unfolded tokens.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
