[
https://issues.apache.org/jira/browse/LUCENE-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12873010#action_12873010
]
Robert Muir commented on LUCENE-2458:
-------------------------------------
i would suggest the following, a combination of mike, mark's, and yonik's ideas:
* in 3.1 we supply a patch that looks like this one, except, there is a toggle
too. this toggle can be per-field.
* in 4.0 we do the same thing, for now.
we open a separate issue where we replace the QP with something better (that
does not split on whitespace
at all and allows multi-word syns, n-gram tokenization, vietnamese, etc to
work)
as part of that issue take the existing one (with per-field toggle) and call it
classicqueryparser or whatever.
i think we should consider the second separate issue carefully and take more
time. I personally would prefer
if somehow we could "tone down" the flexible queryparser (as far as number of
classes, attributes, etc) and
use that as a base. There are things about it I like, and things I don't like,
but overall it seems to be a better
starting point for such a thing.
> queryparser makes all CJK queries phrase queries regardless of analyzer
> -----------------------------------------------------------------------
>
> Key: LUCENE-2458
> URL: https://issues.apache.org/jira/browse/LUCENE-2458
> Project: Lucene - Java
> Issue Type: Bug
> Components: QueryParser
> Reporter: Robert Muir
> Assignee: Robert Muir
> Priority: Blocker
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2458.patch, LUCENE-2458.patch, LUCENE-2458.patch
>
>
> The queryparser automatically makes *ALL* CJK, Thai, Lao, Myanmar, Tibetan,
> ... queries into phrase queries, even though you didn't ask for one, and
> there isn't a way to turn this off.
> This completely breaks lucene for these languages, as it treats all queries
> like 'grep'.
> Example: if you query for f:abcd with standardanalyzer, where a,b,c,d are
> chinese characters, you get a phrasequery of "a b c d". if you use cjk
> analyzer, its no better, its a phrasequery of "ab bc cd", and if you use
> smartchinese analyzer, you get a phrasequery like "ab cd". But the user
> didn't ask for one, and they cannot turn it off.
> The reason is that the code to form phrase queries is not internationally
> appropriate and assumes whitespace tokenization. If more than one token comes
> out of whitespace delimited text, its automatically a phrase query no matter
> what.
> The proposed patch fixes the core queryparser (with all backwards compat
> kept) to only form phrase queries when the double quote operator is used.
> Implementing subclasses can always extend the QP and auto-generate whatever
> kind of queries they want that might completely break search for languages
> they don't care about, but core general-purpose QPs should be language
> independent.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
