github-actions[bot] commented on code in PR #64667:
URL: https://github.com/apache/doris/pull/64667#discussion_r3575796344
##########
fe/fe-catalog/src/main/java/org/apache/doris/analysis/InvertedIndexProperties.java:
##########
@@ -87,7 +91,8 @@ public static String getInvertedIndexParserMode(Map<String,
String> properties)
}
return mode != null ? mode :
INVERTED_INDEX_PARSER_IK.equals(parser) ?
INVERTED_INDEX_PARSER_SMART :
- INVERTED_INDEX_PARSER_COARSE_GRANULARITY;
Review Comment:
This adds the Kuromoji default for the `parser` path, but the PR also makes
`PROPERTIES("analyzer"="kuromoji")` FE-valid via
`IndexPolicy.BUILTIN_ANALYZERS`. That accepted analyzer-only path has no
`parser` key, so this helper still falls through to `coarse_grained`; BE has
the same parser-only defaulting before
`create_builtin_analyzer(PARSER_KUROMOJI, ...)`, and
`kuromoji_mode_from_string("coarse_grained")` rejects it during index
build/TOKENIZE/MATCH. Please either reject analyzer-only `kuromoji` and require
`parser="kuromoji"`, or make the analyzer key participate in the Kuromoji mode
default/validation and add coverage for omitted and explicit modes on
`analyzer="kuromoji"`.
##########
fe/fe-catalog/src/main/java/org/apache/doris/analysis/InvertedIndexProperties.java:
##########
@@ -87,7 +91,8 @@ public static String getInvertedIndexParserMode(Map<String,
String> properties)
}
return mode != null ? mode :
INVERTED_INDEX_PARSER_IK.equals(parser) ?
INVERTED_INDEX_PARSER_SMART :
- INVERTED_INDEX_PARSER_COARSE_GRANULARITY;
+ INVERTED_INDEX_PARSER_KUROMOJI.equals(parser) ?
INVERTED_INDEX_PARSER_KUROMOJI_SEARCH :
Review Comment:
This adds the Kuromoji default for the `parser` path, but the PR also makes
`PROPERTIES("analyzer"="kuromoji")` FE-valid via
`IndexPolicy.BUILTIN_ANALYZERS`. That accepted analyzer-only path has no
`parser` key, so this helper still falls through to `coarse_grained`; BE has
the same parser-only defaulting before
`create_builtin_analyzer(PARSER_KUROMOJI, ...)`, and
`kuromoji_mode_from_string("coarse_grained")` rejects it during index
build/TOKENIZE/MATCH. Please either reject analyzer-only `kuromoji` and require
`parser="kuromoji"`, or make the analyzer key participate in the Kuromoji mode
default/validation and add coverage for omitted and explicit modes on
`analyzer="kuromoji"`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
