yiguolei commented on code in PR #64667:
URL: https://github.com/apache/doris/pull/64667#discussion_r3464089336
##########
be/src/storage/index/inverted/analyzer/analyzer.cpp:
##########
@@ -107,6 +109,11 @@ AnalyzerPtr
InvertedIndexAnalyzer::create_builtin_analyzer(InvertedIndexParserTy
ik_analyzer->setMode(false);
}
analyzer = std::move(ik_analyzer);
+ } else if (parser_type == InvertedIndexParserType::PARSER_KUROMOJI) {
+ auto kuromoji_analyzer = std::make_shared<KuromojiAnalyzer>();
+ kuromoji_analyzer->initDict(config::inverted_index_dict_path +
"/kuromoji");
Review Comment:
there should be a config in be.conf to enable_KUROMOJI_analyzer == false by
default. Because it is very dangerous to loading dict and may occupy a lot of
memory.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
