Yulei-Yang commented on code in PR #13914:
URL: https://github.com/apache/doris/pull/13914#discussion_r1013668593
##########
be/src/vec/functions/function_string.h:
##########
@@ -1278,63 +1278,58 @@ class FunctionSplitPart : public IFunction {
auto str = str_col->get_data_at(i);
if (delimiter.size == 0) {
StringOP::push_empty_string(i, res_chars, res_offsets);
- } else if (delimiter.size == 1) {
- // If delimiter is a char, use memchr to split
- int32_t pre_offset = -1;
- int32_t offset = -1;
- int32_t num = 0;
- while (num < part_number) {
- pre_offset = offset;
- size_t n = str.size - offset - 1;
- const char* pos = reinterpret_cast<const char*>(
- memchr(str.data + offset + 1, delimiter_str[0],
n));
- if (pos != nullptr) {
- offset = pos - str.data;
- num++;
- } else {
- offset = str.size;
- num = (num == 0) ? 0 : num + 1;
- break;
- }
- }
-
- if (num == part_number) {
- StringOP::push_value_string(
- std::string_view {
- reinterpret_cast<const char*>(str.data +
pre_offset + 1),
- (size_t)offset - pre_offset - 1},
- i, res_chars, res_offsets);
- } else {
- StringOP::push_null_string(i, res_chars, res_offsets,
null_map_data);
- }
} else {
- // If delimiter is a string, use memmem to split
- int32_t pre_offset = -delimiter.size;
int32_t offset = -delimiter.size;
int32_t num = 0;
- while (num < part_number) {
- pre_offset = offset;
+ std::vector<int> find(str.size, -1); //store delimiter position
+ while (num < str.size) {
size_t n = str.size - offset - delimiter.size;
- char* pos = reinterpret_cast<char*>(memmem(str.data +
offset + delimiter.size,
- n,
delimiter.data, delimiter.size));
- if (pos != nullptr) {
- offset = pos - str.data;
- num++;
+ if (delimiter.size == 1) {
Review Comment:
Thank you very much for your review. While I have a question here. If I keep
origin logic, there will be two parts of code which looks similar and seems
duplicated, like line 1282-1284 and 1317-1333。Can you please give me an advice
which issue is more critical? @Gabriel39
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
