Yulei-Yang commented on code in PR #13914:
URL: https://github.com/apache/doris/pull/13914#discussion_r1014648987
##########
be/src/vec/functions/function_string.h:
##########
@@ -1278,63 +1278,58 @@ class FunctionSplitPart : public IFunction {
auto str = str_col->get_data_at(i);
if (delimiter.size == 0) {
StringOP::push_empty_string(i, res_chars, res_offsets);
- } else if (delimiter.size == 1) {
- // If delimiter is a char, use memchr to split
- int32_t pre_offset = -1;
- int32_t offset = -1;
- int32_t num = 0;
- while (num < part_number) {
- pre_offset = offset;
- size_t n = str.size - offset - 1;
- const char* pos = reinterpret_cast<const char*>(
- memchr(str.data + offset + 1, delimiter_str[0],
n));
- if (pos != nullptr) {
- offset = pos - str.data;
- num++;
- } else {
- offset = str.size;
- num = (num == 0) ? 0 : num + 1;
- break;
- }
- }
-
- if (num == part_number) {
- StringOP::push_value_string(
- std::string_view {
- reinterpret_cast<const char*>(str.data +
pre_offset + 1),
- (size_t)offset - pre_offset - 1},
- i, res_chars, res_offsets);
- } else {
- StringOP::push_null_string(i, res_chars, res_offsets,
null_map_data);
- }
} else {
- // If delimiter is a string, use memmem to split
- int32_t pre_offset = -delimiter.size;
int32_t offset = -delimiter.size;
int32_t num = 0;
- while (num < part_number) {
- pre_offset = offset;
+ std::vector<int> find(str.size, -1); //store delimiter position
+ while (num < str.size) {
size_t n = str.size - offset - delimiter.size;
- char* pos = reinterpret_cast<char*>(memmem(str.data +
offset + delimiter.size,
- n,
delimiter.data, delimiter.size));
- if (pos != nullptr) {
- offset = pos - str.data;
- num++;
+ if (delimiter.size == 1) {
Review Comment:
> Keep origin logics. We should try to not make this if-else inside a loop
I get your point. It's about efficiency, if/else in a loop will prevent loop
unrolling. I update the code according to your opinion. Please review, thx in
advance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
