Re: [PR] [refactor](nereids)make NormalizeAggregate rule more clear and readable [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28607:
URL: https://github.com/apache/doris/pull/28607#issuecomment-1864025771

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit a28ee684c0d6d29e6351f7009441e8f4b9c60893, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4696442444784424
   q2   361 154 159 154
   q3   1463125212381238
   q4   1120906 894 894
   q5   3179314331633143
   q6   249 126 127 126
   q7   994 485 497 485
   q8   2208223121922192
   q9   6721668867106688
   q10  3233327832743274
   q11  304 184 181 181
   q12  353 211 205 205
   q13  4564381537913791
   q14  243 213 217 213
   q15  567 531 520 520
   q16  443 389 390 389
   q17  998 569 555 555
   q18  7120696969656965
   q19  1528139914011399
   q20  546 328 317 317
   q21  3074263627252636
   q22  347 283 280 280
   Total cold run time: 44311 ms
   Total hot run time: 40069 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4356435643354335
   q2   271 168 175 168
   q3   3533354535233523
   q4   2393240223842384
   q5   5751576057425742
   q6   241 124 125 124
   q7   2387187518791875
   q8   3543354235433542
   q9   9068900990709009
   q10  3929401240144012
   q11  493 371 379 371
   q12  776 599 595 595
   q13  4319355635633556
   q14  288 256 244 244
   q15  573 526 528 526
   q16  509 445 473 445
   q17  1897186418481848
   q18  8669815682928156
   q19  1742175717551755
   q20  2246192319311923
   q21  6589623162496231
   q22  511 423 416 416
   Total cold run time: 64084 ms
   Total hot run time: 60780 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [enhance](tablet) Reduce log in tablet meta [doris]

2023-12-20 Thread via GitHub


platoneko opened a new pull request, #28719:
URL: https://github.com/apache/doris/pull/28719

   ## Proposed changes
   
   Reduce log in tablet meta
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhance](tablet) Reduce log in tablet meta [doris]

2023-12-20 Thread via GitHub


platoneko commented on PR #28719:
URL: https://github.com/apache/doris/pull/28719#issuecomment-1864026859

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [branch-1.2](memory) Fix AggFunc memory leak due to incorrect destroy [doris]

2023-12-20 Thread via GitHub


xy720 commented on PR #28485:
URL: https://github.com/apache/doris/pull/28485#issuecomment-1864027589

   Not merge until we confirm that it will not cause new problem.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](meta_scanner) fix meta_scanner process ColumnNullable [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on PR #28711:
URL: https://github.com/apache/doris/pull/28711#issuecomment-1864029576

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [test](Nereids): add test for scalar agg PushDownFilterThroughAggregation [doris]

2023-12-20 Thread via GitHub


jackwener commented on PR #28712:
URL: https://github.com/apache/doris/pull/28712#issuecomment-1864029703

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](meta_scanner) fix meta_scanner process ColumnNullable [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on PR #28711:
URL: https://github.com/apache/doris/pull/28711#issuecomment-1864029523

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](Nereids): eliminate groupby [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on PR #28615:
URL: https://github.com/apache/doris/pull/28615#issuecomment-1864030794

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](partition) Comment out some exceptions with partition equal to … [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28702:
URL: https://github.com/apache/doris/pull/28702#issuecomment-1864032572

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.47 seconds
stream load tsv:  573 seconds loaded 74807831229 Bytes, about 124 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  29.0 seconds inserted 1000 Rows, about 
344K ops/s
storage size: 17188265442 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](tvf)Fixed the avro-scanner projection pushdown failing to query… [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28709:
URL: https://github.com/apache/doris/pull/28709#issuecomment-1864033308

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit 189064e3cdc5c488b00a689d665ba74caf39, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4664440444294404
   q2   361 154 154 154
   q3   1481124212211221
   q4   1117914 886 886
   q5   3169317531653165
   q6   247 127 126 126
   q7   1009487 484 484
   q8   2181222521772177
   q9   6674664866716648
   q10  3257328432773277
   q11  310 196 180 180
   q12  350 209 207 207
   q13  4557383438053805
   q14  244 211 212 211
   q15  568 517 530 517
   q16  441 397 389 389
   q17  1004612 558 558
   q18  7207687169006871
   q19  1516137014341370
   q20  496 322 320 320
   q21  3095265827312658
   q22  345 276 274 274
   Total cold run time: 44293 ms
   Total hot run time: 39902 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4322435343424342
   q2   265 165 179 165
   q3   3534352935423529
   q4   2393237023742370
   q5   5755574557375737
   q6   242 120 123 120
   q7   2381183518851835
   q8   3514353335163516
   q9   9010900989798979
   q10  3916402740024002
   q11  487 356 358 356
   q12  759 592 623 592
   q13  4296356935603560
   q14  293 258 261 258
   q15  580 520 517 517
   q16  508 451 469 451
   q17  1895186218811862
   q18  8578819782278197
   q19  1717176017501750
   q20  2266194319461943
   q21  6520616361626162
   q22  487 442 419 419
   Total cold run time: 63718 ms
   Total hot run time: 60662 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](limit) cancel running fragments when query with limit is finished [doris]

2023-12-20 Thread via GitHub


AshinGau commented on PR #28610:
URL: https://github.com/apache/doris/pull/28610#issuecomment-1864033882

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](scan) Implement parallel scanning by dividing the tablets based on either the row range or the key range. [doris]

2023-12-20 Thread via GitHub


yiguolei commented on code in PR #28103:
URL: https://github.com/apache/doris/pull/28103#discussion_r1418537513


##
be/src/olap/parallel_scanner_builder.cpp:
##
@@ -0,0 +1,170 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "parallel_scanner_builder.h"
+
+#include "olap/rowset/beta_rowset.h"
+#include "vec/exec/scan/new_olap_scanner.h"
+
+namespace doris {
+

Review Comment:
   70 tablet --> 1 tablet , other 1000 rows
   48 scanner
   1 tablet > 48 scanner
   69 tablet---> 69 scanner 
   
   117 scanner ---> queue/ stack
   
   1 scanner thread
   1 scanner --> 1000 block 2000rows 
   1 scanner
   2 scanner 
   
   scanner thread pool 
48
   
   10 scanner ---> queue/ stack
   



##
be/src/vec/exec/scan/new_olap_scan_node.cpp:
##
@@ -508,48 +503,87 @@ Status 
NewOlapScanNode::_init_scanners(std::list* scanners) {
 std::vector> 
tablets_to_scan;
 tablets_to_scan.reserve(_scan_ranges.size());
 
+std::vector tablets;
+
 for (auto&& scan_range : _scan_ranges) {
 auto tablet = DORIS_TRY(ExecEnv::get_tablet(scan_range->tablet_id));
 int64_t version = 0;
 std::from_chars(scan_range->version.data(),
 scan_range->version.data() + 
scan_range->version.size(), version);
+tablets.emplace_back(
+TabletWithVersion {std::dynamic_pointer_cast(tablet), 
version});
 tablets_to_scan.emplace_back(std::move(tablet), version);
 }
 
+bool is_dup_mow_key = true;
+bool unique_keys_with_mor = false;
+bool enable_parallel_scan = 
_state->query_options().__isset.enable_parallel_scan &&

Review Comment:
   add a function in runtime_state named enable_paralle_scan()  not use query 
optioins directly.



##
be/src/olap/parallel_scanner_builder.h:
##
@@ -0,0 +1,120 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include 
+#include 
+#include 
+
+#include "olap/reader.h"
+#include "olap/rowset/segment_v2/row_ranges.h"
+#include "olap/segment_loader.h"
+#include "olap/tablet.h"
+#include "vec/exec/scan/new_olap_scanner.h"
+
+namespace doris {
+
+namespace vectorized {
+class VScanner;
+}
+
+using TabletSPtr = std::shared_ptr;
+using VScannerSPtr = std::shared_ptr;
+
+struct TabletWithVersion {
+TabletSPtr tablet;
+int64_t version;
+};
+
+struct SegmentGroup {
+RowsetSharedPtr rowset;
+std::vector segments;
+size_t num_blocks;
+size_t num_rows;
+std::string min_key;
+std::string max_key;
+};
+
+template 
+class ParallelScannerBuilder {
+public:
+ParallelScannerBuilder(ParentType* parent, const 
std::vector& tablets,
+   const std::shared_ptr& profile,
+   const std::vector& key_ranges, 
RuntimeState* state,
+   int64_t limit_per_scanner, bool is_dup_mow_key, 
bool is_preaggregation)
+: _parent(parent),
+  _scanner_profile(profile),
+  _state(state),
+  _limit_per_scanner(limit_per_scanner),
+  _is_dup_mow_key(is_dup_mow_key),
+  _is_preaggregation(is_preaggregation),
+  _tablets(tablets.cbegin(), tablets.cend()),
+  _key_ranges(key_ranges.cbegin(), key_ranges.cend()) {}
+
+Status build_scanners(std::list& scanners);
+
+void set_max_scanners_count(size_t count) { _max_scanners_count = count; }
+
+private:
+Statu

Re: [PR] [refactor](nereids)make NormalizeAggregate rule more clear and readable [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28607:
URL: https://github.com/apache/doris/pull/28607#issuecomment-1864034415

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.08 seconds
stream load tsv:  565 seconds loaded 74807831229 Bytes, about 126 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  33 seconds loaded 861443392 Bytes, about 24 
MB/s
insert into select:  28.6 seconds inserted 1000 Rows, about 
349K ops/s
storage size: 17184008236 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](group commit)Fix the issue of duplicate addition of wal path when encouter exception [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on PR #28691:
URL: https://github.com/apache/doris/pull/28691#issuecomment-1864033782

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](meta_scanner) fix meta_scanner process ColumnNullable [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28711:
URL: https://github.com/apache/doris/pull/28711#issuecomment-1864034925

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.05 seconds
stream load tsv:  565 seconds loaded 74807831229 Bytes, about 126 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.8 seconds inserted 1000 Rows, about 
347K ops/s
storage size: 17183538110 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](partition) Add more log for single replica load when partition … [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28707:
URL: https://github.com/apache/doris/pull/28707#issuecomment-1864035017

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.77 seconds
stream load tsv:  573 seconds loaded 74807831229 Bytes, about 124 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  29.3 seconds inserted 1000 Rows, about 
341K ops/s
storage size: 17184123450 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](tvf)Fixed the avro-scanner projection pushdown failing to query… [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28709:
URL: https://github.com/apache/doris/pull/28709#issuecomment-1864035020

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.93 seconds
stream load tsv:  578 seconds loaded 74807831229 Bytes, about 123 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  30.2 seconds inserted 1000 Rows, about 
331K ops/s
storage size: 17184104018 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](Nereids) improve Shape check cases [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28124:
URL: https://github.com/apache/doris/pull/28124#issuecomment-1864035143

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.7 seconds
stream load tsv:  563 seconds loaded 74807831229 Bytes, about 126 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  33 seconds loaded 861443392 Bytes, about 24 
MB/s
insert into select:  28.4 seconds inserted 1000 Rows, about 
352K ops/s
storage size: 17184275923 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](partition) Comment out some exceptions with partition equal to … [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28702:
URL: https://github.com/apache/doris/pull/28702#issuecomment-1864036737

   TeamCity be ut coverage result:
Function Coverage: 36.47% (8534/23397) 
Line Coverage: 28.61% (69389/242555)
Region Coverage: 27.63% (35907/129950)
Branch Coverage: 24.37% (18353/75306)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/14344281303ae3b4fcd8ff250d40974603968801_14344281303ae3b4fcd8ff250d40974603968801/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](meta_scanner) fix meta_scanner process ColumnNullable [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28711:
URL: https://github.com/apache/doris/pull/28711#issuecomment-1864037091

   TeamCity be ut coverage result:
Function Coverage: 36.47% (8533/23398) 
Line Coverage: 28.61% (69383/242554)
Region Coverage: 27.63% (35902/129951)
Branch Coverage: 24.37% (18350/75302)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/defc83a2b258d83d5ee47c1edc3007833fb33efa_defc83a2b258d83d5ee47c1edc3007833fb33efa/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [FIX](explode) fix explode with array has specific precision at old planner [doris]

2023-12-20 Thread via GitHub


amorynan commented on PR #28491:
URL: https://github.com/apache/doris/pull/28491#issuecomment-1864034855

   run pipelinex_p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Bug](cooldown) Prevent making snapshot on remote rowset in single replica compaction [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on code in PR #28716:
URL: https://github.com/apache/doris/pull/28716#discussion_r1432382805


##
be/src/olap/tablet.cpp:
##
@@ -457,8 +475,24 @@ Status Tablet::add_rowset(RowsetSharedPtr rowset) {
 return Status::OK();
 }
 
+Status Tablet::rowsets_in_version_map_unlocked(const 
std::vector& rowsets) {

Review Comment:
   warning: method 'rowsets_in_version_map_unlocked' can be made static 
[readability-convert-member-functions-to-static]
   
   ```suggestion
   static Status Tablet::rowsets_in_version_map_unlocked(const 
std::vector& rowsets) {
   ```
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](scan) Implement parallel scanning by dividing the tablets based on either the row range or the key range. [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on PR #28103:
URL: https://github.com/apache/doris/pull/28103#issuecomment-1864035250

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [pipelineX](improvement) Support global rf [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28692:
URL: https://github.com/apache/doris/pull/28692#issuecomment-1864037065

   TeamCity be ut coverage result:
Function Coverage: 36.46% (8534/23407) 
Line Coverage: 28.59% (69384/242702)
Region Coverage: 27.61% (35904/130048)
Branch Coverage: 24.35% (18348/75348)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/53c40c8993b37b86265a8eb7b320c3b0109e022a_53c40c8993b37b86265a8eb7b320c3b0109e022a/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [imrovement](statistics)Auto analyze priority table/column first. [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on PR #28385:
URL: https://github.com/apache/doris/pull/28385#issuecomment-1864037750

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](Nereids) improve Shape check cases [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28124:
URL: https://github.com/apache/doris/pull/28124#issuecomment-1864040497

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit 1aae7d30781446ebf75987fa159cbed36f60dcd7, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4700444044204420
   q2   360 154 160 154
   q3   1443127412401240
   q4   1114924 887 887
   q5   3162315131663151
   q6   243 127 128 127
   q7   979 484 491 484
   q8   2175217421762174
   q9   6657663266786632
   q10  3237329032783278
   q11  309 188 181 181
   q12  351 212 202 202
   q13  4564382038223820
   q14  239 209 220 209
   q15  564 527 526 526
   q16  446 392 387 387
   q17  996 553 595 553
   q18  7088691568726872
   q19  1530135813741358
   q20  503 328 286 286
   q21  3006261527092615
   q22  357 276 279 276
   Total cold run time: 44023 ms
   Total hot run time: 39832 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4371432143214321
   q2   269 164 173 164
   q3   3554353135143514
   q4   2388237523872375
   q5   5740575957405740
   q6   240 123 118 118
   q7   2375186118741861
   q8   3508350135203501
   q9   9014903589958995
   q10  3917400540184005
   q11  482 375 361 361
   q12  770 612 593 593
   q13  4319360535353535
   q14  286 257 266 257
   q15  564 520 527 520
   q16  526 482 460 460
   q17  1885184318401840
   q18  8599808382768083
   q19  1715174717491747
   q20  2251194219291929
   q21  6544618061836180
   q22  494 424 416 416
   Total cold run time: 63811 ms
   Total hot run time: 60515 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Enhancement] (schema) add column type check [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on PR #28718:
URL: https://github.com/apache/doris/pull/28718#issuecomment-1864041284

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](nereids)subquery unnest need handle subquery in Not expr correnctly [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28713:
URL: https://github.com/apache/doris/pull/28713#issuecomment-1864044452

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.53 seconds
stream load tsv:  565 seconds loaded 74807831229 Bytes, about 126 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.7 seconds inserted 1000 Rows, about 
348K ops/s
storage size: 17183671451 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] Fix typo about checking the progress of building index [doris]

2023-12-20 Thread via GitHub


JaySon-Huang opened a new pull request, #28720:
URL: https://github.com/apache/doris/pull/28720

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](planner)should save original select list item before analyze [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28187:
URL: https://github.com/apache/doris/pull/28187#issuecomment-1864046217

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.11 seconds
stream load tsv:  567 seconds loaded 74807831229 Bytes, about 125 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.9 seconds inserted 1000 Rows, about 
346K ops/s
storage size: 17183302417 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]

2023-12-20 Thread via GitHub


yiguolei commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1864046917

   run compile


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [I] [Bug] jsonb 插入报错 [doris]

2023-12-20 Thread via GitHub


foundation1 commented on issue #26580:
URL: https://github.com/apache/doris/issues/26580#issuecomment-1864048031

   大佬这个问题我也遇到了 你是怎么解决的啊
   
![image](https://github.com/apache/doris/assets/28500506/e0b190bc-ebbc-433b-b0e2-13ee33d5785e)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](partition) Add more log for single replica load when partition … [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28707:
URL: https://github.com/apache/doris/pull/28707#issuecomment-1864048192

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit d7e321fefded775dd79079e6b0471b9c2862e87d, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4688441343974397
   q2   371 143 159 143
   q3   1465125011981198
   q4   1135931 906 906
   q5   3142318331613161
   q6   246 127 130 127
   q7   984 493 492 492
   q8   2204222722062206
   q9   6702667366896673
   q10  3208328832923288
   q11  306 185 179 179
   q12  352 207 206 206
   q13  4549385737793779
   q14  246 209 218 209
   q15  566 524 530 524
   q16  443 379 389 379
   q17  1016559 517 517
   q18  7209682268756822
   q19  1523132614281326
   q20  490 316 313 313
   q21  3109268726692669
   q22  339 281 278 278
   Total cold run time: 44293 ms
   Total hot run time: 39792 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4337431043254310
   q2   270 164 173 164
   q3   3536352935173517
   q4   2373236223662362
   q5   5872573957835739
   q6   246 121 120 120
   q7   2395183818321832
   q8   3531352535213521
   q9   9050897690368976
   q10  3920402340224022
   q11  494 367 367 367
   q12  759 586 610 586
   q13  4277360935373537
   q14  294 249 268 249
   q15  569 522 536 522
   q16  500 450 461 450
   q17  1889185918561856
   q18  8687826282678262
   q19  1728174817441744
   q20  2261196319581958
   q21  6538619861526152
   q22  502 423 423 423
   Total cold run time: 64028 ms
   Total hot run time: 60669 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Feature](inverted index) add lowercase option for inverted index analyzer [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28704:
URL: https://github.com/apache/doris/pull/28704#issuecomment-1864049970

   TeamCity be ut coverage result:
Function Coverage: 36.47% (8534/23398) 
Line Coverage: 28.60% (69376/242568)
Region Coverage: 27.63% (35903/129961)
Branch Coverage: 24.36% (18346/75312)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/aa07cfd3d140ce1597754d113e2a620f1b15603f_aa07cfd3d140ce1597754d113e2a620f1b15603f/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [DNM](Nereids) test disable fallback [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #25344:
URL: https://github.com/apache/doris/pull/25344#issuecomment-1864050429

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.94 seconds
stream load tsv:  566 seconds loaded 74807831229 Bytes, about 126 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.9 seconds inserted 1000 Rows, about 
346K ops/s
storage size: 17183459129 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhance](tablet) Reduce log in tablet meta [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on PR #28719:
URL: https://github.com/apache/doris/pull/28719#issuecomment-1864055392

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](Nereids) delete partition failed [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28717:
URL: https://github.com/apache/doris/pull/28717#issuecomment-1864055526

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.19 seconds
stream load tsv:  569 seconds loaded 74807831229 Bytes, about 125 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.9 seconds inserted 1000 Rows, about 
346K ops/s
storage size: 17184268142 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](meta_scanner) fix meta_scanner process ColumnNullable [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28711:
URL: https://github.com/apache/doris/pull/28711#issuecomment-1864056247

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit defc83a2b258d83d5ee47c1edc3007833fb33efa, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4683436744264367
   q2   358 164 161 161
   q3   1453126512351235
   q4   1101895 869 869
   q5   3152313131453131
   q6   252 126 126 126
   q7   973 485 479 479
   q8   2162221921932193
   q9   6703664366406640
   q10  3226325232823252
   q11  299 194 182 182
   q12  343 212 204 204
   q13  4574383037763776
   q14  245 214 211 211
   q15  568 527 523 523
   q16  438 393 382 382
   q17  989 586 537 537
   q18  7273696869626962
   q19  1519134714211347
   q20  540 311 308 308
   q21  3071264926442644
   q22  345 277 282 277
   Total cold run time: 44267 ms
   Total hot run time: 39806 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4366435543434343
   q2   265 167 170 167
   q3   3544354835303530
   q4   2380237223692369
   q5   5763576257435743
   q6   242 120 121 120
   q7   2379186318431843
   q8   3527351435183514
   q9   9035897090168970
   q10  3922402940034003
   q11  467 364 354 354
   q12  771 603 596 596
   q13  4321355235833552
   q14  293 271 255 255
   q15  569 515 520 515
   q16  530 502 452 452
   q17  1892186618331833
   q18  8602819983228199
   q19  1744175017521750
   q20  2238195219401940
   q21  6533616661586158
   q22  502 426 409 409
   Total cold run time: 63885 ms
   Total hot run time: 60615 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]

2023-12-20 Thread via GitHub


yiguolei commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1864057971

   run  compile


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]

2023-12-20 Thread via GitHub


yiguolei commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1864058219

   run p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](scanner) optimize the number of threads of scanners [doris]

2023-12-20 Thread via GitHub


AshinGau commented on PR #28640:
URL: https://github.com/apache/doris/pull/28640#issuecomment-1864060808

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]

2023-12-20 Thread via GitHub


yiguolei commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1864062232

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](nereids)subquery unnest need handle subquery in Not expr correnctly [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28713:
URL: https://github.com/apache/doris/pull/28713#issuecomment-1864064172

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit 1b7d9204e9736fece00640a1027d5b1f9c2116a7, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4682443544344434
   q2   365 152 159 152
   q3   1474129312351235
   q4   1119921 919 919
   q5   3178314831913148
   q6   252 129 130 129
   q7   1001483 491 483
   q8   220622072207
   q9   6705664966356635
   q10  3257323932933239
   q11  309 190 183 183
   q12  354 211 208 208
   q13  4586383838083808
   q14  247 211 212 211
   q15  564 520 517 517
   q16  441 392 387 387
   q17  1017619 568 568
   q18  7185695268836883
   q19  1519135314201353
   q20  543 315 308 308
   q21  3069266227272662
   q22  350 275 277 275
   Total cold run time: 44423 ms
   Total hot run time: 39944 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4359433743674337
   q2   267 162 174 162
   q3   3541353335173517
   q4   2396238123792379
   q5   5758576557745765
   q6   243 120 122 120
   q7   2370186918681868
   q8   3532352535173517
   q9   8969898689968986
   q10  3909402340034003
   q11  489 390 391 390
   q12  766 586 611 586
   q13  4287358635563556
   q14  298 268 260 260
   q15  570 518 521 518
   q16  499 459 467 459
   q17  1879188818351835
   q18  8674826181298129
   q19  1721175017621750
   q20  2255196219261926
   q21  6539622461716171
   q22  489 420 419 419
   Total cold run time: 63810 ms
   Total hot run time: 60653 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on code in PR #28698:
URL: https://github.com/apache/doris/pull/28698#discussion_r1432407126


##
be/src/pipeline/exec/exchange_sink_buffer.cpp:
##
@@ -45,16 +45,40 @@
 #include "vec/sink/vdata_stream_sender.h"
 
 namespace doris {
+
 namespace vectorized {
+BroadcastPBlockHolder::~BroadcastPBlockHolder() {
+// lock the parent queue, if the queue could lock success, then return the 
block
+// to the queue, to reuse the block
+std::shared_ptr tmp_queue = 
_parent_creator.lock();
+if (tmp_queue != nullptr) {
+
tmp_queue->push(BroadcastPBlockHolder::create_shared(std::move(_pblock)));
+}
+// If the queue already deconstruted, then release pblock automatically 
since it
+// is a unique ptr.
+}
 
-void BroadcastPBlockHolder::unref() noexcept {
-DCHECK_GT(_ref_count._value, 0);
-auto old_value = _ref_count._value.fetch_sub(1);
-if (_dep && old_value == 1) {
-_dep->return_available_block();
+void BroadcastPBlockHolderQueue::push(std::shared_ptr 
holder) {
+std::unique_lock l(_holders_lock);
+holder->set_parent_creator(shared_from_this());
+_holders.push(holder);
+if (_broadcast_dependency) {
+_broadcast_dependency->set_ready();
 }
 }
 
+std::shared_ptr BroadcastPBlockHolderQueue::pop() {

Review Comment:
   warning: method 'pop' can be made static 
[readability-convert-member-functions-to-static]
   
   be/src/pipeline/exec/exchange_sink_buffer.h:110:
   ```diff
   - std::shared_ptr pop();
   + static std::shared_ptr pop();
   ```
   



##
be/src/pipeline/exec/exchange_sink_buffer.cpp:
##
@@ -45,16 +45,40 @@
 #include "vec/sink/vdata_stream_sender.h"
 
 namespace doris {
+
 namespace vectorized {
+BroadcastPBlockHolder::~BroadcastPBlockHolder() {
+// lock the parent queue, if the queue could lock success, then return the 
block
+// to the queue, to reuse the block
+std::shared_ptr tmp_queue = 
_parent_creator.lock();
+if (tmp_queue != nullptr) {
+
tmp_queue->push(BroadcastPBlockHolder::create_shared(std::move(_pblock)));
+}
+// If the queue already deconstruted, then release pblock automatically 
since it
+// is a unique ptr.
+}
 
-void BroadcastPBlockHolder::unref() noexcept {
-DCHECK_GT(_ref_count._value, 0);
-auto old_value = _ref_count._value.fetch_sub(1);
-if (_dep && old_value == 1) {
-_dep->return_available_block();
+void BroadcastPBlockHolderQueue::push(std::shared_ptr 
holder) {

Review Comment:
   warning: method 'push' can be made static 
[readability-convert-member-functions-to-static]
   
   be/src/pipeline/exec/exchange_sink_buffer.h:106:
   ```diff
   - void push(std::shared_ptr holder);
   + static void push(std::shared_ptr holder);
   ```
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [bug](cooldown) Fix incorrect remote rowset dir after restarting BE [doris]

2023-12-20 Thread via GitHub


platoneko commented on PR #28140:
URL: https://github.com/apache/doris/pull/28140#issuecomment-1864067259

   This bug was introduced by https://github.com/apache/doris/pull/23881


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](block) fix be core while mutable block merge may cause different row size between columns in origin block [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #27943:
URL: https://github.com/apache/doris/pull/27943#issuecomment-1864068040

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.01 seconds
stream load tsv:  579 seconds loaded 74807831229 Bytes, about 123 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.2 seconds inserted 1000 Rows, about 
354K ops/s
storage size: 17183622957 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](partition) Comment out some exceptions with partition equal to … [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on PR #28702:
URL: https://github.com/apache/doris/pull/28702#issuecomment-1864069127

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhance](tablet) Reduce log in tablet meta [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28719:
URL: https://github.com/apache/doris/pull/28719#issuecomment-1864069322

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.02 seconds
stream load tsv:  584 seconds loaded 74807831229 Bytes, about 122 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  33 seconds loaded 861443392 Bytes, about 24 
MB/s
insert into select:  28.8 seconds inserted 1000 Rows, about 
347K ops/s
storage size: 17184029740 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [test](Nereids): add test for scalar agg PushDownFilterThroughAggregation [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28712:
URL: https://github.com/apache/doris/pull/28712#issuecomment-1864071344

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.12 seconds
stream load tsv:  564 seconds loaded 74807831229 Bytes, about 126 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.9 seconds inserted 1000 Rows, about 
346K ops/s
storage size: 17183977015 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on code in PR #28698:
URL: https://github.com/apache/doris/pull/28698#discussion_r1432410931


##
be/src/pipeline/exec/exchange_sink_buffer.h:
##
@@ -71,25 +72,53 @@ struct AtomicWrapper {
 // We use BroadcastPBlockHolder to hold a broadcasted PBlock. For broadcast 
shuffle, one PBlock
 // will be shared between different channel, so we have to use a ref count to 
mark if this
 // PBlock is available for next serialization.
+class BroadcastPBlockHolderQueue;
 class BroadcastPBlockHolder {
+ENABLE_FACTORY_CREATOR(BroadcastPBlockHolder);
+
+public:
+BroadcastPBlockHolder() { _pblock = std::make_unique(); }
+BroadcastPBlockHolder(std::unique_ptr&& pblock) { _pblock = 
std::move(pblock); }
+~BroadcastPBlockHolder();
+
+PBlock* get_block() { return _pblock.get(); }
+
+private:
+friend class BroadcastPBlockHolderQueue;
+std::unique_ptr _pblock;
+std::weak_ptr _parent_creator;
+void set_parent_creator(std::shared_ptr 
parent_creator) {
+_parent_creator = parent_creator;
+}
+};
+
+// Use a stack inside to ensure that the PBlock is in cpu cache
+class BroadcastPBlockHolderQueue : public 
std::enable_shared_from_this {
+ENABLE_FACTORY_CREATOR(BroadcastPBlockHolderQueue);
+
 public:
-BroadcastPBlockHolder() : _ref_count(0), _dep(nullptr) {}
-BroadcastPBlockHolder(pipeline::BroadcastDependency* dep) : _ref_count(0), 
_dep(dep) {}
-~BroadcastPBlockHolder() noexcept = default;
+BroadcastPBlockHolderQueue() = default;
 
-void ref(int delta) noexcept { _ref_count._value.fetch_add(delta); }
-void unref() noexcept;
-void ref() noexcept { ref(1); }
+BroadcastPBlockHolderQueue(
+std::shared_ptr& 
broadcast_dependency) {
+_broadcast_dependency = broadcast_dependency;
+}
 
-bool available() { return _ref_count._value == 0; }
+void push(std::shared_ptr holder);
 
-PBlock* get_block() { return &pblock; }
+bool empty() {

Review Comment:
   warning: method 'empty' can be made static 
[readability-convert-member-functions-to-static]
   
   ```suggestion
   static bool empty() {
   ```
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [fix](Nereids) join order is not right after sql parsing [doris]

2023-12-20 Thread via GitHub


morrySnow opened a new pull request, #28721:
URL: https://github.com/apache/doris/pull/28721

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](planner)should save original select list item before analyze [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28187:
URL: https://github.com/apache/doris/pull/28187#issuecomment-1864072289

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit efa15df7d7df7ef618c373dec98091a7357719b8, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4744441044184410
   q2   361 150 155 150
   q3   1471125412021202
   q4   1122886 866 866
   q5   3140314731403140
   q6   247 127 128 127
   q7   978 487 488 487
   q8   2180224022022202
   q9   6673668066536653
   q10  3234326632783266
   q11  309 187 178 178
   q12  341 210 205 205
   q13  4593380237833783
   q14  247 213 215 213
   q15  567 531 527 527
   q16  440 388 385 385
   q17  1005605 548 548
   q18  7110678568996785
   q19  1515136214241362
   q20  537 305 284 284
   q21  3074265026472647
   q22  343 274 278 274
   Total cold run time: 44231 ms
   Total hot run time: 39694 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4357435043274327
   q2   269 165 173 165
   q3   3549352035203520
   q4   2385236623722366
   q5   5736573757335733
   q6   241 122 121 121
   q7   2378184618701846
   q8   3511354335233523
   q9   9017900089688968
   q10  3923399940083999
   q11  492 368 360 360
   q12  768 603 600 600
   q13  4297356235863562
   q14  288 257 245 245
   q15  572 528 524 524
   q16  515 446 462 446
   q17  1861186018521852
   q18  8709819981088108
   q19  1712172117891721
   q20  2250194919391939
   q21  6534617161546154
   q22  508 433 418 418
   Total cold run time: 63872 ms
   Total hot run time: 60497 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [fix](fe ut) fix PropertyConverterTest create double cluster [doris]

2023-12-20 Thread via GitHub


yujun777 opened a new pull request, #28722:
URL: https://github.com/apache/doris/pull/28722

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](fe ut) fix PropertyConverterTest create double cluster [doris]

2023-12-20 Thread via GitHub


yujun777 commented on PR #28722:
URL: https://github.com/apache/doris/pull/28722#issuecomment-1864073347

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](Nereids) join order is not right after sql parsing [doris]

2023-12-20 Thread via GitHub


morrySnow commented on PR #28721:
URL: https://github.com/apache/doris/pull/28721#issuecomment-1864074907

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](api) add profile text api [doris]

2023-12-20 Thread via GitHub


Mryange commented on PR #28697:
URL: https://github.com/apache/doris/pull/28697#issuecomment-1864076468

   run feut


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](Nereids): eliminate groupby [doris]

2023-12-20 Thread via GitHub


jackwener commented on PR #28615:
URL: https://github.com/apache/doris/pull/28615#issuecomment-1864077723

   
   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](mtmv)fix insert overwrite getExecTimeout error [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on PR #28700:
URL: https://github.com/apache/doris/pull/28700#issuecomment-1864079226

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [imrovement](statistics)Auto analyze priority table/column first. [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on PR #28385:
URL: https://github.com/apache/doris/pull/28385#issuecomment-1864080412

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [DNM](Nereids) test disable fallback [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #25344:
URL: https://github.com/apache/doris/pull/25344#issuecomment-1864080963

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit fc942f526c83ed5edf006fa234094890068541c0, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4680442744154415
   q2   372 143 162 143
   q3   1448128711881188
   q4   1110884 870 870
   q5   3163316031623160
   q6   248 128 126 126
   q7   994 485 490 485
   q8   2191222322022202
   q9   6676664166486641
   q10  3232326132983261
   q11  311 194 182 182
   q12  350 208 198 198
   q13  4567383238213821
   q14  242 220 213 213
   q15  569 527 524 524
   q16  442 388 383 383
   q17  1014569 584 569
   q18  7136689069806890
   q19  1518136414201364
   q20  515 295 312 295
   q21  3073263526522635
   q22  354 275 281 275
   Total cold run time: 44205 ms
   Total hot run time: 39840 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4343436143234323
   q2   271 167 180 167
   q3   3532352135143514
   q4   2375237523602360
   q5   5734573257395732
   q6   239 119 119 119
   q7   2372183718721837
   q8   3522351635153515
   q9   9054897190008971
   q10  3945403340054005
   q11  486 371 371 371
   q12  768 591 598 591
   q13  4306356035953560
   q14  282 253 266 253
   q15  574 525 526 525
   q16  522 473 468 468
   q17  1917184818481848
   q18  8615824482588244
   q19  1735174017471740
   q20  2259195219321932
   q21  6510616661706166
   q22  495 430 423 423
   Total cold run time: 63856 ms
   Total hot run time: 60664 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](mtmv)fix thread local reference to checkpoint's Env, causing Env to be unable to be reclaimed, resulting in excessive memory usage by FE [doris]

2023-12-20 Thread via GitHub


zddr commented on PR #28723:
URL: https://github.com/apache/doris/pull/28723#issuecomment-1864084220

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [fix](mtmv)fix thread local reference to checkpoint's Env, causing Env to be unable to be reclaimed, resulting in excessive memory usage by FE [doris]

2023-12-20 Thread via GitHub


zddr opened a new pull request, #28723:
URL: https://github.com/apache/doris/pull/28723

   ## Proposed changes
   
   Issue Number: close #xxx
   
   fix thread local reference to checkpoint's Env, causing Env to be unable to 
be reclaimed, resulting in excessive memory usage by FE
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](nereids) Fix data wrong using mv rewrite and ignore case when getting mv related partition table [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on PR #28699:
URL: https://github.com/apache/doris/pull/28699#issuecomment-1864084250

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [test](Nereids): remove shape plan project and distribute in eager test [doris]

2023-12-20 Thread via GitHub


jackwener opened a new pull request, #28724:
URL: https://github.com/apache/doris/pull/28724

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](function) Fix from_second functions overflow and wrong result [doris]

2023-12-20 Thread via GitHub


zclllyybb commented on PR #28685:
URL: https://github.com/apache/doris/pull/28685#issuecomment-1864088531

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](Nereids) delete partition failed [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28717:
URL: https://github.com/apache/doris/pull/28717#issuecomment-1864089193

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit f681f42276dff1612369026904bd4e988a77a530, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4740443444284428
   q2   360 153 160 153
   q3   1447123112371231
   q4   1118918 878 878
   q5   3189316831613161
   q6   253 128 129 128
   q7   997 485 490 485
   q8   2188220221772177
   q9   6724668066426642
   q10  3245325633113256
   q11  313 187 187 187
   q12  362 206 207 206
   q13  4577382238213821
   q14  241 214 221 214
   q15  566 519 529 519
   q16  445 390 388 388
   q17  989 592 525 525
   q18  7138695769456945
   q19  1504134914261349
   q20  491 290 288 288
   q21  3072264026522640
   q22  350 279 277 277
   Total cold run time: 44309 ms
   Total hot run time: 39898 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4353434143694341
   q2   268 166 168 166
   q3   3533351135133511
   q4   2386237023682368
   q5   5728574357395739
   q6   242 123 121 121
   q7   2404187018611861
   q8   3501350435053504
   q9   9052897289928972
   q10  3904398940163989
   q11  485 374 370 370
   q12  767 650 590 590
   q13  4271355735673557
   q14  297 264 265 264
   q15  572 521 524 521
   q16  522 494 488 488
   q17  1875186618491849
   q18  8666819182348191
   q19  1720173117451731
   q20  2258194019191919
   q21  6498615761886157
   q22  508 421 414 414
   Total cold run time: 63810 ms
   Total hot run time: 60623 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [fix](metrics) fix bvar memtable_input_block_allocated_size [doris]

2023-12-20 Thread via GitHub


kaijchen opened a new pull request, #28725:
URL: https://github.com/apache/doris/pull/28725

   ## Proposed changes
   
   We should update the bvar before clear `_input_mutable_block`.
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](metrics) fix bvar memtable_input_block_allocated_size [doris]

2023-12-20 Thread via GitHub


kaijchen commented on PR #28725:
URL: https://github.com/apache/doris/pull/28725#issuecomment-1864091713

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [test](e2ecase) Optimize doris container log print [doris-flink-connector]

2023-12-20 Thread via GitHub


smallhibiscus opened a new pull request, #277:
URL: https://github.com/apache/doris-flink-connector/pull/277

   # Proposed changes
   
   Optimize doris container log print
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](nereids)support decimalv2 [doris]

2023-12-20 Thread via GitHub


starocean999 commented on PR #28726:
URL: https://github.com/apache/doris/pull/28726#issuecomment-1864096872

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [feature](nereids)support decimalv2 [doris]

2023-12-20 Thread via GitHub


starocean999 opened a new pull request, #28726:
URL: https://github.com/apache/doris/pull/28726

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](scanner) optimize the number of threads of scanners [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on PR #28640:
URL: https://github.com/apache/doris/pull/28640#issuecomment-1864097076

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](block) fix be core while mutable block merge may cause different row size between columns in origin block [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #27943:
URL: https://github.com/apache/doris/pull/27943#issuecomment-1864097873

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit 2a5a5c1bb99b073ae3796a67a181533eef02d25c, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4716445344864453
   q2   379 174 158 158
   q3   1477129812521252
   q4   898 936 898
   q5   3171315631803156
   q6   249 128 129 128
   q7   1018492 495 492
   q8   2223222422252224
   q9   6691672867056705
   q10  3213325032703250
   q11  331 221 194 194
   q12  359 206 201 201
   q13  4563379838173798
   q14  239 214 216 214
   q15  568 529 520 520
   q16  438 377 380 377
   q17  1005593 564 564
   q18  7274685369406853
   q19  1548132214221322
   q20  539 299 297 297
   q21  3106263527042635
   q22  381 313 316 313
   Total cold run time: 44599 ms
   Total hot run time: 40004 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4396440043594359
   q2   267 163 170 163
   q3   3539351735353517
   q4   2393237623822376
   q5   5757573657615736
   q6   239 122 121 121
   q7   2384186118871861
   q8   3544351335423513
   q9   9070899590338995
   q10  3941403740014001
   q11  515 385 395 385
   q12  769 601 596 596
   q13  4318362135623562
   q14  295 256 255 255
   q15  564 525 523 523
   q16  509 482 487 482
   q17  1902186318381838
   q18  8739829782508250
   q19  1732175318291753
   q20  2267195419511951
   q21  6584623962326232
   q22  530 446 451 446
   Total cold run time: 64254 ms
   Total hot run time: 60915 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [pipelineX](improvement) Support global rf [doris]

2023-12-20 Thread via GitHub


Gabriel39 commented on PR #28692:
URL: https://github.com/apache/doris/pull/28692#issuecomment-1864099647

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [test](Nereids): add test for scalar agg PushDownFilterThroughAggregation [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on PR #28712:
URL: https://github.com/apache/doris/pull/28712#issuecomment-1864100523

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [test](Nereids): add test for scalar agg PushDownFilterThroughAggregation [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on PR #28712:
URL: https://github.com/apache/doris/pull/28712#issuecomment-1864100595

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](nereids)support decimalv2 [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on PR #28726:
URL: https://github.com/apache/doris/pull/28726#issuecomment-1864102065

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](nereids)support decimalv2 [doris]

2023-12-20 Thread via GitHub


github-actions[bot] commented on PR #28726:
URL: https://github.com/apache/doris/pull/28726#issuecomment-1864102111

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](limit) cancel running fragments when query with limit is finished [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28610:
URL: https://github.com/apache/doris/pull/28610#issuecomment-1864102234

   TeamCity be ut coverage result:
Function Coverage: 37.79% (7991/21147) 
Line Coverage: 29.49% (64912/220108)
Region Coverage: 28.97% (33406/115319)
Branch Coverage: 24.83% (17137/69004)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/f03c4fb3a193ecfb5622b11f77e056ac1fe6a2c7_f03c4fb3a193ecfb5622b11f77e056ac1fe6a2c7/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](scan) Implement parallel scanning by dividing the tablets based on either the row range or the key range. [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28103:
URL: https://github.com/apache/doris/pull/28103#issuecomment-1864102548

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.47 seconds
stream load tsv:  580 seconds loaded 74807831229 Bytes, about 123 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  33 seconds loaded 861443392 Bytes, about 24 
MB/s
insert into select:  28.9 seconds inserted 1000 Rows, about 
346K ops/s
storage size: 17184099445 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](scan) Implement parallel scanning by dividing the tablets based on either the row range or the key range. [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28103:
URL: https://github.com/apache/doris/pull/28103#issuecomment-1864103711

   TeamCity be ut coverage result:
Function Coverage: 36.46% (8538/23419) 
Line Coverage: 28.56% (69418/243083)
Region Coverage: 27.60% (35930/130192)
Branch Coverage: 24.33% (18365/75488)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/57483d8fb9d5cb1a95ef817bc3bbcbee531108a0_57483d8fb9d5cb1a95ef817bc3bbcbee531108a0/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhance](tablet) Reduce log in tablet meta [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28719:
URL: https://github.com/apache/doris/pull/28719#issuecomment-1864106775

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit 8f0e7a3ed20d473312edd7f74aad0f59162ca6ab, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4736447444394439
   q2   361 150 161 150
   q3   1460125812791258
   q4   1115884 854 854
   q5   3152313531713135
   q6   251 126 125 125
   q7   1017481 491 481
   q8   2219221221982198
   q9   6683667766806677
   q10  3218325232843252
   q11  306 188 187 187
   q12  344 211 201 201
   q13  4556381937993799
   q14  236 212 218 212
   q15  573 528 531 528
   q16  435 393 383 383
   q17  1001595 589 589
   q18  7105692069916920
   q19  1519137013941370
   q20  524 310 291 291
   q21  3063269726192619
   q22  349 294 287 287
   Total cold run time: 44223 ms
   Total hot run time: 39955 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4361433743354335
   q2   270 168 175 168
   q3   3533351735133513
   q4   2379238723792379
   q5   5768574157425741
   q6   242 120 121 120
   q7   2370186818621862
   q8   3525351535353515
   q9   9033899889898989
   q10  3928403440374034
   q11  485 386 375 375
   q12  765 589 604 589
   q13  4308358635433543
   q14  285 271 258 258
   q15  573 527 526 526
   q16  521 462 474 462
   q17  1903187118731871
   q18  8552828781558155
   q19  1712176517531753
   q20  2254194419251925
   q21  6526623161846184
   q22  527 437 428 428
   Total cold run time: 63820 ms
   Total hot run time: 60725 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](mtmv)fix insert overwrite getExecTimeout error [doris]

2023-12-20 Thread via GitHub


morrySnow merged PR #28700:
URL: https://github.com/apache/doris/pull/28700


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch master updated (9d5b9cc452f -> 552a5c16328)

2023-12-20 Thread morrysnow
This is an automated email from the ASF dual-hosted git repository.

morrysnow pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


from 9d5b9cc452f [fix](statistics)Fix drop stats fail silently bug. (#28635)
 add 552a5c16328 [fix](mtmv) fix insert overwrite getExecTimeout error 
(#28700)

No new revisions were added by this update.

Summary of changes:
 .../src/main/java/org/apache/doris/job/extensions/mtmv/MTMVTask.java | 1 +
 fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java   | 5 -
 2 files changed, 5 insertions(+), 1 deletion(-)


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Error while running github feature from .asf.yaml in doris!

2023-12-20 Thread Apache Infrastructure


An error occurred while running github feature in .asf.yaml!:
You can only have a maximum of 10 external triage collaborators, please contact 
vp-in...@apache.org to request an exception.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refactor](broadcastbuffer) using a queue to remove ref and unref codes [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28698:
URL: https://github.com/apache/doris/pull/28698#issuecomment-1864111435

   TeamCity be ut coverage result:
Function Coverage: 36.48% (8534/23393) 
Line Coverage: 28.61% (69382/242519)
Region Coverage: 27.63% (35898/129927)
Branch Coverage: 24.37% (18347/75290)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/776f61933c8c65d96bd9860b2dd3bc1d12257a4a_776f61933c8c65d96bd9860b2dd3bc1d12257a4a/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](fe ut) fix PropertyConverterTest create double cluster [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28722:
URL: https://github.com/apache/doris/pull/28722#issuecomment-1864116089

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.16 seconds
stream load tsv:  567 seconds loaded 74807831229 Bytes, about 125 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  29.0 seconds inserted 1000 Rows, about 
344K ops/s
storage size: 17189218491 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [test](Nereids): add test for scalar agg PushDownFilterThroughAggregation [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28712:
URL: https://github.com/apache/doris/pull/28712#issuecomment-1864116212

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit 0eefa88ab936d025dfb3ec7e339568b9dfc2a05f, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4739443743994399
   q2   363 153 158 153
   q3   1453126012291229
   q4   1118902 940 902
   q5   3086312731123112
   q6   250 133 127 127
   q7   975 489 483 483
   q8   2169219421762176
   q9   6657663166496631
   q10  3238324932893249
   q11  306 183 187 183
   q12  350 220 209 209
   q13  4535380538063805
   q14  242 218 215 215
   q15  561 529 522 522
   q16  440 394 389 389
   q17  989 580 547 547
   q18  7176685969176859
   q19  1505136413981364
   q20  518 306 304 304
   q21  3073265726532653
   q22  341 276 281 276
   Total cold run time: 44084 ms
   Total hot run time: 39787 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4356431343274313
   q2   264 162 172 162
   q3   3538352935103510
   q4   2373237023782370
   q5   5755577357385738
   q6   241 121 123 121
   q7   2402186918501850
   q8   3520351435053505
   q9   9017895090058950
   q10  3928403339993999
   q11  490 370 356 356
   q12  767 599 597 597
   q13  4281358135613561
   q14  286 273 262 262
   q15  573 525 518 518
   q16  517 476 455 455
   q17  1878189018521852
   q18  8742814283298142
   q19  1701175217221722
   q20  2290193719421937
   q21  6527619661646164
   q22  501 414 413 413
   Total cold run time: 63947 ms
   Total hot run time: 60497 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [bugfix](scannercore) scanner will core in deconstructor during collect profile [doris]

2023-12-20 Thread via GitHub


yiguolei opened a new pull request, #28727:
URL: https://github.com/apache/doris/pull/28727

   
   
   ## Proposed changes
   
   EAD of size 8 at 0x603016b1b220 thread T800 (_scanner_scan)
   #0 0x5620cd597d1e in 
doris::vectorized::OrcReader::_collect_profile_on_close() 
/home/zcp/repo_center/doris_master/doris/be/src/vec/exec/format/orc/vorc_reader.cpp:186:9
   #1 0x5620cd5979a2 in doris::vectorized::OrcReader::~OrcReader() 
/home/zcp/repo_center/doris_master/doris/be/src/vec/exec/format/orc/vorc_reader.cpp:178:5
   #2 0x5620cd5985b8 in doris::vectorized::OrcReader::~OrcReader() 
/home/zcp/repo_center/doris_master/doris/be/src/vec/exec/format/orc/vorc_reader.cpp:177:25
   #3 0x5620baac5954 in 
std::default_delete::operator()(doris::vectorized::GenericReader*) const 
/var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:85:2
   #4 0x5620baa9b75b in std::unique_ptr>::~unique_ptr() 
/var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:361:4
   #5 0x5620d4351afb in doris::vectorized::VFileScanner::~VFileScanner() 
/home/zcp/repo_center/doris_master/doris/be/src/vec/exec/scan/vfile_scanner.h:60:7
   #6 0x5620d4351b28 in doris::vectorized::VFileScanner::~VFileScanner() 
/home/zcp/repo_center/doris_master/doris/be/src/vec/exec/scan/vfile_scanner.h:60:7
   #7 0x5620d40d5f14 in 
std::default_delete::operator()(doris::vectorized::VFileScanner*) const 
/var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:85:2
   #8 0x5620d40d6ccb in std::_Sp_counted_deleter, std::allocator, 
(__gnu_cxx::_Lock_policy)2>::_M_dispose() 
/var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:442:9
   #9 0x5620b7a22a41 in 
std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() 
/var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:168:6
   #10 0x5620b7a22779 in 
std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() 
/var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:702:11
   #11 0x5620d405dd2a in std::__shared_ptr::~__shared_ptr() 
/var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1149:31
   #12 0x5620d405d9f4 in std::shared_ptr::~shared_ptr() 
/var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr.h:122:11
   #13 0x5620d42fd5b9 in 
doris::vectorized::ScannerScheduler::_schedule_scanners(std::shared_ptr)::$_5::~$_5()
 
/home/zcp/repo_center/doris_master/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:271:38
   #14 0x5620d4306362 in 
std::_Function_base::_Base_manager)::$_5>::_M_destroy(std::_Any_data&, 
std::integral_constant) 
/var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:174:4
   #15 0x5620d4306209 in 
std::_Function_base::_Base_manager)::$_5>::_M_manager(std::_Any_data&, 
std::_Any_data const&, std::_Manager_operation) 
/var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:200:8
   #16 0x5620d4305cbb in 
std::_Function_handler)::$_5>::_M_manager(std::_Any_data&, std::_Any_data 
const&, std::_Manager_operation) 
/var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:283:6
   #17 0x5620b7aa9112 in std::_Function_base::~_Function_base() 
/var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:245:2
   #18 0x5620b7affe54 in std::function::~function() 
/var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:111:11
   #19 0x5620b7bdd2b8 in doris::WorkThreadPool::Task::~Task() 
/home/zcp/repo_center/doris_master/doris/be/src/util/work_thread_pool.hpp:40:12
   #20 0x5620b7bdfac4 in doris::WorkThreadPool::work_thread(int) 
/home/zcp/repo_center/doris_master/doris/be/src/util/work_thread_pool.hpp:164:9
   #21 0x5620b7be2134 in void std::__invoke_impl::* const&)(int), 
doris::WorkThreadPool*&, int&>(std::__invoke_memfun_deref, void 
(doris::WorkThreadPool::* const&)(int), doris::WorkThreadPool*&, int&) 
/var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:74:14
   #22 0x5620b7be1ff6 in std::__invoke_result::* const&)(int), 
doris::WorkThreadPool*&, int&>::type std::__invoke::* const&)(int), 
doris::WorkThreadPool*&, int&>(void (doris::WorkThreadPool::* const&)(int), 
doris::WorkThreadPool*&, int&) 
/var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14
   #23 0x5620b7be1fb6 in decltype(std::__invoke((*this)._M_pmf, 
std::forward*&>(fp), 

Re: [PR] [fix](group commit)Fix the issue of duplicate addition of wal path when encouter exception [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28691:
URL: https://github.com/apache/doris/pull/28691#issuecomment-1864117638

   TeamCity be ut coverage result:
Function Coverage: 36.48% (8535/23397) 
Line Coverage: 28.61% (69395/242550)
Region Coverage: 27.64% (35915/129951)
Branch Coverage: 24.37% (18350/75302)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/404da44f6116a74cb0200ce8255e02e31dc6_404da44f6116a74cb0200ce8255e02e31dc6/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [bugfix](scannercore) scanner will core in deconstructor during collect profile [doris]

2023-12-20 Thread via GitHub


yiguolei commented on PR #28727:
URL: https://github.com/apache/doris/pull/28727#issuecomment-1864117673

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](Nereids) delete partition failed [doris]

2023-12-20 Thread via GitHub


morrySnow commented on PR #28717:
URL: https://github.com/apache/doris/pull/28717#issuecomment-1864119125

   run p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](Nereids) delete partition failed [doris]

2023-12-20 Thread via GitHub


morrySnow commented on PR #28717:
URL: https://github.com/apache/doris/pull/28717#issuecomment-1864119238

   run feut


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](Nereids) forbid two TVF in one fragment since the limit of coordinator (#28114) [doris]

2023-12-20 Thread via GitHub


morrySnow commented on PR #28569:
URL: https://github.com/apache/doris/pull/28569#issuecomment-1864120672

   run clickbench


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Enhancement] (schema) add column type check [doris]

2023-12-20 Thread via GitHub


doris-robot commented on PR #28718:
URL: https://github.com/apache/doris/pull/28718#issuecomment-1864121741

   TeamCity be ut coverage result:
Function Coverage: 36.48% (8535/23398) 
Line Coverage: 28.60% (69384/242561)
Region Coverage: 27.63% (35912/129954)
Branch Coverage: 24.37% (18354/75308)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/db67477d3681fdf231f1d89a16353f47a1a39ad0_db67477d3681fdf231f1d89a16353f47a1a39ad0/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch master updated (552a5c16328 -> 08066a55f14)

2023-12-20 Thread zhangstar333
This is an automated email from the ASF dual-hosted git repository.

zhangstar333 pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


from 552a5c16328 [fix](mtmv) fix insert overwrite getExecTimeout error 
(#28700)
 add 08066a55f14 [Enhancement](auto-partition) change the behaviour when 
insert overwrite an auto partition table #28683

No new revisions were added by this update.

Summary of changes:
 docs/en/docs/advanced/partition/auto-partition.md  |  1 +
 .../Manipulation/INSERT-OVERWRITE.md   |  8 ++---
 .../docs/advanced/partition/auto-partition.md  |  1 +
 .../Manipulation/INSERT-OVERWRITE.md   |  8 ++---
 .../doris/analysis/CreateTableAsSelectStmt.java|  2 +-
 .../apache/doris/analysis/NativeInsertStmt.java|  7 +++-
 .../trees/plans/commands/InsertExecutor.java   |  6 +++-
 .../plans/commands/InsertIntoTableCommand.java |  9 -
 .../commands/InsertOverwriteTableCommand.java  |  5 ++-
 .../org/apache/doris/planner/OlapTableSink.java|  4 +++
 .../java/org/apache/doris/qe/StmtExecutor.java |  4 +--
 .../test_auto_partition_behavior.out   |  6 
 .../test_auto_partition_behavior.groovy| 39 ++
 13 files changed, 85 insertions(+), 15 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [minor](log) fix wrong error type [doris]

2023-12-20 Thread via GitHub


zclllyybb opened a new pull request, #28729:
URL: https://github.com/apache/doris/pull/28729

   ## Proposed changes
   
   Issue Number: close #xxx
   
   change
   ```
   [INTERNAL_ERROR]The depth of the expression tree is too big, make it less 
than 600
   ```
   to
   ```
   [OUT_OF_BOUND]The depth of the expression tree is too big, make it less than 
600
   ```
   and without stacktrace
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Enhancement](auto-partition) change the behaviour when insert overwrite an auto partition table [doris]

2023-12-20 Thread via GitHub


zhangstar333 merged PR #28683:
URL: https://github.com/apache/doris/pull/28683


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Error while running github feature from .asf.yaml in doris!

2023-12-20 Thread Apache Infrastructure


An error occurred while running github feature in .asf.yaml!:
You can only have a maximum of 10 external triage collaborators, please contact 
vp-in...@apache.org to request an exception.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [minor](log) fix wrong error type [doris]

2023-12-20 Thread via GitHub


zclllyybb commented on PR #28729:
URL: https://github.com/apache/doris/pull/28729#issuecomment-1864125134

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



  1   2   3   4   5   6   7   8   9   >