Re: [PR] [fix](block) fix be core while mutable block merge may cause different row size between columns in origin block [doris]

2023-12-22 Thread via GitHub


HappenLee merged PR #28850:
URL: https://github.com/apache/doris/pull/28850


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch branch-1.2-lts updated: [fix](block) fix be core while mutable block merge may cause different row size between columns in origin block (#28850)

2023-12-22 Thread lihaopeng
This is an automated email from the ASF dual-hosted git repository.

lihaopeng pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
 new 120d69a37b1 [fix](block) fix be core while mutable block merge may 
cause different row size between columns in origin block (#28850)
120d69a37b1 is described below

commit 120d69a37b1fa53ebd3c7ec7f7d761be4f47e634
Author: caiconghui <55968745+caicong...@users.noreply.github.com>
AuthorDate: Fri Dec 22 16:00:36 2023 +0800

[fix](block) fix be core while mutable block merge may cause different row 
size between columns in origin block (#28850)

Co-authored-by: caiconghui1 
---
 be/src/vec/exec/scan/scanner_scheduler.cpp |  4 +++-
 be/src/vec/exec/vunion_node.cpp| 11 +--
 2 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/be/src/vec/exec/scan/scanner_scheduler.cpp 
b/be/src/vec/exec/scan/scanner_scheduler.cpp
index 7b7e3aeac50..32cefe6e822 100644
--- a/be/src/vec/exec/scan/scanner_scheduler.cpp
+++ b/be/src/vec/exec/scan/scanner_scheduler.cpp
@@ -269,7 +269,9 @@ void ScannerScheduler::_scanner_scan(ScannerScheduler* 
scheduler, ScannerContext
 ctx->return_free_block(block);
 } else {
 if (!blocks.empty() && blocks.back()->rows() + block->rows() <= 
state->batch_size()) {
-vectorized::MutableBlock(blocks.back()).merge(*block);
+vectorized::MutableBlock mutable_block(blocks.back());
+mutable_block.merge(*block);
+
blocks.back()->set_columns(std::move(mutable_block.mutable_columns()));
 ctx->return_free_block(block);
 } else {
 blocks.push_back(block);
diff --git a/be/src/vec/exec/vunion_node.cpp b/be/src/vec/exec/vunion_node.cpp
index 8afe5bce8df..199782b577b 100644
--- a/be/src/vec/exec/vunion_node.cpp
+++ b/be/src/vec/exec/vunion_node.cpp
@@ -171,9 +171,9 @@ Status VUnionNode::get_next_materialized(RuntimeState* 
state, Block* block) {
 }
 }
 
-if (!mem_reuse) {
-block->swap(mblock.to_block());
-}
+
+block->swap(mblock.to_block());
+
 
 DCHECK_LE(_child_idx, _children.size());
 return Status::OK();
@@ -204,9 +204,8 @@ Status VUnionNode::get_next_const(RuntimeState* state, 
Block* block) {
 }
 }
 
-if (!mem_reuse) {
-block->swap(mblock.to_block());
-}
+
+block->swap(mblock.to_block());
 
 // some insert query like "insert into string_test select 1, repeat('a', 
1024 * 1024);"
 // the const expr will be in output expr cause the union node return a 
empty block. so here we


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement](group commit) refector replay wal function [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28756:
URL: https://github.com/apache/doris/pull/28756#issuecomment-1867361742

   TeamCity be ut coverage result:
Function Coverage: 36.48% (8538/23404) 
Line Coverage: 28.59% (69433/242838)
Region Coverage: 27.61% (35927/130119)
Branch Coverage: 24.36% (18371/75406)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/b41679ab54ad3eec3c9af8c9db7ad269a1550c57_b41679ab54ad3eec3c9af8c9db7ad269a1550c57/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Feature](inverted index) add lowercase option for inverted index analyzer [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28704:
URL: https://github.com/apache/doris/pull/28704#issuecomment-1867363611

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.94 seconds
stream load tsv:  584 seconds loaded 74807831229 Bytes, about 122 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.7 seconds inserted 1000 Rows, about 
348K ops/s
storage size: 17183622321 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [I] [Bug] 2.0.3 查询异常 [doris]

2023-12-22 Thread via GitHub


learner1212 commented on issue #28741:
URL: https://github.com/apache/doris/issues/28741#issuecomment-1867364326

   > Enable the nereids planner and the query will execute correctly.
   
   这个有效果,多谢


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [fix](pipelineX) fix use global rf when there no shared_scans [doris]

2023-12-22 Thread via GitHub


Mryange opened a new pull request, #28869:
URL: https://github.com/apache/doris/pull/28869

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](pipelineX) fix use global rf when there no shared_scans [doris]

2023-12-22 Thread via GitHub


Mryange commented on PR #28869:
URL: https://github.com/apache/doris/pull/28869#issuecomment-1867366733

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [only test] [doris]

2023-12-22 Thread via GitHub


Mryange commented on PR #28870:
URL: https://github.com/apache/doris/pull/28870#issuecomment-1867368525

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [only test] [doris]

2023-12-22 Thread via GitHub


Mryange opened a new pull request, #28870:
URL: https://github.com/apache/doris/pull/28870

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refactor](pipelineX)do not override dependency() function in pipelineX [doris]

2023-12-22 Thread via GitHub


Mryange commented on PR #28848:
URL: https://github.com/apache/doris/pull/28848#issuecomment-1867370077

   run tpch


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement](group commit) Add an option of enable relay wal on all regression test [doris]

2023-12-22 Thread via GitHub


hust-hhb commented on PR #28757:
URL: https://github.com/apache/doris/pull/28757#issuecomment-1867371191

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](pipelineX) fix use global rf when there no shared_scans [doris]

2023-12-22 Thread via GitHub


github-actions[bot] commented on PR #28869:
URL: https://github.com/apache/doris/pull/28869#issuecomment-1867371564

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Feature](inverted index) add lowercase option for inverted index analyzer [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28704:
URL: https://github.com/apache/doris/pull/28704#issuecomment-1867373469

   TeamCity be ut coverage result:
Function Coverage: 36.47% (8536/23404) 
Line Coverage: 28.59% (69406/242802)
Region Coverage: 27.60% (35913/130113)
Branch Coverage: 24.34% (18352/75406)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/490c05e0613f7144282dca56d2abade4013743e1_490c05e0613f7144282dca56d2abade4013743e1/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement](group commit) refector replay wal function [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28756:
URL: https://github.com/apache/doris/pull/28756#issuecomment-1867374535

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.58 seconds
stream load tsv:  568 seconds loaded 74807831229 Bytes, about 125 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.9 seconds inserted 1000 Rows, about 
346K ops/s
storage size: 17183676197 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [only test] [doris]

2023-12-22 Thread via GitHub


github-actions[bot] commented on PR #28870:
URL: https://github.com/apache/doris/pull/28870#issuecomment-1867374844

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](nereids) "not is null" stats estimation fix [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28860:
URL: https://github.com/apache/doris/pull/28860#issuecomment-1867376292

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.38 seconds
stream load tsv:  582 seconds loaded 74807831229 Bytes, about 122 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  29.3 seconds inserted 1000 Rows, about 
341K ops/s
storage size: 17184032145 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [hotfix](jdbc catalog) fix load table and column names npe [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28865:
URL: https://github.com/apache/doris/pull/28865#issuecomment-1867378600

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.01 seconds
stream load tsv:  584 seconds loaded 74807831229 Bytes, about 122 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.8 seconds inserted 1000 Rows, about 
347K ops/s
storage size: 17183873661 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [fix](hash join) fix stack overflow caused by evaluate case expr on huge build block (#28851) [doris]

2023-12-22 Thread via GitHub


jacktengg opened a new pull request, #28871:
URL: https://github.com/apache/doris/pull/28871

   
   ## Proposed changes
   
   Pick from #28851
   
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](log) regularise some BE error type and fix a load task check [doris]

2023-12-22 Thread via GitHub


github-actions[bot] commented on PR #28729:
URL: https://github.com/apache/doris/pull/28729#issuecomment-1867379816

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement](jdbc catalog) Change the connection pool parameters of Jdbc Catalog to be configurable [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28779:
URL: https://github.com/apache/doris/pull/28779#issuecomment-1867380818

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 49.25 seconds
stream load tsv:  567 seconds loaded 74807831229 Bytes, about 125 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  29.6 seconds inserted 1000 Rows, about 
337K ops/s
storage size: 17163179241 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [exec](load) change default parallel num from 1 to 8 in no pipeline exec engine [doris]

2023-12-22 Thread via GitHub


HappenLee commented on PR #28864:
URL: https://github.com/apache/doris/pull/28864#issuecomment-1867380941

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [I] [Bug] [doris]

2023-12-22 Thread via GitHub


xlostpath commented on issue #28866:
URL: https://github.com/apache/doris/issues/28866#issuecomment-1867382218

   If the data is inserted in a single batch, the ordering will be correct. 
However, if the insertion is done in multiple batches, the ordering will be 
interrupted, resulting in incorrect sorting.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [nereids] fix join fd computing bug [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28849:
URL: https://github.com/apache/doris/pull/28849#issuecomment-1867383420

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.63 seconds
stream load tsv:  585 seconds loaded 74807831229 Bytes, about 121 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  33 seconds loaded 861443392 Bytes, about 24 
MB/s
insert into select:  28.6 seconds inserted 1000 Rows, about 
349K ops/s
storage size: 17183763647 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](block) fix be core while mutable block merge may cause different row size between columns in origin block [doris]

2023-12-22 Thread via GitHub


github-actions[bot] commented on PR #27943:
URL: https://github.com/apache/doris/pull/27943#issuecomment-1867385779

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](block) fix be core while mutable block merge may cause different row size between columns in origin block [doris]

2023-12-22 Thread via GitHub


github-actions[bot] commented on PR #27943:
URL: https://github.com/apache/doris/pull/27943#issuecomment-1867385732

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](pipelineX) fix use global rf when there no shared_scans [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28869:
URL: https://github.com/apache/doris/pull/28869#issuecomment-1867386158

   TeamCity be ut coverage result:
Function Coverage: 36.48% (8537/23403) 
Line Coverage: 28.59% (69416/242816)
Region Coverage: 27.60% (35912/130112)
Branch Coverage: 24.34% (18353/75408)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/cd467df94f2ed4a92a8007ac5582fcf1a05e29bf_cd467df94f2ed4a92a8007ac5582fcf1a05e29bf/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [I] [Bug] 2.0.3 查询异常 [doris]

2023-12-22 Thread via GitHub


learner1212 closed issue #28741: [Bug] 2.0.3 查询异常
URL: https://github.com/apache/doris/issues/28741


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [bugfix](scannercore) scanner will core in deconstructor during collect profile [doris]

2023-12-22 Thread via GitHub


yiguolei commented on PR #28727:
URL: https://github.com/apache/doris/pull/28727#issuecomment-1867390937

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](pipelineX) control exchange sink by memory usage [doris]

2023-12-22 Thread via GitHub


Mryange commented on PR #28814:
URL: https://github.com/apache/doris/pull/28814#issuecomment-1867392342

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [only test] [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28870:
URL: https://github.com/apache/doris/pull/28870#issuecomment-1867393006

   TeamCity be ut coverage result:
Function Coverage: 36.48% (8538/23403) 
Line Coverage: 28.59% (69421/242816)
Region Coverage: 27.61% (35919/130112)
Branch Coverage: 24.34% (18357/75408)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/cc90734f922f0b5576bc8d45a6ce5cde9ee46610_cc90734f922f0b5576bc8d45a6ce5cde9ee46610/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [only test] [doris]

2023-12-22 Thread via GitHub


Mryange commented on PR #28870:
URL: https://github.com/apache/doris/pull/28870#issuecomment-1867395385

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [optimize](invert index) optimize inverted index metadata into file cache [doris]

2023-12-22 Thread via GitHub


zzzxl1993 opened a new pull request, #28872:
URL: https://github.com/apache/doris/pull/28872

   
   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [docs] (DebugPoints) Update docs about Debug Points [doris]

2023-12-22 Thread via GitHub


HowardQin commented on PR #28347:
URL: https://github.com/apache/doris/pull/28347#issuecomment-1867397594

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](invert index) Empty strings are not written to the index in the… [doris]

2023-12-22 Thread via GitHub


zzzxl1993 commented on PR #28822:
URL: https://github.com/apache/doris/pull/28822#issuecomment-1867398396

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [optimize](invert index) optimize inverted index metadata into file cache [doris]

2023-12-22 Thread via GitHub


zzzxl1993 commented on PR #28872:
URL: https://github.com/apache/doris/pull/28872#issuecomment-1867398513

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](pipelineX) control exchange sink by memory usage [doris]

2023-12-22 Thread via GitHub


github-actions[bot] commented on PR #28814:
URL: https://github.com/apache/doris/pull/28814#issuecomment-1867399432

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [only test] [doris]

2023-12-22 Thread via GitHub


github-actions[bot] commented on PR #28870:
URL: https://github.com/apache/doris/pull/28870#issuecomment-1867407902

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [optimize](invert index) optimize inverted index metadata into file cache [doris]

2023-12-22 Thread via GitHub


github-actions[bot] commented on code in PR #28872:
URL: https://github.com/apache/doris/pull/28872#discussion_r1434851121


##
be/src/olap/rowset/beta_rowset.cpp:
##
@@ -318,7 +319,8 @@ Status BetaRowset::copy_files_to(const std::string& dir, 
const RowsetId& new_row
 return Status::OK();
 }
 
-Status BetaRowset::upload_to(io::RemoteFileSystem* dest_fs, const RowsetId& 
new_rowset_id) {
+Status BetaRowset::upload_to(const io::RemoteFileSystemSPtr& dest_fs,

Review Comment:
   warning: function 'upload_to' exceeds recommended size/complexity thresholds 
[readability-function-size]
   ```cpp
   Status BetaRowset::upload_to(const io::RemoteFileSystemSPtr& dest_fs,
  ^
   ```
   
   Additional context
   
   **be/src/olap/rowset/beta_rowset.cpp:321:** 83 lines including whitespace 
and comments (threshold 80)
   ```cpp
   Status BetaRowset::upload_to(const io::RemoteFileSystemSPtr& dest_fs,
  ^
   ```
   
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](pipelineX) fix use global rf when there no shared_scans [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28869:
URL: https://github.com/apache/doris/pull/28869#issuecomment-1867409829

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.88 seconds
stream load tsv:  583 seconds loaded 74807831229 Bytes, about 122 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  33 seconds loaded 861443392 Bytes, about 24 
MB/s
insert into select:  28.9 seconds inserted 1000 Rows, about 
346K ops/s
storage size: 17183964101 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [FIX](complextype)fix count func with complex type [doris]

2023-12-22 Thread via GitHub


amorynan opened a new pull request, #28873:
URL: https://github.com/apache/doris/pull/28873

   ## Proposed changes
   
   Issue Number: close #xxx
   when we use count func with old planner which defined function segnature in 
fe code with hard code, and will not match coplex type with variable nested 
type. 
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](invert index) Empty strings are not written to the index in the… [doris]

2023-12-22 Thread via GitHub


github-actions[bot] commented on PR #28822:
URL: https://github.com/apache/doris/pull/28822#issuecomment-1867410759

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [FIX](complextype) fix complextype has empty literal may make be core [doris]

2023-12-22 Thread via GitHub


amorynan commented on PR #28820:
URL: https://github.com/apache/doris/pull/28820#issuecomment-1867414506

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [exec](load) change default parallel num from 1 to 8 in no pipeline exec engine [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28864:
URL: https://github.com/apache/doris/pull/28864#issuecomment-1867416653

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.14 seconds
stream load tsv:  581 seconds loaded 74807831229 Bytes, about 122 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  33 seconds loaded 861443392 Bytes, about 24 
MB/s
insert into select:  28.8 seconds inserted 1000 Rows, about 
347K ops/s
storage size: 17184071816 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [FIX](complextype)fix count func with complex type [doris]

2023-12-22 Thread via GitHub


amorynan commented on PR #28873:
URL: https://github.com/apache/doris/pull/28873#issuecomment-1867419009

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](paimon)fix `like` predicate [doris]

2023-12-22 Thread via GitHub


wuwenchi commented on PR #28803:
URL: https://github.com/apache/doris/pull/28803#issuecomment-1867421100

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [only test] [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28870:
URL: https://github.com/apache/doris/pull/28870#issuecomment-1867421553

   TeamCity be ut coverage result:
Function Coverage: 36.48% (8538/23403) 
Line Coverage: 28.59% (69421/242816)
Region Coverage: 27.60% (35911/130112)
Branch Coverage: 24.34% (18357/75408)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/c7bebd930ee5cdcab8ccfa61e2b389d1be9c3dab_c7bebd930ee5cdcab8ccfa61e2b389d1be9c3dab/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [fix](ci) tpch pipeline should not re-load data [doris]

2023-12-22 Thread via GitHub


hello-stephen opened a new pull request, #28874:
URL: https://github.com/apache/doris/pull/28874

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [improvement](log) add txn log [doris]

2023-12-22 Thread via GitHub


yujun777 opened a new pull request, #28875:
URL: https://github.com/apache/doris/pull/28875

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [fix](nereids) Fix query rewrite fail when query cache build quickly [doris]

2023-12-22 Thread via GitHub


seawinde opened a new pull request, #28876:
URL: https://github.com/apache/doris/pull/28876

   ## Proposed changes
   
   Fix query rewrite fail by mv when mv build interval  is short and cache 
update quickly.
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement](log) add txn log [doris]

2023-12-22 Thread via GitHub


yujun777 commented on PR #28875:
URL: https://github.com/apache/doris/pull/28875#issuecomment-1867422868

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](ci) tpch pipeline should not re-load data [doris]

2023-12-22 Thread via GitHub


hello-stephen commented on PR #28874:
URL: https://github.com/apache/doris/pull/28874#issuecomment-1867422892

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](nereids) Fix query rewrite fail when query cache build quickly [doris]

2023-12-22 Thread via GitHub


seawinde commented on PR #28876:
URL: https://github.com/apache/doris/pull/28876#issuecomment-1867422950

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](invert index) Empty strings are not written to the index in the… [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28822:
URL: https://github.com/apache/doris/pull/28822#issuecomment-1867424793

   TeamCity be ut coverage result:
Function Coverage: 36.48% (8537/23403) 
Line Coverage: 28.59% (69423/242808)
Region Coverage: 27.61% (35921/130100)
Branch Coverage: 24.35% (18362/75410)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/e50e9ed2c5ca3fa3c3bcac7af26451059f1ffe84_e50e9ed2c5ca3fa3c3bcac7af26451059f1ffe84/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refactor](executor)remove scan group [doris]

2023-12-22 Thread via GitHub


yiguolei merged PR #28847:
URL: https://github.com/apache/doris/pull/28847


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch master updated: [refactor](executor)remove scan group #28847

2023-12-22 Thread yiguolei
This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new aca8406e319 [refactor](executor)remove scan group #28847
aca8406e319 is described below

commit aca8406e319530030e065a34f19a496fab86e85b
Author: wangbo 
AuthorDate: Fri Dec 22 17:05:50 2023 +0800

[refactor](executor)remove scan group #28847
---
 be/src/common/config.cpp   |   2 -
 be/src/common/config.h |   2 -
 be/src/runtime/task_group/task_group.cpp   |   5 -
 be/src/runtime/task_group/task_group.h |   6 -
 be/src/vec/exec/scan/scan_task_queue.cpp   | 221 -
 be/src/vec/exec/scan/scan_task_queue.h |  99 -
 be/src/vec/exec/scan/scanner_context.cpp   |   4 -
 be/src/vec/exec/scan/scanner_context.h |   1 -
 be/src/vec/exec/scan/scanner_scheduler.cpp |  44 --
 be/src/vec/exec/scan/scanner_scheduler.h   |  13 +-
 10 files changed, 1 insertion(+), 396 deletions(-)

diff --git a/be/src/common/config.cpp b/be/src/common/config.cpp
index 03eaee7b23c..ecc44a08e47 100644
--- a/be/src/common/config.cpp
+++ b/be/src/common/config.cpp
@@ -970,8 +970,6 @@ DEFINE_Bool(enable_fuzzy_mode, "false");
 DEFINE_Bool(enable_debug_points, "false");
 
 DEFINE_Int32(pipeline_executor_size, "0");
-DEFINE_Bool(enable_workload_group_for_scan, "false");
-DEFINE_mInt64(workload_group_scan_task_wait_timeout_ms, "1");
 // 128 MB
 DEFINE_mInt64(local_exchange_buffer_mem_limit, "134217728");
 
diff --git a/be/src/common/config.h b/be/src/common/config.h
index e011073d44d..a9508c6e8af 100644
--- a/be/src/common/config.h
+++ b/be/src/common/config.h
@@ -1024,8 +1024,6 @@ DECLARE_Bool(enable_fuzzy_mode);
 DECLARE_Bool(enable_debug_points);
 
 DECLARE_Int32(pipeline_executor_size);
-DECLARE_Bool(enable_workload_group_for_scan);
-DECLARE_mInt64(workload_group_scan_task_wait_timeout_ms);
 
 // Temp config. True to use optimization for bitmap_index apply predicate 
except leaf node of the and node.
 // Will remove after fully test.
diff --git a/be/src/runtime/task_group/task_group.cpp 
b/be/src/runtime/task_group/task_group.cpp
index 137f5ea2345..9e86f8b831b 100644
--- a/be/src/runtime/task_group/task_group.cpp
+++ b/be/src/runtime/task_group/task_group.cpp
@@ -33,7 +33,6 @@
 #include "runtime/memory/mem_tracker_limiter.h"
 #include "util/mem_info.h"
 #include "util/parse_util.h"
-#include "vec/exec/scan/scan_task_queue.h"
 #include "vec/exec/scan/scanner_scheduler.h"
 
 namespace doris {
@@ -102,7 +101,6 @@ std::string TaskGroupEntity::debug_string() 
const {
 }
 
 template class TaskGroupEntity>;
-template class TaskGroupEntity;
 
 TaskGroup::TaskGroup(const TaskGroupInfo& tg_info)
 : _id(tg_info.id),
@@ -112,7 +110,6 @@ TaskGroup::TaskGroup(const TaskGroupInfo& tg_info)
   _enable_memory_overcommit(tg_info.enable_memory_overcommit),
   _cpu_share(tg_info.cpu_share),
   _task_entity(this, "pipeline task entity"),
-  _local_scan_entity(this, "local scan entity"),
   _mem_tracker_limiter_pool(MEM_TRACKER_GROUP_NUM),
   _cpu_hard_limit(tg_info.cpu_hard_limit) {}
 
@@ -150,8 +147,6 @@ void TaskGroup::check_and_update(const TaskGroupInfo& 
tg_info) {
 }
 
ExecEnv::GetInstance()->pipeline_task_group_scheduler()->task_queue()->update_tg_cpu_share(
 tg_info, &_task_entity);
-
ExecEnv::GetInstance()->scanner_scheduler()->local_scan_task_queue()->update_tg_cpu_share(
-tg_info, &_local_scan_entity);
 }
 
 int64_t TaskGroup::memory_used() {
diff --git a/be/src/runtime/task_group/task_group.h 
b/be/src/runtime/task_group/task_group.h
index f1c8523664e..04dbf518f0d 100644
--- a/be/src/runtime/task_group/task_group.h
+++ b/be/src/runtime/task_group/task_group.h
@@ -43,7 +43,6 @@ namespace taskgroup {
 
 class TaskGroup;
 struct TaskGroupInfo;
-class ScanTaskQueue;
 
 template 
 class TaskGroupEntity {
@@ -88,9 +87,6 @@ private:
 using TaskGroupPipelineTaskEntity = 
TaskGroupEntity>;
 using TGPTEntityPtr = TaskGroupPipelineTaskEntity*;
 
-using TaskGroupScanTaskEntity = TaskGroupEntity;
-using TGSTEntityPtr = TaskGroupScanTaskEntity*;
-
 struct TgTrackerLimiterGroup {
 std::unordered_set> trackers;
 std::mutex group_lock;
@@ -101,7 +97,6 @@ public:
 explicit TaskGroup(const TaskGroupInfo& tg_info);
 
 TaskGroupPipelineTaskEntity* task_entity() { return &_task_entity; }
-TGSTEntityPtr local_scan_task_entity() { return &_local_scan_entity; }
 
 int64_t version() const { return _version; }
 
@@ -155,7 +150,6 @@ private:
 bool _enable_memory_overcommit;
 std::atomic _cpu_share;
 TaskGroupPipelineTaskEntity _task_entity;
-TaskGroupScanTaskEntity _local_scan_entity;
 std::vector _mem_tracker_limiter_pool;
 std::atomic _cpu_hard_limit;
 };
diff --git a/be/src/vec/exec/scan/scan_task_

Error while running github feature from .asf.yaml in doris!

2023-12-22 Thread Apache Infrastructure


An error occurred while running github feature in .asf.yaml!:
You can only have a maximum of 10 external triage collaborators, please contact 
vp-in...@apache.org to request an exception.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] (enhance)(S3) Change s3 metric from bvar adder to latency recorder [doris]

2023-12-22 Thread via GitHub


ByteYue commented on PR #28861:
URL: https://github.com/apache/doris/pull/28861#issuecomment-1867428473

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](pipelineX) control exchange sink by memory usage [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28814:
URL: https://github.com/apache/doris/pull/28814#issuecomment-1867429749

   TeamCity be ut coverage result:
Function Coverage: 36.48% (8540/23408) 
Line Coverage: 28.60% (69451/242835)
Region Coverage: 27.62% (35933/130117)
Branch Coverage: 24.37% (18373/75406)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/e45918b43776ee1038d98ab8f06ff17bec8d445b_e45918b43776ee1038d98ab8f06ff17bec8d445b/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement](log) add txn log [doris]

2023-12-22 Thread via GitHub


github-actions[bot] commented on PR #28875:
URL: https://github.com/apache/doris/pull/28875#issuecomment-1867430299

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement](jdbc catalog) Optimize connection pool caching logic [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28859:
URL: https://github.com/apache/doris/pull/28859#issuecomment-1867431272

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit 20119d0346e194fe037ac709de2612736462db0a, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4730443344204420
   q2   378 138 138 138
   q3   1471126512341234
   q4   1109901 925 901
   q5   3182317031813170
   q6   249 130 132 130
   q7   1013496 486 486
   q8   2206228522142214
   q9   6720670866526652
   q10  3219328232753275
   q11  315 187 187 187
   q12  357 214 209 209
   q13  4542377337953773
   q14  245 214 215 214
   q15  577 532 521 521
   q16  449 383 388 383
   q17  1008641 600 600
   q18  7175699269346934
   q19  1553143014591430
   q20  533 331 292 292
   q21  3123269727232697
   q22  359 282 290 282
   Total cold run time: 44513 ms
   Total hot run time: 40142 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4378439143334333
   q2   266 170 175 170
   q3   3518351834943494
   q4   2381237623782376
   q5   5710573257215721
   q6   240 126 124 124
   q7   2343186618551855
   q8   3533352835273527
   q9   9052901689868986
   q10  3938400740234007
   q11  481 367 388 367
   q12  774 594 603 594
   q13  4298353735873537
   q14  285 260 254 254
   q15  584 532 536 532
   q16  521 451 472 451
   q17  1880186418551855
   q18  8474823580318031
   q19  1752175017441744
   q20  2265195019451945
   q21  6485618261416141
   q22  495 418 424 418
   Total cold run time: 63653 ms
   Total hot run time: 60462 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [FIX](map)fix map with rowstore table [doris]

2023-12-22 Thread via GitHub


amorynan opened a new pull request, #28877:
URL: https://github.com/apache/doris/pull/28877

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[I] [Bug] Subquery has bug [doris]

2023-12-22 Thread via GitHub


SinyoWong opened a new issue, #28878:
URL: https://github.com/apache/doris/issues/28878

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no 
similar issues.
   
   
   ### Version
   
   Doris 2.0.3
   
   ### What's Wrong?
   
   SQL 错误 [1105] [HY000]: errCode = 2, detailMessage = 
(192.168.219.30)[INTERNAL_ERROR]couldn't resolve slot descriptor 5, desc: 
tuples:
   Tuple(id=20 slots=[Slot(id=624 type=VARCHAR col=-1, colname=, nullable=1), 
Slot(id=625 type=VARCHAR col=-1, colname=, nullable=1), Slot(id=626 
type=VARCHAR col=-1, colname=, nullable=1)] has_varlen_slots=1)
   Tuple(id=19 slots=[Slot(id=623 type=DECIMAL128(38, 9) col=-1, 
colname=cost_130533_t1, nullable=1)] has_varlen_slots=0)
   Tuple(id=17 slots=[Slot(id=416 type=DECIMALV2(22, 5) col=-1, colname=cost, 
nullable=1), Slot(id=424 type=DATEV2 col=-1, colname=dt, nullable=1), 
Slot(id=444 type=VARCHAR col=-1, colname=, nullable=1), Slot(id=445 
type=VARCHAR col=-1, colname=, nullable=0), Slot(id=447 type=VARCHAR col=-1, 
colname=brand_code, nullable=1), Slot(id=460 type=DATEV2 col=-1, colname=dt, 
nullable=1), Slot(id=480 type=VARCHAR col=-1, colname=brand_code, nullable=1), 
Slot(id=493 type=DATEV2 col=-1, colname=dt, nullable=1)] has_varlen_slots=1)
   Tuple(id=16 slots=[Slot(id=410 type=DECIMAL128(38, 9) col=-1, 
colname=cost_130533_t1_1, nullable=1)] has_varlen_slots=0)
   Tuple(id=14 slots=[Slot(id=394 type=VARCHAR col=-1, colname=, nullable=1), 
Slot(id=395 type=VARCHAR col=-1, colname=, nullable=1), Slot(id=396 
type=VARCHAR col=-1, colname=, nullable=1), Slot(id=397 type=DECIMAL128(38, 9) 
col=-1, colname=, nullable=0)] has_varlen_slots=1)
   Tuple(id=21 slots=[Slot(id=627 type=VARCHAR col=-1, colname=, nullable=1), 
Slot(id=628 type=VARCHAR col=-1, colname=, nullable=1), Slot(id=629 
type=VARCHAR col=-1, colname=, nullable=1), Slot(id=630 type=DECIMAL128(38, 9) 
col=-1, colname=, nullable=1)] has_varlen_slots=1)
   Tuple(id=8 slots=[Slot(id=184 type=VARCHAR col=-1, colname=brand_code, 
nullable=1), Slot(id=189 type=DECIMALV2(22, 5) col=-1, colname=cost, 
nullable=1), Slot(id=197 type=DATEV2 col=-1, colname=dt, nullable=1)] 
has_varlen_slots=1)
   Tuple(id=18 slots=[Slot(id=518 type=DECIMALV2(22, 5) col=-1, colname=cost, 
nullable=1), Slot(id=526 type=DATEV2 col=-1, colname=dt, nullable=1), 
Slot(id=546 type=VARCHAR col=-1, colname=, nullable=1), Slot(id=547 
type=VARCHAR col=-1, colname=, nullable=0), Slot(id=548 type=DECIMALV2(22, 5) 
col=-1, colname=cost, nullable=1), Slot(id=549 type=VARCHAR col=-1, 
colname=brand_code, nullable=1), Slot(id=554 type=DECIMALV2(22, 5) col=-1, 
colname=cost, nullable=1), Slot(id=562 type=DATEV2 col=-1, colname=dt, 
nullable=1), Slot(id=582 type=VARCHAR col=-1, colname=brand_code, nullable=1), 
Slot(id=587 type=DECIMALV2(22, 5) col=-1, colname=cost, nullable=1), 
Slot(id=595 type=DATEV2 col=-1, colname=dt, nullable=1), Slot(id=615 
type=DATEV2 col=-1, colname=, nullable=1)] has_varlen_slots=1)
   Tuple(id=5 slots=[Slot(id=172 type=VARCHAR col=-1, colname=, nullable=1), 
Slot(id=173 type=VARCHAR col=-1, colname=, nullable=1), Slot(id=174 
type=VARCHAR col=-1, colname=, nullable=1)] has_varlen_slots=1)
   Tuple(id=0 slots=[Slot(id=0 type=VARCHAR col=-1, colname=brand_code, 
nullable=1), Slot(id=13 type=DATEV2 col=-1, colname=dt, nullable=1)] 
has_varlen_slots=1)
   
   
   ### What You Expected?
   
   This BUG need fixed.
   
   ### How to Reproduce?
   
   select * from (
   select (select a1,a2,sum(a3) from table_1 group by a1,a2) as tmp1 left join 
   (select a1,a2,sum(a3) from table_2 group by a1,a2) as tmp2 on 
tmp1.a1=tmp2.a1 and tmp1.a2=tmp2.a2
   ) as tmp3
   
   ### Anything Else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [FIX](map)fix map with rowstore table [doris]

2023-12-22 Thread via GitHub


amorynan commented on PR #28877:
URL: https://github.com/apache/doris/pull/28877#issuecomment-1867432558

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hash join) fix stack overflow caused by evaluate case expr on huge build block (#28851) [doris]

2023-12-22 Thread via GitHub


yiguolei merged PR #28871:
URL: https://github.com/apache/doris/pull/28871


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] (enhance)(S3) Change s3 metric from bvar adder to latency recorder [doris]

2023-12-22 Thread via GitHub


github-actions[bot] commented on code in PR #28861:
URL: https://github.com/apache/doris/pull/28861#discussion_r1434872534


##
be/src/util/s3_util.h:
##
@@ -19,6 +19,7 @@
 
 #include 

Review Comment:
   warning: 'aws/core/Aws.h' file not found [clang-diagnostic-error]
   ```cpp
   #include 
^
   ```
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [FIX](map)fix map with rowstore table [doris]

2023-12-22 Thread via GitHub


amorynan commented on PR #28877:
URL: https://github.com/apache/doris/pull/28877#issuecomment-1867435674

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch branch-1.2-lts updated: [fix](hash join) fix stack overflow caused by evaluate case expr on huge build block (#28871)

2023-12-22 Thread yiguolei
This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
 new ed4d5bf951f [fix](hash join) fix stack overflow caused by evaluate 
case expr on huge build block (#28871)
ed4d5bf951f is described below

commit ed4d5bf951f8c827fb0e778010775a2a20bce7c1
Author: TengJianPing <18241664+jackte...@users.noreply.github.com>
AuthorDate: Fri Dec 22 17:13:38 2023 +0800

[fix](hash join) fix stack overflow caused by evaluate case expr on huge 
build block (#28871)
---
 be/src/vec/columns/column_vector.cpp |  3 ++-
 be/src/vec/exec/join/vhash_join_node.cpp | 11 ++-
 be/src/vec/exec/join/vhash_join_node.h   |  1 +
 be/src/vec/functions/function_case.h | 20 ++--
 be/src/vec/functions/function_string.cpp | 28 +---
 be/src/vec/functions/multiply.cpp|  3 ++-
 6 files changed, 46 insertions(+), 20 deletions(-)

diff --git a/be/src/vec/columns/column_vector.cpp 
b/be/src/vec/columns/column_vector.cpp
index e656b97f2e7..93b59bce7ad 100644
--- a/be/src/vec/columns/column_vector.cpp
+++ b/be/src/vec/columns/column_vector.cpp
@@ -479,7 +479,8 @@ ColumnPtr ColumnVector::replicate(const 
IColumn::Offsets& offsets) const {
 res_data.reserve(offsets.back());
 
 // vectorized this code to speed up
-IColumn::Offset counts[size];
+auto counts_uptr = std::unique_ptr(new 
IColumn::Offset[size]);
+IColumn::Offset* counts = counts_uptr.get();
 for (ssize_t i = 0; i < size; ++i) {
 counts[i] = offsets[i] - offsets[i - 1];
 }
diff --git a/be/src/vec/exec/join/vhash_join_node.cpp 
b/be/src/vec/exec/join/vhash_join_node.cpp
index 3efea5d8c6a..b3c7794965e 100644
--- a/be/src/vec/exec/join/vhash_join_node.cpp
+++ b/be/src/vec/exec/join/vhash_join_node.cpp
@@ -656,7 +656,7 @@ Status HashJoinNode::_materialize_build_side(RuntimeState* 
state) {
 RETURN_IF_ERROR(child(1)->open(state));
 
 SCOPED_TIMER(_build_timer);
-MutableBlock mutable_block(child(1)->row_desc().tuple_descriptors());
+MutableBlock mutable_block;
 
 uint8_t index = 0;
 int64_t last_mem_used = 0;
@@ -669,6 +669,7 @@ Status HashJoinNode::_materialize_build_side(RuntimeState* 
state) {
 Block block;
 // If eos or have already met a null value using short-circuit 
strategy, we do not need to pull
 // data from data.
+_build_col_ids.resize(_build_expr_ctxs.size());
 while (!eos && !_short_circuit_for_null_in_probe_side) {
 block.clear_column_data();
 RETURN_IF_CANCELLED(state);
@@ -679,6 +680,8 @@ Status HashJoinNode::_materialize_build_side(RuntimeState* 
state) {
 _mem_used += block.allocated_bytes();
 
 if (block.rows() != 0) {
+RETURN_IF_ERROR(_do_evaluate(block, _build_expr_ctxs, 
*_build_expr_call_timer,
+ _build_col_ids));
 SCOPED_TIMER(_build_side_merge_block_timer);
 RETURN_IF_CATCH_BAD_ALLOC(mutable_block.merge(block));
 }
@@ -889,8 +892,6 @@ Status HashJoinNode::_process_build_block(RuntimeState* 
state, Block& block, uin
 ColumnRawPtrs raw_ptrs(_build_expr_ctxs.size());
 
 ColumnUInt8::MutablePtr null_map_val;
-std::vector res_col_ids(_build_expr_ctxs.size());
-RETURN_IF_ERROR(_do_evaluate(block, _build_expr_ctxs, 
*_build_expr_call_timer, res_col_ids));
 if (_join_op == TJoinOp::LEFT_OUTER_JOIN || _join_op == 
TJoinOp::FULL_OUTER_JOIN) {
 _convert_block_to_null(block);
 }
@@ -898,7 +899,7 @@ Status HashJoinNode::_process_build_block(RuntimeState* 
state, Block& block, uin
 //  so we have to initialize this flag by the first build block.
 if (!_has_set_need_null_map_for_build) {
 _has_set_need_null_map_for_build = true;
-_set_build_ignore_flag(block, res_col_ids);
+_set_build_ignore_flag(block, _build_col_ids);
 }
 if (_short_circuit_for_null_in_build_side || _build_side_ignore_null) {
 null_map_val = ColumnUInt8::create();
@@ -906,7 +907,7 @@ Status HashJoinNode::_process_build_block(RuntimeState* 
state, Block& block, uin
 }
 
 // Get the key column that needs to be built
-Status st = _extract_join_column(block, null_map_val, raw_ptrs, 
res_col_ids);
+Status st = _extract_join_column(block, null_map_val, raw_ptrs, 
_build_col_ids);
 
 st = std::visit(
 Overload {
diff --git a/be/src/vec/exec/join/vhash_join_node.h 
b/be/src/vec/exec/join/vhash_join_node.h
index 97af3818b8d..376ed8f377c 100644
--- a/be/src/vec/exec/join/vhash_join_node.h
+++ b/be/src/vec/exec/join/vhash_join_node.h
@@ -341,6 +341,7 @@ private:
 std::unordered_map> _inserted_rows;
 
 std::vector _runtime_filters;
+std::vector _build_col_ids;
 };
 } // namespace vectorized

Re: [PR] [Feature](inverted index) add lowercase option for inverted index analyzer [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28704:
URL: https://github.com/apache/doris/pull/28704#issuecomment-1867438132

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit 490c05e0613f7144282dca56d2abade4013743e1, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4709442644144414
   q2   377 151 158 151
   q3   1463126712621262
   q4   1129896 934 896
   q5   3193321031863186
   q6   248 130 129 129
   q7   981 488 492 488
   q8   2206220821912191
   q9   6677668966886688
   q10  3204325932653259
   q11  310 188 186 186
   q12  360 214 209 209
   q13  4553378338313783
   q14  240 220 225 220
   q15  578 521 520 520
   q16  453 395 387 387
   q17  1020587 547 547
   q18  7072700069016901
   q19  1549140514911405
   q20  543 310 286 286
   q21  3112268426652665
   q22  347 280 295 280
   Total cold run time: 44324 ms
   Total hot run time: 40053 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4383436543714365
   q2   269 167 176 167
   q3   3517353035023502
   q4   2384237123822371
   q5   5723574057205720
   q6   237 123 123 123
   q7   2369188118581858
   q8   3526352735183518
   q9   9002903090299029
   q10  3931402040034003
   q11  497 371 358 358
   q12  764 599 588 588
   q13  4301357735713571
   q14  299 265 250 250
   q15  577 520 529 520
   q16  509 467 440 440
   q17  1895186818731868
   q18  8594805081458050
   q19  1741174017551740
   q20  2291196519551955
   q21  6516617061766170
   q22  503 440 429 429
   Total cold run time: 63828 ms
   Total hot run time: 60595 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[I] [Enhancement] Catalog配置JDBC权限能否和设置用户权限一致 [doris]

2023-12-22 Thread via GitHub


MRYOG opened a new issue, #28880:
URL: https://github.com/apache/doris/issues/28880

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no 
similar issues.
   
   
   ### Description
   
   Catalog通过JDBC 方式连接的MySQL数据库,执行delete操作时,提示errCode = 2, detailMessage = 
External catalog 'up_mysql' is not allowed in 'DeleteStmt'
   但是配置的用户有对表进行增删改查的权限
   
   ### Solution
   
   catalog 权限能否与用户本身权限一致,不对删除等操作进行限制,可以通过本身数据库对用户的权限限制。或者doris中也可以对catalog 
进行权限限制
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](invert index) Empty strings are not written to the index in the… [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28822:
URL: https://github.com/apache/doris/pull/28822#issuecomment-1867441762

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.49 seconds
stream load tsv:  568 seconds loaded 74807831229 Bytes, about 125 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  29.1 seconds inserted 1000 Rows, about 
343K ops/s
storage size: 17184198005 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [FIX](map)fix map with rowstore table [doris]

2023-12-22 Thread via GitHub


github-actions[bot] commented on code in PR #28877:
URL: https://github.com/apache/doris/pull/28877#discussion_r1434875915


##
be/test/vec/jsonb/serialize_test.cpp:
##
@@ -177,6 +181,159 @@ TEST(BlockSerializeTest, Array) {
 EXPECT_EQ(block.dump_data(), new_block.dump_data());
 }
 
+TEST(BlockSerializeTest, Map) {

Review Comment:
   warning: function 'TEST' exceeds recommended size/complexity thresholds 
[readability-function-size]
   ```cpp
   TEST(BlockSerializeTest, Map) {
   ^
   ```
   
   Additional context
   
   **be/test/vec/jsonb/serialize_test.cpp:183:** 82 lines including whitespace 
and comments (threshold 80)
   ```cpp
   TEST(BlockSerializeTest, Map) {
   ^
   ```
   
   
   



##
be/test/vec/jsonb/serialize_test.cpp:
##
@@ -177,6 +181,159 @@
 EXPECT_EQ(block.dump_data(), new_block.dump_data());
 }
 
+TEST(BlockSerializeTest, Map) {
+TabletSchema schema;
+TabletColumn map;
+map.set_name("m");
+map.set_unique_id(1);
+map.set_type(FieldType::OLAP_FIELD_TYPE_MAP);
+schema.append_column(map);
+// map string string
+DataTypePtr s = 
std::make_shared(std::make_shared());
+DataTypePtr d = 
std::make_shared(std::make_shared());
+DataTypePtr m = std::make_shared(s, d);
+Array k1, k2, v1, v2;
+k1.push_back("null");
+k1.push_back("doris");
+k1.push_back("clever amory");
+v1.push_back("ss");
+v1.push_back(Null());
+v1.push_back("NULL");
+k2.push_back("hello amory");
+k2.push_back("NULL");
+k2.push_back("cute amory");
+k2.push_back("doris");
+v2.push_back("s");
+v2.push_back("0");
+v2.push_back("sf");
+v2.push_back(Null());
+Map m1, m2;
+m1.push_back(k1);
+m1.push_back(v1);
+m2.push_back(k2);
+m2.push_back(v2);
+MutableColumnPtr map_column = m->create_column();
+map_column->reserve(2);
+map_column->insert(m1);
+map_column->insert(m2);
+vectorized::ColumnWithTypeAndName type_and_name(map_column->get_ptr(), m, 
"test_map");
+vectorized::Block block;
+block.insert(type_and_name);
+
+MutableColumnPtr col = ColumnString::create();
+// serialize
+std::cout << "serialize to jsonb" << std::endl;
+JsonbSerializeUtil::block_to_jsonb(schema, block, 
static_cast(*col.get()),
+   block.columns(),
+   
create_data_type_serdes(block.get_data_types()));
+// deserialize
+TupleDescriptor read_desc(PTupleDescriptor(), true);
+// slot
+TSlotDescriptor tslot;
+tslot.__set_colName("m");
+tslot.nullIndicatorBit = -1;
+tslot.nullIndicatorByte = 0;
+TypeDescriptor type_desc(TYPE_MAP);
+type_desc.children.push_back(TypeDescriptor(TYPE_STRING));
+type_desc.children.push_back(TypeDescriptor(TYPE_INT));
+type_desc.contains_nulls.push_back(true);
+type_desc.contains_nulls.push_back(true);
+tslot.__set_col_unique_id(1);
+tslot.__set_slotType(type_desc.to_thrift());
+SlotDescriptor* slot = new SlotDescriptor(tslot);
+read_desc.add_slot(slot);
+
+Block new_block = block.clone_empty();
+std::unordered_map col_uid_to_idx;
+std::vector default_values;
+default_values.resize(read_desc.slots().size());
+for (int i = 0; i < read_desc.slots().size(); ++i) {
+col_uid_to_idx[read_desc.slots()[i]->col_unique_id()] = i;
+default_values[i] = read_desc.slots()[i]->col_default_value();
+std::cout << "uid " << read_desc.slots()[i]->col_unique_id() << ":" << 
i << std::endl;
+}
+std::cout << block.dump_data() << std::endl;
+std::cout << new_block.dump_data() << std::endl;
+std::cout << "deserialize from jsonb" << std::endl;
+
JsonbSerializeUtil::jsonb_to_block(create_data_type_serdes(read_desc.slots()),
+   static_cast(*col.get()), 
col_uid_to_idx,
+   new_block, default_values);
+std::cout << block.dump_data() << std::endl;
+std::cout << new_block.dump_data() << std::endl;
+EXPECT_EQ(block.dump_data(), new_block.dump_data());
+}
+
+TEST(BlockSerializeTest, Struct) {
+TabletSchema schema;
+TabletColumn struct_col;
+struct_col.set_name("struct");
+struct_col.set_unique_id(1);
+struct_col.set_type(FieldType::OLAP_FIELD_TYPE_STRUCT);
+schema.append_column(struct_col);
+vectorized::Block block;
+{
+DataTypePtr s = 
std::make_shared(std::make_shared());
+DataTypePtr d = 
std::make_shared(std::make_shared());
+DataTypePtr m = 
std::make_shared(std::make_shared());
+DataTypePtr st = 
std::make_shared(std::vector {s, d, m});
+Tuple t1, t2;
+t1.push_back(String("amory cute"));
+t1.push_back(__int128_t(37));
+t1.push_back(true);
+t2.push_back("null");
+t2.push_back(__int128_t(26));
+t2.push_back(false);
+MutableColumnPtr struct_column = st->create_colum

Re: [PR] [FIX](map)fix map with rowstore table [doris]

2023-12-22 Thread via GitHub


github-actions[bot] commented on code in PR #28877:
URL: https://github.com/apache/doris/pull/28877#discussion_r1434877793


##
be/test/vec/jsonb/serialize_test.cpp:
##
@@ -177,6 +181,159 @@ TEST(BlockSerializeTest, Array) {
 EXPECT_EQ(block.dump_data(), new_block.dump_data());
 }
 
+TEST(BlockSerializeTest, Map) {
+TabletSchema schema;
+TabletColumn map;
+map.set_name("m");
+map.set_unique_id(1);
+map.set_type(FieldType::OLAP_FIELD_TYPE_MAP);
+schema.append_column(map);
+// map string string
+DataTypePtr s = 
std::make_shared(std::make_shared());
+DataTypePtr d = 
std::make_shared(std::make_shared());
+DataTypePtr m = std::make_shared(s, d);
+Array k1, k2, v1, v2;
+k1.push_back("null");
+k1.push_back("doris");
+k1.push_back("clever amory");
+v1.push_back("ss");
+v1.push_back(Null());
+v1.push_back("NULL");
+k2.push_back("hello amory");
+k2.push_back("NULL");
+k2.push_back("cute amory");
+k2.push_back("doris");
+v2.push_back("s");
+v2.push_back("0");
+v2.push_back("sf");
+v2.push_back(Null());
+Map m1, m2;
+m1.push_back(k1);
+m1.push_back(v1);
+m2.push_back(k2);
+m2.push_back(v2);
+MutableColumnPtr map_column = m->create_column();
+map_column->reserve(2);
+map_column->insert(m1);
+map_column->insert(m2);
+vectorized::ColumnWithTypeAndName type_and_name(map_column->get_ptr(), m, 
"test_map");
+vectorized::Block block;
+block.insert(type_and_name);
+
+MutableColumnPtr col = ColumnString::create();
+// serialize
+std::cout << "serialize to jsonb" << std::endl;
+JsonbSerializeUtil::block_to_jsonb(schema, block, 
static_cast(*col.get()),
+   block.columns(),
+   
create_data_type_serdes(block.get_data_types()));
+// deserialize
+TupleDescriptor read_desc(PTupleDescriptor(), true);
+// slot
+TSlotDescriptor tslot;
+tslot.__set_colName("m");
+tslot.nullIndicatorBit = -1;
+tslot.nullIndicatorByte = 0;
+TypeDescriptor type_desc(TYPE_MAP);
+type_desc.children.push_back(TypeDescriptor(TYPE_STRING));
+type_desc.children.push_back(TypeDescriptor(TYPE_INT));
+type_desc.contains_nulls.push_back(true);
+type_desc.contains_nulls.push_back(true);
+tslot.__set_col_unique_id(1);
+tslot.__set_slotType(type_desc.to_thrift());
+SlotDescriptor* slot = new SlotDescriptor(tslot);

Review Comment:
   warning: use auto when initializing with new to avoid duplicating the type 
name [modernize-use-auto]
   
   ```suggestion
   auto* slot = new SlotDescriptor(tslot);
   ```
   



##
be/test/vec/jsonb/serialize_test.cpp:
##
@@ -177,6 +181,159 @@
 EXPECT_EQ(block.dump_data(), new_block.dump_data());
 }
 
+TEST(BlockSerializeTest, Map) {
+TabletSchema schema;
+TabletColumn map;
+map.set_name("m");
+map.set_unique_id(1);
+map.set_type(FieldType::OLAP_FIELD_TYPE_MAP);
+schema.append_column(map);
+// map string string
+DataTypePtr s = 
std::make_shared(std::make_shared());
+DataTypePtr d = 
std::make_shared(std::make_shared());
+DataTypePtr m = std::make_shared(s, d);
+Array k1, k2, v1, v2;
+k1.push_back("null");
+k1.push_back("doris");
+k1.push_back("clever amory");
+v1.push_back("ss");
+v1.push_back(Null());
+v1.push_back("NULL");
+k2.push_back("hello amory");
+k2.push_back("NULL");
+k2.push_back("cute amory");
+k2.push_back("doris");
+v2.push_back("s");
+v2.push_back("0");
+v2.push_back("sf");
+v2.push_back(Null());
+Map m1, m2;
+m1.push_back(k1);
+m1.push_back(v1);
+m2.push_back(k2);
+m2.push_back(v2);
+MutableColumnPtr map_column = m->create_column();
+map_column->reserve(2);
+map_column->insert(m1);
+map_column->insert(m2);
+vectorized::ColumnWithTypeAndName type_and_name(map_column->get_ptr(), m, 
"test_map");
+vectorized::Block block;
+block.insert(type_and_name);
+
+MutableColumnPtr col = ColumnString::create();
+// serialize
+std::cout << "serialize to jsonb" << std::endl;
+JsonbSerializeUtil::block_to_jsonb(schema, block, 
static_cast(*col.get()),
+   block.columns(),
+   
create_data_type_serdes(block.get_data_types()));
+// deserialize
+TupleDescriptor read_desc(PTupleDescriptor(), true);
+// slot
+TSlotDescriptor tslot;
+tslot.__set_colName("m");
+tslot.nullIndicatorBit = -1;
+tslot.nullIndicatorByte = 0;
+TypeDescriptor type_desc(TYPE_MAP);
+type_desc.children.push_back(TypeDescriptor(TYPE_STRING));
+type_desc.children.push_back(TypeDescriptor(TYPE_INT));
+type_desc.contains_nulls.push_back(true);
+type_desc.contains_nulls.push_back(true);
+tslot.__set_col_unique_id(1);
+tslot.__set_slotType(type_desc.to_

Re: [PR] [test](regression)Add select before analyze. [doris]

2023-12-22 Thread via GitHub


Jibing-Li closed pull request #28714: [test](regression)Add select before 
analyze.
URL: https://github.com/apache/doris/pull/28714


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement](group commit) refector replay wal function [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28756:
URL: https://github.com/apache/doris/pull/28756#issuecomment-1867445230

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit b41679ab54ad3eec3c9af8c9db7ad269a1550c57, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4928465646484648
   q2   375 159 198 159
   q3   1521131212411241
   q4   1175100010391000
   q5   3283325032653250
   q6   273 132 130 130
   q7   1050527 558 527
   q8   2271235523062306
   q9   6858683868786838
   q10  3247331432963296
   q11  325 187 185 185
   q12  392 221 215 215
   q13  4641398439353935
   q14  247 212 212 212
   q15  572 528 530 528
   q16  448 391 396 391
   q17  1017646 508 508
   q18  7140702268436843
   q19  1630149915271499
   q20  563 331 304 304
   q21  3322288129012881
   q22  358 283 288 283
   Total cold run time: 45636 ms
   Total hot run time: 41179 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4579456745764567
   q2   293 190 198 190
   q3   3601358335743574
   q4   2455244124382438
   q5   5827584458315831
   q6   261 124 124 124
   q7   2412188018461846
   q8   3655363536473635
   q9   9063903290019001
   q10  3968405840564056
   q11  520 395 400 395
   q12  789 586 592 586
   q13  4396367837183678
   q14  295 258 256 256
   q15  587 538 539 538
   q16  514 459 482 459
   q17  1971190318821882
   q18  8633835883368336
   q19  1852188318741874
   q20  2274197919591959
   q21  6734630362946294
   q22  509 423 429 423
   Total cold run time: 65188 ms
   Total hot run time: 61942 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [pipelineX](refactor) rename functions [doris]

2023-12-22 Thread via GitHub


yiguolei merged PR #28846:
URL: https://github.com/apache/doris/pull/28846


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch master updated: [pipelineX](refactor) rename functions (#28846)

2023-12-22 Thread yiguolei
This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 9e0a2e861cf [pipelineX](refactor) rename functions (#28846)
9e0a2e861cf is described below

commit 9e0a2e861cf05fcb7cb22fbcfe95955b16ab7f73
Author: Gabriel 
AuthorDate: Fri Dec 22 17:24:39 2023 +0800

[pipelineX](refactor) rename functions (#28846)
---
 be/src/pipeline/exec/aggregation_sink_operator.h   | 4 ++--
 be/src/pipeline/exec/analytic_sink_operator.h  | 4 ++--
 be/src/pipeline/exec/assert_num_rows_operator.h| 2 +-
 be/src/pipeline/exec/exchange_source_operator.h| 2 +-
 be/src/pipeline/exec/hashjoin_build_sink.h | 2 +-
 be/src/pipeline/exec/hashjoin_probe_operator.h | 2 +-
 be/src/pipeline/exec/nested_loop_join_build_operator.h | 2 +-
 be/src/pipeline/exec/nested_loop_join_probe_operator.h | 2 +-
 be/src/pipeline/exec/partition_sort_sink_operator.h| 4 ++--
 be/src/pipeline/exec/scan_operator.h   | 2 +-
 be/src/pipeline/exec/set_probe_sink_operator.h | 2 +-
 be/src/pipeline/exec/set_sink_operator.h   | 2 +-
 be/src/pipeline/exec/sort_sink_operator.h  | 4 ++--
 be/src/pipeline/exec/streaming_aggregation_sink_operator.h | 2 +-
 be/src/pipeline/pipeline.h | 2 +-
 be/src/pipeline/pipeline_x/operator.h  | 4 ++--
 be/src/pipeline/pipeline_x/pipeline_x_fragment_context.cpp | 8 
 17 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/be/src/pipeline/exec/aggregation_sink_operator.h 
b/be/src/pipeline/exec/aggregation_sink_operator.h
index 2cd6ef50939..97be9dcd6a3 100644
--- a/be/src/pipeline/exec/aggregation_sink_operator.h
+++ b/be/src/pipeline/exec/aggregation_sink_operator.h
@@ -366,12 +366,12 @@ public:
 Status sink(RuntimeState* state, vectorized::Block* in_block,
 SourceState source_state) override;
 
-DataDistribution get_local_exchange_type() const override {
+DataDistribution required_data_distribution() const override {
 if (_probe_expr_ctxs.empty()) {
 return _needs_finalize || 
DataSinkOperatorX::_child_x
   ->ignore_data_distribution()
? DataDistribution(ExchangeType::PASSTHROUGH)
-   : 
DataSinkOperatorX::get_local_exchange_type();
+   : 
DataSinkOperatorX::required_data_distribution();
 }
 return _is_colocate ? 
DataDistribution(ExchangeType::BUCKET_HASH_SHUFFLE, _partition_exprs)
 : DataDistribution(ExchangeType::HASH_SHUFFLE, 
_partition_exprs);
diff --git a/be/src/pipeline/exec/analytic_sink_operator.h 
b/be/src/pipeline/exec/analytic_sink_operator.h
index 14ed8c815b1..3e0eb85f76d 100644
--- a/be/src/pipeline/exec/analytic_sink_operator.h
+++ b/be/src/pipeline/exec/analytic_sink_operator.h
@@ -107,7 +107,7 @@ public:
 
 Status sink(RuntimeState* state, vectorized::Block* in_block,
 SourceState source_state) override;
-DataDistribution get_local_exchange_type() const override {
+DataDistribution required_data_distribution() const override {
 if (_partition_by_eq_expr_ctxs.empty()) {
 return {ExchangeType::PASSTHROUGH};
 } else if (_order_by_eq_expr_ctxs.empty()) {
@@ -115,7 +115,7 @@ public:
? 
DataDistribution(ExchangeType::BUCKET_HASH_SHUFFLE, _partition_exprs)
: DataDistribution(ExchangeType::HASH_SHUFFLE, 
_partition_exprs);
 }
-return 
DataSinkOperatorX::get_local_exchange_type();
+return 
DataSinkOperatorX::required_data_distribution();
 }
 
 private:
diff --git a/be/src/pipeline/exec/assert_num_rows_operator.h 
b/be/src/pipeline/exec/assert_num_rows_operator.h
index 1e796b622dc..bb5e65168b6 100644
--- a/be/src/pipeline/exec/assert_num_rows_operator.h
+++ b/be/src/pipeline/exec/assert_num_rows_operator.h
@@ -57,7 +57,7 @@ public:
 
 [[nodiscard]] bool is_source() const override { return false; }
 
-DataDistribution get_local_exchange_type() const override {
+DataDistribution required_data_distribution() const override {
 return {ExchangeType::PASSTHROUGH};
 }
 
diff --git a/be/src/pipeline/exec/exchange_source_operator.h 
b/be/src/pipeline/exec/exchange_source_operator.h
index 221a43779a1..b621da38072 100644
--- a/be/src/pipeline/exec/exchange_source_operator.h
+++ b/be/src/pipeline/exec/exchange_source_operator.h
@@ -117,7 +117,7 @@ public:
 return _sub_plan_query_statistics_recvr;
 }
 
-DataDistribution get_local_exchange_type() const override {
+DataDistribution required_data_distribution() const override {
 if (Oper

Error while running github feature from .asf.yaml in doris!

2023-12-22 Thread Apache Infrastructure


An error occurred while running github feature in .asf.yaml!:
You can only have a maximum of 10 external triage collaborators, please contact 
vp-in...@apache.org to request an exception.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hash join) fix stack overflow caused by evaluate case expr on huge build block (#28823) [doris]

2023-12-22 Thread via GitHub


yiguolei closed pull request #28836: [fix](hash join) fix stack overflow caused 
by evaluate case expr on huge build block (#28823)
URL: https://github.com/apache/doris/pull/28836


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hash join) fix stack overflow caused by evaluate case expr on huge build block (#28823) [doris]

2023-12-22 Thread via GitHub


yiguolei commented on PR #28836:
URL: https://github.com/apache/doris/pull/28836#issuecomment-1867450195

   close the PR temporary, keep the branch


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [FIX](complextype) fix complextype has empty literal may make be core [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28820:
URL: https://github.com/apache/doris/pull/28820#issuecomment-1867451565

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 51.73 seconds
stream load tsv:  569 seconds loaded 74807831229 Bytes, about 125 
MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc:  65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  29.7 seconds inserted 1000 Rows, about 
336K ops/s
storage size: 17164713746 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [exec](compress) use FragmentTransmissionCompressionCodec control the exchange compress behavior [doris]

2023-12-22 Thread via GitHub


yiguolei commented on PR #28818:
URL: https://github.com/apache/doris/pull/28818#issuecomment-1867451815

   run p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](nereids) "not is null" stats estimation fix [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28860:
URL: https://github.com/apache/doris/pull/28860#issuecomment-1867452270

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit f149a78140b036af21fbd049036f397fd228bd8a, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4716439344544393
   q2   371 145 139 139
   q3   1460131812551255
   q4   1112914 881 881
   q5   3154317231483148
   q6   249 131 135 131
   q7   1004492 490 490
   q8   2192222821752175
   q9   6701662366216621
   q10  3224325932773259
   q11  310 185 182 182
   q12  349 203 200 200
   q13  4528379937893789
   q14  245 212 212 212
   q15  581 528 531 528
   q16  440 382 384 382
   q17  1016636 502 502
   q18  7109687068126812
   q19  1548136414811364
   q20  550 313 322 313
   q21  3074268126772677
   q22  344 277 283 277
   Total cold run time: 44277 ms
   Total hot run time: 39730 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4336436343524352
   q2   271 161 170 161
   q3   3516351034933493
   q4   2370237423652365
   q5   5726569557135695
   q6   240 123 124 123
   q7   2367187118441844
   q8   3517353435223522
   q9   8996900289678967
   q10  3905402540334025
   q11  477 369 371 369
   q12  774 609 607 607
   q13  4313357735513551
   q14  285 251 260 251
   q15  590 520 527 520
   q16  507 451 473 451
   q17  1893185119321851
   q18  8625816381728163
   q19  1731178617341734
   q20  2267194019471940
   q21  6529613261116111
   q22  490 419 412 412
   Total cold run time: 63725 ms
   Total hot run time: 60507 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [FIX](complextype)fix count func with complex type [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28873:
URL: https://github.com/apache/doris/pull/28873#issuecomment-1867452646

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.77 seconds
stream load tsv:  583 seconds loaded 74807831229 Bytes, about 122 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.8 seconds inserted 1000 Rows, about 
347K ops/s
storage size: 17183949907 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](paimon)fix `like` predicate [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28803:
URL: https://github.com/apache/doris/pull/28803#issuecomment-1867454758

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.58 seconds
stream load tsv:  582 seconds loaded 74807831229 Bytes, about 122 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.8 seconds inserted 1000 Rows, about 
347K ops/s
storage size: 17183490722 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [nereids] fix join fd computing bug [doris]

2023-12-22 Thread via GitHub


xzj7019 commented on PR #28849:
URL: https://github.com/apache/doris/pull/28849#issuecomment-1867457643

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](Nereids) Pull up join from union all [doris]

2023-12-22 Thread via GitHub


xzj7019 commented on PR #28682:
URL: https://github.com/apache/doris/pull/28682#issuecomment-1867458426

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [hotfix](jdbc catalog) fix load table and column names npe [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28865:
URL: https://github.com/apache/doris/pull/28865#issuecomment-1867459505

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit a2a5adbe6d2499a5f47f604175473056e66f5975, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4682441444284414
   q2   372 143 157 143
   q3   1465125212081208
   q4   1114881 927 881
   q5   3190318131893181
   q6   250 133 131 131
   q7   1002484 496 484
   q8   2204222522072207
   q9   6706662566596625
   q10  3211329632823282
   q11  315 189 184 184
   q12  352 210 208 208
   q13  4587383038293829
   q14  241 219 217 217
   q15  569 523 529 523
   q16  441 374 381 374
   q17  1027663 561 561
   q18  7025697467556755
   q19  1556143314241424
   q20  510 328 283 283
   q21  3093260426492604
   q22  350 278 282 278
   Total cold run time: 44262 ms
   Total hot run time: 39796 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4351434343654343
   q2   272 165 173 165
   q3   3534350035223500
   q4   2386238223712371
   q5   5740572557155715
   q6   243 125 126 125
   q7   2379185918951859
   q8   3512352835453528
   q9   9055902590479025
   q10  3937400040044000
   q11  480 372 364 364
   q12  775 606 590 590
   q13  4277354635253525
   q14  286 250 256 250
   q15  584 525 521 521
   q16  514 458 459 458
   q17  1903188218431843
   q18  8529796481057964
   q19  1737174317581743
   q20  2251195419411941
   q21  6486614161186118
   q22  508 426 432 426
   Total cold run time: 63739 ms
   Total hot run time: 60374 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [FIX](map)fix map with rowstore table [doris]

2023-12-22 Thread via GitHub


github-actions[bot] commented on PR #28877:
URL: https://github.com/apache/doris/pull/28877#issuecomment-1867459955

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [FIX](map)fix map with rowstore table [doris]

2023-12-22 Thread via GitHub


github-actions[bot] commented on PR #28877:
URL: https://github.com/apache/doris/pull/28877#issuecomment-1867460007

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](ci) tpch pipeline should not re-load data [doris]

2023-12-22 Thread via GitHub


hello-stephen commented on PR #28874:
URL: https://github.com/apache/doris/pull/28874#issuecomment-1867462805

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](pipelineX) control exchange sink by memory usage [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28814:
URL: https://github.com/apache/doris/pull/28814#issuecomment-1867463778

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.79 seconds
stream load tsv:  566 seconds loaded 74807831229 Bytes, about 126 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  27.9 seconds inserted 1000 Rows, about 
358K ops/s
storage size: 17183821535 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refactor](pipelineX)do not override dependency() function in pipelineX [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28848:
URL: https://github.com/apache/doris/pull/28848#issuecomment-1867466558

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit 5af4df47e484d94afe251b74a67dc31c3c02c62a, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4673440344124403
   q2   367 160 158 158
   q3   1464123112481231
   q4   1102944 863 863
   q5   3165317531773175
   q6   247 132 127 127
   q7   994 488 488 488
   q8   2151224721892189
   q9   6670665866986658
   q10  3236326632713266
   q11  311 188 188 188
   q12  354 213 208 208
   q13  4551378538073785
   q14  247 222 217 217
   q15  565 522 526 522
   q16  437 382 388 382
   q17  1020625 589 589
   q18  7142684668576846
   q19  1547144914581449
   q20  550 325 294 294
   q21  3060267326682668
   q22  350 283 295 283
   Total cold run time: 44203 ms
   Total hot run time: 39989 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4346434343354335
   q2   274 165 166 165
   q3   3508350434903490
   q4   2387236723652365
   q5   5746572157275721
   q6   239 121 122 121
   q7   2382188218361836
   q8   3516351835233518
   q9   9027895890128958
   q10  3911401040474010
   q11  491 368 368 368
   q12  771 593 590 590
   q13  4295356735243524
   q14  288 257 247 247
   q15  578 522 518 518
   q16  505 476 461 461
   q17  1875183518461835
   q18  8588821281278127
   q19  1728175917361736
   q20  2263194119351935
   q21  6512615261436143
   q22  500 424 419 419
   Total cold run time: 63730 ms
   Total hot run time: 60422 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[I] [Bug] doris be 自动退出 [doris]

2023-12-22 Thread via GitHub


haoyan19881215 opened a new issue, #28881:
URL: https://github.com/apache/doris/issues/28881

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no 
similar issues.
   
   
   ### Version
   
   2.0.2
   
   ### What's Wrong?
   
   there are 8 be in my doris cluster,but 2 be Automatic stop when they running 
for 1~2 hours。but I cannot find any exception in logs
   **be.out**
   `
   start time: Fri Dec 22 09:47:07 CST 2023
   INFO: java_cmd /opt/jdk1.8.0_191/bin/java
   INFO: jdk_version 8
   SLF4J: Class path contains multiple SLF4J bindings.
   SLF4J: Found binding in 
[jar:file:/opt/apache-doris-2.0.2-bin-x64-noavx2/be/lib/java_extensions/preload-extensions/preload-extensions-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   SLF4J: Found binding in 
[jar:file:/opt/apache-doris-2.0.2-bin-x64-noavx2/be/lib/java_extensions/java-udf/java-udf-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   SLF4J: Found binding in 
[jar:file:/opt/apache-doris-2.0.2-bin-x64-noavx2/be/lib/hadoop_hdfs/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
explanation.
   SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
   Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library 
/opt/apache-doris-2.0.2-bin-x64-noavx2/be/lib/hadoop_hdfs/native/libhadoop.so.1.0.0
 which might have disabled stack guard. The VM will try to fix the stack guard 
now.
   It's highly recommended that you fix the library with 'execstack -c 
', or link it with '-z noexecstack'.
   `
   **be.INFO**
   `
   I1222 11:57:49.784793 24196 olap_server.cpp:1064] cooldown producer get 
tablet num: 0
   I1222 11:57:54.344010 24947 heartbeat_server.cpp:61] get heartbeat from 
FE.host:10.237.22.118, port:9020, cluster id:1914286766, counter:1561, BE start 
time: 1703209629783
   I1222 11:57:57.641873 24283 task_worker_pool.cpp:1068] successfully report 
TASK|host=10.237.22.118|port=9020
   I1222 11:58:09.578222 24038 load_channel_mgr.cpp:250] cleaning timed out 
load channels
   I1222 11:58:09.578332 24038 load_channel_mgr.cpp:282] load mem 
consumption(bytes). limit: 13420866764, current: 0, peak: 0, total running load 
channels: 0
   I1222 11:58:09.785163 24196 olap_server.cpp:1064] cooldown producer get 
tablet num: 0
   I1222 11:58:10.643011 24283 task_worker_pool.cpp:1068] successfully report 
TASK|host=10.237.22.118|port=9020
   I1222 11:58:23.643994 24283 task_worker_pool.cpp:1068] successfully report 
TASK|host=10.237.22.118|port=9020
   I1222 11:58:29.785368 24196 olap_server.cpp:1064] cooldown producer get 
tablet num: 0
   I1222 11:58:31.761829 24285 tablet_manager.cpp:1016] find expired 
transactions for 0 tablets
   I1222 11:58:31.761873 24285 tablet_manager.cpp:1048] success to build all 
report tablets info. tablet_count=0
   I1222 11:58:31.762550 24285 task_worker_pool.cpp:1068] successfully report 
TABLET|host=10.237.22.118|port=9020
   I1222 11:58:36.864832 23487 daemon.cpp:397] doris start to exit
   I1222 11:58:38.206190 24284 data_dir.cpp:810] path: 
/opt/apache-doris-2.0.2-bin-x64-noavx2/be/storage total capacity: 
1022174953472, available capacity: 1013264003072
   I1222 11:58:38.206262 24284 storage_engine.cpp:383] get root path info cost: 
0 ms. tablet counter: 0
   I1222 11:58:38.207151 24284 task_worker_pool.cpp:1068] successfully report 
DISK|host=10.237.22.118|port=9020
   I1222 11:58:38.644892 24283 task_worker_pool.cpp:1068] successfully report 
TASK|host=10.237.22.118|port=9020
   I1222 11:58:39.867292 23487 server.cpp:1167] 
Server[doris::PInternalServiceImpl] is going to quit
   I1222 11:58:39.892066 24882 thrift_server.cpp:170] ThriftServer heartbeat 
exited
   I1222 11:58:39.892594 24289 thrift_server.cpp:170] ThriftServer backend 
exited
   I1222 11:58:39.892784 23487 storage_engine.cpp:546] begin stopping storage 
engine
   I1222 11:58:39.892843 24190 olap_server.cpp:364] try to perform path gc by 
rowsetid!
   I1222 11:58:39.893136 23487 storage_engine.cpp:566] start join garbage 
sweeper thread
   I1222 11:58:39.893158 23487 storage_engine.cpp:568] end join garbage sweeper 
thread
   I1222 11:58:39.893258 23487 storage_engine.cpp:588] end stopping storage 
engine
   I1222 11:58:40.893028 24284 data_dir.cpp:810] path: 
/opt/apache-doris-2.0.2-bin-x64-noavx2/be/storage total capacity: 
1022174953472, available capacity: 1013264003072
   I1222 11:58:40.893131 24284 storage_engine.cpp:383] get root path info cost: 
0 ms. tablet counter: 0
   I1222 11:58:40.893143 24285 tablet_manager.cpp:1016] find expired 
transactions for 0 tablets
   I1222 11:58:40.893206 24285 tablet_manager.cpp:1048] success to build all 
report tablets info. tablet_count=0
   I1222 11:58:40.893572 24283 task_worker_pool.cpp:1068] successfully report 
TASK|host=10.237.22.118|port=9020
   I1222 11:58:40.8

Re: [PR] [fix](pipelineX) fix use global rf when there no shared_scans [doris]

2023-12-22 Thread via GitHub


doris-robot commented on PR #28869:
URL: https://github.com/apache/doris/pull/28869#issuecomment-1867473443

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit cd467df94f2ed4a92a8007ac5582fcf1a05e29bf, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4673441744524417
   q2   377 138 157 138
   q3   1451127611761176
   q4   1115922 919 919
   q5   3153314131693141
   q6   250 128 124 124
   q7   1009491 499 491
   q8   2189221621982198
   q9   6714665566836655
   q10  3216326832533253
   q11  301 191 187 187
   q12  355 204 202 202
   q13  4529378537763776
   q14  242 217 216 216
   q15  579 529 520 520
   q16  440 394 385 385
   q17  1013613 533 533
   q18  7033674868516748
   q19  1536142015301420
   q20  518 317 304 304
   q21  3136266926832669
   q22  352 275 287 275
   Total cold run time: 44181 ms
   Total hot run time: 39747 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4346435443364336
   q2   269 166 171 166
   q3   3508349234853485
   q4   2377236623662366
   q5   5707569657315696
   q6   240 122 120 120
   q7   2366186118851861
   q8   3545351736523517
   q9   9033902789778977
   q10  3935400339783978
   q11  484 363 369 363
   q12  754 593 606 593
   q13  4274355235443544
   q14  295 256 258 256
   q15  576 514 527 514
   q16  506 473 452 452
   q17  1896186818601860
   q18  8594824079927992
   q19  1747178717861786
   q20  2245197219271927
   q21  6497615061256125
   q22  507 442 416 416
   Total cold run time: 63701 ms
   Total hot run time: 60330 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [fix](hash join) fix stack overflow caused by evaluate case expr on huge build block (#28851) [doris]

2023-12-22 Thread via GitHub


jacktengg opened a new pull request, #28882:
URL: https://github.com/apache/doris/pull/28882

   
   ## Proposed changes
   
   Pick from #28851
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hash join) fix stack overflow caused by evaluate case expr on huge build block (#28851) [doris]

2023-12-22 Thread via GitHub


jacktengg commented on PR #28882:
URL: https://github.com/apache/doris/pull/28882#issuecomment-1867474871

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [improve](load) limit delta writer flush task parallelism [doris]

2023-12-22 Thread via GitHub


kaijchen opened a new pull request, #28883:
URL: https://github.com/apache/doris/pull/28883

   ## Proposed changes
   
   Add a flush task parallelism limit for each delta writer.
   
   If flush is fast enough, this limit will do nothing since there will be no 
task waiting.
   If flush is the bottleneck, this limit could reduce memory usage.
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improve](load) limit delta writer flush task parallelism [doris]

2023-12-22 Thread via GitHub


kaijchen commented on PR #28883:
URL: https://github.com/apache/doris/pull/28883#issuecomment-1867478451

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [branch-2-0](http) add show_table_data http api [doris]

2023-12-22 Thread via GitHub


nextdreamblue commented on PR #28633:
URL: https://github.com/apache/doris/pull/28633#issuecomment-1867479357

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](hash join) fix stack overflow caused by evaluate case expr on huge build block (#28851) [doris]

2023-12-22 Thread via GitHub


github-actions[bot] commented on PR #28882:
URL: https://github.com/apache/doris/pull/28882#issuecomment-1867479832

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



  1   2   3   4   5   6   >