[GitHub] [doris] XieJiann commented on pull request #23864: [feature](Nereids) multi array contains to array overlap
XieJiann commented on PR #23864: URL: https://github.com/apache/doris/pull/23864#issuecomment-1707782001 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23236: [Feature](partitions) Support auto partition
github-actions[bot] commented on PR #23236: URL: https://github.com/apache/doris/pull/23236#issuecomment-1707784284 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23236: [Feature](partitions) Support auto partition
github-actions[bot] commented on PR #23236: URL: https://github.com/apache/doris/pull/23236#issuecomment-1707784347 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #23954: [pipelineX](feature) support select operator and add TPCH test cases
doris-robot commented on PR #23954: URL: https://github.com/apache/doris/pull/23954#issuecomment-1707788148 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 48.22 seconds stream load tsv: 538 seconds loaded 74807831229 Bytes, about 132 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 28.9 seconds inserted 1000 Rows, about 346K ops/s storage size: 17162081587 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23933: [opt](Nereids) optimize error msg of unbound slot
github-actions[bot] commented on PR #23933: URL: https://github.com/apache/doris/pull/23933#issuecomment-1707789774 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] platoneko commented on pull request #23953: [feature](merge-cloud) Rewrite FS interface
platoneko commented on PR #23953: URL: https://github.com/apache/doris/pull/23953#issuecomment-1707789527 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23933: [opt](Nereids) optimize error msg of unbound slot
github-actions[bot] commented on PR #23933: URL: https://github.com/apache/doris/pull/23933#issuecomment-1707789722 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #23933: [opt](Nereids) optimize error msg of unbound slot
morrySnow commented on PR #23933: URL: https://github.com/apache/doris/pull/23933#issuecomment-1707789961 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman commented on pull request #23845: [fix](explain) fix explain physical plan with external table issue
morningman commented on PR #23845: URL: https://github.com/apache/doris/pull/23845#issuecomment-1707791428 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #23863: [Improvement] (pipeline) Cancel related query if backend restarts or dead
doris-robot commented on PR #23863: URL: https://github.com/apache/doris/pull/23863#issuecomment-1707792010 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 47.68 seconds stream load tsv: 536 seconds loaded 74807831229 Bytes, about 133 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 28.8 seconds inserted 1000 Rows, about 347K ops/s storage size: 17162120996 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yujun777 commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
yujun777 commented on PR #21118: URL: https://github.com/apache/doris/pull/21118#issuecomment-1707792471 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #23898: [fix](auth)Fix create user no permissions of information_schema database
doris-robot commented on PR #23898: URL: https://github.com/apache/doris/pull/23898#issuecomment-1707799320 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 49.81 seconds stream load tsv: 537 seconds loaded 74807831229 Bytes, about 132 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 28.8 seconds inserted 1000 Rows, about 347K ops/s storage size: 17162184339 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei opened a new pull request, #23957: [bugfix](datastreamsender) receiver will hang during close
yiguolei opened a new pull request, #23957: URL: https://github.com/apache/doris/pull/23957 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] TangSiyang2001 commented on pull request #23793: [Draft][cherry-pick](branch-2.0) support delete sub pred v2
TangSiyang2001 commented on PR #23793: URL: https://github.com/apache/doris/pull/23793#issuecomment-1707801106 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23793: [Draft][cherry-pick](branch-2.0) support delete sub pred v2
github-actions[bot] commented on PR #23793: URL: https://github.com/apache/doris/pull/23793#issuecomment-1707808483 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly opened a new pull request, #23958: [fix](nereids) bug: runtimefilter target is not SlotReference
englefly opened a new pull request, #23958: URL: https://github.com/apache/doris/pull/23958 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yujun777 commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
yujun777 commented on PR #21118: URL: https://github.com/apache/doris/pull/21118#issuecomment-1707812158 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #23863: [Improvement] (pipeline) Cancel related query if backend restarts or dead
yiguolei commented on code in PR #23863:
URL: https://github.com/apache/doris/pull/23863#discussion_r1316851604
##
fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java:
##
@@ -1252,6 +1253,92 @@ public RowBatch getNext() throws Exception {
return resultBatch;
}
+
+// We use a very conservative cancel strategy.
+// 0. If backends has zero process epoch, do not cancel. Zero process
epoch usually arises in cluster upgrading.
+// 1. If process epoch is same, do not cancel. Means backends does not
restart or die.
+public boolean shouldCancel(List currentBackends) {
+Map curBeMap = Maps.newHashMap();
+for (Backend be : currentBackends) {
+curBeMap.put(be.getId(), be);
+}
+lock();
Review Comment:
use try (lock()) like RAII in cpp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jacktengg opened a new pull request, #23959: [improvement](bitmap) support version for ser/deser of bitmap
jacktengg opened a new pull request, #23959: URL: https://github.com/apache/doris/pull/23959 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23954: [pipelineX](feature) support select operator and add TPCH test cases
github-actions[bot] commented on PR #23954: URL: https://github.com/apache/doris/pull/23954#issuecomment-1707818227 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] bobhan1 opened a new pull request, #23960: [Fix](regression) Fix `test_partial_update_schema_change`
bobhan1 opened a new pull request, #23960: URL: https://github.com/apache/doris/pull/23960 ## Proposed changes ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23959: [improvement](bitmap) support version for ser/deser of bitmap
github-actions[bot] commented on PR #23959: URL: https://github.com/apache/doris/pull/23959#issuecomment-1707818876 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] bobhan1 commented on pull request #23960: [Fix](regression) Fix `test_partial_update_schema_change`
bobhan1 commented on PR #23960: URL: https://github.com/apache/doris/pull/23960#issuecomment-1707820203 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #23864: [feature](Nereids) multi array contains to array overlap
doris-robot commented on PR #23864: URL: https://github.com/apache/doris/pull/23864#issuecomment-1707822281 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 48.46 seconds stream load tsv: 529 seconds loaded 74807831229 Bytes, about 134 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 28.9 seconds inserted 1000 Rows, about 346K ops/s storage size: 17162053514 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhiqiang-hhhh commented on a diff in pull request #23863: [Improvement] (pipeline) Cancel related query if backend restarts or dead
zhiqiang- commented on code in PR #23863:
URL: https://github.com/apache/doris/pull/23863#discussion_r1316859355
##
fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java:
##
@@ -1252,6 +1253,92 @@ public RowBatch getNext() throws Exception {
return resultBatch;
}
+
+// We use a very conservative cancel strategy.
+// 0. If backends has zero process epoch, do not cancel. Zero process
epoch usually arises in cluster upgrading.
+// 1. If process epoch is same, do not cancel. Means backends does not
restart or die.
+public boolean shouldCancel(List currentBackends) {
+Map curBeMap = Maps.newHashMap();
+for (Backend be : currentBackends) {
+curBeMap.put(be.getId(), be);
+}
+lock();
Review Comment:
but here we do not have potential thrown exception? can we use try {lock()}
finally {unlock();} either?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HappenLee opened a new pull request, #23961: [pipeline](log) disable some unless error status warning log
HappenLee opened a new pull request, #23961: URL: https://github.com/apache/doris/pull/23961 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #23961: [pipeline](log) disable some unless error status warning log
BiteThet commented on PR #23961: URL: https://github.com/apache/doris/pull/23961#issuecomment-1707823019 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman opened a new pull request, #23962: [improvement](jdbc) add profile for jdbc read and convert phase
morningman opened a new pull request, #23962: URL: https://github.com/apache/doris/pull/23962 ## Proposed changes Add 2 metrics in jdbc scan node profile: - `CallJniNextTime`: call get next from jdbc result set - `ConvertBatchTime`: call convert jobject to columm block Also fix a potential concurrency issue when init jdbc connection cache pool ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 merged pull request #23954: [pipelineX](feature) support select operator and add TPCH test cases
Gabriel39 merged PR #23954: URL: https://github.com/apache/doris/pull/23954 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (3047d7dd07 -> 7625d1514a)
This is an automated email from the ASF dual-hosted git repository. gabriellee pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 3047d7dd07 [fix](Nereids) fix or to in rule (#23940) add 7625d1514a [pipelineX](feature) support select operator and add TPCH test cases (#23954) No new revisions were added by this update. Summary of changes: be/src/pipeline/exec/exchange_sink_operator.cpp| 13 +- be/src/pipeline/exec/scan_operator.cpp |2 +- be/src/pipeline/exec/scan_operator.h |2 +- be/src/pipeline/exec/select_operator.h | 31 + be/src/pipeline/pipeline_x/operator.cpp|2 + .../pipeline_x/pipeline_x_fragment_context.cpp |6 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q01.out |7 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q02.out | 47 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q03.out | 13 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q04.out |8 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q05.out |8 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q06.out |4 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q07.out |7 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q08.out |5 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q09.out | 178 ++ .../tpch_unique_sql_zstd_p0/sql/pipelinex_q10.out | 23 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q11.out | 2544 ++ .../tpch_unique_sql_zstd_p0/sql/pipelinex_q12.out |5 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q13.out | 40 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q14.out |4 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q15.out |4 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q16.out | 2765 .../tpch_unique_sql_zstd_p0/sql/pipelinex_q17.out |4 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q18.out |8 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q19.out |4 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q20.out | 12 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q21.out | 50 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q22.out | 10 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q01.sql | 22 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q02.sql | 42 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q03.sql | 24 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q04.sql | 19 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q05.sql | 25 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q06.sql | 11 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q07.sql | 39 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q08.sql | 38 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q09.sql | 32 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q10.sql | 33 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q11.sql | 28 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q12.sql | 29 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q13.sql | 21 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q14.sql | 13 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q15.sql | 20 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q16.sql | 30 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q17.sql | 16 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q18.sql | 34 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q19.sql | 35 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q20.sql | 33 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q21.sql | 39 + .../tpch_unique_sql_zstd_p0/sql/pipelinex_q22.sql | 35 + 50 files changed, 6418 insertions(+), 6 deletions(-) create mode 100644 regression-test/data/tpch_unique_sql_zstd_p0/sql/pipelinex_q01.out create mode 100644 regression-test/data/tpch_unique_sql_zstd_p0/sql/pipelinex_q02.out create mode 100644 regression-test/data/tpch_unique_sql_zstd_p0/sql/pipelinex_q03.out create mode 100644 regression-test/data/tpch_unique_sql_zstd_p0/sql/pipelinex_q04.out create mode 100644 regression-test/data/tpch_unique_sql_zstd_p0/sql/pipelinex_q05.out create mode 100644 regression-test/data/tpch_unique_sql_zstd_p0/sql/pipelinex_q06.out create mode 100644 regression-test/data/tpch_unique_sql_zstd_p0/sql/pipelinex_q07.out create mode 100644 regression-test/data/tpch_unique_sql_zstd_p0/sql/pipelinex_q08.out create mode 100644 regression-test/data/tpch_unique_sql_zstd_p0/sql/pipelinex_q09.out create mode 100644 regression-test/data/tpch_unique_sql_zstd_p0/sql/pipelinex_q10.out create mode 100644 regression-test/data/tpch_unique_sql_zstd_p0/sql/pipelinex_q11.out create mode 100644 regression-test/data/tpch_unique_sql_zstd_p0/sql/pipelinex_q12.out create mode 100644 regression-test/data/tpch_unique_sql_zstd_p0/sql/pipelinex_q13.out create mode 100644 regression-test/data/tpch_unique_sql_zstd_p0/sql/pipelinex_q14.out create mode 100644 regression-test/data/tpch_unique_sql_zstd_p0/sql/pipelinex_q15.out create mode 100644 regression-test/data/tpch_unique_sql_zstd_p0/sql/pipelinex_q16.out create mode 100644 regression-test/data/tpc
[GitHub] [doris] github-actions[bot] commented on pull request #23961: [pipeline](log) disable some unless error status warning log
github-actions[bot] commented on PR #23961: URL: https://github.com/apache/doris/pull/23961#issuecomment-1707823998 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23961: [pipeline](log) disable some unless error status warning log
github-actions[bot] commented on PR #23961: URL: https://github.com/apache/doris/pull/23961#issuecomment-1707823952 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #23933: [opt](Nereids) optimize error msg of unbound slot
doris-robot commented on PR #23933: URL: https://github.com/apache/doris/pull/23933#issuecomment-1707829240 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 47.12 seconds stream load tsv: 529 seconds loaded 74807831229 Bytes, about 134 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.1 seconds inserted 1000 Rows, about 343K ops/s storage size: 17162180723 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt merged pull request #23862: [Chore](function) refactor of quantile_state
BiteThet merged PR #23862: URL: https://github.com/apache/doris/pull/23862 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23961: [pipeline](log) disable some unless error status warning log
github-actions[bot] commented on PR #23961: URL: https://github.com/apache/doris/pull/23961#issuecomment-1707831120 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (7625d1514a -> a96adc01aa)
This is an automated email from the ASF dual-hosted git repository.
panxiaolei pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
from 7625d1514a [pipelineX](feature) support select operator and add TPCH
test cases (#23954)
add a96adc01aa [Chore](function) refactor of quantile_state (#23862)
No new revisions were added by this update.
Summary of changes:
be/src/olap/tablet_schema.cpp | 14 +--
be/src/util/quantile_state.cpp | 75 ++
be/src/util/quantile_state.h | 14 ++-
.../aggregate_function_quantile_state.cpp | 10 +-
.../aggregate_function_quantile_state.h| 46 -
be/src/vec/columns/column_complex.h| 16 +--
be/src/vec/core/field.h| 6 +-
be/src/vec/core/types.h| 9 +-
be/src/vec/data_types/data_type_decimal.h | 2 +-
be/src/vec/data_types/data_type_factory.cpp| 8 +-
be/src/vec/data_types/data_type_quantilestate.cpp | 54 --
be/src/vec/data_types/data_type_quantilestate.h| 18 ++--
.../serde/data_type_quantilestate_serde.cpp| 22
.../serde/data_type_quantilestate_serde.h | 115 -
be/src/vec/functions/function_quantile_state.cpp | 115 -
be/src/vec/olap/olap_data_convertor.cpp| 16 +--
be/test/util/quantile_state_test.cpp | 5 +-
be/test/vec/core/column_complex_test.cpp | 14 +--
.../data_types/serde/data_type_serde_pb_test.cpp | 8 +-
.../vec/data_types/serde/data_type_serde_test.cpp | 8 +-
.../sql-reference/Data-Types/AGG_STATE.md | 1 +
.../sql-reference/Data-Types/QUANTILE_STATE.md | 4 +-
.../sql-reference/Data-Types/AGG_STATE.md | 2 +
.../sql-reference/Data-Types/QUANTILE_STATE.md | 5 +-
.../functions/scalar/ToQuantileState.java | 4 +-
gensrc/script/doris_builtins_functions.py | 3 -
.../test_agg_state_quantile_union.out} | 11 +-
.../test_agg_state_quantile_union.groovy} | 40 +++
28 files changed, 238 insertions(+), 407 deletions(-)
delete mode 100644
be/src/vec/data_types/serde/data_type_quantilestate_serde.cpp
copy regression-test/data/{correctness_p0/test_outer_join_sort.out =>
datatype_p0/agg_state/quantile_union/test_agg_state_quantile_union.out} (61%)
copy
regression-test/suites/{mv_p0/test_mv_useless/where_invalid/where_invalid.groovy
=> datatype_p0/agg_state/quantile_union/test_agg_state_quantile_union.groovy}
(51%)
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23962: [improvement](jdbc) add profile for jdbc read and convert phase
github-actions[bot] commented on PR #23962: URL: https://github.com/apache/doris/pull/23962#issuecomment-1707831915 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly commented on pull request #23948: [fix](nereids) rf not generated after postprocessor
englefly commented on PR #23948: URL: https://github.com/apache/doris/pull/23948#issuecomment-1707838181 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Jibing-Li opened a new pull request, #23963: [Improvement](statistics)Support basic jdbc external table stats collection.
Jibing-Li opened a new pull request, #23963: URL: https://github.com/apache/doris/pull/23963 Support jdbc external table stats collection. ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly commented on pull request #23767: [feature](nereids)push runtime filter to right child of left outer join
englefly commented on PR #23767: URL: https://github.com/apache/doris/pull/23767#issuecomment-1707839606 A inner join (B left outer join C) runtimer filter A->C is valid, and can be pushed to C. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly merged pull request #23767: [feature](nereids)push runtime filter to right child of left outer join
englefly merged PR #23767: URL: https://github.com/apache/doris/pull/23767 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [feature](nereids)push runtime filter to right child of left outer join #23767
This is an automated email from the ASF dual-hosted git repository.
englefly pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new df1f4f843a [feature](nereids)push runtime filter to right child of
left outer join #23767
df1f4f843a is described below
commit df1f4f843a316784ca9aa6a2f53db7a14d39d0ab
Author: minghong
AuthorDate: Wed Sep 6 15:47:00 2023 +0800
[feature](nereids)push runtime filter to right child of left outer join
#23767
A inner join (B left outer join C)
runtimer filter A->C is valid, and can be pushed to C.
---
.../processor/post/RuntimeFilterGenerator.java | 30 --
.../trees/plans/physical/PhysicalHashJoin.java | 8 ++
2 files changed, 13 insertions(+), 25 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/processor/post/RuntimeFilterGenerator.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/processor/post/RuntimeFilterGenerator.java
index e8b37af3ae..5111e82309 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/processor/post/RuntimeFilterGenerator.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/processor/post/RuntimeFilterGenerator.java
@@ -74,7 +74,7 @@ import java.util.stream.Collectors;
*/
public class RuntimeFilterGenerator extends PlanPostProcessor {
-private static final ImmutableSet DENIED_JOIN_TYPES =
ImmutableSet.of(
+public static final ImmutableSet DENIED_JOIN_TYPES =
ImmutableSet.of(
JoinType.LEFT_ANTI_JOIN,
JoinType.FULL_OUTER_JOIN,
JoinType.LEFT_OUTER_JOIN,
@@ -110,33 +110,13 @@ public class RuntimeFilterGenerator extends
PlanPostProcessor {
public PhysicalPlan visitPhysicalHashJoin(PhysicalHashJoin join,
CascadesContext context) {
RuntimeFilterContext ctx = context.getRuntimeFilterContext();
-Map> aliasTransferMap =
ctx.getAliasTransferMap();
join.right().accept(this, context);
join.left().accept(this, context);
-if (DENIED_JOIN_TYPES.contains(join.getJoinType()) ||
join.isMarkJoin()) {
-// aliasTransMap is also used for judging whether the slot can be
as rf target.
-// for denied join type, the forbidden slots will be removed from
the map.
-// for example: a full outer join b on a.id = b.id, all slots will
be removed out.
-// for left outer join, only remove the right side slots and leave
the left side.
-// in later visit, the additional checking for the join type will
be invoked for different cases:
-// case 1: a left join b on a.id = b.id, checking whether rf on
b.id can be pushed to a, the answer is no,
-// since current join type is left outer join which is in
denied list;
-// case 2: (a left join b on a.id = b.id) inner join c on a.id2 =
c.id2, checking whether rf on c.id2 can
-// be pushed to a, the answer is yes, since the current
join is inner join which is permitted.
-if (join.getJoinType() == JoinType.LEFT_OUTER_JOIN) {
-Set slots = join.right().getOutputSet();
-slots.forEach(aliasTransferMap::remove);
-} else {
-Set slots = join.getOutputSet();
-slots.forEach(aliasTransferMap::remove);
-}
+collectPushDownCTEInfos(join, context);
+if (!getPushDownCTECandidates(ctx).isEmpty()) {
+pushDownRuntimeFilterIntoCTE(ctx);
} else {
-collectPushDownCTEInfos(join, context);
-if (!getPushDownCTECandidates(ctx).isEmpty()) {
-pushDownRuntimeFilterIntoCTE(ctx);
-} else {
-pushDownRuntimeFilterCommon(join, context);
-}
+pushDownRuntimeFilterCommon(join, context);
}
return join;
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalHashJoin.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalHashJoin.java
index 94e49a6889..eda81e5fef 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalHashJoin.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalHashJoin.java
@@ -204,6 +204,14 @@ public class PhysicalHashJoin<
public boolean pushDownRuntimeFilter(CascadesContext context,
IdGenerator generator,
AbstractPhysicalJoin builderNode,
Expression srcExpr, Expression probeExpr,
TRuntimeFilterType type, long
buildSideNdv, int exprOrder) {
+if (RuntimeFilterGenerator.DENIED_JOIN_TYPES.contains(getJoinType())
|| isMarkJoin()) {
+if (builderNode instanceof PhysicalHashJoin) {
+Phys
[GitHub] [doris] github-actions[bot] commented on pull request #23962: [improvement](jdbc) add profile for jdbc read and convert phase
github-actions[bot] commented on PR #23962: URL: https://github.com/apache/doris/pull/23962#issuecomment-1707847666 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly commented on a diff in pull request #23935: [opt](nereids)push down filter through window
englefly commented on code in PR #23935: URL: https://github.com/apache/doris/pull/23935#discussion_r1316884016 ## fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PushdownFilterThroughWindow.java: ## Review Comment: done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] XieJiann commented on pull request #23598: [feature](Nereids) add lambda argument and array_map function
XieJiann commented on PR #23598: URL: https://github.com/apache/doris/pull/23598#issuecomment-1707850642 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #21118: [Feature](x-load) support config min replica num for loading data
doris-robot commented on PR #21118: URL: https://github.com/apache/doris/pull/21118#issuecomment-1707850940 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 49.87 seconds stream load tsv: 532 seconds loaded 74807831229 Bytes, about 134 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 28.9 seconds inserted 1000 Rows, about 346K ops/s storage size: 17161970282 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xzj7019 commented on a diff in pull request #23935: [opt](nereids)push down filter through window
xzj7019 commented on code in PR #23935:
URL: https://github.com/apache/doris/pull/23935#discussion_r1316877390
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PushdownFilterThroughWindow.java:
##
@@ -81,88 +55,36 @@ public Rule build() {
return logicalFilter(logicalWindow()).thenApply(ctx -> {
LogicalFilter> filter = ctx.root;
LogicalWindow window = filter.child();
-
-// We have already done such optimization rule, so just ignore it.
-if (window.child(0) instanceof LogicalPartitionTopN) {
-return filter;
-}
-
-List windowExprs = window.getWindowExpressions();
-if (windowExprs.size() != 1) {
-return filter;
-}
-NamedExpression windowExpr = windowExprs.get(0);
-if (windowExpr.children().size() != 1 || !(windowExpr.child(0)
instanceof WindowExpression)) {
-return filter;
-}
-
-// Check the filter conditions. Now, we currently only support
simple conditions of the form
-// 'column conjuncts = filter.getConjuncts();
-Set relatedConjuncts =
extractRelatedConjuncts(conjuncts, windowExpr.getExprId());
-
-boolean hasPartitionLimit = false;
-long partitionLimit = Long.MAX_VALUE;
-
-for (Expression conjunct : relatedConjuncts) {
-Preconditions.checkArgument(conjunct instanceof
BinaryOperator);
-BinaryOperator op = (BinaryOperator) conjunct;
-Expression leftChild = op.children().get(0);
-Expression rightChild = op.children().get(1);
-
-Preconditions.checkArgument(leftChild instanceof SlotReference
-&& rightChild instanceof IntegerLikeLiteral);
-
-long limitVal = ((IntegerLikeLiteral)
rightChild).getLongValue();
-// Adjust the value for 'limitVal' based on the comparison
operators.
-if (conjunct instanceof LessThan) {
-limitVal--;
-}
-if (limitVal < 0) {
-return new
LogicalEmptyRelation(ctx.statementContext.getNextRelationId(),
filter.getOutput());
+Set commonPartitionKeys =
window.getCommonPartitionKeyFromWindowExpressions();
+Set bottomConjuncts = Sets.newHashSet();
+Set upperConjuncts = Sets.newHashSet();
+for (Expression expr : filter.getConjuncts()) {
+boolean pushed = false;
+for (Expression partitionKey : commonPartitionKeys) {
+if
(partitionKey.getInputSlots().containsAll(expr.getInputSlots())) {
Review Comment:
select * from (select c.*, rank() over(partition by c2+c3 order by c4) rn
from c) abc where c2=1;
Seems that c2=1 can't be pushed down. We can just support column currently.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly commented on a diff in pull request #23935: [opt](nereids)push down filter through window
englefly commented on code in PR #23935:
URL: https://github.com/apache/doris/pull/23935#discussion_r1316886879
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PushdownFilterThroughWindow.java:
##
@@ -81,88 +55,36 @@ public Rule build() {
return logicalFilter(logicalWindow()).thenApply(ctx -> {
LogicalFilter> filter = ctx.root;
LogicalWindow window = filter.child();
-
-// We have already done such optimization rule, so just ignore it.
-if (window.child(0) instanceof LogicalPartitionTopN) {
-return filter;
-}
-
-List windowExprs = window.getWindowExpressions();
-if (windowExprs.size() != 1) {
-return filter;
-}
-NamedExpression windowExpr = windowExprs.get(0);
-if (windowExpr.children().size() != 1 || !(windowExpr.child(0)
instanceof WindowExpression)) {
-return filter;
-}
-
-// Check the filter conditions. Now, we currently only support
simple conditions of the form
-// 'column conjuncts = filter.getConjuncts();
-Set relatedConjuncts =
extractRelatedConjuncts(conjuncts, windowExpr.getExprId());
-
-boolean hasPartitionLimit = false;
-long partitionLimit = Long.MAX_VALUE;
-
-for (Expression conjunct : relatedConjuncts) {
-Preconditions.checkArgument(conjunct instanceof
BinaryOperator);
-BinaryOperator op = (BinaryOperator) conjunct;
-Expression leftChild = op.children().get(0);
-Expression rightChild = op.children().get(1);
-
-Preconditions.checkArgument(leftChild instanceof SlotReference
-&& rightChild instanceof IntegerLikeLiteral);
-
-long limitVal = ((IntegerLikeLiteral)
rightChild).getLongValue();
-// Adjust the value for 'limitVal' based on the comparison
operators.
-if (conjunct instanceof LessThan) {
-limitVal--;
-}
-if (limitVal < 0) {
-return new
LogicalEmptyRelation(ctx.statementContext.getNextRelationId(),
filter.getOutput());
+Set commonPartitionKeys =
window.getCommonPartitionKeyFromWindowExpressions();
+Set bottomConjuncts = Sets.newHashSet();
+Set upperConjuncts = Sets.newHashSet();
+for (Expression expr : filter.getConjuncts()) {
+boolean pushed = false;
+for (Expression partitionKey : commonPartitionKeys) {
+if
(partitionKey.getInputSlots().containsAll(expr.getInputSlots())) {
Review Comment:
thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jacktengg commented on pull request #23959: [improvement](bitmap) support version for ser/deser of bitmap
jacktengg commented on PR #23959: URL: https://github.com/apache/doris/pull/23959#issuecomment-1707862966 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] TangSiyang2001 commented on pull request #23882: [enhancement](broker-load) support compress type for old broker load, and split compress type from file format
TangSiyang2001 commented on PR #23882: URL: https://github.com/apache/doris/pull/23882#issuecomment-1707866538 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] TangSiyang2001 commented on pull request #23882: [enhancement](broker-load) support compress type for old broker load, and split compress type from file format
TangSiyang2001 commented on PR #23882: URL: https://github.com/apache/doris/pull/23882#issuecomment-1707871595 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] XieJiann commented on pull request #23598: [feature](Nereids) add lambda argument and array_map function
XieJiann commented on PR #23598: URL: https://github.com/apache/doris/pull/23598#issuecomment-1707870049 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23871: [Optimize](invert index) Optimize multiple terms conjunction query
github-actions[bot] commented on PR #23871: URL: https://github.com/apache/doris/pull/23871#issuecomment-1707876493 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] TangSiyang2001 commented on pull request #23882: [enhancement](broker-load) support compress type for old broker load, and split compress type from file format
TangSiyang2001 commented on PR #23882: URL: https://github.com/apache/doris/pull/23882#issuecomment-1707877269 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23860: [fix](planner)fix 'char' function's toSql implementation is wrong
github-actions[bot] commented on PR #23860: URL: https://github.com/apache/doris/pull/23860#issuecomment-1707877487 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23860: [fix](planner)fix 'char' function's toSql implementation is wrong
github-actions[bot] commented on PR #23860: URL: https://github.com/apache/doris/pull/23860#issuecomment-1707877560 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xiaokang merged pull request #23860: [fix](planner)fix 'char' function's toSql implementation is wrong
xiaokang merged PR #23860: URL: https://github.com/apache/doris/pull/23860 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [fix](planner)fix 'char' function's toSql implementation is wrong (#23860)
This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new cb9acf4918 [fix](planner)fix 'char' function's toSql implementation is
wrong (#23860)
cb9acf4918 is described below
commit cb9acf4918717ef819b34423bde00dec84e964bf
Author: starocean999 <40539150+starocean...@users.noreply.github.com>
AuthorDate: Wed Sep 6 16:16:16 2023 +0800
[fix](planner)fix 'char' function's toSql implementation is wrong (#23860)
---
.../org/apache/doris/analysis/FunctionCallExpr.java | 20
regression-test/data/view_p0/view_p0.out | 3 +++
regression-test/suites/view_p0/view_p0.groovy| 9 -
3 files changed, 31 insertions(+), 1 deletion(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
index 480877e666..53e3458c3e 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
@@ -551,6 +551,26 @@ public class FunctionCallExpr extends Expr {
sb.append("DISTINCT ");
}
int len = children.size();
+
+if (fnName.getFunction().equalsIgnoreCase("char")) {
+for (int i = 1; i < len; ++i) {
+sb.append(children.get(i).toSql());
+if (i < len - 1) {
+sb.append(", ");
+}
+}
+sb.append(" using ");
+String encodeType = children.get(0).toSql();
+if (encodeType.charAt(0) == '\'') {
+encodeType = encodeType.substring(1, encodeType.length());
+}
+if (encodeType.charAt(encodeType.length() - 1) == '\'') {
+encodeType = encodeType.substring(0, encodeType.length() - 1);
+}
+sb.append(encodeType).append(")");
+return sb.toString();
+}
+
// XXX_diff are used by nereids only
if (fnName.getFunction().equalsIgnoreCase("years_diff") ||
fnName.getFunction().equalsIgnoreCase("months_diff")
|| fnName.getFunction().equalsIgnoreCase("days_diff")
diff --git a/regression-test/data/view_p0/view_p0.out
b/regression-test/data/view_p0/view_p0.out
index ff952fdaa7..976d4a3cb2 100644
--- a/regression-test/data/view_p0/view_p0.out
+++ b/regression-test/data/view_p0/view_p0.out
@@ -15,3 +15,6 @@
-- !sql --
960
+-- !sql2 --
+
+
diff --git a/regression-test/suites/view_p0/view_p0.groovy
b/regression-test/suites/view_p0/view_p0.groovy
index bb6a5ab5a6..cf55380a71 100644
--- a/regression-test/suites/view_p0/view_p0.groovy
+++ b/regression-test/suites/view_p0/view_p0.groovy
@@ -122,5 +122,12 @@ suite("view_p0") {
qt_sql "select * from test_time_diff"
sql "drop view if exists test_time_diff"
-
+
+sql "drop view if exists test_vv1;"
+
+sql "create view test_vv1 as select char(field2) from test_array_tbl_2;"
+
+qt_sql2 "select * from test_vv1;"
+
+sql "drop view if exists test_vv1;"
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] 08/08: [fix](planner)fix 'char' function's toSql implementation is wrong (#23860)
This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
commit b767749e500aaa0f3209fe1be0cbbce0c6abbe58
Author: starocean999 <40539150+starocean...@users.noreply.github.com>
AuthorDate: Wed Sep 6 16:16:16 2023 +0800
[fix](planner)fix 'char' function's toSql implementation is wrong (#23860)
---
.../org/apache/doris/analysis/FunctionCallExpr.java | 20
regression-test/data/view_p0/view_p0.out | 3 +++
regression-test/suites/view_p0/view_p0.groovy| 9 -
3 files changed, 31 insertions(+), 1 deletion(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
index 90f01e09cc..0ffcd5ab0a 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
@@ -551,6 +551,26 @@ public class FunctionCallExpr extends Expr {
sb.append("DISTINCT ");
}
int len = children.size();
+
+if (fnName.getFunction().equalsIgnoreCase("char")) {
+for (int i = 1; i < len; ++i) {
+sb.append(children.get(i).toSql());
+if (i < len - 1) {
+sb.append(", ");
+}
+}
+sb.append(" using ");
+String encodeType = children.get(0).toSql();
+if (encodeType.charAt(0) == '\'') {
+encodeType = encodeType.substring(1, encodeType.length());
+}
+if (encodeType.charAt(encodeType.length() - 1) == '\'') {
+encodeType = encodeType.substring(0, encodeType.length() - 1);
+}
+sb.append(encodeType).append(")");
+return sb.toString();
+}
+
// XXX_diff are used by nereids only
if (fnName.getFunction().equalsIgnoreCase("years_diff") ||
fnName.getFunction().equalsIgnoreCase("months_diff")
|| fnName.getFunction().equalsIgnoreCase("days_diff")
diff --git a/regression-test/data/view_p0/view_p0.out
b/regression-test/data/view_p0/view_p0.out
index ff952fdaa7..976d4a3cb2 100644
--- a/regression-test/data/view_p0/view_p0.out
+++ b/regression-test/data/view_p0/view_p0.out
@@ -15,3 +15,6 @@
-- !sql --
960
+-- !sql2 --
+
+
diff --git a/regression-test/suites/view_p0/view_p0.groovy
b/regression-test/suites/view_p0/view_p0.groovy
index bb6a5ab5a6..cf55380a71 100644
--- a/regression-test/suites/view_p0/view_p0.groovy
+++ b/regression-test/suites/view_p0/view_p0.groovy
@@ -122,5 +122,12 @@ suite("view_p0") {
qt_sql "select * from test_time_diff"
sql "drop view if exists test_time_diff"
-
+
+sql "drop view if exists test_vv1;"
+
+sql "create view test_vv1 as select char(field2) from test_array_tbl_2;"
+
+qt_sql2 "select * from test_vv1;"
+
+sql "drop view if exists test_vv1;"
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-2.0 updated (84365bdbd9 -> b767749e50)
This is an automated email from the ASF dual-hosted git repository. kxiao pushed a change to branch branch-2.0 in repository https://gitbox.apache.org/repos/asf/doris.git from 84365bdbd9 [be](pagecache) enable storage page cache by default new 00925def65 [opt](hudi) reduce the memory usage of avro reader (#23745) new 3f239bd8d6 [fe](default parameters) change remote_fragment_exec_timeout_ms from 5s to 30s (#23909) new 4a1cdfec1b [fix](statistics)Set default database while forwarding stmt to master. #23938 new 86d5fa4c02 [fix](segcompaction) disable segcompaction when calculating delete bitmap (#23927) new 129869fd5b [enhancement](merge-on-write) print the detailed error message on calculating delete bitmap failed (#23919) new aec036ea22 [FIX](array)fix array batch append data with right next_array_item_rowid (#23779) new ab228fd71b [fix](Nereids) fix or to in rule (#23940) new b767749e50 [fix](planner)fix 'char' function's toSql implementation is wrong (#23860) The 8 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: be/src/olap/calc_delete_bitmap_executor.cpp| 3 +- be/src/olap/rowset/segment_v2/column_writer.cpp| 10 +- be/src/olap/rowset/segment_v2/column_writer.h | 7 +- be/src/vec/olap/olap_data_convertor.cpp| 6 +- docs/en/docs/admin-manual/config/fe-config.md | 2 +- docs/zh-CN/docs/admin-manual/config/fe-config.md | 2 +- .../java/org/apache/doris/hudi/HudiJniScanner.java | 141 - .../org/apache/doris/hudi/BaseSplitReader.scala| 34 +++-- .../main/java/org/apache/doris/common/Config.java | 2 +- .../apache/doris/analysis/FunctionCallExpr.java| 20 +++ .../nereids/rules/expression/rules/OrToIn.java | 55 +++- .../java/org/apache/doris/qe/ConnectProcessor.java | 3 + .../java/org/apache/doris/qe/MasterOpExecutor.java | 1 + .../doris/nereids/rules/rewrite/OrToInTest.java| 27 +++- gensrc/thrift/FrontendService.thrift | 1 + .../es/test_external_catalog_es.out| 21 +-- regression-test/data/view_p0/view_p0.out | 3 + .../es/test_external_catalog_es.groovy | 11 +- regression-test/suites/view_p0/view_p0.groovy | 9 +- 19 files changed, 230 insertions(+), 128 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] 06/08: [FIX](array)fix array batch append data with right next_array_item_rowid (#23779)
This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
commit aec036ea222c6dc79a2e710d92910ccd38a11c4c
Author: amory
AuthorDate: Wed Sep 6 14:47:37 2023 +0800
[FIX](array)fix array batch append data with right next_array_item_rowid
(#23779)
---
be/src/olap/rowset/segment_v2/column_writer.cpp | 10 ++
be/src/olap/rowset/segment_v2/column_writer.h | 7 +++
be/src/vec/olap/olap_data_convertor.cpp | 6 ++
3 files changed, 7 insertions(+), 16 deletions(-)
diff --git a/be/src/olap/rowset/segment_v2/column_writer.cpp
b/be/src/olap/rowset/segment_v2/column_writer.cpp
index acf5c029d0..7a446f1123 100644
--- a/be/src/olap/rowset/segment_v2/column_writer.cpp
+++ b/be/src/olap/rowset/segment_v2/column_writer.cpp
@@ -221,7 +221,7 @@ Status ColumnWriter::create(const ColumnWriterOptions&
opts, const TabletColumn*
length_column.set_index_length(-1); // no short key index
std::unique_ptr
bigint_field(FieldFactory::create(length_column));
auto* length_writer =
-new ScalarColumnWriter(length_options,
std::move(bigint_field), file_writer);
+new OffsetColumnWriter(length_options,
std::move(bigint_field), file_writer);
// if nullable, create null writer
ScalarColumnWriter* null_writer = nullptr;
@@ -895,7 +895,7 @@ Status StructColumnWriter::finish_current_page() {
ArrayColumnWriter::ArrayColumnWriter(const ColumnWriterOptions& opts,
std::unique_ptr field,
- ScalarColumnWriter* offset_writer,
+ OffsetColumnWriter* offset_writer,
ScalarColumnWriter* null_writer,
std::unique_ptr item_writer)
: ColumnWriter(std::move(field), opts.meta->is_nullable()),
@@ -913,7 +913,6 @@ Status ArrayColumnWriter::init() {
RETURN_IF_ERROR(_null_writer->init());
}
RETURN_IF_ERROR(_item_writer->init());
-_offset_writer->register_flush_page_callback(this);
if (_opts.inverted_index) {
auto writer = dynamic_cast(_item_writer.get());
if (writer != nullptr) {
@@ -927,11 +926,6 @@ Status ArrayColumnWriter::init() {
return Status::OK();
}
-Status ArrayColumnWriter::put_extra_info_in_page(DataPageFooterPB* footer) {
-footer->set_next_array_item_ordinal(_item_writer->get_next_rowid());
-return Status::OK();
-}
-
Status ArrayColumnWriter::write_inverted_index() {
if (_opts.inverted_index) {
return _inverted_index_builder->finish();
diff --git a/be/src/olap/rowset/segment_v2/column_writer.h
b/be/src/olap/rowset/segment_v2/column_writer.h
index dfda60b53f..1bc0afb972 100644
--- a/be/src/olap/rowset/segment_v2/column_writer.h
+++ b/be/src/olap/rowset/segment_v2/column_writer.h
@@ -349,10 +349,10 @@ private:
ColumnWriterOptions _opts;
};
-class ArrayColumnWriter final : public ColumnWriter, public FlushPageCallback {
+class ArrayColumnWriter final : public ColumnWriter {
public:
explicit ArrayColumnWriter(const ColumnWriterOptions& opts,
std::unique_ptr field,
- ScalarColumnWriter* offset_writer,
ScalarColumnWriter* null_writer,
+ OffsetColumnWriter* offset_writer,
ScalarColumnWriter* null_writer,
std::unique_ptr item_writer);
~ArrayColumnWriter() override = default;
@@ -394,12 +394,11 @@ public:
ordinal_t get_next_rowid() const override { return
_offset_writer->get_next_rowid(); }
private:
-Status put_extra_info_in_page(DataPageFooterPB* header) override;
Status write_null_column(size_t num_rows, bool is_null); //
写入num_rows个null标记
bool has_empty_items() const { return _item_writer->get_next_rowid() == 0;
}
private:
-std::unique_ptr _offset_writer;
+std::unique_ptr _offset_writer;
std::unique_ptr _null_writer;
std::unique_ptr _item_writer;
std::unique_ptr _inverted_index_builder;
diff --git a/be/src/vec/olap/olap_data_convertor.cpp
b/be/src/vec/olap/olap_data_convertor.cpp
index d339accc7b..e7b59033c4 100644
--- a/be/src/vec/olap/olap_data_convertor.cpp
+++ b/be/src/vec/olap/olap_data_convertor.cpp
@@ -943,8 +943,7 @@ Status
OlapBlockDataConvertor::OlapColumnDataConvertorArray::convert_to_olap(
auto elem_size = end_offset - start_offset;
_offsets.clear();
-// we need all offsets, so reserve num_rows + 1 to make sure last offset
can be got in offset column,
-// instead of according to nested item column
+// we need all offsets, so reserve num_rows + 1 to make sure last offset
can be got in offset column, instead of according to nested item column
_offsets.reserve(_num_rows + 1);
for (
[doris] 07/08: [fix](Nereids) fix or to in rule (#23940)
This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
commit ab228fd71b5cfac9e1096dadf57b0dd6cfb6e26f
Author: 谢健
AuthorDate: Wed Sep 6 14:58:20 2023 +0800
[fix](Nereids) fix or to in rule (#23940)
or expression context can't propagation cross or expression.
for example:
```
select (a = 1 or a = 2 or a = 3) + (a = 4 or a = 5 or a = 6)
= select a in [1, 2, 3] + a in [4,5,6]
!= select a in [1, 2, 3] + a in [1, 2, 3, 4, 5, 6]
```
---
.../nereids/rules/expression/rules/OrToIn.java | 55 +-
.../doris/nereids/rules/rewrite/OrToInTest.java| 27 +--
2 files changed, 44 insertions(+), 38 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/OrToIn.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/OrToIn.java
index a54d5f5369..aaa077d199 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/OrToIn.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/OrToIn.java
@@ -19,13 +19,11 @@ package org.apache.doris.nereids.rules.expression.rules;
import org.apache.doris.nereids.rules.expression.ExpressionRewriteContext;
import org.apache.doris.nereids.rules.expression.ExpressionRewriteRule;
-import org.apache.doris.nereids.rules.expression.rules.OrToIn.OrToInContext;
-import org.apache.doris.nereids.trees.expressions.And;
-import org.apache.doris.nereids.trees.expressions.CompoundPredicate;
import org.apache.doris.nereids.trees.expressions.EqualTo;
import org.apache.doris.nereids.trees.expressions.Expression;
import org.apache.doris.nereids.trees.expressions.InPredicate;
import org.apache.doris.nereids.trees.expressions.NamedExpression;
+import org.apache.doris.nereids.trees.expressions.Or;
import org.apache.doris.nereids.trees.expressions.literal.Literal;
import
org.apache.doris.nereids.trees.expressions.visitor.DefaultExpressionRewriter;
import org.apache.doris.nereids.util.ExpressionUtils;
@@ -57,7 +55,7 @@ import java.util.Set;
* adding any additional rule-specific fields to the default
ExpressionRewriteContext. However, the entire expression
* rewrite framework always passes an ExpressionRewriteContext of type context
to all rules.
*/
-public class OrToIn extends DefaultExpressionRewriter implements
+public class OrToIn extends
DefaultExpressionRewriter implements
ExpressionRewriteRule {
public static final OrToIn INSTANCE = new OrToIn();
@@ -66,25 +64,20 @@ public class OrToIn extends
DefaultExpressionRewriter implements
@Override
public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
-return expr.accept(this, new OrToInContext());
+return expr.accept(this, null);
}
@Override
-public Expression visitCompoundPredicate(CompoundPredicate
compoundPredicate, OrToInContext context) {
-if (compoundPredicate instanceof And) {
-return
compoundPredicate.withChildren(compoundPredicate.child(0).accept(new OrToIn(),
-new OrToInContext()),
-compoundPredicate.child(1).accept(new OrToIn(),
-new OrToInContext()));
-}
-List expressions =
ExpressionUtils.extractDisjunction(compoundPredicate);
+public Expression visitOr(Or or, ExpressionRewriteContext ctx) {
+Map> slotNameToLiteral = new HashMap<>();
+List expressions = ExpressionUtils.extractDisjunction(or);
for (Expression expression : expressions) {
if (expression instanceof EqualTo) {
-addSlotToLiteralMap((EqualTo) expression, context);
+addSlotToLiteralMap((EqualTo) expression, slotNameToLiteral);
}
}
List rewrittenOr = new ArrayList<>();
-for (Map.Entry> entry :
context.slotNameToLiteral.entrySet()) {
+for (Map.Entry> entry :
slotNameToLiteral.entrySet()) {
Set literals = entry.getValue();
if (literals.size() >= REWRITE_OR_TO_IN_PREDICATE_THRESHOLD) {
InPredicate inPredicate = new InPredicate(entry.getKey(),
ImmutableList.copyOf(entry.getValue()));
@@ -92,26 +85,26 @@ public class OrToIn extends
DefaultExpressionRewriter implements
}
}
for (Expression expression : expressions) {
-if (!ableToConvertToIn(expression, context)) {
-rewrittenOr.add(expression);
+if (!ableToConvertToIn(expression, slotNameToLiteral)) {
+rewrittenOr.add(expression.accept(this, null));
}
}
return ExpressionUtils.or(rewrittenOr);
}
-private void addSlotToLiteralMap(EqualTo equal, OrToInContext context) {
+private void addSlotToLiteralMap(EqualTo equal, Map> slotNa
[doris] 05/08: [enhancement](merge-on-write) print the detailed error message on calculating delete bitmap failed (#23919)
This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 129869fd5b1abff602ac55f002e02b77781135f4
Author: zhannngchen <48427519+zhannngc...@users.noreply.github.com>
AuthorDate: Wed Sep 6 14:28:17 2023 +0800
[enhancement](merge-on-write) print the detailed error message on
calculating delete bitmap failed (#23919)
---
be/src/olap/calc_delete_bitmap_executor.cpp | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/be/src/olap/calc_delete_bitmap_executor.cpp
b/be/src/olap/calc_delete_bitmap_executor.cpp
index 284c03c985..51e77a3628 100644
--- a/be/src/olap/calc_delete_bitmap_executor.cpp
+++ b/be/src/olap/calc_delete_bitmap_executor.cpp
@@ -50,7 +50,8 @@ Status CalcDeleteBitmapToken::submit(TabletSharedPtr tablet,
RowsetSharedPtr cur
if (!st.ok()) {
LOG(WARNING) << "failed to calc segment delete bitmap, tablet_id: "
<< tablet->tablet_id() << " rowset: " <<
cur_rowset->rowset_id()
- << " seg_id: " << cur_segment->id() << " version: "
<< end_version;
+ << " seg_id: " << cur_segment->id() << " version: "
<< end_version
+ << " error: " << st;
std::lock_guard wlock(_lock);
if (_status.ok()) {
_status = st;
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] 01/08: [opt](hudi) reduce the memory usage of avro reader (#23745)
This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 00925def650c37302682995ef1721f03f2dc1718
Author: Ashin Gau
AuthorDate: Tue Sep 5 23:59:23 2023 +0800
[opt](hudi) reduce the memory usage of avro reader (#23745)
1. Reduce the number of threads reading avro logs and keep the readers in a
fixed thread pool.
2. Regularly cleaning the cached resolvers in the thread local map by
reflection.
---
.../java/org/apache/doris/hudi/HudiJniScanner.java | 141 -
.../org/apache/doris/hudi/BaseSplitReader.scala| 34 +++--
2 files changed, 125 insertions(+), 50 deletions(-)
diff --git
a/fe/be-java-extensions/hudi-scanner/src/main/java/org/apache/doris/hudi/HudiJniScanner.java
b/fe/be-java-extensions/hudi-scanner/src/main/java/org/apache/doris/hudi/HudiJniScanner.java
index 539ab8f7a8..417b338115 100644
---
a/fe/be-java-extensions/hudi-scanner/src/main/java/org/apache/doris/hudi/HudiJniScanner.java
+++
b/fe/be-java-extensions/hudi-scanner/src/main/java/org/apache/doris/hudi/HudiJniScanner.java
@@ -22,6 +22,9 @@ import org.apache.doris.common.jni.JniScanner;
import org.apache.doris.common.jni.vec.ColumnType;
import org.apache.doris.common.jni.vec.ScanPredicate;
+import com.google.common.util.concurrent.ThreadFactoryBuilder;
+import org.apache.avro.generic.GenericDatumReader;
+import org.apache.avro.util.WeakIdentityHashMap;
import org.apache.hadoop.security.UserGroupInformation;
import org.apache.log4j.Logger;
import org.apache.spark.sql.catalyst.InternalRow;
@@ -30,14 +33,21 @@ import scala.collection.Iterator;
import java.io.Closeable;
import java.io.IOException;
+import java.lang.reflect.Field;
import java.security.PrivilegedExceptionAction;
import java.util.Collections;
import java.util.List;
import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
+import java.util.concurrent.Future;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.concurrent.locks.ReadWriteLock;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
import java.util.stream.Collectors;
/**
@@ -56,6 +66,56 @@ public class HudiJniScanner extends JniScanner {
private long getRecordReaderTimeNs = 0;
private Iterator recordIterator;
+/**
+ * `GenericDatumReader` of avro is a thread local map, that stores
`WeakIdentityHashMap`.
+ * `WeakIdentityHashMap` has cached the avro resolving decoder, and the
cached resolver can only be cleaned when
+ * its avro schema is recycled and become a week reference. However, the
behavior of the week reference queue
+ * of `WeakIdentityHashMap` is unpredictable. Secondly, the decoder is
very memory intensive, the number of threads
+ * to call the thread local map cannot be too many.
+ * Two solutions:
+ * 1. Reduce the number of threads reading avro logs and keep the readers
in a fixed thread pool.
+ * 2. Regularly cleaning the cached resolvers in the thread local map by
reflection.
+ */
+private static final AtomicLong lastUpdateTime = new
AtomicLong(System.currentTimeMillis());
+private static final long RESOLVER_TIME_OUT = 6;
+private static final ExecutorService avroReadPool;
+private static ThreadLocal> AVRO_RESOLVER_CACHE;
+private static final Map> cachedResolvers
= new ConcurrentHashMap<>();
+private static final ReadWriteLock cleanResolverLock = new
ReentrantReadWriteLock();
+private static final ScheduledExecutorService cleanResolverService =
Executors.newScheduledThreadPool(1);
+
+static {
+int numThreads = Math.max(Runtime.getRuntime().availableProcessors() *
2 + 1, 4);
+avroReadPool = Executors.newFixedThreadPool(numThreads,
+new
ThreadFactoryBuilder().setNameFormat("avro-log-reader-%d").build());
+LOG.info("Create " + numThreads + " daemon threads to load avro logs");
+
+Class avroReader = GenericDatumReader.class;
+try {
+Field field = avroReader.getDeclaredField("RESOLVER_CACHE");
+field.setAccessible(true);
+AVRO_RESOLVER_CACHE = (ThreadLocal>)
field.get(null);
+LOG.info("Get the resolved cache for avro reader");
+} catch (Exception e) {
+AVRO_RESOLVER_CACHE = null;
+LOG.warn("Failed to get the resolved cache for avro reader");
+}
+
+cleanResolverService.scheduleAtFixedRate(() -> {
+cleanResolverLock.writeLock().lock();
+try {
+if (System.currentTimeMillis() - lastUpdateTime.get() >
RESOLVER_TIME_OUT) {
+for (WeakIdentityHashMap solver :
cached
[doris] 04/08: [fix](segcompaction) disable segcompaction when calculating delete bitmap (#23927)
This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 86d5fa4c02cc0e16aa29df33624f386299e98993
Author: zhangguoqiang <18372634...@163.com>
AuthorDate: Wed Sep 6 12:12:43 2023 +0800
[fix](segcompaction) disable segcompaction when calculating delete bitmap
(#23927)
Calc delete bitmap may generate segments, as a result, may trigger
segcompaction. But the BetaRowsetWriter is a transient one and in
Publishing context, which is bug-prone and hard to rollback. So let us
disable the triggering in delete bitmap calc code path.
Signed-off-by: freemandealer
---
.../es/test_external_catalog_es.out | 21 +++--
.../es/test_external_catalog_es.groovy | 11 ++-
2 files changed, 13 insertions(+), 19 deletions(-)
diff --git
a/regression-test/data/external_table_p2/es/test_external_catalog_es.out
b/regression-test/data/external_table_p2/es/test_external_catalog_es.out
index 3f61e7c809..d2104f34fc 100644
--- a/regression-test/data/external_table_p2/es/test_external_catalog_es.out
+++ b/regression-test/data/external_table_p2/es/test_external_catalog_es.out
@@ -1,12 +1,13 @@
-- This file is automatically generated. You should know what you did if you
want to edit this
-- !sql --
-\N \N \N \N \N \N \N \N \N \N
{"properties":{"account_number":{"type":"long"},"firstname":{"type":"text"},"address":{"type":"text"},"balance":{"type":"long"},"gender":{"type":"text"},"city":{"type":"text"},"employer":{"type":"text"},"state":{"type":"text"},"age":{"type":"text"},"email":{"type":"text"},"lastname":{"type":"text"}}}
{"index":{"number_of_shards":5,"number_of_replicas":1}} \N
-0 244 Columbus Place 29 16623 Hobuckenbradshawmcken...@euron.com
Euron BradshawF Mckenzie\N \N CO
-1 880 Holmes Lane 32 39225 Brogan amberd...@pyrami.comPyrami Amber
M Duke\N \N IL
-2 560 Kingsway Place 22 28838 Bennett robertaben...@chillium.com
ChilliumRoberta F Bender \N \N LA
-3 328 Wilson Avenue 26 44947 Cochranvillelevinebu...@amtap.com
Amtap Levine F Burks \N \N HI
-4 986 Wyckoff Avenue 31 27658 Eastvalerodriquezflo...@tourmania.com
Tourmania Rodriquez F Flores \N \N HI
-5 311 Elm Place 30 29342 Fairviewleolastew...@diginetic.com
Diginetic Leola F Stewart \N \N NJ
-6 671 Bristol Street 36 5686Dante hattieb...@netagy.com Netagy
Hattie M Bond\N \N TN
-7 820 Logan Street22 39121 Shrewsbury levyrich...@teraprene.com
Teraprene LevyM Richard \N \N MO
-8 699 Visitation Place35 48868 Wakulla janbu...@glasstep.com
GlasstepJan M Burns \N \N AZ
+\N \N \N \N \N \N \N \N \N \N
{"properties":{"account_number":{"type":"long"},"firstname":{"type":"text"},"address":{"type":"text"},"balance":{"type":"long"},"gender":{"type":"text"},"city":{"type":"text"},"employer":{"type":"text"},"state":{"type":"text"},"age":{"type":"text"},"email":{"type":"text"},"lastname":{"type":"text"}}}
{"index":{"number_of_shards":5,"number_of_replicas":1}} \N
+0 244 Columbus Place 29 16623 Hobucken
bradshawmcken...@euron.com Euron BradshawF Mckenzie
\N \N CO
+1 880 Holmes Lane 32 39225 Brogan amberd...@pyrami.comPyrami
Amber M Duke\N \N IL
+2 560 Kingsway Place 22 28838 Bennett
robertaben...@chillium.com ChilliumRoberta F Bender \N
\N LA
+3 328 Wilson Avenue 26 44947 Cochranville
levinebu...@amtap.com Amtap Levine F Burks \N \N HI
+4 986 Wyckoff Avenue 31 27658 Eastvale
rodriquezflo...@tourmania.com Tourmania Rodriquez F Flores
\N \N HI
+5 311 Elm Place 30 29342 Fairview
leolastew...@diginetic.com Diginetic Leola F Stewart \N
\N NJ
+6 671 Bristol Street 36 5686Dante hattieb...@netagy.com
Netagy Hattie M Bond\N \N TN
+7 820 Logan Street22 39121 Shrewsbury
levyrich...@teraprene.com Teraprene LevyM Richard \N
\N MO
+8 699 Visitation Place35 48868 Wakulla janbu...@glasstep.com
GlasstepJan M Burns \N \N AZ
+
diff --git
a/regression-test/suites/external_table_p2/es/test_external_catalog_es.groovy
b/regression-test/suites/external_table_p2/es/test_external_catalog_es.groovy
index 3f8581930e..6e1cf4e0d8 100644
---
a/regression-test/suites/external_table_p2/es/test_external_catalog_es.groovy
+++
b/regression-test/suites/external_table_p2/es/test_external_catalog_es.groovy
@@
[GitHub] [doris] morningman commented on pull request #22195: [Feature](Nereids) support create table and CTAS
morningman commented on PR #22195: URL: https://github.com/apache/doris/pull/22195#issuecomment-1707884432 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] 02/08: [fe](default parameters) change remote_fragment_exec_timeout_ms from 5s to 30s (#23909)
This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 3f239bd8d653dd9fb006bdae9419488efaf7adc1
Author: yiguolei <676222...@qq.com>
AuthorDate: Wed Sep 6 00:16:23 2023 +0800
[fe](default parameters) change remote_fragment_exec_timeout_ms from 5s to
30s (#23909)
---
docs/en/docs/admin-manual/config/fe-config.md | 2 +-
docs/zh-CN/docs/admin-manual/config/fe-config.md | 2 +-
fe/fe-common/src/main/java/org/apache/doris/common/Config.java | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/docs/en/docs/admin-manual/config/fe-config.md
b/docs/en/docs/admin-manual/config/fe-config.md
index fc16291720..7f11a4a99c 100644
--- a/docs/en/docs/admin-manual/config/fe-config.md
+++ b/docs/en/docs/admin-manual/config/fe-config.md
@@ -613,7 +613,7 @@ max num of thread to handle agent task in agent task
thread-pool.
`remote_fragment_exec_timeout_ms`
-Default：5000 （ms）
+Default：3 （ms）
IsMutable：true
diff --git a/docs/zh-CN/docs/admin-manual/config/fe-config.md
b/docs/zh-CN/docs/admin-manual/config/fe-config.md
index 4f45de2076..fc71a939a4 100644
--- a/docs/zh-CN/docs/admin-manual/config/fe-config.md
+++ b/docs/zh-CN/docs/admin-manual/config/fe-config.md
@@ -613,7 +613,7 @@ FE向BE的BackendService发送rpc请求时的超时时间，单位：毫秒。
`remote_fragment_exec_timeout_ms`
-默认值：5000 （ms）
+默认值：3 （ms）
是否可以动态配置：true
diff --git a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
index 3ca13c6437..705c93b8c3 100644
--- a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
+++ b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
@@ -1611,7 +1611,7 @@ public class Config extends ConfigBase {
* condition，try to set this timeout longer.
*/
@ConfField(mutable = true)
-public static long remote_fragment_exec_timeout_ms = 5000; // 5 sec
+public static long remote_fragment_exec_timeout_ms = 3; // 30 sec
/**
* Max data version of backends serialize block.
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] 03/08: [fix](statistics)Set default database while forwarding stmt to master. #23938
This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 4a1cdfec1bacfa174cbc319c5e5f19236ed7d702
Author: Jibing-Li <64681310+jibing...@users.noreply.github.com>
AuthorDate: Wed Sep 6 08:53:30 2023 +0800
[fix](statistics)Set default database while forwarding stmt to master.
#23938
Set default database while forwarding request to master. Avoid No database
selected error.
---
fe/fe-core/src/main/java/org/apache/doris/qe/ConnectProcessor.java | 3 +++
fe/fe-core/src/main/java/org/apache/doris/qe/MasterOpExecutor.java | 1 +
gensrc/thrift/FrontendService.thrift | 1 +
3 files changed, 5 insertions(+)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/ConnectProcessor.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/ConnectProcessor.java
index 2bdfba3548..73c6debb69 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/ConnectProcessor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/ConnectProcessor.java
@@ -748,6 +748,9 @@ public class ConnectProcessor {
if (request.isSetDefaultCatalog()) {
ctx.getEnv().changeCatalog(ctx, request.getDefaultCatalog());
}
+if (request.isSetDefaultDatabase() &&
!request.getDefaultDatabase().isEmpty()) {
+ctx.getEnv().changeDb(ctx, request.getDefaultDatabase());
+}
TUniqueId queryId; // This query id will be set in ctx
if (request.isSetQueryId()) {
queryId = request.getQueryId();
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/MasterOpExecutor.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/MasterOpExecutor.java
index 05fc4e2d21..5c15007ffa 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/MasterOpExecutor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/MasterOpExecutor.java
@@ -168,6 +168,7 @@ public class MasterOpExecutor {
params.setStmtIdx(originStmt.idx);
params.setUser(ctx.getQualifiedUser());
params.setDefaultCatalog(ctx.getDefaultCatalog());
+params.setDefaultDatabase(ctx.getDatabase());
params.setDb(ctx.getDatabase());
params.setUserIp(ctx.getRemoteIP());
params.setStmtId(ctx.getStmtId());
diff --git a/gensrc/thrift/FrontendService.thrift
b/gensrc/thrift/FrontendService.thrift
index 6a005450cc..14bf4ac1ae 100644
--- a/gensrc/thrift/FrontendService.thrift
+++ b/gensrc/thrift/FrontendService.thrift
@@ -470,6 +470,7 @@ struct TMasterOpRequest {
23: optional i32 clientNodePort
24: optional bool syncJournalOnly // if set to true, this request means to
do nothing but just sync max journal id of master
25: optional string defaultCatalog
+26: optional string defaultDatabase
}
struct TColumnDefinition {
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen opened a new pull request, #23964: [fix](case) enable case test_bitmap_index
hello-stephen opened a new pull request, #23964: URL: https://github.com/apache/doris/pull/23964 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #23964: [fix](case) enable case test_bitmap_index
hello-stephen commented on PR #23964: URL: https://github.com/apache/doris/pull/23964#issuecomment-1707889097 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23964: [fix](case) enable case test_bitmap_index
github-actions[bot] commented on PR #23964: URL: https://github.com/apache/doris/pull/23964#issuecomment-1707892483 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman commented on pull request #23962: [improvement](jdbc) add profile for jdbc read and convert phase
morningman commented on PR #23962: URL: https://github.com/apache/doris/pull/23962#issuecomment-1707893636 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HappenLee closed issue #18547: [Feature] 根据分区列自动创建分区
HappenLee closed issue #18547: [Feature] 根据分区列自动创建分区 URL: https://github.com/apache/doris/issues/18547 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HappenLee merged pull request #23236: [Feature](partitions) Support auto partition
HappenLee merged PR #23236: URL: https://github.com/apache/doris/pull/23236 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Jibing-Li opened a new pull request, #23965: [Improvement](statistics)Support basic jdbc external table stats collection.
Jibing-Li opened a new pull request, #23965: URL: https://github.com/apache/doris/pull/23965 Support basic jdbc external table stats collection. ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhannngchen merged pull request #23855: [Fix](BinaryPrefixPage) stop to read values when current pos reached the end of the page in `BinaryPrefixPageDecoder::next_batch`
zhannngchen merged PR #23855: URL: https://github.com/apache/doris/pull/23855 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Fix](BinaryPrefixPage) stop to read values when current pos reached the end of the page in `BinaryPrefixPageDecoder::next_batch` (#23855)
This is an automated email from the ASF dual-hosted git repository.
zhangchen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 95ae5376f3 [Fix](BinaryPrefixPage) stop to read values when current
pos reached the end of the page in `BinaryPrefixPageDecoder::next_batch`
(#23855)
95ae5376f3 is described below
commit 95ae5376f306177d2f92ad109548e7a1cfd6ae14
Author: bobhan1
AuthorDate: Wed Sep 6 16:34:38 2023 +0800
[Fix](BinaryPrefixPage) stop to read values when current pos reached the
end of the page in `BinaryPrefixPageDecoder::next_batch` (#23855)
---
be/src/olap/rowset/segment_v2/binary_prefix_page.cpp | 9 +++--
be/test/olap/primary_key_index_test.cpp | 2 +-
2 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/be/src/olap/rowset/segment_v2/binary_prefix_page.cpp
b/be/src/olap/rowset/segment_v2/binary_prefix_page.cpp
index e3e893a0dd..ab9056def1 100644
--- a/be/src/olap/rowset/segment_v2/binary_prefix_page.cpp
+++ b/be/src/olap/rowset/segment_v2/binary_prefix_page.cpp
@@ -23,6 +23,7 @@
#include
#include
+#include "common/status.h"
#include "gutil/port.h"
#include "gutil/strings/substitute.h"
#include "util/coding.h"
@@ -114,12 +115,13 @@ const uint8_t*
BinaryPrefixPageDecoder::_decode_value_lengths(const uint8_t* ptr
Status BinaryPrefixPageDecoder::_read_next_value() {
if (_cur_pos >= _num_values) {
-return Status::NotFound("no more value to read");
+return Status::EndOfFile("no more value to read");
}
uint32_t shared_len;
uint32_t non_shared_len;
auto data_ptr = _decode_value_lengths(_next_ptr, &shared_len,
&non_shared_len);
if (data_ptr == nullptr) {
+DCHECK(false) << "[BinaryPrefixPageDecoder::_read_next_value]
corruption!";
return Status::Corruption("Failed to decode value at position {}",
_cur_pos);
}
_current_value.resize(shared_len);
@@ -215,8 +217,11 @@ Status BinaryPrefixPageDecoder::next_batch(size_t* n,
vectorized::MutableColumnP
// read and copy values
for (size_t i = 0; i < max_fetch; ++i) {
dst->insert_data((char*)(_current_value.data()),
_current_value.size());
-_read_next_value();
_cur_pos++;
+// reach the end of the page, should not read the next value
+if (_cur_pos < _num_values) {
+RETURN_IF_ERROR(_read_next_value());
+}
}
*n = max_fetch;
diff --git a/be/test/olap/primary_key_index_test.cpp
b/be/test/olap/primary_key_index_test.cpp
index 837d258700..46b87b712d 100644
--- a/be/test/olap/primary_key_index_test.cpp
+++ b/be/test/olap/primary_key_index_test.cpp
@@ -129,7 +129,7 @@ TEST_F(PrimaryKeyIndexTest, builder) {
EXPECT_FALSE(exists);
auto status = index_iterator->seek_at_or_after(&slice, &exact_match);
EXPECT_FALSE(exact_match);
-EXPECT_TRUE(status.is());
+EXPECT_TRUE(status.is());
}
// read all key
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xzj7019 commented on a diff in pull request #23948: [fix](nereids) rf not generated after postprocessor
xzj7019 commented on code in PR #23948: URL: https://github.com/apache/doris/pull/23948#discussion_r1316935567 ## regression-test/pipeline/p0/conf/regression-conf.groovy: ## @@ -55,10 +55,10 @@ testDirectories = "" excludeGroups = "" // this suites will not be executed -excludeSuites = "test_bitmap_index,test_ddl,test_analyze,test_leading,test_stream_load_move_memtable,ds_rf95,test_profile,test_broker_load,test_spark_load,test_refresh_mtmv,test_bitmap_filter,test_export_parquet,test_doris_jdbc_catalog,nereids_delete_mow_partial_update" +excludeSuites = "test_bitmap_index,test_ddl,test_analyze,test_leading,test_stream_load_move_memtable,ds_rf95ds_rf16,ds_rf94,test_profile,test_broker_load,test_spark_load,test_refresh_mtmv,test_bitmap_filter,test_export_parquet,test_doris_jdbc_catalog,nereids_delete_mow_partial_update" Review Comment: 1）typo? ds_rf95ds_rf16 2) why exclude 95/94/16 ? ## regression-test/pipeline/p0/conf/regression-conf.groovy: ## @@ -55,10 +55,10 @@ testDirectories = "" excludeGroups = "" // this suites will not be executed -excludeSuites = "test_bitmap_index,test_ddl,test_analyze,test_leading,test_stream_load_move_memtable,ds_rf95,test_profile,test_broker_load,test_spark_load,test_refresh_mtmv,test_bitmap_filter,test_export_parquet,test_doris_jdbc_catalog,nereids_delete_mow_partial_update" +excludeSuites = "test_bitmap_index,test_ddl,test_analyze,test_leading,test_stream_load_move_memtable,ds_rf95ds_rf16,ds_rf94,test_profile,test_broker_load,test_spark_load,test_refresh_mtmv,test_bitmap_filter,test_export_parquet,test_doris_jdbc_catalog,nereids_delete_mow_partial_update" Review Comment: 1）typo? ds_rf95ds_rf16 2) why exclude 95/94/16 ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] SWJTU-ZhangLei commented on pull request #20243: [feature](regression) Add a p2 level case for schema change
SWJTU-ZhangLei commented on PR #20243: URL: https://github.com/apache/doris/pull/20243#issuecomment-1707913847 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] amorynan opened a new pull request, #23966: [FIX](func)fix size function for array map
amorynan opened a new pull request, #23966: URL: https://github.com/apache/doris/pull/23966 ## Proposed changes Issue Number: close #xxx link: https://github.com/apache/doris/pull/23920/files ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] vinlee19 opened a new pull request, #23967: [typo[(docs) modify flink connector document error
vinlee19 opened a new pull request, #23967: URL: https://github.com/apache/doris/pull/23967 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] vinlee19 commented on pull request #23967: [typo[(docs) modify flink connector document error
vinlee19 commented on PR #23967: URL: https://github.com/apache/doris/pull/23967#issuecomment-1707922971 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Jibing-Li commented on pull request #23963: [Improvement](statistics)Support basic jdbc external table stats collection.
Jibing-Li commented on PR #23963: URL: https://github.com/apache/doris/pull/23963#issuecomment-1707923561 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BePPPower commented on pull request #23635: [fix](Export) Concatenation the outfile sql for Export
BePPPower commented on PR #23635: URL: https://github.com/apache/doris/pull/23635#issuecomment-1707926919 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] vinlee19 commented on issue #23955: how to stop sync job
vinlee19 commented on issue #23955: URL: https://github.com/apache/doris/issues/23955#issuecomment-1707929002 https://doris.apache.org/zh-CN/docs/1.2/sql-manual/sql-reference/Data-Manipulation-Statements/Load/PAUSE-SYNC-JOB?_highlight=jobs PAUSE SYNC JOB `job_name`; -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] LemonLiTree opened a new pull request, #23968: (imporve)Improve error message
LemonLiTree opened a new pull request, #23968: URL: https://github.com/apache/doris/pull/23968 before: https://github.com/apache/doris/assets/109049295/fdc1cda1-7f1e-4477-b374-fa1d0d6db7f6";> now: https://github.com/apache/doris/assets/109049295/59feb51a-2ac2-49ce-8df5-3ed50416826c";> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zzzzzzzs opened a new pull request, #23969: [fix] fix http_stream retry mechanism
zzzs opened a new pull request, #23969: URL: https://github.com/apache/doris/pull/23969 ## Proposed changes Issue Number: close #xxx ## Further comments 1. For the original global variable need_schema, the Doris retry mechanism cannot be guaranteed, so the method of using local variables is used 2. Removed useless code -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] doris-robot commented on pull request #23961: [pipeline](log) disable some unless error status warning log
doris-robot commented on PR #23961: URL: https://github.com/apache/doris/pull/23961#issuecomment-1707934414 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 49.62 seconds stream load tsv: 538 seconds loaded 74807831229 Bytes, about 132 MB/s stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 28.8 seconds inserted 1000 Rows, about 347K ops/s storage size: 17161956411 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23966: [FIX](func)fix size function for array map
github-actions[bot] commented on PR #23966: URL: https://github.com/apache/doris/pull/23966#issuecomment-1707934946 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zzzzzzzs commented on pull request #23969: [fix] fix http_stream retry mechanism
zzzs commented on PR #23969: URL: https://github.com/apache/doris/pull/23969#issuecomment-1707934762 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23968: (imporve)Improve error message
github-actions[bot] commented on PR #23968: URL: https://github.com/apache/doris/pull/23968#issuecomment-1707938001 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23968: (imporve)Improve error message
github-actions[bot] commented on PR #23968: URL: https://github.com/apache/doris/pull/23968#issuecomment-1707938066 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23969: [fix] fix http_stream retry mechanism
github-actions[bot] commented on PR #23969: URL: https://github.com/apache/doris/pull/23969#issuecomment-1707942265 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] XieJiann commented on pull request #23598: [feature](Nereids) add lambda argument and array_map function
XieJiann commented on PR #23598: URL: https://github.com/apache/doris/pull/23598#issuecomment-1707943312 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei closed pull request #23957: [bugfix](datastreamsender) receiver will hang during close
yiguolei closed pull request #23957: [bugfix](datastreamsender) receiver will hang during close URL: https://github.com/apache/doris/pull/23957 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei opened a new pull request, #23970: [bugfix&enhancement](exchange) not use thread pool to handle exchange block
yiguolei opened a new pull request, #23970: URL: https://github.com/apache/doris/pull/23970 … ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] mrhhsg opened a new pull request, #23971: [fix](nested_loop_join) null value should be output in semi-anti join
mrhhsg opened a new pull request, #23971: URL: https://github.com/apache/doris/pull/23971 ## Proposed changes Schema: ```sql create table t1 (k1 bigint, k2 bigint) ENGINE=OLAP DUPLICATE KEY(k1, k2) COMMENT 'OLAP' DISTRIBUTED BY HASH(k2) BUCKETS 1 PROPERTIES ( "replication_allocation" = "tag.location.default: 1", "is_being_synced" = "false", "storage_format" = "V2", "light_schema_change" = "true", "disable_auto_compaction" = "false", "enable_single_replica_compaction" = "false" ); create table t3 (k1 bigint, k2 bigint) ENGINE=OLAP DUPLICATE KEY(k1, k2) COMMENT 'OLAP' DISTRIBUTED BY HASH(k2) BUCKETS 1 PROPERTIES ( "replication_allocation" = "tag.location.default: 1", "is_being_synced" = "false", "storage_format" = "V2", "light_schema_change" = "true", "disable_auto_compaction" = "false", "enable_single_replica_compaction" = "false" ); ``` Data: ```sql insert into t1 values (1,null),(null,1),(1,2), (null,2),(1,3), (2,4), (2,5), (3,3), (3,4), (20,2), (22,3), (24,4),(null,null); insert into t3 values (1,null),(null,1),(1,4), (1,2), (null,3), (2,4), (3,7), (3,9),(null,null),(5,1); ``` Query: ```sql select t1.* from t1 where not exists ( select k1 from t3 where t1.k2 < t3.k2 ); ``` Result: ``` Empty set ``` Expect result: ``` +--+--+ | k1 | k2 | +--+--+ | NULL | NULL | |1 | NULL | +--+--+ ``` ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] mrhhsg commented on pull request #23971: [fix](nested_loop_join) null value should be output in semi-anti join
mrhhsg commented on PR #23971: URL: https://github.com/apache/doris/pull/23971#issuecomment-1707944504 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt opened a new pull request, #23972: [Improvement](join) some improvement of hash join
BiteThet opened a new pull request, #23972: URL: https://github.com/apache/doris/pull/23972 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #23972: [Improvement](join) some improvement of hash join
BiteThet commented on PR #23972: URL: https://github.com/apache/doris/pull/23972#issuecomment-1707948747 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
