[GitHub] [doris] Gabriel39 commented on pull request #19641: [DRAFT](decimalv3) test auto conversion
Gabriel39 commented on PR #19641: URL: https://github.com/apache/doris/pull/19641#issuecomment-1556646743 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 opened a new pull request, #19922: [feat](stats) Delete auto analysis task
Kikyou1997 opened a new pull request, #19922: URL: https://github.com/apache/doris/pull/19922 # Proposed changes Support delete expired auto analysis tasks. ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] 1HanJing opened a new issue, #19923: k8s doris-follower failed
1HanJing opened a new issue, #19923: URL: https://github.com/apache/doris/issues/19923 [ERROR] [Entrypoint]: FE_SERVERS rule error!example: $FE_NAME:$FE_HOST_IP:$FE_EDIT_LOG_PORT[,$FE_NAME:$FE_HOST_IP:$FE_EDIT_LOG_PORT]... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on pull request #19922: [feat](stats) Delete auto analysis task
Kikyou1997 commented on PR #19922: URL: https://github.com/apache/doris/pull/19922#issuecomment-1556651007 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #19413: [Feature] (Multi-Catalog) support query hll column in doris jdbc table - part 1
hello-stephen commented on PR #19413: URL: https://github.com/apache/doris/pull/19413#issuecomment-1556655291 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.38 seconds stream load tsv: 435 seconds loaded 74807831229 Bytes, about 164 MB/s stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 79.2 seconds inserted 1000 Rows, about 126K ops/s https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522070622_clickbench_pr_148156.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] liutang123 opened a new pull request, #19924: [improvement](pipeline) task group scan entity
liutang123 opened a new pull request, #19924: URL: https://github.com/apache/doris/pull/19924 scan task group queue # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19681: [feature](multi-catalog)support paimon catalog
github-actions[bot] commented on PR #19681: URL: https://github.com/apache/doris/pull/19681#issuecomment-1556668987 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] qidaye commented on pull request #19868: [opt](compaction) add pick rowset to compact interval config
qidaye commented on PR #19868: URL: https://github.com/apache/doris/pull/19868#issuecomment-1556670570 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19924: [improvement](pipeline) task group scan entity
github-actions[bot] commented on PR #19924: URL: https://github.com/apache/doris/pull/19924#issuecomment-1556671267 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19868: [opt](compaction) add pick rowset to compact interval config
github-actions[bot] commented on PR #19868: URL: https://github.com/apache/doris/pull/19868#issuecomment-1556672232 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #19921: [Bug](s3-tvf) s3 tvf specify region and s3.region params failed
hello-stephen commented on PR #19921: URL: https://github.com/apache/doris/pull/19921#issuecomment-1556673205 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.38 seconds stream load tsv: 420 seconds loaded 74807831229 Bytes, about 169 MB/s stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 80.5 seconds inserted 1000 Rows, about 124K ops/s https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522071840_clickbench_pr_148166.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yixiutt commented on pull request #19864: [enhancement](page cache) use separate pk index cache
yixiutt commented on PR #19864: URL: https://github.com/apache/doris/pull/19864#issuecomment-1556676975 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19864: [enhancement](page cache) use separate pk index cache
github-actions[bot] commented on PR #19864: URL: https://github.com/apache/doris/pull/19864#issuecomment-1556681142 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19924: [improvement](pipeline) task group scan entity
github-actions[bot] commented on PR #19924: URL: https://github.com/apache/doris/pull/19924#issuecomment-1556684884 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] luzhijing commented on pull request #19910: [doc](fqdn)broker fqdn doc
luzhijing commented on PR #19910: URL: https://github.com/apache/doris/pull/19910#issuecomment-1556685879 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #19641: [DRAFT](decimalv3) test auto conversion
hello-stephen commented on PR #19641: URL: https://github.com/apache/doris/pull/19641#issuecomment-1556686377 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 36.84 seconds stream load tsv: 421 seconds loaded 74807831229 Bytes, about 169 MB/s stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 77.6 seconds inserted 1000 Rows, about 128K ops/s https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522072922_clickbench_pr_148175.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19101: [improvement](bitmap) Use shared_ptr in BitmapValue to avoid deep copying
github-actions[bot] commented on PR #19101: URL: https://github.com/apache/doris/pull/19101#issuecomment-1556687822 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19895: [typo](doc)fix typo in readme
github-actions[bot] commented on PR #19895: URL: https://github.com/apache/doris/pull/19895#issuecomment-1556688600 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] lvshaokang commented on pull request #19921: [Bug](s3-tvf) s3 tvf specify region and s3.region params failed
lvshaokang commented on PR #19921: URL: https://github.com/apache/doris/pull/19921#issuecomment-1556690182 cc @TangSiyang2001 @Yukang-Lian -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #18869: [Feature](Nereids)support insert into select.
sohardforaname commented on PR #18869: URL: https://github.com/apache/doris/pull/18869#issuecomment-1556697003 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Reminiscent commented on a diff in pull request #18784: [feature](Nereids): pushdown filter through window
Reminiscent commented on code in PR #18784: URL: https://github.com/apache/doris/pull/18784#discussion_r1200088484 ## fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticalType.java: ## @@ -38,6 +38,7 @@ public enum StatisticalType { MYSQL_SCAN_NODE, ODBC_SCAN_NODE, OLAP_SCAN_NODE, +PARTITION_TOPN_MODE, Review Comment: PARTITION_TOPN_NODE -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 opened a new pull request, #19925: [Bug](decimal) fix `mod` function
Gabriel39 opened a new pull request, #19925: URL: https://github.com/apache/doris/pull/19925 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 commented on pull request #19925: [Bug](decimal) fix `mod` function
Gabriel39 commented on PR #19925: URL: https://github.com/apache/doris/pull/19925#issuecomment-1556709704 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19925: [Bug](decimal) fix `mod` function
github-actions[bot] commented on PR #19925: URL: https://github.com/apache/doris/pull/19925#issuecomment-1556713336 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Mryange opened a new pull request, #19926: [fix](DECIMALV3) Fix the error in DECIMALV3 when explicitly casting.
Mryange opened a new pull request, #19926: URL: https://github.com/apache/doris/pull/19926 # Proposed changes before ``` mysql [test]>select cast(1 as DECIMALV3(16, 2)) / cast(3 as DECIMALV3(16, 2)); +---+ | CAST(1 AS DECIMALV3(16, 2)) / CAST(3 AS DECIMALV3(16, 2)) | +---+ | 0.00 | +---+ mysql [test]>select * from divtest; +--+--+ | id | val | +--+--+ |3 | 5.00 | |2 | 4.00 | |1 | 3.00 | +--+--+ mysql [test]>select cast(1 as decimalv3(16,2)) / val from divtest; +-+ | CAST(1 AS DECIMALV3(16, 2)) / `val` | +-+ | 0 | | 0 | | 0 | +-+ ``` after ``` mysql [test]>select cast(1 as DECIMALV3(16, 2)) / cast(3 as DECIMALV3(16, 2)); +---+ | CAST(1 AS DECIMALV3(16, 2)) / CAST(3 AS DECIMALV3(16, 2)) | +---+ | 0.33 | +---+ mysql [test]>select cast(1 as decimalv3(16,2)) / val from divtest; +-+ | CAST(1 AS DECIMALV3(16, 2)) / `val` | +-+ |0.25 | |0.20 | |0.33 | +-+ ``` This is because in the previous code, the constant 1.000 would be transformed into 1. Therefore, an additional check is added here. If the constant is converted through casting, "ReduceType" should not be applied. ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Mryange commented on pull request #19926: [fix](DECIMALV3) Fix the error in DECIMALV3 when explicitly casting.
Mryange commented on PR #19926: URL: https://github.com/apache/doris/pull/19926#issuecomment-1556720149 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19829: [Fix]Fix lazy open null point
github-actions[bot] commented on PR #19829: URL: https://github.com/apache/doris/pull/19829#issuecomment-1556722194 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19413: [Feature] (Multi-Catalog) support query hll column in doris jdbc table - part 1
github-actions[bot] commented on PR #19413: URL: https://github.com/apache/doris/pull/19413#issuecomment-1556724236 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dangerousfeng opened a new issue, #19927: [Feature] Create a materialized view as a new table, using the new table name
dangerousfeng opened a new issue, #19927: URL: https://github.com/apache/doris/issues/19927 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description 物化视图的表能否有一个自定义的表名作为一张新的表存在 Whether a table with a materialized view can have a custom table name as a new table ### Use case 我现在创建了一个物化视图: `create materialized view k1_k2_sumk3 as select k1, k2, sum(k3) from example_table group by k1, k2;` 为了后续使用方便,我需要现创建一个对应的视图: `create view example_db.view_that_target_table (k1, k2, k3) AS select k1, k2, sum(k3) as v1 from example_table group by k1, k2;` 然后再基于这个视图做后续的查询使用,或者JOIN操作: `select k1, v1 from view_that_target_table` `select k1, v1 from view_that_target_table join ...` 是否可以在创建物化视图时指定其表名,因为本身物化视图的数据就是一个新的表数据。 ### Related issues _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19829: [Fix]Fix lazy open null point
github-actions[bot] commented on PR #19829: URL: https://github.com/apache/doris/pull/19829#issuecomment-1556726465 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] GoGoWen commented on pull request #19413: [Feature] (Multi-Catalog) support query hll column in doris jdbc table - part 1
GoGoWen commented on PR #19413: URL: https://github.com/apache/doris/pull/19413#issuecomment-1556727397 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] GoGoWen commented on pull request #19414: [typo](doc) fix some typo
GoGoWen commented on PR #19414: URL: https://github.com/apache/doris/pull/19414#issuecomment-1556728669 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #18869: [Feature](Nereids)support insert into select.
sohardforaname commented on PR #18869: URL: https://github.com/apache/doris/pull/18869#issuecomment-1556733470 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dangerousfeng commented on issue #19927: [Feature] Create a materialized view as a new table, using the new table name
dangerousfeng commented on issue #19927: URL: https://github.com/apache/doris/issues/19927#issuecomment-1556738283 我找到了类似的需求在 #15847 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19829: [Fix]Fix lazy open null point
github-actions[bot] commented on PR #19829: URL: https://github.com/apache/doris/pull/19829#issuecomment-1556737989 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dangerousfeng closed issue #19927: [Feature] Create a materialized view as a new table, using the new table name
dangerousfeng closed issue #19927: [Feature] Create a materialized view as a new table, using the new table name URL: https://github.com/apache/doris/issues/19927 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #19925: [Bug](decimal) fix `mod` function
hello-stephen commented on PR #19925: URL: https://github.com/apache/doris/pull/19925#issuecomment-1556755166 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.55 seconds stream load tsv: 424 seconds loaded 74807831229 Bytes, about 168 MB/s stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 80.3 seconds inserted 1000 Rows, about 124K ops/s https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522081706_clickbench_pr_148212.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.2-lts updated: [branch1.2](hudi) skip hidden file of hudi
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch branch-1.2-lts in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-1.2-lts by this push: new f777d3b36a [branch1.2](hudi) skip hidden file of hudi f777d3b36a is described below commit f777d3b36af16564f1252b97c5a4d855ca01d20d Author: morningman AuthorDate: Mon May 22 16:15:55 2023 +0800 [branch1.2](hudi) skip hidden file of hudi When listing data files from hudi's data dir, there are some hidden files such as `.hoodie_metadata`, we should skip these files. Otherwise, Doris will read these file as data file, and return file format error --- .../doris/planner/external/HudiScanProvider.java| 21 + 1 file changed, 21 insertions(+) diff --git a/fe/fe-core/src/main/java/org/apache/doris/planner/external/HudiScanProvider.java b/fe/fe-core/src/main/java/org/apache/doris/planner/external/HudiScanProvider.java index 59274ec521..355ea6c732 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/planner/external/HudiScanProvider.java +++ b/fe/fe-core/src/main/java/org/apache/doris/planner/external/HudiScanProvider.java @@ -17,13 +17,19 @@ package org.apache.doris.planner.external; +import org.apache.doris.analysis.Expr; import org.apache.doris.analysis.TupleDescriptor; import org.apache.doris.catalog.external.HMSExternalTable; import org.apache.doris.common.DdlException; import org.apache.doris.common.MetaNotFoundException; +import org.apache.doris.common.UserException; import org.apache.doris.planner.ColumnRange; import org.apache.doris.thrift.TFileFormatType; +import com.google.common.collect.Lists; +import org.apache.hadoop.mapred.FileSplit; +import org.apache.hadoop.mapred.InputSplit; + import java.util.Collections; import java.util.List; import java.util.Map; @@ -48,4 +54,19 @@ public class HudiScanProvider extends HiveScanProvider { public List getPathPartitionKeys() throws DdlException, MetaNotFoundException { return Collections.emptyList(); } + +@Override +public List getSplits(List exprs) throws UserException { +List splits = super.getSplits(exprs); +List filterSplits = Lists.newArrayList(); +for (InputSplit split : splits) { +FileSplit fileSplit = (FileSplit) split; +// skip hidden files start with "." +if (fileSplit.getPath().getName().startsWith(".")) { +continue; +} +filterSplits.add(split); +} +return filterSplits; +} } - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19829: [Fix]Fix lazy open null point
github-actions[bot] commented on PR #19829: URL: https://github.com/apache/doris/pull/19829#issuecomment-1556758099 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] AshinGau commented on pull request #19912: [fix](multi-catalog) JDBC Catalog Unknown UNSIGNED type of mysql, type: [DOUBLE]
AshinGau commented on PR #19912: URL: https://github.com/apache/doris/pull/19912#issuecomment-1556758759 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] amorynan opened a new pull request, #19928: [Improve](complex-type)add switch for array/struct/map nesting complex type
amorynan opened a new pull request, #19928: URL: https://github.com/apache/doris/pull/19928 # Proposed changes Now we not support array/map/struct nesting each other for many action in be , If we do prohibit it in fe, we will meet many undefined action in be , so I just add switch to prohibit nesting complex type . When we fully support , can make it able. Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener opened a new pull request, #19929: [fix](Nereids): commute non-inner join for DPHyp
jackwener opened a new pull request, #19929: URL: https://github.com/apache/doris/pull/19929 # Proposed changes Issue Number: close #xxx ## Problem summary commute non-inner join for DPHyp ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #18869: [Feature](Nereids)support insert into select.
sohardforaname commented on PR #18869: URL: https://github.com/apache/doris/pull/18869#issuecomment-1556773761 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HHoflittlefish777 opened a new pull request, #19930: [Fix]fix lazy open nullpoint bug
HHoflittlefish777 opened a new pull request, #19930: URL: https://github.com/apache/doris/pull/19930 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener commented on pull request #19929: [fix](Nereids): commute non-inner join for DPHyp
jackwener commented on PR #19929: URL: https://github.com/apache/doris/pull/19929#issuecomment-1556778369 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19829: [Fix]Fix lazy open null point
github-actions[bot] commented on PR #19829: URL: https://github.com/apache/doris/pull/19829#issuecomment-1556780245 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19829: [Fix]Fix lazy open null point
github-actions[bot] commented on PR #19829: URL: https://github.com/apache/doris/pull/19829#issuecomment-1556780292 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HHoflittlefish777 closed pull request #19930: [Fix]fix lazy open nullpoint bug
HHoflittlefish777 closed pull request #19930: [Fix]fix lazy open nullpoint bug URL: https://github.com/apache/doris/pull/19930 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HHoflittlefish777 commented on pull request #19829: [Fix]Fix lazy open null point
HHoflittlefish777 commented on PR #19829: URL: https://github.com/apache/doris/pull/19829#issuecomment-1556785273 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 opened a new pull request, #19931: [Improvement](function) support decimalv3 for function `least` and `greatest`
Gabriel39 opened a new pull request, #19931: URL: https://github.com/apache/doris/pull/19931 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 commented on pull request #19931: [Improvement](function) support decimalv3 for function `least` and `greatest`
Gabriel39 commented on PR #19931: URL: https://github.com/apache/doris/pull/19931#issuecomment-1556797632 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xiaokang commented on a diff in pull request #19928: [Improve](complex-type)add switch for array/struct/map nesting complex type
xiaokang commented on code in PR #19928: URL: https://github.com/apache/doris/pull/19928#discussion_r1200178095 ## fe/fe-common/src/main/java/org/apache/doris/common/Config.java: ## @@ -1932,6 +1932,13 @@ public class Config extends ConfigBase { @ConfField(mutable = true) public static boolean disable_datev1 = true; +/** + * Now we not fully support array/struct/map nesting complex type in many situation, + * so just disable creating nesting complex data type when create table. + * We can make it able after we fully support + */ +@ConfField(mutable = true) +public static boolean disable_nesting_complex_type = true; Review Comment: name 'nested' is better. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19931: [Improvement](function) support decimalv3 for function `least` and `greatest`
github-actions[bot] commented on PR #19931: URL: https://github.com/apache/doris/pull/19931#issuecomment-1556803821 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener opened a new pull request, #19932: [refactor](Nereids): remove flag hasPushedFilter in Union
jackwener opened a new pull request, #19932: URL: https://github.com/apache/doris/pull/19932 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HHoflittlefish777 commented on pull request #19829: [Fix]Fix lazy open null point
HHoflittlefish777 commented on PR #19829: URL: https://github.com/apache/doris/pull/19829#issuecomment-1556814704 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19829: [Fix]Fix lazy open null point
github-actions[bot] commented on PR #19829: URL: https://github.com/apache/doris/pull/19829#issuecomment-1556819229 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19099: [performance](stream-load) use vector instead of skiplist when insert agg keys
github-actions[bot] commented on PR #19099: URL: https://github.com/apache/doris/pull/19099#issuecomment-1556820324 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 commented on pull request #19931: [Improvement](function) support decimalv3 for function `least` and `greatest`
Gabriel39 commented on PR #19931: URL: https://github.com/apache/doris/pull/19931#issuecomment-1556830647 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #19413: [Feature] (Multi-Catalog) support query hll column in doris jdbc table - part 1
hello-stephen commented on PR #19413: URL: https://github.com/apache/doris/pull/19413#issuecomment-1556832768 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 37.11 seconds stream load tsv: 427 seconds loaded 74807831229 Bytes, about 167 MB/s stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 82.0 seconds inserted 1000 Rows, about 121K ops/s https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522090232_clickbench_pr_148228.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19931: [Improvement](function) support decimalv3 for function `least` and `greatest`
github-actions[bot] commented on PR #19931: URL: https://github.com/apache/doris/pull/19931#issuecomment-1556833480 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] kaijchen commented on a diff in pull request #19099: [performance](stream-load) use vector instead of skiplist when insert agg keys
kaijchen commented on code in PR #19099: URL: https://github.com/apache/doris/pull/19099#discussion_r1200211199 ## be/src/olap/memtable.cpp: ## @@ -262,108 +228,168 @@ void MemTable::_aggregate_two_row_in_block(RowInBlock* new_row, RowInBlock* row_ } // dst is non-sequence row, or dst sequence is smaller for (uint32_t cid = _schema->num_key_columns(); cid < _num_columns; ++cid) { -auto col_ptr = _input_mutable_block.mutable_columns()[cid].get(); +auto col_ptr = mutable_block.mutable_columns()[cid].get(); _agg_functions[cid]->add(row_in_skiplist->agg_places(cid), const_cast(&col_ptr), new_row->_row_pos, nullptr); } } -template -void MemTable::_collect_vskiplist_results() { -if (_keys_type == KeysType::DUP_KEYS) { -if (_schema->num_key_columns() > 0) { -_collect_dup_table_with_keys(); -} else { -// skip sort if the table is dup table without keys -_collect_dup_table_without_keys(); -} -} else { -VecTable::Iterator it(_vec_skip_list.get()); -vectorized::Block in_block = _input_mutable_block.to_block(); -size_t idx = 0; -for (it.SeekToFirst(); it.Valid(); it.Next()) { -auto& block_data = in_block.get_columns_with_type_and_name(); -// move key columns -for (size_t i = 0; i < _schema->num_key_columns(); ++i) { -_output_mutable_block.get_column_by_position(i)->insert_from( -*block_data[i].column.get(), it.key()->_row_pos); -} -// get value columns from agg_places -for (size_t i = _schema->num_key_columns(); i < _num_columns; ++i) { -auto function = _agg_functions[i]; -auto agg_place = it.key()->agg_places(i); -auto col_ptr = _output_mutable_block.get_column_by_position(i).get(); -function->insert_result_into(agg_place, *col_ptr); -if constexpr (is_final) { -function->destroy(agg_place); +void MemTable::prepare_block_for_flush(vectorized::Block& in_block) { +if (_keys_type == KeysType::DUP_KEYS && _schema->num_key_columns() == 0) { +// skip sort if the table is dup table without keys +_output_mutable_block.swap(_input_mutable_block); +return; +} +std::vector row_pos_vec; +DCHECK(in_block.rows() <= std::numeric_limits::max()); +row_pos_vec.reserve(in_block.rows()); +for (int i = 0; i < _row_in_blocks.size(); i++) { +row_pos_vec.emplace_back(_row_in_blocks[i]->_row_pos); +} +_output_mutable_block.add_rows(&in_block, row_pos_vec.data(), + row_pos_vec.data() + in_block.rows()); +} +int MemTable::_sort() { +_vec_row_comparator->set_block(&_input_mutable_block); +auto new_row_it = std::next(_row_in_blocks.begin(), _last_sorted_pos); +size_t same_keys_num = 0; +bool is_dup = (_keys_type == KeysType::DUP_KEYS); +// sort new rows +std::sort(new_row_it, _row_in_blocks.end(), + [this, is_dup, &same_keys_num](const RowInBlock* l, const RowInBlock* r) -> bool { + auto value = (*(this->_vec_row_comparator))(l, r); + if (value == 0) { + same_keys_num++; + return is_dup ? l->_row_pos > r->_row_pos : l->_row_pos < r->_row_pos; + } else { + return value < 0; + } + }); Review Comment: Save this lambda or use a named function to avoid repeating it twice? Just for better readability. ## be/src/olap/memtable.h: ## @@ -178,12 +186,18 @@ class MemTable { //for vectorized vectorized::MutableBlock _input_mutable_block; vectorized::MutableBlock _output_mutable_block; +size_t _last_sorted_pos = 0; +//return number of same keys +int _sort(); +template +void finalize_one_row(RowInBlock* row, const vectorized::ColumnsWithTypeAndName& block_data, + int row_pos); template -void _collect_vskiplist_results(); -void _collect_dup_table_with_keys(); -void _collect_dup_table_without_keys(); +void _aggregate(); +void prepare_block_for_flush(vectorized::Block& in_block); Review Comment: ```suggestion void _prepare_block_for_flush(vectorized::Block& in_block); ``` ## be/src/olap/memtable.h: ## @@ -178,12 +186,18 @@ class MemTable { //for vectorized vectorized::MutableBlock _input_mutable_block; vectorized::MutableBlock _output_mutable_block; +size_t _last_sorted_pos = 0; +//return number of same keys +int _sort(); +template +void finalize_one_row(RowInBlock* row, const vectorized::ColumnsWithTypeAndName& block_data, Review Comment: Private method na
[GitHub] [doris] mrhhsg commented on pull request #19101: [improvement](bitmap) Use shared_ptr in BitmapValue to avoid deep copying
mrhhsg commented on PR #19101: URL: https://github.com/apache/doris/pull/19101#issuecomment-1556840244 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener commented on pull request #19932: [refactor](Nereids): remove flag hasPushedFilter in Union
jackwener commented on PR #19932: URL: https://github.com/apache/doris/pull/19932#issuecomment-1556853676 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #19929: [fix](Nereids): commute non-inner join for DPHyp
hello-stephen commented on PR #19929: URL: https://github.com/apache/doris/pull/19929#issuecomment-1556858348 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.33 seconds stream load tsv: 416 seconds loaded 74807831229 Bytes, about 171 MB/s stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 78.8 seconds inserted 1000 Rows, about 126K ops/s https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522091725_clickbench_pr_148257.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19681: [feature](multi-catalog)support paimon catalog
github-actions[bot] commented on PR #19681: URL: https://github.com/apache/doris/pull/19681#issuecomment-1556862324 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #19829: [Fix]Fix lazy open null point
hello-stephen commented on PR #19829: URL: https://github.com/apache/doris/pull/19829#issuecomment-1556871006 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 36.38 seconds stream load tsv: 422 seconds loaded 74807831229 Bytes, about 169 MB/s stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 78.3 seconds inserted 1000 Rows, about 127K ops/s https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522092553_clickbench_pr_148262.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] weizhengte commented on pull request #19901: [regressiontest](statistics) Collate and supplement statistics regression test
weizhengte commented on PR #19901: URL: https://github.com/apache/doris/pull/19901#issuecomment-1556871208 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] weizhengte commented on pull request #19902: [regressiontest](statistics) draft for stats test-1
weizhengte commented on PR #19902: URL: https://github.com/apache/doris/pull/19902#issuecomment-1556871763 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] weizhengte commented on pull request #19903: [regressiontest](statistics) draft for stats test-2
weizhengte commented on PR #19903: URL: https://github.com/apache/doris/pull/19903#issuecomment-1556872510 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] weizhengte commented on pull request #19901: [regressiontest](statistics) Collate and supplement statistics regression test
weizhengte commented on PR #19901: URL: https://github.com/apache/doris/pull/19901#issuecomment-1556879913 run compile -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #18869: [Feature](Nereids)support insert into select.
sohardforaname commented on PR #18869: URL: https://github.com/apache/doris/pull/18869#issuecomment-1556884198 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wsjz opened a new pull request, #19933: [fix](s3)fix s3 resource check
wsjz opened a new pull request, #19933: URL: https://github.com/apache/doris/pull/19933 # Proposed changes fix s3 resource check: ERROR 1105 (HY000): Unexpected exception: org.apache.doris.common.DdlException: errCode = 2, detailMessage = Missing [AWS_ACCESS_KEY] in properties. ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #19543: [Refact] (serde) refact mysql serde with data type
github-actions[bot] commented on PR #19543: URL: https://github.com/apache/doris/pull/19543#issuecomment-1556887885 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] weizhengte commented on pull request #19901: [regressiontest](statistics) Collate and supplement statistics regression test
weizhengte commented on PR #19901: URL: https://github.com/apache/doris/pull/19901#issuecomment-1556900034 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wsjz commented on pull request #19933: [fix](s3)fix s3 resource check
wsjz commented on PR #19933: URL: https://github.com/apache/doris/pull/19933#issuecomment-1556902043 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #19931: [Improvement](function) support decimalv3 for function `least` and `greatest`
hello-stephen commented on PR #19931: URL: https://github.com/apache/doris/pull/19931#issuecomment-1556892023 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.3 seconds stream load tsv: 419 seconds loaded 74807831229 Bytes, about 170 MB/s stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 78.3 seconds inserted 1000 Rows, about 127K ops/s https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522094008_clickbench_pr_148268.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] weizhengte commented on pull request #19903: [regressiontest](statistics) draft for stats test-2
weizhengte commented on PR #19903: URL: https://github.com/apache/doris/pull/19903#issuecomment-1556890885 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] weizhengte commented on pull request #19902: [regressiontest](statistics) draft for stats test-1
weizhengte commented on PR #19902: URL: https://github.com/apache/doris/pull/19902#issuecomment-1556890261 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #19829: [Fix]Fix lazy open null point
hello-stephen commented on PR #19829: URL: https://github.com/apache/doris/pull/19829#issuecomment-1556905423 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 36.49 seconds stream load tsv: 425 seconds loaded 74807831229 Bytes, about 167 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 82.5 seconds inserted 1000 Rows, about 121K ops/s https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522094916_clickbench_pr_148277.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] amorynan commented on pull request #19841: Use keytab and principal to login kerberos
amorynan commented on PR #19841: URL: https://github.com/apache/doris/pull/19841#issuecomment-1556914295 is this pr should we upgrade hdfs lib ? I can't build be locally -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] AshinGau commented on a diff in pull request #19909: [Enhancement](multi catalog) Support hudi mor java side
AshinGau commented on code in PR #19909: URL: https://github.com/apache/doris/pull/19909#discussion_r1200253792 ## fe/fe-core/src/main/java/org/apache/doris/catalog/external/HudiExternalTable.java: ## @@ -0,0 +1,125 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + + +package org.apache.doris.catalog.external; + +import org.apache.doris.catalog.Column; +import org.apache.doris.catalog.Type; +import org.apache.doris.datasource.hudi.HudiHMSExternalCatalog; +import org.apache.doris.thrift.THudiTable; +import org.apache.doris.thrift.TTableDescriptor; +import org.apache.doris.thrift.TTableType; + +import com.google.common.base.Preconditions; +import com.google.common.collect.Lists; +import org.apache.avro.Schema; +import org.apache.hadoop.hive.metastore.api.FieldSchema; +import org.apache.hadoop.hive.metastore.api.Table; +import org.apache.hudi.common.table.HoodieTableConfig; +import org.apache.hudi.common.util.Option; + +import java.util.Arrays; +import java.util.List; +import java.util.stream.Collectors; + +public class HudiExternalTable extends ExternalTable { + +public HudiExternalTable(long id, String name, String dbName, HudiHMSExternalCatalog catalog) { +super(id, name, catalog, dbName, TableType.HUDI_EXTERNAL_TABLE); +} + +public String getHudiCatalogType() { +return ((HudiHMSExternalCatalog) catalog).getHudiCatalogType(); +} + +protected synchronized void makeSureInitialized() { +super.makeSureInitialized(); +if (!objectCreated) { +objectCreated = true; +} +} + +@Override +public TTableDescriptor toThrift() { +List schema = getFullSchema(); +String hudiCatalogType = getHudiCatalogType(); +if (hudiCatalogType.equals("hms")) { +THudiTable hudiTable = new THudiTable(); +hudiTable.setDbName(dbName); +hudiTable.setTableName(dbName); + +TTableDescriptor tTableDescriptor = new TTableDescriptor(getId(), TTableType.HIVE_TABLE, schema.size(), 0, +getName(), dbName); +tTableDescriptor.setHudiTable(hudiTable); +return tTableDescriptor; +} +throw new UnsupportedOperationException("hudi not supported the catalog type :" + hudiCatalogType); +} + +public static String fromAvroHudiTypeToHiveTypeString(Schema avroSchema) { +Schema.Type columnType = avroSchema.getType(); +switch (columnType) { Review Comment: How about `decimal` and `timestamp` ... types? ## fe/fe-core/src/main/java/org/apache/doris/catalog/external/HudiExternalDatabase.java: ## @@ -0,0 +1,72 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.catalog.external; + +import org.apache.doris.catalog.TableIf; +import org.apache.doris.datasource.ExternalCatalog; +import org.apache.doris.datasource.InitDatabaseLog; +import org.apache.doris.datasource.hudi.HudiHMSExternalCatalog; +import org.apache.doris.persist.gson.GsonPostProcessable; + +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; + +import java.util.Comparator; +import java.util.List; +import java.util.stream.Collectors; + +public class HudiExternalDatabase extends ExternalDatabase implements GsonPostProcessable { Review Comment: We should register the subtype of `DatabaseIf` in `org.apache.doris.persist.gson.GsonUtils` ## fe/fe-cor
[GitHub] [doris] dataroaring commented on a diff in pull request #17881: [Feature-WIP](CCR): Add ccr doris interface (WIP)
dataroaring commented on code in PR #17881: URL: https://github.com/apache/doris/pull/17881#discussion_r1193801045 ## be/src/http/action/download_binlog_action.cpp: ## @@ -0,0 +1,190 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +#include "http/action/download_binlog_action.h" + +#include +#include + +#include +#include +#include +#include + +#include "common/config.h" +#include "common/logging.h" +#include "http/http_channel.h" +#include "http/http_request.h" +#include "http/utils.h" +#include "io/fs/local_file_system.h" +#include "olap/storage_engine.h" +#include "olap/tablet.h" +#include "olap/tablet_manager.h" +#include "runtime/exec_env.h" + +namespace doris { + +namespace { +const std::string kMethodParameter = "method"; +const std::string kTokenParameter = "token"; +const std::string kTabletIdParameter = "tablet_id"; +const std::string kBinlogVersionParameter = "binlog_version"; +const std::string kRowsetIdParameter = "rowset_id"; +const std::string kSegmentIndexParameter = "segment_index"; + +// get http param, if no value throw exception +const auto& get_http_param(HttpRequest* req, const std::string& param_name) { +const auto& param = req->param(param_name); +if (param.empty()) { +auto error_msg = fmt::format("parameter {} not specified in url.", param_name); +throw std::runtime_error(error_msg); +} +return param; +} + +auto get_tablet(const std::string& tablet_id_str) { +int64_t tablet_id = std::atoll(tablet_id_str.data()); + +TabletSharedPtr tablet = StorageEngine::instance()->tablet_manager()->get_tablet(tablet_id); +if (tablet == nullptr) { +auto error = fmt::format("tablet is not exist, tablet_id={}", tablet_id); +LOG(WARNING) << error; +throw std::runtime_error(error); +} + +return tablet; +} + +// need binlog_version, tablet_id +void handle_get_binlog_info(HttpRequest* req) { +try { +const auto& binlog_version = get_http_param(req, kBinlogVersionParameter); +const auto& tablet_id = get_http_param(req, kTabletIdParameter); +auto tablet = get_tablet(tablet_id); + +const auto& [rowset_id, num_segments] = tablet->get_binlog_info(binlog_version); +auto binlog_info_msg = fmt::format("{}:{}", rowset_id, num_segments); Review Comment: We should return a json message? ## be/src/http/action/monitor_action.cpp: ## @@ -1,60 +0,0 @@ -// Licensed to the Apache Software Foundation (ASF) under one -// or more contributor license agreements. See the NOTICE file -// distributed with this work for additional information -// regarding copyright ownership. The ASF licenses this file -// to you under the Apache License, Version 2.0 (the -// "License"); you may not use this file except in compliance -// with the License. You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, -// software distributed under the License is distributed on an -// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -// KIND, either express or implied. See the License for the -// specific language governing permissions and limitations -// under the License. - -#include "http/action/monitor_action.h" - -#include - -#include -#include - -#include "http/http_channel.h" -#include "http/http_request.h" -#include "http/http_status.h" -#include "http/rest_monitor_iface.h" - -namespace doris { - -const std::string MODULE_KEY = "module"; - -MonitorAction::MonitorAction() {} - -void MonitorAction::register_module(const std::string& name, RestMonitorIface* module) { -_module_by_name.insert(std::make_pair(name, module)); -} - -void MonitorAction::handle(HttpRequest* req) { -LOG(INFO) << req->debug_string(); -const std::string& module = req->param(MODULE_KEY); -if (module.empty()) { -std::string err_msg = "No module params\n"; -HttpChannel::send_reply(req, HttpStatus::OK, err_msg); -return; -} -if (_module_by_name.find(module) == _module_by_name.end()) { -std::string err_msg = "Unknown module("; -err_msg += module + ")\n"; -HttpChan
[GitHub] [doris] LiBinfeng-01 commented on pull request #19807: [Fix](Nereids) Fix bitmap type in compare operator bug
LiBinfeng-01 commented on PR #19807: URL: https://github.com/apache/doris/pull/19807#issuecomment-1556923313 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] AshinGau commented on pull request #19909: [Enhancement](multi catalog) Support hudi mor java side
AshinGau commented on PR #19909: URL: https://github.com/apache/doris/pull/19909#issuecomment-1556925875 Cloud you please add in the description that the FE part of the current PR is completed, and the BE part will be supplemented in next PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #18869: [Feature](Nereids)support insert into select.
sohardforaname commented on PR #18869: URL: https://github.com/apache/doris/pull/18869#issuecomment-1556925645 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #19931: [Improvement](function) support decimalv3 for function `least` and `greatest`
hello-stephen commented on PR #19931: URL: https://github.com/apache/doris/pull/19931#issuecomment-1556926940 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.47 seconds stream load tsv: 420 seconds loaded 74807831229 Bytes, about 169 MB/s stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 75.6 seconds inserted 1000 Rows, about 132K ops/s https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522100255_clickbench_pr_148286.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhengshiJ commented on pull request #19650: [Feature](Nereids) support advance mv
zhengshiJ commented on PR #19650: URL: https://github.com/apache/doris/pull/19650#issuecomment-1556943352 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 opened a new pull request, #19934: [feat](optimizer) Support materailze CTE
Kikyou1997 opened a new pull request, #19934: URL: https://github.com/apache/doris/pull/19934 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow opened a new pull request, #19935: [fix](Nereids) join reorder lead to circle in memo
morrySnow opened a new pull request, #19935: URL: https://github.com/apache/doris/pull/19935 # Proposed changes Issue Number: close #xxx ## Problem summary If we have join as the root node, then after some join reorder join, the root Group in Memo will have a GroupExpression including LogicalProject as its plan and the children is its ownerGroup. This PR add a write rule to ensure we have a Project on the top of the top join of plan to avoid circle in Memo. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #19935: [fix](Nereids) join reorder lead to circle in memo
morrySnow commented on PR #19935: URL: https://github.com/apache/doris/pull/19935#issuecomment-1556959210 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HHoflittlefish777 commented on pull request #19829: [Fix]Fix lazy open null point
HHoflittlefish777 commented on PR #19829: URL: https://github.com/apache/doris/pull/19829#issuecomment-1556960650 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #19910: [doc](fqdn)broker fqdn doc
yiguolei merged PR #19910: URL: https://github.com/apache/doris/pull/19910 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [doc](fqdn)broker fqdn doc #19910
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new 750a3ea1b4 [doc](fqdn)broker fqdn doc #19910 750a3ea1b4 is described below commit 750a3ea1b4ca5671a7a0e86c4b0df22dfba3dc5d Author: zhangdong <493738...@qq.com> AuthorDate: Mon May 22 18:30:57 2023 +0800 [doc](fqdn)broker fqdn doc #19910 --- docs/en/docs/admin-manual/cluster-management/fqdn.md| 1 + .../Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md| 5 + docs/zh-CN/docs/admin-manual/cluster-management/fqdn.md | 1 + .../Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md| 6 ++ 4 files changed, 13 insertions(+) diff --git a/docs/en/docs/admin-manual/cluster-management/fqdn.md b/docs/en/docs/admin-manual/cluster-management/fqdn.md index c1738fe7b4..4df88b0abd 100644 --- a/docs/en/docs/admin-manual/cluster-management/fqdn.md +++ b/docs/en/docs/admin-manual/cluster-management/fqdn.md @@ -59,6 +59,7 @@ After Doris supports FQDN, communication between nodes is entirely based on FQDN 4. Verification: It can 'ping fe2' on FE1, and can resolve the correct IP address and ping it, indicating that the network environment is available. 5. fe.conf settings for each FE node ` enable_ fqdn_ mode = true`. 6. Refer to[Standard deployment](../../install/standard-deployment.md) +7. Select several machines to deploy broker on six machines as needed, and execute `ALTER SYSTEM ADD BROKER broker_name "fe1:8000","be1:8000",...;`. ### Deployment of Doris for K8S diff --git a/docs/en/docs/sql-manual/sql-reference/Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md b/docs/en/docs/sql-manual/sql-reference/Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md index 0dd6e44632..b1d01552c0 100644 --- a/docs/en/docs/sql-manual/sql-reference/Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md +++ b/docs/en/docs/sql-manual/sql-reference/Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md @@ -48,6 +48,11 @@ ALTER SYSTEM ADD BROKER broker_name "broker_host1:broker_ipc_port1","broker_host ```sql ALTER SYSTEM ADD BROKER "host1:port", "host2:port"; +2. When fe enable fqdn([fqdn](../../../admin-manual/cluster-management/fqdn.md)),add one Broker + + ```sql +ALTER SYSTEM ADD BROKER "broker_fqdn1:port"; + ``` ### Keywords diff --git a/docs/zh-CN/docs/admin-manual/cluster-management/fqdn.md b/docs/zh-CN/docs/admin-manual/cluster-management/fqdn.md index 9ca7abde87..ac129f57e6 100644 --- a/docs/zh-CN/docs/admin-manual/cluster-management/fqdn.md +++ b/docs/zh-CN/docs/admin-manual/cluster-management/fqdn.md @@ -59,6 +59,7 @@ Doris 支持 FQDN 之后,各节点之间通信完全基于 FQDN。添加各类 4. 验证:可以在 FE1 上 `ping fe2` 等,能解析出正确的 IP 并且能 Ping 通,代表网络环境可用。 5. 每个 FE 节点的 fe.conf 设置 `enable_fqdn_mode = true`。 6. 参考[标准部署](../../install/standard-deployment.md) +7. 按需在六台机器上选择几台机器部署broker,执行`ALTER SYSTEM ADD BROKER broker_name "fe1:8000","be1:8000",...;`。 ### K8s 部署 Doris diff --git a/docs/zh-CN/docs/sql-manual/sql-reference/Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md b/docs/zh-CN/docs/sql-manual/sql-reference/Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md index 072aefb7fd..9dee64135a 100644 --- a/docs/zh-CN/docs/sql-manual/sql-reference/Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md +++ b/docs/zh-CN/docs/sql-manual/sql-reference/Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md @@ -48,6 +48,12 @@ ALTER SYSTEM ADD BROKER broker_name "broker_host1:broker_ipc_port1","broker_host ```sql ALTER SYSTEM ADD BROKER "host1:port", "host2:port"; ``` +2. fe开启fqdn([fqdn](../../../admin-manual/cluster-management/fqdn.md))时添加一个Broker + + ```sql +ALTER SYSTEM ADD BROKER "broker_fqdn1:port"; + ``` + ### Keywords - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #19895: [typo](doc)fix typo in readme
yiguolei merged PR #19895: URL: https://github.com/apache/doris/pull/19895 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (750a3ea1b4 -> 7ae3724acd)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 750a3ea1b4 [doc](fqdn)broker fqdn doc #19910 add 7ae3724acd [typo](doc)fix typo in readme #19895 No new revisions were added by this update. Summary of changes: README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Bug](function) make VcompoundPred optimization work well (#19870)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new 9945067e3c [Bug](function) make VcompoundPred optimization work well (#19870) 9945067e3c is described below commit 9945067e3c9bc6a985bb834c631b22e1ef17fafd Author: Pxl AuthorDate: Mon May 22 18:32:17 2023 +0800 [Bug](function) make VcompoundPred optimization work well (#19870) make VcompoundPred optimization work well #19818 this pr try to enable VcompoundPred optimization but get wrong result on tpcds q28. The reason is some nullable logic on mysql need special handling. mysql [regression_test_tpcds_sf1_p1]>select null and false; ++ | NULL AND FALSE | ++ | 0 | ++ 1 row in set (0.00 sec) mysql [regression_test_tpcds_sf1_p1]>select null and true; +---+ | NULL AND TRUE | +---+ | NULL | +---+ 1 row in set (0.00 sec) mysql [regression_test_tpcds_sf1_p1]>select null or false; +---+ | NULL OR FALSE | +---+ | NULL | +---+ 1 row in set (0.00 sec) mysql [regression_test_tpcds_sf1_p1]>select null or true; +--+ | NULL OR TRUE | +--+ |1 | +--+ 1 row in set (0.00 sec) --- be/src/vec/exprs/vcompound_pred.h | 74 +- .../data/query_p0/sql_functions/test_predicate.out | 48 ++ .../query_p0/sql_functions/test_predicate.groovy | 57 + 3 files changed, 148 insertions(+), 31 deletions(-) diff --git a/be/src/vec/exprs/vcompound_pred.h b/be/src/vec/exprs/vcompound_pred.h index 9ce992d0ec..47f12fa13e 100644 --- a/be/src/vec/exprs/vcompound_pred.h +++ b/be/src/vec/exprs/vcompound_pred.h @@ -54,9 +54,13 @@ public: const std::string& expr_name() const override { return _expr_name; } -Status execute(VExprContext* context, doris::vectorized::Block* block, +Status execute(VExprContext* context, vectorized::Block* block, int* result_column_id) override { -if (children().size() == 1 || !_all_child_is_compound_and_not_const()) { +if (children().size() == 1 || !_all_child_is_compound_and_not_const() || +_children[0]->is_nullable() || _children[1]->is_nullable()) { +// TODO: +// When the child is nullable, make the optimization also take effect, and the processing of this piece may be more complicated +// https://dev.mysql.com/doc/refman/8.0/en/logical-operators.html return VectorizedFnCall::execute(context, block, result_column_id); } @@ -65,15 +69,14 @@ public: RETURN_IF_ERROR(_children[0]->execute(context, block, &lhs_id)); ColumnPtr lhs_column = block->get_by_position(lhs_id).column; -ColumnPtr rhs_column = nullptr; - size_t size = lhs_column->size(); uint8* __restrict data = _get_raw_data(lhs_column); int filted = simd::count_zero_num((int8_t*)data, size); bool full = filted == 0; bool empty = filted == size; -const uint8* __restrict data_rhs = nullptr; +ColumnPtr rhs_column = nullptr; +uint8* __restrict data_rhs = nullptr; bool full_rhs = false; bool empty_rhs = false; @@ -82,15 +85,6 @@ public: RETURN_IF_ERROR(_children[1]->execute(context, block, &rhs_id)); rhs_column = block->get_by_position(rhs_id).column; data_rhs = _get_raw_data(rhs_column); -if (!empty) { -if (const uint8* null_map = - _get_null_map(block->get_by_position(rhs_id).column); -null_map != nullptr) { -for (size_t i = 0; i < size; i++) { -data[i] &= !null_map[i]; -} -} -} int filted = simd::count_zero_num((int8_t*)data_rhs, size); full_rhs = filted == 0; empty_rhs = filted == size; @@ -99,34 +93,54 @@ public: }; if (_op == TExprOpcode::COMPOUND_AND) { -if (!empty) { // empty and any = empty, so lhs should not empty +if (empty) { +// empty and any = empty, return lhs +*result_column_id = lhs_id; +} else { RETURN_IF_ERROR(get_rhs_colum()); -if (empty_rhs) { // any and empty = empty + +if (full) { +// full and any = any, return rhs *result_column_id = rhs_id; -return
[GitHub] [doris] yiguolei merged pull request #19870: [Bug](function) make VcompoundPred optimization work well
yiguolei merged PR #19870: URL: https://github.com/apache/doris/pull/19870 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt closed pull request #19818: [fix](compound_predicate) fix function _all_child_is_compound_and_not_const always return false
BiteThet closed pull request #19818: [fix](compound_predicate) fix function _all_child_is_compound_and_not_const always return false URL: https://github.com/apache/doris/pull/19818 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #19818: [fix](compound_predicate) fix function _all_child_is_compound_and_not_const always return false
BiteThet commented on PR #19818: URL: https://github.com/apache/doris/pull/19818#issuecomment-1556982790 fixed on #19870 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #19101: [improvement](bitmap) Use shared_ptr in BitmapValue to avoid deep copying
hello-stephen commented on PR #19101: URL: https://github.com/apache/doris/pull/19101#issuecomment-1556983446 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 36.68 seconds stream load tsv: 433 seconds loaded 74807831229 Bytes, about 164 MB/s stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 80.4 seconds inserted 1000 Rows, about 124K ops/s https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522103952_clickbench_pr_148293.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org