[GitHub] [doris] Gabriel39 commented on pull request #19641: [DRAFT](decimalv3) test auto conversion

2023-05-22 Thread via GitHub


Gabriel39 commented on PR #19641:
URL: https://github.com/apache/doris/pull/19641#issuecomment-1556646743

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] Kikyou1997 opened a new pull request, #19922: [feat](stats) Delete auto analysis task

2023-05-22 Thread via GitHub


Kikyou1997 opened a new pull request, #19922:
URL: https://github.com/apache/doris/pull/19922

   # Proposed changes
   
   Support delete expired auto analysis tasks.
   
   ## Problem summary
   
   Describe your changes.
   
   ## Checklist(Required)
   
   * [ ] Does it affect the original behavior
   * [ ] Has unit tests been added
   * [ ] Has document been added or modified
   * [ ] Does it need to update dependencies
   * [ ] Is this PR support rollback (If NO, please explain WHY)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] 1HanJing opened a new issue, #19923: k8s doris-follower failed

2023-05-22 Thread via GitHub


1HanJing opened a new issue, #19923:
URL: https://github.com/apache/doris/issues/19923

   [ERROR] [Entrypoint]: FE_SERVERS rule error!example: 
$FE_NAME:$FE_HOST_IP:$FE_EDIT_LOG_PORT[,$FE_NAME:$FE_HOST_IP:$FE_EDIT_LOG_PORT]...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] Kikyou1997 commented on pull request #19922: [feat](stats) Delete auto analysis task

2023-05-22 Thread via GitHub


Kikyou1997 commented on PR #19922:
URL: https://github.com/apache/doris/pull/19922#issuecomment-1556651007

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] hello-stephen commented on pull request #19413: [Feature] (Multi-Catalog) support query hll column in doris jdbc table - part 1

2023-05-22 Thread via GitHub


hello-stephen commented on PR #19413:
URL: https://github.com/apache/doris/pull/19413#issuecomment-1556655291

   TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 35.38 seconds
stream load tsv:  435 seconds loaded 74807831229 Bytes, about 164 
MB/s
stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s
stream load orc:  59 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet:  30 seconds loaded 861443392 Bytes, about 27 
MB/s
insert into select:  79.2 seconds inserted 1000 Rows, about 
126K ops/s

https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522070622_clickbench_pr_148156.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] liutang123 opened a new pull request, #19924: [improvement](pipeline) task group scan entity

2023-05-22 Thread via GitHub


liutang123 opened a new pull request, #19924:
URL: https://github.com/apache/doris/pull/19924

   scan task group queue
   
   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem summary
   
   Describe your changes.
   
   ## Checklist(Required)
   
   * [ ] Does it affect the original behavior
   * [ ] Has unit tests been added
   * [ ] Has document been added or modified
   * [ ] Does it need to update dependencies
   * [ ] Is this PR support rollback (If NO, please explain WHY)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19681: [feature](multi-catalog)support paimon catalog

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19681:
URL: https://github.com/apache/doris/pull/19681#issuecomment-1556668987

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] qidaye commented on pull request #19868: [opt](compaction) add pick rowset to compact interval config

2023-05-22 Thread via GitHub


qidaye commented on PR #19868:
URL: https://github.com/apache/doris/pull/19868#issuecomment-1556670570

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19924: [improvement](pipeline) task group scan entity

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19924:
URL: https://github.com/apache/doris/pull/19924#issuecomment-1556671267

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19868: [opt](compaction) add pick rowset to compact interval config

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19868:
URL: https://github.com/apache/doris/pull/19868#issuecomment-1556672232

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] hello-stephen commented on pull request #19921: [Bug](s3-tvf) s3 tvf specify region and s3.region params failed

2023-05-22 Thread via GitHub


hello-stephen commented on PR #19921:
URL: https://github.com/apache/doris/pull/19921#issuecomment-1556673205

   TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.38 seconds
stream load tsv:  420 seconds loaded 74807831229 Bytes, about 169 
MB/s
stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s
stream load orc:  59 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet:  30 seconds loaded 861443392 Bytes, about 27 
MB/s
insert into select:  80.5 seconds inserted 1000 Rows, about 
124K ops/s

https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522071840_clickbench_pr_148166.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] yixiutt commented on pull request #19864: [enhancement](page cache) use separate pk index cache

2023-05-22 Thread via GitHub


yixiutt commented on PR #19864:
URL: https://github.com/apache/doris/pull/19864#issuecomment-1556676975

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19864: [enhancement](page cache) use separate pk index cache

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19864:
URL: https://github.com/apache/doris/pull/19864#issuecomment-1556681142

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19924: [improvement](pipeline) task group scan entity

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19924:
URL: https://github.com/apache/doris/pull/19924#issuecomment-1556684884

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] luzhijing commented on pull request #19910: [doc](fqdn)broker fqdn doc

2023-05-22 Thread via GitHub


luzhijing commented on PR #19910:
URL: https://github.com/apache/doris/pull/19910#issuecomment-1556685879

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] hello-stephen commented on pull request #19641: [DRAFT](decimalv3) test auto conversion

2023-05-22 Thread via GitHub


hello-stephen commented on PR #19641:
URL: https://github.com/apache/doris/pull/19641#issuecomment-1556686377

   TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 36.84 seconds
stream load tsv:  421 seconds loaded 74807831229 Bytes, about 169 
MB/s
stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s
stream load orc:  59 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet:  30 seconds loaded 861443392 Bytes, about 27 
MB/s
insert into select:  77.6 seconds inserted 1000 Rows, about 
128K ops/s

https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522072922_clickbench_pr_148175.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19101: [improvement](bitmap) Use shared_ptr in BitmapValue to avoid deep copying

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19101:
URL: https://github.com/apache/doris/pull/19101#issuecomment-1556687822

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19895: [typo](doc)fix typo in readme

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19895:
URL: https://github.com/apache/doris/pull/19895#issuecomment-1556688600

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] lvshaokang commented on pull request #19921: [Bug](s3-tvf) s3 tvf specify region and s3.region params failed

2023-05-22 Thread via GitHub


lvshaokang commented on PR #19921:
URL: https://github.com/apache/doris/pull/19921#issuecomment-1556690182

   cc @TangSiyang2001 @Yukang-Lian 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] sohardforaname commented on pull request #18869: [Feature](Nereids)support insert into select.

2023-05-22 Thread via GitHub


sohardforaname commented on PR #18869:
URL: https://github.com/apache/doris/pull/18869#issuecomment-1556697003

   run p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] Reminiscent commented on a diff in pull request #18784: [feature](Nereids): pushdown filter through window

2023-05-22 Thread via GitHub


Reminiscent commented on code in PR #18784:
URL: https://github.com/apache/doris/pull/18784#discussion_r1200088484


##
fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticalType.java:
##
@@ -38,6 +38,7 @@ public enum StatisticalType {
 MYSQL_SCAN_NODE,
 ODBC_SCAN_NODE,
 OLAP_SCAN_NODE,
+PARTITION_TOPN_MODE,

Review Comment:
   PARTITION_TOPN_NODE



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] Gabriel39 opened a new pull request, #19925: [Bug](decimal) fix `mod` function

2023-05-22 Thread via GitHub


Gabriel39 opened a new pull request, #19925:
URL: https://github.com/apache/doris/pull/19925

   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem summary
   
   Describe your changes.
   
   ## Checklist(Required)
   
   * [ ] Does it affect the original behavior
   * [ ] Has unit tests been added
   * [ ] Has document been added or modified
   * [ ] Does it need to update dependencies
   * [ ] Is this PR support rollback (If NO, please explain WHY)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] Gabriel39 commented on pull request #19925: [Bug](decimal) fix `mod` function

2023-05-22 Thread via GitHub


Gabriel39 commented on PR #19925:
URL: https://github.com/apache/doris/pull/19925#issuecomment-1556709704

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19925: [Bug](decimal) fix `mod` function

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19925:
URL: https://github.com/apache/doris/pull/19925#issuecomment-1556713336

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] Mryange opened a new pull request, #19926: [fix](DECIMALV3) Fix the error in DECIMALV3 when explicitly casting.

2023-05-22 Thread via GitHub


Mryange opened a new pull request, #19926:
URL: https://github.com/apache/doris/pull/19926

   # Proposed changes
   
   before
   ```
   mysql [test]>select cast(1 as DECIMALV3(16, 2)) /  cast(3 as DECIMALV3(16, 
2));
   +---+
   | CAST(1 AS DECIMALV3(16, 2)) / CAST(3 AS DECIMALV3(16, 2)) |
   +---+
   |  0.00 |
   +---+
   
   
   mysql [test]>select * from divtest;
   +--+--+
   | id   | val  |
   +--+--+
   |3 | 5.00 |
   |2 | 4.00 |
   |1 | 3.00 |
   +--+--+
   
   mysql [test]>select cast(1 as decimalv3(16,2)) / val from divtest;
   +-+
   | CAST(1 AS DECIMALV3(16, 2)) / `val` |
   +-+
   |   0 |
   |   0 |
   |   0 |
   +-+
   ```
   
   after
   ```
   mysql [test]>select cast(1 as DECIMALV3(16, 2)) /  cast(3 as DECIMALV3(16, 
2));
   +---+
   | CAST(1 AS DECIMALV3(16, 2)) / CAST(3 AS DECIMALV3(16, 2)) |
   +---+
   |  0.33 |
   +---+
   
   mysql [test]>select cast(1 as decimalv3(16,2)) / val from divtest;
   +-+
   | CAST(1 AS DECIMALV3(16, 2)) / `val` |
   +-+
   |0.25 |
   |0.20 |
   |0.33 |
   +-+
   ```
   This is because in the previous code, the constant 1.000 would be 
transformed into 1.
   
   Therefore, an additional check is added here. If the constant is converted 
through casting, "ReduceType" should not be applied.
   
   ## Problem summary
   
   Describe your changes.
   
   ## Checklist(Required)
   
   * [ ] Does it affect the original behavior
   * [ ] Has unit tests been added
   * [ ] Has document been added or modified
   * [ ] Does it need to update dependencies
   * [ ] Is this PR support rollback (If NO, please explain WHY)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] Mryange commented on pull request #19926: [fix](DECIMALV3) Fix the error in DECIMALV3 when explicitly casting.

2023-05-22 Thread via GitHub


Mryange commented on PR #19926:
URL: https://github.com/apache/doris/pull/19926#issuecomment-1556720149

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19829: [Fix]Fix lazy open null point

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19829:
URL: https://github.com/apache/doris/pull/19829#issuecomment-1556722194

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19413: [Feature] (Multi-Catalog) support query hll column in doris jdbc table - part 1

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19413:
URL: https://github.com/apache/doris/pull/19413#issuecomment-1556724236

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] dangerousfeng opened a new issue, #19927: [Feature] Create a materialized view as a new table, using the new table name

2023-05-22 Thread via GitHub


dangerousfeng opened a new issue, #19927:
URL: https://github.com/apache/doris/issues/19927

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no 
similar issues.
   
   
   ### Description
   
   物化视图的表能否有一个自定义的表名作为一张新的表存在
   Whether a table with a materialized view can have a custom table name as a 
new table
   
   ### Use case
   
   我现在创建了一个物化视图:
   `create materialized view k1_k2_sumk3 as
   select k1, k2, sum(k3) from example_table group by k1, k2;`
   为了后续使用方便,我需要现创建一个对应的视图:
   `create view example_db.view_that_target_table (k1, k2, k3)
   AS
   select  k1, k2, sum(k3) as v1  from example_table
   group by k1, k2;`
   然后再基于这个视图做后续的查询使用,或者JOIN操作:
   `select k1, v1 from view_that_target_table`
   `select k1, v1 from view_that_target_table join ...`
   是否可以在创建物化视图时指定其表名,因为本身物化视图的数据就是一个新的表数据。
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19829: [Fix]Fix lazy open null point

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19829:
URL: https://github.com/apache/doris/pull/19829#issuecomment-1556726465

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] GoGoWen commented on pull request #19413: [Feature] (Multi-Catalog) support query hll column in doris jdbc table - part 1

2023-05-22 Thread via GitHub


GoGoWen commented on PR #19413:
URL: https://github.com/apache/doris/pull/19413#issuecomment-1556727397

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] GoGoWen commented on pull request #19414: [typo](doc) fix some typo

2023-05-22 Thread via GitHub


GoGoWen commented on PR #19414:
URL: https://github.com/apache/doris/pull/19414#issuecomment-1556728669

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] sohardforaname commented on pull request #18869: [Feature](Nereids)support insert into select.

2023-05-22 Thread via GitHub


sohardforaname commented on PR #18869:
URL: https://github.com/apache/doris/pull/18869#issuecomment-1556733470

   run p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] dangerousfeng commented on issue #19927: [Feature] Create a materialized view as a new table, using the new table name

2023-05-22 Thread via GitHub


dangerousfeng commented on issue #19927:
URL: https://github.com/apache/doris/issues/19927#issuecomment-1556738283

   我找到了类似的需求在 #15847 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19829: [Fix]Fix lazy open null point

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19829:
URL: https://github.com/apache/doris/pull/19829#issuecomment-1556737989

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] dangerousfeng closed issue #19927: [Feature] Create a materialized view as a new table, using the new table name

2023-05-22 Thread via GitHub


dangerousfeng closed issue #19927: [Feature] Create a materialized view as a 
new table, using the new table name
URL: https://github.com/apache/doris/issues/19927


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] hello-stephen commented on pull request #19925: [Bug](decimal) fix `mod` function

2023-05-22 Thread via GitHub


hello-stephen commented on PR #19925:
URL: https://github.com/apache/doris/pull/19925#issuecomment-1556755166

   TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.55 seconds
stream load tsv:  424 seconds loaded 74807831229 Bytes, about 168 
MB/s
stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s
stream load orc:  59 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet:  30 seconds loaded 861443392 Bytes, about 27 
MB/s
insert into select:  80.3 seconds inserted 1000 Rows, about 
124K ops/s

https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522081706_clickbench_pr_148212.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[doris] branch branch-1.2-lts updated: [branch1.2](hudi) skip hidden file of hudi

2023-05-22 Thread morningman
This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
 new f777d3b36a [branch1.2](hudi) skip hidden file of hudi
f777d3b36a is described below

commit f777d3b36af16564f1252b97c5a4d855ca01d20d
Author: morningman 
AuthorDate: Mon May 22 16:15:55 2023 +0800

[branch1.2](hudi) skip hidden file of hudi

When listing data files from hudi's data dir, there are some hidden files
such as `.hoodie_metadata`, we should skip these files.
Otherwise, Doris will read these file as data file, and return file format 
error
---
 .../doris/planner/external/HudiScanProvider.java| 21 +
 1 file changed, 21 insertions(+)

diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/planner/external/HudiScanProvider.java
 
b/fe/fe-core/src/main/java/org/apache/doris/planner/external/HudiScanProvider.java
index 59274ec521..355ea6c732 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/planner/external/HudiScanProvider.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/planner/external/HudiScanProvider.java
@@ -17,13 +17,19 @@
 
 package org.apache.doris.planner.external;
 
+import org.apache.doris.analysis.Expr;
 import org.apache.doris.analysis.TupleDescriptor;
 import org.apache.doris.catalog.external.HMSExternalTable;
 import org.apache.doris.common.DdlException;
 import org.apache.doris.common.MetaNotFoundException;
+import org.apache.doris.common.UserException;
 import org.apache.doris.planner.ColumnRange;
 import org.apache.doris.thrift.TFileFormatType;
 
+import com.google.common.collect.Lists;
+import org.apache.hadoop.mapred.FileSplit;
+import org.apache.hadoop.mapred.InputSplit;
+
 import java.util.Collections;
 import java.util.List;
 import java.util.Map;
@@ -48,4 +54,19 @@ public class HudiScanProvider extends HiveScanProvider {
 public List getPathPartitionKeys() throws DdlException, 
MetaNotFoundException {
 return Collections.emptyList();
 }
+
+@Override
+public List getSplits(List exprs) throws UserException {
+List splits = super.getSplits(exprs);
+List filterSplits = Lists.newArrayList();
+for (InputSplit split : splits) {
+FileSplit fileSplit = (FileSplit) split;
+// skip hidden files start with "."
+if (fileSplit.getPath().getName().startsWith(".")) {
+continue;
+}
+filterSplits.add(split);
+}
+return filterSplits;
+}
 }


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19829: [Fix]Fix lazy open null point

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19829:
URL: https://github.com/apache/doris/pull/19829#issuecomment-1556758099

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] AshinGau commented on pull request #19912: [fix](multi-catalog) JDBC Catalog Unknown UNSIGNED type of mysql, type: [DOUBLE]

2023-05-22 Thread via GitHub


AshinGau commented on PR #19912:
URL: https://github.com/apache/doris/pull/19912#issuecomment-1556758759

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] amorynan opened a new pull request, #19928: [Improve](complex-type)add switch for array/struct/map nesting complex type

2023-05-22 Thread via GitHub


amorynan opened a new pull request, #19928:
URL: https://github.com/apache/doris/pull/19928

   # Proposed changes
   Now we not support array/map/struct nesting each other for many action in be 
, If we do prohibit it in fe, we will meet many undefined action in be , so I 
just add switch to prohibit nesting complex type . When we fully support , can 
make it able.
   Issue Number: close #xxx
   
   ## Problem summary
   
   Describe your changes.
   
   ## Checklist(Required)
   
   * [ ] Does it affect the original behavior
   * [ ] Has unit tests been added
   * [ ] Has document been added or modified
   * [ ] Does it need to update dependencies
   * [ ] Is this PR support rollback (If NO, please explain WHY)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] jackwener opened a new pull request, #19929: [fix](Nereids): commute non-inner join for DPHyp

2023-05-22 Thread via GitHub


jackwener opened a new pull request, #19929:
URL: https://github.com/apache/doris/pull/19929

   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem summary
   
   commute non-inner join for DPHyp
   
   ## Checklist(Required)
   
   * [ ] Does it affect the original behavior
   * [ ] Has unit tests been added
   * [ ] Has document been added or modified
   * [ ] Does it need to update dependencies
   * [ ] Is this PR support rollback (If NO, please explain WHY)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] sohardforaname commented on pull request #18869: [Feature](Nereids)support insert into select.

2023-05-22 Thread via GitHub


sohardforaname commented on PR #18869:
URL: https://github.com/apache/doris/pull/18869#issuecomment-1556773761

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] HHoflittlefish777 opened a new pull request, #19930: [Fix]fix lazy open nullpoint bug

2023-05-22 Thread via GitHub


HHoflittlefish777 opened a new pull request, #19930:
URL: https://github.com/apache/doris/pull/19930

   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem summary
   
   Describe your changes.
   
   ## Checklist(Required)
   
   * [ ] Does it affect the original behavior
   * [ ] Has unit tests been added
   * [ ] Has document been added or modified
   * [ ] Does it need to update dependencies
   * [ ] Is this PR support rollback (If NO, please explain WHY)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] jackwener commented on pull request #19929: [fix](Nereids): commute non-inner join for DPHyp

2023-05-22 Thread via GitHub


jackwener commented on PR #19929:
URL: https://github.com/apache/doris/pull/19929#issuecomment-1556778369

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19829: [Fix]Fix lazy open null point

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19829:
URL: https://github.com/apache/doris/pull/19829#issuecomment-1556780245

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19829: [Fix]Fix lazy open null point

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19829:
URL: https://github.com/apache/doris/pull/19829#issuecomment-1556780292

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] HHoflittlefish777 closed pull request #19930: [Fix]fix lazy open nullpoint bug

2023-05-22 Thread via GitHub


HHoflittlefish777 closed pull request #19930: [Fix]fix lazy open nullpoint bug
URL: https://github.com/apache/doris/pull/19930


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] HHoflittlefish777 commented on pull request #19829: [Fix]Fix lazy open null point

2023-05-22 Thread via GitHub


HHoflittlefish777 commented on PR #19829:
URL: https://github.com/apache/doris/pull/19829#issuecomment-1556785273

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] Gabriel39 opened a new pull request, #19931: [Improvement](function) support decimalv3 for function `least` and `greatest`

2023-05-22 Thread via GitHub


Gabriel39 opened a new pull request, #19931:
URL: https://github.com/apache/doris/pull/19931

   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem summary
   
   Describe your changes.
   
   ## Checklist(Required)
   
   * [ ] Does it affect the original behavior
   * [ ] Has unit tests been added
   * [ ] Has document been added or modified
   * [ ] Does it need to update dependencies
   * [ ] Is this PR support rollback (If NO, please explain WHY)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] Gabriel39 commented on pull request #19931: [Improvement](function) support decimalv3 for function `least` and `greatest`

2023-05-22 Thread via GitHub


Gabriel39 commented on PR #19931:
URL: https://github.com/apache/doris/pull/19931#issuecomment-1556797632

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] xiaokang commented on a diff in pull request #19928: [Improve](complex-type)add switch for array/struct/map nesting complex type

2023-05-22 Thread via GitHub


xiaokang commented on code in PR #19928:
URL: https://github.com/apache/doris/pull/19928#discussion_r1200178095


##
fe/fe-common/src/main/java/org/apache/doris/common/Config.java:
##
@@ -1932,6 +1932,13 @@ public class Config extends ConfigBase {
 @ConfField(mutable = true)
 public static boolean disable_datev1  = true;
 
+/**
+ * Now we not fully support array/struct/map nesting complex type in many 
situation,
+ * so just disable creating nesting complex data type when create table.
+ * We can make it able after we fully support
+ */
+@ConfField(mutable = true)
+public static boolean disable_nesting_complex_type  = true;

Review Comment:
   name 'nested' is better.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19931: [Improvement](function) support decimalv3 for function `least` and `greatest`

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19931:
URL: https://github.com/apache/doris/pull/19931#issuecomment-1556803821

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] jackwener opened a new pull request, #19932: [refactor](Nereids): remove flag hasPushedFilter in Union

2023-05-22 Thread via GitHub


jackwener opened a new pull request, #19932:
URL: https://github.com/apache/doris/pull/19932

   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem summary
   
   Describe your changes.
   
   ## Checklist(Required)
   
   * [ ] Does it affect the original behavior
   * [ ] Has unit tests been added
   * [ ] Has document been added or modified
   * [ ] Does it need to update dependencies
   * [ ] Is this PR support rollback (If NO, please explain WHY)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] HHoflittlefish777 commented on pull request #19829: [Fix]Fix lazy open null point

2023-05-22 Thread via GitHub


HHoflittlefish777 commented on PR #19829:
URL: https://github.com/apache/doris/pull/19829#issuecomment-1556814704

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19829: [Fix]Fix lazy open null point

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19829:
URL: https://github.com/apache/doris/pull/19829#issuecomment-1556819229

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19099: [performance](stream-load) use vector instead of skiplist when insert agg keys

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19099:
URL: https://github.com/apache/doris/pull/19099#issuecomment-1556820324

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] Gabriel39 commented on pull request #19931: [Improvement](function) support decimalv3 for function `least` and `greatest`

2023-05-22 Thread via GitHub


Gabriel39 commented on PR #19931:
URL: https://github.com/apache/doris/pull/19931#issuecomment-1556830647

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] hello-stephen commented on pull request #19413: [Feature] (Multi-Catalog) support query hll column in doris jdbc table - part 1

2023-05-22 Thread via GitHub


hello-stephen commented on PR #19413:
URL: https://github.com/apache/doris/pull/19413#issuecomment-1556832768

   TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 37.11 seconds
stream load tsv:  427 seconds loaded 74807831229 Bytes, about 167 
MB/s
stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s
stream load orc:  59 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet:  30 seconds loaded 861443392 Bytes, about 27 
MB/s
insert into select:  82.0 seconds inserted 1000 Rows, about 
121K ops/s

https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522090232_clickbench_pr_148228.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19931: [Improvement](function) support decimalv3 for function `least` and `greatest`

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19931:
URL: https://github.com/apache/doris/pull/19931#issuecomment-1556833480

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] kaijchen commented on a diff in pull request #19099: [performance](stream-load) use vector instead of skiplist when insert agg keys

2023-05-22 Thread via GitHub


kaijchen commented on code in PR #19099:
URL: https://github.com/apache/doris/pull/19099#discussion_r1200211199


##
be/src/olap/memtable.cpp:
##
@@ -262,108 +228,168 @@ void MemTable::_aggregate_two_row_in_block(RowInBlock* 
new_row, RowInBlock* row_
 }
 // dst is non-sequence row, or dst sequence is smaller
 for (uint32_t cid = _schema->num_key_columns(); cid < _num_columns; ++cid) 
{
-auto col_ptr = _input_mutable_block.mutable_columns()[cid].get();
+auto col_ptr = mutable_block.mutable_columns()[cid].get();
 _agg_functions[cid]->add(row_in_skiplist->agg_places(cid),
  const_cast(&col_ptr),
  new_row->_row_pos, nullptr);
 }
 }
-template 
-void MemTable::_collect_vskiplist_results() {
-if (_keys_type == KeysType::DUP_KEYS) {
-if (_schema->num_key_columns() > 0) {
-_collect_dup_table_with_keys();
-} else {
-// skip sort if the table is dup table without keys
-_collect_dup_table_without_keys();
-}
-} else {
-VecTable::Iterator it(_vec_skip_list.get());
-vectorized::Block in_block = _input_mutable_block.to_block();
-size_t idx = 0;
-for (it.SeekToFirst(); it.Valid(); it.Next()) {
-auto& block_data = in_block.get_columns_with_type_and_name();
-// move key columns
-for (size_t i = 0; i < _schema->num_key_columns(); ++i) {
-_output_mutable_block.get_column_by_position(i)->insert_from(
-*block_data[i].column.get(), it.key()->_row_pos);
-}
-// get value columns from agg_places
-for (size_t i = _schema->num_key_columns(); i < _num_columns; ++i) 
{
-auto function = _agg_functions[i];
-auto agg_place = it.key()->agg_places(i);
-auto col_ptr = 
_output_mutable_block.get_column_by_position(i).get();
-function->insert_result_into(agg_place, *col_ptr);
-if constexpr (is_final) {
-function->destroy(agg_place);
+void MemTable::prepare_block_for_flush(vectorized::Block& in_block) {
+if (_keys_type == KeysType::DUP_KEYS && _schema->num_key_columns() == 0) {
+// skip sort if the table is dup table without keys
+_output_mutable_block.swap(_input_mutable_block);
+return;
+}
+std::vector row_pos_vec;
+DCHECK(in_block.rows() <= std::numeric_limits::max());
+row_pos_vec.reserve(in_block.rows());
+for (int i = 0; i < _row_in_blocks.size(); i++) {
+row_pos_vec.emplace_back(_row_in_blocks[i]->_row_pos);
+}
+_output_mutable_block.add_rows(&in_block, row_pos_vec.data(),
+   row_pos_vec.data() + in_block.rows());
+}
+int MemTable::_sort() {
+_vec_row_comparator->set_block(&_input_mutable_block);
+auto new_row_it = std::next(_row_in_blocks.begin(), _last_sorted_pos);
+size_t same_keys_num = 0;
+bool is_dup = (_keys_type == KeysType::DUP_KEYS);
+// sort new rows
+std::sort(new_row_it, _row_in_blocks.end(),
+  [this, is_dup, &same_keys_num](const RowInBlock* l, const 
RowInBlock* r) -> bool {
+  auto value = (*(this->_vec_row_comparator))(l, r);
+  if (value == 0) {
+  same_keys_num++;
+  return is_dup ? l->_row_pos > r->_row_pos : l->_row_pos 
< r->_row_pos;
+  } else {
+  return value < 0;
+  }
+  });

Review Comment:
   Save this lambda or use a named function to avoid repeating it twice?
   Just for better readability.



##
be/src/olap/memtable.h:
##
@@ -178,12 +186,18 @@ class MemTable {
 //for vectorized
 vectorized::MutableBlock _input_mutable_block;
 vectorized::MutableBlock _output_mutable_block;
+size_t _last_sorted_pos = 0;
 
+//return number of same keys
+int _sort();
+template 
+void finalize_one_row(RowInBlock* row, const 
vectorized::ColumnsWithTypeAndName& block_data,
+  int row_pos);
 template 
-void _collect_vskiplist_results();
-void _collect_dup_table_with_keys();
-void _collect_dup_table_without_keys();
+void _aggregate();
+void prepare_block_for_flush(vectorized::Block& in_block);

Review Comment:
   ```suggestion
   void _prepare_block_for_flush(vectorized::Block& in_block);
   ```



##
be/src/olap/memtable.h:
##
@@ -178,12 +186,18 @@ class MemTable {
 //for vectorized
 vectorized::MutableBlock _input_mutable_block;
 vectorized::MutableBlock _output_mutable_block;
+size_t _last_sorted_pos = 0;
 
+//return number of same keys
+int _sort();
+template 
+void finalize_one_row(RowInBlock* row, const 
vectorized::ColumnsWithTypeAndName& block_data,

Review Comment:
   Private method na

[GitHub] [doris] mrhhsg commented on pull request #19101: [improvement](bitmap) Use shared_ptr in BitmapValue to avoid deep copying

2023-05-22 Thread via GitHub


mrhhsg commented on PR #19101:
URL: https://github.com/apache/doris/pull/19101#issuecomment-1556840244

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] jackwener commented on pull request #19932: [refactor](Nereids): remove flag hasPushedFilter in Union

2023-05-22 Thread via GitHub


jackwener commented on PR #19932:
URL: https://github.com/apache/doris/pull/19932#issuecomment-1556853676

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] hello-stephen commented on pull request #19929: [fix](Nereids): commute non-inner join for DPHyp

2023-05-22 Thread via GitHub


hello-stephen commented on PR #19929:
URL: https://github.com/apache/doris/pull/19929#issuecomment-1556858348

   TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.33 seconds
stream load tsv:  416 seconds loaded 74807831229 Bytes, about 171 
MB/s
stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s
stream load orc:  59 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet:  30 seconds loaded 861443392 Bytes, about 27 
MB/s
insert into select:  78.8 seconds inserted 1000 Rows, about 
126K ops/s

https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522091725_clickbench_pr_148257.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19681: [feature](multi-catalog)support paimon catalog

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19681:
URL: https://github.com/apache/doris/pull/19681#issuecomment-1556862324

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] hello-stephen commented on pull request #19829: [Fix]Fix lazy open null point

2023-05-22 Thread via GitHub


hello-stephen commented on PR #19829:
URL: https://github.com/apache/doris/pull/19829#issuecomment-1556871006

   TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 36.38 seconds
stream load tsv:  422 seconds loaded 74807831229 Bytes, about 169 
MB/s
stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s
stream load orc:  59 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet:  30 seconds loaded 861443392 Bytes, about 27 
MB/s
insert into select:  78.3 seconds inserted 1000 Rows, about 
127K ops/s

https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522092553_clickbench_pr_148262.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] weizhengte commented on pull request #19901: [regressiontest](statistics) Collate and supplement statistics regression test

2023-05-22 Thread via GitHub


weizhengte commented on PR #19901:
URL: https://github.com/apache/doris/pull/19901#issuecomment-1556871208

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] weizhengte commented on pull request #19902: [regressiontest](statistics) draft for stats test-1

2023-05-22 Thread via GitHub


weizhengte commented on PR #19902:
URL: https://github.com/apache/doris/pull/19902#issuecomment-1556871763

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] weizhengte commented on pull request #19903: [regressiontest](statistics) draft for stats test-2

2023-05-22 Thread via GitHub


weizhengte commented on PR #19903:
URL: https://github.com/apache/doris/pull/19903#issuecomment-1556872510

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] weizhengte commented on pull request #19901: [regressiontest](statistics) Collate and supplement statistics regression test

2023-05-22 Thread via GitHub


weizhengte commented on PR #19901:
URL: https://github.com/apache/doris/pull/19901#issuecomment-1556879913

   run compile


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] sohardforaname commented on pull request #18869: [Feature](Nereids)support insert into select.

2023-05-22 Thread via GitHub


sohardforaname commented on PR #18869:
URL: https://github.com/apache/doris/pull/18869#issuecomment-1556884198

   run p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] wsjz opened a new pull request, #19933: [fix](s3)fix s3 resource check

2023-05-22 Thread via GitHub


wsjz opened a new pull request, #19933:
URL: https://github.com/apache/doris/pull/19933

   # Proposed changes
   
   fix s3 resource check:
   
   ERROR 1105 (HY000): Unexpected exception: 
org.apache.doris.common.DdlException: errCode = 2, detailMessage = Missing 
[AWS_ACCESS_KEY] in properties.
   
   
   ## Problem summary
   
   Describe your changes.
   
   ## Checklist(Required)
   
   * [ ] Does it affect the original behavior
   * [ ] Has unit tests been added
   * [ ] Has document been added or modified
   * [ ] Does it need to update dependencies
   * [ ] Is this PR support rollback (If NO, please explain WHY)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] github-actions[bot] commented on pull request #19543: [Refact] (serde) refact mysql serde with data type

2023-05-22 Thread via GitHub


github-actions[bot] commented on PR #19543:
URL: https://github.com/apache/doris/pull/19543#issuecomment-1556887885

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] weizhengte commented on pull request #19901: [regressiontest](statistics) Collate and supplement statistics regression test

2023-05-22 Thread via GitHub


weizhengte commented on PR #19901:
URL: https://github.com/apache/doris/pull/19901#issuecomment-1556900034

   run p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] wsjz commented on pull request #19933: [fix](s3)fix s3 resource check

2023-05-22 Thread via GitHub


wsjz commented on PR #19933:
URL: https://github.com/apache/doris/pull/19933#issuecomment-1556902043

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] hello-stephen commented on pull request #19931: [Improvement](function) support decimalv3 for function `least` and `greatest`

2023-05-22 Thread via GitHub


hello-stephen commented on PR #19931:
URL: https://github.com/apache/doris/pull/19931#issuecomment-1556892023

   TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.3 seconds
stream load tsv:  419 seconds loaded 74807831229 Bytes, about 170 
MB/s
stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s
stream load orc:  59 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet:  30 seconds loaded 861443392 Bytes, about 27 
MB/s
insert into select:  78.3 seconds inserted 1000 Rows, about 
127K ops/s

https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522094008_clickbench_pr_148268.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] weizhengte commented on pull request #19903: [regressiontest](statistics) draft for stats test-2

2023-05-22 Thread via GitHub


weizhengte commented on PR #19903:
URL: https://github.com/apache/doris/pull/19903#issuecomment-1556890885

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] weizhengte commented on pull request #19902: [regressiontest](statistics) draft for stats test-1

2023-05-22 Thread via GitHub


weizhengte commented on PR #19902:
URL: https://github.com/apache/doris/pull/19902#issuecomment-1556890261

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] hello-stephen commented on pull request #19829: [Fix]Fix lazy open null point

2023-05-22 Thread via GitHub


hello-stephen commented on PR #19829:
URL: https://github.com/apache/doris/pull/19829#issuecomment-1556905423

   TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 36.49 seconds
stream load tsv:  425 seconds loaded 74807831229 Bytes, about 167 
MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc:  59 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet:  30 seconds loaded 861443392 Bytes, about 27 
MB/s
insert into select:  82.5 seconds inserted 1000 Rows, about 
121K ops/s

https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522094916_clickbench_pr_148277.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] amorynan commented on pull request #19841: Use keytab and principal to login kerberos

2023-05-22 Thread via GitHub


amorynan commented on PR #19841:
URL: https://github.com/apache/doris/pull/19841#issuecomment-1556914295

   is this pr should we upgrade hdfs lib ? I can't build be locally


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] AshinGau commented on a diff in pull request #19909: [Enhancement](multi catalog) Support hudi mor java side

2023-05-22 Thread via GitHub


AshinGau commented on code in PR #19909:
URL: https://github.com/apache/doris/pull/19909#discussion_r1200253792


##
fe/fe-core/src/main/java/org/apache/doris/catalog/external/HudiExternalTable.java:
##
@@ -0,0 +1,125 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+
+package org.apache.doris.catalog.external;
+
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.datasource.hudi.HudiHMSExternalCatalog;
+import org.apache.doris.thrift.THudiTable;
+import org.apache.doris.thrift.TTableDescriptor;
+import org.apache.doris.thrift.TTableType;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.Lists;
+import org.apache.avro.Schema;
+import org.apache.hadoop.hive.metastore.api.FieldSchema;
+import org.apache.hadoop.hive.metastore.api.Table;
+import org.apache.hudi.common.table.HoodieTableConfig;
+import org.apache.hudi.common.util.Option;
+
+import java.util.Arrays;
+import java.util.List;
+import java.util.stream.Collectors;
+
+public class HudiExternalTable extends ExternalTable {
+
+public HudiExternalTable(long id, String name, String dbName, 
HudiHMSExternalCatalog catalog) {
+super(id, name, catalog, dbName, TableType.HUDI_EXTERNAL_TABLE);
+}
+
+public String getHudiCatalogType() {
+return ((HudiHMSExternalCatalog) catalog).getHudiCatalogType();
+}
+
+protected synchronized void makeSureInitialized() {
+super.makeSureInitialized();
+if (!objectCreated) {
+objectCreated = true;
+}
+}
+
+@Override
+public TTableDescriptor toThrift() {
+List schema = getFullSchema();
+String hudiCatalogType = getHudiCatalogType();
+if (hudiCatalogType.equals("hms")) {
+THudiTable hudiTable = new THudiTable();
+hudiTable.setDbName(dbName);
+hudiTable.setTableName(dbName);
+
+TTableDescriptor tTableDescriptor = new TTableDescriptor(getId(), 
TTableType.HIVE_TABLE, schema.size(), 0,
+getName(), dbName);
+tTableDescriptor.setHudiTable(hudiTable);
+return tTableDescriptor;
+}
+throw new UnsupportedOperationException("hudi not supported  the 
catalog type :" + hudiCatalogType);
+}
+
+public static String fromAvroHudiTypeToHiveTypeString(Schema avroSchema) {
+Schema.Type columnType = avroSchema.getType();
+switch (columnType) {

Review Comment:
   How about `decimal` and `timestamp` ... types?



##
fe/fe-core/src/main/java/org/apache/doris/catalog/external/HudiExternalDatabase.java:
##
@@ -0,0 +1,72 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog.external;
+
+import org.apache.doris.catalog.TableIf;
+import org.apache.doris.datasource.ExternalCatalog;
+import org.apache.doris.datasource.InitDatabaseLog;
+import org.apache.doris.datasource.hudi.HudiHMSExternalCatalog;
+import org.apache.doris.persist.gson.GsonPostProcessable;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.util.Comparator;
+import java.util.List;
+import java.util.stream.Collectors;
+
+public class HudiExternalDatabase extends ExternalDatabase 
implements GsonPostProcessable {

Review Comment:
   We should register the subtype of `DatabaseIf` in 
`org.apache.doris.persist.gson.GsonUtils`



##
fe/fe-cor

[GitHub] [doris] dataroaring commented on a diff in pull request #17881: [Feature-WIP](CCR): Add ccr doris interface (WIP)

2023-05-22 Thread via GitHub


dataroaring commented on code in PR #17881:
URL: https://github.com/apache/doris/pull/17881#discussion_r1193801045


##
be/src/http/action/download_binlog_action.cpp:
##
@@ -0,0 +1,190 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "http/action/download_binlog_action.h"
+
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+
+#include "common/config.h"
+#include "common/logging.h"
+#include "http/http_channel.h"
+#include "http/http_request.h"
+#include "http/utils.h"
+#include "io/fs/local_file_system.h"
+#include "olap/storage_engine.h"
+#include "olap/tablet.h"
+#include "olap/tablet_manager.h"
+#include "runtime/exec_env.h"
+
+namespace doris {
+
+namespace {
+const std::string kMethodParameter = "method";
+const std::string kTokenParameter = "token";
+const std::string kTabletIdParameter = "tablet_id";
+const std::string kBinlogVersionParameter = "binlog_version";
+const std::string kRowsetIdParameter = "rowset_id";
+const std::string kSegmentIndexParameter = "segment_index";
+
+// get http param, if no value throw exception
+const auto& get_http_param(HttpRequest* req, const std::string& param_name) {
+const auto& param = req->param(param_name);
+if (param.empty()) {
+auto error_msg = fmt::format("parameter {} not specified in url.", 
param_name);
+throw std::runtime_error(error_msg);
+}
+return param;
+}
+
+auto get_tablet(const std::string& tablet_id_str) {
+int64_t tablet_id = std::atoll(tablet_id_str.data());
+
+TabletSharedPtr tablet = 
StorageEngine::instance()->tablet_manager()->get_tablet(tablet_id);
+if (tablet == nullptr) {
+auto error = fmt::format("tablet is not exist, tablet_id={}", 
tablet_id);
+LOG(WARNING) << error;
+throw std::runtime_error(error);
+}
+
+return tablet;
+}
+
+// need binlog_version, tablet_id
+void handle_get_binlog_info(HttpRequest* req) {
+try {
+const auto& binlog_version = get_http_param(req, 
kBinlogVersionParameter);
+const auto& tablet_id = get_http_param(req, kTabletIdParameter);
+auto tablet = get_tablet(tablet_id);
+
+const auto& [rowset_id, num_segments] = 
tablet->get_binlog_info(binlog_version);
+auto binlog_info_msg = fmt::format("{}:{}", rowset_id, num_segments);

Review Comment:
   We should return a json message?



##
be/src/http/action/monitor_action.cpp:
##
@@ -1,60 +0,0 @@
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements.  See the NOTICE file
-// distributed with this work for additional information
-// regarding copyright ownership.  The ASF licenses this file
-// to you under the Apache License, Version 2.0 (the
-// "License"); you may not use this file except in compliance
-// with the License.  You may obtain a copy of the License at
-//
-//   http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing,
-// software distributed under the License is distributed on an
-// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-// KIND, either express or implied.  See the License for the
-// specific language governing permissions and limitations
-// under the License.
-
-#include "http/action/monitor_action.h"
-
-#include 
-
-#include 
-#include 
-
-#include "http/http_channel.h"
-#include "http/http_request.h"
-#include "http/http_status.h"
-#include "http/rest_monitor_iface.h"
-
-namespace doris {
-
-const std::string MODULE_KEY = "module";
-
-MonitorAction::MonitorAction() {}
-
-void MonitorAction::register_module(const std::string& name, RestMonitorIface* 
module) {
-_module_by_name.insert(std::make_pair(name, module));
-}
-
-void MonitorAction::handle(HttpRequest* req) {
-LOG(INFO) << req->debug_string();
-const std::string& module = req->param(MODULE_KEY);
-if (module.empty()) {
-std::string err_msg = "No module params\n";
-HttpChannel::send_reply(req, HttpStatus::OK, err_msg);
-return;
-}
-if (_module_by_name.find(module) == _module_by_name.end()) {
-std::string err_msg = "Unknown module(";
-err_msg += module + ")\n";
-HttpChan

[GitHub] [doris] LiBinfeng-01 commented on pull request #19807: [Fix](Nereids) Fix bitmap type in compare operator bug

2023-05-22 Thread via GitHub


LiBinfeng-01 commented on PR #19807:
URL: https://github.com/apache/doris/pull/19807#issuecomment-1556923313

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] AshinGau commented on pull request #19909: [Enhancement](multi catalog) Support hudi mor java side

2023-05-22 Thread via GitHub


AshinGau commented on PR #19909:
URL: https://github.com/apache/doris/pull/19909#issuecomment-1556925875

   Cloud you please add in the description that the FE part of the current PR 
is completed, and the BE part will be supplemented in next PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] sohardforaname commented on pull request #18869: [Feature](Nereids)support insert into select.

2023-05-22 Thread via GitHub


sohardforaname commented on PR #18869:
URL: https://github.com/apache/doris/pull/18869#issuecomment-1556925645

   run p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] hello-stephen commented on pull request #19931: [Improvement](function) support decimalv3 for function `least` and `greatest`

2023-05-22 Thread via GitHub


hello-stephen commented on PR #19931:
URL: https://github.com/apache/doris/pull/19931#issuecomment-1556926940

   TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.47 seconds
stream load tsv:  420 seconds loaded 74807831229 Bytes, about 169 
MB/s
stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s
stream load orc:  59 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet:  30 seconds loaded 861443392 Bytes, about 27 
MB/s
insert into select:  75.6 seconds inserted 1000 Rows, about 
132K ops/s

https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522100255_clickbench_pr_148286.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] zhengshiJ commented on pull request #19650: [Feature](Nereids) support advance mv

2023-05-22 Thread via GitHub


zhengshiJ commented on PR #19650:
URL: https://github.com/apache/doris/pull/19650#issuecomment-1556943352

   run p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] Kikyou1997 opened a new pull request, #19934: [feat](optimizer) Support materailze CTE

2023-05-22 Thread via GitHub


Kikyou1997 opened a new pull request, #19934:
URL: https://github.com/apache/doris/pull/19934

   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem summary
   
   Describe your changes.
   
   ## Checklist(Required)
   
   * [ ] Does it affect the original behavior
   * [ ] Has unit tests been added
   * [ ] Has document been added or modified
   * [ ] Does it need to update dependencies
   * [ ] Is this PR support rollback (If NO, please explain WHY)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] morrySnow opened a new pull request, #19935: [fix](Nereids) join reorder lead to circle in memo

2023-05-22 Thread via GitHub


morrySnow opened a new pull request, #19935:
URL: https://github.com/apache/doris/pull/19935

   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem summary
   
   If we have join as the root node, then after some join reorder join, the 
root Group in Memo will have a GroupExpression including LogicalProject as its 
plan and the children is its ownerGroup.
   This PR add a write rule to ensure we have a Project on the top of the top 
join of plan to avoid circle in Memo.
   
   ## Checklist(Required)
   
   * [ ] Does it affect the original behavior
   * [ ] Has unit tests been added
   * [ ] Has document been added or modified
   * [ ] Does it need to update dependencies
   * [ ] Is this PR support rollback (If NO, please explain WHY)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] morrySnow commented on pull request #19935: [fix](Nereids) join reorder lead to circle in memo

2023-05-22 Thread via GitHub


morrySnow commented on PR #19935:
URL: https://github.com/apache/doris/pull/19935#issuecomment-1556959210

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] HHoflittlefish777 commented on pull request #19829: [Fix]Fix lazy open null point

2023-05-22 Thread via GitHub


HHoflittlefish777 commented on PR #19829:
URL: https://github.com/apache/doris/pull/19829#issuecomment-1556960650

   run p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] yiguolei merged pull request #19910: [doc](fqdn)broker fqdn doc

2023-05-22 Thread via GitHub


yiguolei merged PR #19910:
URL: https://github.com/apache/doris/pull/19910


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[doris] branch master updated: [doc](fqdn)broker fqdn doc #19910

2023-05-22 Thread yiguolei
This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 750a3ea1b4 [doc](fqdn)broker fqdn doc #19910
750a3ea1b4 is described below

commit 750a3ea1b4ca5671a7a0e86c4b0df22dfba3dc5d
Author: zhangdong <493738...@qq.com>
AuthorDate: Mon May 22 18:30:57 2023 +0800

[doc](fqdn)broker fqdn doc #19910
---
 docs/en/docs/admin-manual/cluster-management/fqdn.md| 1 +
 .../Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md| 5 +
 docs/zh-CN/docs/admin-manual/cluster-management/fqdn.md | 1 +
 .../Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md| 6 ++
 4 files changed, 13 insertions(+)

diff --git a/docs/en/docs/admin-manual/cluster-management/fqdn.md 
b/docs/en/docs/admin-manual/cluster-management/fqdn.md
index c1738fe7b4..4df88b0abd 100644
--- a/docs/en/docs/admin-manual/cluster-management/fqdn.md
+++ b/docs/en/docs/admin-manual/cluster-management/fqdn.md
@@ -59,6 +59,7 @@ After Doris supports FQDN, communication between nodes is 
entirely based on FQDN
 4. Verification: It can 'ping fe2' on FE1, and can resolve the correct IP 
address and ping it, indicating that the network environment is available.
 5. fe.conf settings for each FE node ` enable_ fqdn_ mode = true`.
 6. Refer to[Standard deployment](../../install/standard-deployment.md)
+7. Select several machines to deploy broker on six machines as needed, and 
execute `ALTER SYSTEM ADD BROKER broker_name "fe1:8000","be1:8000",...;`.
 
 ### Deployment of Doris for K8S
 
diff --git 
a/docs/en/docs/sql-manual/sql-reference/Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md
 
b/docs/en/docs/sql-manual/sql-reference/Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md
index 0dd6e44632..b1d01552c0 100644
--- 
a/docs/en/docs/sql-manual/sql-reference/Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md
+++ 
b/docs/en/docs/sql-manual/sql-reference/Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md
@@ -48,6 +48,11 @@ ALTER SYSTEM ADD BROKER broker_name 
"broker_host1:broker_ipc_port1","broker_host
 ```sql
  ALTER SYSTEM ADD BROKER "host1:port", "host2:port";
 
+2. When fe enable 
fqdn([fqdn](../../../admin-manual/cluster-management/fqdn.md)),add one Broker
+
+   ```sql
+ALTER SYSTEM ADD BROKER "broker_fqdn1:port";
+   ```
 
 ### Keywords
 
diff --git a/docs/zh-CN/docs/admin-manual/cluster-management/fqdn.md 
b/docs/zh-CN/docs/admin-manual/cluster-management/fqdn.md
index 9ca7abde87..ac129f57e6 100644
--- a/docs/zh-CN/docs/admin-manual/cluster-management/fqdn.md
+++ b/docs/zh-CN/docs/admin-manual/cluster-management/fqdn.md
@@ -59,6 +59,7 @@ Doris 支持 FQDN 之后,各节点之间通信完全基于 FQDN。添加各类
 4. 验证:可以在 FE1 上 `ping fe2` 等,能解析出正确的 IP 并且能 Ping 通,代表网络环境可用。
 5. 每个 FE 节点的 fe.conf 设置 `enable_fqdn_mode = true`。
 6. 参考[标准部署](../../install/standard-deployment.md)
+7. 按需在六台机器上选择几台机器部署broker,执行`ALTER SYSTEM ADD BROKER broker_name 
"fe1:8000","be1:8000",...;`。
 
 ### K8s 部署 Doris
 
diff --git 
a/docs/zh-CN/docs/sql-manual/sql-reference/Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md
 
b/docs/zh-CN/docs/sql-manual/sql-reference/Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md
index 072aefb7fd..9dee64135a 100644
--- 
a/docs/zh-CN/docs/sql-manual/sql-reference/Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md
+++ 
b/docs/zh-CN/docs/sql-manual/sql-reference/Cluster-Management-Statements/ALTER-SYSTEM-ADD-BROKER.md
@@ -48,6 +48,12 @@ ALTER SYSTEM ADD BROKER broker_name 
"broker_host1:broker_ipc_port1","broker_host
```sql
 ALTER SYSTEM ADD BROKER "host1:port", "host2:port";
```
+2. 
fe开启fqdn([fqdn](../../../admin-manual/cluster-management/fqdn.md))时添加一个Broker
+
+   ```sql
+ALTER SYSTEM ADD BROKER "broker_fqdn1:port";
+   ```
+
 
 ### Keywords
 


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] yiguolei merged pull request #19895: [typo](doc)fix typo in readme

2023-05-22 Thread via GitHub


yiguolei merged PR #19895:
URL: https://github.com/apache/doris/pull/19895


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[doris] branch master updated (750a3ea1b4 -> 7ae3724acd)

2023-05-22 Thread yiguolei
This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


from 750a3ea1b4 [doc](fqdn)broker fqdn doc #19910
 add 7ae3724acd [typo](doc)fix typo in readme #19895

No new revisions were added by this update.

Summary of changes:
 README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[doris] branch master updated: [Bug](function) make VcompoundPred optimization work well (#19870)

2023-05-22 Thread yiguolei
This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 9945067e3c [Bug](function) make VcompoundPred optimization work well 
(#19870)
9945067e3c is described below

commit 9945067e3c9bc6a985bb834c631b22e1ef17fafd
Author: Pxl 
AuthorDate: Mon May 22 18:32:17 2023 +0800

[Bug](function) make VcompoundPred optimization work well (#19870)

make VcompoundPred optimization work well
#19818 this pr try to enable VcompoundPred optimization but get wrong 
result on tpcds q28.
The reason is some nullable logic on mysql need special handling.

mysql [regression_test_tpcds_sf1_p1]>select null and false;
++
| NULL AND FALSE |
++
|  0 |
++
1 row in set (0.00 sec)

mysql [regression_test_tpcds_sf1_p1]>select null and true;
+---+
| NULL AND TRUE |
+---+
| NULL  |
+---+
1 row in set (0.00 sec)

mysql [regression_test_tpcds_sf1_p1]>select null or false;
+---+
| NULL OR FALSE |
+---+
| NULL  |
+---+
1 row in set (0.00 sec)

mysql [regression_test_tpcds_sf1_p1]>select null or true;
+--+
| NULL OR TRUE |
+--+
|1 |
+--+
1 row in set (0.00 sec)
---
 be/src/vec/exprs/vcompound_pred.h  | 74 +-
 .../data/query_p0/sql_functions/test_predicate.out | 48 ++
 .../query_p0/sql_functions/test_predicate.groovy   | 57 +
 3 files changed, 148 insertions(+), 31 deletions(-)

diff --git a/be/src/vec/exprs/vcompound_pred.h 
b/be/src/vec/exprs/vcompound_pred.h
index 9ce992d0ec..47f12fa13e 100644
--- a/be/src/vec/exprs/vcompound_pred.h
+++ b/be/src/vec/exprs/vcompound_pred.h
@@ -54,9 +54,13 @@ public:
 
 const std::string& expr_name() const override { return _expr_name; }
 
-Status execute(VExprContext* context, doris::vectorized::Block* block,
+Status execute(VExprContext* context, vectorized::Block* block,
int* result_column_id) override {
-if (children().size() == 1 || !_all_child_is_compound_and_not_const()) 
{
+if (children().size() == 1 || !_all_child_is_compound_and_not_const() 
||
+_children[0]->is_nullable() || _children[1]->is_nullable()) {
+// TODO:
+// When the child is nullable, make the optimization also take 
effect, and the processing of this piece may be more complicated
+// https://dev.mysql.com/doc/refman/8.0/en/logical-operators.html
 return VectorizedFnCall::execute(context, block, result_column_id);
 }
 
@@ -65,15 +69,14 @@ public:
 RETURN_IF_ERROR(_children[0]->execute(context, block, &lhs_id));
 ColumnPtr lhs_column = block->get_by_position(lhs_id).column;
 
-ColumnPtr rhs_column = nullptr;
-
 size_t size = lhs_column->size();
 uint8* __restrict data = _get_raw_data(lhs_column);
 int filted = simd::count_zero_num((int8_t*)data, size);
 bool full = filted == 0;
 bool empty = filted == size;
 
-const uint8* __restrict data_rhs = nullptr;
+ColumnPtr rhs_column = nullptr;
+uint8* __restrict data_rhs = nullptr;
 bool full_rhs = false;
 bool empty_rhs = false;
 
@@ -82,15 +85,6 @@ public:
 RETURN_IF_ERROR(_children[1]->execute(context, block, 
&rhs_id));
 rhs_column = block->get_by_position(rhs_id).column;
 data_rhs = _get_raw_data(rhs_column);
-if (!empty) {
-if (const uint8* null_map =
-
_get_null_map(block->get_by_position(rhs_id).column);
-null_map != nullptr) {
-for (size_t i = 0; i < size; i++) {
-data[i] &= !null_map[i];
-}
-}
-}
 int filted = simd::count_zero_num((int8_t*)data_rhs, size);
 full_rhs = filted == 0;
 empty_rhs = filted == size;
@@ -99,34 +93,54 @@ public:
 };
 
 if (_op == TExprOpcode::COMPOUND_AND) {
-if (!empty) { // empty and any = empty, so lhs should not empty
+if (empty) {
+// empty and any = empty, return lhs
+*result_column_id = lhs_id;
+} else {
 RETURN_IF_ERROR(get_rhs_colum());
-if (empty_rhs) { // any and empty = empty
+
+if (full) {
+// full and any = any, return rhs
 *result_column_id = rhs_id;
-return

[GitHub] [doris] yiguolei merged pull request #19870: [Bug](function) make VcompoundPred optimization work well

2023-05-22 Thread via GitHub


yiguolei merged PR #19870:
URL: https://github.com/apache/doris/pull/19870


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] BiteTheDDDDt closed pull request #19818: [fix](compound_predicate) fix function _all_child_is_compound_and_not_const always return false

2023-05-22 Thread via GitHub


BiteThet closed pull request #19818: [fix](compound_predicate) fix function 
_all_child_is_compound_and_not_const always return false
URL: https://github.com/apache/doris/pull/19818


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] BiteTheDDDDt commented on pull request #19818: [fix](compound_predicate) fix function _all_child_is_compound_and_not_const always return false

2023-05-22 Thread via GitHub


BiteThet commented on PR #19818:
URL: https://github.com/apache/doris/pull/19818#issuecomment-1556982790

   fixed on #19870


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [doris] hello-stephen commented on pull request #19101: [improvement](bitmap) Use shared_ptr in BitmapValue to avoid deep copying

2023-05-22 Thread via GitHub


hello-stephen commented on PR #19101:
URL: https://github.com/apache/doris/pull/19101#issuecomment-1556983446

   TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 36.68 seconds
stream load tsv:  433 seconds loaded 74807831229 Bytes, about 164 
MB/s
stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s
stream load orc:  59 seconds loaded 1101869774 Bytes, about 17 MB/s
stream load parquet:  31 seconds loaded 861443392 Bytes, about 26 
MB/s
insert into select:  80.4 seconds inserted 1000 Rows, about 
124K ops/s

https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230522103952_clickbench_pr_148293.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



  1   2   3   4   >