[PR] [fix](Outfile) optimize the error message of `SELECT INTO OUTFILE` [doris]
BePPPower opened a new pull request, #45859: URL: https://github.com/apache/doris/pull/45859 optimize the error message of `SELECT INTO OUTFILE` ### Check List (For Author) - Test - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [x] No need to test or manual test. Explain why: - [x] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason - Behavior changed: - [x] No. - [ ] Yes. - Does this need documentation? - [x] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Outfile) optimize the error message of `SELECT INTO OUTFILE` [doris]
hello-stephen commented on PR #45859: URL: https://github.com/apache/doris/pull/45859#issuecomment-2560814296 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) fix fe folding constant of string functions and add more cases [doris]
doris-robot commented on PR #45233: URL: https://github.com/apache/doris/pull/45233#issuecomment-2560815111 TPC-H: Total hot run time: 39889 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit f71027c12bd90c2b0b45458a52d4f9703a470911, data reload: false -- Round 1 -- q1 17612 744673177317 q2 2041182 178 178 q3 10534 113011891130 q4 10228 701 695 695 q5 7586270326962696 q6 243 147 150 147 q7 969 626 634 626 q8 9256186018731860 q9 6553644064076407 q10 6996232323422323 q11 462 256 274 256 q12 408 226 218 218 q13 17767 287329812873 q14 244 217 224 217 q15 554 510 483 483 q16 664 611 585 585 q17 985 547 526 526 q18 7420665866106610 q19 1340100010251000 q20 494 195 186 186 q21 4007324233743242 q22 382 317 314 314 Total cold run time: 106745 ms Total hot run time: 39889 ms - Round 2, with runtime_filter_mode=off - q1 7223726673257266 q2 337 229 234 229 q3 2947278329952783 q4 2093183917741774 q5 5681562056485620 q6 224 140 139 139 q7 2207182718471827 q8 3310358335043504 q9 8907889789328897 q10 3627356235843562 q11 611 506 511 506 q12 802 605 631 605 q13 12760 307531603075 q14 296 269 263 263 q15 562 492 533 492 q16 687 629 615 615 q17 1868160416021602 q18 7885745375227453 q19 1676165013461346 q20 2033180117771777 q21 5454535853265326 q22 647 555 588 555 Total cold run time: 71837 ms Total hot run time: 59216 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](regression-test) use unified trigger_and_wait_compaction method [doris]
py023 commented on PR #45761: URL: https://github.com/apache/doris/pull/45761#issuecomment-2560815110 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [feat](docker)Modify the init_be and start_be scripts to meet the requirements for rapid Docker startup(Merge 2.1). [doris]
FreeOnePlus opened a new pull request, #45858: URL: https://github.com/apache/doris/pull/45858 ### What problem does this PR solve? Issue Number: close #xxx Related PR: #45267 Master PR: #45269 Problem Summary: To meet the needs of rapid Docker startup, I have made adjustments to two related scripts in the Docker startup process. First, I added a env `SKIP_CHECK_ULIMIT` to the `start_be.sh` script, which will skip the size checks for `swap`, `ulimit`, and `max_map_count`. At the same time, I used `--console` to start the process and print logs. The reason why I did not use the `--daemon` daemon command to execute is that starting with a foreground log printing method in a Docker container is the correct and reliable approach. At the same time, I added a check logic for a `be.conf` configuration item in the `init_be.sh` script: if it is the first time starting, append the export `SKIP_CHECK_ULIMIT=true` to skip the `ulimit` value check in the BE process. In summary, these adjustments can meet the basic requirements for rapid Docker startup usage. ### Release note None ### Check List (For Author) - Test - [ ] Regression test - [ ] Unit Test - [x] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason 1. Set the `ulimit`, `swap`, and `max_map_count` check values on the host machine to values that do not meet the requirements. 2. Run `export SKIP_CHECK_ULIMIT=true` and `sh start_be.sh --console` to start the BE process, which starts up normally. 3. Run `export SKIP_CHECK_ULIMIT=false`，then using the `--daemon` or `--console` option to start, the BE process fails to start normally. - Behavior changed: - [x] No. - [ ] Yes. - Does this need documentation? - [x] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [x] Confirm the release note - [x] Confirm test cases - [x] Confirm document - [x] Add branch pick label -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](nereids) add rule count distinct split [doris]
924060929 commented on code in PR #45209:
URL: https://github.com/apache/doris/pull/45209#discussion_r1896498949
##
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/DistinctSplitTest.java:
##
@@ -0,0 +1,97 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.rewrite;
+
+import org.apache.doris.nereids.trees.plans.Plan;
+import org.apache.doris.nereids.util.MatchingUtils;
+import org.apache.doris.nereids.util.MemoPatternMatchSupported;
+import org.apache.doris.nereids.util.PlanChecker;
+import org.apache.doris.utframe.TestWithFeService;
+
+import org.junit.jupiter.api.Test;
+
+public class DistinctSplitTest extends TestWithFeService implements
MemoPatternMatchSupported {
+@Override
+protected void runBeforeAll() throws Exception {
+createDatabase("test");
+createTable("create table test.test_distinct_multi(a int, b int, c
int, d varchar(10), e date)"
++ "distributed by hash(a) properties('replication_num'='1');");
+connectContext.setDatabase("test");
+
connectContext.getSessionVariable().setDisableNereidsRules("PRUNE_EMPTY_PARTITION");
+}
+
+@Test
+void multiCountWithoutGby() {
+String sql = "select count(distinct b), count(distinct a) from
test_distinct_multi";
+PlanChecker.from(connectContext).checkExplain(sql, planner -> {
+Plan plan = planner.getOptimizedPlan();
+MatchingUtils.assertMatches(plan,
physicalCTEAnchor(physicalCTEProducer(any()),
physicalResultSink(physicalProject(physicalNestedLoopJoin(
Review Comment:
Use line break and indentation to format code to see the shape of plan, this
lines width than 120 characters?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](metrics) Remove IntAtomicCounter & CoreLocal #45742 [doris]
zhiqiang- commented on PR #45870: URL: https://github.com/apache/doris/pull/45870#issuecomment-2560969738 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](nereids)support create function command in nereids [doris]
Thearas commented on PR #45874: URL: https://github.com/apache/doris/pull/45874#issuecomment-2560974674 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [feature](nereids)support create function command in nereids [doris]
starocean999 opened a new pull request, #45874: URL: https://github.com/apache/doris/pull/45874 ### What problem does this PR solve? Issue Number: close #xxx Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason - Behavior changed: - [ ] No. - [ ] Yes. - Does this need documentation? - [ ] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](parser) Syntax error for add partition with null null [doris]
doris-robot commented on PR #45865: URL: https://github.com/apache/doris/pull/45865#issuecomment-2560976441 TPC-DS: Total hot run time: 197684 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 8267636b46bcaa2f1c5b2bfb3d05d23dd8563385, data reload: false query1 1298960 916 916 query2 6250234023232323 query3 11108 484048724840 query4 33296 23512 23439 23439 query5 3923469 457 457 query6 286 182 185 182 query7 4004315 321 315 query8 295 241 248 241 query9 9725275227472747 query10 448 244 247 244 query11 17862 15182 15311 15182 query12 154 104 98 98 query13 1563408 412 408 query14 9069787774877487 query15 249 216 189 189 query16 7464451 449 449 query17 1642614 607 607 query18 1403334 345 334 query19 382 174 165 165 query20 135 122 121 121 query21 216 108 115 108 query22 4697456643434343 query23 34410 33525 33639 33525 query24 10479 252525982525 query25 630 399 388 388 query26 1201154 154 154 query27 2353341 350 341 query28 7702249125132491 query29 732 420 422 420 query30 230 153 150 150 query31 1018841 854 841 query32 90 58 56 56 query33 767 291 302 291 query34 1186559 520 520 query35 894 749 799 749 query36 1123981 969 969 query37 154 77 79 77 query38 4282411942344119 query39 1537147414571457 query40 213 102 100 100 query41 47 42 44 42 query42 123 111 111 111 query43 538 536 498 498 query44 1326836 846 836 query45 194 174 167 167 query46 1212746 701 701 query47 2047198119861981 query48 413 343 325 325 query49 1000384 392 384 query50 819 407 383 383 query51 7471723371507150 query52 107 97 95 95 query53 260 190 186 186 query54 1213431 419 419 query55 79 83 86 83 query56 279 260 239 239 query57 1336116011751160 query58 246 227 225 225 query59 3471331932773277 query60 322 259 249 249 query61 115 105 110 105 query62 933 740 765 740 query63 226 186 193 186 query64 4020744 677 677 query65 3314329432633263 query66 881 306 316 306 query67 16591 15559 15509 15509 query68 5639569 573 569 query69 494 258 260 258 query70 1268115111291129 query71 492 265 254 254 query72 6976407941584079 query73 793 374 372 372 query74 9974894787568756 query75 3514262126152615 query76 3821109610781078 query77 569 276 267 267 query78 10371 940494359404 query79 1882603 609 603 query80 1208423 434 423 query81 511 230 233 230 query82 494 121 119 119 query83 207 151 141 141 query84 282 71 76 71 query85 1101325 304 304 query86 354 302 302 302 query87 4573460045724572 query88 419022172217 query89 433 304 289 289 query90 2095184 185 184 query91 141 101 115 101 query92 70 52 51 51 query93 4007551 541 541 query94 913 300 282 282 query95 358 255 296 255 query96 628 279 283 279 query97 2827267026612661 query98 220 195 195 195 query99 1710142314181418 Total cold run time: 305168 ms Total hot run time: 197684 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [feature](nereids)support create function command in nereids [doris]
starocean999 commented on PR #45874: URL: https://github.com/apache/doris/pull/45874#issuecomment-2560974771 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch revert-1580-polish_query_data created (now 3d418089b0a)
This is an automated email from the ASF dual-hosted git repository. kassiez pushed a change to branch revert-1580-polish_query_data in repository https://gitbox.apache.org/repos/asf/doris-website.git at 3d418089b0a Revert "[polish](query-data) polish query data docs (#1580)" This branch includes the following new commits: new 3d418089b0a Revert "[polish](query-data) polish query data docs (#1580)" The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement adminSetPartitionVersionCommand in nereids [doris]
doris-robot commented on PR #45426: URL: https://github.com/apache/doris/pull/45426#issuecomment-2560978071 TPC-H: Total hot run time: 40341 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 46fe09cb87ea4527f868e0e359d81bfe14523864, data reload: false -- Round 1 -- q1 16144 757373407340 q2 1162185 178 178 q3 9321117411441144 q4 6235766 688 688 q5 2765268427022684 q6 241 154 157 154 q7 1022636 616 616 q8 1825187119381871 q9 6513644364006400 q10 2643237723542354 q11 378 264 269 264 q12 417 238 229 229 q13 17536 294629802946 q14 259 226 225 225 q15 569 518 497 497 q16 638 618 590 590 q17 965 613 564 564 q18 7374679867846784 q19 1151955 1012955 q20 401 196 192 192 q21 3983339033573357 q22 382 309 316 309 Total cold run time: 81924 ms Total hot run time: 40341 ms - Round 2, with runtime_filter_mode=off - q1 7298729172627262 q2 326 238 238 238 q3 2926277228262772 q4 1995175516801680 q5 5386540054385400 q6 217 137 141 137 q7 2124174217171717 q8 3235340334183403 q9 8616857586958575 q10 3534347534603460 q11 591 505 491 491 q12 752 574 597 574 q13 12414 296929352935 q14 301 263 266 263 q15 553 489 509 489 q16 669 622 649 622 q17 1813155315511551 q18 7874743273837383 q19 1745152315921523 q20 2140180418971804 q21 5446528852515251 q22 632 614 599 599 Total cold run time: 70587 ms Total hot run time: 58129 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](metrics) Remove IntAtomicCounter & CoreLocal #45742 [doris]
zhiqiang- commented on PR #45870: URL: https://github.com/apache/doris/pull/45870#issuecomment-2560978089 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) 01/01: Revert "[polish](query-data) polish query data docs (#1580)"
This is an automated email from the ASF dual-hosted git repository. kassiez pushed a commit to branch revert-1580-polish_query_data in repository https://gitbox.apache.org/repos/asf/doris-website.git commit 3d418089b0a3b191b2208d2ff1aad5a875ba394c Author: KassieZ <139741991+kass...@users.noreply.github.com> AuthorDate: Tue Dec 24 18:46:59 2024 +0800 Revert "[polish](query-data) polish query data docs (#1580)" This reverts commit 23d22e617174806f71ede7cd3979e1a0b9dfe213. --- docs/query-data/complex-type.md| 4 +++- docs/query-data/cte.md | 26 ++ docs/query-data/join.md| 22 ++ docs/query-data/subquery.md| 6 ++--- docs/query-data/udf/alias-function.md | 4 .../current/query-data/complex-type.md | 4 +++- .../current/query-data/cte.md | 26 ++ .../current/query-data/join.md | 14 +++- .../current/query-data/udf/alias-function.md | 4 .../current/query-data/window-function.md | 12 +- .../version-2.1/query-data/complex-type.md | 4 +++- .../version-2.1/query-data/cte.md | 26 ++ .../version-2.1/query-data/join.md | 14 +++- .../version-2.1/query-data/udf/alias-function.md | 4 .../version-2.1/query-data/window-function.md | 13 +-- .../version-3.0/query-data/complex-type.md | 4 +++- .../version-3.0/query-data/cte.md | 26 ++ .../version-3.0/query-data/join.md | 14 +++- .../version-3.0/query-data/udf/alias-function.md | 4 .../version-3.0/query-data/window-function.md | 12 +- .../version-2.1/query-data/complex-type.md | 5 - versioned_docs/version-2.1/query-data/cte.md | 26 ++ versioned_docs/version-2.1/query-data/join.md | 22 ++ versioned_docs/version-2.1/query-data/subquery.md | 6 ++--- .../version-2.1/query-data/udf/alias-function.md | 4 .../version-3.0/query-data/complex-type.md | 5 - versioned_docs/version-3.0/query-data/cte.md | 26 ++ versioned_docs/version-3.0/query-data/join.md | 22 ++ versioned_docs/version-3.0/query-data/subquery.md | 6 ++--- .../version-3.0/query-data/udf/alias-function.md | 4 30 files changed, 149 insertions(+), 220 deletions(-) diff --git a/docs/query-data/complex-type.md b/docs/query-data/complex-type.md index 9db1b523fd8..92dd4862cf7 100644 --- a/docs/query-data/complex-type.md +++ b/docs/query-data/complex-type.md @@ -28,4 +28,6 @@ Doris supports complex types such as Array, Map, Struct, and JSON. Doris provides various functions specifically designed for these complex types. -For detailed information on the supported functions, please refer to the [SQL manual - Array functions](../sql-manual/sql-functions/array-functions/array), [SQL manual - Map functions](../sql-manual/sql-data-types/semi-structured/MAP) data type, [SQL manual - Struct functions](../sql-manual/sql-functions/struct-functions/struct), and [SQL manual - Json functions](../sql-manual/sql-functions/json-functions/json-parse). +For detailed information on the supported functions, please refer to the SQL manual- [Array functions](../sql-manual/sql-functions/array-functions/array), [SQL manual-Struct functions](../sql-manual/sql-functions/struct-functions/struct), and [SQL manual-JSON functions](../sql-manual/sql-functions/json-functions/json-parse). + +To check the support for Map functions, you can refer to the SQL manual docs under the section for the [MAP](../sql-manual/sql-data-types/semi-structured/MAP) data type. diff --git a/docs/query-data/cte.md b/docs/query-data/cte.md index 04299e4afec..941d74639a1 100644 --- a/docs/query-data/cte.md +++ b/docs/query-data/cte.md @@ -30,14 +30,12 @@ Common Table Expression (CTE) define a temporary result set that can be referenc To specify a CTE, use the `WITH` clause with one or more comma-separated clauses. Each clause provides a subquery that generates a result set and associates a name with the subquery. -Doris supports nested CTE。 Within the statement that contains the `WITH` clause, you can reference each CTE name to access the corresponding CTE result set. CTE names can be referenced in other CTE, allowing you to define CTE based on other CTE. +Within the statement that contains the `WITH` clause, you can reference each CTE name to access the corresponding CTE result set. CTE names can be referenced in other CTE, allowing you to define CTE based on other CTE. -Doris **DOES NOT** support recursive CTE. For more information, please read MySQL manual about [recursive CTE](https://dev.mysql.com/doc/refman/8.4/en/with.html#com
(doris-website) branch master updated: [polish](query-data) polish query data docs (#1580)
This is an automated email from the ASF dual-hosted git repository. kassiez pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 23d22e61717 [polish](query-data) polish query data docs (#1580) 23d22e61717 is described below commit 23d22e617174806f71ede7cd3979e1a0b9dfe213 Author: morrySnow AuthorDate: Tue Dec 24 18:46:25 2024 +0800 [polish](query-data) polish query data docs (#1580) ## Versions - [x] dev - [x] 3.0 - [x] 2.1 - [ ] 2.0 ## Languages - [x] Chinese - [x] English ## Docs Checklist - [ ] Checked by AI - [ ] Test Cases Built --- docs/query-data/complex-type.md| 4 +--- docs/query-data/cte.md | 26 -- docs/query-data/join.md| 22 -- docs/query-data/subquery.md| 6 ++--- docs/query-data/udf/alias-function.md | 4 .../current/query-data/complex-type.md | 4 +--- .../current/query-data/cte.md | 26 -- .../current/query-data/join.md | 14 +--- .../current/query-data/udf/alias-function.md | 4 .../current/query-data/window-function.md | 12 +- .../version-2.1/query-data/complex-type.md | 4 +--- .../version-2.1/query-data/cte.md | 26 -- .../version-2.1/query-data/join.md | 14 +--- .../version-2.1/query-data/udf/alias-function.md | 4 .../version-2.1/query-data/window-function.md | 13 ++- .../version-3.0/query-data/complex-type.md | 4 +--- .../version-3.0/query-data/cte.md | 26 -- .../version-3.0/query-data/join.md | 14 +--- .../version-3.0/query-data/udf/alias-function.md | 4 .../version-3.0/query-data/window-function.md | 12 +- .../version-2.1/query-data/complex-type.md | 5 + versioned_docs/version-2.1/query-data/cte.md | 26 -- versioned_docs/version-2.1/query-data/join.md | 22 -- versioned_docs/version-2.1/query-data/subquery.md | 6 ++--- .../version-2.1/query-data/udf/alias-function.md | 4 .../version-3.0/query-data/complex-type.md | 5 + versioned_docs/version-3.0/query-data/cte.md | 26 -- versioned_docs/version-3.0/query-data/join.md | 22 -- versioned_docs/version-3.0/query-data/subquery.md | 6 ++--- .../version-3.0/query-data/udf/alias-function.md | 4 30 files changed, 220 insertions(+), 149 deletions(-) diff --git a/docs/query-data/complex-type.md b/docs/query-data/complex-type.md index 92dd4862cf7..9db1b523fd8 100644 --- a/docs/query-data/complex-type.md +++ b/docs/query-data/complex-type.md @@ -28,6 +28,4 @@ Doris supports complex types such as Array, Map, Struct, and JSON. Doris provides various functions specifically designed for these complex types. -For detailed information on the supported functions, please refer to the SQL manual- [Array functions](../sql-manual/sql-functions/array-functions/array), [SQL manual-Struct functions](../sql-manual/sql-functions/struct-functions/struct), and [SQL manual-JSON functions](../sql-manual/sql-functions/json-functions/json-parse). - -To check the support for Map functions, you can refer to the SQL manual docs under the section for the [MAP](../sql-manual/sql-data-types/semi-structured/MAP) data type. +For detailed information on the supported functions, please refer to the [SQL manual - Array functions](../sql-manual/sql-functions/array-functions/array), [SQL manual - Map functions](../sql-manual/sql-data-types/semi-structured/MAP) data type, [SQL manual - Struct functions](../sql-manual/sql-functions/struct-functions/struct), and [SQL manual - Json functions](../sql-manual/sql-functions/json-functions/json-parse). diff --git a/docs/query-data/cte.md b/docs/query-data/cte.md index 941d74639a1..04299e4afec 100644 --- a/docs/query-data/cte.md +++ b/docs/query-data/cte.md @@ -30,12 +30,14 @@ Common Table Expression (CTE) define a temporary result set that can be referenc To specify a CTE, use the `WITH` clause with one or more comma-separated clauses. Each clause provides a subquery that generates a result set and associates a name with the subquery. -Within the statement that contains the `WITH` clause, you can reference each CTE name to access the corresponding CTE result set. CTE names can be referenced in other CTE, allowing you to define CTE based on other CTE. +Doris supports nested CTE。 Within the statement that contains the `WITH` clause, you can reference each CTE name to access the corresponding CTE result set.
Re: [PR] [polish](query-data) polish query data docs [doris-website]
KassieZ merged PR #1580: URL: https://github.com/apache/doris-website/pull/1580 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [regression-test](fix) two tests using same table (#45312) [doris]
hello-stephen merged PR #45867: URL: https://github.com/apache/doris/pull/45867 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](csv reader) fix core dump when parsing csv with enclose [doris]
doris-robot commented on PR #45485: URL: https://github.com/apache/doris/pull/45485#issuecomment-2560963426 ClickBench: Total hot run time: 32.47 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit a922acae5b0a7a32e4c569d4c06627f8f873f70e, data reload: false query1 0.030.030.03 query2 0.060.040.03 query3 0.240.070.06 query4 1.610.100.09 query5 0.430.390.41 query6 1.170.650.65 query7 0.030.020.01 query8 0.040.040.03 query9 0.560.520.49 query10 0.560.580.55 query11 0.140.110.11 query12 0.140.110.11 query13 0.620.600.60 query14 2.742.762.71 query15 0.900.830.82 query16 0.400.390.38 query17 1.041.020.98 query18 0.230.220.21 query19 1.881.831.89 query20 0.010.010.01 query21 15.38 0.620.58 query22 2.642.871.86 query23 16.95 0.960.89 query24 2.921.440.75 query25 0.270.350.05 query26 0.330.150.14 query27 0.040.040.05 query28 10.57 1.111.08 query29 12.82 3.283.24 query30 0.250.060.07 query31 2.850.380.39 query32 3.240.480.46 query33 3.063.183.17 query34 17.11 4.474.44 query35 4.504.484.47 query36 0.680.480.48 query37 0.090.050.06 query38 0.040.040.03 query39 0.030.020.03 query40 0.170.140.12 query41 0.080.030.02 query42 0.040.020.02 query43 0.030.030.03 Total cold run time: 106.92 s Total hot run time: 32.47 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) push down encode slot [doris]
doris-robot commented on PR #44748: URL: https://github.com/apache/doris/pull/44748#issuecomment-2560961984 ClickBench: Total hot run time: 32.69 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit e5add51746dcc7dae49ce8d88296ff3a68da6c66, data reload: false query1 0.040.030.03 query2 0.070.030.03 query3 0.240.070.08 query4 1.600.110.10 query5 0.410.410.42 query6 1.150.660.65 query7 0.020.020.02 query8 0.040.030.04 query9 0.570.510.50 query10 0.550.570.56 query11 0.150.110.11 query12 0.150.100.12 query13 0.600.600.59 query14 2.822.732.74 query15 0.890.820.81 query16 0.390.380.38 query17 1.041.041.05 query18 0.230.210.21 query19 1.961.782.01 query20 0.010.010.01 query21 15.37 0.590.58 query22 2.842.462.08 query23 16.86 1.040.80 query24 3.030.641.50 query25 0.120.310.06 query26 0.540.140.14 query27 0.040.040.04 query28 10.62 1.101.08 query29 12.57 3.283.28 query30 0.250.070.06 query31 2.860.380.38 query32 3.280.470.46 query33 3.053.143.09 query34 16.94 4.514.52 query35 4.554.574.53 query36 0.670.480.49 query37 0.100.060.06 query38 0.050.040.04 query39 0.040.030.03 query40 0.180.120.12 query41 0.080.030.02 query42 0.030.020.02 query43 0.040.030.03 Total cold run time: 107.04 s Total hot run time: 32.69 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch asf-site updated (2fe4a238e4e -> ec93b7dfded)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/doris-website.git discard 2fe4a238e4e Automated deployment with doris branch master@ a9dfa175230d7bc083d278e8ab69693f1dc68186 new ec93b7dfded Automated deployment with doris branch @ a9dfa175230d7bc083d278e8ab69693f1dc68186 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (2fe4a238e4e) \ N -- N -- N refs/heads/asf-site (ec93b7dfded) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: docs/1.2/search-index.json |2 +- docs/2.0/search-index.json |2 +- docs/3.0/search-index.json |2 +- docs/dev/search-index.json |2 +- search-index.json|2 +- worker.js| 4898 -- zh-CN/docs/1.2/search-index.json |2 +- zh-CN/docs/2.0/search-index.json |2 +- zh-CN/docs/3.0/search-index.json |2 +- zh-CN/docs/dev/search-index.json |2 +- zh-CN/search-index.json |2 +- 11 files changed, 10 insertions(+), 4908 deletions(-) delete mode 100644 worker.js - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](metrics) Remove IntAtomicCounter & CoreLocal #45742 [doris]
zhiqiang- commented on PR #45870: URL: https://github.com/apache/doris/pull/45870#issuecomment-2560953127 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](tvf) fix azure tvf: can not build s3() [doris]
kaijchen commented on PR #45872: URL: https://github.com/apache/doris/pull/45872#issuecomment-2560959499 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [case](auth)Add case for auth #45478 [doris]
zddr commented on PR #45819: URL: https://github.com/apache/doris/pull/45819#issuecomment-2560959874 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](csv reader) fix core dump when parsing csv with enclose [doris]
doris-robot commented on PR #45485: URL: https://github.com/apache/doris/pull/45485#issuecomment-2560957766 TPC-DS: Total hot run time: 190626 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit a922acae5b0a7a32e4c569d4c06627f8f873f70e, data reload: false query1 964 372 374 372 query2 6523241624392416 query3 6703215 215 215 query4 33831 23361 23841 23361 query5 4743481 468 468 query6 284 184 197 184 query7 4629302 296 296 query8 295 226 219 219 query9 9834275427552754 query10 484 266 250 250 query11 18216 15176 15223 15176 query12 153 103 99 99 query13 1649415 402 402 query14 9529779070567056 query15 311 175 182 175 query16 7459425 459 425 query17 1673579 548 548 query18 1941297 297 297 query19 374 150 147 147 query20 117 107 110 107 query21 207 105 102 102 query22 4446419441914191 query23 34264 33365 33540 33365 query24 11469 250524462446 query25 715 419 414 414 query26 1867157 166 157 query27 2938329 329 329 query28 8054247324562456 query29 982 431 438 431 query30 309 152 150 150 query31 1062800 819 800 query32 104 59 58 58 query33 791 307 299 299 query34 981 521 557 521 query35 858 761 739 739 query36 1092960 949 949 query37 235 73 71 71 query38 4133430641514151 query39 1483143914271427 query40 289 99 98 98 query41 50 45 49 45 query42 118 100 98 98 query43 523 496 480 480 query44 1241809 805 805 query45 186 168 159 159 query46 1163693 696 693 query47 1952184218741842 query48 409 328 314 314 query49 1280392 404 392 query50 812 378 379 378 query51 7209708370467046 query52 100 93 89 89 query53 257 187 182 182 query54 1259414 412 412 query55 83 77 77 77 query56 254 264 231 231 query57 1293111311451113 query58 230 218 222 218 query59 3190313030933093 query60 274 245 248 245 query61 110 114 111 111 query62 946 738 729 729 query63 218 184 196 184 query64 5131675 639 639 query65 3321320532203205 query66 1414314 310 310 query67 15915 15495 15572 15495 query68 5816554 554 554 query69 445 249 261 249 query70 1211113611491136 query71 430 244 247 244 query72 6403410340474047 query73 773 369 357 357 query74 10681 925888848884 query75 3483264926832649 query76 3552124110671067 query77 569 273 270 270 query78 10289 938392919291 query79 2534622 617 617 query80 1078479 439 439 query81 531 228 232 228 query82 838 124 118 118 query83 245 159 147 147 query84 231 71 71 71 query85 1438327 310 310 query86 465 298 297 297 query87 4524441745144417 query88 415521892189 query89 412 297 299 297 query90 2139188 189 188 query91 137 101 116 101 query92 66 50 56 50 query93 1492555 561 555 query94 1051281 288 281 query95 350 250 250 250 query96 618 277 285 277 query97 2862269327002693 query98 215 201 201 201 query99 1692144614501446 Total cold run time: 305654 ms Total hot run time: 190626 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t
Re: [PR] [fix](parser) Syntax error for add partition with null null [doris]
doris-robot commented on PR #45865: URL: https://github.com/apache/doris/pull/45865#issuecomment-2560964107 TPC-H: Total hot run time: 39945 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 8267636b46bcaa2f1c5b2bfb3d05d23dd8563385, data reload: false -- Round 1 -- q1 17582 748672917291 q2 2047176 165 165 q3 10592 110112261101 q4 10588 803 839 803 q5 7619268726992687 q6 235 151 151 151 q7 1001659 601 601 q8 9226185519431855 q9 6599637364336373 q10 7000233623182318 q11 462 261 257 257 q12 412 228 218 218 q13 17813 291729132913 q14 241 209 207 207 q15 562 506 497 497 q16 657 586 601 586 q17 968 532 557 532 q18 7189670967586709 q19 1352966 928 928 q20 468 186 187 186 q21 4020325633193256 q22 363 325 311 311 Total cold run time: 106996 ms Total hot run time: 39945 ms - Round 2, with runtime_filter_mode=off - q1 7298727272117211 q2 331 233 231 231 q3 2886274629462746 q4 2044183217831783 q5 5652569356285628 q6 225 142 143 142 q7 2238179318421793 q8 3394358235863582 q9 8926901390009000 q10 3632357536083575 q11 609 509 538 509 q12 812 583 586 583 q13 13130 314630673067 q14 324 292 290 290 q15 552 504 501 501 q16 671 651 668 651 q17 1861165016201620 q18 8334776978037769 q19 1763159314531453 q20 2094181918161816 q21 5541541454375414 q22 669 591 608 591 Total cold run time: 72986 ms Total hot run time: 59955 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](parser) Syntax error for add partition with null null [doris]
wyxxxcat commented on PR #45865: URL: https://github.com/apache/doris/pull/45865#issuecomment-2560965844 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[I] [Bug] dbt-doris extention does not execute post_hook [doris]
SmiMi opened a new issue, #45873:
URL: https://github.com/apache/doris/issues/45873
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no
similar issues.
### Version
- dbt-core version: 1.5.10 (Latest: 1.7.9 - Update available!)
- dbt-doris plugin version: 0.3.4
- Doris version: doris-2.1.5-rc02
- Python version: 3.9.13
### What's Wrong?
During my modeling process with dbt-doris, I found that the SQL statements
in the post-hook are not taking effect. I tried to add comments to the table
fields in the post-hook, but after the model execution is complete, the field
comments are still empty.
Below are my YML and SQL files:
```
---
version: 2
models:
- name: "dim_test_after"
description: ""
config:
contract:
enforced: false
columns:
- name: "customer_code"
description: ""
data_type: "varchar"
constraints: []
- name: "id"
description: ""
data_type: "varchar"
constraints: []
- name: "customer_name"
description: ""
data_type: "varchar"
constraints: []
```
```
{{config(
post_hook=[
"ALTER TABLE dim_test_after
MODIFY COLUMN customer_code COMMENT 'customer code',
MODIFY COLUMN id COMMENT 'customer id',
MODIFY COLUMN customer_name COMMENT 'customer name'"],
materialized='table',
)
}}
with middle_steo as (
select 1)
select customer_code, id,customer_name from dim_customer
```
Below is the log of the successful execution.
```
{
"exitCode": 0,
"output": "10:10:51 Running with dbt=1.5.10
10:10:52 Registered adapter: doris=0.3.4
10:10:52 Found 8 models, 0 tests, 0 snapshots, 0 analyses, 328 macros, 0
operations, 0 seed files, 19 sources, 0 exposures, 0 metrics, 0 groups
10:10:52
10:10:52 Concurrency: 4 threads (target='fat')
10:10:52
10:10:52 1 of 1 START sql table model bigdata.dim_test_after
[RUN]
10:10:52 1 of 1 OK created sql table model bigdata.dim_test_after
... [351 rows affected in 0.65s]
10:10:53
10:10:53 Finished running 1 table model in 0 hours 0 minutes and 0.83
seconds (0.83s).
10:10:53
10:10:53 Completed successfully
10:10:53
10:10:53 Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1
Command exited with return code 0"
}
```
But even though the execution was successful, the ALTER TABLE statement did
not take effect.
### What You Expected?
I expect the post-hook to take effect.
### How to Reproduce?
_No response_
### Anything Else?
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](metrics) Remove IntAtomicCounter & CoreLocal #45742 [doris]
zhiqiang- commented on PR #45870: URL: https://github.com/apache/doris/pull/45870#issuecomment-2560980927 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](parser) Syntax error for add partition with null null [doris]
doris-robot commented on PR #45865: URL: https://github.com/apache/doris/pull/45865#issuecomment-2560982051 ClickBench: Total hot run time: 33.52 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 8267636b46bcaa2f1c5b2bfb3d05d23dd8563385, data reload: false query1 0.030.030.03 query2 0.080.040.04 query3 0.240.070.07 query4 1.600.110.10 query5 0.420.430.41 query6 1.150.650.65 query7 0.030.020.02 query8 0.040.030.03 query9 0.580.510.50 query10 0.560.580.55 query11 0.160.110.11 query12 0.140.110.11 query13 0.610.600.61 query14 2.852.862.75 query15 0.900.830.82 query16 0.390.380.39 query17 1.001.061.08 query18 0.230.210.23 query19 1.911.851.95 query20 0.020.010.01 query21 15.37 0.580.58 query22 2.582.282.47 query23 17.37 0.930.91 query24 2.731.031.18 query25 0.220.140.18 query26 0.330.140.14 query27 0.030.040.05 query28 10.58 1.101.08 query29 12.54 3.283.27 query30 0.240.060.06 query31 2.860.390.38 query32 3.230.470.45 query33 3.133.073.04 query34 16.97 4.534.53 query35 4.564.574.50 query36 0.680.500.48 query37 0.100.060.06 query38 0.040.030.03 query39 0.030.020.02 query40 0.170.120.12 query41 0.070.030.03 query42 0.030.020.02 query43 0.030.030.03 Total cold run time: 106.83 s Total hot run time: 33.52 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Update JSON.md [doris-website]
KassieZ commented on code in PR #1576:
URL: https://github.com/apache/doris-website/pull/1576#discussion_r1896646929
##
docs/sql-manual/sql-data-types/semi-structured/JSON.md:
##
@@ -56,6 +56,8 @@ INSERT INTO table_name(id, json_column_name) VALUES (1,
'{"k1": "100"}')
14 [123, 456]
```
+- When the newline character "\n" appears in JSON, you need to use the replace
function to replace "\n" with "\\n"
Review Comment:
\n 使用行内代码包裹
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](tvf) fix azure tvf: can not build s3() [doris]
doris-robot commented on PR #45872: URL: https://github.com/apache/doris/pull/45872#issuecomment-2560984148 TPC-H: Total hot run time: 39978 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 364e0a1688f58d5c72100283e29ffdff7048459b, data reload: false -- Round 1 -- q1 17602 748272957295 q2 2051185 175 175 q3 10537 112811911128 q4 10240 744 740 740 q5 7600269126642664 q6 245 149 149 149 q7 992 622 611 611 q8 9235182418961824 q9 6691646763956395 q10 6995231723692317 q11 460 258 260 258 q12 407 224 224 224 q13 17770 297729062906 q14 248 220 209 209 q15 533 506 496 496 q16 690 607 579 579 q17 972 631 666 631 q18 7407667466886674 q19 1342991 1010991 q20 491 192 185 185 q21 4093322633013226 q22 377 301 311 301 Total cold run time: 106978 ms Total hot run time: 39978 ms - Round 2, with runtime_filter_mode=off - q1 7251725572367236 q2 329 228 239 228 q3 2939273429972734 q4 2132181519121815 q5 5628560556235605 q6 227 139 139 139 q7 2167178517581758 q8 3368349335003493 q9 8865887189528871 q10 3671358035763576 q11 610 499 499 499 q12 788 598 602 598 q13 13297 314730133013 q14 285 272 271 271 q15 552 499 534 499 q16 699 658 653 653 q17 1846159515761576 q18 7842747274887472 q19 1682150114651465 q20 2061179218571792 q21 5343528853265288 q22 657 580 563 563 Total cold run time: 72239 ms Total hot run time: 59144 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](nereids)support alter table command [doris]
doris-robot commented on PR #44058: URL: https://github.com/apache/doris/pull/44058#issuecomment-2560835636 ClickBench: Total hot run time: 32.65 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit e7230a1602ac8219c13335287a8443e734890061, data reload: false query1 0.030.040.05 query2 0.070.030.03 query3 0.230.070.07 query4 1.620.100.11 query5 0.420.410.42 query6 1.140.670.65 query7 0.020.010.02 query8 0.040.040.03 query9 0.580.510.50 query10 0.550.550.56 query11 0.140.100.11 query12 0.140.110.10 query13 0.620.620.60 query14 2.852.832.79 query15 0.870.840.82 query16 0.370.380.39 query17 1.061.061.08 query18 0.220.210.20 query19 1.911.871.97 query20 0.020.020.01 query21 15.36 0.600.56 query22 2.412.411.70 query23 17.19 0.860.86 query24 3.300.831.85 query25 0.190.200.22 query26 0.570.140.13 query27 0.040.040.04 query28 9.941.121.08 query29 12.54 3.273.28 query30 0.250.070.06 query31 2.850.390.38 query32 3.250.480.45 query33 3.083.053.13 query34 16.73 4.464.52 query35 4.464.504.47 query36 0.700.480.48 query37 0.100.060.06 query38 0.050.040.04 query39 0.040.020.02 query40 0.170.120.12 query41 0.080.030.02 query42 0.030.020.02 query43 0.040.030.03 Total cold run time: 106.27 s Total hot run time: 32.65 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [ci](perf) fix conf [doris]
hello-stephen commented on PR #45847: URL: https://github.com/apache/doris/pull/45847#issuecomment-2560834983 skip buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] regexp_extract_all execute empty string will make be shutdown [doris]
ScalaFirst closed issue #43357: [Bug] regexp_extract_all execute empty string will make be shutdown URL: https://github.com/apache/doris/issues/43357 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] regexp_extract_all execute empty string will make be shutdown [doris]
ScalaFirst commented on issue #43357: URL: https://github.com/apache/doris/issues/43357#issuecomment-2560837004 > this issue has been fixed, you can upgrade your doris to 2.0.15 version or 2.1.7 has anyway to fix with version 1.2.x ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cloud-mow) FE should release delete bitmap lock when calculating delete bitmap failed [doris]
hust-hhb commented on PR #45673: URL: https://github.com/apache/doris/pull/45673#issuecomment-2560837597 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](doc) add job-scheduler example in doris's doc to regression test [doris]
yagagagaga commented on PR #42904: URL: https://github.com/apache/doris/pull/42904#issuecomment-2560838810 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Outfile) optimize the error message of `SELECT INTO OUTFILE` [doris]
doris-robot commented on PR #45859: URL: https://github.com/apache/doris/pull/45859#issuecomment-2560837181 TPC-H: Total hot run time: 39760 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 3e2b022f51929a2808145730d1f721739fb1d424, data reload: false -- Round 1 -- q1 17591 741072417241 q2 2045181 179 179 q3 10767 107711551077 q4 10586 723 715 715 q5 7593266226082608 q6 242 155 154 154 q7 985 635 617 617 q8 9238187118921871 q9 6670640264566402 q10 7013233823322332 q11 464 259 260 259 q12 410 218 222 218 q13 17794 290729682907 q14 247 208 213 208 q15 568 524 509 509 q16 663 604 587 587 q17 968 557 614 557 q18 7300681865896589 q19 13931021978 978 q20 476 188 189 188 q21 4277327332553255 q22 374 323 309 309 Total cold run time: 107664 ms Total hot run time: 39760 ms - Round 2, with runtime_filter_mode=off - q1 7209717572317175 q2 327 234 228 228 q3 2902282829452828 q4 2043175217841752 q5 5680564856505648 q6 223 142 141 141 q7 2284179018471790 q8 3351350235323502 q9 9005897089498949 q10 3624356135773561 q11 605 516 519 516 q12 800 618 613 613 q13 12770 310931563109 q14 326 290 275 275 q15 550 514 503 503 q16 697 651 655 651 q17 1835164516181618 q18 8422797176057605 q19 3447145414581454 q20 2057189518581858 q21 5577542654495426 q22 664 592 595 592 Total cold run time: 74398 ms Total hot run time: 59794 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](information_schema)Show view definition in information_schema.views. [doris]
doris-robot commented on PR #45857: URL: https://github.com/apache/doris/pull/45857#issuecomment-2560838480 TPC-DS: Total hot run time: 191341 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 9d8d5eb726083556b7e2ec219cab5dd2611181aa, data reload: false query1 866 393 378 378 query2 3731248425252484 query3 6590215 210 210 query4 33697 23658 23781 23658 query5 2547477 474 474 query6 283 184 182 182 query7 4588295 301 295 query8 315 240 233 233 query9 9607276127642761 query10 467 257 246 246 query11 18645 15057 15017 15017 query12 157 99 106 99 query13 1686423 399 399 query14 10671 768475947594 query15 272 182 190 182 query16 7756466 457 457 query17 1911593 583 583 query18 2023307 322 307 query19 354 151 143 143 query20 111 104 109 104 query21 208 100 100 100 query22 4543443443324332 query23 34753 33865 8 8 query24 10995 255523522352 query25 614 367 380 367 query26 1342150 152 150 query27 2868330 337 330 query28 8008244224852442 query29 880 395 401 395 query30 298 151 145 145 query31 1014827 819 819 query32 105 62 58 58 query33 787 287 284 284 query34 1121536 509 509 query35 887 730 729 729 query36 1087916 939 916 query37 211 75 72 72 query38 4319404140554041 query39 1603144614401440 query40 252 102 108 102 query41 46 47 43 43 query42 114 101 101 101 query43 526 494 482 482 query44 1202795 800 795 query45 180 160 170 160 query46 1166695 716 695 query47 1986186618561856 query48 416 320 321 320 query49 1186386 377 377 query50 802 386 378 378 query51 7158697970436979 query52 108 97 98 97 query53 256 184 184 184 query54 1274400 407 400 query55 80 79 81 79 query56 251 237 239 237 query57 1253112411091109 query58 230 221 231 221 query59 3267296430322964 query60 274 246 266 246 query61 112 104 103 103 query62 933 725 737 725 query63 220 190 189 189 query64 4358674 645 645 query65 3217319132213191 query66 1270307 310 307 query67 15797 15560 1 1 query68 5797570 567 567 query69 427 251 249 249 query70 1204116111401140 query71 404 251 252 251 query72 6496408141594081 query73 770 358 356 356 query74 9966901590219015 query75 3416264426482644 query76 3620117811461146 query77 552 276 272 272 query78 10245 960094769476 query79 2218621 603 603 query80 917 424 416 416 query81 544 246 228 228 query82 840 120 128 120 query83 179 148 143 143 query84 259 71 67 67 query85 1191306 303 303 query86 401 296 289 289 query87 4681431045644310 query88 4053219922012199 query89 413 298 303 298 query90 2235196 186 186 query91 133 101 103 101 query92 68 56 50 50 query93 1772565 551 551 query94 1129288 290 288 query95 349 248 248 248 query96 610 275 277 275 query97 2882270426762676 query98 224 202 198 198 query99 1687145214301430 Total cold run time: 299536 ms Total hot run time: 191341 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [ci](perf) fix conf [doris]
hello-stephen merged PR #45847: URL: https://github.com/apache/doris/pull/45847 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-3.0 updated (a5791833c91 -> aeeb4520938)
This is an automated email from the ASF dual-hosted git repository. hellostephen pushed a change to branch branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git from a5791833c91 branch-3.0: [fix](nereids) fix bug when sum0 distinct appear in cte #45447 (#45510) add aeeb4520938 [ci](perf) fix conf (#45847) No new revisions were added by this update. Summary of changes: regression-test/pipeline/performance/run-clickbench.sh | 4 regression-test/pipeline/performance/run-load.sh | 4 2 files changed, 8 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] memtable flush reserve process mem and improve logs [doris]
yiguolei merged PR #45743: URL: https://github.com/apache/doris/pull/45743 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [test](nereids) add test simplify comparison predicate #44886 [doris]
doris-robot commented on PR #45803: URL: https://github.com/apache/doris/pull/45803#issuecomment-2560842887 TPC-DS: Total hot run time: 190224 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit e5d5b35a39387a6b732fc374e10d2c941d5967da, data reload: false query1 969 365 367 365 query2 6512209020922090 query3 6710213 223 213 query4 33708 23302 23492 23302 query5 4318448 448 448 query6 273 178 170 170 query7 4629308 321 308 query8 291 229 225 225 query9 9564269026692669 query10 494 261 264 261 query11 17909 14981 15051 14981 query12 146 102 103 102 query13 1636426 410 410 query14 9290739069096909 query15 225 185 172 172 query16 7912489 510 489 query17 1620576 553 553 query18 2036309 320 309 query19 219 156 157 156 query20 115 108 110 108 query21 61 47 45 45 query22 4570432744034327 query23 34420 33737 33745 33737 query24 12349 287229002872 query25 716 413 402 402 query26 1847173 169 169 query27 2981301 303 301 query28 8314245224282428 query29 1109458 436 436 query30 341 172 165 165 query31 1089792 809 792 query32 98 58 57 57 query33 778 288 297 288 query34 981 491 519 491 query35 853 711 720 711 query36 1090932 959 932 query37 271 73 76 73 query38 3982385638503850 query39 1587141614281416 query40 229 85 84 84 query41 55 53 50 50 query42 112 99 100 99 query43 539 499 494 494 query44 1282792 787 787 query45 186 170 171 170 query46 1150722 715 715 query47 1869180818231808 query48 454 384 361 361 query49 1297401 399 399 query50 795 422 412 412 query51 7104704969846984 query52 101 97 95 95 query53 274 199 188 188 query54 1272467 458 458 query55 75 79 75 75 query56 272 268 258 258 query57 1181110411031103 query58 245 216 235 216 query59 3186300030983000 query60 299 248 247 247 query61 109 105 107 105 query62 867 654 684 654 query63 212 185 182 182 query64 5352636 630 630 query65 3291317931613161 query66 1428302 326 302 query67 15876 15400 15449 15400 query68 5604582 552 552 query69 425 268 256 256 query70 1170112311331123 query71 346 258 250 250 query72 6560389839333898 query73 762 343 336 336 query74 10272 888088858880 query75 3348265626012601 query76 3502106611351066 query77 407 275 270 270 query78 10660 979494739473 query79 8915584 582 582 query80 2178423 414 414 query81 564 231 233 231 query82 1404117 115 115 query83 258 141 142 141 query84 303 77 78 77 query85 2302298 293 293 query86 500 306 299 299 query87 4518432441894189 query88 5919236923872369 query89 560 300 293 293 query90 2291188 182 182 query91 175 141 142 141 query92 68 48 48 48 query93 6745555 528 528 query94 1071298 269 269 query95 349 249 251 249 query96 625 278 278 278 query97 3323317031743170 query98 217 201 193 193 query99 1599132612891289 Total cold run time: 322065 ms Total hot run time: 190224 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t
Re: [PR] [improvement](information_schema)Show view definition in information_schema.views. [doris]
doris-robot commented on PR #45857: URL: https://github.com/apache/doris/pull/45857#issuecomment-2560844366 ClickBench: Total hot run time: 32.05 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 9d8d5eb726083556b7e2ec219cab5dd2611181aa, data reload: false query1 0.030.030.03 query2 0.070.020.04 query3 0.240.080.07 query4 1.600.100.11 query5 0.440.410.41 query6 1.170.650.65 query7 0.030.020.02 query8 0.040.030.03 query9 0.560.530.51 query10 0.550.570.56 query11 0.150.100.10 query12 0.140.110.12 query13 0.620.590.60 query14 2.712.742.87 query15 0.890.810.84 query16 0.380.370.38 query17 1.051.060.99 query18 0.220.200.20 query19 1.991.892.00 query20 0.020.000.01 query21 15.38 0.590.60 query22 2.743.311.79 query23 17.04 1.080.83 query24 2.690.561.68 query25 0.210.070.17 query26 0.420.140.15 query27 0.030.050.04 query28 10.67 1.101.07 query29 12.59 3.253.22 query30 0.250.070.06 query31 2.840.380.39 query32 3.250.460.45 query33 3.063.043.06 query34 17.13 4.414.46 query35 4.484.424.52 query36 0.670.480.48 query37 0.100.060.05 query38 0.040.030.03 query39 0.020.020.02 query40 0.160.130.14 query41 0.070.020.02 query42 0.030.020.02 query43 0.030.030.03 Total cold run time: 106.8 s Total hot run time: 32.05 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Support for global dictionary functions. [doris]
doris-robot commented on PR #45536: URL: https://github.com/apache/doris/pull/45536#issuecomment-2560845804 TPC-H: Total hot run time: 39789 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit b1f1e9699bd72af34c776fe77d88614a232ac11e, data reload: false -- Round 1 -- q1 17581 745972507250 q2 2057176 171 171 q3 10617 107212101072 q4 10543 708 768 708 q5 7605275427112711 q6 245 152 156 152 q7 981 635 625 625 q8 9243182619021826 q9 6712640364556403 q10 7034227223542272 q11 465 269 259 259 q12 445 230 231 230 q13 17797 295530422955 q14 247 226 231 226 q15 558 506 504 504 q16 669 575 600 575 q17 974 507 553 507 q18 7346665168006651 q19 1353978 903 903 q20 477 194 192 192 q21 4066335032933293 q22 376 319 304 304 Total cold run time: 107391 ms Total hot run time: 39789 ms - Round 2, with runtime_filter_mode=off - q1 7320726672127212 q2 329 224 238 224 q3 2930281629792816 q4 2132187318911873 q5 5661569556065606 q6 234 148 151 148 q7 2276187118321832 q8 3437354735123512 q9 8805902389808980 q10 3646361535833583 q11 593 502 499 499 q12 834 625 606 606 q13 11142 319831713171 q14 304 267 273 267 q15 569 503 505 503 q16 702 648 647 647 q17 1862162716171617 q18 8376790277737773 q19 3515151715181517 q20 2053193518621862 q21 5659554355495543 q22 671 569 596 569 Total cold run time: 73050 ms Total hot run time: 60360 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](join) Fuzzy disable runtime filters in BE [doris]
doris-robot commented on PR #45654: URL: https://github.com/apache/doris/pull/45654#issuecomment-2560846089 TPC-H: Total hot run time: 39988 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit ac680479658ecde4be168926ce79f09efed70d0d, data reload: false -- Round 1 -- q1 17629 747472667266 q2 2048188 186 186 q3 10517 113911571139 q4 10226 729 745 729 q5 7611270027512700 q6 241 147 148 147 q7 993 625 616 616 q8 9248187619351876 q9 6675648464096409 q10 6981234423742344 q11 451 265 260 260 q12 420 221 224 221 q13 17772 299429542954 q14 251 213 218 213 q15 548 515 483 483 q16 670 580 582 580 q17 991 614 534 534 q18 7383665566166616 q19 1368102310031003 q20 471 190 189 189 q21 4142321732543217 q22 374 306 320 306 Total cold run time: 107010 ms Total hot run time: 39988 ms - Round 2, with runtime_filter_mode=off - q1 7257719372547193 q2 331 234 230 230 q3 2914275027892750 q4 1965176917111711 q5 5400541354305413 q6 225 140 139 139 q7 2123173517181718 q8 3265342334173417 q9 8600866485798579 q10 3529345234423442 q11 591 491 493 491 q12 761 575 576 575 q13 12188 296029342934 q14 298 265 276 265 q15 546 495 485 485 q16 676 644 644 644 q17 1801158415671567 q18 7912753474187418 q19 1692156815611561 q20 2075182818731828 q21 5419539652935293 q22 635 593 592 592 Total cold run time: 70203 ms Total hot run time: 58245 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](information_schema)Show view definition in information_schema.views. [doris]
doris-robot commented on PR #45857: URL: https://github.com/apache/doris/pull/45857#issuecomment-2560846076 TeamCity be ut coverage result: Function Coverage: 38.80% (10093/26015) Line Coverage: 29.77% (85111/285869) Region Coverage: 28.90% (43462/150406) Branch Coverage: 25.43% (22152/87118) Coverage Report: http://coverage.selectdb-in.cc/coverage/9d8d5eb726083556b7e2ec219cab5dd2611181aa_9d8d5eb726083556b7e2ec219cab5dd2611181aa/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [enhancement](blacklist) ignore shutdown message to avoid add the backend to blacklist [doris]
yiguolei opened a new pull request, #45875: URL: https://github.com/apache/doris/pull/45875 ### What problem does this PR solve? Issue Number: close #xxx Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason - Behavior changed: - [ ] No. - [ ] Yes. - Does this need documentation? - [ ] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](blacklist) ignore shutdown message to avoid add the backend to blacklist [doris]
hello-stephen commented on PR #45875: URL: https://github.com/apache/doris/pull/45875#issuecomment-2560988773 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](memory) Storage page cache use LRU-K cache, K=2 [doris]
yiguolei merged PR #45719: URL: https://github.com/apache/doris/pull/45719 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [improvement](memory) Storage page cache use LRU-K cache, K=2 (#45719)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 4e6197ca60d [improvement](memory) Storage page cache use LRU-K cache,
K=2 (#45719)
4e6197ca60d is described below
commit 4e6197ca60de0ea07e9cdf597dce68008bfad3f0
Author: Xinyi Zou
AuthorDate: Tue Dec 24 19:02:14 2024 +0800
[improvement](memory) Storage page cache use LRU-K cache, K=2 (#45719)
### What problem does this PR solve?
Storage page cache uses plain LRU Cache, occasional batch operations can
cause "cache pollution" in plain LRU Cache. This will cause hotspot data
to be squeezed out of the cache by non-hotspot data, reduce cache hit
rate.
In extreme cases, if the number of pages inserted each time is greater
than the cache capacity, the cache hit rate will be 0.
Introducing LRU-K Cache avoids "cache pollution" in most cases.
Last submitted #28794
Similar PR: https://github.com/apache/doris/pull/23546
Performance test on branch-2.1:
1-FE, 3-BE (32 Core, 128GB Memory) Doris cluster
`ssb_sf_1000` all query cost: 18.85s -> 14.44s, 23% improvement
`ssb_flat_1000` all query cost: 12.21s -> 8.54s, 30% improvement, cache
hit rate 1% -> 40%
---
be/src/olap/lru_cache.cpp| 104 +++--
be/src/olap/lru_cache.h | 46 ++---
be/src/olap/page_cache.h | 3 +-
be/src/runtime/memory/lru_cache_policy.h | 9 +--
be/test/olap/lru_cache_test.cpp | 108 +--
be/test/olap/page_cache_test.cpp | 29 -
6 files changed, 240 insertions(+), 59 deletions(-)
diff --git a/be/src/olap/lru_cache.cpp b/be/src/olap/lru_cache.cpp
index 9895a013894..bd92142c312 100644
--- a/be/src/olap/lru_cache.cpp
+++ b/be/src/olap/lru_cache.cpp
@@ -167,7 +167,7 @@ uint32_t HandleTable::element_count() const {
return _elems;
}
-LRUCache::LRUCache(LRUCacheType type) : _type(type) {
+LRUCache::LRUCache(LRUCacheType type, bool is_lru_k) : _type(type),
_is_lru_k(is_lru_k) {
// Make empty circular linked list
_lru_normal.next = &_lru_normal;
_lru_normal.prev = &_lru_normal;
@@ -305,6 +305,17 @@ Cache::Handle* LRUCache::lookup(const CacheKey& key,
uint32_t hash) {
} else {
++_miss_count;
}
+
+// If key not exist in cache, and is lru k cache, and key in visits list,
+// then move the key to beginning of the visits list.
+// key in visits list indicates that the key has been inserted once after
the cache is full.
+if (e == nullptr && _is_lru_k) {
+auto it = _visits_lru_cache_map.find(hash);
+if (it != _visits_lru_cache_map.end()) {
+_visits_lru_cache_list.splice(_visits_lru_cache_list.begin(),
_visits_lru_cache_list,
+ it->second);
+}
+}
return reinterpret_cast(e);
}
@@ -316,10 +327,10 @@ void LRUCache::release(Cache::Handle* handle) {
bool last_ref = false;
{
std::lock_guard l(_mutex);
+// if last_ref is true, key may have been evict from the cache,
+// or if it is lru k, first insert of key may have failed.
last_ref = _unref(e);
-if (last_ref) {
-_usage -= e->total_size;
-} else if (e->in_cache && e->refs == 1) {
+if (e->in_cache && e->refs == 1) {
// only exists in cache
if (_usage > _capacity) {
// take this opportunity and remove the item
@@ -327,6 +338,8 @@ void LRUCache::release(Cache::Handle* handle) {
DCHECK(removed);
e->in_cache = false;
_unref(e);
+// `entry->in_cache = false` and `_usage -=
entry->total_size;` and `_unref(entry)` should appear together.
+// see the comment for old entry in `LRUCache::insert`.
_usage -= e->total_size;
last_ref = true;
} else {
@@ -401,6 +414,8 @@ void LRUCache::_evict_one_entry(LRUHandle* e) {
DCHECK(removed);
e->in_cache = false;
_unref(e);
+// `entry->in_cache = false` and `_usage -= entry->total_size;` and
`_unref(entry)` should appear together.
+// see the comment for old entry in `LRUCache::insert`.
_usage -= e->total_size;
}
@@ -408,6 +423,42 @@ bool LRUCache::_check_element_count_limit() {
return _element_count_capacity != 0 && _table.element_count() >=
_element_count_capacity;
}
+// After cache is full,
+// 1.Return false. If key has been inserted into the visits list before,
+// key is allowed to be inserted into cache this time (this will trigger cache
evict),
+// and key is removed from the visits list.
+// 2. Return true. If key not in visits list, insert it into visits list.
+bool LRUC
Re: [PR] [enhancement](blacklist) ignore shutdown message to avoid add the backend to blacklist [doris]
yiguolei commented on PR #45875: URL: https://github.com/apache/doris/pull/45875#issuecomment-2560992289 skip buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](auth)Add the default configuration file for ranger under `fe/conf` [doris]
doris-robot commented on PR #45869: URL: https://github.com/apache/doris/pull/45869#issuecomment-2560990405 TPC-H: Total hot run time: 39972 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 67468ef0153dbf2589860f2bb5ccfac68b90ca38, data reload: false -- Round 1 -- q1 17654 741673047304 q2 2045189 170 170 q3 10599 109611561096 q4 10578 717 759 717 q5 7613273926972697 q6 240 147 146 146 q7 991 610 595 595 q8 9410189919651899 q9 6681647165556471 q10 6958230723782307 q11 458 255 256 255 q12 442 220 224 220 q13 17750 294029212921 q14 244 215 217 215 q15 568 497 513 497 q16 644 585 576 576 q17 982 545 585 545 q18 7184680366306630 q19 1347103410081008 q20 496 190 188 188 q21 3994320732893207 q22 370 308 313 308 Total cold run time: 107248 ms Total hot run time: 39972 ms - Round 2, with runtime_filter_mode=off - q1 7235718171827181 q2 354 230 227 227 q3 2901278429642784 q4 2050180917831783 q5 5632563256325632 q6 229 143 140 140 q7 2252179018081790 q8 3359353834753475 q9 8926894089058905 q10 3603356335993563 q11 610 502 510 502 q12 822 599 600 599 q13 10964 308231463082 q14 328 275 271 271 q15 553 519 506 506 q16 697 652 636 636 q17 1860163016221622 q18 8394775477817754 q19 2913160414461446 q20 2042188619001886 q21 5530545553555355 q22 657 576 616 576 Total cold run time: 71911 ms Total hot run time: 59715 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement adminSetPartitionVersionCommand in nereids [doris]
doris-robot commented on PR #45426: URL: https://github.com/apache/doris/pull/45426#issuecomment-2560990041 TPC-DS: Total hot run time: 190454 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 46fe09cb87ea4527f868e0e359d81bfe14523864, data reload: false query1 966 396 379 379 query2 6554248724032403 query3 6721227 223 223 query4 33866 23364 23903 23364 query5 5001479 446 446 query6 274 176 184 176 query7 4640299 311 299 query8 312 224 222 222 query9 9857273527302730 query10 513 268 244 244 query11 18158 15257 15142 15142 query12 165 107 106 106 query13 1675422 433 422 query14 10520 760672777277 query15 290 171 196 171 query16 8163479 445 445 query17 1717573 544 544 query18 2152290 297 290 query19 381 164 152 152 query20 117 107 111 107 query21 221 102 106 102 query22 4580414741554147 query23 34326 34784 33355 33355 query24 11474 248325052483 query25 574 374 404 374 query26 1272153 148 148 query27 2789320 325 320 query28 7839245324892453 query29 669 405 401 401 query30 302 147 148 147 query31 1051820 837 820 query32 99 56 61 56 query33 780 297 299 297 query34 960 505 506 505 query35 886 739 748 739 query36 1084945 965 945 query37 212 73 80 73 query38 4180422540684068 query39 1498145914211421 query40 290 109 108 108 query41 49 46 49 46 query42 128 103 104 103 query43 543 513 491 491 query44 1264805 817 805 query45 187 169 207 169 query46 1162726 694 694 query47 1962182518311825 query48 418 322 318 318 query49 1262396 400 396 query50 794 380 392 380 query51 7293699970856999 query52 103 92 90 90 query53 255 186 178 178 query54 1148397 409 397 query55 79 74 81 74 query56 261 247 237 237 query57 1246110310991099 query58 230 222 225 222 query59 3219301730703017 query60 271 257 228 228 query61 155 113 106 106 query62 910 733 740 733 query63 217 183 189 183 query64 4949659 690 659 query65 3302321932393219 query66 1280312 325 312 query67 15860 15550 15642 15550 query68 5887550 571 550 query69 436 249 263 249 query70 1225109111091091 query71 438 254 254 254 query72 6302407041084070 query73 779 363 363 363 query74 10599 888790778887 query75 3416265026392639 query76 3551107611071076 query77 546 277 273 273 query78 10150 956193969396 query79 2237616 610 610 query80 1067426 415 415 query81 543 247 228 228 query82 860 125 126 125 query83 250 148 222 148 query84 239 71 71 71 query85 1307307 299 299 query86 424 304 291 291 query87 4570439043224322 query88 3948225021732173 query89 399 294 292 292 query90 2251188 187 187 query91 140 103 104 103 query92 60 51 51 51 query93 1635550 552 550 query94 1113350 285 285 query95 364 248 248 248 query96 610 285 275 275 query97 2871268126692669 query98 238 201 197 197 query99 1637145214331433 Total cold run time: 305767 ms Total hot run time: 190454 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
(doris) branch branch-2.1 updated: [enhancement](blacklist) ignore shutdown message to avoid add the backend to blacklist (#45875)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new 956138da5c2 [enhancement](blacklist) ignore shutdown message to avoid
add the backend to blacklist (#45875)
956138da5c2 is described below
commit 956138da5c2b84040bf393f026c94cf2a6c65d26
Author: yiguolei
AuthorDate: Tue Dec 24 19:03:51 2024 +0800
[enhancement](blacklist) ignore shutdown message to avoid add the backend
to blacklist (#45875)
### What problem does this PR solve?
1. If one request failed to send fragment to BE then it will call
removeProxy;
2. removeProxy will call shutdown and shutdown will cancel all requests
related with this client.
3. Other request will get exception when receive data from be and will
add this be to blacklist.
Should avoid this scenario.
None
---
fe/fe-core/src/main/java/org/apache/doris/qe/ResultReceiver.java | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/ResultReceiver.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/ResultReceiver.java
index 43ad573bf79..9d234ae9e69 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/ResultReceiver.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/ResultReceiver.java
@@ -179,7 +179,10 @@ public class ResultReceiver {
status.updateStatus(TStatusCode.TIMEOUT, e.getMessage());
} else {
status.updateStatus(TStatusCode.THRIFT_RPC_ERROR,
e.getMessage());
-SimpleScheduler.addToBlacklist(backendId, e.getMessage());
+// Shutdown maybe called by other request, should ignore this
case.
+if (!e.getMessage().contains("shutdown")) {
+SimpleScheduler.addToBlacklist(backendId, e.getMessage());
+}
}
} catch (TimeoutException e) {
LOG.warn("fetch result timeout, finstId={}",
DebugUtil.printId(finstId), e);
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](blacklist) ignore shutdown message to avoid add the backend to blacklist [doris]
yiguolei merged PR #45875: URL: https://github.com/apache/doris/pull/45875 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [test] fix partition prune unstable cases (#44976) (#45846)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new bcc2c155dff [test] fix partition prune unstable cases (#44976) (#45846)
bcc2c155dff is described below
commit bcc2c155dffc768adba347e64a6e0cea8592b951
Author: feiniaofeiafei
AuthorDate: Tue Dec 24 19:06:26 2024 +0800
[test] fix partition prune unstable cases (#44976) (#45846)
cherry-pick #44976 to branch-2.1
---
.../nereids_rules_p0/partition_prune/int_as_date_partition_col.groovy| 1 +
.../nereids_rules_p0/partition_prune/one_col_range_partition.groovy | 1 +
.../partition_prune/varchar_as_date_partition_col.groovy | 1 +
3 files changed, 3 insertions(+)
diff --git
a/regression-test/suites/nereids_rules_p0/partition_prune/int_as_date_partition_col.groovy
b/regression-test/suites/nereids_rules_p0/partition_prune/int_as_date_partition_col.groovy
index 4554947d5ea..e9afd352727 100644
---
a/regression-test/suites/nereids_rules_p0/partition_prune/int_as_date_partition_col.groovy
+++
b/regression-test/suites/nereids_rules_p0/partition_prune/int_as_date_partition_col.groovy
@@ -16,6 +16,7 @@
// under the License.
suite("int_as_date_partition_col") {
+sql "set ENABLE_FOLD_CONSTANT_BY_BE=false"
sql "drop table if exists partition_int"
sql """CREATE TABLE partition_int(a int, dt int) PARTITION BY range(dt) (
partition p20240101 values less than ("20240101"),
diff --git
a/regression-test/suites/nereids_rules_p0/partition_prune/one_col_range_partition.groovy
b/regression-test/suites/nereids_rules_p0/partition_prune/one_col_range_partition.groovy
index 4e04fed0cc4..a52f2577477 100644
---
a/regression-test/suites/nereids_rules_p0/partition_prune/one_col_range_partition.groovy
+++
b/regression-test/suites/nereids_rules_p0/partition_prune/one_col_range_partition.groovy
@@ -16,6 +16,7 @@
// under the License.
suite("one_col_range_partition") {
+sql "set ENABLE_FOLD_CONSTANT_BY_BE=false"
sql "drop table if exists one_col_range_partition_date"
sql """
create table one_col_range_partition_date(a int, dt datetime, d date, c
varchar(100)) duplicate key(a)
diff --git
a/regression-test/suites/nereids_rules_p0/partition_prune/varchar_as_date_partition_col.groovy
b/regression-test/suites/nereids_rules_p0/partition_prune/varchar_as_date_partition_col.groovy
index 0cfe4963f27..a5b3a4375c1 100644
---
a/regression-test/suites/nereids_rules_p0/partition_prune/varchar_as_date_partition_col.groovy
+++
b/regression-test/suites/nereids_rules_p0/partition_prune/varchar_as_date_partition_col.groovy
@@ -16,6 +16,7 @@
// under the License.
suite("varchar_as_date_partition_col") {
+sql "set ENABLE_FOLD_CONSTANT_BY_BE=false"
sql"""drop table if exists partition_varchar;"""
sql """CREATE TABLE partition_varchar(a int, dt varchar(10), rdt datetime)
PARTITION BY list(dt) (
partition p20240101 values in ("20240101","20240102"),
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test] fix partition prune unstable cases (#44976) [doris]
yiguolei merged PR #45846: URL: https://github.com/apache/doris/pull/45846 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](tvf) fix azure tvf: can not build s3() [doris]
doris-robot commented on PR #45872: URL: https://github.com/apache/doris/pull/45872#issuecomment-2560996244 TPC-DS: Total hot run time: 190952 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 364e0a1688f58d5c72100283e29ffdff7048459b, data reload: false query1 976 381 389 381 query2 6527255324402440 query3 6720223 211 211 query4 33632 23441 23427 23427 query5 4370490 444 444 query6 275 188 200 188 query7 4638311 309 309 query8 301 237 243 237 query9 9824278027702770 query10 475 257 264 257 query11 18109 15956 15486 15486 query12 155 104 103 103 query13 1746441 428 428 query14 12163 682069566820 query15 289 176 182 176 query16 8131472 493 472 query17 1752595 605 595 query18 2144299 288 288 query19 362 149 150 149 query20 121 108 111 108 query21 201 105 117 105 query22 4789457743874387 query23 34317 33440 33596 33440 query24 11056 258124102410 query25 659 405 404 404 query26 1821151 149 149 query27 2843327 330 327 query28 8139245624702456 query29 1038404 406 404 query30 294 152 149 149 query31 1043810 801 801 query32 95 85 57 57 query33 770 313 286 286 query34 961 510 515 510 query35 914 750 753 750 query36 967 942 942 query37 284 68 77 68 query38 4157399140393991 query39 1515142614471426 query40 280 101 99 99 query41 48 42 46 42 query42 122 104 103 103 query43 528 502 494 494 query44 1253804 800 800 query45 183 161 165 161 query46 1176704 715 704 query47 1980187618901876 query48 410 327 317 317 query49 1275390 400 390 query50 813 373 392 373 query51 7276711371337113 query52 104 92 92 92 query53 257 181 189 181 query54 1244420 413 413 query55 86 79 78 78 query56 259 243 247 243 query57 1276118311251125 query58 238 232 228 228 query59 3281312528392839 query60 277 249 256 249 query61 111 110 115 110 query62 939 712 734 712 query63 213 192 191 191 query64 5043655 648 648 query65 3275318432103184 query66 1410308 319 308 query67 15840 15551 15647 15551 query68 5859617 556 556 query69 433 245 249 245 query70 1177109111381091 query71 436 257 246 246 query72 7277418142124181 query73 809 372 380 372 query74 10491 880991188809 query75 3414264826452645 query76 3625115611361136 query77 555 267 283 267 query78 10193 939295939392 query79 2443611 616 611 query80 1059428 441 428 query81 546 229 222 222 query82 831 115 120 115 query83 241 146 147 146 query84 236 75 79 75 query85 1395310 306 306 query86 446 304 302 302 query87 4436458142954295 query88 4162223122082208 query89 408 294 291 291 query90 2182187 186 186 query91 144 105 106 105 query92 68 53 52 52 query93 1472561 562 561 query94 1133294 289 289 query95 370 260 257 257 query96 623 297 285 285 query97 2859269526672667 query98 216 196 195 195 query99 1654144814411441 Total cold run time: 309107 ms Total hot run time: 190952 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [typo](docker) Adjust the indentation format of the init_be and entry_point scripts, as well as the duration of loop execution(Merge-2.1). [doris]
yiguolei commented on PR #45860: URL: https://github.com/apache/doris/pull/45860#issuecomment-2560997078 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement adminSetPartitionVersionCommand in nereids [doris]
doris-robot commented on PR #45426: URL: https://github.com/apache/doris/pull/45426#issuecomment-2560995403 ClickBench: Total hot run time: 33.09 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 46fe09cb87ea4527f868e0e359d81bfe14523864, data reload: false query1 0.030.040.04 query2 0.070.040.03 query3 0.230.070.07 query4 1.600.110.11 query5 0.420.410.44 query6 1.180.650.67 query7 0.020.020.02 query8 0.030.030.03 query9 0.580.500.52 query10 0.550.560.56 query11 0.140.100.10 query12 0.140.120.11 query13 0.610.620.61 query14 2.842.872.76 query15 0.900.820.83 query16 0.380.380.37 query17 1.051.071.06 query18 0.220.220.21 query19 1.911.862.03 query20 0.010.010.01 query21 15.38 0.590.58 query22 2.542.241.90 query23 17.13 0.850.92 query24 2.941.651.08 query25 0.210.140.11 query26 0.500.140.13 query27 0.050.050.04 query28 10.15 1.111.08 query29 12.57 3.283.27 query30 0.250.060.07 query31 2.850.380.40 query32 3.240.480.46 query33 3.053.053.06 query34 16.93 4.474.51 query35 4.564.554.50 query36 0.690.490.51 query37 0.100.050.07 query38 0.050.040.04 query39 0.040.030.02 query40 0.160.140.13 query41 0.070.020.02 query42 0.040.020.02 query43 0.040.030.03 Total cold run time: 106.45 s Total hot run time: 33.09 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](mtmv)mtmv support paimon partition refresh(#43959,#44911) [doris]
github-actions[bot] commented on PR #45660: URL: https://github.com/apache/doris/pull/45660#issuecomment-2560998221 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](mtmv)mtmv support paimon partition refresh(#43959,#44911) [doris]
github-actions[bot] commented on PR #45660: URL: https://github.com/apache/doris/pull/45660#issuecomment-2560998254 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-3.0 updated: [test] fix partition prune unstable cases (#44976) (#45849)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 36abe97a24a [test] fix partition prune unstable cases (#44976) (#45849)
36abe97a24a is described below
commit 36abe97a24affd03b8fdbdeacd0b4747879daa55
Author: feiniaofeiafei
AuthorDate: Tue Dec 24 19:07:50 2024 +0800
[test] fix partition prune unstable cases (#44976) (#45849)
cherry-pick #44976 to branch-3.0
---
.../nereids_rules_p0/partition_prune/int_as_date_partition_col.groovy| 1 +
.../nereids_rules_p0/partition_prune/one_col_range_partition.groovy | 1 +
.../partition_prune/varchar_as_date_partition_col.groovy | 1 +
3 files changed, 3 insertions(+)
diff --git
a/regression-test/suites/nereids_rules_p0/partition_prune/int_as_date_partition_col.groovy
b/regression-test/suites/nereids_rules_p0/partition_prune/int_as_date_partition_col.groovy
index 5e65e6445ef..9d4e2a16490 100644
---
a/regression-test/suites/nereids_rules_p0/partition_prune/int_as_date_partition_col.groovy
+++
b/regression-test/suites/nereids_rules_p0/partition_prune/int_as_date_partition_col.groovy
@@ -16,6 +16,7 @@
// under the License.
suite("int_as_date_partition_col") {
+sql "set ENABLE_FOLD_CONSTANT_BY_BE=false"
sql "drop table if exists partition_int"
sql """CREATE TABLE partition_int(a int, dt int) PARTITION BY range(dt) (
partition p20240101 values less than ("20240101"),
diff --git
a/regression-test/suites/nereids_rules_p0/partition_prune/one_col_range_partition.groovy
b/regression-test/suites/nereids_rules_p0/partition_prune/one_col_range_partition.groovy
index e8b7621e6d7..4d1e3eb223b 100644
---
a/regression-test/suites/nereids_rules_p0/partition_prune/one_col_range_partition.groovy
+++
b/regression-test/suites/nereids_rules_p0/partition_prune/one_col_range_partition.groovy
@@ -16,6 +16,7 @@
// under the License.
suite("one_col_range_partition") {
+sql "set ENABLE_FOLD_CONSTANT_BY_BE=false"
sql "drop table if exists one_col_range_partition_date"
sql """
create table one_col_range_partition_date(a int, dt datetime, d date, c
varchar(100)) duplicate key(a)
diff --git
a/regression-test/suites/nereids_rules_p0/partition_prune/varchar_as_date_partition_col.groovy
b/regression-test/suites/nereids_rules_p0/partition_prune/varchar_as_date_partition_col.groovy
index 0cfe4963f27..a5b3a4375c1 100644
---
a/regression-test/suites/nereids_rules_p0/partition_prune/varchar_as_date_partition_col.groovy
+++
b/regression-test/suites/nereids_rules_p0/partition_prune/varchar_as_date_partition_col.groovy
@@ -16,6 +16,7 @@
// under the License.
suite("varchar_as_date_partition_col") {
+sql "set ENABLE_FOLD_CONSTANT_BY_BE=false"
sql"""drop table if exists partition_varchar;"""
sql """CREATE TABLE partition_varchar(a int, dt varchar(10), rdt datetime)
PARTITION BY list(dt) (
partition p20240101 values in ("20240101","20240102"),
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45478-branch-2.1 updated (e3a322d03e6 -> 139ec1d85db)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch auto-pick-45478-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git discard e3a322d03e6 [case](auth)Add case for auth (#45478) add 8b35b0e4774 branch-2.1: [fix](function) fixed some nested type func's param type which is not suitable and make result wrong #44923 (#45798) add c9d3f1122bd [regression-test](framework) disable defining global variable in test… (#45840) add 956138da5c2 [enhancement](blacklist) ignore shutdown message to avoid add the backend to blacklist (#45875) add bcc2c155dff [test] fix partition prune unstable cases (#44976) (#45846) add 139ec1d85db [case](auth)Add case for auth (#45478) This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (e3a322d03e6) \ N -- N -- N refs/heads/auto-pick-45478-branch-2.1 (139ec1d85db) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .../expressions/functions/scalar/ArrayApply.java | 16 -- .../functions/scalar/ArrayContains.java| 15 -- .../functions/scalar/ArrayPosition.java| 16 -- .../functions/scalar/ArrayPushBack.java| 16 -- .../functions/scalar/ArrayPushFront.java | 16 -- .../expressions/functions/scalar/ArrayRemove.java | 16 -- .../expressions/functions/scalar/CountEqual.java | 19 ++-- .../functions/scalar/MapContainsKey.java | 17 +-- .../functions/scalar/MapContainsValue.java | 17 +-- .../org/apache/doris/nereids/types/DataType.java | 35 ++ .../java/org/apache/doris/qe/ResultReceiver.java | 5 +++- .../nereids_function_p0/scalar_function/Array.out | 27 + .../test_uniq_vals_schema_change.out | 3 ++ .../doris/regression/suite/ScriptSource.groovy | 9 +- .../backup_restore/test_backup_restore_db.groovy | 2 +- .../test_backup_restore_exclude.groovy | 2 +- .../test_backup_restore_multi_tables.groovy| 2 +- ...st_backup_restore_multi_tables_overwrite.groovy | 2 +- .../backup_restore/test_backup_restore_mv.groovy | 10 +++ .../test_restore_mix_exists_and_new_table.groovy | 2 +- .../ccr_mow_syncer_p0/test_ingest_binlog.groovy| 2 +- .../inverted_index/test_ingest_binlog.groovy | 2 +- .../suites/ccr_syncer_p0/test_ingest_binlog.groovy | 2 +- .../auth/test_disable_revoke_admin_auth.groovy | 1 + .../suites/compaction/test_full_compaction.groovy | 2 +- .../correctness/test_trim_new_parameters.groovy| 2 +- .../ddl_p0/test_create_table_properties.groovy | 2 +- .../suites/export/test_array_export.groovy | 6 ++-- .../suites/export/test_map_export.groovy | 4 +-- .../suites/export/test_struct_export.groovy| 2 +- .../outfile/csv/test_outfile_empty_data.groovy | 14 - .../test_outfile_expr_generate_col_name.groovy | 10 +++ .../suites/export_p0/test_export_basic.groovy | 8 ++--- .../export_p0/test_outfile_file_suffix.groovy | 2 +- .../export_p0/test_show_create_database.groovy | 2 +- .../hive/test_autoinc_broker_load.groovy | 22 +++--- .../hive/test_hive_parquet_alter_column.groovy | 2 +- .../hive/test_hive_statistic.groovy| 2 +- .../hive/test_partial_update_broker_load.groovy| 16 +- .../iceberg/test_iceberg_optimize_count.groovy | 10 +++ .../test_iceberg_predicate_conversion.groovy | 2 +- .../insert_group_commit_with_exception.groovy | 4 +-- .../test_add_drop_index_on_table_with_mv.groovy| 2 +- .../index_change/test_pk_uk_index_change.groovy| 2 +- .../test_schema_change_storage_format.groovy | 2 +- .../test_array_contains_with_inverted_index.groovy | 4 +-- .../test_index_match_select.groovy | 8 ++--- .../test_primary_key_simple_case.groovy| 2 +- .../http_stream/test_http_stream_2pc.groovy| 8 ++--- .../test_insert_random_distribution_table.groovy | 4 +-- .../load_p0/mysql_load/test_mysql_load.groovy | 2 +- ...t_stream_load_with_nonexist_db_and_table.groovy | 6 ++-- .../test_map_load_and_compaction.groovy| 4 ++- .../load_p0/stream_load/test_stream_load.groovy| 22 +++--- .../test_stream_lo
Re: [PR] branch-2.1: [case](auth)Add case for auth #45478 [doris]
yiguolei commented on PR #45819: URL: https://github.com/apache/doris/pull/45819#issuecomment-2560999741 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](tvf) fix azure tvf: can not build s3() [doris]
github-actions[bot] commented on PR #45872: URL: https://github.com/apache/doris/pull/45872#issuecomment-2561001333 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](tvf) fix azure tvf: can not build s3() [doris]
github-actions[bot] commented on PR #45872: URL: https://github.com/apache/doris/pull/45872#issuecomment-2561001370 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](tvf) fix azure tvf: can not build s3() [doris]
doris-robot commented on PR #45872: URL: https://github.com/apache/doris/pull/45872#issuecomment-2561001593 ClickBench: Total hot run time: 32.7 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 364e0a1688f58d5c72100283e29ffdff7048459b, data reload: false query1 0.040.030.03 query2 0.080.030.04 query3 0.240.070.07 query4 1.600.100.10 query5 0.430.420.40 query6 1.180.650.65 query7 0.030.020.01 query8 0.040.030.03 query9 0.580.500.51 query10 0.540.590.55 query11 0.150.100.10 query12 0.140.100.11 query13 0.610.610.60 query14 2.692.862.74 query15 0.890.820.82 query16 0.390.380.39 query17 1.041.071.04 query18 0.220.210.21 query19 1.891.802.00 query20 0.020.010.01 query21 15.36 0.620.56 query22 3.112.431.71 query23 17.00 0.870.79 query24 2.931.261.15 query25 0.200.060.13 query26 0.420.140.15 query27 0.050.040.04 query28 10.48 1.101.07 query29 12.56 3.253.23 query30 0.250.070.07 query31 2.850.400.38 query32 3.240.460.46 query33 3.053.053.11 query34 16.53 4.544.51 query35 4.594.514.52 query36 0.690.480.48 query37 0.100.060.06 query38 0.050.040.04 query39 0.030.030.02 query40 0.170.130.13 query41 0.080.030.02 query42 0.040.020.02 query43 0.040.040.03 Total cold run time: 106.62 s Total hot run time: 32.7 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](auth)Add the default configuration file for ranger under `fe/conf` [doris]
doris-robot commented on PR #45869: URL: https://github.com/apache/doris/pull/45869#issuecomment-2561002472 TPC-DS: Total hot run time: 197674 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 67468ef0153dbf2589860f2bb5ccfac68b90ca38, data reload: false query1 1288940 925 925 query2 6225237124882371 query3 10952 458145834581 query4 33253 23436 23449 23436 query5 4590471 444 444 query6 268 188 178 178 query7 3994304 310 304 query8 298 232 223 223 query9 9452275927512751 query10 489 254 267 254 query11 18083 15081 15037 15037 query12 161 103 106 103 query13 1586446 425 425 query14 9941735672757275 query15 260 204 195 195 query16 8155486 426 426 query17 1637600 613 600 query18 2160323 333 323 query19 384 178 166 166 query20 124 121 116 116 query21 215 111 111 111 query22 4760459445424542 query23 34475 33814 33718 33718 query24 10393 259925272527 query25 575 394 396 394 query26 719 153 157 153 query27 2153338 352 338 query28 7043249925262499 query29 753 452 428 428 query30 232 159 150 150 query31 1072853 867 853 query32 92 56 53 53 query33 778 302 288 288 query34 1075522 515 515 query35 905 779 735 735 query36 1119969 974 969 query37 125 75 73 73 query38 4264416241564156 query39 1513147514491449 query40 215 107 102 102 query41 51 45 44 44 query42 117 106 107 106 query43 539 500 499 499 query44 1247839 830 830 query45 191 166 168 166 query46 1200726 728 726 query47 2073193619311931 query48 427 328 332 328 query49 883 388 436 388 query50 826 411 400 400 query51 7362719671197119 query52 105 93 100 93 query53 261 184 185 184 query54 1285418 419 418 query55 81 80 76 76 query56 282 268 238 238 query57 1296115511701155 query58 247 226 222 222 query59 3396331832763276 query60 317 251 261 251 query61 109 105 110 105 query62 921 770 747 747 query63 227 196 199 196 query64 3656697 652 652 query65 3320324932893249 query66 837 303 309 303 query67 16365 15739 15595 15595 query68 6626562 553 553 query69 505 257 251 251 query70 1193111411411114 query71 515 251 255 251 query72 7064398739673967 query73 803 369 387 369 query74 10110 905092449050 query75 4260267426532653 query76 4481107110751071 query77 673 272 278 272 query78 10283 945898579458 query79 1452612 619 612 query80 1036423 420 420 query81 499 245 228 228 query82 222 126 115 115 query83 196 148 152 148 query84 287 80 67 67 query85 1019316 301 301 query86 366 312 304 304 query87 4730445545374455 query88 4244228022042204 query89 445 291 290 290 query90 2102190 186 186 query91 137 101 103 101 query92 66 50 51 50 query93 3783562 559 559 query94 890 271 277 271 query95 347 255 250 250 query96 631 278 281 278 query97 2870269626942694 query98 216 217 188 188 query99 1740147714611461 Total cold run time: 307188 ms Total hot run time: 197674 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [fix](nereids) fix illegal import of guava's Throwables [doris]
hello-stephen commented on PR #45876: URL: https://github.com/apache/doris/pull/45876#issuecomment-2561003875 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](nereids) fix illegal import of guava's Throwables [doris]
924060929 opened a new pull request, #45876: URL: https://github.com/apache/doris/pull/45876 ### What problem does this PR solve? fix illegal import of guava's Throwables ### Check List (For Author) - Test - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [x] No need to test or manual test. Explain why: - [x] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason - Behavior changed: - [x] No. - [ ] Yes. - Does this need documentation? - [x] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test] fix partition prune unstable cases (#44976) [doris]
yiguolei merged PR #45849: URL: https://github.com/apache/doris/pull/45849 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](inverted index)Optimize code to get rid of heap use after free [doris]
doris-robot commented on PR #45745: URL: https://github.com/apache/doris/pull/45745#issuecomment-2561004600 TPC-H: Total hot run time: 39526 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 6a35a236e35617e79529523d9e9416729e52563e, data reload: false -- Round 1 -- q1 17601 740872187218 q2 2047177 172 172 q3 10549 112211791122 q4 10218 706 746 706 q5 7593269926462646 q6 235 145 146 145 q7 983 628 627 627 q8 9247185418791854 q9 6671641564656415 q10 6976227923352279 q11 468 262 252 252 q12 418 218 215 215 q13 17759 298629362936 q14 241 211 216 211 q15 546 502 485 485 q16 684 591 606 591 q17 985 513 592 513 q18 7211655066586550 q19 13391078957 957 q20 478 185 187 185 q21 4061314832333148 q22 374 299 320 299 Total cold run time: 106684 ms Total hot run time: 39526 ms - Round 2, with runtime_filter_mode=off - q1 7231722172517221 q2 321 229 224 224 q3 2876276729442767 q4 2119188517391739 q5 5552563056135613 q6 220 141 140 140 q7 2188189017921792 q8 3332354634833483 q9 8931891589458915 q10 3651359035573557 q11 604 501 506 501 q12 797 591 635 591 q13 12251 313130953095 q14 306 279 273 273 q15 542 518 503 503 q16 682 634 662 634 q17 1808161315931593 q18 7820739174597391 q19 1653159415111511 q20 2050177418641774 q21 5432524852095209 q22 628 574 578 574 Total cold run time: 70994 ms Total hot run time: 59100 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids) fix illegal import of guava's Throwables [doris]
924060929 commented on PR #45876: URL: https://github.com/apache/doris/pull/45876#issuecomment-2561007103 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](auth)Add the default configuration file for ranger under `fe/conf` [doris]
doris-robot commented on PR #45869: URL: https://github.com/apache/doris/pull/45869#issuecomment-2561007633 ClickBench: Total hot run time: 32.58 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 67468ef0153dbf2589860f2bb5ccfac68b90ca38, data reload: false query1 0.040.040.03 query2 0.070.030.03 query3 0.240.080.07 query4 1.610.100.11 query5 0.430.420.42 query6 1.140.650.64 query7 0.020.010.02 query8 0.040.040.03 query9 0.580.510.53 query10 0.550.600.55 query11 0.150.100.10 query12 0.140.110.11 query13 0.600.600.59 query14 2.912.872.80 query15 0.900.830.82 query16 0.380.370.38 query17 1.051.031.07 query18 0.220.210.21 query19 1.991.792.00 query20 0.010.010.02 query21 15.37 0.570.58 query22 2.782.692.10 query23 16.70 1.240.78 query24 2.950.631.38 query25 0.330.120.05 query26 0.400.140.14 query27 0.040.030.04 query28 10.62 1.101.07 query29 12.61 3.273.28 query30 0.250.070.06 query31 2.860.380.38 query32 3.240.470.45 query33 3.073.203.13 query34 16.88 4.444.49 query35 4.494.484.48 query36 0.690.480.49 query37 0.090.060.06 query38 0.040.030.04 query39 0.040.020.02 query40 0.170.130.12 query41 0.080.020.03 query42 0.030.020.02 query43 0.030.030.03 Total cold run time: 106.83 s Total hot run time: 32.58 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](parser) Syntax error for add partition with null null [doris]
zclllyybb commented on code in PR #45865: URL: https://github.com/apache/doris/pull/45865#discussion_r1896663702 ## fe/fe-core/src/main/cup/sql_parser.cup: ## Review Comment: How about the Nereids? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](nereids)support create function command in nereids [doris]
doris-robot commented on PR #45874: URL: https://github.com/apache/doris/pull/45874#issuecomment-2561012350 TPC-H: Total hot run time: 39927 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 611affda70d2a764ad8e63ab264fcf19c7c6ae61, data reload: false -- Round 1 -- q1 17624 746172977297 q2 2056179 169 169 q3 10694 105911331059 q4 10567 751 733 733 q5 7614269926752675 q6 243 150 144 144 q7 1000631 605 605 q8 9246188819201888 q9 6576643464136413 q10 6992228823232288 q11 467 260 269 260 q12 432 222 219 219 q13 17761 297929802979 q14 243 210 214 210 q15 575 516 489 489 q16 654 576 583 576 q17 992 516 510 510 q18 7158680667956795 q19 1356957 1009957 q20 444 179 184 179 q21 4004318431773177 q22 379 329 305 305 Total cold run time: 107077 ms Total hot run time: 39927 ms - Round 2, with runtime_filter_mode=off - q1 7253723872237223 q2 332 235 231 231 q3 2895274628862746 q4 2052176418041764 q5 5679561656155615 q6 219 149 143 143 q7 2248178618271786 q8 3339353935043504 q9 8923892390048923 q10 3610356135403540 q11 606 494 518 494 q12 837 611 586 586 q13 10883 311831493118 q14 310 273 288 273 q15 543 514 496 496 q16 692 667 666 666 q17 1856162016201620 q18 8288769275477547 q19 3346157915231523 q20 2068186719131867 q21 5474552554565456 q22 682 567 615 567 Total cold run time: 72135 ms Total hot run time: 59688 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](nerieds) date_format, to_monday, last_day, to_date, year implement Monotonic [doris]
doris-robot commented on PR #44789: URL: https://github.com/apache/doris/pull/44789#issuecomment-2561230069 ClickBench: Total hot run time: 32.04 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 3ce9cf6533c26f44962487d799878369cbe7228a, data reload: false query1 0.030.030.03 query2 0.060.040.03 query3 0.240.070.07 query4 1.620.110.11 query5 0.430.420.41 query6 1.150.650.64 query7 0.020.010.02 query8 0.040.040.03 query9 0.580.500.51 query10 0.540.550.55 query11 0.160.110.10 query12 0.140.110.11 query13 0.600.610.60 query14 2.712.762.90 query15 0.890.820.83 query16 0.390.390.40 query17 1.041.051.01 query18 0.230.210.21 query19 1.901.731.99 query20 0.010.020.01 query21 15.38 0.970.59 query22 0.750.940.99 query23 14.91 1.380.57 query24 2.601.461.55 query25 0.240.190.19 query26 0.220.140.14 query27 0.070.050.05 query28 14.35 1.471.04 query29 12.62 3.953.29 query30 0.250.080.06 query31 2.820.620.37 query32 3.230.540.46 query33 3.043.083.04 query34 16.69 5.124.48 query35 4.554.464.46 query36 0.650.480.47 query37 0.100.060.06 query38 0.040.040.04 query39 0.040.020.03 query40 0.170.130.13 query41 0.080.020.02 query42 0.040.020.02 query43 0.040.030.02 Total cold run time: 105.66 s Total hot run time: 32.04 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](cooldown)backup cooldown data [doris]
doris-robot commented on PR #45589: URL: https://github.com/apache/doris/pull/45589#issuecomment-2561234555 TPC-H: Total hot run time: 32670 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit f033662aa968c16724c008f46eccb80f2c1083f3, data reload: false -- Round 1 -- q1 17615 620060316031 q2 2045307 177 177 q3 10412 1299740 740 q4 10215 881 436 436 q5 7535222420072007 q6 202 179 147 147 q7 916 753 616 616 q8 9251138311851185 q9 5303495748844884 q10 6752232218691869 q11 472 284 260 260 q12 341 365 218 218 q13 17768 363729922992 q14 252 237 224 224 q15 566 504 494 494 q16 620 625 581 581 q17 588 855 331 331 q18 7391668864896489 q19 2130988 554 554 q20 301 309 181 181 q21 2866219519451945 q22 361 329 309 309 Total cold run time: 103902 ms Total hot run time: 32670 ms - Round 2, with runtime_filter_mode=off - q1 6323624762376237 q2 241 331 229 229 q3 2266266423102310 q4 1413184613441344 q5 4339477649014776 q6 192 183 145 145 q7 2109202118311831 q8 2638281826972697 q9 7414725473637254 q10 3081332528202820 q11 588 518 496 496 q12 702 769 599 599 q13 3379369031313131 q14 291 312 288 288 q15 565 512 498 498 q16 666 699 633 633 q17 1227176012511251 q18 7741759271257125 q19 799 111210421042 q20 1922203018611861 q21 5515503749584958 q22 619 593 565 565 Total cold run time: 54030 ms Total hot run time: 52090 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch asf-site updated (30595e0ec42 -> c2ce7f6c072)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/doris-website.git discard 30595e0ec42 Automated deployment with doris branch @ 4ae5a7f921c356ee2a1549735c001d6705732839 new c2ce7f6c072 Automated deployment with doris branch @ c3b46990c7e65969aeca99280e780fd1a2815ed5 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (30595e0ec42) \ N -- N -- N refs/heads/asf-site (c2ce7f6c072) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: docs/1.2/search-index.json | 2 +- docs/2.0/search-index.json | 2 +- docs/3.0/search-index.json | 2 +- docs/dev/search-index.json | 2 +- search-index.json| 2 +- zh-CN/docs/1.2/search-index.json | 2 +- zh-CN/docs/2.0/search-index.json | 2 +- zh-CN/docs/3.0/search-index.json | 2 +- zh-CN/docs/dev/search-index.json | 2 +- zh-CN/search-index.json | 2 +- 10 files changed, 10 insertions(+), 10 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](backup)(cooldown) backup/restore for cooldown data [doris]
justfortaste closed pull request #43642: [feature](backup)(cooldown) backup/restore for cooldown data URL: https://github.com/apache/doris/pull/43642 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](backup)(cooldown) backup/restore for cooldown data [doris]
justfortaste commented on PR #43642: URL: https://github.com/apache/doris/pull/43642#issuecomment-2561238052 squash pr https://github.com/apache/doris/pull/45589 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](nestedtype)fix nested type with timezone datetime [doris]
amorynan opened a new pull request, #45885: URL: https://github.com/apache/doris/pull/45885 ### What problem does this PR solve? 1. fix nestedtype like struct/map/array nested datetime type with timezone data for load, issue: https://github.com/apache/doris/issues/36963 2. fix coredump with new_json_reader if we read invalid data will core  Issue Number: close #xxx Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason - Behavior changed: - [ ] No. - [ ] Yes. - Does this need documentation? - [ ] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nestedtype)fix nested type with timezone datetime [doris]
amorynan commented on PR #45885: URL: https://github.com/apache/doris/pull/45885#issuecomment-2561238206 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nestedtype)fix nested type with timezone datetime [doris]
hello-stephen commented on PR #45885: URL: https://github.com/apache/doris/pull/45885#issuecomment-2561238238 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](backup) Backup and restore cooldown data [doris]
justfortaste commented on PR #44993: URL: https://github.com/apache/doris/pull/44993#issuecomment-2561237389 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](cooldown)backup cooldown data [doris]
doris-robot commented on PR #45589: URL: https://github.com/apache/doris/pull/45589#issuecomment-2561243299 TPC-DS: Total hot run time: 191266 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit f033662aa968c16724c008f46eccb80f2c1083f3, data reload: false query1 969 379 380 379 query2 6519249524602460 query3 6704217 226 217 query4 33600 23677 23623 23623 query5 4395630 488 488 query6 292 216 189 189 query7 4627494 313 313 query8 320 265 250 250 query9 9382272527122712 query10 481 311 261 261 query11 18552 15585 15247 15247 query12 171 107 103 103 query13 1672567 429 429 query14 10541 721071447144 query15 228 199 187 187 query16 7906615 432 432 query17 1581725 552 552 query18 2053402 297 297 query19 221 180 168 168 query20 117 115 116 115 query21 207 125 106 106 query22 4432439341124112 query23 34501 33636 33535 33535 query24 6411228423572284 query25 488 456 398 398 query26 1221277 150 150 query27 2235467 331 331 query28 5297247324242424 query29 677 546 410 410 query30 231 182 148 148 query31 1018907 814 814 query32 70 64 61 61 query33 559 347 299 299 query34 776 835 507 507 query35 797 839 738 738 query36 982 1073959 959 query37 124 105 76 76 query38 4463442341084108 query39 1511145514401440 query40 206 111 98 98 query41 49 45 48 45 query42 126 102 104 102 query43 532 529 517 517 query44 1367813 817 813 query45 184 175 177 175 query46 862 1046650 650 query47 1928190218671867 query48 386 406 318 318 query49 776 474 396 396 query50 638 664 407 407 query51 7151725771547154 query52 98 104 93 93 query53 229 252 186 186 query54 489 496 409 409 query55 85 86 126 86 query56 269 269 240 240 query57 1205121211401140 query58 234 222 233 222 query59 3149320329872987 query60 272 263 265 263 query61 111 110 133 110 query62 882 793 775 775 query63 236 194 198 194 query64 4138996 662 662 query65 3329323032903230 query66 1038418 312 312 query67 16034 15869 15477 15477 query68 7403794 518 518 query69 456 288 259 259 query70 1221115511461146 query71 444 299 250 250 query72 5721383739313837 query73 667 743 366 366 query74 9830913290769076 query75 3991314826452645 query76 36901200766 766 query77 766 374 276 276 query78 10099 10259 94139413 query79 3798903 585 585 query80 741 520 448 448 query81 504 268 236 236 query82 663 149 127 127 query83 201 169 143 143 query84 289 94 75 75 query85 782 353 308 308 query86 403 328 307 307 query87 4724449044684468 query88 4844222122102210 query89 406 339 327 327 query90 1832191 192 191 query91 145 133 107 107 query92 66 58 59 58 query93 1955903 538 538 query94 659 384 287 287 query95 343 271 269 269 query96 496 618 281 281 query97 2711277627202720 query98 225 206 195 195 query99 1694153414401440 Total cold run time: 294103 ms Total hot run time: 191266 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
[PR] [Feature](backup) backup cooldown data [doris]
justfortaste opened a new pull request, #45886: URL: https://github.com/apache/doris/pull/45886 ### What problem does this PR solve? Issue Number: close #xxx Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason - Behavior changed: - [ ] No. - [ ] Yes. - Does this need documentation? - [ ] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](backup) backup cooldown data [doris]
hello-stephen commented on PR #45886: URL: https://github.com/apache/doris/pull/45886#issuecomment-2561247012 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](cooldown)backup cooldown data [doris]
doris-robot commented on PR #45589: URL: https://github.com/apache/doris/pull/45589#issuecomment-2561247225 ClickBench: Total hot run time: 31.82 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit f033662aa968c16724c008f46eccb80f2c1083f3, data reload: false query1 0.030.040.04 query2 0.070.030.03 query3 0.230.070.07 query4 1.600.110.11 query5 0.410.390.40 query6 1.180.660.65 query7 0.020.010.01 query8 0.040.030.04 query9 0.580.510.50 query10 0.560.570.56 query11 0.150.120.11 query12 0.140.110.11 query13 0.600.600.59 query14 2.802.862.78 query15 0.900.820.83 query16 0.390.390.39 query17 1.061.041.03 query18 0.230.220.21 query19 1.951.782.01 query20 0.010.010.01 query21 15.36 0.920.59 query22 0.740.820.71 query23 15.22 1.430.52 query24 2.621.371.94 query25 0.200.220.06 query26 0.190.150.13 query27 0.050.050.04 query28 14.23 1.501.04 query29 12.60 3.943.24 query30 0.250.090.07 query31 2.810.600.39 query32 3.220.550.47 query33 3.083.113.12 query34 16.90 5.184.54 query35 4.524.554.54 query36 0.670.500.48 query37 0.100.060.05 query38 0.040.040.03 query39 0.030.020.02 query40 0.170.150.13 query41 0.080.020.02 query42 0.030.020.02 query43 0.030.030.04 Total cold run time: 106.09 s Total hot run time: 31.82 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](docker)Modify the init_be and start_be scripts to meet the requirements for rapid Docker startup(Merge 2.1). [doris]
Thearas commented on PR #45858: URL: https://github.com/apache/doris/pull/45858#issuecomment-2560810479 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](docker)Modify the init_be and start_be scripts to meet the requirements for rapid Docker startup(Merge 2.1). [doris]
FreeOnePlus commented on PR #45858: URL: https://github.com/apache/doris/pull/45858#issuecomment-2560810827 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feat](Nereids) Support use mv hint [doris]
doris-robot commented on PR #45855: URL: https://github.com/apache/doris/pull/45855#issuecomment-2560811446 TPC-H: Total hot run time: 40176 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit ba21a85474e2e66a8bac88edea74d999cacbda42, data reload: false -- Round 1 -- q1 17596 739773127312 q2 2041201 177 177 q3 10518 112911961129 q4 10218 772 741 741 q5 7588269526222622 q6 239 148 146 146 q7 997 625 625 625 q8 9248184418981844 q9 6597647463926392 q10 6994234023392339 q11 467 261 266 261 q12 437 221 221 221 q13 17768 294529652945 q14 249 213 213 213 q15 551 501 496 496 q16 663 593 593 593 q17 983 585 511 511 q18 7341678668306786 q19 1391102410681024 q20 487 183 185 183 q21 4003331933043304 q22 367 312 316 312 Total cold run time: 106743 ms Total hot run time: 40176 ms - Round 2, with runtime_filter_mode=off - q1 7287725572357235 q2 335 231 234 231 q3 2888275530932755 q4 2120190817941794 q5 5554558756245587 q6 228 138 139 138 q7 2217178118101781 q8 3390352834903490 q9 8896891489498914 q10 3634355235593552 q11 603 508 506 506 q12 803 623 614 614 q13 10913 306830683068 q14 309 276 281 276 q15 567 504 510 504 q16 714 634 671 634 q17 1834160415951595 q18 7825740272867286 q19 1711161015391539 q20 2058185018371837 q21 5436525952595259 q22 630 609 544 544 Total cold run time: 69952 ms Total hot run time: 59139 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch master updated: [improve] add data-distribution-concept (#1568)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 721074180f4 [improve] add data-distribution-concept (#1568)
721074180f4 is described below
commit 721074180f41470291d66a53e009029bf628b39e
Author: Yongqiang YANG
AuthorDate: Tue Dec 24 17:18:23 2024 +0800
[improve] add data-distribution-concept (#1568)
## Versions
- [x] dev
- [ ] 3.0
- [ ] 2.1
- [ ] 2.0
## Languages
- [x] Chinese
- [x] English
## Docs Checklist
- [ ] Checked by AI
- [ ] Test Cases Built
-
Co-authored-by: Yongqiang YANG
---
.../data-partitioning/data-distribution.md | 123 +
.../data-partitioning/data-distribution.md | 123 +
sidebars.json | 2 +-
3 files changed, 247 insertions(+), 1 deletion(-)
diff --git a/docs/table-design/data-partitioning/data-distribution.md
b/docs/table-design/data-partitioning/data-distribution.md
new file mode 100644
index 000..f9036d97244
--- /dev/null
+++ b/docs/table-design/data-partitioning/data-distribution.md
@@ -0,0 +1,123 @@
+---
+{
+"title": "Data Distribution Concept",
+"language": "en"
+}
+---
+
+
+
+In Doris, the core of **data distribution** is to efficiently map the rows of
data written to the table onto various **data shards (Tablets)** in the
underlying storage through reasonable partitioning and bucketing strategies.
Through data distribution strategies, Doris can fully utilize the storage and
computing capabilities of multiple nodes, thereby supporting efficient storage
and querying of large-scale data.
+
+---
+
+## Overview of Data Distribution
+
+### Data Writing
+
+When writing data, Doris first allocates the rows of data to the corresponding
partitions based on the table's partitioning strategy. Then, according to the
bucketing strategy, the rows of data are further mapped to specific shards
within the partition, determining the storage location of the data rows.
+
+### Query Execution
+
+During query execution, Doris's optimizer will trim data based on partitioning
and bucketing strategies to maximize the reduction of the scanning range. In
cases involving JOIN or aggregation queries, data transfer across nodes
(Shuffle) may occur. Reasonable partitioning and bucketing design can reduce
Shuffle and fully utilize **Colocate Join** to optimize query performance.
+
+---
+
+## Node and Storage Architecture
+
+### Node Types
+
+A Doris cluster consists of the following two types of nodes:
+
+- **FE Node (Frontend)**: Manages cluster metadata (such as tables and shards)
and is responsible for SQL parsing and execution planning.
+- **BE Node (Backend)**: Stores data and is responsible for executing
computational tasks. The results from BE are aggregated and returned to FE,
which then returns them to the user.
+
+### Data Shard (Tablet)
+
+The data stored in the BE node is divided into shards, with each shard being
the smallest unit of data management in Doris and the basic unit for data
movement and replication.
+
+---
+
+## Partitioning Strategy
+
+Partitioning is the first layer of logical division for data organization,
used to divide the data in the table into smaller subsets. Doris provides the
following two **partition types** and three **partition modes**:
+
+### Partition Types
+
+- **Range Partitioning**: Allocates data rows to corresponding partitions
based on the value range of the partition column.
+- **List Partitioning**: Allocates data rows to corresponding partitions based
on specific values of the partition column.
+
+### Partition Modes
+
+- **Manual Partitioning**: Users manually create partitions (e.g., specified
during table creation or added via `ALTER` statements).
+- **Dynamic Partitioning**: The system automatically creates partitions based
on time scheduling rules, but does not create partitions on demand when writing
data.
+- **Automatic Partitioning**: The system automatically creates corresponding
partitions as needed during data writing, but care should be taken to avoid
generating too many partitions with dirty data.
+
+---
+
+## Bucketing Strategy
+
+Bucketing is the second layer of logical division for data organization, used
to further divide data rows into smaller units within a partition. Doris
supports the following two bucketing methods:
+
+- **Hash Bucketing**: Distributes data rows evenly across shards by
calculating the `crc32` hash value of the bucketing column and taking the
modulus of the number of buckets.
+- **Random Bucketing**: Randomly allocates data rows to shards. When using
Random bucketing, the `load_to_single_tablet` option can be used to optimize
the quick writing of small-scale data.
+
Re: [PR] [opt] improvement docs for tiered storage [doris-website]
dataroaring merged PR #1567: URL: https://github.com/apache/doris-website/pull/1567 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve] add data-distribution-concept [doris-website]
dataroaring merged PR #1568: URL: https://github.com/apache/doris-website/pull/1568 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch master updated: [improvement] imrovement doc of auto inc column (#1510)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 6abea9f0c99 [improvement] imrovement doc of auto inc column (#1510)
6abea9f0c99 is described below
commit 6abea9f0c9910dbb44bc765037dbce071c9faa1a
Author: Yongqiang YANG
AuthorDate: Tue Dec 24 17:19:18 2024 +0800
[improvement] imrovement doc of auto inc column (#1510)
## Versions
- [ ] dev
- [ ] 3.0
- [ ] 2.1
- [ ] 2.0
## Languages
- [ ] Chinese
- [ ] English
## Docs Checklist
- [ ] Checked by AI
- [ ] Test Cases Built
-
Co-authored-by: Yongqiang YANG
Co-authored-by: bobhan1
---
docs/table-design/auto-increment.md | 104 +---
1 file changed, 61 insertions(+), 43 deletions(-)
diff --git a/docs/table-design/auto-increment.md
b/docs/table-design/auto-increment.md
index 1e1eae38451..487c47b89f0 100644
--- a/docs/table-design/auto-increment.md
+++ b/docs/table-design/auto-increment.md
@@ -1,6 +1,6 @@
---
{
-"title": "Using AUTO_INCREMENT",
+"title": "Auto-Increment Column",
"language": "en"
}
---
@@ -24,22 +24,42 @@ specific language governing permissions and limitations
under the License.
-->
+When importing data, Doris automatically assigns unique values to rows that do
not have specified values in the **auto-increment column**. This feature
simplifies data import workflows while maintaining flexibility.
-When importing data, Doris assigns a table-unique value to rows that do not
have specified values in the auto-increment column.
+---
## Functionality
-For tables containing an auto-increment column, during data import:
-- If the target columns don't include the auto-increment column, Doris will
populate the auto-increment column with generated values.
-- If the target columns include the auto-increment column, null values in the
imported data for that column will be replaced by values generated by Doris,
while non-null values will remain unchanged. Note that **non-null values can
disrupt the uniqueness of the auto-increment column values**.
+For tables with an auto-increment column, Doris processes data imports as
follows:
+
+- **Auto-Population (Column Excluded)**:
+ If the imported data does not include the auto-increment column, Doris
generates and populates unique values for this column.
+
+- **Partial Specification (Column Included)**:
+ - **Null Values**: Doris replaces null values in the imported data with
system-generated unique values.
+ - **Non-Null Values**: User-provided values remain unchanged.
+ > **Important**: User-provided non-null values can disrupt the uniqueness of
the auto-increment column.
+
+---
### Uniqueness
-Doris ensures that values generated on the auto-increment column have
**table-wide uniqueness**. However, it's important to note that **the
uniqueness of the auto-increment column only guarantees uniqueness for values
automatically filled by Doris and does not consider values provided by users**.
If a user explicitly inserts user-provided values for this table by specifying
the auto-increment column, this uniqueness cannot be guaranteed.
+Doris guarantees **table-wide uniqueness** for values it generates in the
auto-increment column. However:
+
+- **Guaranteed Uniqueness**: This applies only to system-generated values.
+- **User-Provided Values**: Doris does not validate or enforce uniqueness for
values specified by users in the auto-increment column. This may result in
duplicate entries.
+
+---
### Density
-Doris ensures that the values generated on the auto-increment column are
dense, but it **cannot guarantee** that the values automatically generated in
the auto-increment column during an import will be entirely contiguous. Thus,
there might be some jumps in the values generated by the auto-increment column
during an import. This is because, for performance consideration, each BE
caches a portion of pre-allocated auto-increment column values, and these
cached values do not intersect betwe [...]
+Auto-increment values generated by Doris are generally **dense** but with some
considerations:
+
+- **Potential Gaps**: Gaps may appear due to performance optimizations. Each
backend node (BE) pre-allocates a block of unique values for efficiency, and
these blocks do not overlap between nodes.
+- **Non-Chronological Values**: Doris does not guarantee that values generated
in later imports are larger than those from earlier imports.
+ > **Note**: Auto-increment values cannot be used to infer the chronological
order of imports.
+
+---
## Syntax
@@ -47,7 +67,7 @@ To use auto-increment columns, you need to add the
`AUTO_INCREMENT` attribute to
### Examples
-1. Creating a Duplicate table with one key col
Re: [PR] [improvement] imrovement doc of auto inc column [doris-website]
dataroaring merged PR #1510: URL: https://github.com/apache/doris-website/pull/1510 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
