Re: [PR] Support for global dictionary functions. [doris]
Mryange commented on PR #45536: URL: https://github.com/apache/doris/pull/45536#issuecomment-2564133378 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](azure) let backup to azure work and use https for azure [doris]
doris-robot commented on PR #46064: URL: https://github.com/apache/doris/pull/46064#issuecomment-2564155182 ClickBench: Total hot run time: 31.14 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 219d90ec9f6af35d639f6828169253b7b25d2268, data reload: false query1 0.040.040.03 query2 0.070.030.03 query3 0.240.070.07 query4 1.620.100.10 query5 0.410.420.43 query6 1.180.650.65 query7 0.020.020.01 query8 0.030.030.03 query9 0.570.490.49 query10 0.550.590.55 query11 0.140.100.10 query12 0.140.120.11 query13 0.610.620.60 query14 2.712.842.75 query15 0.910.830.82 query16 0.390.400.39 query17 1.081.021.07 query18 0.220.210.21 query19 1.851.841.95 query20 0.010.000.02 query21 15.39 0.910.55 query22 0.760.730.58 query23 15.44 1.460.57 query24 2.610.911.54 query25 0.190.100.07 query26 0.230.140.13 query27 0.050.080.05 query28 13.84 1.541.05 query29 12.56 4.003.26 query30 0.250.100.07 query31 2.810.580.38 query32 3.230.550.48 query33 3.063.063.18 query34 16.72 5.114.45 query35 4.604.534.52 query36 0.640.480.50 query37 0.100.060.06 query38 0.050.040.04 query39 0.030.020.02 query40 0.170.130.12 query41 0.070.030.02 query42 0.030.020.02 query43 0.030.030.03 Total cold run time: 105.65 s Total hot run time: 31.14 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Workload management - update circuit breaking & job [doris-website]
wangtianyi2004 closed pull request #1650: Workload management - update circuit breaking & job URL: https://github.com/apache/doris-website/pull/1650 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enchement](utf8)import enable_text_validate_utf8 session var (#45537) [doris]
morningman merged PR #46070: URL: https://github.com/apache/doris/pull/46070 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [enchement](utf8)import enable_text_validate_utf8 session var (#45537) (#46070)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-2.1 by this push: new a380f5d2228 [enchement](utf8)import enable_text_validate_utf8 session var (#45537) (#46070) a380f5d2228 is described below commit a380f5d2228517492b863e6998c05093842b599f Author: daidai AuthorDate: Sat Dec 28 10:05:03 2024 +0800 [enchement](utf8)import enable_text_validate_utf8 session var (#45537) (#46070) bp #45537 --- be/src/util/utf8_check.cpp | 7 ++ be/src/util/utf8_check.h | 4 + be/src/vec/exec/format/csv/csv_reader.cpp | 6 +- .../scripts/create_preinstalled_scripts/run72.hql | 31 +++ .../text/utf8_check/utf8_check_fail.csv| 5 ++ .../doris/datasource/hive/source/HiveScanNode.java | 2 + .../java/org/apache/doris/qe/SessionVariable.java | 9 ++ .../ExternalFileTableValuedFunction.java | 2 + gensrc/thrift/PlanNodes.thrift | 2 + .../external_table_p0/hive/test_utf8_check.out | 55 .../external_table_p0/hive/test_utf8_check.groovy | 100 + 11 files changed, 220 insertions(+), 3 deletions(-) diff --git a/be/src/util/utf8_check.cpp b/be/src/util/utf8_check.cpp index 5355b901420..f90c27e5e91 100644 --- a/be/src/util/utf8_check.cpp +++ b/be/src/util/utf8_check.cpp @@ -327,4 +327,11 @@ bool validate_utf8(const char* src, size_t len) { return validate_utf8_naive(src, len); } #endif + +bool validate_utf8(const TFileScanRangeParams& params, const char* src, size_t len) { +if (params.__isset.file_attributes && !params.file_attributes.enable_text_validate_utf8) { +return true; +} +return validate_utf8(src, len); +} } // namespace doris diff --git a/be/src/util/utf8_check.h b/be/src/util/utf8_check.h index 4214e186b71..7e9b7a2a9de 100644 --- a/be/src/util/utf8_check.h +++ b/be/src/util/utf8_check.h @@ -17,6 +17,8 @@ #pragma once +#include + #include namespace doris { @@ -25,4 +27,6 @@ namespace doris { bool validate_utf8(const char* src, size_t len); // check utf8 use naive c++ bool validate_utf8_naive(const char* data, size_t len); + +bool validate_utf8(const TFileScanRangeParams& params, const char* src, size_t len); } // namespace doris diff --git a/be/src/vec/exec/format/csv/csv_reader.cpp b/be/src/vec/exec/format/csv/csv_reader.cpp index 1fc3bbad294..77a5b65d512 100644 --- a/be/src/vec/exec/format/csv/csv_reader.cpp +++ b/be/src/vec/exec/format/csv/csv_reader.cpp @@ -715,7 +715,7 @@ Status CsvReader::_fill_empty_line(Block* block, std::vector& } Status CsvReader::_validate_line(const Slice& line, bool* success) { -if (!_is_proto_format && !validate_utf8(line.data, line.size)) { +if (!_is_proto_format && !validate_utf8(_params, line.data, line.size)) { if (!_is_load) { return Status::InternalError("Only support csv data in utf8 codec"); } else { @@ -951,7 +951,7 @@ Status CsvReader::_parse_col_nums(size_t* col_nums) { return Status::InternalError( "The first line is empty, can not parse column numbers"); } -if (!validate_utf8(const_cast(reinterpret_cast(ptr)), size)) { +if (!validate_utf8(_params, const_cast(reinterpret_cast(ptr)), size)) { return Status::InternalError("Only support csv data in utf8 codec"); } ptr = _remove_bom(ptr, size); @@ -968,7 +968,7 @@ Status CsvReader::_parse_col_names(std::vector* col_names) { if (size == 0) { return Status::InternalError("The first line is empty, can not parse column names"); } -if (!validate_utf8(const_cast(reinterpret_cast(ptr)), size)) { +if (!validate_utf8(_params, const_cast(reinterpret_cast(ptr)), size)) { return Status::InternalError("Only support csv data in utf8 codec"); } ptr = _remove_bom(ptr, size); diff --git a/docker/thirdparties/docker-compose/hive/scripts/create_preinstalled_scripts/run72.hql b/docker/thirdparties/docker-compose/hive/scripts/create_preinstalled_scripts/run72.hql new file mode 100644 index 000..1ab754b5042 --- /dev/null +++ b/docker/thirdparties/docker-compose/hive/scripts/create_preinstalled_scripts/run72.hql @@ -0,0 +1,31 @@ +CREATE TABLE invalid_utf8_data ( +id INT, +corrupted_data STRING, +string_data1 STRING, +string_data2 STRING +) +ROW FORMAT DELIMITED +FIELDS TERMINATED BY ',' +LINES TERMINATED BY '\n' +location '/user/doris/preinstalled_data/text/utf8_check'; + + +CREATE TABLE invalid_utf8_data2 ( +id INT, +corrupted_data STRING, +string_data1 STRING, +string_data2 STRING +) +ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde' +WITH SERDEPROPERTIES ( +"separatorChar" = ",", +"quoteChar" = "\"", +"escapeChar" = "\\" +) +lo
Re: [PR] [fix](azure) let backup to azure work and use https for azure [doris]
dataroaring commented on PR #46064: URL: https://github.com/apache/doris/pull/46064#issuecomment-2564143531 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](schema) support temporary table like mysql [doris]
Yulei-Yang commented on PR #40680: URL: https://github.com/apache/doris/pull/40680#issuecomment-2564145645 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](schema) support temporary table like mysql [doris]
Yulei-Yang commented on PR #40680: URL: https://github.com/apache/doris/pull/40680#issuecomment-2564157047 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] update sql-block and job-scheduler for en [doris-website]
wangtianyi2004 opened a new pull request, #1657: URL: https://github.com/apache/doris-website/pull/1657 ## Versions - [ ] dev - [ ] 3.0 - [ ] 2.1 - [ ] 2.0 ## Languages - [ ] Chinese - [ ] English ## Docs Checklist - [ ] Checked by AI - [ ] Test Cases Built -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [fix](auth) fix use database stmt access unauthorized catalog #45720 [doris]
vinlee19 commented on PR #45978: URL: https://github.com/apache/doris/pull/45978#issuecomment-2564157972 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](azure) let backup to azure work and use https for azure [doris]
doris-robot commented on PR #46064: URL: https://github.com/apache/doris/pull/46064#issuecomment-2564154141 TPC-DS: Total hot run time: 198297 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 219d90ec9f6af35d639f6828169253b7b25d2268, data reload: false query1 1279928 896 896 query2 6385241423642364 query3 11142 483250754832 query4 33371 24027 23701 23701 query5 4920614 474 474 query6 311 196 193 193 query7 3991487 312 312 query8 300 233 225 225 query9 9297273627272727 query10 482 303 267 267 query11 18092 15297 15214 15214 query12 164 104 104 104 query13 1566550 467 467 query14 9780772577557725 query15 245 224 195 195 query16 7951662 455 455 query17 1513788 580 580 query18 2082427 321 321 query19 216 190 174 174 query20 126 122 116 116 query21 210 136 115 115 query22 4583481745054505 query23 34666 33776 33608 33608 query24 6439234423152315 query25 479 475 388 388 query26 830 284 150 150 query27 2030466 348 348 query28 5278250724842484 query29 605 564 452 452 query30 209 186 149 149 query31 1013947 862 862 query32 82 58 58 58 query33 491 345 305 305 query34 777 867 526 526 query35 820 813 759 759 query36 1024106010001000 query37 115 101 79 79 query38 4420429043004290 query39 1513150814651465 query40 207 117 100 100 query41 44 46 44 44 query42 123 98 111 98 query43 531 535 501 501 query44 1325859 849 849 query45 190 177 166 166 query46 912 1062674 674 query47 1988198419481948 query48 400 424 326 326 query49 696 477 384 384 query50 666 701 406 406 query51 7245725772367236 query52 99 101 93 93 query53 237 261 183 183 query54 494 511 404 404 query55 90 82 80 80 query56 261 253 232 232 query57 1268124711821182 query58 251 225 219 219 query59 3329339231273127 query60 289 260 253 253 query61 116 109 118 109 query62 855 825 742 742 query63 229 196 197 196 query64 32831046645 645 query65 3296321232983212 query66 693 462 302 302 query67 16455 16214 15499 15499 query68 8982757 504 504 query69 496 284 259 259 query70 1226115011551150 query71 444 296 256 256 query72 6195394639723946 query73 658 772 367 367 query74 10542 917090059005 query75 4514315726492649 query76 44471206795 795 query77 859 380 281 281 query78 10086 10242 94749474 query79 2439925 626 626 query80 683 516 486 486 query81 500 281 223 223 query82 644 146 120 120 query83 190 162 152 152 query84 280 85 74 74 query85 782 415 299 299 query86 355 327 307 307 query87 4600438143984381 query88 3337223522142214 query89 420 331 317 317 query90 1868188 189 188 query91 133 129 103 103 query92 68 55 48 48 query93 1168950 517 517 query94 650 392 256 256 query95 329 265 244 244 query96 481 608 279 279 query97 2728287627102710 query98 231 202 201 201 query99 1650154114561456 Total cold run time: 296354 ms Total hot run time: 198297 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [fix](azure) let backup to azure work and use https for azure [doris]
doris-robot commented on PR #46064: URL: https://github.com/apache/doris/pull/46064#issuecomment-2564153517 TeamCity be ut coverage result: Function Coverage: 38.89% (10119/26022) Line Coverage: 29.87% (85490/286185) Region Coverage: 29.02% (43705/150621) Branch Coverage: 25.55% (22298/87256) Coverage Report: http://coverage.selectdb-in.cc/coverage/219d90ec9f6af35d639f6828169253b7b25d2268_219d90ec9f6af35d639f6828169253b7b25d2268/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](case) fix some cases [doris]
github-actions[bot] commented on PR #37038: URL: https://github.com/apache/doris/pull/37038#issuecomment-2564110169 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](azure) let backup to azure work and use https for azure [doris]
doris-robot commented on PR #46064: URL: https://github.com/apache/doris/pull/46064#issuecomment-2563560049 TeamCity be ut coverage result: Function Coverage: 38.89% (10121/26022) Line Coverage: 29.89% (85509/286125) Region Coverage: 29.03% (43710/150574) Branch Coverage: 25.56% (22293/87218) Coverage Report: http://coverage.selectdb-in.cc/coverage/cb9cefb97b3d3e434cdb4ed31a92620bfeb4bd71_cb9cefb97b3d3e434cdb4ed31a92620bfeb4bd71/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](azure) let backup to azure work and use https for azure [doris]
doris-robot commented on PR #46064: URL: https://github.com/apache/doris/pull/46064#issuecomment-2563561105 TPC-DS: Total hot run time: 196358 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit cb9cefb97b3d3e434cdb4ed31a92620bfeb4bd71, data reload: false query1 1319969 886 886 query2 6343231623232316 query3 10987 467646264626 query4 33099 23826 23508 23508 query5 4368622 482 482 query6 295 209 194 194 query7 3981488 303 303 query8 310 246 244 244 query9 9587275027372737 query10 462 305 229 229 query11 17926 15236 15069 15069 query12 155 107 102 102 query13 1551520 384 384 query14 10766 718473877184 query15 247 206 187 187 query16 8214602 466 466 query17 1542761 569 569 query18 2140430 336 336 query19 213 206 166 166 query20 133 118 114 114 query21 212 129 110 110 query22 4645456845474547 query23 34565 34282 33725 33725 query24 6546238322652265 query25 457 463 401 401 query26 750 256 153 153 query27 2113458 328 328 query28 5641247324692469 query29 537 556 420 420 query30 223 209 161 161 query31 983 903 858 858 query32 101 64 69 64 query33 517 363 322 322 query34 786 883 507 507 query35 846 861 744 744 query36 10381073977 977 query37 128 103 78 78 query38 4255427941344134 query39 1536149514661466 query40 225 124 108 108 query41 47 48 54 48 query42 119 112 107 107 query43 518 547 507 507 query44 1339843 849 843 query45 188 177 185 177 query46 872 1064666 666 query47 1996201719021902 query48 376 418 328 328 query49 708 477 381 381 query50 665 663 393 393 query51 7257735272397239 query52 101 102 93 93 query53 219 261 183 183 query54 481 494 414 414 query55 84 84 78 78 query56 261 252 236 236 query57 1245122511501150 query58 236 215 223 215 query59 3256335231253125 query60 293 291 255 255 query61 106 108 110 108 query62 869 813 743 743 query63 225 201 185 185 query64 36641039637 637 query65 3310325232253225 query66 777 398 332 332 query67 16566 15827 15521 15521 query68 729 501 501 query69 476 294 243 243 query70 1221116011461146 query71 463 275 248 248 query72 6238396538033803 query73 650 741 357 357 query74 9930909788858885 query75 4544313728192819 query76 50111212752 752 query77 925 371 270 270 query78 996610365 93339333 query79 3018892 590 590 query80 594 507 436 436 query81 496 268 232 232 query82 659 154 120 120 query83 161 158 138 138 query84 241 181 69 69 query85 785 369 300 300 query86 358 308 305 305 query87 4442439245284392 query88 4175219822062198 query89 424 332 296 296 query90 1843183 188 183 query91 140 129 103 103 query92 67 54 51 51 query93 1036864 513 513 query94 646 402 269 269 query95 331 257 255 255 query96 483 606 292 292 query97 2742282426142614 query98 216 205 188 188 query99 2504154414271427 Total cold run time: 300304 ms Total hot run time: 196358 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [enhance](auth)Reduce the scope of permission locks [doris]
doris-robot commented on PR #46115: URL: https://github.com/apache/doris/pull/46115#issuecomment-2563566178 TPC-H: Total hot run time: 32399 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit b26c683bc18f5c8306b3937de0e3d2bd398cf366, data reload: false -- Round 1 -- q1 17606 611159985998 q2 2045312 191 191 q3 10384 1251727 727 q4 10213 854 428 428 q5 7510217319711971 q6 208 183 149 149 q7 890 753 609 609 q8 9242134811531153 q9 5161487948794879 q10 6746233318891889 q11 466 271 251 251 q12 345 347 230 230 q13 17751 353429032903 q14 230 237 212 212 q15 541 496 493 493 q16 637 620 609 609 q17 582 842 324 324 q18 7240640564916405 q19 2314979 539 539 q20 304 310 190 190 q21 2798218019371937 q22 361 326 312 312 Total cold run time: 103574 ms Total hot run time: 32399 ms - Round 2, with runtime_filter_mode=off - q1 6276628862416241 q2 240 322 238 238 q3 2242263023292329 q4 1415179313561356 q5 4360475448094754 q6 189 181 142 142 q7 2116196217771777 q8 2564279526472647 q9 7345725172017201 q10 3059329628422842 q11 592 510 503 503 q12 664 777 628 628 q13 3383380731173117 q14 304 308 278 278 q15 559 512 513 512 q16 661 674 625 625 q17 1200170712441244 q18 7809731069776977 q19 786 114410371037 q20 1920193018231823 q21 5443520048944894 q22 597 624 568 568 Total cold run time: 53724 ms Total hot run time: 51733 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](azure) let backup to azure work and use https for azure [doris]
doris-robot commented on PR #46064: URL: https://github.com/apache/doris/pull/46064#issuecomment-2563565567 ClickBench: Total hot run time: 31.53 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit cb9cefb97b3d3e434cdb4ed31a92620bfeb4bd71, data reload: false query1 0.040.040.04 query2 0.070.030.04 query3 0.240.070.06 query4 1.610.110.10 query5 0.430.420.42 query6 1.170.640.64 query7 0.020.020.02 query8 0.040.030.03 query9 0.590.500.52 query10 0.560.600.56 query11 0.160.100.10 query12 0.140.100.11 query13 0.610.600.61 query14 2.702.852.75 query15 0.880.820.82 query16 0.390.380.40 query17 1.061.081.06 query18 0.220.200.20 query19 1.961.862.00 query20 0.010.010.01 query21 15.36 0.970.60 query22 0.761.020.68 query23 15.03 1.430.66 query24 3.350.992.17 query25 0.130.140.14 query26 0.400.140.14 query27 0.050.050.05 query28 14.13 1.481.04 query29 12.61 3.933.30 query30 0.250.090.06 query31 2.820.600.38 query32 3.220.540.45 query33 3.083.043.14 query34 16.66 5.054.50 query35 4.484.474.45 query36 0.630.490.47 query37 0.100.060.06 query38 0.040.040.03 query39 0.040.030.02 query40 0.180.150.12 query41 0.080.030.02 query42 0.040.020.03 query43 0.030.030.03 Total cold run time: 106.37 s Total hot run time: 31.53 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [env](compile) open compile_check in some file [doris]
doris-robot commented on PR #46061: URL: https://github.com/apache/doris/pull/46061#issuecomment-2563565614 TeamCity be ut coverage result: Function Coverage: 38.89% (10121/26022) Line Coverage: 29.89% (85525/286126) Region Coverage: 29.03% (43717/150568) Branch Coverage: 25.56% (22297/87218) Coverage Report: http://coverage.selectdb-in.cc/coverage/f72081cf2c0862346b0734e653889e3af8576967_f72081cf2c0862346b0734e653889e3af8576967/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix] make create table and load data procedure easily [doris-website]
KassieZ merged PR #1603: URL: https://github.com/apache/doris-website/pull/1603 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cluster key) skip unsupported case [doris]
dataroaring merged PR #45579: URL: https://github.com/apache/doris/pull/45579 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cluster key) fix compaction and schema change [doris]
dataroaring merged PR #45383: URL: https://github.com/apache/doris/pull/45383 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch master updated: [fix] make create table and load data procedure easily (#1603)
This is an automated email from the ASF dual-hosted git repository. kassiez pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 2babf360221 [fix] make create table and load data procedure easily (#1603) 2babf360221 is described below commit 2babf36022152ebebacdde38f243b4518bcc7612 Author: feiniaofeiafei AuthorDate: Fri Dec 27 19:27:33 2024 +0800 [fix] make create table and load data procedure easily (#1603) ## Versions - [x] dev - [x] 3.0 - [x] 2.1 - [ ] 2.0 ## Languages - [x] Chinese - [x] English ## Docs Checklist - [ ] Checked by AI - [ ] Test Cases Built --- docs/query-data/window-function.md| 19 ++- .../current/query-data/window-function.md | 19 ++- .../version-2.1/query-data/window-function.md | 19 ++- .../version-3.0/query-data/window-function.md | 19 ++- .../version-2.1/query-data/window-function.md | 19 ++- .../version-3.0/query-data/window-function.md | 19 ++- 6 files changed, 84 insertions(+), 30 deletions(-) diff --git a/docs/query-data/window-function.md b/docs/query-data/window-function.md index d1b09006f26..5df4ab55ede 100644 --- a/docs/query-data/window-function.md +++ b/docs/query-data/window-function.md @@ -583,9 +583,12 @@ For more information on analytic functions, refer to the Oracle official documen ## Reference -Create table and load data : +The table creation statement used in the example is as follows: ```sql +CREATE DATABASE IF NOT EXISTS doc_tpcds; +USE doc_tpcds; + CREATE TABLE IF NOT EXISTS item ( i_item_sk bigint not null, i_item_id char(16) not null, @@ -703,33 +706,39 @@ DISTRIBUTED BY HASH(ca_address_sk) BUCKETS 12 PROPERTIES ( "replication_num" = "1" ); +``` + +Execute the following command on the terminal to download the data to the local computer and load the data into the table using the Stream Load method: + +```shell +curl -L https://cdn.selectdb.com/static/doc_ddl_dir_d27a752a7b.tar -o - | tar -Jxf - curl --location-trusted \ -u "root:" \ -H "column_separator:|" \ -H "columns: i_item_sk, i_item_id, i_rec_start_date, i_rec_end_date, i_item_desc, i_current_price, i_wholesale_cost, i_brand_id, i_brand, i_class_id, i_class, i_category_id, i_category, i_manufact_id, i_manufact, i_size, i_formulation, i_color, i_units, i_container, i_manager_id, i_product_name" \ --T "/path/to/data/item_1_10.dat" \ +-T "doc_ddl_dir/item_1_10.dat" \ http://127.0.0.1:8030/api/doc_tpcds/item/_stream_load curl --location-trusted \ -u "root:" \ -H "column_separator:|" \ -H "columns: d_date_sk, d_date_id, d_date, d_month_seq, d_week_seq, d_quarter_seq, d_year, d_dow, d_moy, d_dom, d_qoy, d_fy_year, d_fy_quarter_seq, d_fy_week_seq, d_day_name, d_quarter_name, d_holiday, d_weekend, d_following_holiday, d_first_dom, d_last_dom, d_same_day_ly, d_same_day_lq, d_current_day, d_current_week, d_current_month, d_current_quarter, d_current_year" \ --T "/path/to/data/date_dim_1_10.dat" \ +-T "doc_ddl_dir/date_dim_1_10.dat" \ http://127.0.0.1:8030/api/doc_tpcds/date_dim/_stream_load curl --location-trusted \ -u "root:" \ -H "column_separator:|" \ -H "columns: ss_sold_date_sk, ss_sold_time_sk, ss_item_sk, ss_customer_sk, ss_cdemo_sk, ss_hdemo_sk, ss_addr_sk, ss_store_sk, ss_promo_sk, ss_ticket_number, ss_quantity, ss_wholesale_cost, ss_list_price, ss_sales_price, ss_ext_discount_amt, ss_ext_sales_price, ss_ext_wholesale_cost, ss_ext_list_price, ss_ext_tax, ss_coupon_amt, ss_net_paid, ss_net_paid_inc_tax, ss_net_profit" \ --T "/path/to/data/store_sales.csv" \ +-T "doc_ddl_dir/store_sales.csv" \ http://127.0.0.1:8030/api/doc_tpcds/store_sales/_stream_load curl --location-trusted \ -u "root:" \ -H "column_separator:|" \ -H "ca_address_sk, ca_address_id, ca_street_number, ca_street_name, ca_street_type, ca_suite_number, ca_city, ca_county, ca_state, ca_zip, ca_country, ca_gmt_offset, ca_location_type" \ --T "/path/to/data/customer_address_1_10.dat" \ +-T "doc_ddl_dir/customer_address_1_10.dat" \ http://127.0.0.1:8030/api/doc_tpcds/customer_address/_stream_load ``` diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/query-data/window-function.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/query-data/window-function.md index a74ab33daf4..175b7b472b1 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/query-data/window-function.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/query-data/window-function.md @@ -575,9 +575,12 @@ FROM ## 附录 -建表和加载数据: +示例中使用到的表的建表语句如下: ```sql +CREATE DATABASE IF NOT EXISTS doc_tpcds; +USE doc_tpcds; + CREATE TABLE IF NOT EXISTS item ( i_item_sk bigint not null, i_item_id char(
(doris) branch master updated: [fix](cluster key) skip unsupported case (#45579)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new c596e81416c [fix](cluster key) skip unsupported case (#45579) c596e81416c is described below commit c596e81416cfe84b9a861f7bf0bf0cba188fde1a Author: meiyi AuthorDate: Fri Dec 27 19:27:33 2024 +0800 [fix](cluster key) skip unsupported case (#45579) --- .../src/main/groovy/org/apache/doris/regression/Config.groovy | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/regression-test/framework/src/main/groovy/org/apache/doris/regression/Config.groovy b/regression-test/framework/src/main/groovy/org/apache/doris/regression/Config.groovy index 246558eaf06..a852441e7f8 100644 --- a/regression-test/framework/src/main/groovy/org/apache/doris/regression/Config.groovy +++ b/regression-test/framework/src/main/groovy/org/apache/doris/regression/Config.groovy @@ -1028,7 +1028,9 @@ class Config { excludeDirectorySet.add("fault_injection_p0/partial_update") excludeDirectorySet.add("fault_injection_p0/flexible") excludeDirectorySet.add("doc") -excludeDirectorySet.add("schema_change_p0/unique_ck") +excludeDirectorySet.add("schema_change") +excludeDirectorySet.add("schema_change_p0") +excludeDirectorySet.add("schema_change_p2") List excludeCases = ["test_table_properties", "test_create_table" , "test_default_hll", "test_default_pi", "test_default_bitmap_empty" , "test_full_compaction", "test_full_compaction_by_table_id" - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] update Installation - delete socket checking [doris-website]
KassieZ merged PR #1638: URL: https://github.com/apache/doris-website/pull/1638 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch master updated: update Installation - delete socket checking (#1638)
This is an automated email from the ASF dual-hosted git repository. kassiez pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new e08f509300a update Installation - delete socket checking (#1638) e08f509300a is described below commit e08f509300a83d3c0f60eafa57158fcdd1e107f0 Author: wangtianyi2004 <376612...@qq.com> AuthorDate: Fri Dec 27 19:31:21 2024 +0800 update Installation - delete socket checking (#1638) ## Versions - [ ] dev - [ ] 3.0 - [ ] 2.1 - [ ] 2.0 ## Languages - [ ] Chinese - [ ] English ## Docs Checklist - [ ] Checked by AI - [ ] Test Cases Built --- docs/install/preparation/os-checking.md | 1 - .../current/install/preparation/os-checking.md| 4 +--- .../version-2.1/install/preparation/os-checking.md| 4 +--- .../version-3.0/install/preparation/os-checking.md| 4 +--- versioned_docs/version-2.1/install/preparation/os-checking.md | 1 - versioned_docs/version-3.0/install/preparation/os-checking.md | 1 - 6 files changed, 3 insertions(+), 12 deletions(-) diff --git a/docs/install/preparation/os-checking.md b/docs/install/preparation/os-checking.md index 728b9e0bd8e..e6e80a13599 100644 --- a/docs/install/preparation/os-checking.md +++ b/docs/install/preparation/os-checking.md @@ -31,7 +31,6 @@ When deploying Doris, the following operating system items need to be checked: - Ensure the system has a sufficiently large virtual memory area - Ensure CPU is not using power-saving mode - Ensure network connections automatically reset new connections when overflow occurs -- Ensure socket queue is sufficiently large - Ensure Doris-related ports are open or the system firewall is disabled - Ensure the system has a sufficiently large number of open file descriptors - Ensure cluster deployment machines have NTP service installed diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/install/preparation/os-checking.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/install/preparation/os-checking.md index 9cb777d9462..4d97d1810b1 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/install/preparation/os-checking.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/install/preparation/os-checking.md @@ -35,9 +35,7 @@ under the License. - 确保 CPU 不使用省电模式 - 确保网络连接溢出时自动重置新连接 - -- 确保 socket 队列足够大 - + - 确保 Doris 相关端口畅通或关闭系统防火墙 - 确保系统有足够大的打开文件句柄数 diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/install/preparation/os-checking.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/install/preparation/os-checking.md index 9cb777d9462..4d97d1810b1 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/install/preparation/os-checking.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/install/preparation/os-checking.md @@ -35,9 +35,7 @@ under the License. - 确保 CPU 不使用省电模式 - 确保网络连接溢出时自动重置新连接 - -- 确保 socket 队列足够大 - + - 确保 Doris 相关端口畅通或关闭系统防火墙 - 确保系统有足够大的打开文件句柄数 diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/install/preparation/os-checking.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/install/preparation/os-checking.md index 9cb777d9462..4d97d1810b1 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/install/preparation/os-checking.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/install/preparation/os-checking.md @@ -35,9 +35,7 @@ under the License. - 确保 CPU 不使用省电模式 - 确保网络连接溢出时自动重置新连接 - -- 确保 socket 队列足够大 - + - 确保 Doris 相关端口畅通或关闭系统防火墙 - 确保系统有足够大的打开文件句柄数 diff --git a/versioned_docs/version-2.1/install/preparation/os-checking.md b/versioned_docs/version-2.1/install/preparation/os-checking.md index 728b9e0bd8e..e6e80a13599 100644 --- a/versioned_docs/version-2.1/install/preparation/os-checking.md +++ b/versioned_docs/version-2.1/install/preparation/os-checking.md @@ -31,7 +31,6 @@ When deploying Doris, the following operating system items need to be checked: - Ensure the system has a sufficiently large virtual memory area - Ensure CPU is not using power-saving mode - Ensure network connections automatically reset new connections when overflow occurs -- Ensure socket queue is sufficiently large - Ensure Doris-related ports are open or the system firewall is disabled - Ensure the system has a sufficiently large number of open file descriptors - Ensure cluster deployment machines have NTP service installed diff --git a/versioned_docs/version-3.0/install/preparation/os-checking.md b/versioned_docs/version-3.0/install/preparation/os-checking.md index 728b9e0bd8e..e6e80a13599 100644 --- a/versioned_docs/version-3.0/install/preparation/os-checking.md +++ b/versioned_docs/version-3.0/install/p
Re: [PR] branch-3.0: [regression-test](fix) remove interrupt() in test_routine_load_eof.groovy #46044 [doris]
dataroaring merged PR #46097: URL: https://github.com/apache/doris/pull/46097 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-46044-branch-3.0 deleted (was 0a1e5fc8143)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-46044-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git was 0a1e5fc8143 [regression-test](fix) remove interrupt() in test_routine_load_eof.groovy (#46044) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Fix http url error [doris-website]
KassieZ merged PR #1630: URL: https://github.com/apache/doris-website/pull/1630 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch master updated (2babf360221 -> a662eb3dded)
This is an automated email from the ASF dual-hosted git repository. kassiez pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git from 2babf360221 [fix] make create table and load data procedure easily (#1603) add a662eb3dded Fix http url error (#1630) No new revisions were added by this update. Summary of changes: .../memory-management/memory-analysis/doris-cache-memory-analysis.md | 4 ++-- .../memory-management/memory-analysis/metadata-memory-analysis.md | 4 ++-- docs/admin-manual/memory-management/overview.md | 2 +- .../memory-management/memory-analysis/doris-cache-memory-analysis.md | 4 ++-- .../memory-management/memory-analysis/metadata-memory-analysis.md | 4 ++-- .../current/admin-manual/memory-management/overview.md| 2 +- .../memory-management/memory-analysis/doris-cache-memory-analysis.md | 4 ++-- .../memory-management/memory-analysis/metadata-memory-analysis.md | 4 ++-- .../version-2.1/admin-manual/memory-management/overview.md| 2 +- .../memory-management/memory-analysis/doris-cache-memory-analysis.md | 4 ++-- .../memory-management/memory-analysis/metadata-memory-analysis.md | 4 ++-- .../version-3.0/admin-manual/memory-management/overview.md| 2 +- .../memory-management/memory-analysis/doris-cache-memory-analysis.md | 4 ++-- .../memory-management/memory-analysis/metadata-memory-analysis.md | 4 ++-- versioned_docs/version-2.1/admin-manual/memory-management/overview.md | 2 +- .../memory-management/memory-analysis/doris-cache-memory-analysis.md | 4 ++-- .../memory-management/memory-analysis/metadata-memory-analysis.md | 4 ++-- versioned_docs/version-3.0/admin-manual/memory-management/overview.md | 2 +- 18 files changed, 30 insertions(+), 30 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) fix cast string to date [doris]
doris-robot commented on PR #46065: URL: https://github.com/apache/doris/pull/46065#issuecomment-2563556755 ClickBench: Total hot run time: 31.25 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 78829e45659d236c801d0ce3b7c069480dc4d85b, data reload: false query1 0.030.030.04 query2 0.070.030.03 query3 0.240.070.07 query4 1.610.100.10 query5 0.400.420.41 query6 1.140.660.64 query7 0.030.020.01 query8 0.040.030.03 query9 0.580.510.51 query10 0.560.560.56 query11 0.150.110.12 query12 0.130.120.11 query13 0.600.600.59 query14 2.842.732.71 query15 0.890.830.84 query16 0.400.390.39 query17 1.041.041.01 query18 0.230.220.21 query19 1.951.831.98 query20 0.010.010.02 query21 15.38 0.930.55 query22 0.750.700.65 query23 15.36 1.530.61 query24 3.541.071.38 query25 0.210.050.18 query26 0.370.160.15 query27 0.050.050.05 query28 13.27 1.581.05 query29 12.56 3.903.22 query30 0.240.100.06 query31 2.830.600.37 query32 3.230.550.47 query33 3.093.063.06 query34 16.72 5.164.49 query35 4.474.464.42 query36 0.660.490.48 query37 0.090.060.06 query38 0.050.040.04 query39 0.030.020.03 query40 0.160.140.13 query41 0.070.020.03 query42 0.040.030.02 query43 0.040.030.02 Total cold run time: 106.15 s Total hot run time: 31.25 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [fix](jdbc catalog) Get statistics using remote name #46040 [doris]
Thearas commented on PR #46116: URL: https://github.com/apache/doris/pull/46116#issuecomment-2563543644 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Profile](scan) Add profile to detail the scan [doris]
doris-robot commented on PR #46110: URL: https://github.com/apache/doris/pull/46110#issuecomment-2563570378 TeamCity be ut coverage result: Function Coverage: 38.54% (8308/21555) Line Coverage: 30.23% (68693/227205) Region Coverage: 29.67% (35383/119273) Branch Coverage: 25.43% (18186/71526) Coverage Report: http://coverage.selectdb-in.cc/coverage/4d0bd4ddfff995ea3baedbc18d9fe72c78603e37_4d0bd4ddfff995ea3baedbc18d9fe72c78603e37/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nerieds)Push encode slot [doris]
doris-robot commented on PR #45958: URL: https://github.com/apache/doris/pull/45958#issuecomment-2563580683 TPC-DS: Total hot run time: 197229 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 2a8d67bb6e2c3d0b5693644992c2446063c4bc8f, data reload: false query1 12971017926 926 query2 6464235723072307 query3 11130 476348234763 query4 32748 23782 23329 23329 query5 3961608 474 474 query6 293 206 186 186 query7 3984528 307 307 query8 311 267 241 241 query9 9467274127222722 query10 470 319 246 246 query11 17864 15381 15487 15381 query12 158 104 101 101 query13 1671562 412 412 query14 11151 801871877187 query15 256 233 196 196 query16 7441678 493 493 query17 1568751 600 600 query18 1374397 339 339 query19 269 187 165 165 query20 125 110 111 110 query21 205 137 109 109 query22 4726447144104410 query23 34484 33688 33577 33577 query24 6364228223072282 query25 482 453 379 379 query26 793 305 155 155 query27 1986489 334 334 query28 5402249124542454 query29 549 555 410 410 query30 204 189 148 148 query31 999 900 815 815 query32 83 55 56 55 query33 497 364 292 292 query34 806 927 533 533 query35 822 821 762 762 query36 10611094973 973 query37 123 111 79 79 query38 4276430842054205 query39 1511144814721448 query40 203 113 102 102 query41 45 42 48 42 query42 118 106 118 106 query43 518 510 498 498 query44 1374832 842 832 query45 188 172 170 170 query46 959 1156662 662 query47 2029196819531953 query48 389 434 324 324 query49 716 494 397 397 query50 695 703 403 403 query51 7255758572207220 query52 99 102 96 96 query53 233 263 190 190 query54 491 521 420 420 query55 89 78 81 78 query56 252 261 263 261 query57 1223124011771177 query58 236 224 252 224 query59 3266335930253025 query60 277 266 257 257 query61 111 105 113 105 query62 913 824 749 749 query63 226 191 188 188 query64 35061027662 662 query65 3298323732753237 query66 839 432 302 302 query67 16681 15684 15574 15574 query68 9807778 511 511 query69 473 302 247 247 query70 1231112011391120 query71 431 281 255 255 query72 6029386038853860 query73 666 788 347 347 query74 10120 935689848984 query75 4576316226182618 query76 53881279810 810 query77 936 365 273 273 query78 10030 10295 96519651 query79 2487935 608 608 query80 730 511 528 511 query81 480 274 236 236 query82 602 148 131 131 query83 156 165 145 145 query84 238 88 77 77 query85 777 371 298 298 query86 357 315 310 310 query87 4430455945374537 query88 3342223222572232 query89 430 327 297 297 query90 1871182 181 181 query91 128 138 106 106 query92 60 56 50 50 query93 1064898 514 514 query94 655 419 289 289 query95 321 266 251 251 query96 486 673 289 289 query97 2738281027132713 query98 225 202 194 194 query99 1657157014251425 Total cold run time: 296525 ms Total hot run time: 197229 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] branch-3.0: [Fix](PreparedStatement) nondeterministic functions in prepared statement should not be short circuited #46003 [doris]
doris-robot commented on PR #46108: URL: https://github.com/apache/doris/pull/46108#issuecomment-2563580319 ClickBench: Total hot run time: 33.35 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 63c916a907667ebdb829606ac622d2f14fdd6bbf, data reload: false query1 0.030.030.03 query2 0.060.030.03 query3 0.230.060.06 query4 1.630.100.11 query5 0.530.530.50 query6 1.140.730.74 query7 0.020.020.02 query8 0.040.030.03 query9 0.560.500.52 query10 0.550.560.58 query11 0.150.110.10 query12 0.140.100.12 query13 0.620.600.59 query14 2.922.932.94 query15 0.890.830.82 query16 0.390.390.39 query17 1.061.031.03 query18 0.240.210.22 query19 1.951.901.96 query20 0.010.010.01 query21 15.35 0.580.59 query22 2.292.932.70 query23 17.02 0.930.77 query24 2.941.690.58 query25 0.320.190.04 query26 0.350.140.14 query27 0.040.050.04 query28 10.59 1.101.07 query29 12.53 3.203.23 query30 0.260.060.06 query31 2.860.390.38 query32 3.240.460.47 query33 3.022.993.06 query34 17.10 4.464.42 query35 4.474.494.50 query36 0.670.480.49 query37 0.090.060.06 query38 0.050.040.04 query39 0.030.030.02 query40 0.160.130.13 query41 0.080.020.02 query42 0.030.020.02 query43 0.040.030.02 Total cold run time: 106.69 s Total hot run time: 33.35 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](metrics) Unify metrics of thread pool [doris]
doris-robot commented on PR #43144: URL: https://github.com/apache/doris/pull/43144#issuecomment-2563580567 TPC-DS: Total hot run time: 190439 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 5e27cf55d6ca7d5a38c27ed4ad03a9665acaa154, data reload: false query1 1010379 388 379 query2 6514232422522252 query3 6710211 217 211 query4 33766 23952 23543 23543 query5 4316612 453 453 query6 281 205 192 192 query7 4639497 301 301 query8 311 252 248 248 query9 9578279027892789 query10 477 346 257 257 query11 18592 15294 15248 15248 query12 175 108 112 108 query13 1672550 421 421 query14 12168 766371707170 query15 248 193 189 189 query16 8096584 423 423 query17 1563764 578 578 query18 2135441 291 291 query19 224 180 146 146 query20 115 110 110 110 query21 209 123 105 105 query22 4535461243414341 query23 34637 33276 33297 33276 query24 6140225021982198 query25 502 435 386 386 query26 1207273 151 151 query27 2031448 358 358 query28 5062249224562456 query29 751 556 415 415 query30 222 181 146 146 query31 1083901 816 816 query32 88 61 59 59 query33 500 357 298 298 query34 783 855 522 522 query35 805 806 739 739 query36 995 1053967 967 query37 115 101 71 71 query38 4216426941264126 query39 1512142014181418 query40 203 128 105 105 query41 47 46 45 45 query42 118 104 100 100 query43 518 530 496 496 query44 1347807 802 802 query45 181 172 167 167 query46 858 1030648 648 query47 1905189218781878 query48 391 404 329 329 query49 781 479 404 404 query50 627 654 400 400 query51 7227709371097093 query52 106 99 93 93 query53 221 253 181 181 query54 477 480 420 420 query55 83 76 77 76 query56 248 277 260 260 query57 1177116211281128 query58 243 229 231 229 query59 3051324429422942 query60 277 258 250 250 query61 109 110 111 110 query62 857 830 751 751 query63 227 192 192 192 query64 45921032658 658 query65 3296319332683193 query66 836 419 304 304 query67 15917 15892 15647 15647 query68 9830769 530 530 query69 450 286 265 265 query70 1243115110291029 query71 444 295 259 259 query72 5798383038723830 query73 680 741 366 366 query74 10091 901488258825 query75 4648309126592659 query76 55391208778 778 query77 1109364 275 275 query78 10240 10167 93279327 query79 3392904 588 588 query80 690 513 431 431 query81 462 279 229 229 query82 629 154 127 127 query83 194 164 224 164 query84 292 88 75 75 query85 782 362 306 306 query86 355 317 303 303 query87 4349458744534453 query88 4453223522412235 query89 400 333 310 310 query90 1904189 188 188 query91 132 146 105 105 query92 64 56 52 52 query93 1207892 544 544 query94 651 399 286 286 query95 336 257 247 247 query96 492 609 291 291 query97 2724278627002700 query98 235 210 199 199 query99 1679154414051405 Total cold run time: 299397 ms Total hot run time: 190439 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] branch-3.0: [test](nereids) add test for normalize sort #45890 [doris]
doris-robot commented on PR #46111: URL: https://github.com/apache/doris/pull/46111#issuecomment-2563580422 ClickBench: Total hot run time: 33.86 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 6beabab5744a6c7a2c3026ed32fd6942a78aca33, data reload: false query1 0.040.040.03 query2 0.060.030.03 query3 0.230.070.07 query4 1.630.100.10 query5 0.510.490.51 query6 1.130.720.73 query7 0.020.020.02 query8 0.040.030.03 query9 0.580.510.49 query10 0.550.560.56 query11 0.140.100.11 query12 0.130.110.12 query13 0.610.600.60 query14 3.063.032.95 query15 0.900.830.83 query16 0.390.380.39 query17 1.041.081.00 query18 0.240.210.22 query19 1.881.821.99 query20 0.010.010.01 query21 15.38 0.570.57 query22 2.611.972.09 query23 16.96 0.910.79 query24 2.991.911.90 query25 0.240.030.11 query26 0.590.140.14 query27 0.040.040.05 query28 9.081.101.06 query29 12.55 3.193.20 query30 0.250.060.06 query31 2.850.380.38 query32 3.270.490.47 query33 2.992.983.05 query34 16.90 4.454.49 query35 4.594.494.51 query36 0.660.490.49 query37 0.090.060.06 query38 0.050.030.03 query39 0.040.020.03 query40 0.160.120.13 query41 0.080.020.02 query42 0.040.020.02 query43 0.030.030.03 Total cold run time: 105.63 s Total hot run time: 33.86 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0 [Fix](Catalog)Standardize the use of authentication attributes in the Catalog to avoid creating a large number of Authenticator objects. ( [doris]
doris-robot commented on PR #46107: URL: https://github.com/apache/doris/pull/46107#issuecomment-2563580449 ClickBench: Total hot run time: 33.63 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 220fc68fffebaed2f9546626ba5614c876578420, data reload: false query1 0.030.040.03 query2 0.080.030.02 query3 0.220.070.06 query4 1.620.100.10 query5 0.530.530.52 query6 1.130.730.72 query7 0.020.020.02 query8 0.040.030.03 query9 0.570.510.48 query10 0.560.550.56 query11 0.140.110.11 query12 0.150.120.12 query13 0.620.590.60 query14 2.932.892.94 query15 0.900.830.83 query16 0.390.370.38 query17 1.071.091.00 query18 0.230.230.22 query19 1.981.931.96 query20 0.010.010.02 query21 15.37 0.600.58 query22 2.792.502.03 query23 16.80 1.230.87 query24 3.501.492.41 query25 0.260.060.17 query26 0.580.150.14 query27 0.050.050.04 query28 8.861.121.07 query29 12.58 3.203.18 query30 0.240.060.05 query31 2.870.410.40 query32 3.230.470.47 query33 3.013.063.01 query34 16.83 4.444.40 query35 4.584.464.45 query36 0.680.490.49 query37 0.100.060.07 query38 0.050.040.03 query39 0.030.020.02 query40 0.150.120.13 query41 0.080.020.02 query42 0.030.030.02 query43 0.040.030.03 Total cold run time: 105.93 s Total hot run time: 33.63 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](sql) Wrong result for alter set property [doris]
dataroaring merged PR #46028: URL: https://github.com/apache/doris/pull/46028 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [fix](sql) Wrong result for alter set property (#46028)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new 4e92bd61586 [fix](sql) Wrong result for alter set property (#46028) 4e92bd61586 is described below commit 4e92bd615867617b1ae2b13e906c3a3b7beef8e2 Author: Uniqueyou AuthorDate: Fri Dec 27 19:14:44 2024 +0800 [fix](sql) Wrong result for alter set property (#46028) Related PR: https://github.com/selectdb/ccr-syncer/pull/353 --- .../doris/persist/ModifyTablePropertyOperationLog.java | 7 ++- .../doris/persist/ModifyDynamicPartitionInfoTest.java | 13 + 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/persist/ModifyTablePropertyOperationLog.java b/fe/fe-core/src/main/java/org/apache/doris/persist/ModifyTablePropertyOperationLog.java index 50b49074ce1..1d64029bcfd 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/persist/ModifyTablePropertyOperationLog.java +++ b/fe/fe-core/src/main/java/org/apache/doris/persist/ModifyTablePropertyOperationLog.java @@ -55,7 +55,8 @@ public class ModifyTablePropertyOperationLog implements Writable { StringBuilder sb = new StringBuilder(); sb.append("SET ("); for (Map.Entry entry : properties.entrySet()) { - sb.append(entry.getKey()).append("=").append(entry.getValue()).append(","); +sb.append("\"").append(entry.getKey()).append("\"").append(" = ").append("\"").append(entry.getValue()) +.append("\" ").append(","); } sb.deleteCharAt(sb.length() - 1); // remove last ',' sb.append(")"); @@ -106,4 +107,8 @@ public class ModifyTablePropertyOperationLog implements Writable { public String toJson() { return GsonUtils.GSON.toJson(this); } + +public String toSql() { +return sql; +} } diff --git a/fe/fe-core/src/test/java/org/apache/doris/persist/ModifyDynamicPartitionInfoTest.java b/fe/fe-core/src/test/java/org/apache/doris/persist/ModifyDynamicPartitionInfoTest.java index bff50dcf768..a54f91dd826 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/persist/ModifyDynamicPartitionInfoTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/persist/ModifyDynamicPartitionInfoTest.java @@ -67,4 +67,17 @@ public class ModifyDynamicPartitionInfoTest { Assert.assertEquals(readModifyDynamicPartitionInfo.getProperties(), properties); in.close(); } + +@Test +public void testToSql() { +HashMap properties = new HashMap<>(); +properties.put(DynamicPartitionProperty.ENABLE, "true"); +properties.put(DynamicPartitionProperty.TIME_UNIT, "day"); +properties.put(DynamicPartitionProperty.START, "-3"); +ModifyTablePropertyOperationLog modifyDynamicPartitionInfo = new ModifyTablePropertyOperationLog(100L, 200L, +"test", properties); + Assert.assertTrue(modifyDynamicPartitionInfo.toSql().contains("\"dynamic_partition.enable\" = \"true\"")); + Assert.assertTrue(modifyDynamicPartitionInfo.toSql().contains("\"dynamic_partition.time_unit\" = \"day\"")); + Assert.assertTrue(modifyDynamicPartitionInfo.toSql().contains("\"dynamic_partition.start\" = \"-3\"")); +} } - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch master updated: [doc](mtmv) Async materialized view doc optimize (#1473)
This is an automated email from the ASF dual-hosted git repository. kassiez pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new f2402bc79af [doc](mtmv) Async materialized view doc optimize (#1473) f2402bc79af is described below commit f2402bc79af090e76360d15cbef52bcb9e4d123b Author: seawinde AuthorDate: Fri Dec 27 19:32:30 2024 +0800 [doc](mtmv) Async materialized view doc optimize (#1473) ## Versions Need preview - [x] dev - [ ] 3.0 - [ ] 2.1 - [ ] 2.0 ## Languages - [x] Chinese - [ ] English ## Docs Checklist - [ ] Checked by AI - [ ] Test Cases Built --- .../async-materialized-view/faq.md | 32 +- .../functions-and-demands.md | 745 - .../async-materialized-view/overview.md| 548 ++- .../async-materialized-view/use-guide.md | 478 - .../Create/CREATE-ASYNC-MATERIALIZED-VIEW.md | 5 + 5 files changed, 938 insertions(+), 870 deletions(-) diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/query-acceleration/materialized-view/async-materialized-view/faq.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/query-acceleration/materialized-view/async-materialized-view/faq.md index 72068e39c7c..53e066e5149 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/query-acceleration/materialized-view/async-materialized-view/faq.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/query-acceleration/materialized-view/async-materialized-view/faq.md @@ -92,7 +92,7 @@ Unable to find a suitable base table for partitioning - 最新的代码可以提示分区构建失败的原因,原因摘要和说明见附录 2 -**举例如下:** +**例如:** ```sql CREATE TABLE IF NOT EXISTS orders ( @@ -108,7 +108,7 @@ CREATE TABLE IF NOT EXISTS orders ( ) DUPLICATE KEY(o_orderkey, o_custkey) PARTITION BY RANGE(o_orderdate) ( FROM ('2024-05-01') TO ('2024-06-30') INTERVAL 1 DAY -) DISTRIBUTED BY HASH(o_orderkey) BUCKETS 3 PROPERTIES ("replication_num" = "1"); +) DISTRIBUTED BY HASH(o_orderkey) BUCKETS 3; CREATE TABLE IF NOT EXISTS lineitem ( @@ -131,7 +131,7 @@ CREATE TABLE IF NOT EXISTS lineitem ( ) DUPLICATE KEY( l_orderkey, l_partkey, l_suppkey, l_linenumber -) DISTRIBUTED BY HASH(l_orderkey) BUCKETS 3 PROPERTIES ("replication_num" = "1"); +) DISTRIBUTED BY HASH(l_orderkey) BUCKETS 3; ``` 物化视图定义如下,可以进行分区增量更新。如果选择`orders.o_orderdate`作为物化视图的分区字段,那么它是可以支持增量分区更新的。相反,如果使用了`lineitem.l_shipdate`,则不能实现增量更新。 @@ -143,7 +143,12 @@ CREATE TABLE IF NOT EXISTS lineitem ( 2. `lineitem.l_shipdate`是`outer join`操作中产生`null`值那一端的列。 ```sql -CREATE MATERIALIZED VIEW mv_1 BUILD IMMEDIATE REFRESH AUTO ON MANUAL partition by(o_orderdate) DISTRIBUTED BY RANDOM BUCKETS 2 PROPERTIES ('replication_num' = '1') AS +CREATE MATERIALIZED VIEW mv_1 + BUILD IMMEDIATE + REFRESH AUTO ON MANUAL + partition by(o_orderdate) + DISTRIBUTED BY RANDOM BUCKETS 2 + AS SELECT l_linestatus, sum( @@ -193,7 +198,6 @@ CREATE MATERIALIZED VIEW mv11 BUILD IMMEDIATE REFRESH AUTO ON MANUAL partition by(l_shipdate) DISTRIBUTED BY HASH(l_orderkey) BUCKETS 10 -PROPERTIES ('replication_num' = '1') AS SELECT l_shipdate, l_orderkey, O_ORDERDATE, count(*) FROM lineitem @@ -221,7 +225,7 @@ GROUP BY l_shipdate, l_orderkey, O_ORDERDATE; - 如果 explain 最后没有出现 `MaterializedView` 等信息,那么意味着此物化视图状态不可用,因此不能参与透明改写。(关于什么情况下会导致物化视图状态不可用,可详细参考使用与实践 - 查看物化视图状态)。 -举例如下: +例如: ```sql | MaterializedView | @@ -362,19 +366,7 @@ Explain 显示的信息如下: ## 附录 -### 1 物化视图相关开关介绍 - -| 开关 | 说明 | -| | | -| SET enable_nereids_planner = true; | 异步物化视图只有在新优化器下才支持,所以物化视图透明改写没有生效时,需要开启新优化器 | -| SET enable_materialized_view_rewrite = true; | 开启或者关闭查询透明改写,默认开启 | -| SET materialized_view_rewrite_enable_contain_external_table = true; | 参与透明改写的物化视图是否允许包含外表,默认不允许,如果物化视图的定义 SQL 中包含外表,也想参与到透明改写,可以打开此开关。 | -| SET materialized_view_rewrite_success_candidate_num = 3; | 透明改写成功的结果集合,允许参与到 CBO 候选的最大数量,默认是 3。如果发现透明改写的性能很慢,可以考虑把这个值调小。 | -| SET enable_materialized_view_union_rewrite = true; | 当分区物化视图不足以提供查询的全部数据时,是否允许基表和物化视图 union all 来响应查询,默认允许。如果发现命中物化视图时数据错误,可以把这个开关关闭。 | -| SET enable_materialized_view_nest_rewrite = true;| 是否允许嵌套改写,默认不允许。如果查询 SQL 很复杂,需要构建嵌套物化视图才可以命中,那么需要打开此开关。 | -| SET materialized_view_relation_mapping_max_count = 8;| 透明改写过程中,relation mapping 最大允许数量,如果超过,进行截取。relation mapping 通常由表自关联产生,数量一般会是笛卡尔积,比如 3 张表,可能会产生 8 种组合。默认是 8
(doris) branch master updated (c596e81416c -> 22400530cc0)
This is an automated email from the ASF dual-hosted git repository. zhangchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from c596e81416c [fix](cluster key) skip unsupported case (#45579) add 22400530cc0 [Fix](merge-on-write) Should update pending delete bitmap KVs in MS when no need to calc delete bitmaps in publish phase (#46039) No new revisions were added by this update. Summary of changes: .../cloud/cloud_engine_calc_delete_bitmap_task.cpp | 9 +- be/src/cloud/cloud_tablet.cpp | 38 -- be/src/cloud/cloud_tablet.h| 3 + cloud/src/meta-service/meta_service.cpp| 1 + cloud/src/meta-service/meta_service_txn.cpp| 31 + cloud/test/meta_service_test.cpp | 139 - gensrc/proto/cloud.proto | 2 + ...ending_delete_bitmaps_removed_by_other_txn.out} | 2 +- ...ing_delete_bitmaps_removed_by_other_txn.groovy} | 59 + 9 files changed, 242 insertions(+), 42 deletions(-) copy regression-test/data/fault_injection_p0/cloud/{test_cloud_mow_insert_timeout.out => test_cloud_pending_delete_bitmaps_removed_by_other_txn.out} (93%) copy regression-test/suites/fault_injection_p0/cloud/{test_cloud_mow_stale_resp_load_load_conflict.groovy => test_cloud_pending_delete_bitmaps_removed_by_other_txn.groovy} (53%) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch master updated (f2402bc79af -> 1d006c92df2)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git from f2402bc79af [doc](mtmv) Async materialized view doc optimize (#1473) add 1d006c92df2 refine compute group en docs (#1649) No new revisions were added by this update. Summary of changes: .../workload-management/compute-group.md | 59 +- .../workload-management/compute-group.md | 59 +- 2 files changed, 70 insertions(+), 48 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-3.0 updated: branch-3.0: [regression-test](fix) remove interrupt() in test_routine_load_eof.groovy #46044 (#46097)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a commit to branch branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-3.0 by this push: new bf74831f46c branch-3.0: [regression-test](fix) remove interrupt() in test_routine_load_eof.groovy #46044 (#46097) bf74831f46c is described below commit bf74831f46c9e879d94d98e14b321f0dedbfb5ee Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> AuthorDate: Fri Dec 27 19:31:45 2024 +0800 branch-3.0: [regression-test](fix) remove interrupt() in test_routine_load_eof.groovy #46044 (#46097) Cherry-picked from #46044 Co-authored-by: shuke --- regression-test/suites/load_p0/routine_load/test_routine_load_eof.groovy | 1 - 1 file changed, 1 deletion(-) diff --git a/regression-test/suites/load_p0/routine_load/test_routine_load_eof.groovy b/regression-test/suites/load_p0/routine_load/test_routine_load_eof.groovy index 6eeb9a4e51c..5e61a3ec1d0 100644 --- a/regression-test/suites/load_p0/routine_load/test_routine_load_eof.groovy +++ b/regression-test/suites/load_p0/routine_load/test_routine_load_eof.groovy @@ -173,6 +173,5 @@ suite("test_routine_load_eof","p0") { sql "stop routine load for ${jobName}" sql "DROP TABLE IF EXISTS ${tableName}" } -thread.interrupt() } } \ No newline at end of file - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [test](doc) add job-scheduler example in doris's doc to regression test #42904 [doris]
dataroaring commented on PR #45967: URL: https://github.com/apache/doris/pull/45967#issuecomment-2563607504 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](merge-on-write) Should update pending delete bitmap KVs in MS when no need to calc delete bitmaps in publish phase [doris]
zhannngchen merged PR #46039: URL: https://github.com/apache/doris/pull/46039 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [doc](mtmv) Async materialized view doc optimize [doris-website]
KassieZ merged PR #1473: URL: https://github.com/apache/doris-website/pull/1473 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] refine compute group en docs [doris-website]
yiguolei merged PR #1649: URL: https://github.com/apache/doris-website/pull/1649 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-42904-branch-3.0 updated (99f58c7a0c0 -> 1e006eecbe0)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a change to branch auto-pick-42904-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git discard 99f58c7a0c0 [test](doc) add job-scheduler example in doris's doc to regression test (#42904) add aa496914ff7 branch-3.0: [opt](cloud) Make get tablet stats and update delete bitmap update lock be able to be in different fdb txns #45206 (#45559) add c04a8da75bc branch-3.0: [fix](vault) Fix bugs about altering storage vault name #45685 (#45963) add 9f2097c882e branch-3.0: [fix](cloud) fix abort transaction in runningTxns list when show routine load #45629 (#45721) add 4ea5b16105e branch-3.0: [fix](meta-mgr) Real-time parsing meta service endpoint to avoid rpc failed after config muted #45877 (#45899) add af2d6287611 branch-3.0: [enhancement](cloud) add profile counter for file cache #45678 (#45892) add d729009c95d branch-3.0: [fix](csv reader) fix core dump when parsing csv with enclose #45485 (#45888) add 3a1743a1109 branch-3.0: [fix](olap) Set the original tablet state to TABLET_SHUTDOWN #45812 (#45891) add 44cbeaca577 branch-3.0: [enhancement](meta) Sync tablet meta even if local state is not running #45821 (#45962) add 47050877928 brnahc-3.0:[improvement](jdbc catalog) Optimize JdbcCatalog case mapping stability (#45221) add 0cfdf4bd290 branch-3.0: [feature](table-function)support posexplode table function #43221 (#45782) add 3c5797bd99c branch-3.0: [fix](mem leak) fe non_heap mem leak while use jdbc catalog #45806 (#45979) add b06cf95e32c branch-3.0: [fix](regression)Fix paimon row count case. #45994 (#46005) add ee733c0ec44 branch-3.0: [improvement](statistics)Async drop table stats while doing truncate and schema change. #45923 (#45997) add 86369bb2c3d branch-3.0: [enchement](mc)opt maxcompute read arrow skip repeated check of isNull #45989 (#46022) add 9a49926ee09 branch-3.0: [enhance](mtmv)When drop MTMV, no longer wait for task cancel to complete #45995 (#46014) add 24a23d7e976 branch-3.0: [fix](case)fix manager regression case. #45993 (#46032) add fc0078e07ca branch-3.0: [fix](show)show tables should be case insensitive when lowerCaseTableNames is 1 or 2. #46030 (#46057) add cac25bec45a branch-3.0: [opt](bloomfilter index) optimize memory usage for bloom filter index writer #45833 (#46048) add 949ad8be043 branch-3.0: [opt](inverted index)Optimize code to get rid of heap use after free #45745 (#46051) add 4746f167892 branch-3.0: [test](index compaction) Add index compaction full flow UT test #45746 (#46049) add c4f8591652e branch-3.0: [fix](profile) Change the check of whether the username is admin and root to check whether the user specifically corresponds to admin permissions #41714 (#44864) add 9e4a0a910c7 branch-3.0: [fix](code) remove unused files #46078 (#46080) add 1cb1d94b87d branch-3.0: [test](regression)Fix unstable write inverted index case #46046 (#46066) add 64f30f3d8f9 branch-3.0: [fix](clone) Fix the skipped link file due to the stale value #46009 (#46067) add dfe7db7094c branch-3.0: [chore](binlog) Add table type to getMeta method #45942 (#45971) add 01dc5bcd276 [opt](hms table)Some optimizations for hms external table for 3.0 (#44909) (#46086) add bbef3ec6903 [enchement](utf8)import enable_text_validate_utf8 session var (#45537) (#46071) add 64578dafcb4 branch-3.0: [test](nereids) add arthmetic comparison ut #45690 (#45894) add 06b7485e5ee branch-3.0: [fix](variant) disable column name with dot character for variant type #45927 (#45990) add c36eead3063 branch-3.0: [fix](regression test) fix stream load 2pc case fail #45919 (#45975) add bf74831f46c branch-3.0: [regression-test](fix) remove interrupt() in test_routine_load_eof.groovy #46044 (#46097) add 1e006eecbe0 [test](doc) add job-scheduler example in doris's doc to regression test (#42904) This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (99f58c7a0c0) \ N -- N -- N refs/heads/auto-pick-42904-branch-3.0 (1e006eecbe0) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: aazcp.tar.gz | Bin 4218 -> 0 bytes be/src/cloud/cloud_meta_mgr.cpp| 17 +- be/src/cloud/cloud_tablet.cpp
(doris) branch branch-3.0 updated (bf74831f46c -> 1eb361b117b)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a change to branch branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git from bf74831f46c branch-3.0: [regression-test](fix) remove interrupt() in test_routine_load_eof.groovy #46044 (#46097) add 1eb361b117b branch-3.0: [improvement](http) improve table schema api for catalog table #45933 (#45981) No new revisions were added by this update. Summary of changes: .../doris/httpv2/controller/BaseController.java| 10 +++- .../doris/httpv2/rest/RestBaseController.java | 1 + .../doris/httpv2/rest/TableSchemaAction.java | 39 ++ .../http_rest_api/get/test_schema_api.groovy | 60 ++ 4 files changed, 98 insertions(+), 12 deletions(-) create mode 100644 regression-test/suites/http_rest_api/get/test_schema_api.groovy - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45933-branch-3.0 deleted (was 6cea9dd32ef)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45933-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git was 6cea9dd32ef [improvement](http) improve table schema api for catalog table (#45933) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [case](cloud) add case that partial update fail to update tmp rowset after it write new segment in publish phase #45795 [doris]
dataroaring merged PR #45961: URL: https://github.com/apache/doris/pull/45961 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-3.0 updated (1eb361b117b -> 6d5df5d1dfa)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a change to branch branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git from 1eb361b117b branch-3.0: [improvement](http) improve table schema api for catalog table #45933 (#45981) add 6d5df5d1dfa branch-3.0: [case](cloud) add case that partial update fail to update tmp rowset after it write new segment in publish phase #45795 (#45961) No new revisions were added by this update. Summary of changes: be/src/cloud/cloud_tablet.cpp | 3 + ...loud_partial_update_update_tmp_rowset_fail.out} | 5 +- ...ud_partial_update_update_tmp_rowset_fail.groovy | 93 ++ 3 files changed, 100 insertions(+), 1 deletion(-) copy regression-test/data/{correctness_p0/test_with_false_predicate.out => fault_injection_p0/cloud/test_cloud_partial_update_update_tmp_rowset_fail.out} (78%) create mode 100644 regression-test/suites/fault_injection_p0/cloud/test_cloud_partial_update_update_tmp_rowset_fail.groovy - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix]Remove be special handling of date types and bugs in registration functions [doris]
doris-robot commented on PR #45159: URL: https://github.com/apache/doris/pull/45159#issuecomment-2563608527 ClickBench: Total hot run time: 31.32 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 9d5dc10b514cfba4259ad6ad7fca7c71b0942a6e, data reload: false query1 0.030.030.04 query2 0.080.030.03 query3 0.240.070.06 query4 1.620.100.10 query5 0.450.400.40 query6 1.160.660.65 query7 0.020.020.01 query8 0.040.030.04 query9 0.550.520.50 query10 0.590.590.55 query11 0.140.100.11 query12 0.140.100.10 query13 0.610.600.60 query14 2.812.772.79 query15 0.900.820.84 query16 0.380.390.39 query17 1.071.061.07 query18 0.230.210.20 query19 1.991.772.02 query20 0.010.010.01 query21 15.36 0.880.58 query22 0.760.780.63 query23 15.33 1.380.53 query24 3.270.961.17 query25 0.260.140.08 query26 0.240.140.14 query27 0.070.040.04 query28 13.30 1.581.05 query29 12.63 3.903.26 query30 0.260.090.06 query31 2.830.620.37 query32 3.220.560.49 query33 3.183.203.15 query34 16.70 5.094.53 query35 4.554.484.49 query36 0.620.480.48 query37 0.100.070.06 query38 0.050.040.04 query39 0.030.020.02 query40 0.170.130.12 query41 0.080.030.02 query42 0.030.020.02 query43 0.030.030.03 Total cold run time: 106.13 s Total hot run time: 31.32 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-3.0 updated: branch-3.0: [fix](sql) Wrong result for partition item tosql #45918 (#45959)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a commit to branch branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-3.0 by this push: new ef860142e7c branch-3.0: [fix](sql) Wrong result for partition item tosql #45918 (#45959) ef860142e7c is described below commit ef860142e7c814a78b30815bffd7c3f4c7724fcb Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> AuthorDate: Fri Dec 27 19:34:54 2024 +0800 branch-3.0: [fix](sql) Wrong result for partition item tosql #45918 (#45959) Cherry-picked from #45918 Co-authored-by: Uniqueyou --- .../apache/doris/catalog/ListPartitionItem.java| 9 +--- .../doris/catalog/ListPartitionInfoTest.java | 57 ++ 2 files changed, 59 insertions(+), 7 deletions(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/ListPartitionItem.java b/fe/fe-core/src/main/java/org/apache/doris/catalog/ListPartitionItem.java index 98585381244..6edabea442e 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/catalog/ListPartitionItem.java +++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/ListPartitionItem.java @@ -189,10 +189,7 @@ public class ListPartitionItem extends PartitionItem { public String toSql() { StringBuilder sb = new StringBuilder(); -int size = partitionKeys.size(); -if (size > 1) { -sb.append("("); -} +sb.append("("); int i = 0; for (PartitionKey partitionKey : partitionKeys) { @@ -203,9 +200,7 @@ public class ListPartitionItem extends PartitionItem { i++; } -if (size > 1) { -sb.append(")"); -} +sb.append(")"); return sb.toString(); } diff --git a/fe/fe-core/src/test/java/org/apache/doris/catalog/ListPartitionInfoTest.java b/fe/fe-core/src/test/java/org/apache/doris/catalog/ListPartitionInfoTest.java index fe5f2fa0070..2caee8e7b68 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/catalog/ListPartitionInfoTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/catalog/ListPartitionInfoTest.java @@ -248,4 +248,61 @@ public class ListPartitionInfoTest { String expected = "AUTO PARTITION BY LIST (`k1`, `k2`)"; Assert.assertTrue("got: " + sql + ", should have: " + expected, sql.contains(expected)); } + +@Test +public void testListPartitionNullMax() throws AnalysisException, DdlException { +PartitionItem partitionItem = null; +Column k1 = new Column("k1", new ScalarType(PrimitiveType.INT), true, null, "", ""); +Column k2 = new Column("k2", new ScalarType(PrimitiveType.INT), true, null, "", ""); +partitionColumns.add(k1); +partitionColumns.add(k2); +partitionInfo = new ListPartitionInfo(partitionColumns); + +List> inValues = new ArrayList<>(); +inValues.add(Lists.newArrayList(new PartitionValue("", true), PartitionValue.MAX_VALUE)); +SinglePartitionDesc singlePartitionDesc = new SinglePartitionDesc(false, "p1", +PartitionKeyDesc.createIn(inValues), null); +singlePartitionDesc.analyze(2, null); +partitionItem = partitionInfo.handleNewSinglePartitionDesc(singlePartitionDesc, 2L, false); + +Assert.assertEquals("((NULL, MAXVALUE))", ((ListPartitionItem) partitionItem).toSql()); + +inValues = new ArrayList<>(); +inValues.add(Lists.newArrayList(new PartitionValue("", true), new PartitionValue("", true))); +singlePartitionDesc = new SinglePartitionDesc(false, "p2", +PartitionKeyDesc.createIn(inValues), null); +singlePartitionDesc.analyze(2, null); +partitionItem = partitionInfo.handleNewSinglePartitionDesc(singlePartitionDesc, 2L, false); + +Assert.assertEquals("((NULL, NULL))", ((ListPartitionItem) partitionItem).toSql()); + +inValues = new ArrayList<>(); +inValues.add(Lists.newArrayList(PartitionValue.MAX_VALUE, new PartitionValue("", true))); +singlePartitionDesc = new SinglePartitionDesc(false, "p3", +PartitionKeyDesc.createIn(inValues), null); +singlePartitionDesc.analyze(2, null); +partitionItem = partitionInfo.handleNewSinglePartitionDesc(singlePartitionDesc, 2L, false); + +Assert.assertEquals("((MAXVALUE, NULL))", ((ListPartitionItem) partitionItem).toSql()); + +inValues = new ArrayList<>(); +inValues.add(Lists.newArrayList(PartitionValue.MAX_VALUE, PartitionValue.MAX_VALUE)); +singlePartitionDesc = new SinglePartitionDesc(false, "p4", +PartitionKeyDesc.createIn(inValues), null); +singlePartitionDesc.analyze(2, null); +partitionItem = partitionInfo.handleNewSinglePartitionDesc(singlePartitionDesc, 2L, false); + +Assert.assertEquals("((MAXVALUE, MAXVALUE))"
(doris) branch auto-pick-45795-branch-3.0 deleted (was a70df99ab58)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45795-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git was a70df99ab58 [case](cloud) add case that partial update fail to update tmp rowset after it write new segment in publish phase (#45795) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [fix](sql) Wrong result for partition item tosql #45918 [doris]
dataroaring merged PR #45959: URL: https://github.com/apache/doris/pull/45959 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45918-branch-3.0 deleted (was 781aaeaead0)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45918-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git was 781aaeaead0 [fix](sql) Wrong result for partition item tosql (#45918) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [docs]add concurrency control en [doris-website]
wangbo opened a new pull request, #1653: URL: https://github.com/apache/doris-website/pull/1653 ## Versions - [ ] dev - [ ] 3.0 - [ ] 2.1 - [ ] 2.0 ## Languages - [ ] Chinese - [ ] English ## Docs Checklist - [ ] Checked by AI - [ ] Test Cases Built -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](cloud) Cloud enable fe deploy mode from master-observers to mul… [doris]
doris-robot commented on PR #45255: URL: https://github.com/apache/doris/pull/45255#issuecomment-2563809654 TPC-DS: Total hot run time: 197747 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 920dbac97dcb6d7bddda90e53d9e724ebf67b8ae, data reload: false query1 1284938 894 894 query2 6406227023332270 query3 11030 492846854685 query4 33198 23887 23493 23493 query5 4351601 482 482 query6 287 211 191 191 query7 3987491 311 311 query8 314 263 247 247 query9 9418273327252725 query10 481 316 260 260 query11 17988 15564 15622 15564 query12 168 111 120 111 query13 1574539 411 411 query14 10152 741972987298 query15 246 204 187 187 query16 7726653 499 499 query17 1555773 626 626 query18 1873403 362 362 query19 205 192 168 168 query20 127 120 114 114 query21 208 133 108 108 query22 4871451344594459 query23 34014 33856 34249 33856 query24 6672233523172317 query25 493 475 419 419 query26 812 279 160 160 query27 2006460 339 339 query28 5934249324772477 query29 608 561 443 443 query30 207 191 154 154 query31 1019958 862 862 query32 89 60 59 59 query33 469 351 316 316 query34 777 870 521 521 query35 811 854 751 751 query36 10401056976 976 query37 123 101 70 70 query38 4283418643574186 query39 1537151014591459 query40 223 117 108 108 query41 47 49 45 45 query42 120 105 110 105 query43 533 543 488 488 query44 1372846 840 840 query45 191 175 170 170 query46 892 1060686 686 query47 2029200019091909 query48 384 426 345 345 query49 699 470 397 397 query50 649 684 422 422 query51 7211719570787078 query52 104 110 96 96 query53 232 266 187 187 query54 476 535 411 411 query55 90 84 85 84 query56 263 272 250 250 query57 1221123311601160 query58 241 231 230 230 query59 3157323429612961 query60 292 286 248 248 query61 110 106 114 106 query62 856 812 782 782 query63 236 197 199 197 query64 35061073677 677 query65 3350326332753263 query66 723 416 333 333 query67 16454 15875 15567 15567 query68 9264744 513 513 query69 492 285 258 258 query70 1236110411311104 query71 439 280 262 262 query72 6360379638353796 query73 652 753 363 363 query74 10586 911892289118 query75 4614311626852685 query76 47461217775 775 query77 848 388 285 285 query78 10051 10239 94209420 query79 3468910 584 584 query80 701 519 462 462 query81 514 262 237 237 query82 676 150 122 122 query83 190 159 142 142 query84 282 90 66 66 query85 759 368 297 297 query86 337 325 308 308 query87 4420471644484448 query88 4065222022052205 query89 427 331 298 298 query90 1830187 193 187 query91 136 136 102 102 query92 68 53 50 50 query93 1184875 526 526 query94 638 402 275 275 query95 325 269 264 264 query96 492 605 280 280 query97 2746285427022702 query98 217 202 195 195 query99 1694156714431443 Total cold run time: 298461 ms Total hot run time: 197747 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [Feature](Schema Change)light schema change support add key column [doris]
doris-robot commented on PR #42911: URL: https://github.com/apache/doris/pull/42911#issuecomment-2563807780 TeamCity be ut coverage result: Function Coverage: 38.88% (10117/26022) Line Coverage: 29.87% (85488/286183) Region Coverage: 29.02% (43701/150615) Branch Coverage: 25.55% (22292/87256) Coverage Report: http://coverage.selectdb-in.cc/coverage/324eed62c2a6987e306d6dbb9542bbbebd3aef30_324eed62c2a6987e306d6dbb9542bbbebd3aef30/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [opt](inverted index) Opt Inverted Index Composite File Order [doris]
zzzxl1993 opened a new pull request, #46120: URL: https://github.com/apache/doris/pull/46120 ### What problem does this PR solve? Issue Number: close #xxx Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason - Behavior changed: - [ ] No. - [ ] Yes. - Does this need documentation? - [ ] No. - [ ] Yes. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](cloud) Cloud enable fe deploy mode from master-observers to mul… [doris]
doris-robot commented on PR #45255: URL: https://github.com/apache/doris/pull/45255#issuecomment-2563813873 ClickBench: Total hot run time: 31.71 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 920dbac97dcb6d7bddda90e53d9e724ebf67b8ae, data reload: false query1 0.030.030.03 query2 0.090.040.04 query3 0.240.060.06 query4 1.640.070.08 query5 0.420.390.41 query6 1.170.660.65 query7 0.020.020.01 query8 0.050.050.05 query9 0.540.490.53 query10 0.560.560.56 query11 0.160.130.13 query12 0.150.130.13 query13 0.610.610.59 query14 2.722.742.90 query15 0.910.840.83 query16 0.380.390.38 query17 1.071.061.08 query18 0.190.180.19 query19 1.931.882.05 query20 0.020.010.02 query21 15.35 0.960.65 query22 0.770.790.70 query23 14.94 1.480.69 query24 2.220.370.22 query25 0.150.090.09 query26 0.280.200.18 query27 0.080.080.08 query28 13.32 1.841.13 query29 12.68 4.093.45 query30 0.250.080.05 query31 2.840.610.41 query32 3.220.580.48 query33 3.143.083.10 query34 16.70 5.154.56 query35 4.554.574.59 query36 0.620.510.47 query37 0.210.160.17 query38 0.170.150.15 query39 0.050.050.04 query40 0.170.140.12 query41 0.100.050.05 query42 0.070.050.06 query43 0.050.050.05 Total cold run time: 104.83 s Total hot run time: 31.71 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch master updated: update 2.1/3.0 document (#1646)
This is an automated email from the ASF dual-hosted git repository. wangbo pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 7e605831ba2 update 2.1/3.0 document (#1646) 7e605831ba2 is described below commit 7e605831ba2a04ef2289f1e5ca4098a31b1c7ea6 Author: wangbo AuthorDate: Fri Dec 27 18:24:14 2024 +0800 update 2.1/3.0 document (#1646) ## Versions - [ ] dev - [x] 3.0 - [x] 2.1 - [ ] 2.0 ## Languages - [x] Chinese - [ ] English ## Docs Checklist - [ ] Checked by AI - [ ] Test Cases Built --- .../best-practice/group-workload-groups.md | 155 .../best-practice/workload-group-best-practice.md | 436 - .../concurrency-control-and-queuing.md | 10 +- .../workload-management/sql-blocking.md| 55 +-- .../workload-management/workload-group.md | 128 +++--- .../best-practice/group-workload-groups.md | 155 .../best-practice/workload-group-best-practice.md | 436 - .../concurrency-control-and-queuing.md | 10 +- .../workload-management/sql-blocking.md| 87 ++-- .../workload-management/workload-group.md | 128 +++--- 10 files changed, 198 insertions(+), 1402 deletions(-) diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/workload-management/best-practice/group-workload-groups.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/workload-management/best-practice/group-workload-groups.md deleted file mode 100644 index d36ecc7d41e..000 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/workload-management/best-practice/group-workload-groups.md +++ /dev/null @@ -1,155 +0,0 @@ -{ -"title": "Workload Group 分组功能", -"language": "zh-CN" -} - - - -Workload Group 分组功能常用于当一个 Doris 集群中有多个物理隔离的 BE 集群时,可以将 Workload Group 进行分组,不同分组的 Workload Group 可以绑定到不同的 BE 集群中。 - -## 推荐用法 -假如目前集群中已有了两个隔离的 BE 子集群,命名为 rg1 和 rg2,且这两个分组之间是完全物理隔离的,数据和计算不会有共享的情况。 -那么比较推荐的配置方式是: -1. 把 normal group 的资源配置量尽量调小,作为保底的查询分组,比如查询如果不携带任何 Workload Group 信息,那么就会自动使用这个默认的 group,作用是避免查询失败。 -2. 为这两个子集群分别创建对应的 Workload Group,绑定到对应的子集群上。 - 例如,为 rg1 集群创建第一个名为 wg1 的 Workload Group 分组,包含 Workload Group a 和 Workload Group b 两个 Workload Group。为 rg2 集群创建第二个名为 wg2 的 Workload Group 分组,包含 Workload Group c 和 Workload Group d。 - 那么最终效果如下: - - - -操作流程如下: - -第一步:把数据副本绑定到 BE 节点,其实也就是完成 rg1 子集群和 rg2 子集群的划分,实现数据副本的隔离,如果集群已经完成了子集群的划分,那么可以跳过这个步骤,直接进入第二步。 - -1. 把数据副本绑定到 rg1 集群和 rg2 集群 - -```sql --- 为 rg1 集群建表时需要指定副本分布到 rg1 -create table table1 -(k1 int, k2 int) -distributed by hash(k1) buckets 1 -properties( -"replication_allocation"="tag.location.rg1:3" -) - --- 为 rg2 集群建表时需要指定副本分布到 rg2 -create table table2 -(k1 int, k2 int) -distributed by hash(k1) buckets 1 -properties( -"replication_allocation"="tag.location.rg2:3" -) -``` - -2. 把 BE 节点绑定到 rg1 集群和 rg2 集群 - -```sql --- 把 be1 和 be2 绑定到 rg1 集群 -alter system modify backend "be1:9050" set ("tag.location" = "rg1"); -alter system modify backend "be2:9050" set ("tag.location" = "rg1"); - --- 把 be3 和 be4 绑定到 rg2 集群 -alter system modify backend "be3:9050" set ("tag.location" = "rg2"); -alter system modify backend "be4:9050" set ("tag.location" = "rg2"); -``` - -第二步:把 workload group 绑定到 BE 节点 - -1. 新建 workload group,并把 workload group 分别绑定到 wg1 和 wg2 - -```sql --- 创建 wg1 分组的 workload group -create workload group a properties ("memory_limit"="45%","tag"="wg1") -create workload group b properties ("memory_limit"="45%","tag"="wg1") - --- 创建 wg2 分组的 workload group -create workload group c properties ("memory_limit"="45%","tag"="wg2") -create workload group d properties ("memory_limit"="45%","tag"="wg2") -``` - -2. 把 BE 绑定到 wg1 和 wg2,此时 Workload Group a 和 b 只会在 be1 和 be2 上生效。Workload Group c 和 d 只会在 be3 和 be4 上生效。 - (需要注意的是这里在修改时指定了 tag.location,原因是修改 BE 配置的接口目前暂时不支持增量更新,因此在新加属性时要把存量的属性也携带上) - -```sql --- 把 be1 和 be2 绑定到 wg1 -alter system modify backend "be1:9050" set ("tag.location" = "rg1",tag.workload_group="wg1"); -alter system modify backend "be2:9050" set ("tag.location" = "rg1",tag.workload_group="wg1"); - --- 把 be3 和 be4 绑定到 wg2 -alter system modify backend "be3:9050" set ("tag.location" = "rg2",tag.workload_group="wg2"); -alter system modify backend "be4:9050" set ("tag.location" = "rg2",tag.workload_group="wg2"); -``` - -3. 调小 normal workload group 的资源用量,作为用户不携带 Workload Group 信息时保底可用的 Workload Group,可以看到没有为 normal group 指定 tag 属性,因此 normal 可以在所有 BE 生效。 - -```sql -alter workload group normal properties("memory_limit"="1%") -
Re: [PR] [fix]Remove be special handling of date types and bugs in registration functions [doris]
koarz commented on PR #45159: URL: https://github.com/apache/doris/pull/45159#issuecomment-2563555244 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enchancement](sort) change priority_queue to ck SortingQueue [doris]
BiteThet commented on PR #45952: URL: https://github.com/apache/doris/pull/45952#issuecomment-2563555130 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] update 2.1/3.0 document [doris-website]
wangbo merged PR #1646: URL: https://github.com/apache/doris-website/pull/1646 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Profile](scan) Add profile to detail the scan [doris]
doris-robot commented on PR #46110: URL: https://github.com/apache/doris/pull/46110#issuecomment-2563564182 TPC-H: Total hot run time: 49099 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 4d0bd4ddfff995ea3baedbc18d9fe72c78603e37, data reload: false -- Round 1 -- q1 17678 444643914391 q2 2065153 158 153 q3 10482 194319441943 q4 10260 126713221267 q5 8425388339503883 q6 238 124 121 121 q7 2056162415921592 q8 9536278627412741 q9 10302 993297559755 q10 8671360635263526 q11 423 256 256 256 q12 477 302 299 299 q13 18371 396140363961 q14 362 327 329 327 q15 501 452 465 452 q16 553 454 469 454 q17 1143938 918 918 q18 7278695867856785 q19 1716159215151515 q20 543 294 303 294 q21 4386414940764076 q22 498 415 390 390 Total cold run time: 115964 ms Total hot run time: 49099 ms - Round 2, with runtime_filter_mode=off - q1 4312434043544340 q2 324 232 223 223 q3 4177412441544124 q4 2783275127622751 q5 7216707571567075 q6 240 117 118 117 q7 3254283828442838 q8 4449447144674467 q9 13738 13594 13688 13594 q10 4225429242514251 q11 738 694 682 682 q12 1020858 853 853 q13 6780373537293729 q14 468 421 429 421 q15 492 457 459 457 q16 628 587 608 587 q17 3893388338273827 q18 8761876188088761 q19 1733173516781678 q20 2342216521222122 q21 8617848584188418 q22 984 905 960 905 Total cold run time: 81174 ms Total hot run time: 76220 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45927-branch-3.0 deleted (was f4e60617b45)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45927-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git was f4e60617b45 [fix](variant) disable column name with dot character for variant type (#45927) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [fix](variant) disable column name with dot character for variant type #45927 [doris]
eldenmoon merged PR #45990: URL: https://github.com/apache/doris/pull/45990 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-3.0 updated: branch-3.0: [fix](variant) disable column name with dot character for variant type #45927 (#45990)
This is an automated email from the ASF dual-hosted git repository. eldenmoon pushed a commit to branch branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-3.0 by this push: new 06b7485e5ee branch-3.0: [fix](variant) disable column name with dot character for variant type #45927 (#45990) 06b7485e5ee is described below commit 06b7485e5ee0c358f2e0afac6898ac8aeb546830 Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> AuthorDate: Fri Dec 27 18:47:37 2024 +0800 branch-3.0: [fix](variant) disable column name with dot character for variant type #45927 (#45990) Cherry-picked from #45927 Co-authored-by: camby --- .../org/apache/doris/analysis/CreateTableStmt.java | 5 .../trees/plans/commands/info/CreateTableInfo.java | 5 .../suites/ddl_p0/test_create_table.groovy | 30 ++ 3 files changed, 40 insertions(+) diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateTableStmt.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateTableStmt.java index 1338632f3fb..1051b7b1748 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateTableStmt.java +++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateTableStmt.java @@ -325,6 +325,11 @@ public class CreateTableStmt extends DdlStmt implements NotFallbackInParser { throw new AnalysisException( "Disable to create table column with name start with __DORIS_: " + columnNameUpperCase); } +if (Objects.equals(columnDef.getType(), Type.VARIANT) && columnNameUpperCase.indexOf('.') != -1) { +throw new AnalysisException( +"Disable to create table of `VARIANT` type column named with a `.` character: " ++ columnNameUpperCase); +} if (Objects.equals(columnDef.getType(), Type.DATE) && Config.disable_datev1) { throw new AnalysisException("Disable to create table with `DATE` type columns, please use `DATEV2`."); } diff --git a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/info/CreateTableInfo.java b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/info/CreateTableInfo.java index 0e3c037ecc3..821625576f6 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/info/CreateTableInfo.java +++ b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/info/CreateTableInfo.java @@ -307,6 +307,11 @@ public class CreateTableInfo { throw new AnalysisException( "Disable to create table column with name start with __DORIS_: " + columnNameUpperCase); } +if (columnDef.getType().isVariantType() && columnNameUpperCase.indexOf('.') != -1) { +throw new AnalysisException( +"Disable to create table of `VARIANT` type column named with a `.` character: " ++ columnNameUpperCase); +} if (columnDef.getType().isDateType() && Config.disable_datev1) { throw new AnalysisException( "Disable to create table with `DATE` type columns, please use `DATEV2`."); diff --git a/regression-test/suites/ddl_p0/test_create_table.groovy b/regression-test/suites/ddl_p0/test_create_table.groovy index 1d3b8409d65..03de74bc289 100644 --- a/regression-test/suites/ddl_p0/test_create_table.groovy +++ b/regression-test/suites/ddl_p0/test_create_table.groovy @@ -46,6 +46,21 @@ suite("sql_create_time_range_table") { ); """ +// variant type column named with '.' character is disabled +sql "DROP TABLE IF EXISTS disable_variant_column_with_dot" +test { +sql """ +CREATE TABLE disable_variant_column_with_dot ( +k int, +`v1.v2` variant +) +DUPLICATE KEY (`k`) +DISTRIBUTED BY HASH(`k`) BUCKETS 1 +PROPERTIES ( "replication_num" = "1"); +""" +exception "Disable to create table" +} + // DDL/DML return 1 row and 1 column, the only value is update row count assertTrue(result1.size() == 1) assertTrue(result1[0].size() == 1) @@ -58,4 +73,19 @@ suite("sql_create_time_range_table") { def res_show = sql "show create table varchar_0_char_0" mustContain(res_show[0][1], "varchar(65533)") mustContain(res_show[0][1], "char(1)") + +// variant type column named with '.' character is disabled +sql "DROP TABLE IF EXISTS disable_variant_column_with_dot" +test { +sql """ +CREATE TABLE disable_variant_column_with_dot ( +k int, +`v1.v2` variant +) +DUPLICATE KEY (`k`) +
Re: [PR] [test](regression)Add index compaction exception case [doris]
doris-robot commented on PR #46118: URL: https://github.com/apache/doris/pull/46118#issuecomment-2563587996 TeamCity be ut coverage result: Function Coverage: 38.89% (10119/26022) Line Coverage: 29.88% (85495/286124) Region Coverage: 29.02% (43694/150572) Branch Coverage: 25.56% (22292/87220) Coverage Report: http://coverage.selectdb-in.cc/coverage/7454a75757486167225378b1e68b64c524101f34_7454a75757486167225378b1e68b64c524101f34/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [chore](binlog) Add table type to getMeta method #45942 [doris]
w41ter merged PR #45972: URL: https://github.com/apache/doris/pull/45972 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45942-branch-2.1 deleted (was d0dd45e41bf)
This is an automated email from the ASF dual-hosted git repository. w41ter pushed a change to branch auto-pick-45942-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git was d0dd45e41bf [chore](binlog) Add table type to getMeta method (#45942) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: branch-2.1: [chore](binlog) Add table type to getMeta method #45942 (#45972)
This is an automated email from the ASF dual-hosted git repository. w41ter pushed a commit to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-2.1 by this push: new fab89c199e5 branch-2.1: [chore](binlog) Add table type to getMeta method #45942 (#45972) fab89c199e5 is described below commit fab89c199e5475b4f9722e21214573852d2ae35f Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> AuthorDate: Fri Dec 27 20:03:53 2024 +0800 branch-2.1: [chore](binlog) Add table type to getMeta method #45942 (#45972) Cherry-picked from #45942 Co-authored-by: walter --- fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java | 1 + gensrc/thrift/FrontendService.thrift | 1 + 2 files changed, 2 insertions(+) diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java index 0bbe662ec6e..3e3de2e6676 100755 --- a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java +++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java @@ -6159,6 +6159,7 @@ public class Env { try { tableMeta.setId(olapTable.getId()); tableMeta.setName(olapTable.getName()); +tableMeta.setType(olapTable.getType().name()); PartitionInfo tblPartitionInfo = olapTable.getPartitionInfo(); diff --git a/gensrc/thrift/FrontendService.thrift b/gensrc/thrift/FrontendService.thrift index 4853d3a474f..2b753a45ac1 100644 --- a/gensrc/thrift/FrontendService.thrift +++ b/gensrc/thrift/FrontendService.thrift @@ -1557,6 +1557,7 @@ struct TGetMetaTableMeta { 2: optional string name 3: optional bool in_trash 4: optional list partitions +5: optional string type } struct TGetMetaDBMeta { - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Cherry-pick](branch-2.1) Pick "[Enhancement](compaction) Do not set failure time when cumulative compaction dealing with delete rowset (#43466)" [doris]
doris-robot commented on PR #46117: URL: https://github.com/apache/doris/pull/46117#issuecomment-2563639836 TeamCity be ut coverage result: Function Coverage: 36.39% (9540/26218) Line Coverage: 27.90% (78603/281755) Region Coverage: 26.57% (40354/151859) Branch Coverage: 23.33% (20436/87610) Coverage Report: http://coverage.selectdb-in.cc/coverage/d95effbbc587b25f7ba63b51b07e39062d6f24c5_d95effbbc587b25f7ba63b51b07e39062d6f24c5/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated (75e9aa3fb4f -> 829d9b3454f)
This is an automated email from the ASF dual-hosted git repository. eldenmoon pushed a change to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git from 75e9aa3fb4f branch-2.1: [fix](jdbc catalog) Get statistics using remote name #46040 (#46116) add 829d9b3454f branch-2.1: [Fix](PreparedStatement) nondeterministic functions in prepared statement should not be short circuited #46003 (#46109) No new revisions were added by this update. Summary of changes: .../org/apache/doris/nereids/StatementContext.java | 10 .../nereids/rules/analysis/ExpressionAnalyzer.java | 5 .../trees/plans/commands/ExecuteCommand.java | 7 +++--- .../suites/point_query_p0/test_point_query.groovy | 27 ++ 4 files changed, 46 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-45941-branch-3.0 deleted (was 584c3b5d288)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45941-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git was 584c3b5d288 [opt](profile) add summary metric for file scanner (#45941) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch master updated: refactor workload group en (#1651)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 9528f0e0a47 refactor workload group en (#1651) 9528f0e0a47 is described below commit 9528f0e0a4715e19a94a37d046a7336b1c671894 Author: wangbo AuthorDate: Fri Dec 27 21:12:32 2024 +0800 refactor workload group en (#1651) ## Versions - [x] dev - [ ] 3.0 - [ ] 2.1 - [ ] 2.0 ## Languages - [ ] Chinese - [x] English ## Docs Checklist - [ ] Checked by AI - [ ] Test Cases Built --- .../workload-management/workload-group.md | 737 + .../workload-management/workload-group.md | 6 +- .../workload-management/workload-group.md | 6 +- .../workload-management/workload-group.md | 6 +- 4 files changed, 622 insertions(+), 133 deletions(-) diff --git a/docs/admin-manual/workload-management/workload-group.md b/docs/admin-manual/workload-management/workload-group.md index dc491a96259..50e3cb65ca3 100644 --- a/docs/admin-manual/workload-management/workload-group.md +++ b/docs/admin-manual/workload-management/workload-group.md @@ -24,137 +24,171 @@ specific language governing permissions and limitations under the License. --> -Use Workload Groups in Doris to manage and limit resources. By employing resource control, you can effectively limit the CPU, memory, and IO resources used by queries and imports, and create query queues to manage the maximum concurrency of queries in the cluster. Since Doris version 2.1, CPU resource limitations are enforced using CGroup. Before using the Workload resource control feature, you need to configure the CGroup environment. When setting up Workload resource control, you must [...] -- Soft Limit: Allows borrowing resources from other Workload Groups when there is no resource contention, potentially exceeding the soft limit. -- Hard Limit: Ensures that the resource allocation cannot exceed the specified quota, regardless of resource contention. -To use Workload resource control, you need to perform the following steps: -1. Create a Workload Group. -2. Add resource limitation rules to the Workload Group. -3. Bind tenants to the Workload Group. +Workload Group is an in-process mechanism for isolating workloads. +It achieves resource isolation by finely partitioning or limiting resources (CPU, IO, Memory) within the BE process. +Its principle is illustrated in the diagram below: -## Version Upgrade Notes -Workload resource control has been available since Doris version 2.0. In Doris 2.0, Workload resource control did not depend on CGroup, but Doris 2.1 requires CGroup. + -Upgrading from Doris 2.0 to 2.1: Since Workload resource control in version 2.1 depends on CGroup, you must first configure the CGroup environment before upgrading to Doris 2.1. +The currently supported isolation capabilities include: -## Configuring the CGroup Environment -In Doris version 2.0, CPU resource limitation was implemented based on Doris's scheduling, which provided great flexibility but lacked precise CPU isolation. From version 2.1, Doris uses CGroup for CPU resource limitation. Users requiring strong resource isolation are recommended to upgrade to version 2.1 and ensure CGroup is installed on all BE nodes. +* Managing CPU resources, with support for both cpu hard limit and cpu soft limit; +* Managing memory resources, with support for both memory hard limit and memory soft limit; +* Managing IO resources, including IO generated by reading local and remote files. -If you used soft limits in Workload Groups in version 2.0 and upgraded to 2.1, you also need to configure CGroup to avoid losing soft limit functionality. Without CGroup configured, users can use all Workload Group features except CPU limitation. -:::tip -1. The Doris BE node can effectively utilize the CPU and memory resources of the machine. It is recommended to deploy only one BE instance per machine. Currently, the workload resource management does not support deploying multiple BE instances on a single machine. -2. After a machine restart, the following CGroup configurations will be cleared. If you want the configurations to persist after a reboot, you can use systemd to set the operation as a custom system service. This way, the creation and authorization operations will be automatically performed each time the machine restarts. -3. If CGroup is used within a container, the container needs to have permissions to operate on the host machine. - ::: +## Version Notes -### Verifying CGroup Installation on BE Nodes -Check /proc/filesystems to determine if CGroup is installed: -cat /proc/filesystems | grep cgroup -nodev cgroup -nodev cgroup2 -nodev c
(doris-website) branch master updated (9528f0e0a47 -> bf7e729c12c)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git from 9528f0e0a47 refactor workload group en (#1651) add bf7e729c12c [docs]add concurrency control en (#1653) No new revisions were added by this update. Summary of changes: .../concurrency-control-and-queuing.md | 104 + .../concurrency-control-and-queuing.md | 2 +- .../concurrency-control-and-queuing.md | 2 +- .../concurrency-control-and-queuing.md | 2 +- 4 files changed, 107 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] refactor workload group en [doris-website]
yiguolei merged PR #1651: URL: https://github.com/apache/doris-website/pull/1651 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](schema) support temporary table like mysql [doris]
doris-robot commented on PR #40680: URL: https://github.com/apache/doris/pull/40680#issuecomment-2563569637 TPC-H: Total hot run time: 32695 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 78a93210171a3a3267ccedc083bb36ac6dd0dcf8, data reload: false -- Round 1 -- q1 17582 615460706070 q2 2055306 171 171 q3 10416 1240743 743 q4 10210 903 432 432 q5 7553223019821982 q6 211 185 147 147 q7 921 740 611 611 q8 9244141612091209 q9 5405499249354935 q10 6769234618741874 q11 494 274 267 267 q12 353 367 217 217 q13 17760 362030103010 q14 235 237 230 230 q15 577 508 498 498 q16 631 617 586 586 q17 567 863 326 326 q18 7170652963536353 q19 38301005598 598 q20 294 316 183 183 q21 2807217219441944 q22 368 342 309 309 Total cold run time: 105452 ms Total hot run time: 32695 ms - Round 2, with runtime_filter_mode=off - q1 6454624362826243 q2 236 332 235 235 q3 2294269323402340 q4 1449190514031403 q5 4364476848964768 q6 177 176 145 145 q7 2156201118491849 q8 2649282427092709 q9 7264738773537353 q10 3125337227252725 q11 583 514 508 508 q12 691 808 607 607 q13 3454385131213121 q14 280 315 271 271 q15 569 512 517 512 q16 662 712 657 657 q17 1225176312561256 q18 7885759475367536 q19 842 118610921092 q20 1989205419111911 q21 5878507750705070 q22 665 643 587 587 Total cold run time: 54891 ms Total hot run time: 52898 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](auth)Reduce the scope of permission locks [doris]
doris-robot commented on PR #46115: URL: https://github.com/apache/doris/pull/46115#issuecomment-2563579429 ClickBench: Total hot run time: 31.22 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit b26c683bc18f5c8306b3937de0e3d2bd398cf366, data reload: false query1 0.040.040.04 query2 0.070.030.03 query3 0.240.070.07 query4 1.610.110.11 query5 0.420.410.40 query6 1.150.650.66 query7 0.020.010.02 query8 0.030.040.04 query9 0.580.510.52 query10 0.560.600.56 query11 0.150.110.12 query12 0.150.110.11 query13 0.610.610.59 query14 2.872.792.77 query15 0.900.830.82 query16 0.380.380.38 query17 1.051.021.09 query18 0.220.210.20 query19 1.951.812.03 query20 0.010.010.01 query21 15.36 0.910.58 query22 0.750.840.61 query23 15.32 1.530.57 query24 2.691.190.96 query25 0.190.110.21 query26 0.230.150.13 query27 0.080.060.05 query28 13.88 1.531.05 query29 12.54 3.903.22 query30 0.250.090.07 query31 2.830.590.39 query32 3.220.550.45 query33 3.203.063.01 query34 16.58 5.084.51 query35 4.514.474.48 query36 0.650.480.48 query37 0.100.060.05 query38 0.050.040.04 query39 0.040.020.03 query40 0.170.140.13 query41 0.080.030.03 query42 0.040.020.03 query43 0.040.030.03 Total cold run time: 105.81 s Total hot run time: 31.22 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Profile](scan) Add profile to detail the scan [doris]
doris-robot commented on PR #46110: URL: https://github.com/apache/doris/pull/46110#issuecomment-2563578261 ClickBench: Total hot run time: 31.41 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 4d0bd4ddfff995ea3baedbc18d9fe72c78603e37, data reload: false query1 0.030.020.02 query2 0.080.030.03 query3 0.250.050.06 query4 1.750.080.08 query5 0.540.530.52 query6 1.270.610.66 query7 0.020.010.01 query8 0.030.020.02 query9 0.520.500.49 query10 0.560.530.54 query11 0.130.090.09 query12 0.120.090.10 query13 0.620.620.61 query14 0.790.780.77 query15 0.780.760.76 query16 0.390.380.37 query17 1.001.021.02 query18 0.220.260.25 query19 1.871.881.88 query20 0.020.010.01 query21 15.47 0.560.55 query22 2.152.351.98 query23 17.33 0.980.97 query24 3.761.071.50 query25 0.350.070.05 query26 0.550.160.16 query27 0.050.040.05 query28 8.750.710.71 query29 12.64 2.432.35 query30 0.560.500.51 query31 2.780.380.36 query32 3.410.490.50 query33 3.093.123.10 query34 15.27 4.844.78 query35 4.874.844.84 query36 1.061.001.02 query37 0.060.040.05 query38 0.040.020.02 query39 0.020.020.01 query40 0.160.150.14 query41 0.070.020.01 query42 0.030.010.01 query43 0.020.020.02 Total cold run time: 103.48 s Total hot run time: 31.41 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [fix](jdbc catalog) Get statistics using remote name #46040 [doris]
doris-robot commented on PR #46113: URL: https://github.com/apache/doris/pull/46113#issuecomment-2563580245 ClickBench: Total hot run time: 32.85 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit c836102c80cf124bf4489c14cf39e1830fe12535, data reload: false query1 0.030.030.03 query2 0.070.030.03 query3 0.240.070.06 query4 1.620.100.10 query5 0.530.500.53 query6 1.170.730.73 query7 0.030.010.02 query8 0.040.030.03 query9 0.560.500.49 query10 0.550.560.55 query11 0.150.100.10 query12 0.140.110.11 query13 0.610.630.59 query14 3.043.062.93 query15 0.900.830.83 query16 0.370.390.38 query17 1.031.001.01 query18 0.250.210.22 query19 1.971.851.95 query20 0.010.010.01 query21 15.35 0.590.57 query22 2.661.901.46 query23 16.93 1.000.77 query24 3.052.031.19 query25 0.270.150.19 query26 0.470.140.14 query27 0.040.050.04 query28 9.811.111.08 query29 12.54 3.233.21 query30 0.240.060.06 query31 2.860.390.39 query32 3.260.460.47 query33 3.003.003.05 query34 17.00 4.504.49 query35 4.574.534.52 query36 0.640.480.47 query37 0.090.070.06 query38 0.050.030.03 query39 0.040.030.02 query40 0.160.120.12 query41 0.080.030.03 query42 0.040.030.02 query43 0.040.030.04 Total cold run time: 106.5 s Total hot run time: 32.85 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](regression)Add index compaction exception case [doris]
doris-robot commented on PR #46118: URL: https://github.com/apache/doris/pull/46118#issuecomment-2563580093 ClickBench: Total hot run time: 32.14 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 7454a75757486167225378b1e68b64c524101f34, data reload: false query1 0.030.030.03 query2 0.060.040.03 query3 0.230.070.07 query4 1.610.110.11 query5 0.440.400.39 query6 1.140.640.64 query7 0.020.020.01 query8 0.050.040.03 query9 0.570.520.49 query10 0.540.580.57 query11 0.160.110.11 query12 0.150.110.11 query13 0.610.610.60 query14 2.732.742.72 query15 0.890.840.81 query16 0.380.410.37 query17 0.991.051.04 query18 0.220.210.22 query19 1.931.821.99 query20 0.020.020.01 query21 15.37 0.910.58 query22 0.750.790.57 query23 15.37 1.420.53 query24 2.941.801.81 query25 0.220.190.06 query26 0.220.150.14 query27 0.070.060.06 query28 14.50 1.471.04 query29 12.57 3.993.30 query30 0.240.090.06 query31 2.840.600.36 query32 3.230.560.47 query33 3.113.213.17 query34 16.91 5.194.55 query35 4.544.504.59 query36 0.660.490.47 query37 0.090.060.06 query38 0.050.040.03 query39 0.030.030.02 query40 0.160.150.13 query41 0.070.020.03 query42 0.030.020.02 query43 0.030.030.03 Total cold run time: 106.77 s Total hot run time: 32.14 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enchement](utf8)import enable_text_validate_utf8 session var (#45537) [doris]
doris-robot commented on PR #46070: URL: https://github.com/apache/doris/pull/46070#issuecomment-2563578929 TeamCity be ut coverage result: Function Coverage: 36.38% (9538/26219) Line Coverage: 27.89% (78571/281752) Region Coverage: 26.56% (40336/151859) Branch Coverage: 23.32% (20428/87608) Coverage Report: http://coverage.selectdb-in.cc/coverage/2cb72a05d84b95ee34e15c4105527fed510805c3_2cb72a05d84b95ee34e15c4105527fed510805c3/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](schema) support temporary table like mysql [doris]
doris-robot commented on PR #40680: URL: https://github.com/apache/doris/pull/40680#issuecomment-2563578966 TPC-DS: Total hot run time: 199138 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 78a93210171a3a3267ccedc083bb36ac6dd0dcf8, data reload: false query1 1300977 932 932 query2 6318251822622262 query3 11121 503149074907 query4 33084 23656 23543 23543 query5 4313620 449 449 query6 294 204 181 181 query7 3995500 313 313 query8 319 265 243 243 query9 9539276427842764 query10 468 304 259 259 query11 18076 15615 15315 15315 query12 170 108 109 108 query13 1631549 406 406 query14 11524 744478077444 query15 260 211 196 196 query16 8096674 514 514 query17 1549798 617 617 query18 2145428 334 334 query19 244 187 157 157 query20 123 116 117 116 query21 201 125 107 107 query22 4577451245024502 query23 34665 33888 33599 33599 query24 6485242523842384 query25 506 447 388 388 query26 910 277 151 151 query27 2191468 338 338 query28 5371252124922492 query29 673 566 430 430 query30 217 208 153 153 query31 986 953 864 864 query32 93 61 60 60 query33 515 375 313 313 query34 773 860 510 510 query35 808 838 755 755 query36 10301077970 970 query37 118 100 77 77 query38 4343423042334230 query39 1508148514591459 query40 207 117 101 101 query41 48 46 41 41 query42 116 103 107 103 query43 528 540 510 510 query44 1404849 842 842 query45 185 177 171 171 query46 877 1076674 674 query47 1977201319451945 query48 393 401 322 322 query49 721 482 382 382 query50 635 701 408 408 query51 7285725972327232 query52 105 101 97 97 query53 230 252 190 190 query54 482 493 408 408 query55 83 83 83 83 query56 257 245 228 228 query57 1279125011761176 query58 246 231 214 214 query59 3404351934953495 query60 277 267 258 258 query61 111 104 107 104 query62 859 840 763 763 query63 240 201 195 195 query64 37701044665 665 query65 3403325932873259 query66 799 411 303 303 query67 16539 15800 15513 15513 query68 9842756 503 503 query69 472 299 252 252 query70 1232114511481145 query71 440 297 252 252 query72 6216389438483848 query73 660 761 376 376 query74 10164 905393439053 query75 4414310826712671 query76 50711208824 824 query77 925 377 283 283 query78 10205 10331 10449 10331 query79 2892850 612 612 query80 747 529 424 424 query81 495 277 224 224 query82 484 147 122 122 query83 195 156 160 156 query84 280 97 78 78 query85 790 387 315 315 query86 390 317 302 302 query87 4382456744044404 query88 4581222721782178 query89 423 325 296 296 query90 1830190 191 190 query91 137 136 106 106 query92 64 57 52 52 query93 1391832 524 524 query94 642 411 290 290 query95 332 261 254 254 query96 482 611 293 293 query97 2775281326322632 query98 228 228 197 197 query99 1799151514271427 Total cold run time: 301979 ms Total hot run time: 199138 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [feature](schema) support temporary table like mysql [doris]
Yulei-Yang commented on PR #40680: URL: https://github.com/apache/doris/pull/40680#issuecomment-2563588760 run feut -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] update sidebar for import, update and export [doris-website]
KassieZ merged PR #1629: URL: https://github.com/apache/doris-website/pull/1629 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1 [Fix](Catalog)Standardize the use of authentication attributes in the Catalog to avoid creating a large number of Authenticator objects. [doris]
CalvinKirs commented on PR #46106: URL: https://github.com/apache/doris/pull/46106#issuecomment-2563591201 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch master updated: update sidebar for import, update and export (#1629)
This is an automated email from the ASF dual-hosted git repository. kassiez pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 0289353bddb update sidebar for import, update and export (#1629) 0289353bddb is described below commit 0289353bddba649d0bfff907a9d38ccb9a277a2b Author: wangtianyi2004 <376612...@qq.com> AuthorDate: Fri Dec 27 19:10:54 2024 +0800 update sidebar for import, update and export (#1629) ## Versions - [x] dev - [x] 3.0 - [x] 2.1 - [ ] 2.0 ## Languages - [x] Chinese - [x] English ## Docs Checklist - [ ] Checked by AI - [ ] Test Cases Built - Co-authored-by: KassieZ <139741991+kass...@users.noreply.github.com> --- .../docusaurus-plugin-content-docs/current.json| 8 .../version-2.1.json | 4 ++ .../version-3.0.json | 4 ++ sidebars.json | 55 -- versioned_sidebars/version-2.1-sidebars.json | 54 +++-- versioned_sidebars/version-3.0-sidebars.json | 54 +++-- 6 files changed, 136 insertions(+), 43 deletions(-) diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current.json b/i18n/zh-CN/docusaurus-plugin-content-docs/current.json index a8c5e3c014b..de58e14d98f 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current.json +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current.json @@ -99,6 +99,14 @@ "message": "数据导入", "description": "The label for category Loading Data in sidebar docs" }, + "sidebar.docs.category.Data Update and Delete": { +"message": "数据更新与删除", +"description": "The label for category Data Update and Delete in sidebar docs" + }, + "sidebar.docs.category.Complex Type": { +"message": "复杂类型", +"description": "The label for category Complex Type in sidebar docs" + }, "sidebar.docs.category.Data Source": { "message": "数据源", "description": "The label for category Data Source in sidebar docs" diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1.json b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1.json index e79a42af59f..f7ba177768f 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1.json +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1.json @@ -119,6 +119,10 @@ "message": "数据删除", "description": "The label for category Deleting Data in sidebar docs" }, + "sidebar.docs.category.Data Update and Delete": { +"message": "数据更新与删除", +"description": "The label for category Data Update and Delete in sidebar docs" + }, "sidebar.docs.category.Exporting Data": { "message": "数据导出", "description": "The label for category Exporting Data in sidebar docs" diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0.json b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0.json index 283c6ffdcff..2994e1f5977 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0.json +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0.json @@ -107,6 +107,10 @@ "message": "导入方式", "description": "The label for category Import Way in sidebar docs" }, + "sidebar.docs.category.Data Update and Delete": { +"message": "数据更新与删除", +"description": "The label for category Data Update and Delete in sidebar docs" + }, "sidebar.docs.category.Complex Data Types": { "message": "复杂数据类型", "description": "The label for category Complex Data Types in sidebar docs" diff --git a/sidebars.json b/sidebars.json index dfe1fe79719..3e178507f61 100644 --- a/sidebars.json +++ b/sidebars.json @@ -167,13 +167,14 @@ "table-design/best-practice" ] }, -{ -"type": "category", -"label": "Data Manipulation", + { + "type": "category", +"label": "Loading Data", "items": [ +"data-operate/import/load-manual", { "type": "category", -"label": "Loading Data", +"label": "Data Source", "items": [ "data-operate/import/load-manual", { @@ -226,6 +227,30 @@ "data-operate/import/group-commit-manual" ] }, +{ +"type": "category", +"label": "Import Way", +"items": [ + "data-operate/import/import-way/stream-load-manual", +
(doris) branch auto-pick-45927-branch-2.1 deleted (was cb41cd63664)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-45927-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git was cb41cd63664 [fix](variant) disable column name with dot character for variant type (#45927) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](variant) fix potential use after free [doris]
eldenmoon merged PR #46056: URL: https://github.com/apache/doris/pull/46056 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [fix](variant) fix potential use after free (#46056)
This is an automated email from the ASF dual-hosted git repository. eldenmoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new 6f063bc86da [fix](variant) fix potential use after free (#46056) 6f063bc86da is described below commit 6f063bc86da6025c11fb66fb2c7bf6988697f35d Author: lihangyu AuthorDate: Fri Dec 27 19:11:33 2024 +0800 [fix](variant) fix potential use after free (#46056) --- be/src/vec/exprs/table_function/vexplode.cpp | 17 - 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/be/src/vec/exprs/table_function/vexplode.cpp b/be/src/vec/exprs/table_function/vexplode.cpp index 5fa378f6351..6f1db0ac1a9 100644 --- a/be/src/vec/exprs/table_function/vexplode.cpp +++ b/be/src/vec/exprs/table_function/vexplode.cpp @@ -44,27 +44,26 @@ VExplodeTableFunction::VExplodeTableFunction() { Status VExplodeTableFunction::_process_init_variant(Block* block, int value_column_idx) { // explode variant array -const auto& variant_column = check_and_get_column( -remove_nullable(block->get_by_position(value_column_idx) -.column->convert_to_full_column_if_const()) -.get()); +auto column_without_nullable = remove_nullable(block->get_by_position(value_column_idx).column); +auto column = column_without_nullable->convert_to_full_column_if_const(); +const auto& variant_column = assert_cast(*column); _detail.output_as_variant = true; -if (!variant_column->is_null_root()) { -_array_column = variant_column->get_root(); +if (!variant_column.is_null_root()) { +_array_column = variant_column.get_root(); // We need to wrap the output nested column within a variant column. // Otherwise the type is missmatched const auto* array_type = check_and_get_data_type( -remove_nullable(variant_column->get_root_type()).get()); +remove_nullable(variant_column.get_root_type()).get()); if (array_type == nullptr) { return Status::NotSupported("explode not support none array type {}", - variant_column->get_root_type()->get_name()); + variant_column.get_root_type()->get_name()); } _detail.nested_type = array_type->get_nested_type(); } else { // null root, use nothing type _array_column = ColumnNullable::create(ColumnArray::create(ColumnNothing::create(0)), ColumnUInt8::create(0)); - _array_column->assume_mutable()->insert_many_defaults(variant_column->size()); + _array_column->assume_mutable()->insert_many_defaults(variant_column.size()); _detail.nested_type = std::make_shared(); } return Status::OK(); - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [fix](variant) disable column name with dot character for variant type #45927 [doris]
eldenmoon merged PR #45992: URL: https://github.com/apache/doris/pull/45992 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: branch-2.1: [fix](variant) disable column name with dot character for variant type #45927 (#45992)
This is an automated email from the ASF dual-hosted git repository. eldenmoon pushed a commit to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-2.1 by this push: new 6cee95540dc branch-2.1: [fix](variant) disable column name with dot character for variant type #45927 (#45992) 6cee95540dc is described below commit 6cee95540dced1fac31e2a518b271e4b637dbcad Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> AuthorDate: Fri Dec 27 19:12:44 2024 +0800 branch-2.1: [fix](variant) disable column name with dot character for variant type #45927 (#45992) Cherry-picked from #45927 Co-authored-by: camby --- .../org/apache/doris/analysis/CreateTableStmt.java | 5 .../trees/plans/commands/info/CreateTableInfo.java | 5 .../suites/ddl_p0/test_create_table.groovy | 30 ++ 3 files changed, 40 insertions(+) diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateTableStmt.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateTableStmt.java index 333f4eabc6b..3dccc18730a 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateTableStmt.java +++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateTableStmt.java @@ -317,6 +317,11 @@ public class CreateTableStmt extends DdlStmt { throw new AnalysisException( "Disable to create table column with name start with __DORIS_: " + columnNameUpperCase); } +if (Objects.equals(columnDef.getType(), Type.VARIANT) && columnNameUpperCase.indexOf('.') != -1) { +throw new AnalysisException( +"Disable to create table of `VARIANT` type column named with a `.` character: " ++ columnNameUpperCase); +} if (Objects.equals(columnDef.getType(), Type.DATE) && Config.disable_datev1) { throw new AnalysisException("Disable to create table with `DATE` type columns, please use `DATEV2`."); } diff --git a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/info/CreateTableInfo.java b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/info/CreateTableInfo.java index 638b87c07e0..09c44626e78 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/info/CreateTableInfo.java +++ b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/info/CreateTableInfo.java @@ -278,6 +278,11 @@ public class CreateTableInfo { throw new AnalysisException( "Disable to create table column with name start with __DORIS_: " + columnNameUpperCase); } +if (columnDef.getType().isVariantType() && columnNameUpperCase.indexOf('.') != -1) { +throw new AnalysisException( +"Disable to create table of `VARIANT` type column named with a `.` character: " ++ columnNameUpperCase); +} if (columnDef.getType().isDateType() && Config.disable_datev1) { throw new AnalysisException( "Disable to create table with `DATE` type columns, please use `DATEV2`."); diff --git a/regression-test/suites/ddl_p0/test_create_table.groovy b/regression-test/suites/ddl_p0/test_create_table.groovy index 1d3b8409d65..03de74bc289 100644 --- a/regression-test/suites/ddl_p0/test_create_table.groovy +++ b/regression-test/suites/ddl_p0/test_create_table.groovy @@ -46,6 +46,21 @@ suite("sql_create_time_range_table") { ); """ +// variant type column named with '.' character is disabled +sql "DROP TABLE IF EXISTS disable_variant_column_with_dot" +test { +sql """ +CREATE TABLE disable_variant_column_with_dot ( +k int, +`v1.v2` variant +) +DUPLICATE KEY (`k`) +DISTRIBUTED BY HASH(`k`) BUCKETS 1 +PROPERTIES ( "replication_num" = "1"); +""" +exception "Disable to create table" +} + // DDL/DML return 1 row and 1 column, the only value is update row count assertTrue(result1.size() == 1) assertTrue(result1[0].size() == 1) @@ -58,4 +73,19 @@ suite("sql_create_time_range_table") { def res_show = sql "show create table varchar_0_char_0" mustContain(res_show[0][1], "varchar(65533)") mustContain(res_show[0][1], "char(1)") + +// variant type column named with '.' character is disabled +sql "DROP TABLE IF EXISTS disable_variant_column_with_dot" +test { +sql """ +CREATE TABLE disable_variant_column_with_dot ( +k int, +`v1.v2` variant +) +DUPLICATE KEY (`k`) +DISTRIBUTED BY HASH(`k`) BUCK
Re: [I] [Bug] IcebergScanNode throws exception "Split size must be > 0: 0" [doris]
morningman commented on issue #44573: URL: https://github.com/apache/doris/issues/44573#issuecomment-2563691908 You can try 1. set enable_fallback_to_original_planner=false; run sql again to see if there are errors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [docs]add concurrency control en [doris-website]
yiguolei merged PR #1653: URL: https://github.com/apache/doris-website/pull/1653 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showStorageVaultCommand in nereids [doris]
msridhar78 commented on PR #44805: URL: https://github.com/apache/doris/pull/44805#issuecomment-2563929914 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showStagesCommand in nereids [doris]
msridhar78 commented on PR #45373: URL: https://github.com/apache/doris/pull/45373#issuecomment-2563930708 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org