Re: [PR] [feature-wip](Cloud) Support create storage vault stmt [doris]
github-actions[bot] commented on PR #31537: URL: https://github.com/apache/doris/pull/31537#issuecomment-1978163971 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](jdbc catalog) Change Druid Pool dependency to version 1.2.5 [doris]
doris-robot commented on PR #31783: URL: https://github.com/apache/doris/pull/31783#issuecomment-1978164476 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit 11f6fb4b8fdd5e7dfa8d693f321799b676527258 with default session variables Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s Insert into select: 18.4 seconds inserted 1000 Rows, about 543K ops/s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cast) fix wrong result while cast string to float [doris]
doris-robot commented on PR #31781: URL: https://github.com/apache/doris/pull/31781#issuecomment-1978164699 TPC-H: Total hot run time: 38163 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 164945d24157ba7ecfb53ccc1e558cf1c99695f0, data reload: false -- Round 1 -- q1 17925 414941614149 q2 2665154 149 149 q3 11259 1029980 980 q4 4972968 968 968 q5 7844293930132939 q6 180 124 132 124 q7 1315818 814 814 q8 9486204020502040 q9 7386643864276427 q10 8237258625512551 q11 421 219 220 219 q12 744 328 325 325 q13 17957 290229152902 q14 278 254 259 254 q15 502 461 449 449 q16 498 396 404 396 q17 943 874 823 823 q18 6697591360685913 q19 1581154215101510 q20 550 278 283 278 q21 7541371536573657 q22 801 320 296 296 Total cold run time: 109782 ms Total hot run time: 38163 ms - Round 2, with runtime_filter_mode=off - q1 4008399740063997 q2 316 232 238 232 q3 2943293628982898 q4 1858182817771777 q5 5238523752415237 q6 201 116 116 116 q7 2253182518451825 q8 3191324132923241 q9 8521853985438539 q10 6132372036873687 q11 529 435 454 435 q12 697 568 530 530 q13 15503 278627712771 q14 282 266 246 246 q15 482 438 440 438 q16 456 410 423 410 q17 1687168416581658 q18 7771754472777277 q19 1605160416021602 q20 1941170617381706 q21 4923483048834830 q22 542 484 495 484 Total cold run time: 71079 ms Total hot run time: 53936 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [pick][fix](jdbc catalog) Change Druid Pool dependency to version 1.2.5 [doris]
doris-robot commented on PR #31784: URL: https://github.com/apache/doris/pull/31784#issuecomment-1978165080 ClickBench: Total hot run time: 30.94 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit a6e4d5b8f4966b37ea4eb3cadb10bf62cd3185e1, data reload: false query1 0.020.020.02 query2 0.070.030.02 query3 0.250.050.04 query4 1.800.060.07 query5 0.530.520.52 query6 1.240.620.62 query7 0.020.010.01 query8 0.030.030.02 query9 0.510.480.48 query10 0.540.530.53 query11 0.110.090.08 query12 0.120.090.08 query13 0.620.620.62 query14 0.780.800.78 query15 0.770.760.76 query16 0.370.360.36 query17 1.021.021.03 query18 0.220.240.25 query19 1.911.821.82 query20 0.020.010.00 query21 15.46 0.560.56 query22 2.182.371.87 query23 17.64 0.900.91 query24 4.362.950.95 query25 0.360.150.05 query26 0.680.130.14 query27 0.040.040.03 query28 6.960.740.79 query29 12.64 2.322.29 query30 0.580.520.51 query31 2.810.370.37 query32 3.390.500.49 query33 3.023.093.06 query34 15.63 4.834.80 query35 4.874.854.87 query36 1.091.021.03 query37 0.060.050.04 query38 0.030.020.02 query39 0.020.010.01 query40 0.160.140.14 query41 0.070.020.01 query42 0.020.010.02 query43 0.030.010.02 Total cold run time: 103.05 s Total hot run time: 30.94 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhance](multi-catalog) Use MetaIdMappingsLog to replace InitCatalogLog/InitDatabaseLog. [doris]
dutyu commented on PR #31120: URL: https://github.com/apache/doris/pull/31120#issuecomment-1978165813 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Variant) fix variant null bitmap set incorrect [doris]
doris-robot commented on PR #31769: URL: https://github.com/apache/doris/pull/31769#issuecomment-1978165961 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit 33eb01fb13c1dad95b756c90d655f78965875ebd with default session variables Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s Insert into select: 18.4 seconds inserted 1000 Rows, about 543K ops/s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](fe) Speed up fe-core checkstyle [doris]
github-actions[bot] commented on PR #31785: URL: https://github.com/apache/doris/pull/31785#issuecomment-1978167484 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](fe) Speed up fe-core checkstyle [doris]
github-actions[bot] commented on PR #31785: URL: https://github.com/apache/doris/pull/31785#issuecomment-1978167441 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] ci improve [doris-flink-connector]
JNSimba closed pull request #328: ci improve URL: https://github.com/apache/doris-flink-connector/pull/328 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](doris compose) Add no-detach mode && use java 17 image [doris]
github-actions[bot] commented on PR #31775: URL: https://github.com/apache/doris/pull/31775#issuecomment-1978168283 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](doris compose) Add no-detach mode && use java 17 image [doris]
dataroaring merged PR #31775: URL: https://github.com/apache/doris/pull/31775 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](merge-cloud) Make MetaService compatible with selectdb cloud [doris]
github-actions[bot] commented on PR #31620: URL: https://github.com/apache/doris/pull/31620#issuecomment-1978168461 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (4a3e0bdb135 -> 2a77d8b6f38)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 4a3e0bdb135 [fix](jdbc catalog) fix close abort in sqlserver (#31718) add 2a77d8b6f38 [feature](doris compose) doris compose use jdk 17 image (#31775) No new revisions were added by this update. Summary of changes: docker/runtime/doris-compose/Dockerfile | 32 docker/runtime/doris-compose/command.py | 28 docker/runtime/doris-compose/utils.py | 28 3 files changed, 68 insertions(+), 20 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [pick][fix](jdbc catalog) Change Druid Pool dependency to version 1.2.5 [doris]
doris-robot commented on PR #31784: URL: https://github.com/apache/doris/pull/31784#issuecomment-1978169937 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit a6e4d5b8f4966b37ea4eb3cadb10bf62cd3185e1 with default session variables Stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s Insert into select: 20.5 seconds inserted 1000 Rows, about 487K ops/s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](fe) Speed up fe-core checkstyle [doris]
doris-robot commented on PR #31785: URL: https://github.com/apache/doris/pull/31785#issuecomment-1978171093 TPC-H: Total hot run time: 38172 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 93dd3a80c459cebeef0d13d0a4e58b5bdcb5952a, data reload: false -- Round 1 -- q1 18772 415441324132 q2 2341152 153 152 q3 11100 1035970 970 q4 4979979 958 958 q5 8258286929922869 q6 175 123 128 123 q7 1298837 816 816 q8 9489204020702040 q9 7376649864766476 q10 8234254325702543 q11 418 218 219 218 q12 784 332 321 321 q13 17950 298529572957 q14 282 262 262 262 q15 491 446 447 446 q16 480 402 411 402 q17 949 842 827 827 q18 6792597559085908 q19 1572153315321532 q20 556 291 288 288 q21 7497365836273627 q22 790 305 322 305 Total cold run time: 110583 ms Total hot run time: 38172 ms - Round 2, with runtime_filter_mode=off - q1 3989399139673967 q2 312 217 216 216 q3 2969291529032903 q4 1880181117871787 q5 5231523952475239 q6 202 121 122 121 q7 2283183218181818 q8 3188325532373237 q9 8532860885548554 q10 6153373837133713 q11 530 452 439 439 q12 702 528 562 528 q13 8500281627922792 q14 281 263 259 259 q15 493 448 451 448 q16 435 412 401 401 q17 1686166216661662 q18 7849742773047304 q19 1598162116321621 q20 1959175717541754 q21 4944474948414749 q22 529 473 470 470 Total cold run time: 64245 ms Total hot run time: 53982 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [enhancement](Nereids) refine and speedup analyzer [doris]
924060929 opened a new pull request, #31792: URL: https://github.com/apache/doris/pull/31792 ## Proposed changes draft -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](Nereids) refine and speedup analyzer [doris]
doris-robot commented on PR #31792: URL: https://github.com/apache/doris/pull/31792#issuecomment-1978172791 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](partial-update) Fix wrong column number passing to be when partial and enable nereids [doris]
Yukang-Lian commented on PR #31461: URL: https://github.com/apache/doris/pull/31461#issuecomment-1978173327 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](TrinoConnector-Catalog) Fe supports `trino-connector` catalog [doris]
doris-robot commented on PR #31768: URL: https://github.com/apache/doris/pull/31768#issuecomment-1978174092 TPC-DS: Total hot run time: 177955 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit d0b8d45454e6122c55beff6e18e9dfea52e6b69c, data reload: false query1 929 364 344 344 query2 7444215422112154 query3 6720218 213 213 query4 27816 21178 20947 20947 query5 4325492 4022492 query6 280 184 182 182 query7 4628306 297 297 query8 247 176 181 176 query9 8437220621982198 query10 438 243 224 224 query11 14899 14336 14498 14336 query12 136 90 91 90 query13 1665428 419 419 query14 8879699971206999 query15 236 197 201 197 query16 7258296 267 267 query17 955 591 543 543 query18 1911295 282 282 query19 203 161 154 154 query20 90 90 88 88 query21 200 126 123 123 query22 4743451145394511 query23 31774 30795 30962 30795 query24 12221 308030033003 query25 683 376 382 376 query26 1908162 168 162 query27 3010361 356 356 query28 6611181118241811 query29 1307620 608 608 query30 315 153 161 153 query31 959 747 756 747 query32 104 63 61 61 query33 759 262 265 262 query34 1011455 471 455 query35 935 808 813 808 query36 937 812 839 812 query37 282 72 71 71 query38 3323311331163113 query39 1402137613601360 query40 301 123 132 123 query41 62 55 55 55 query42 109 102 101 101 query43 455 436 420 420 query44 1090708 711 708 query45 220 202 193 193 query46 1057810 790 790 query47 1629155915651559 query48 416 353 348 348 query49 1205352 334 334 query50 788 379 377 377 query51 6781666166136613 query52 109 98 96 96 query53 355 281 288 281 query54 317 250 255 250 query55 87 89 83 83 query56 240 237 228 228 query57 1104100110101001 query58 253 222 222 222 query59 2696239524232395 query60 270 263 271 263 query61 115 114 113 113 query62 661 419 416 416 query63 305 284 287 284 query64 6390339432053205 query65 3039303730153015 query66 1463341 338 338 query67 15225 14246 14294 14246 query68 12224 574 581 574 query69 718 386 410 386 query70 1353111010871087 query71 635 279 302 279 query72 10111 258224542454 query73 2148343 328 328 query74 7191679768176797 query75 6789262626962626 query76 8099111211821112 query77 945 252 260 252 query78 10466 976195329532 query79 10915 513 516 513 query80 952 469 443 443 query81 483 206 211 206 query82 325 91 93 91 query83 265 141 141 141 query84 283 75 80 75 query85 1232332 333 332 query86 379 278 275 275 query87 3433320031343134 query88 3166228622642264 query89 522 395 354 354 query90 2342177 171 171 query91 178 132 126 126 query92 66 49 48 48 query93 4667524 510 510 query94 1559186 185 185 query95 448 344 353 344 query96 597 262 260 260 query97 4002385038863850 query98 231 226 212 212 query99 1094780 748 748 Total cold run time: 314540 ms Total hot run time: 177955 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t
Re: [PR] [chore](ci) remove required of pipelinex [doris]
github-actions[bot] commented on PR #31791: URL: https://github.com/apache/doris/pull/31791#issuecomment-1978176259 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](ci) remove required of pipelinex [doris]
github-actions[bot] commented on PR #31791: URL: https://github.com/apache/doris/pull/31791#issuecomment-1978175092 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](ci) remove required of pipelinex [doris]
zhangstar333 merged PR #31791: URL: https://github.com/apache/doris/pull/31791 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (2a77d8b6f38 -> c1fae97b3e8)
This is an automated email from the ASF dual-hosted git repository. zhangstar333 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 2a77d8b6f38 [feature](doris compose) doris compose use jdk 17 image (#31775) add c1fae97b3e8 [chore](ci) remove required of pipelinex (#31791) No new revisions were added by this update. Summary of changes: .asf.yaml | 1 - 1 file changed, 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cloud-merge) Fix missing the whereColumn field [doris]
luwei16 commented on PR #31786: URL: https://github.com/apache/doris/pull/31786#issuecomment-1978177658 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] pr2 [doris]
hello-stephen opened a new pull request, #31793: URL: https://github.com/apache/doris/pull/31793 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] pr2 [doris]
doris-robot commented on PR #31793: URL: https://github.com/apache/doris/pull/31793#issuecomment-1978178556 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] pr2 [doris]
hello-stephen commented on PR #31793: URL: https://github.com/apache/doris/pull/31793#issuecomment-1978178734 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](txn insert) txn insert support insert into select [doris]
zhannngchen commented on code in PR #31666:
URL: https://github.com/apache/doris/pull/31666#discussion_r1512308584
##
fe/fe-core/src/main/java/org/apache/doris/transaction/TransactionEntry.java:
##
@@ -18,32 +18,68 @@
package org.apache.doris.transaction;
import org.apache.doris.catalog.Database;
+import org.apache.doris.catalog.DatabaseIf;
+import org.apache.doris.catalog.Env;
import org.apache.doris.catalog.Table;
+import org.apache.doris.catalog.TableIf;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.DdlException;
+import org.apache.doris.common.MetaNotFoundException;
+import org.apache.doris.common.QuotaExceedException;
+import org.apache.doris.common.UserException;
import org.apache.doris.proto.InternalService;
import org.apache.doris.proto.Types;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.InsertStreamTxnExecutor;
+import org.apache.doris.qe.MasterTxnExecutor;
+import org.apache.doris.service.FrontendOptions;
import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TTabletCommitInfo;
import org.apache.doris.thrift.TTxnParams;
+import org.apache.doris.thrift.TWaitingTxnStatusRequest;
+import org.apache.doris.thrift.TWaitingTxnStatusResult;
+import org.apache.doris.transaction.TransactionState.LoadJobSourceType;
+import org.apache.doris.transaction.TransactionState.TxnCoordinator;
+import org.apache.doris.transaction.TransactionState.TxnSourceType;
+
+import com.google.common.collect.Lists;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.apache.thrift.TException;
import java.util.ArrayList;
import java.util.List;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.TimeoutException;
public class TransactionEntry {
+private static final Logger LOG =
LogManager.getLogger(TransactionEntry.class);
+
private String label = "";
-private Database db;
+private DatabaseIf database;
+
+// for insert into values for one table
private Table table;
private Backend backend;
private TTxnParams txnConf;
private List dataToSend = new ArrayList<>();
private long rowsInTransaction = 0;
private Types.PUniqueId pLoadId;
+// for insert into select for multi tables
+private boolean isTransactionBegan = false;
Review Comment:
Using `isTransactionBegan` to differentiate between insert into values and
insert into select doesn't quite MAKE SENSE, I totally missed it the first
time. It would be better to use a meaningful variable name to do so, or add an
enum variable and use a function like check_compatibility() to check the
compatibility of different inserts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](TrinoConnector-Catalog) Fe supports `trino-connector` catalog [doris]
doris-robot commented on PR #31768: URL: https://github.com/apache/doris/pull/31768#issuecomment-1978181641 ClickBench: Total hot run time: 30.34 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit d0b8d45454e6122c55beff6e18e9dfea52e6b69c, data reload: false query1 0.040.040.04 query2 0.060.020.02 query3 0.240.060.06 query4 1.640.100.10 query5 0.530.500.48 query6 1.300.670.67 query7 0.020.010.01 query8 0.040.030.03 query9 0.560.520.52 query10 0.570.580.58 query11 0.130.100.10 query12 0.120.100.10 query13 0.580.570.57 query14 0.750.750.75 query15 0.820.800.80 query16 0.380.370.39 query17 0.950.980.96 query18 0.260.260.26 query19 1.811.681.72 query20 0.010.010.01 query21 15.41 0.640.66 query22 3.695.242.25 query23 17.47 0.981.12 query24 2.290.610.20 query25 0.210.070.04 query26 0.160.130.13 query27 0.040.030.03 query28 12.12 0.850.82 query29 12.59 3.193.12 query30 0.620.570.53 query31 2.810.350.33 query32 3.370.440.43 query33 2.932.912.86 query34 15.56 4.314.29 query35 4.304.294.32 query36 1.091.011.02 query37 0.070.060.05 query38 0.040.030.03 query39 0.020.020.02 query40 0.170.140.12 query41 0.070.030.02 query42 0.020.020.02 query43 0.040.020.02 Total cold run time: 105.9 s Total hot run time: 30.34 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cast) fix wrong result while cast string to float [doris]
doris-robot commented on PR #31781: URL: https://github.com/apache/doris/pull/31781#issuecomment-1978182056 TPC-DS: Total hot run time: 177768 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 164945d24157ba7ecfb53ccc1e558cf1c99695f0, data reload: false query1 915 356 336 336 query2 7334218820392039 query3 6702216 212 212 query4 25020 21145 21205 21145 query5 4337447 447 447 query6 280 188 189 188 query7 4636333 299 299 query8 253 178 169 169 query9 8402220221972197 query10 433 230 226 226 query11 15169 14296 14419 14296 query12 130 90 84 84 query13 1632437 434 434 query14 8958737070777077 query15 228 197 201 197 query16 7239266 282 266 query17 961 613 573 573 query18 1930290 286 286 query19 208 169 165 165 query20 95 94 95 94 query21 200 133 126 126 query22 4584448644714471 query23 31844 30780 30917 30780 query24 12043 302930913029 query25 704 372 374 372 query26 1902161 172 161 query27 2962357 364 357 query28 6690180218171802 query29 1285628 591 591 query30 315 145 149 145 query31 934 740 745 740 query32 100 58 61 58 query33 749 253 244 244 query34 1021458 463 458 query35 910 817 793 793 query36 951 821 805 805 query37 272 71 67 67 query38 3214309231103092 query39 1403139713661366 query40 297 116 113 113 query41 58 50 55 50 query42 106 101 103 101 query43 435 401 397 397 query44 1076715 709 709 query45 202 192 199 192 query46 1056792 762 762 query47 1632154315551543 query48 415 369 337 337 query49 1189330 324 324 query50 799 369 368 368 query51 6761654665946546 query52 105 102 98 98 query53 346 284 285 284 query54 314 229 238 229 query55 86 86 85 85 query56 236 284 214 214 query57 1077993 1002993 query58 237 209 229 209 query59 2414228322172217 query60 292 274 264 264 query61 117 111 112 111 query62 636 409 417 409 query63 308 287 289 287 query64 6360302531723025 query65 3043300829942994 query66 1470338 336 336 query67 15115 14564 14396 14396 query68 9670575 582 575 query69 620 393 388 388 query70 1275113510511051 query71 570 279 287 279 query72 8746278326192619 query73 1019326 333 326 query74 7154690567646764 query75 5022264226602642 query76 6332112511881125 query77 903 260 268 260 query78 10474 955495489548 query79 12241 526 523 523 query80 1254385 380 380 query81 484 208 203 203 query82 667 89 88 88 query83 199 154 147 147 query84 283 80 80 80 query85 1470336 324 324 query86 429 309 316 309 query87 3412319532233195 query88 3709228522862285 query89 521 364 361 361 query90 2109171 186 171 query91 160 129 144 129 query92 60 51 51 51 query93 6562535 522 522 query94 1325186 184 184 query95 444 348 343 343 query96 610 260 259 259 query97 3970392438943894 query98 222 218 208 208 query99 1225774 755 755 Total cold run time: 306268 ms Total hot run time: 177768 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th
Re: [PR] [enhance](mtmv)MTMV Use partial partition of base table [doris]
morrySnow commented on code in PR #31632:
URL: https://github.com/apache/doris/pull/31632#discussion_r1512296667
##
fe/fe-core/src/main/java/org/apache/doris/mtmv/MTMVUtil.java:
##
@@ -66,4 +83,115 @@ public static boolean mtmvContainsExternalTable(MTMV mtmv) {
}
return false;
}
+
+/**
+ * Obtain the minimum second from `syncLimit` `timeUnit` ago
+ *
+ * @param timeUnit
+ * @param syncLimit
+ * @return
+ * @throws AnalysisException
+ */
+public static long getNowTruncSubSec(MTMVPartitionSyncTimeUnit timeUnit,
int syncLimit)
+throws AnalysisException {
+if (syncLimit < 1) {
+throw new AnalysisException("Unexpected syncLimit, syncLimit: " +
syncLimit);
+}
+// get current time
+Expression now = DateTimeAcquire.now();
+if (!(now instanceof DateTimeLiteral)) {
+throw new AnalysisException("Obtaining current time does not meet
expectations, now: " + now);
Review Comment:
error msg should tell user what do u expect
##
fe/fe-core/src/main/java/org/apache/doris/catalog/MTMV.java:
##
@@ -240,10 +242,14 @@ public Optional getWorkloadGroup() {
}
}
+public MTMVPartitionSyncConfig getPartitionSyncConfig() {
+return
MTMVUtil.generateMTMVPartitionSyncConfigByProperties(mvProperties);
+}
Review Comment:
if we use this function multi times, do we need merialize the result of this
function for speed up?
if we only use this function one time, why wrap it?
##
fe/fe-core/src/main/java/org/apache/doris/catalog/ListPartitionItem.java:
##
@@ -99,6 +101,25 @@ public PartitionKeyDesc toPartitionKeyDesc(int pos) throws
AnalysisException {
return PartitionKeyDesc.createIn(Lists.newArrayList(res));
}
+@Override
+public boolean isSatisfyConfig(int pos, Optional
dateFormatOptional, long nowTruncSubSec)
Review Comment:
function name is not very good. satisfy config is very common, cannot know
satisfy what config
##
fe/fe-core/src/main/java/org/apache/doris/mtmv/MTMVPartitionSyncTimeUnit.java:
##
@@ -0,0 +1,37 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.mtmv;
+
+import java.util.Optional;
+
+public enum MTMVPartitionSyncTimeUnit {
+YEAR,
+MONTH,
+DAY;
+
Review Comment:
mulit blank line
##
fe/fe-core/src/main/java/org/apache/doris/mtmv/MTMVUtil.java:
##
@@ -17,6 +17,10 @@
package org.apache.doris.mtmv;
+import org.apache.doris.analysis.DateLiteral;
Review Comment:
if u mixed-use same name class in analysis and Nereids, u should use simple
name from one package to avoid typo. for this file, u should always use simple
name for Nereids' expression, and use full-qualifier name for legacy planner's
expression.
##
fe/fe-core/src/main/java/org/apache/doris/catalog/MTMV.java:
##
@@ -217,7 +219,7 @@ public Map alterMvProperties(Map mvProperties) {
public long getGracePeriod() {
readMvLock();
try {
-if
(mvProperties.containsKey(PropertyAnalyzer.PROPERTIES_GRACE_PERIOD)) {
+if
(!StringUtils.isEmpty(mvProperties.get(PropertyAnalyzer.PROPERTIES_GRACE_PERIOD)))
{
Review Comment:
because we allow empty string? should we add a check for properties?
##
fe/fe-core/src/main/java/org/apache/doris/mtmv/MTMVUtil.java:
##
@@ -66,4 +83,115 @@ public static boolean mtmvContainsExternalTable(MTMV mtmv) {
}
return false;
}
+
+/**
+ * Obtain the minimum second from `syncLimit` `timeUnit` ago
+ *
+ * @param timeUnit
+ * @param syncLimit
+ * @return
+ * @throws AnalysisException
+ */
+public static long getNowTruncSubSec(MTMVPartitionSyncTimeUnit timeUnit,
int syncLimit)
+throws AnalysisException {
+if (syncLimit < 1) {
+throw new AnalysisException("Unexpected syncLimit, syncLimit: " +
syncLimit);
+}
+// get current time
+Expression now = DateTimeAcquire.now();
+if (!(now instanceof DateTimeLiteral)) {
+throw new AnalysisExce
Re: [PR] [Feature](TrinoConnector-Catalog) Fe supports `trino-connector` catalog [doris]
doris-robot commented on PR #31768: URL: https://github.com/apache/doris/pull/31768#issuecomment-1978186889 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit d0b8d45454e6122c55beff6e18e9dfea52e6b69c with default session variables Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s Insert into select: 18.3 seconds inserted 1000 Rows, about 546K ops/s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] ci improve [doris-flink-connector]
JNSimba closed pull request #328: ci improve URL: https://github.com/apache/doris-flink-connector/pull/328 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] pr2 [doris]
github-actions[bot] commented on PR #31793: URL: https://github.com/apache/doris/pull/31793#issuecomment-1978187014 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] pr2 [doris]
github-actions[bot] commented on PR #31793: URL: https://github.com/apache/doris/pull/31793#issuecomment-1978187082 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](fe) Speed up fe-core checkstyle [doris]
doris-robot commented on PR #31785: URL: https://github.com/apache/doris/pull/31785#issuecomment-1978188791 TPC-DS: Total hot run time: 177378 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 93dd3a80c459cebeef0d13d0a4e58b5bdcb5952a, data reload: false query1 915 358 350 350 query2 7471200720442007 query3 6696209 215 209 query4 27088 21039 20982 20982 query5 4328475 511 475 query6 292 179 177 177 query7 4649307 291 291 query8 235 169 167 167 query9 8512218821872187 query10 434 252 230 230 query11 14852 14353 14359 14353 query12 133 100 92 92 query13 1656429 449 429 query14 8846721368126812 query15 240 189 193 189 query16 7233270 270 270 query17 934 589 551 551 query18 1924281 279 279 query19 189 162 164 162 query20 95 92 89 89 query21 204 135 124 124 query22 4732456345184518 query23 31818 30623 30495 30495 query24 12002 308231313082 query25 691 398 410 398 query26 1854174 165 165 query27 2926357 372 357 query28 6752184118191819 query29 1268619 603 603 query30 321 148 148 148 query31 911 782 761 761 query32 106 65 60 60 query33 766 272 250 250 query34 970 462 482 462 query35 918 826 832 826 query36 950 814 817 814 query37 273 72 69 69 query38 3250316931113111 query39 1396136013761360 query40 302 117 115 115 query41 59 55 57 55 query42 105 103 101 101 query43 448 406 414 406 query44 1076712 713 712 query45 208 206 199 199 query46 1046782 786 782 query47 1640159415651565 query48 416 347 346 346 query49 1206349 353 349 query50 798 379 385 379 query51 6625663565416541 query52 110 101 96 96 query53 352 286 283 283 query54 319 252 254 252 query55 88 84 84 84 query56 247 261 220 220 query57 1099100110231001 query58 258 224 219 219 query59 2569235522982298 query60 269 264 254 254 query61 117 113 130 113 query62 637 404 418 404 query63 301 281 280 280 query64 6373327032373237 query65 3013305929742974 query66 1461341 341 341 query67 15295 14588 14504 14504 query68 15732 546 594 546 query69 718 390 387 387 query70 1455111510991099 query71 624 267 281 267 query72 10182 260724632463 query73 3528347 333 333 query74 7261666469406664 query75 6904263426612634 query76 8114111012151110 query77 994 254 252 252 query78 10133 945396309453 query79 8529504 514 504 query80 1098398 441 398 query81 473 209 209 209 query82 244 85 92 85 query83 402 152 141 141 query84 280 82 78 78 query85 1185335 330 330 query86 389 278 329 278 query87 3438319232913192 query88 3055228122582258 query89 507 372 351 351 query90 2486182 173 173 query91 158 126 129 126 query92 61 51 58 51 query93 2937527 513 513 query94 1588190 187 187 query95 459 356 366 356 query96 602 261 264 261 query97 4054389438813881 query98 226 220 211 211 query99 1077777 754 754 Total cold run time: 314160 ms Total hot run time: 177378 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t
[PR] Support column level health value. [doris]
Jibing-Li opened a new pull request, #31794: URL: https://github.com/apache/doris/pull/31794 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Support column level health value. [doris]
doris-robot commented on PR #31794: URL: https://github.com/apache/doris/pull/31794#issuecomment-1978189929 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](Nereids) refine and speedup analyzer [doris]
924060929 commented on PR #31792: URL: https://github.com/apache/doris/pull/31792#issuecomment-1978190992 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cast) fix wrong result while cast string to float [doris]
doris-robot commented on PR #31781: URL: https://github.com/apache/doris/pull/31781#issuecomment-1978190776 ClickBench: Total hot run time: 30.07 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 164945d24157ba7ecfb53ccc1e558cf1c99695f0, data reload: false query1 0.040.030.03 query2 0.060.020.02 query3 0.240.060.06 query4 1.650.100.10 query5 0.490.480.49 query6 1.290.660.67 query7 0.020.020.01 query8 0.040.020.03 query9 0.550.520.51 query10 0.560.570.57 query11 0.130.100.10 query12 0.120.100.10 query13 0.580.590.56 query14 0.750.740.75 query15 0.810.810.80 query16 0.390.370.37 query17 1.050.980.99 query18 0.270.260.27 query19 1.831.701.71 query20 0.010.020.01 query21 15.41 0.610.62 query22 3.953.511.67 query23 17.38 1.091.03 query24 2.190.220.49 query25 0.250.050.04 query26 0.190.130.13 query27 0.030.030.03 query28 12.41 0.840.82 query29 12.52 3.223.68 query30 0.630.590.59 query31 2.800.340.34 query32 3.400.440.44 query33 2.932.942.93 query34 15.47 4.314.30 query35 4.284.314.33 query36 1.091.021.03 query37 0.060.050.05 query38 0.050.030.03 query39 0.020.020.02 query40 0.160.130.14 query41 0.080.020.02 query42 0.030.020.02 query43 0.030.020.03 Total cold run time: 106.24 s Total hot run time: 30.07 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Support column level health value. [doris]
Jibing-Li merged PR #31794: URL: https://github.com/apache/doris/pull/31794 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](Nereids) change consumer rules name [doris]
morrySnow merged PR #31777: URL: https://github.com/apache/doris/pull/31777 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [chore](ci) add back required to test [doris]
hello-stephen opened a new pull request, #31795: URL: https://github.com/apache/doris/pull/31795 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (2bbbdae6a2e -> 021cbfbe0e4)
This is an automated email from the ASF dual-hosted git repository. morrysnow pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 2bbbdae6a2e [fix](hive) support partition prune for _HIVE_DEFAULT_PARTITION_ (#31736) add 021cbfbe0e4 [chore](Nereids) change consumer rules name (#31777) No new revisions were added by this update. Summary of changes: .../src/main/java/org/apache/doris/nereids/rules/RuleType.java | 6 +++--- .../doris/nereids/rules/rewrite/CollectFilterAboveConsumer.java | 2 +- .../doris/nereids/rules/rewrite/CollectProjectAboveConsumer.java| 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](ci) add back required to test [doris]
doris-robot commented on PR #31795: URL: https://github.com/apache/doris/pull/31795#issuecomment-1978191791 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch high-priority-column updated: Support column level health value. (#31794)
This is an automated email from the ASF dual-hosted git repository.
lijibing pushed a commit to branch high-priority-column
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/high-priority-column by this
push:
new 65a71e86856 Support column level health value. (#31794)
65a71e86856 is described below
commit 65a71e86856721ff0304e8d8889b3ddbe5ba8ee5
Author: Jibing-Li <64681310+jibing...@users.noreply.github.com>
AuthorDate: Tue Mar 5 16:19:08 2024 +0800
Support column level health value. (#31794)
---
.../apache/doris/analysis/ShowColumnStatsStmt.java | 4 +
.../apache/doris/datasource/InternalCatalog.java | 6 +-
.../java/org/apache/doris/qe/SessionVariable.java | 7 ++
.../org/apache/doris/statistics/AnalysisInfo.java | 17 ++--
.../doris/statistics/AnalysisInfoBuilder.java | 17 ++--
.../org/apache/doris/statistics/AnalysisJob.java | 8 +-
.../apache/doris/statistics/AnalysisManager.java | 13 ++-
.../org/apache/doris/statistics/ColStatsMeta.java | 16 ++--
.../apache/doris/statistics/OlapAnalysisTask.java | 2 +-
.../doris/statistics/StatisticsAutoCollector.java | 95 --
.../doris/statistics/StatisticsJobAppender.java| 28 +--
.../apache/doris/statistics/TableStatsMeta.java| 33
.../doris/statistics/util/StatisticsUtil.java | 10 +++
.../apache/doris/statistics/AnalysisJobTest.java | 4 +-
.../doris/statistics/AnalysisManagerTest.java | 2 +-
15 files changed, 198 insertions(+), 64 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowColumnStatsStmt.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowColumnStatsStmt.java
index 37be76b20df..749bfa7d360 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowColumnStatsStmt.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowColumnStatsStmt.java
@@ -61,6 +61,8 @@ public class ShowColumnStatsStmt extends ShowStmt {
.add("trigger")
.add("query_times")
.add("updated_time")
+.add("update_rows")
+.add("last_analyze_row_count")
.build();
private final TableName tableName;
@@ -160,6 +162,8 @@ public class ShowColumnStatsStmt extends ShowStmt {
row.add(String.valueOf(colStatsMeta == null ? "N/A" :
colStatsMeta.jobType));
row.add(String.valueOf(colStatsMeta == null ? "N/A" :
colStatsMeta.queriedTimes));
row.add(String.valueOf(p.second.updatedTime));
+row.add(String.valueOf(colStatsMeta == null ? "N/A" :
colStatsMeta.updatedRows));
+row.add(String.valueOf(colStatsMeta == null ? "N/A" :
colStatsMeta.rowCount));
result.add(row);
});
return new ShowResultSet(getMetaData(), result);
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
b/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
index 95abc9e06c4..61a1073da34 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
@@ -3116,7 +3116,6 @@ public class InternalCatalog implements
CatalogIf {
rowsToTruncate += partition.getBaseIndex().getRowCount();
}
} else {
-rowsToTruncate = olapTable.getRowCount();
for (Partition partition : olapTable.getPartitions()) {
// If need absolutely correct, should check running txn
here.
// But if the txn is in prepare state, cann't known which
partitions had load data.
@@ -3125,6 +3124,7 @@ public class InternalCatalog implements
CatalogIf {
}
origPartitions.put(partition.getName(), partition.getId());
partitionsDistributionInfo.put(partition.getId(),
partition.getDistributionInfo());
+rowsToTruncate += partition.getBaseIndex().getRowCount();
}
}
// if table currently has no partitions, this sql like empty
command and do nothing, should return directly.
@@ -3285,10 +3285,8 @@ public class InternalCatalog implements
CatalogIf {
if (truncateEntireTable) {
// Drop the whole table stats after truncate the entire table
Env.getCurrentEnv().getAnalysisManager().dropStats(olapTable);
-} else {
-// Update the updated rows in table stats after truncate some
partitions.
-
Env.getCurrentEnv().getAnalysisManager().updateUpdatedRows(updateRecords);
}
+
Env.getCurrentEnv().getAnalysisManager().updateUpdatedRows(updateRecords);
LOG.info("finished to truncate table {}, partitions: {}",
tblRef.getName().toSql(), tblRef.getPartitionNames());
}
dif
Re: [PR] [chore](ci) add back required to test [doris]
hello-stephen commented on PR #31795: URL: https://github.com/apache/doris/pull/31795#issuecomment-1978192297 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](mtmv)MTMV Use partial partition of base table [doris]
zddr commented on code in PR #31632:
URL: https://github.com/apache/doris/pull/31632#discussion_r1512317402
##
fe/fe-core/src/main/java/org/apache/doris/catalog/MTMV.java:
##
@@ -217,7 +219,7 @@ public Map alterMvProperties(Map mvProperties) {
public long getGracePeriod() {
readMvLock();
try {
-if
(mvProperties.containsKey(PropertyAnalyzer.PROPERTIES_GRACE_PERIOD)) {
+if
(!StringUtils.isEmpty(mvProperties.get(PropertyAnalyzer.PROPERTIES_GRACE_PERIOD)))
{
Review Comment:
we use set xxx="" restore property to default configuration
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cloud-merge) Fix missing the whereColumn field [doris]
doris-robot commented on PR #31786: URL: https://github.com/apache/doris/pull/31786#issuecomment-1978193629 TPC-H: Total hot run time: 37729 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit d6018d7e7afe8a755967ae7d0f1595ed38bbdaac, data reload: false -- Round 1 -- q1 17653 402840304028 q2 2022148 146 146 q3 10557 945 929 929 q4 4663936 932 932 q5 7592286630052866 q6 182 127 125 125 q7 1294821 816 816 q8 9475201720732017 q9 7204645764326432 q10 8221255025682550 q11 426 222 214 214 q12 770 324 312 312 q13 17949 292029622920 q14 284 245 252 245 q15 480 449 442 442 q16 464 395 396 395 q17 930 837 835 835 q18 576960025769 q19 1558151514991499 q20 554 287 296 287 q21 7520366136973661 q22 803 331 309 309 Total cold run time: 107267 ms Total hot run time: 37729 ms - Round 2, with runtime_filter_mode=off - q1 4041399939973997 q2 328 233 229 229 q3 2956292628722872 q4 1819181518101810 q5 5221525252285228 q6 208 119 120 119 q7 2292183318191819 q8 3221325732503250 q9 8529854185358535 q10 6145377536963696 q11 528 434 431 431 q12 697 533 498 498 q13 10863 275727662757 q14 286 258 253 253 q15 471 443 441 441 q16 454 416 419 416 q17 1685167416421642 q18 7745740872687268 q19 4044162215911591 q20 1959176116741674 q21 4998479648614796 q22 514 473 401 401 Total cold run time: 69004 ms Total hot run time: 53723 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cast) fix wrong result while cast string to float [doris]
doris-robot commented on PR #31781: URL: https://github.com/apache/doris/pull/31781#issuecomment-1978195743 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit 164945d24157ba7ecfb53ccc1e558cf1c99695f0 with default session variables Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s Insert into select: 18.2 seconds inserted 1000 Rows, about 549K ops/s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] ci improve [doris-flink-connector]
JNSimba closed pull request #328: ci improve URL: https://github.com/apache/doris-flink-connector/pull/328 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](fe) Speed up fe-core checkstyle [doris]
doris-robot commented on PR #31785: URL: https://github.com/apache/doris/pull/31785#issuecomment-1978197217 ClickBench: Total hot run time: 31.38 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 93dd3a80c459cebeef0d13d0a4e58b5bdcb5952a, data reload: false query1 0.040.040.04 query2 0.060.030.02 query3 0.240.060.05 query4 1.670.100.10 query5 0.500.510.49 query6 1.300.680.67 query7 0.020.010.02 query8 0.040.030.04 query9 0.570.520.52 query10 0.580.570.57 query11 0.120.100.10 query12 0.120.100.10 query13 0.590.570.57 query14 0.740.740.74 query15 0.820.820.80 query16 0.390.360.38 query17 1.011.000.98 query18 0.270.260.26 query19 1.931.801.76 query20 0.020.010.01 query21 15.41 0.630.62 query22 3.144.222.88 query23 17.60 0.960.92 query24 2.150.570.37 query25 0.290.040.03 query26 0.190.130.14 query27 0.040.030.03 query28 12.17 0.840.82 query29 12.51 3.513.30 query30 0.620.540.59 query31 2.790.340.34 query32 3.370.430.44 query33 2.952.892.90 query34 15.50 4.324.30 query35 4.324.314.29 query36 1.081.021.02 query37 0.070.060.05 query38 0.040.040.04 query39 0.020.020.01 query40 0.170.130.16 query41 0.070.020.02 query42 0.030.020.02 query43 0.020.020.03 Total cold run time: 105.58 s Total hot run time: 31.38 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](ci) add back required to test [doris]
github-actions[bot] commented on PR #31795: URL: https://github.com/apache/doris/pull/31795#issuecomment-1978198793 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](ci) add back required to test [doris]
github-actions[bot] commented on PR #31795: URL: https://github.com/apache/doris/pull/31795#issuecomment-1978198861 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Improve] fix connection close in test case [doris-flink-connector]
JNSimba opened a new pull request, #329: URL: https://github.com/apache/doris-flink-connector/pull/329 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Bug](load) fix wrong data in mv when routine load with function mapping [doris]
doris-robot commented on PR #31787: URL: https://github.com/apache/doris/pull/31787#issuecomment-1978201784 TPC-H: Total hot run time: 37743 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 17df853e10256a4add62cbed770eef2b0162749f, data reload: false -- Round 1 -- q1 17649 402940564029 q2 2021150 153 150 q3 10652 942 931 931 q4 4676936 951 936 q5 7621291929782919 q6 180 122 126 122 q7 1311834 795 795 q8 9520203520312031 q9 7278639863936393 q10 8233259126112591 q11 421 221 217 217 q12 733 323 307 307 q13 17980 292928782878 q14 287 248 249 248 q15 490 446 443 443 q16 503 395 393 393 q17 938 863 826 826 q18 6675598457595759 q19 1554150815131508 q20 545 295 277 277 q21 7500371336953695 q22 795 295 299 295 Total cold run time: 107562 ms Total hot run time: 37743 ms - Round 2, with runtime_filter_mode=off - q1 3999396639683966 q2 313 223 227 223 q3 2929294628962896 q4 1842181018381810 q5 5225520552075205 q6 215 114 122 114 q7 2271182918331829 q8 3191324532453245 q9 8469848985348489 q10 6094372536963696 q11 528 447 439 439 q12 683 547 558 547 q13 13503 278827742774 q14 280 249 256 249 q15 468 433 453 433 q16 451 393 406 393 q17 1678165616671656 q18 7897737172867286 q19 1740159615911591 q20 1958171917431719 q21 4920481847834783 q22 549 473 477 473 Total cold run time: 69203 ms Total hot run time: 53816 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](fe) Speed up fe-core checkstyle [doris]
doris-robot commented on PR #31785: URL: https://github.com/apache/doris/pull/31785#issuecomment-1978202467 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit 93dd3a80c459cebeef0d13d0a4e58b5bdcb5952a with default session variables Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s Stream load orc: 60 seconds loaded 1101869774 Bytes, about 17 MB/s Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s Insert into select: 18.3 seconds inserted 1000 Rows, about 546K ops/s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [pick][fix](jdbc catalog) Change Druid Pool dependency to version 1.2.5 [doris]
zy-kkk merged PR #31784: URL: https://github.com/apache/doris/pull/31784 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated (8c5da176799 -> 546cb1601b6)
This is an automated email from the ASF dual-hosted git repository. zykkk pushed a change to branch branch-2.0 in repository https://gitbox.apache.org/repos/asf/doris.git from 8c5da176799 [log](tablet) add gc log (#31185) add 546cb1601b6 [fix](jdbc catalog) Change Druid Pool dependency to version 1.2.5 (#31784) No new revisions were added by this update. Summary of changes: fe/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [nereids](topn-filter) support multi-topn filter (FE part) [doris]
englefly commented on PR #31485: URL: https://github.com/apache/doris/pull/31485#issuecomment-1978205010 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Nereids] add leading tpc-ds [doris]
github-actions[bot] commented on PR #31681: URL: https://github.com/apache/doris/pull/31681#issuecomment-1978208200 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](partial-update) Fix wrong column number passing to be when partial and enable nereids [doris]
doris-robot commented on PR #31461: URL: https://github.com/apache/doris/pull/31461#issuecomment-1978208205 TPC-H: Total hot run time: 37658 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit da83631c761db9122e1ea456d4b15bcea5570757, data reload: false -- Round 1 -- q1 17716 402440334024 q2 2018156 146 146 q3 10556 922 930 922 q4 4665918 934 918 q5 7619282629512826 q6 175 121 129 121 q7 1293829 800 800 q8 9473205920332033 q9 7185639763986397 q10 8171257725492549 q11 417 216 219 216 q12 787 326 308 308 q13 17972 290329302903 q14 285 249 252 249 q15 483 440 444 440 q16 504 396 398 396 q17 926 888 883 883 q18 6781603758535853 q19 1558150914961496 q20 563 283 275 275 q21 7483362136153615 q22 785 288 340 288 Total cold run time: 107415 ms Total hot run time: 37658 ms - Round 2, with runtime_filter_mode=off - q1 3990403539733973 q2 323 231 223 223 q3 2939293228962896 q4 1825180217961796 q5 5226520552175205 q6 199 116 117 116 q7 2239177318051773 q8 3193322832553228 q9 8465847984408440 q10 6094372437123712 q11 518 446 434 434 q12 694 553 543 543 q13 14445 281027562756 q14 276 250 263 250 q15 469 438 431 431 q16 451 418 408 408 q17 1663162416161616 q18 7778748671927192 q19 2895159015861586 q20 1930172616721672 q21 4917472847134713 q22 538 468 462 462 Total cold run time: 71067 ms Total hot run time: 53425 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Nereids] add leading tpc-ds [doris]
github-actions[bot] commented on PR #31681: URL: https://github.com/apache/doris/pull/31681#issuecomment-1978208248 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) 01/03: Collect high priority columns. (#31235)
This is an automated email from the ASF dual-hosted git repository.
lijibing pushed a commit to branch high-priority-column
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 0f97217b72bda1c6e1b2172af244e8dcfe31c701
Author: Jibing-Li <64681310+jibing...@users.noreply.github.com>
AuthorDate: Wed Feb 28 11:54:40 2024 +0800
Collect high priority columns. (#31235)
---
.../doris/nereids/jobs/executor/Rewriter.java | 4 +-
.../org/apache/doris/nereids/rules/RuleType.java | 1 +
.../expression/HighPriorityColumnCollector.java| 202 +
.../apache/doris/statistics/AnalysisManager.java | 44 +
.../doris/statistics/HighPriorityColumn.java | 55 ++
5 files changed, 305 insertions(+), 1 deletion(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/executor/Rewriter.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/executor/Rewriter.java
index db463354120..8976e274893 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/executor/Rewriter.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/executor/Rewriter.java
@@ -32,6 +32,7 @@ import
org.apache.doris.nereids.rules.expression.CheckLegalityAfterRewrite;
import org.apache.doris.nereids.rules.expression.ExpressionNormalization;
import org.apache.doris.nereids.rules.expression.ExpressionOptimization;
import org.apache.doris.nereids.rules.expression.ExpressionRewrite;
+import org.apache.doris.nereids.rules.expression.HighPriorityColumnCollector;
import org.apache.doris.nereids.rules.rewrite.AddDefaultLimit;
import org.apache.doris.nereids.rules.rewrite.AdjustConjunctsReturnType;
import org.apache.doris.nereids.rules.rewrite.AdjustNullable;
@@ -453,7 +454,8 @@ public class Rewriter extends AbstractBatchJobExecutor {
new CollectFilterAboveConsumer(),
new CollectProjectAboveConsumer()
)
-)
+),
+topic("Collect used column", custom(RuleType.COLLECT_COLUMNS,
HighPriorityColumnCollector::new))
);
private static final List WHOLE_TREE_REWRITE_JOBS
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/RuleType.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/RuleType.java
index 282c131faa4..963b2145a91 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/RuleType.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/RuleType.java
@@ -300,6 +300,7 @@ public enum RuleType {
LEADING_JOIN(RuleTypeClass.REWRITE),
REWRITE_SENTINEL(RuleTypeClass.REWRITE),
+COLLECT_COLUMNS(RuleTypeClass.REWRITE),
// topn opts
DEFER_MATERIALIZE_TOP_N_RESULT(RuleTypeClass.REWRITE),
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/HighPriorityColumnCollector.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/HighPriorityColumnCollector.java
new file mode 100644
index 000..ed67ad97005
--- /dev/null
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/HighPriorityColumnCollector.java
@@ -0,0 +1,202 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression;
+
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.TableIf;
+import org.apache.doris.nereids.jobs.JobContext;
+import
org.apache.doris.nereids.rules.expression.HighPriorityColumnCollector.CollectorContext;
+import org.apache.doris.nereids.trees.expressions.NamedExpression;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.expressions.SlotReference;
+import org.apache.doris.nereids.trees.plans.Plan;
+import org.apache.doris.nereids.trees.plans.logical.LogicalAggregate;
+import org.apache.doris.nereids.trees.plans.logical.LogicalCatalogRelation;
+import org.apache.doris.nereids.trees.plans.logical.LogicalFileScan;
+import org.apache.doris.nereids.trees.plans.logical.LogicalFilter;
+import org.apache.doris.nereids.trees.plans.logical.LogicalHaving;
+import org.apache.doris.nereids.trees.plans.logical.LogicalJoin;
+im
(doris) 02/03: High priority queue and map. (#31509)
This is an automated email from the ASF dual-hosted git repository.
lijibing pushed a commit to branch high-priority-column
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 2a0b3add07d0ca1f61ab1d9416d19f3d1e21f1b1
Author: Jibing-Li <64681310+jibing...@users.noreply.github.com>
AuthorDate: Wed Feb 28 12:01:55 2024 +0800
High priority queue and map. (#31509)
---
.../main/java/org/apache/doris/catalog/Env.java| 20 +-
.../apache/doris/statistics/AnalysisManager.java | 33 +-
.../doris/statistics/StatisticsAutoCollector.java | 202 ++--
.../doris/statistics/StatisticsCollector.java | 4 -
.../doris/statistics/StatisticsJobAppender.java| 135 +
.../statistics/StatisticsAutoCollectorTest.java| 560 -
6 files changed, 213 insertions(+), 741 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
index 10417dca367..823b7f18fda 100755
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
@@ -243,6 +243,7 @@ import org.apache.doris.statistics.AnalysisManager;
import org.apache.doris.statistics.StatisticsAutoCollector;
import org.apache.doris.statistics.StatisticsCache;
import org.apache.doris.statistics.StatisticsCleaner;
+import org.apache.doris.statistics.StatisticsJobAppender;
import org.apache.doris.statistics.query.QueryStats;
import org.apache.doris.system.Backend;
import org.apache.doris.system.Frontend;
@@ -516,6 +517,8 @@ public class Env {
private StatisticsAutoCollector statisticsAutoCollector;
+private StatisticsJobAppender statisticsJobAppender;
+
private HiveTransactionMgr hiveTransactionMgr;
private TopicPublisherThread topicPublisherThread;
@@ -743,6 +746,7 @@ public class Env {
this.analysisManager = new AnalysisManager();
this.statisticsCleaner = new StatisticsCleaner();
this.statisticsAutoCollector = new StatisticsAutoCollector();
+this.statisticsJobAppender = new StatisticsJobAppender();
this.globalFunctionMgr = new GlobalFunctionMgr();
this.workloadGroupMgr = new WorkloadGroupMgr();
this.workloadSchedPolicyMgr = new WorkloadSchedPolicyMgr();
@@ -1019,13 +1023,6 @@ public class Env {
// If not using bdb, we need to notify the FE type transfer
manually.
notifyNewFETypeTransfer(FrontendNodeType.MASTER);
}
-if (statisticsCleaner != null) {
-statisticsCleaner.start();
-}
-if (statisticsAutoCollector != null) {
-statisticsAutoCollector.start();
-}
-
queryCancelWorker.start();
TopicPublisher wgPublisher = new WorkloadGroupPublisher(this);
@@ -1663,6 +1660,11 @@ public class Env {
binlogGcer.start();
columnIdFlusher.start();
insertOverwriteManager.start();
+
+// auto analyze related threads.
+statisticsCleaner.start();
+statisticsAutoCollector.start();
+statisticsJobAppender.start();
}
// start threads that should running on all FE
@@ -5998,6 +6000,10 @@ public class Env {
return statisticsAutoCollector;
}
+public StatisticsJobAppender getStatisticsJobAppender() {
+return statisticsJobAppender;
+}
+
public void alterMTMVRefreshInfo(AlterMTMVRefreshInfo info) {
AlterMTMV alter = new AlterMTMV(info.getMvName(),
info.getRefreshInfo(), MTMVAlterOpType.ALTER_REFRESH_INFO);
this.alter.processAlterMTMV(alter, false);
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/statistics/AnalysisManager.java
b/fe/fe-core/src/main/java/org/apache/doris/statistics/AnalysisManager.java
index 61579a2c0cc..e7ab342d091 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/statistics/AnalysisManager.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/statistics/AnalysisManager.java
@@ -85,6 +85,7 @@ import java.util.Collections;
import java.util.Comparator;
import java.util.HashMap;
import java.util.HashSet;
+import java.util.LinkedHashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
@@ -108,15 +109,12 @@ public class AnalysisManager implements Writable {
private static final Logger LOG =
LogManager.getLogger(AnalysisManager.class);
-/**
- * Mem only.
- */
-public final Queue predicateColumns = new
ArrayBlockingQueue<>(100);
-
-/**
- * Mem only.
- */
-public final Queue queryColumns = new
ArrayBlockingQueue<>(100);
+private static final int COLUMN_QUEUE_SIZE = 1000;
+public final Queue highPriorityColumns = new
ArrayBlockingQueue<>(COLUMN_QUEUE_SIZE);
+public final Queue midPriorityColumns = new
ArrayBlockingQueue<>(COLUMN_QUEUE_SIZE);
+public final Map> highPriorityJobs = new
LinkedHashMap<>();
+public final Map> midPriorityJobs = new
LinkedHashMap<
(doris) 03/03: Support column level health value. (#31794)
This is an automated email from the ASF dual-hosted git repository.
lijibing pushed a commit to branch high-priority-column
in repository https://gitbox.apache.org/repos/asf/doris.git
commit a2c22c63e63f6fc0108d90f338ddb7091434196e
Author: Jibing-Li <64681310+jibing...@users.noreply.github.com>
AuthorDate: Tue Mar 5 16:19:08 2024 +0800
Support column level health value. (#31794)
---
.../apache/doris/analysis/ShowColumnStatsStmt.java | 4 +
.../apache/doris/datasource/InternalCatalog.java | 6 +-
.../java/org/apache/doris/qe/SessionVariable.java | 7 ++
.../org/apache/doris/statistics/AnalysisInfo.java | 17 ++--
.../doris/statistics/AnalysisInfoBuilder.java | 17 ++--
.../org/apache/doris/statistics/AnalysisJob.java | 8 +-
.../apache/doris/statistics/AnalysisManager.java | 13 ++-
.../org/apache/doris/statistics/ColStatsMeta.java | 16 ++--
.../apache/doris/statistics/OlapAnalysisTask.java | 2 +-
.../doris/statistics/StatisticsAutoCollector.java | 95 --
.../doris/statistics/StatisticsJobAppender.java| 28 +--
.../apache/doris/statistics/TableStatsMeta.java| 33
.../doris/statistics/util/StatisticsUtil.java | 10 +++
.../apache/doris/statistics/AnalysisJobTest.java | 4 +-
.../doris/statistics/AnalysisManagerTest.java | 2 +-
15 files changed, 198 insertions(+), 64 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowColumnStatsStmt.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowColumnStatsStmt.java
index 37be76b20df..749bfa7d360 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowColumnStatsStmt.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowColumnStatsStmt.java
@@ -61,6 +61,8 @@ public class ShowColumnStatsStmt extends ShowStmt {
.add("trigger")
.add("query_times")
.add("updated_time")
+.add("update_rows")
+.add("last_analyze_row_count")
.build();
private final TableName tableName;
@@ -160,6 +162,8 @@ public class ShowColumnStatsStmt extends ShowStmt {
row.add(String.valueOf(colStatsMeta == null ? "N/A" :
colStatsMeta.jobType));
row.add(String.valueOf(colStatsMeta == null ? "N/A" :
colStatsMeta.queriedTimes));
row.add(String.valueOf(p.second.updatedTime));
+row.add(String.valueOf(colStatsMeta == null ? "N/A" :
colStatsMeta.updatedRows));
+row.add(String.valueOf(colStatsMeta == null ? "N/A" :
colStatsMeta.rowCount));
result.add(row);
});
return new ShowResultSet(getMetaData(), result);
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
b/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
index 95abc9e06c4..61a1073da34 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
@@ -3116,7 +3116,6 @@ public class InternalCatalog implements
CatalogIf {
rowsToTruncate += partition.getBaseIndex().getRowCount();
}
} else {
-rowsToTruncate = olapTable.getRowCount();
for (Partition partition : olapTable.getPartitions()) {
// If need absolutely correct, should check running txn
here.
// But if the txn is in prepare state, cann't known which
partitions had load data.
@@ -3125,6 +3124,7 @@ public class InternalCatalog implements
CatalogIf {
}
origPartitions.put(partition.getName(), partition.getId());
partitionsDistributionInfo.put(partition.getId(),
partition.getDistributionInfo());
+rowsToTruncate += partition.getBaseIndex().getRowCount();
}
}
// if table currently has no partitions, this sql like empty
command and do nothing, should return directly.
@@ -3285,10 +3285,8 @@ public class InternalCatalog implements
CatalogIf {
if (truncateEntireTable) {
// Drop the whole table stats after truncate the entire table
Env.getCurrentEnv().getAnalysisManager().dropStats(olapTable);
-} else {
-// Update the updated rows in table stats after truncate some
partitions.
-
Env.getCurrentEnv().getAnalysisManager().updateUpdatedRows(updateRecords);
}
+
Env.getCurrentEnv().getAnalysisManager().updateUpdatedRows(updateRecords);
LOG.info("finished to truncate table {}, partitions: {}",
tblRef.getName().toSql(), tblRef.getPartitionNames());
}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java
index 5f7e8ebc32e..a8830f6fc31 100644
(doris) branch high-priority-column updated (65a71e86856 -> a2c22c63e63)
This is an automated email from the ASF dual-hosted git repository. lijibing pushed a change to branch high-priority-column in repository https://gitbox.apache.org/repos/asf/doris.git discard 65a71e86856 Support column level health value. (#31794) omit 1e71f334bdf High priority queue and map. (#31509) omit eafbcb51507 Collect high priority columns. (#31235) add d8b4edbd60a [bugfix](wgcore) map at only get reference and it will core in multithread (#31702) add b71686dfb28 [pipelineX](bug) Fix incorrect join operator judgement (#31690) add d459ac239f0 [chore](ci) prepare jdk17 in cloud_p* and performance (#31685) add 1ef6a1bbc55 [fix](jdbc catalog) Change Druid Pool dependency to version 1.2.11 (#31703) add a970fef2517 [refactor](jdbc catalog) split sqlserver jdbc executor (#31679) add 5048c6ec6a4 [feature](Nereids) parallel output file (#31623) add 3e910206991 [fix](testcases) make all auto partition P2 cases nonConcurrent (#31708) add c90728cd6af fix in stats estimation (#31684) add fb2c4011e4f [fix](scan) Fix missing sync rowsets in cloud mode (#31717) add be418110cb4 [enhancement](nereids)support null partition for list partition (#31613) add 183ff65f2e3 [Feature](topn) support multiple topn filter on backend (#31665) add 237929d0cbc [Bug](resource) fix npe on Resource read from json (#31723) add 28b15b2521e [fix](connections) fix connection hang after too many connections (#31594) add 966d500ed9e [fix](merge-cloud) fix mac compile error (#31732) add 18f98710e14 [fix](fe) Fix NPE in FrontendServiceImpl.loadTxnCommit if table is dropped (#31715) add 1d6e2f96768 [fix](feut) fix TabletRepairAndBalanceTest (#31653) add 7d55e16df5e [fix](partial update) compaction may cause update failue (#31551) add f73ea301534 [Enhencement](env) Checking Master branch must use JDK17 (#31587) add b6696da4ca6 [fix](merge-cloud) fix full compaction core (#31745) add 98671ffcf7e [fix](internal table) fix internal replica num in cloud (#31740) add 08bc414a20d [fix](cloud) Fix regression case `test_nereids_row_policy` run in cloud mode (#31726) add a47d15f5e65 [fix](test) Fix cloud_p0/catalog_recycle_bin_p0/show.groovy failed (#31727) add dae59ad7fa9 [bug](scan) Fix missing sync rowsets in cloud mode (#31756) add da5b6d38664 [feature](insert)add hive table sink definition (#31662) add ea027f3ad31 [Performance](exec) Support runtime filter in <=> join (#31754) add b2887367b2d [feature](function) round function defaults to rounding normally add 0e4191d9f44 [docs](kafka-connector) Add doris-kafka-connector usage documentation (#31729) add 28678ca68ee [Fix](partial update) Fix partial update load false when schema includes auto increment column (#31725) add 12305ce4c24 [fix](broker-load) fix broker load statement type conversion failure (#31746) add 1c60d29206b [opt](doc) update release verify (#31773) add 7fbace480e3 [fix](cast) wrong result while cast const to double then to string (#31657) add 207aba4d0b4 Refactor active queries (#31742) add f71b531ead9 fix](Nereids): support max_value in range partition (#31721) add 4a3e0bdb135 [fix](jdbc catalog) fix close abort in sqlserver (#31718) add 2a77d8b6f38 [feature](doris compose) doris compose use jdk 17 image (#31775) add c1fae97b3e8 [chore](ci) remove required of pipelinex (#31791) add 2bbbdae6a2e [fix](hive) support partition prune for _HIVE_DEFAULT_PARTITION_ (#31736) add 021cbfbe0e4 [chore](Nereids) change consumer rules name (#31777) new 0f97217b72b Collect high priority columns. (#31235) new 2a0b3add07d High priority queue and map. (#31509) new a2c22c63e63 Support column level health value. (#31794) This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (65a71e86856) \ N -- N -- N refs/heads/high-priority-column (a2c22c63e63) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .asf.yaml | 1 - be/src/cloud/cloud_base_compaction.cpp | 4 + be/src/cloud/cloud_cumulative_compaction.cpp | 2 + be/src/cloud/cl
[PR] [fix](partition) add log when tablet partition id eq 0 [doris]
deardeng opened a new pull request, #31796: URL: https://github.com/apache/doris/pull/31796 ## Proposed changes Issue Number: close #xxx 加log，之前认为可以通过grep be中的log，得到异常tablet的个数。后面发现真遇到 grep 还是不直观，增加点log，增加可观测性，减少解决异常时间。 ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](partition) add log when tablet partition id eq 0 [doris]
doris-robot commented on PR #31796: URL: https://github.com/apache/doris/pull/31796#issuecomment-1978212206 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](partition) add log when tablet partition id eq 0 [doris]
deardeng commented on PR #31796: URL: https://github.com/apache/doris/pull/31796#issuecomment-1978212561 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cloud-merge) Fix missing the whereColumn field [doris]
doris-robot commented on PR #31786: URL: https://github.com/apache/doris/pull/31786#issuecomment-1978212748 TPC-DS: Total hot run time: 177276 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit d6018d7e7afe8a755967ae7d0f1595ed38bbdaac, data reload: false query1 927 364 341 341 query2 7377215220402040 query3 6703222 211 211 query4 25626 21026 21106 21026 query5 4360459 515 459 query6 284 186 174 174 query7 4624318 318 318 query8 243 176 170 170 query9 8441219321692169 query10 429 224 234 224 query11 14919 14238 14333 14238 query12 137 90 92 90 query13 1656439 446 439 query14 8782680975316809 query15 222 194 193 193 query16 7258264 264 264 query17 949 587 561 561 query18 1930289 295 289 query19 205 168 163 163 query20 112 97 86 86 query21 200 136 130 130 query22 4639444744244424 query23 31962 30733 31088 30733 query24 11868 314630783078 query25 698 389 392 389 query26 1899163 165 163 query27 2934351 370 351 query28 6706181218031803 query29 1310619 608 608 query30 316 148 150 148 query31 917 744 754 744 query32 108 61 60 60 query33 762 258 258 258 query34 983 467 480 467 query35 922 828 819 819 query36 952 810 827 810 query37 280 70 69 69 query38 3195309731133097 query39 1445135613951356 query40 295 123 119 119 query41 61 56 53 53 query42 111 110 110 110 query43 442 398 395 395 query44 1082728 722 722 query45 205 197 190 190 query46 1066789 784 784 query47 1637157615461546 query48 419 343 359 343 query49 1202366 350 350 query50 800 394 393 393 query51 6695661765496549 query52 107 89 100 89 query53 349 290 284 284 query54 325 250 248 248 query55 86 81 83 81 query56 249 229 232 229 query57 10681003992 992 query58 262 221 221 221 query59 2413228121512151 query60 272 256 273 256 query61 120 114 116 114 query62 650 414 412 412 query63 307 279 283 279 query64 6366327631643164 query65 3044300429692969 query66 1454354 330 330 query67 15119 14334 14297 14297 query68 16052 561 586 561 query69 712 387 375 375 query70 1907109511231095 query71 619 278 289 278 query72 9990261424652465 query73 3817344 330 330 query74 7145682468416824 query75 6563266426742664 query76 7165109411801094 query77 1131256 260 256 query78 10104 965395609560 query79 8401513 517 513 query80 1160403 379 379 query81 473 211 212 211 query82 245 94 90 90 query83 402 145 148 145 query84 284 77 76 76 query85 1251331 325 325 query86 398 292 302 292 query87 3472319932333199 query88 3374229122902290 query89 525 383 355 355 query90 2417176 173 173 query91 166 146 126 126 query92 61 52 54 52 query93 3626539 511 511 query94 1558190 181 181 query95 454 346 355 346 query96 599 275 274 274 query97 3993382539253825 query98 229 214 210 210 query99 1116760 783 760 Total cold run time: 312895 ms Total hot run time: 177276 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th
Re: [PR] [fix](cloud) Fix cloud create table need cluster auth and no cluster … [doris]
yujun777 commented on code in PR #31772:
URL: https://github.com/apache/doris/pull/31772#discussion_r1512345776
##
regression-test/suites/account_p0/test_nereids_authentication.groovy:
##
@@ -26,6 +26,11 @@ suite("test_nereids_authentication", "query") {
PROPERTIES ('replication_num' = '1')
"""
}
+def isCloudMode = {
Review Comment:
use Suite.isCloudCluster() ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](update) Update set value should consider sequence column [doris]
hust-hhb commented on PR #31626: URL: https://github.com/apache/doris/pull/31626#issuecomment-1978218150 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cloud) Fix cloud create table need cluster auth and no cluster … [doris]
yujun777 commented on code in PR #31772:
URL: https://github.com/apache/doris/pull/31772#discussion_r1512348306
##
regression-test/suites/account_p0/test_nereids_authentication.groovy:
##
@@ -26,6 +26,11 @@ suite("test_nereids_authentication", "query") {
PROPERTIES ('replication_num' = '1')
"""
}
+def isCloudMode = {
Review Comment:
maybe change Suite.isCloudCluster() to Suite.isCloudMode() is better
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Bug](load) fix wrong data in mv when routine load with function mapping [doris]
doris-robot commented on PR #31787: URL: https://github.com/apache/doris/pull/31787#issuecomment-1978220726 TPC-DS: Total hot run time: 178059 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 17df853e10256a4add62cbed770eef2b0162749f, data reload: false query1 919 358 339 339 query2 7379207421212074 query3 6719218 206 206 query4 25237 20950 20973 20950 query5 4387467 489 467 query6 277 181 182 181 query7 4625312 298 298 query8 251 166 173 166 query9 8450219521982195 query10 447 235 236 235 query11 15149 14252 14338 14252 query12 142 89 85 85 query13 1683440 421 421 query14 9074694269406940 query15 247 190 193 190 query16 7306268 274 268 query17 966 602 585 585 query18 1939295 284 284 query19 202 166 161 161 query20 101 92 90 90 query21 208 134 131 131 query22 4669450744314431 query23 31285 30667 30836 30667 query24 12361 307731233077 query25 703 402 393 393 query26 1924165 163 163 query27 3015357 386 357 query28 6607183717991799 query29 1302634 606 606 query30 311 147 154 147 query31 915 750 726 726 query32 107 62 57 57 query33 765 268 265 265 query34 975 461 480 461 query35 946 793 820 793 query36 916 843 817 817 query37 289 72 69 69 query38 3193311731353117 query39 1418135614661356 query40 301 121 123 121 query41 60 55 56 55 query42 108 105 111 105 query43 458 410 386 386 query44 1081722 715 715 query45 206 196 193 193 query46 1043773 777 773 query47 1642157615711571 query48 439 345 344 344 query49 1213363 348 348 query50 781 381 381 381 query51 6728659766776597 query52 109 91 98 91 query53 354 280 284 280 query54 342 251 265 251 query55 86 90 91 90 query56 256 246 267 246 query57 1090997 1000997 query58 250 228 230 228 query59 2455233622522252 query60 279 311 252 252 query61 123 118 117 117 query62 656 406 431 406 query63 303 279 288 279 query64 6466326635233266 query65 3014301430173014 query66 1474352 325 325 query67 15810 14656 14481 14481 query68 13731 561 568 561 query69 729 389 388 388 query70 1378112910981098 query71 643 279 272 272 query72 10048 261425072507 query73 3403337 335 335 query74 7084693270166932 query75 6676268926812681 query76 7918108611921086 query77 1104268 250 250 query78 10282 978495299529 query79 10080 520 515 515 query80 826 420 378 378 query81 480 212 209 209 query82 282 90 90 90 query83 277 145 141 141 query84 280 74 77 74 query85 1132346 341 341 query86 375 280 296 280 query87 3390322732343227 query88 3046226922622262 query89 525 395 377 377 query90 2334188 184 184 query91 162 132 129 129 query92 64 50 49 49 query93 3059539 520 520 query94 1525200 187 187 query95 456 337 351 337 query96 602 272 259 259 query97 4016392239203920 query98 234 216 216 216 query99 1035770 742 742 Total cold run time: 311712 ms Total hot run time: 178059 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th
Re: [PR] [fix](cloud) Fix cloud create table need cluster auth and no cluster … [doris]
deardeng commented on code in PR #31772:
URL: https://github.com/apache/doris/pull/31772#discussion_r1512351681
##
regression-test/suites/account_p0/test_nereids_authentication.groovy:
##
@@ -26,6 +26,11 @@ suite("test_nereids_authentication", "query") {
PROPERTIES ('replication_num' = '1')
"""
}
+def isCloudMode = {
Review Comment:
Suite.isCloudCluster() 是判断配置文件是否配置了cloud_unique_id，依赖配置文件。
我感觉我自己def这个方法，更好点可以自适应。show backends，然后看tag里面是否包含cloud的信息。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cloud-merge) Fix missing the whereColumn field [doris]
doris-robot commented on PR #31786: URL: https://github.com/apache/doris/pull/31786#issuecomment-1978221536 ClickBench: Total hot run time: 31.15 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit d6018d7e7afe8a755967ae7d0f1595ed38bbdaac, data reload: false query1 0.040.030.03 query2 0.060.020.03 query3 0.230.050.05 query4 1.670.100.10 query5 0.520.490.50 query6 1.240.670.69 query7 0.020.010.02 query8 0.030.020.03 query9 0.550.530.51 query10 0.580.560.58 query11 0.130.100.09 query12 0.130.110.10 query13 0.570.570.56 query14 0.740.750.74 query15 0.820.810.80 query16 0.390.380.38 query17 1.020.971.01 query18 0.270.270.25 query19 1.841.761.73 query20 0.020.010.01 query21 15.43 0.630.67 query22 3.463.732.83 query23 17.39 0.990.87 query24 2.130.370.58 query25 0.250.080.03 query26 0.160.140.13 query27 0.030.030.04 query28 12.18 0.860.83 query29 12.66 3.463.26 query30 0.640.540.58 query31 2.780.340.35 query32 3.380.440.44 query33 2.902.882.98 query34 15.49 4.314.32 query35 4.324.274.32 query36 1.101.011.02 query37 0.070.060.05 query38 0.040.020.03 query39 0.020.020.02 query40 0.170.140.15 query41 0.080.020.02 query42 0.030.020.02 query43 0.030.020.02 Total cold run time: 105.61 s Total hot run time: 31.15 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](update) Update set value should consider sequence column [doris]
github-actions[bot] commented on PR #31626: URL: https://github.com/apache/doris/pull/31626#issuecomment-1978222803 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhance](multi-catalog) Use MetaIdMappingsLog to replace InitCatalogLog/InitDatabaseLog. [doris]
dutyu closed pull request #31120: [Enhance](multi-catalog) Use MetaIdMappingsLog to replace InitCatalogLog/InitDatabaseLog. URL: https://github.com/apache/doris/pull/31120 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cloud-merge) Fix missing the whereColumn field [doris]
doris-robot commented on PR #31786: URL: https://github.com/apache/doris/pull/31786#issuecomment-1978226945 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit d6018d7e7afe8a755967ae7d0f1595ed38bbdaac with default session variables Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s Insert into select: 17.7 seconds inserted 1000 Rows, about 564K ops/s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](partial-update) Fix wrong column number passing to be when partial and enable nereids [doris]
doris-robot commented on PR #31461: URL: https://github.com/apache/doris/pull/31461#issuecomment-1978227880 TPC-DS: Total hot run time: 177893 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit da83631c761db9122e1ea456d4b15bcea5570757, data reload: false query1 934 363 332 332 query2 7465216520512051 query3 6710217 212 212 query4 27982 21114 21033 21033 query5 42915608505 505 query6 271 183 185 183 query7 4628305 292 292 query8 241 169 172 169 query9 8441218121932181 query10 437 237 234 234 query11 14834 14327 14530 14327 query12 128 85 89 85 query13 1647413 425 413 query14 8978708473867084 query15 237 196 194 194 query16 7246285 282 282 query17 956 598 574 574 query18 1919307 285 285 query19 214 175 175 175 query20 96 89 92 89 query21 202 135 127 127 query22 4560447844674467 query23 31353 30638 30858 30638 query24 11757 303130263026 query25 699 399 370 370 query26 1916157 174 157 query27 2937348 355 348 query28 6679181717921792 query29 1295625 592 592 query30 305 149 147 147 query31 947 745 750 745 query32 103 64 59 59 query33 747 250 246 246 query34 993 459 462 459 query35 940 831 796 796 query36 925 814 822 814 query37 274 69 68 68 query38 3222312830683068 query39 1404136313931363 query40 297 120 121 120 query41 56 52 52 52 query42 111 101 101 101 query43 453 402 418 402 query44 1080691 699 691 query45 203 198 187 187 query46 1044790 754 754 query47 1634155715471547 query48 401 346 350 346 query49 1191347 329 329 query50 767 379 370 370 query51 6670652465256524 query52 103 94 101 94 query53 354 280 279 279 query54 308 237 232 232 query55 92 83 80 80 query56 236 231 219 219 query57 1102997 986 986 query58 242 207 219 207 query59 2563248024262426 query60 299 256 265 256 query61 119 116 121 116 query62 664 406 418 406 query63 313 281 284 281 query64 6307322332723223 query65 3038300429882988 query66 1456354 337 337 query67 15151 14694 14396 14396 query68 10519 571 592 571 query69 702 389 389 389 query70 1388111011061106 query71 623 285 298 285 query72 9634274926792679 query73 1616326 323 323 query74 7317684368186818 query75 6232267926122612 query76 7834108511861085 query77 945 264 249 249 query78 10440 963995519551 query79 10660 514 512 512 query80 884 423 384 384 query81 471 210 203 203 query82 228 92 92 92 query83 194 147 145 145 query84 288 84 79 79 query85 1124348 327 327 query86 390 290 316 290 query87 3429321631503150 query88 3876229522812281 query89 535 363 374 363 query90 2138171 175 171 query91 157 128 131 128 query92 62 52 50 50 query93 6051540 515 515 query94 1392191 186 186 query95 459 346 362 346 query96 594 259 258 258 query97 4035382539473825 query98 237 215 203 203 query99 1067786 788 786 Total cold run time: 310688 ms Total hot run time: 177893 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th
Re: [PR] [Nereids] add leading tpc-h [doris]
LiBinfeng-01 commented on PR #30405: URL: https://github.com/apache/doris/pull/30405#issuecomment-1978228359 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](group commit) handle group commit create plan error [doris]
dataroaring commented on PR #31757: URL: https://github.com/apache/doris/pull/31757#issuecomment-1978228541 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](group commit) handle group commit create plan error [doris]
github-actions[bot] commented on PR #31757: URL: https://github.com/apache/doris/pull/31757#issuecomment-1978229797 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Bug](load) fix wrong data in mv when routine load with function mapping [doris]
doris-robot commented on PR #31787: URL: https://github.com/apache/doris/pull/31787#issuecomment-1978230023 ClickBench: Total hot run time: 30.85 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 17df853e10256a4add62cbed770eef2b0162749f, data reload: false query1 0.050.030.04 query2 0.060.030.03 query3 0.230.060.07 query4 1.640.100.10 query5 0.500.480.50 query6 1.270.660.65 query7 0.020.020.02 query8 0.040.030.03 query9 0.570.530.52 query10 0.580.560.56 query11 0.140.100.10 query12 0.120.100.11 query13 0.570.570.57 query14 0.730.760.74 query15 0.810.820.81 query16 0.370.370.37 query17 0.991.010.95 query18 0.270.260.26 query19 1.821.741.68 query20 0.020.010.01 query21 15.43 0.660.62 query22 3.533.712.01 query23 17.30 1.131.05 query24 2.290.610.58 query25 0.240.030.05 query26 0.230.140.14 query27 0.040.030.03 query28 11.75 0.840.83 query29 12.70 3.443.45 query30 0.590.560.53 query31 2.800.340.35 query32 3.350.430.44 query33 2.932.932.88 query34 15.50 4.304.28 query35 4.294.304.31 query36 1.091.011.02 query37 0.070.050.05 query38 0.040.030.03 query39 0.020.020.02 query40 0.180.130.13 query41 0.080.020.02 query42 0.020.010.02 query43 0.030.020.03 Total cold run time: 105.3 s Total hot run time: 30.85 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) json_object can't be odd parameters, need even parameters [doris]
iwanttobepowerful commented on PR #31767: URL: https://github.com/apache/doris/pull/31767#issuecomment-1978232292 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](jdbc catalog) Change Druid Pool dependency to version 1.2.5 [doris]
zy-kkk merged PR #31783: URL: https://github.com/apache/doris/pull/31783 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (021cbfbe0e4 -> a1fa1daeeaa)
This is an automated email from the ASF dual-hosted git repository. zykkk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 021cbfbe0e4 [chore](Nereids) change consumer rules name (#31777) add a1fa1daeeaa [fix](jdbc catalog) Change Druid Pool dependency to version 1.2.5 (#31783) No new revisions were added by this update. Summary of changes: fe/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Bug](load) fix wrong data in mv when routine load with function mapping [doris]
doris-robot commented on PR #31787: URL: https://github.com/apache/doris/pull/31787#issuecomment-1978235213 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit 17df853e10256a4add62cbed770eef2b0162749f with default session variables Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s Insert into select: 19.0 seconds inserted 1000 Rows, about 526K ops/s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhance](multi-catalog) Use MetaIdMappingsLog to replace InitCatalogLog/InitDatabaseLog. [doris]
doris-robot commented on PR #31120: URL: https://github.com/apache/doris/pull/31120#issuecomment-1978236303 TPC-H: Total hot run time: 38294 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit c6886f27be53688272d515ab5c5b256ba6db857b, data reload: false -- Round 1 -- q1 17573 407940444044 q2 2026150 145 145 q3 10561 963 960 960 q4 5470972 967 967 q5 7590290930892909 q6 179 125 125 125 q7 1310856 808 808 q8 9501213621102110 q9 7347649565296495 q10 8220254725652547 q11 422 225 218 218 q12 727 317 314 314 q13 17954 299429342934 q14 272 249 252 249 q15 497 459 436 436 q16 494 388 404 388 q17 983 881 857 857 q18 6899602759775977 q19 1568153515291529 q20 557 288 272 272 q21 7529374737183718 q22 778 304 292 292 Total cold run time: 108457 ms Total hot run time: 38294 ms - Round 2, with runtime_filter_mode=off - q1 4039396840583968 q2 320 219 234 219 q3 2961293929792939 q4 1895183918341834 q5 5297528352755275 q6 219 116 119 116 q7 2309182218471822 q8 3239329233033292 q9 8581859686388596 q10 6186378137413741 q11 531 443 450 443 q12 713 553 542 542 q13 7280281928172817 q14 277 257 256 256 q15 472 448 431 431 q16 458 421 421 421 q17 1709165416771654 q18 8008746473067306 q19 1683162716321627 q20 1935170117331701 q21 5003491448734873 q22 569 478 467 467 Total cold run time: 63684 ms Total hot run time: 54340 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](partial-update) Fix wrong column number passing to be when partial and enable nereids [doris]
doris-robot commented on PR #31461: URL: https://github.com/apache/doris/pull/31461#issuecomment-1978236634 ClickBench: Total hot run time: 30.82 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit da83631c761db9122e1ea456d4b15bcea5570757, data reload: false query1 0.040.040.03 query2 0.060.030.03 query3 0.240.060.06 query4 1.640.100.10 query5 0.500.530.52 query6 1.250.690.68 query7 0.020.010.02 query8 0.040.030.03 query9 0.580.510.52 query10 0.560.580.55 query11 0.140.100.10 query12 0.120.100.10 query13 0.580.570.57 query14 0.740.740.74 query15 0.830.810.80 query16 0.360.370.38 query17 1.001.000.98 query18 0.270.260.26 query19 1.851.741.72 query20 0.020.010.01 query21 15.41 0.610.61 query22 2.763.812.53 query23 17.48 1.111.04 query24 2.310.280.19 query25 0.200.070.05 query26 0.170.140.13 query27 0.040.030.03 query28 12.47 0.850.85 query29 12.52 3.423.18 query30 0.620.560.54 query31 2.790.350.35 query32 3.340.440.43 query33 2.912.882.89 query34 15.48 4.294.28 query35 4.294.274.30 query36 1.101.011.02 query37 0.070.060.06 query38 0.040.020.02 query39 0.030.020.02 query40 0.170.130.13 query41 0.080.020.02 query42 0.030.010.02 query43 0.030.020.02 Total cold run time: 105.18 s Total hot run time: 30.82 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-streamloader) branch master updated: header parsing compatible with multiple ':' (#14)
This is an automated email from the ASF dual-hosted git repository.
liaoxin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-streamloader.git
The following commit(s) were added to refs/heads/master by this push:
new 3b2e1ce header parsing compatible with multiple ':' (#14)
3b2e1ce is described below
commit 3b2e1ce0dd5e3f7083993dbe1bdba4b90e3c6da4
Author: ZSiHM <38096673+zs...@users.noreply.github.com>
AuthorDate: Tue Mar 5 16:46:54 2024 +0800
header parsing compatible with multiple ':' (#14)
header parse wrong when contains multiple ':',like
header="columns:c1,c2,deviceip=c1.endtime=c2,p_hour=date_format(c2,
'%Y-%m-%d %H:%i:%s')"
---
main.go | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/main.go b/main.go
index 64d7de3..779148e 100644
--- a/main.go
+++ b/main.go
@@ -245,7 +245,11 @@ func paramCheck() {
continue
}
kv := strings.Split(v, ":")
- headers[kv[0]] = kv[1]
+ if len(kv) > 2 {
+ headers[kv[0]] = strings.Join(kv[1:], ":")
+ } else {
+ headers[kv[0]] = kv[1]
+ }
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] header parsing compatible with multiple ':' [doris-streamloader]
liaoxin01 merged PR #14: URL: https://github.com/apache/doris-streamloader/pull/14 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](README) rename StreamLoader to Streamloader [doris-streamloader]
liaoxin01 merged PR #13: URL: https://github.com/apache/doris-streamloader/pull/13 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [nereids](topn-filter) support multi-topn filter (FE part) [doris]
doris-robot commented on PR #31485: URL: https://github.com/apache/doris/pull/31485#issuecomment-1978237643 TPC-H: Total hot run time: 38072 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 9c8222a6c686f27f9f32d3efaf739a0b832f548f, data reload: false -- Round 1 -- q1 17806 407140404040 q2 2022154 148 148 q3 10559 940 937 937 q4 4987961 936 936 q5 7620285830392858 q6 183 131 128 128 q7 1307845 820 820 q8 9494211020982098 q9 7338645964876459 q10 8246253926142539 q11 435 227 223 223 q12 797 315 307 307 q13 17964 294329442943 q14 283 255 268 255 q15 498 445 443 443 q16 478 395 404 395 q17 952 853 849 849 q18 6896606858585858 q19 1587155215401540 q20 558 296 286 286 q21 7560374737053705 q22 790 305 311 305 Total cold run time: 108360 ms Total hot run time: 38072 ms - Round 2, with runtime_filter_mode=off - q1 4038400739973997 q2 317 231 225 225 q3 2971292829062906 q4 1877181518181815 q5 5222524652485246 q6 204 122 125 122 q7 2309187218311831 q8 3214328433123284 q9 8580859885408540 q10 6189375136753675 q11 531 458 449 449 q12 719 557 555 555 q13 13313 287827682768 q14 277 265 263 263 q15 476 444 439 439 q16 460 411 446 411 q17 1707166216611661 q18 7756746572577257 q19 1639161816401618 q20 1909176117281728 q21 5045487347644764 q22 553 475 411 411 Total cold run time: 69306 ms Total hot run time: 53965 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-streamloader) branch master updated: [fix](README) rename StreamLoader to Streamloader (#13)
This is an automated email from the ASF dual-hosted git repository. liaoxin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-streamloader.git The following commit(s) were added to refs/heads/master by this push: new 6e59062 [fix](README) rename StreamLoader to Streamloader (#13) 6e59062 is described below commit 6e59062a9043219c5f887e39aa6160f0ce92335c Author: zhengyu AuthorDate: Tue Mar 5 16:47:25 2024 +0800 [fix](README) rename StreamLoader to Streamloader (#13) --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 15a2a96..86b2139 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,7 @@ specific language governing permissions and limitations under the License. --> -# Apache Doris StreamLoader +# Apache Doris Streamloader A robust, high-performance and user-friendly alternative to the traditional curl-based Stream Load. @@ -65,7 +65,7 @@ For additional details and options, refer to our comprehensive docs below. ## Build -To build StreamLoader, ensure you have golang installed (version >= 1.19.9). For example, on CentOS: +To build Streamloader, ensure you have golang installed (version >= 1.19.9). For example, on CentOS: ``` yum install golang - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) let OrToIn rewritten result have stable order [doris]
morrySnow commented on PR #31797: URL: https://github.com/apache/doris/pull/31797#issuecomment-1978238791 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) let OrToIn rewritten result have stable order [doris]
doris-robot commented on PR #31797: URL: https://github.com/apache/doris/pull/31797#issuecomment-1978238717 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](Nereids) let OrToIn rewritten result have stable order [doris]
morrySnow opened a new pull request, #31797: URL: https://github.com/apache/doris/pull/31797 pick from master #31731 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](partial-update) Fix wrong column number passing to be when partial and enable nereids [doris]
doris-robot commented on PR #31461: URL: https://github.com/apache/doris/pull/31461#issuecomment-1978242026 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit da83631c761db9122e1ea456d4b15bcea5570757 with default session variables Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s Insert into select: 18.4 seconds inserted 1000 Rows, about 543K ops/s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
