Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]

2023-12-23 Thread via GitHub


dutyu commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1868235259

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [opt](information_schema) support information_schema in external catalog [doris]

2023-12-23 Thread via GitHub


morningman opened a new pull request, #28919:
URL: https://github.com/apache/doris/pull/28919

   ## Proposed changes
   
   Draft
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refactor](create tablet) default create tablet round robin [doris]

2023-12-23 Thread via GitHub


yujun777 commented on PR #28911:
URL: https://github.com/apache/doris/pull/28911#issuecomment-1868238982

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1868241922

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit 2086add8ec732822b5c32ac2344748500a36f784, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4685442844194419
   q2   373 146 158 146
   q3   1460126612371237
   q4   886 903 886
   q5   3168313331773133
   q6   247 129 129 129
   q7   986 486 492 486
   q8   2171222521852185
   q9   6677666766676667
   q10  3226325432663254
   q11  308 186 192 186
   q12  348 212 206 206
   q13  4535380937963796
   q14  244 210 211 210
   q15  561 521 521 521
   q16  436 384 381 381
   q17  1002620 526 526
   q18  7035675868506758
   q19  1532143914071407
   q20  524 319 314 314
   q21  3075264226312631
   q22  348 280 283 280
   Total cold run time: 44052 ms
   Total hot run time: 39758 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4332439243284328
   q2   269 165 170 165
   q3   3502349934963496
   q4   2393235723682357
   q5   5694573757115711
   q6   239 122 123 122
   q7   2356187218771872
   q8   3518351535073507
   q9   9034899089778977
   q10  3915400339973997
   q11  488 373 368 368
   q12  761 579 599 579
   q13  4266358135763576
   q14  288 250 255 250
   q15  581 520 519 519
   q16  503 466 447 447
   q17  1888186818491849
   q18  8499813581508135
   q19  1738175917561756
   q20  2269193719421937
   q21  6490614461316131
   q22  499 414 407 407
   Total cold run time: 63522 ms
   Total hot run time: 60486 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refactor](create tablet) default create tablet round robin [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28911:
URL: https://github.com/apache/doris/pull/28911#issuecomment-1868243010

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit 8719a14529dff4ce5d3e27e197da3634e1917613, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4754441844294418
   q2   368 147 158 147
   q3   1469133512301230
   q4   1118892 879 879
   q5   3182315231823152
   q6   250 131 133 131
   q7   990 489 488 488
   q8   2158222022012201
   q9   6657666166696661
   q10  3232325132833251
   q11  310 186 182 182
   q12  349 211 202 202
   q13  4570382237813781
   q14  244 212 217 212
   q15  577 528 528 528
   q16  451 382 385 382
   q17  999 590 524 524
   q18  7000682967416741
   q19  1516143514101410
   q20  504 295 313 295
   q21  3132264426672644
   q22  350 279 282 279
   Total cold run time: 44180 ms
   Total hot run time: 39738 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4339432843304328
   q2   269 168 168 168
   q3   3480349534743474
   q4   2401236423552355
   q5   5705570057135700
   q6   242 126 125 125
   q7   2377187218401840
   q8   3516352935233523
   q9   8977899089688968
   q10  3937400039933993
   q11  488 377 360 360
   q12  759 587 594 587
   q13  4318356935633563
   q14  285 255 250 250
   q15  575 520 515 515
   q16  502 442 477 442
   q17  1880186618421842
   q18  8509801482158014
   q19  1758173717491737
   q20  2251195419401940
   q21  6517614661416141
   q22  507 427 427 427
   Total cold run time: 63592 ms
   Total hot run time: 60292 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Improvement](execute) optimize for ColumnNullable's serialize_vec/deserialize_vec [doris]

2023-12-23 Thread via GitHub


BiteThet commented on PR #28788:
URL: https://github.com/apache/doris/pull/28788#issuecomment-1868243699

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Refactor](dialect) Add sql dialect converter plugins [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28890:
URL: https://github.com/apache/doris/pull/28890#issuecomment-1868244056

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.45 seconds
stream load tsv:  564 seconds loaded 74807831229 Bytes, about 126 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  29.0 seconds inserted 1000 Rows, about 
344K ops/s
storage size: 17183825534 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refactor](create tablet) default create tablet round robin [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28911:
URL: https://github.com/apache/doris/pull/28911#issuecomment-1868244574

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.62 seconds
stream load tsv:  563 seconds loaded 74807831229 Bytes, about 126 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.6 seconds inserted 1000 Rows, about 
349K ops/s
storage size: 17183790601 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Improvement](execute) optimize for ColumnNullable's serialize_vec/deserialize_vec [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28788:
URL: https://github.com/apache/doris/pull/28788#issuecomment-1868247543

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit 0487a9b89fbd3ac311fa22b84111c3b435375abc, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4703441244834412
   q2   371 144 159 144
   q3   1448126712171217
   q4   1105900 844 844
   q5   3157315631473147
   q6   249 132 127 127
   q7   979 491 475 475
   q8   2169221521692169
   q9   6691670466476647
   q10  3227327432723272
   q11  304 183 186 183
   q12  346 212 203 203
   q13  4571379037623762
   q14  252 212 209 209
   q15  565 529 523 523
   q16  440 390 386 386
   q17  1030633 548 548
   q18  7089689268136813
   q19  1517137214811372
   q20  550 301 288 288
   q21  3085264526352635
   q22  348 282 282 282
   Total cold run time: 44196 ms
   Total hot run time: 39658 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4353437043684368
   q2   267 163 176 163
   q3   3502349434863486
   q4   2387236723652365
   q5   5719571557135713
   q6   238 124 123 123
   q7   2357184718871847
   q8   3510352335233523
   q9   9032901789518951
   q10  3905400340134003
   q11  481 360 370 360
   q12  763 591 608 591
   q13  4310358635763576
   q14  287 264 248 248
   q15  575 519 522 519
   q16  500 451 477 451
   q17  1895187118711871
   q18  8441813881128112
   q19  1725178217121712
   q20  2252192819341928
   q21  6473612961576129
   q22  497 448 411 411
   Total cold run time: 63469 ms
   Total hot run time: 60450 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refactor](create tablet) default create tablet round robin [doris]

2023-12-23 Thread via GitHub


yujun777 commented on PR #28911:
URL: https://github.com/apache/doris/pull/28911#issuecomment-1868247799

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhancement](bulk-load) cancel loading tasks directly without retrying when timeout exceeded [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28666:
URL: https://github.com/apache/doris/pull/28666#issuecomment-1868249411

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](parquet) the end offset of column chunk may be wrong in parquet metadata [doris]

2023-12-23 Thread via GitHub


AshinGau commented on PR #28891:
URL: https://github.com/apache/doris/pull/28891#issuecomment-1868249505

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](parquet) the end offset of column chunk may be wrong in parquet metadata [doris]

2023-12-23 Thread via GitHub


AshinGau commented on PR #28893:
URL: https://github.com/apache/doris/pull/28893#issuecomment-1868249791

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Improvement](execute) optimize for ColumnNullable's serialize_vec/deserialize_vec [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28788:
URL: https://github.com/apache/doris/pull/28788#issuecomment-1868250024

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.41 seconds
stream load tsv:  568 seconds loaded 74807831229 Bytes, about 125 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.6 seconds inserted 1000 Rows, about 
349K ops/s
storage size: 17183821545 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](parquet) the end offset of column chunk may be wrong in parquet metadata [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28891:
URL: https://github.com/apache/doris/pull/28891#issuecomment-1868250588

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](parquet) the end offset of column chunk may be wrong in parquet metadata [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28893:
URL: https://github.com/apache/doris/pull/28893#issuecomment-1868250746

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [branch-2.0-var](disk balance) Impr disk rebalancer sched #26412 [doris]

2023-12-23 Thread via GitHub


yujun777 commented on PR #28920:
URL: https://github.com/apache/doris/pull/28920#issuecomment-1868250789

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [branch-2.0-var](disk balance) Impr disk rebalancer sched #26412 [doris]

2023-12-23 Thread via GitHub


yujun777 opened a new pull request, #28920:
URL: https://github.com/apache/doris/pull/28920

   pick: #26412
   
   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](chore) update DCHECK to avoid core during stress test [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28895:
URL: https://github.com/apache/doris/pull/28895#issuecomment-1868251025

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Enhancement](page cache) insert into setting to disable page cache [doris]

2023-12-23 Thread via GitHub


yiguolei commented on code in PR #28913:
URL: https://github.com/apache/doris/pull/28913#discussion_r1435546767


##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/InsertIntoTableCommand.java:
##
@@ -102,6 +102,8 @@ public void setJobId(long jobId) {
 
 @Override
 public void run(ConnectContext ctx, StmtExecutor executor) throws 
Exception {
+// insert into setting to disable page cache
+ctx.getSessionVariable().enablePageCache = false;

Review Comment:
   If you modify the code like this. when the insert into select command 
finished. and user want to execute select command, the page cache value is 
false too. The performance maybe bad.
   I think you could refer the code insert into xxx select /* SET_VAR=xxx*/
   to know how to set the session variable at sql level not at session level.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [branch-2.0-var](disk balance) Impr disk rebalancer sched #26412 [doris]

2023-12-23 Thread via GitHub


yujun777 commented on PR #28920:
URL: https://github.com/apache/doris/pull/28920#issuecomment-1868253039

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](parquet) the end offset of column chunk may be wrong in parquet metadata [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28891:
URL: https://github.com/apache/doris/pull/28891#issuecomment-1868253030

   TeamCity be ut coverage result:
Function Coverage: 36.59% (8550/23364) 
Line Coverage: 28.66% (69505/242492)
Region Coverage: 27.67% (35958/129943)
Branch Coverage: 24.40% (18380/75324)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/7037d3db0a423f5f471d2503c83b5e6343b03183_7037d3db0a423f5f471d2503c83b5e6343b03183/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](parquet) the end offset of column chunk may be wrong in parquet metadata [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28891:
URL: https://github.com/apache/doris/pull/28891#issuecomment-1868253642

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit 7037d3db0a423f5f471d2503c83b5e6343b03183, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4703437244434372
   q2   369 147 159 147
   q3   1460125312491249
   q4   1107889 908 889
   q5   3173315631783156
   q6   247 136 135 135
   q7   981 487 487 487
   q8   2176224621912191
   q9   6679665066776650
   q10  3196327132833271
   q11  305 189 178 178
   q12  351 211 209 209
   q13  4572379438133794
   q14  245 211 221 211
   q15  575 524 524 524
   q16  441 379 392 379
   q17  1014674 602 602
   q18  7144676967796769
   q19  1540143814221422
   q20  543 311 289 289
   q21  3084265726892657
   q22  354 285 289 285
   Total cold run time: 44259 ms
   Total hot run time: 39866 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4338432043444320
   q2   274 167 169 167
   q3   3514350934893489
   q4   2397237823722372
   q5   5707570657085706
   q6   250 121 125 121
   q7   2369187118611861
   q8   3523352235403522
   q9   9011902089888988
   q10  3927401540214015
   q11  487 363 364 363
   q12  760 616 601 601
   q13  4303359835313531
   q14  293 261 253 253
   q15  570 535 521 521
   q16  514 462 507 462
   q17  1898185018731850
   q18  8581800179797979
   q19  1780177017791770
   q20  2271195019341934
   q21  6481616561296129
   q22  489 431 429 429
   Total cold run time: 63737 ms
   Total hot run time: 60383 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refactor](create tablet) default create tablet round robin [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28911:
URL: https://github.com/apache/doris/pull/28911#issuecomment-1868254861

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit e531f11f36be4fc70cacbca4f957597ffa7ecdcc, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4714442844784428
   q2   369 157 159 157
   q3   1464127712201220
   q4   1103900 900 900
   q5   3222317331943173
   q6   250 134 133 133
   q7   994 485 490 485
   q8   2218220421832183
   q9   6689674566956695
   q10  3190325133033251
   q11  306 194 193 193
   q12  355 213 215 213
   q13  4599379238253792
   q14  243 211 218 211
   q15  565 525 524 524
   q16  440 391 395 391
   q17  1022677 590 590
   q18  7080677768466777
   q19  1519143714291429
   q20  551 327 292 292
   q21  3086265326572653
   q22  350 277 284 277
   Total cold run time: 44329 ms
   Total hot run time: 39967 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4363432843324328
   q2   274 170 175 170
   q3   3495348834923488
   q4   2405236323722363
   q5   5739571657105710
   q6   241 123 122 122
   q7   2371182918781829
   q8   3538352135353521
   q9   9036901389418941
   q10  3900399840173998
   q11  480 356 370 356
   q12  771 601 629 601
   q13  4289360735573557
   q14  283 253 256 253
   q15  581 523 526 523
   q16  499 472 474 472
   q17  1877186618561856
   q18  8606813981688139
   q19  1746175517921755
   q20  2285195819471947
   q21  6502612561236123
   q22  499 433 417 417
   Total cold run time: 63780 ms
   Total hot run time: 60469 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](parquet) the end offset of column chunk may be wrong in parquet metadata [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28893:
URL: https://github.com/apache/doris/pull/28893#issuecomment-1868255585

   TeamCity be ut coverage result:
Function Coverage: 37.74% (7990/21170) 
Line Coverage: 29.45% (64914/220412)
Region Coverage: 28.92% (33391/115464)
Branch Coverage: 24.79% (17131/69094)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/00c92dc619375f720158682affae764c91c269c9_00c92dc619375f720158682affae764c91c269c9/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](parquet) the end offset of column chunk may be wrong in parquet metadata [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28891:
URL: https://github.com/apache/doris/pull/28891#issuecomment-1868256021

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.25 seconds
stream load tsv:  566 seconds loaded 74807831229 Bytes, about 126 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  33 seconds loaded 861443392 Bytes, about 24 
MB/s
insert into select:  28.7 seconds inserted 1000 Rows, about 
348K ops/s
storage size: 17183597790 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refactor](create tablet) default create tablet round robin [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28911:
URL: https://github.com/apache/doris/pull/28911#issuecomment-1868256348

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.6 seconds
stream load tsv:  564 seconds loaded 74807831229 Bytes, about 126 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.6 seconds inserted 1000 Rows, about 
349K ops/s
storage size: 17183704084 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](parquet) the end offset of column chunk may be wrong in parquet metadata [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28893:
URL: https://github.com/apache/doris/pull/28893#issuecomment-1868257923

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 51.3 seconds
stream load tsv:  574 seconds loaded 74807831229 Bytes, about 124 
MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  30.5 seconds inserted 1000 Rows, about 
327K ops/s
storage size: 17167903773 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Improvement](execute) optimize for ColumnNullable's serialize_vec/deserialize_vec [doris]

2023-12-23 Thread via GitHub


BiteThet commented on PR #28788:
URL: https://github.com/apache/doris/pull/28788#issuecomment-1868260993

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement] add a lower bound for bytes in scanner queue [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28912:
URL: https://github.com/apache/doris/pull/28912#issuecomment-1868266539

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Improvement](execute) optimize for ColumnNullable's serialize_vec/deserialize_vec [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28788:
URL: https://github.com/apache/doris/pull/28788#issuecomment-1868267018

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit 2a83c1f25a87f09e5a9cbedca5d51875c9baa2de, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4745443344584433
   q2   373 144 157 144
   q3   1463124412271227
   q4   908 919 908
   q5   3160317731783177
   q6   249 136 133 133
   q7   1011495 492 492
   q8   2184221622022202
   q9   6690668966636663
   q10  3243326832883268
   q11  309 179 183 179
   q12  352 216 212 212
   q13  4565382638263826
   q14  242 215 216 215
   q15  573 521 533 521
   q16  438 389 383 383
   q17  1028693 533 533
   q18  7013674468736744
   q19  1523140014261400
   q20  509 292 309 292
   q21  3064265626932656
   q22  355 280 290 280
   Total cold run time: 44200 ms
   Total hot run time: 39888 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4391433743444337
   q2   272 164 180 164
   q3   3518349934963496
   q4   2402237723782377
   q5   5714570457075704
   q6   240 121 123 121
   q7   2366185518931855
   q8   3532352835193519
   q9   9080902490149014
   q10  3904401940274019
   q11  483 371 367 367
   q12  771 592 611 592
   q13  4275357535513551
   q14  296 251 257 251
   q15  577 529 527 527
   q16  512 454 462 454
   q17  1895183918651839
   q18  8559814180208020
   q19  1774175217271727
   q20  2256196119241924
   q21  6544615761156115
   q22  505 441 446 441
   Total cold run time: 63866 ms
   Total hot run time: 60414 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch master updated: [fix](chore) update dcheck to avoid core during stress test (#28895)

2023-12-23 Thread dataroaring
This is an automated email from the ASF dual-hosted git repository.

dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new de6c7a792e9 [fix](chore) update dcheck to avoid core during stress 
test (#28895)
de6c7a792e9 is described below

commit de6c7a792e95ea8d12b0f20677d1ee567bbec81a
Author: zhannngchen <48427519+zhannngc...@users.noreply.github.com>
AuthorDate: Sat Dec 23 18:49:57 2023 +0800

[fix](chore) update dcheck to avoid core during stress test (#28895)
---
 be/src/olap/tablet.cpp | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/be/src/olap/tablet.cpp b/be/src/olap/tablet.cpp
index 47fd726b375..a5f34633d79 100644
--- a/be/src/olap/tablet.cpp
+++ b/be/src/olap/tablet.cpp
@@ -3016,7 +3016,10 @@ Status 
Tablet::calc_segment_delete_bitmap(RowsetSharedPtr rowset,
 auto st = lookup_row_key(key, true, specified_rowsets, &loc, 
dummy_version.first - 1,
  segment_caches, &rowset_find);
 bool expected_st = st.ok() || st.is() || 
st.is();
-DCHECK(expected_st) << "unexpected error status while 
lookup_row_key:" << st;
+// It's a defensive DCHECK, we need to exclude some common errors 
to avoid core-dump
+// while stress test
+DCHECK(expected_st || st.is())
+<< "unexpected error status while lookup_row_key:" << st;
 if (!expected_st) {
 return st;
 }


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Error while running github feature from .asf.yaml in doris!

2023-12-23 Thread Apache Infrastructure


An error occurred while running github feature in .asf.yaml!:
You can only have a maximum of 10 external triage collaborators, please contact 
vp-in...@apache.org to request an exception.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](chore) update DCHECK to avoid core during stress test [doris]

2023-12-23 Thread via GitHub


dataroaring merged PR #28895:
URL: https://github.com/apache/doris/pull/28895


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement] add a lower bound for bytes in scanner queue [doris]

2023-12-23 Thread via GitHub


dataroaring commented on PR #28912:
URL: https://github.com/apache/doris/pull/28912#issuecomment-1868267755

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement] add a lower bound for bytes in scanner queue [doris]

2023-12-23 Thread via GitHub


dataroaring commented on PR #28912:
URL: https://github.com/apache/doris/pull/28912#issuecomment-1868267919

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhance-wip](multi-catalog) Speed up consume rate of hms events. [doris]

2023-12-23 Thread via GitHub


dutyu commented on PR #27666:
URL: https://github.com/apache/doris/pull/27666#issuecomment-1868268009

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement] add a lower bound for bytes in scanner queue [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28912:
URL: https://github.com/apache/doris/pull/28912#issuecomment-1868268502

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Improvement](execute) optimize for ColumnNullable's serialize_vec/deserialize_vec [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28788:
URL: https://github.com/apache/doris/pull/28788#issuecomment-1868269171

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.42 seconds
stream load tsv:  578 seconds loaded 74807831229 Bytes, about 123 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  29.1 seconds inserted 1000 Rows, about 
343K ops/s
storage size: 17184166728 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement] add a lower bound for bytes in scanner queue [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28912:
URL: https://github.com/apache/doris/pull/28912#issuecomment-1868269260

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhance-wip](multi-catalog) Speed up consume rate of hms events. [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #27666:
URL: https://github.com/apache/doris/pull/27666#issuecomment-1868273738

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit c0c0e137de47d6b3b2ac634161f9027cb4bcba0a, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4718441044154410
   q2   372 143 159 143
   q3   1449124612501246
   q4   1095907 914 907
   q5   3168316531833165
   q6   252 138 133 133
   q7   1004499 486 486
   q8   2175220721972197
   q9   6676667366496649
   q10  3204328732833283
   q11  308 182 189 182
   q12  350 210 208 208
   q13  4525382937653765
   q14  243 218 215 215
   q15  568 522 521 521
   q16  437 394 391 391
   q17  1000653 602 602
   q18  7125672168056721
   q19  1530144213871387
   q20  543 298 286 286
   q21  3075264126452641
   q22  351 280 281 280
   Total cold run time: 44168 ms
   Total hot run time: 39818 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4347436343434343
   q2   277 166 174 166
   q3   3521350334963496
   q4   2404237123832371
   q5   5702570256945694
   q6   241 121 121 121
   q7   2365190018351835
   q8   3517352235183518
   q9   9025905990189018
   q10  3893399740133997
   q11  496 368 367 367
   q12  765 589 586 586
   q13  4282354935703549
   q14  290 255 251 251
   q15  579 522 520 520
   q16  515 464 462 462
   q17  1883184918531849
   q18  8444813281598132
   q19  1731178217401740
   q20  2263195619601956
   q21  6482617861306130
   q22  519 412 420 412
   Total cold run time: 63541 ms
   Total hot run time: 60513 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement] add a lower bound for bytes in scanner queue [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28912:
URL: https://github.com/apache/doris/pull/28912#issuecomment-1868275504

   TeamCity be ut coverage result:
Function Coverage: 36.59% (8549/23364) 
Line Coverage: 28.66% (69510/242505)
Region Coverage: 27.66% (35945/129952)
Branch Coverage: 24.40% (18376/75320)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/bcf21a69ffe78a7c2454e2c7ed078fe27b3de9b9_bcf21a69ffe78a7c2454e2c7ed078fe27b3de9b9/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhance-wip](multi-catalog) Speed up consume rate of hms events. [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #27666:
URL: https://github.com/apache/doris/pull/27666#issuecomment-1868275698

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.32 seconds
stream load tsv:  568 seconds loaded 74807831229 Bytes, about 125 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.5 seconds inserted 1000 Rows, about 
350K ops/s
storage size: 17183123141 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement] add a lower bound for bytes in scanner queue [doris]

2023-12-23 Thread via GitHub


dataroaring commented on PR #28912:
URL: https://github.com/apache/doris/pull/28912#issuecomment-1868277601

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [fix](mtmv)fix can not create mtmv all use default value [doris]

2023-12-23 Thread via GitHub


zddr opened a new pull request, #28922:
URL: https://github.com/apache/doris/pull/28922

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](mtmv)fix can not create mtmv all use default value [doris]

2023-12-23 Thread via GitHub


zddr commented on PR #28922:
URL: https://github.com/apache/doris/pull/28922#issuecomment-1868278255

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement] add a lower bound for bytes in scanner queue [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28912:
URL: https://github.com/apache/doris/pull/28912#issuecomment-1868278807

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement] add a lower bound for bytes in scanner queue [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28912:
URL: https://github.com/apache/doris/pull/28912#issuecomment-1868282857

   TeamCity be ut coverage result:
Function Coverage: 36.59% (8548/23364) 
Line Coverage: 28.66% (69502/242505)
Region Coverage: 27.66% (35940/129952)
Branch Coverage: 24.39% (18371/75320)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/d06f492f37e2aa3aaffb3dd8fc7abfb745671b48_d06f492f37e2aa3aaffb3dd8fc7abfb745671b48/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](mtmv)fix can not create mtmv all use default value [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28922:
URL: https://github.com/apache/doris/pull/28922#issuecomment-1868284498

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit a7a86d997664870b1af3fdb6b30dc48362912038, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4683437944484379
   q2   369 143 158 143
   q3   1465125012881250
   q4   1113944 879 879
   q5   3159317531713171
   q6   248 131 133 131
   q7   989 482 490 482
   q8   2183219221892189
   q9   6678664666726646
   q10  3231326132793261
   q11  307 179 184 179
   q12  363 208 209 208
   q13  4545383437493749
   q14  245 212 215 212
   q15  574 525 519 519
   q16  440 384 378 378
   q17  1009564 542 542
   q18  7000672968256729
   q19  1511142414741424
   q20  550 331 284 284
   q21  3115262726782627
   q22  350 275 278 275
   Total cold run time: 44127 ms
   Total hot run time: 39657 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4359434343454343
   q2   272 164 177 164
   q3   3505349934743474
   q4   2380236423622362
   q5   5705569156855685
   q6   241 123 119 119
   q7   2332187319031873
   q8   3523352335163516
   q9   8983897189048904
   q10  3899397140133971
   q11  492 378 376 376
   q12  756 598 598 598
   q13  4285353335603533
   q14  288 261 255 255
   q15  572 522 521 521
   q16  479 444 471 444
   q17  1883185618561856
   q18  8470813779307930
   q19  1775177117541754
   q20  2260194819331933
   q21  6507612061296120
   q22  489 412 432 412
   Total cold run time: 63455 ms
   Total hot run time: 60143 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement] add a lower bound for bytes in scanner queue [doris]

2023-12-23 Thread via GitHub


dataroaring commented on PR #28912:
URL: https://github.com/apache/doris/pull/28912#issuecomment-1868285980

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](mtmv)fix can not create mtmv all use default value [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28922:
URL: https://github.com/apache/doris/pull/28922#issuecomment-1868286434

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.64 seconds
stream load tsv:  566 seconds loaded 74807831229 Bytes, about 126 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.8 seconds inserted 1000 Rows, about 
347K ops/s
storage size: 17188656333 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](nereids) Fix query mv rewrite fail when mv cache build quickly [doris]

2023-12-23 Thread via GitHub


seawinde commented on PR #28876:
URL: https://github.com/apache/doris/pull/28876#issuecomment-1868286552

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [fix](merge-on-write) migration may cause duplicate keys for mow table [doris]

2023-12-23 Thread via GitHub


liaoxin01 opened a new pull request, #28923:
URL: https://github.com/apache/doris/pull/28923

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Fix](statistics) Fix partition name NPE and sample for all table during auto analyze. [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28916:
URL: https://github.com/apache/doris/pull/28916#issuecomment-1868286892

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement] add a lower bound for bytes in scanner queue [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on code in PR #28912:
URL: https://github.com/apache/doris/pull/28912#discussion_r1435592850


##
be/src/util/bvar_helper.h:
##
@@ -16,6 +16,7 @@
 // under the License.
 
 #pragma once
+#include 

Review Comment:
   warning: 'bvar/bvar.h' file not found [clang-diagnostic-error]
   ```cpp
   #include 
^
   ```
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](merge-on-write) fix migration may cause duplicate keys for mow table [doris]

2023-12-23 Thread via GitHub


liaoxin01 commented on PR #28923:
URL: https://github.com/apache/doris/pull/28923#issuecomment-1868287243

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](mtmv)fix can not create mtmv all use default value [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28922:
URL: https://github.com/apache/doris/pull/28922#issuecomment-1868287407

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](mtmv)fix can not create mtmv all use default value [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28922:
URL: https://github.com/apache/doris/pull/28922#issuecomment-1868287412

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [I] [Bug] The documentation of s3 tvf does not include the usage example of BOS [doris]

2023-12-23 Thread via GitHub


morningman closed issue #28897: [Bug] The documentation of s3 tvf does not 
include the usage example of BOS
URL: https://github.com/apache/doris/issues/28897


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](mtmv)fix can not create mtmv all use default value [doris]

2023-12-23 Thread via GitHub


morningman commented on code in PR #28922:
URL: https://github.com/apache/doris/pull/28922#discussion_r1435593376


##
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##
@@ -616,6 +616,9 @@ private String getOriginSql(ParserRuleContext ctx) {
 
 @Override
 public MTMVRefreshTriggerInfo visitRefreshTrigger(RefreshTriggerContext 
ctx) {
+if (ctx == null) {

Review Comment:
   If all are default value, better create a default RefreshTriggerContext?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](doc) Add the usage example of bos to the documentation of s3 tvf [doris]

2023-12-23 Thread via GitHub


morningman merged PR #28899:
URL: https://github.com/apache/doris/pull/28899


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch master updated: [fix](doc) Add the usage example of bos to the documentation of s3 tvf (#28899)

2023-12-23 Thread morningman
This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 5e9e199ca0d [fix](doc) Add the usage example of bos to the 
documentation of s3 tvf (#28899)
5e9e199ca0d is described below

commit 5e9e199ca0deffc1bb5a5923efd63b433cf6a647
Author: nanfeng <42513321+nanfeng1...@users.noreply.github.com>
AuthorDate: Sat Dec 23 20:47:50 2023 +0800

[fix](doc) Add the usage example of bos to the documentation of s3 tvf 
(#28899)
---
 docs/en/docs/sql-manual/sql-functions/table-functions/s3.md| 10 ++
 docs/zh-CN/docs/sql-manual/sql-functions/table-functions/s3.md | 10 ++
 2 files changed, 20 insertions(+)

diff --git a/docs/en/docs/sql-manual/sql-functions/table-functions/s3.md 
b/docs/en/docs/sql-manual/sql-functions/table-functions/s3.md
index c482542595d..e410bf39649 100644
--- a/docs/en/docs/sql-manual/sql-functions/table-functions/s3.md
+++ b/docs/en/docs/sql-manual/sql-functions/table-functions/s3.md
@@ -160,6 +160,16 @@ select * from s3(
 "region" = "ap-hongkong",
 "format" = "parquet",
 "use_path_style" = "false");
+
+// The BOS on Baidu Cloud will use 'virtual-hosted style' compatible with the 
S3 protocol to access s3.
+// BOS
+select * from s3(
+"uri" = "https://example-bucket.s3.bj.bcebos.com/your-folder/file.parquet";,
+"s3.access_key"= "ak",
+"s3.secret_key" = "sk",
+"s3.region" = "bj",
+"format" = "parquet",
+"use_path_style" = "false");
 ```
 
 Example of s3://:
diff --git a/docs/zh-CN/docs/sql-manual/sql-functions/table-functions/s3.md 
b/docs/zh-CN/docs/sql-manual/sql-functions/table-functions/s3.md
index 5a9ffd60404..5ee99684a90 100644
--- a/docs/zh-CN/docs/sql-manual/sql-functions/table-functions/s3.md
+++ b/docs/zh-CN/docs/sql-manual/sql-functions/table-functions/s3.md
@@ -160,6 +160,16 @@ select * from s3(
 "s3.region" = "ap-hongkong",
 "format" = "parquet",
 "use_path_style" = "false");
+
+// 百度云bos采用兼容s3协议的virtual-hosted style方式访问s3。
+// BOS
+select * from s3(
+"uri" = "https://example-bucket.s3.bj.bcebos.com/your-folder/file.parquet";,
+"s3.access_key"= "ak",
+"s3.secret_key" = "sk",
+"s3.region" = "bj",
+"format" = "parquet",
+"use_path_style" = "false");
 ```
 
 s3:// 使用示例:


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](merge-on-write) fix migration may cause duplicate keys for mow table [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28923:
URL: https://github.com/apache/doris/pull/28923#issuecomment-1868287713

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Error while running github feature from .asf.yaml in doris!

2023-12-23 Thread Apache Infrastructure


An error occurred while running github feature in .asf.yaml!:
You can only have a maximum of 10 external triage collaborators, please contact 
vp-in...@apache.org to request an exception.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](nereids) Fix query mv rewrite fail when mv cache build quickly [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28876:
URL: https://github.com/apache/doris/pull/28876#issuecomment-1868288037

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](nereids) Fix query mv rewrite fail when mv cache build quickly [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28876:
URL: https://github.com/apache/doris/pull/28876#issuecomment-1868288042

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] Related partition exclude null generate column when increment build materialized view [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28855:
URL: https://github.com/apache/doris/pull/28855#issuecomment-1868288266

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] Related partition exclude null generate column when increment build materialized view [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28855:
URL: https://github.com/apache/doris/pull/28855#issuecomment-1868288271

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement](statistics)Remove retry load when load stats cache fail. [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28904:
URL: https://github.com/apache/doris/pull/28904#issuecomment-1868288646

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](parquet) the end offset of column chunk may be wrong in parquet metadata [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28891:
URL: https://github.com/apache/doris/pull/28891#issuecomment-1868288935

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement] add a lower bound for bytes in scanner queue [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28912:
URL: https://github.com/apache/doris/pull/28912#issuecomment-1868291291

   TeamCity be ut coverage result:
Function Coverage: 36.60% (8551/23364) 
Line Coverage: 28.68% (69548/242505)
Region Coverage: 27.67% (35963/129952)
Branch Coverage: 24.41% (18383/75320)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/c7ea8d4a861012c14faed9615c9b77f9eb2582eb_c7ea8d4a861012c14faed9615c9b77f9eb2582eb/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](nereids) Fix query mv rewrite fail when mv cache build quickly [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28876:
URL: https://github.com/apache/doris/pull/28876#issuecomment-1868292351

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit ccd458dfbc6ba05f57e34b86e37683e0a5f98ace, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4678440844504408
   q2   371 141 161 141
   q3   1470125212511251
   q4   1127907 885 885
   q5   3212316031513151
   q6   245 130 130 130
   q7   976 497 483 483
   q8   2188222021912191
   q9   6710667366496649
   q10  3239328032743274
   q11  308 193 181 181
   q12  354 211 204 204
   q13  4546379538023795
   q14  249 220 208 208
   q15  572 524 516 516
   q16  440 383 390 383
   q17  1019576 529 529
   q18  7040675568446755
   q19  1523143014541430
   q20  537 297 313 297
   q21  3087263427012634
   q22  345 276 277 276
   Total cold run time: 44236 ms
   Total hot run time: 39771 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4347434543394339
   q2   268 163 174 163
   q3   3502349834853485
   q4   2398237723782377
   q5   5699570757235707
   q6   240 124 122 122
   q7   2345184418771844
   q8   3507352435233523
   q9   8998896289798962
   q10  3917400639953995
   q11  475 374 360 360
   q12  770 599 589 589
   q13  4303357435353535
   q14  300 242 258 242
   q15  574 519 520 519
   q16  511 486 473 473
   q17  1876186118511851
   q18  8549815780908090
   q19  1746177717691769
   q20  2249196419291929
   q21  6488618261556155
   q22  501 423 429 423
   Total cold run time: 63563 ms
   Total hot run time: 60452 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](mtmv)fix can not create mtmv all use default value [doris]

2023-12-23 Thread via GitHub


morningman commented on code in PR #28922:
URL: https://github.com/apache/doris/pull/28922#discussion_r1435599823


##
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##
@@ -616,6 +616,9 @@ private String getOriginSql(ParserRuleContext ctx) {
 
 @Override
 public MTMVRefreshTriggerInfo visitRefreshTrigger(RefreshTriggerContext 
ctx) {
+if (ctx == null) {

Review Comment:
   I got it



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](merge-on-write) fix migration may cause duplicate keys for mow table [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28923:
URL: https://github.com/apache/doris/pull/28923#issuecomment-1868293238

   TeamCity be ut coverage result:
Function Coverage: 36.59% (8550/23364) 
Line Coverage: 28.67% (69517/242500)
Region Coverage: 27.66% (35941/129945)
Branch Coverage: 24.40% (18376/75326)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/3f165e583563f3611dd739f033c98c21276a8661_3f165e583563f3611dd739f033c98c21276a8661/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](merge-on-write) fix migration may cause duplicate keys for mow table [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28923:
URL: https://github.com/apache/doris/pull/28923#issuecomment-1868293974

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit 3f165e583563f3611dd739f033c98c21276a8661, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4686441044354410
   q2   369 146 159 146
   q3   1472125212191219
   q4   1113905 903 903
   q5   3163315332043153
   q6   254 133 138 133
   q7   999 484 495 484
   q8   2198222021872187
   q9   6700664166986641
   q10  3216330932673267
   q11  310 188 194 188
   q12  356 213 210 210
   q13  4593383338123812
   q14  240 215 214 214
   q15  583 520 520 520
   q16  445 384 385 384
   q17  1017717 563 563
   q18  7160689467996799
   q19  1521146214351435
   q20  508 323 309 309
   q21  3118269726742674
   q22  350 281 290 281
   Total cold run time: 44371 ms
   Total hot run time: 39932 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4346434243374337
   q2   272 170 180 170
   q3   3508348334983483
   q4   2415238423772377
   q5   5711573557225722
   q6   243 123 126 123
   q7   2375186618381838
   q8   3528353935313531
   q9   9034900689988998
   q10  3887397440113974
   q11  486 360 373 360
   q12  779 600 594 594
   q13  4299357235573557
   q14  286 248 262 248
   q15  580 521 512 512
   q16  506 479 451 451
   q17  1886187818831878
   q18  8563811881138113
   q19  1752175417441744
   q20  2251193119261926
   q21  6512616961656165
   q22  521 433 423 423
   Total cold run time: 63740 ms
   Total hot run time: 60524 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Fix](statistics) Fix partition name NPE and sample for all table during auto analyze. [doris]

2023-12-23 Thread via GitHub


Jibing-Li commented on PR #28916:
URL: https://github.com/apache/doris/pull/28916#issuecomment-1868294074

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](nereids) Fix query mv rewrite fail when mv cache build quickly [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28876:
URL: https://github.com/apache/doris/pull/28876#issuecomment-1868294520

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.29 seconds
stream load tsv:  570 seconds loaded 74807831229 Bytes, about 125 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  33 seconds loaded 861443392 Bytes, about 24 
MB/s
insert into select:  28.7 seconds inserted 1000 Rows, about 
348K ops/s
storage size: 17184424703 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](mtmv)fix can not create mtmv all use default value [doris]

2023-12-23 Thread via GitHub


morningman merged PR #28922:
URL: https://github.com/apache/doris/pull/28922


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch master updated: [fix](mtmv)fix can not create mtmv all use default value (#28922)

2023-12-23 Thread morningman
This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 66b14f4db1f [fix](mtmv)fix can not create mtmv all use default value 
(#28922)
66b14f4db1f is described below

commit 66b14f4db1f34371237f45b9bca64fc9ef951cc4
Author: zhangdong <493738...@qq.com>
AuthorDate: Sat Dec 23 21:27:01 2023 +0800

[fix](mtmv)fix can not create mtmv all use default value (#28922)
---
 .../org/apache/doris/nereids/parser/LogicalPlanBuilder.java |  6 ++
 regression-test/suites/mtmv_p0/test_build_mtmv.groovy   | 13 +
 2 files changed, 19 insertions(+)

diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java
 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java
index dcbfb9ef3f0..37b42c7123f 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java
@@ -616,6 +616,9 @@ public class LogicalPlanBuilder extends 
DorisParserBaseVisitor {
 
 @Override
 public MTMVRefreshTriggerInfo visitRefreshTrigger(RefreshTriggerContext 
ctx) {
+if (ctx == null) {
+return new MTMVRefreshTriggerInfo(RefreshTrigger.MANUAL);
+}
 if (ctx.MANUAL() != null) {
 return new MTMVRefreshTriggerInfo(RefreshTrigger.MANUAL);
 }
@@ -662,6 +665,9 @@ public class LogicalPlanBuilder extends 
DorisParserBaseVisitor {
 
 @Override
 public BuildMode visitBuildMode(BuildModeContext ctx) {
+if (ctx == null) {
+return BuildMode.IMMEDIATE;
+}
 if (ctx.DEFERRED() != null) {
 return BuildMode.DEFERRED;
 } else if (ctx.IMMEDIATE() != null) {
diff --git a/regression-test/suites/mtmv_p0/test_build_mtmv.groovy 
b/regression-test/suites/mtmv_p0/test_build_mtmv.groovy
index 882a7eff22e..eb4560f8b06 100644
--- a/regression-test/suites/mtmv_p0/test_build_mtmv.groovy
+++ b/regression-test/suites/mtmv_p0/test_build_mtmv.groovy
@@ -152,6 +152,19 @@ suite("test_build_mtmv") {
 DROP MATERIALIZED VIEW ${mvName}
 """
 
+// use default value
+sql """
+CREATE MATERIALIZED VIEW ${mvName}
+DISTRIBUTED BY RANDOM BUCKETS 2
+PROPERTIES ('replication_num' = '1')
+AS
+SELECT ${tableName}.username, ${tableNamePv}.pv FROM ${tableName}, 
${tableNamePv} WHERE ${tableName}.id=${tableNamePv}.id;
+"""
+
+sql """
+DROP MATERIALIZED VIEW ${mvName}
+"""
+
 // IMMEDIATE schedule interval
 sql """
 CREATE MATERIALIZED VIEW ${mvName}


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Error while running github feature from .asf.yaml in doris!

2023-12-23 Thread Apache Infrastructure


An error occurred while running github feature in .asf.yaml!:
You can only have a maximum of 10 external triage collaborators, please contact 
vp-in...@apache.org to request an exception.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Enhancement](Wal)Support dynamic wal space limit [doris]

2023-12-23 Thread via GitHub


Yukang-Lian commented on PR #27726:
URL: https://github.com/apache/doris/pull/27726#issuecomment-1868294892

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Fix](statistics) Fix partition name NPE and sample for all table during auto analyze. [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28916:
URL: https://github.com/apache/doris/pull/28916#issuecomment-1868295154

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](merge-on-write) fix migration may cause duplicate keys for mow table [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28923:
URL: https://github.com/apache/doris/pull/28923#issuecomment-1868296156

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.07 seconds
stream load tsv:  566 seconds loaded 74807831229 Bytes, about 126 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.8 seconds inserted 1000 Rows, about 
347K ops/s
storage size: 17184037176 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [branch-2.0-var](disk balance) Impr disk rebalancer sched #26412 [doris]

2023-12-23 Thread via GitHub


yujun777 commented on PR #28920:
URL: https://github.com/apache/doris/pull/28920#issuecomment-1868296612

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](nereids) Fix query mv rewrite fail when mv cache build quickly [doris]

2023-12-23 Thread via GitHub


seawinde commented on PR #28876:
URL: https://github.com/apache/doris/pull/28876#issuecomment-1868297143

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Fix](statistics) Fix partition name NPE and sample for all table during auto analyze. [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28916:
URL: https://github.com/apache/doris/pull/28916#issuecomment-1868297805

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit 389c80c8461afd01ca28ea07b375c076a404b855, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4724443944804439
   q2   378 185 167 167
   q3   1467127112491249
   q4   1124907 920 907
   q5   3182314631643146
   q6   248 129 130 129
   q7   994 487 493 487
   q8   220621742174
   q9   6688662866596628
   q10  3220328032443244
   q11  312 194 196 194
   q12  358 218 212 212
   q13  4564383138013801
   q14  243 210 213 210
   q15  574 528 528 528
   q16  439 387 388 387
   q17  1005559 509 509
   q18  7059675867836758
   q19  1511144014551440
   q20  537 300 337 300
   q21  3087265326672653
   q22  351 278 287 278
   Total cold run time: 44271 ms
   Total hot run time: 39840 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4389437643874376
   q2   268 165 173 165
   q3   3502349835083498
   q4   2397237123662366
   q5   5715571957185718
   q6   240 126 124 124
   q7   2360183718861837
   q8   3525352235063506
   q9   9087900489638963
   q10  3915397040053970
   q11  487 357 375 357
   q12  769 607 606 606
   q13  4289354035613540
   q14  288 250 255 250
   q15  576 530 526 526
   q16  503 455 455 455
   q17  1888184718341834
   q18  8438806780138013
   q19  1754177417501750
   q20  2241195719201920
   q21  6494614160976097
   q22  514 423 444 423
   Total cold run time: 63639 ms
   Total hot run time: 60294 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Enhancement](Wal)Support dynamic wal space limit [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #27726:
URL: https://github.com/apache/doris/pull/27726#issuecomment-1868299348

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit 5ee0e280a4c6dbb488d8b1da7221546ee51903e7, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4687445344434443
   q2   387 138 157 138
   q3   1458127812221222
   q4   1105884 884 884
   q5   3178320832023202
   q6   252 130 136 130
   q7   1009486 496 486
   q8   2194223522152215
   q9   6700669566576657
   q10  3205328232843282
   q11  306 192 185 185
   q12  358 204 212 204
   q13  4570379937793779
   q14  238 213 223 213
   q15  574 524 514 514
   q16  442 389 386 386
   q17  1017696 581 581
   q18  7046674869786748
   q19  1535142014791420
   q20  531 312 310 310
   q21  3101267026442644
   q22  345 282 281 281
   Total cold run time: 44238 ms
   Total hot run time: 39924 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4360439143864386
   q2   274 165 172 165
   q3   3529349835043498
   q4   2408237523802375
   q5   5731569857215698
   q6   246 123 124 123
   q7   2372183618261826
   q8   3533354535433543
   q9   9081900690299006
   q10  3920398639953986
   q11  488 368 371 368
   q12  772 592 611 592
   q13  4279354135353535
   q14  290 252 254 252
   q15  567 522 520 520
   q16  499 478 467 467
   q17  1909189318771877
   q18  8570822181408140
   q19  1761176917701769
   q20  2250193919431939
   q21  6518620461516151
   q22  494 431 416 416
   Total cold run time: 63851 ms
   Total hot run time: 60632 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Enhancement](Wal)Support dynamic wal space limit [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #27726:
URL: https://github.com/apache/doris/pull/27726#issuecomment-1868299849

   TeamCity be ut coverage result:
Function Coverage: 36.61% (8560/23383) 
Line Coverage: 28.68% (69623/242747)
Region Coverage: 27.66% (35996/130121)
Branch Coverage: 24.39% (18399/75448)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/5ee0e280a4c6dbb488d8b1da7221546ee51903e7_5ee0e280a4c6dbb488d8b1da7221546ee51903e7/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Fix](statistics) Fix partition name NPE and sample for all table during auto analyze. [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28916:
URL: https://github.com/apache/doris/pull/28916#issuecomment-1868299805

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.77 seconds
stream load tsv:  568 seconds loaded 74807831229 Bytes, about 125 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  33 seconds loaded 861443392 Bytes, about 24 
MB/s
insert into select:  29.0 seconds inserted 1000 Rows, about 
344K ops/s
storage size: 17184573743 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](nereids) Fix query mv rewrite fail when mv cache build quickly [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28876:
URL: https://github.com/apache/doris/pull/28876#issuecomment-1868301024

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit a98e4398046eed969d936669e007bcfff403bdc8, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4692443044474430
   q2   378 138 158 138
   q3   1444121111941194
   q4   1112890 919 890
   q5   3151314432043144
   q6   245 132 129 129
   q7   1028515 491 491
   q8   2190223821882188
   q9   6698667866476647
   q10  3220328032713271
   q11  310 181 183 181
   q12  354 211 210 210
   q13  4545382437983798
   q14  246 211 211 211
   q15  568 523 520 520
   q16  444 387 394 387
   q17  1021614 565 565
   q18  7082681767436743
   q19  1532140113681368
   q20  543 311 284 284
   q21  3083263026402630
   q22  346 285 278 278
   Total cold run time: 44232 ms
   Total hot run time: 39697 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4326434643444344
   q2   269 166 169 166
   q3   3517349534843484
   q4   2404237323792373
   q5   5687569657715696
   q6   240 122 125 122
   q7   2385183818771838
   q8   3535353335263526
   q9   9024898689768976
   q10  3914399439803980
   q11  489 386 375 375
   q12  760 603 591 591
   q13  4302356035543554
   q14  291 251 263 251
   q15  575 517 518 517
   q16  494 468 448 448
   q17  1895183418361834
   q18  8560818881378137
   q19  1738175317361736
   q20  2257194519271927
   q21  6502617661086108
   q22  503 413 429 413
   Total cold run time: 63667 ms
   Total hot run time: 60396 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Enhancement](Wal)Support dynamic wal space limit [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #27726:
URL: https://github.com/apache/doris/pull/27726#issuecomment-1868301669

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.19 seconds
stream load tsv:  566 seconds loaded 74807831229 Bytes, about 126 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.3 seconds inserted 1000 Rows, about 
353K ops/s
storage size: 17183750098 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement] add a lower bound for bytes in scanner queue [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28912:
URL: https://github.com/apache/doris/pull/28912#issuecomment-1868302410

   
   TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Tpch sf100 test result on commit c7ea8d4a861012c14faed9615c9b77f9eb2582eb, 
data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1   4760432144714321
   q2   374 178 158 158
   q3   1425118212071182
   q4   1073862 760 760
   q5   3215312131853121
   q6   228 130 136 130
   q7   948 466 474 466
   q8   2145221321772177
   q9   6608657165556555
   q10  3192311530773077
   q11  297 180 182 180
   q12  358 205 205 205
   q13  4533376637713766
   q14  234 211 211 211
   q15  550 511 508 508
   q16  432 405 409 405
   q17  995 682 529 529
   q18  6390603063826030
   q19  1560137414371374
   q20  521 335 307 307
   q21  2935250524652465
   q22  346 274 284 274
   Total cold run time: 43119 ms
   Total hot run time: 38201 ms
   
   run tpch-sf100 query with default conf and set session variable 
runtime_filter_mode=off
   q1   4323435342174217
   q2   303 228 230 228
   q3   3224299530182995
   q4   2100190218891889
   q5   5291525052535250
   q6   240 122 122 122
   q7   2209177718071777
   q8   3317342634043404
   q9   8584856584688468
   q10  3866376537853765
   q11  545 433 422 422
   q12  743 633 581 581
   q13  4319359635213521
   q14  286 266 264 264
   q15  560 513 509 509
   q16  524 501 486 486
   q17  1852166616741666
   q18  7758740480027404
   q19  1785172917141714
   q20  2257198119711971
   q21  5179517250025002
   q22  550 431 439 431
   Total cold run time: 59815 ms
   Total hot run time: 56086 ms
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](nereids) Fix query mv rewrite fail when mv cache build quickly [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28876:
URL: https://github.com/apache/doris/pull/28876#issuecomment-1868303534

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.55 seconds
stream load tsv:  566 seconds loaded 74807831229 Bytes, about 126 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.9 seconds inserted 1000 Rows, about 
346K ops/s
storage size: 17183822725 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](parquet) the end offset of column chunk may be wrong in parquet metadata [doris]

2023-12-23 Thread via GitHub


morningman merged PR #28891:
URL: https://github.com/apache/doris/pull/28891


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch master updated: [fix](parquet) the end offset of column chunk may be wrong in parquet metadata (#28891)

2023-12-23 Thread morningman
This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 96d4778f2ec [fix](parquet) the end offset of column chunk may be wrong 
in parquet metadata (#28891)
96d4778f2ec is described below

commit 96d4778f2ec837eef5bfe94b18bde4ccd0400c23
Author: Ashin Gau 
AuthorDate: Sat Dec 23 22:21:04 2023 +0800

[fix](parquet) the end offset of column chunk may be wrong in parquet 
metadata (#28891)
---
 .../vec/exec/format/parquet/vparquet_column_chunk_reader.cpp |  2 ++
 .../vec/exec/format/parquet/vparquet_column_chunk_reader.h   | 11 ++-
 be/src/vec/exec/format/parquet/vparquet_page_reader.h|  5 +
 regression-test/data/external_table_p2/tvf/test_tvf_p2.out   | 12 
 .../suites/external_table_p2/tvf/test_tvf_p2.groovy  |  6 ++
 5 files changed, 35 insertions(+), 1 deletion(-)

diff --git a/be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.cpp 
b/be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.cpp
index 928b5ae70bd..6feb9bc1025 100644
--- a/be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.cpp
+++ b/be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.cpp
@@ -97,10 +97,12 @@ Status ColumnChunkReader::next_page() {
 return next_page();
 } else if (_page_reader->get_page_header()->type == 
tparquet::PageType::DATA_PAGE_V2) {
 _remaining_num_values = 
_page_reader->get_page_header()->data_page_header_v2.num_values;
+_chunk_parsed_values += _remaining_num_values;
 _state = HEADER_PARSED;
 return Status::OK();
 } else {
 _remaining_num_values = 
_page_reader->get_page_header()->data_page_header.num_values;
+_chunk_parsed_values += _remaining_num_values;
 _state = HEADER_PARSED;
 return Status::OK();
 }
diff --git a/be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.h 
b/be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.h
index 21ee808e48f..c8a49e098a5 100644
--- a/be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.h
+++ b/be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.h
@@ -91,9 +91,17 @@ public:
 Status init();
 
 // Whether the chunk reader has a more page to read.
-bool has_next_page() { return _page_reader->has_next_page(); }
+bool has_next_page() { return _chunk_parsed_values < _metadata.num_values; 
}
 
+// Deprecated
 // Seek to the specific page, page_header_offset must be the start offset 
of the page header.
+// _end_offset may exceed the actual data area, so we can only use the 
number of parsed values
+// to determine whether there are remaining pages to read. That's to say 
we can't use the
+// PageLocation in parquet metadata to seek to the specified page. We 
should call next_page()
+// and skip_page() to skip pages one by one.
+// todo: change this interface to seek_to_page(int64_t page_header_offset, 
size_t num_parsed_values)
+// and set _chunk_parsed_values = num_parsed_values
+// [[deprecated]]
 void seek_to_page(int64_t page_header_offset) {
 _remaining_num_values = 0;
 _page_reader->seek_to_page(page_header_offset);
@@ -201,6 +209,7 @@ private:
 
 LevelDecoder _rep_level_decoder;
 LevelDecoder _def_level_decoder;
+size_t _chunk_parsed_values = 0;
 uint32_t _remaining_num_values = 0;
 Slice _page_data;
 std::unique_ptr _decompress_buf;
diff --git a/be/src/vec/exec/format/parquet/vparquet_page_reader.h 
b/be/src/vec/exec/format/parquet/vparquet_page_reader.h
index 730b9a3001b..bdd0a8d0f5f 100644
--- a/be/src/vec/exec/format/parquet/vparquet_page_reader.h
+++ b/be/src/vec/exec/format/parquet/vparquet_page_reader.h
@@ -45,6 +45,11 @@ public:
uint64_t length);
 ~PageReader() = default;
 
+// Deprecated
+// Parquet file may not be standardized,
+// _end_offset may exceed the actual data area.
+// ColumnChunkReader::has_next_page() use the number of parsed values for 
judgment
+// [[deprecated]]
 bool has_next_page() const { return _offset < _end_offset; }
 
 Status next_page_header();
diff --git a/regression-test/data/external_table_p2/tvf/test_tvf_p2.out 
b/regression-test/data/external_table_p2/tvf/test_tvf_p2.out
index 6a44b7322dc..53b454df858 100644
--- a/regression-test/data/external_table_p2/tvf/test_tvf_p2.out
+++ b/regression-test/data/external_table_p2/tvf/test_tvf_p2.out
@@ -50,6 +50,18 @@
 32.024 64. 128.901468  32.024  64. 128.901468  2023-07-07  
2023-07-07  2021-07-07T19:15:31.123456  2023-07-07  2023-07-07  
2021-07-07T19:15:31.123456
 32.689 64.2580 128.745382  32.689  64.2580 128.745382  2023-11-11  
2023-11-11  2022-11-11T16:35:37.123456  2023-11-11  2023-11-11  
2022-11-11T16:35:37.1

Re: [PR] [fix](parquet) the end offset of column chunk may be wrong in parquet metadata [doris]

2023-12-23 Thread via GitHub


github-actions[bot] commented on PR #28891:
URL: https://github.com/apache/doris/pull/28891#issuecomment-1868304185

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Error while running github feature from .asf.yaml in doris!

2023-12-23 Thread Apache Infrastructure


An error occurred while running github feature in .asf.yaml!:
You can only have a maximum of 10 external triage collaborators, please contact 
vp-in...@apache.org to request an exception.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](parquet) the end offset of column chunk may be wrong in parquet metadata [doris]

2023-12-23 Thread via GitHub


morningman merged PR #28893:
URL: https://github.com/apache/doris/pull/28893


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch branch-2.0 updated: [fix](parquet) the end offset of column chunk may be wrong in parquet metadata #28891 (#28893)

2023-12-23 Thread morningman
This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/branch-2.0 by this push:
 new adb7730b64b [fix](parquet) the end offset of column chunk may be wrong 
in parquet metadata #28891 (#28893)
adb7730b64b is described below

commit adb7730b64b0a0d882fec6517af9c2dc73cdb7b5
Author: Ashin Gau 
AuthorDate: Sat Dec 23 22:23:07 2023 +0800

[fix](parquet) the end offset of column chunk may be wrong in parquet 
metadata #28891 (#28893)

backport: #28891
---
 .../vec/exec/format/parquet/vparquet_column_chunk_reader.cpp |  2 ++
 .../vec/exec/format/parquet/vparquet_column_chunk_reader.h   | 11 ++-
 be/src/vec/exec/format/parquet/vparquet_page_reader.h|  5 +
 regression-test/data/external_table_p2/tvf/test_tvf_p2.out   | 12 
 .../suites/external_table_p2/tvf/test_tvf_p2.groovy  |  6 ++
 5 files changed, 35 insertions(+), 1 deletion(-)

diff --git a/be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.cpp 
b/be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.cpp
index aca5ae96e81..579bfa5a4ae 100644
--- a/be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.cpp
+++ b/be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.cpp
@@ -98,10 +98,12 @@ Status ColumnChunkReader::next_page() {
 return next_page();
 } else if (_page_reader->get_page_header()->type == 
tparquet::PageType::DATA_PAGE_V2) {
 _remaining_num_values = 
_page_reader->get_page_header()->data_page_header_v2.num_values;
+_chunk_parsed_values += _remaining_num_values;
 _state = HEADER_PARSED;
 return Status::OK();
 } else {
 _remaining_num_values = 
_page_reader->get_page_header()->data_page_header.num_values;
+_chunk_parsed_values += _remaining_num_values;
 _state = HEADER_PARSED;
 return Status::OK();
 }
diff --git a/be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.h 
b/be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.h
index 9f62fddce70..aa30af1e94c 100644
--- a/be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.h
+++ b/be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.h
@@ -91,9 +91,17 @@ public:
 Status init();
 
 // Whether the chunk reader has a more page to read.
-bool has_next_page() { return _page_reader->has_next_page(); }
+bool has_next_page() { return _chunk_parsed_values < _metadata.num_values; 
}
 
+// Deprecated
 // Seek to the specific page, page_header_offset must be the start offset 
of the page header.
+// _end_offset may exceed the actual data area, so we can only use the 
number of parsed values
+// to determine whether there are remaining pages to read. That's to say 
we can't use the
+// PageLocation in parquet metadata to seek to the specified page. We 
should call next_page()
+// and skip_page() to skip pages one by one.
+// todo: change this interface to seek_to_page(int64_t page_header_offset, 
size_t num_parsed_values)
+// and set _chunk_parsed_values = num_parsed_values
+// [[deprecated]]
 void seek_to_page(int64_t page_header_offset) {
 _remaining_num_values = 0;
 _page_reader->seek_to_page(page_header_offset);
@@ -201,6 +209,7 @@ private:
 
 LevelDecoder _rep_level_decoder;
 LevelDecoder _def_level_decoder;
+size_t _chunk_parsed_values = 0;
 uint32_t _remaining_num_values = 0;
 Slice _page_data;
 std::unique_ptr _decompress_buf;
diff --git a/be/src/vec/exec/format/parquet/vparquet_page_reader.h 
b/be/src/vec/exec/format/parquet/vparquet_page_reader.h
index 089409786f4..3b816629c6e 100644
--- a/be/src/vec/exec/format/parquet/vparquet_page_reader.h
+++ b/be/src/vec/exec/format/parquet/vparquet_page_reader.h
@@ -45,6 +45,11 @@ public:
uint64_t length);
 ~PageReader() = default;
 
+// Deprecated
+// Parquet file may not be standardized,
+// _end_offset may exceed the actual data area.
+// ColumnChunkReader::has_next_page() use the number of parsed values for 
judgment
+// [[deprecated]]
 bool has_next_page() const { return _offset < _end_offset; }
 
 Status next_page_header();
diff --git a/regression-test/data/external_table_p2/tvf/test_tvf_p2.out 
b/regression-test/data/external_table_p2/tvf/test_tvf_p2.out
index 6a44b7322dc..53b454df858 100644
--- a/regression-test/data/external_table_p2/tvf/test_tvf_p2.out
+++ b/regression-test/data/external_table_p2/tvf/test_tvf_p2.out
@@ -50,6 +50,18 @@
 32.024 64. 128.901468  32.024  64. 128.901468  2023-07-07  
2023-07-07  2021-07-07T19:15:31.123456  2023-07-07  2023-07-07  
2021-07-07T19:15:31.123456
 32.689 64.2580 128.745382  32.689  64.2580 128.745382  2023-11-11  
2023-11-11  2022-11-11T16:35:37.123456  2023-1

Re: [PR] [improvement] add a lower bound for bytes in scanner queue [doris]

2023-12-23 Thread via GitHub


doris-robot commented on PR #28912:
URL: https://github.com/apache/doris/pull/28912#issuecomment-1868304670

   (From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.68 seconds
stream load tsv:  579 seconds loaded 74807831229 Bytes, about 123 
MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc:  66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet:  32 seconds loaded 861443392 Bytes, about 25 
MB/s
insert into select:  28.9 seconds inserted 1000 Rows, about 
346K ops/s
storage size: 17183644549 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



  1   2   3   >