Re: [PR] [fix](planner) remove input slot for aggregate slot which is not materialized [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32092:
URL: https://github.com/apache/doris/pull/32092#issuecomment-1990909281

   
   
   TPC-DS: Total hot run time: 187409 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit c325499b1b02c504cf6414c4b1d94da1b0eaf894, 
data reload: false
   
   query1   918 363 342 342
   query2   7520199719651965
   query3   6699212 213 212
   query4   31178   20603   20614   20603
   query5   4399476 406 406
   query6   277 179 174 174
   query7   4623320 290 290
   query8   227 177 168 168
   query9   9965232223212321
   query10  450 242 247 242
   query11  14617   14170   14136   14136
   query12  130 91  83  83
   query13  1648411 432 411
   query14  11238   10209   10588   10209
   query15  252 207 194 194
   query16  7842263 252 252
   query17  1962556 523 523
   query18  1973276 279 276
   query19  214 160 142 142
   query20  87  83  82  82
   query21  198 131 121 121
   query22  4837454145994541
   query23  31660   31063   31013   31013
   query24  11068   282728322827
   query25  629 356 407 356
   query26  1575152 158 152
   query27  3139361 356 356
   query28  8241192218761876
   query29  1002640 624 624
   query30  306 148 150 148
   query31  950 690 729 690
   query32  91  59  53  53
   query33  767 261 252 252
   query34  1063473 474 473
   query35  846 628 603 603
   query36  983 884 863 863
   query37  122 75  76  75
   query38  3336319132043191
   query39  1421141113801380
   query40  233 112 111 111
   query41  51  47  48  47
   query42  96  97  96  96
   query43  490 447 461 447
   query44  1261711 719 711
   query45  273 227 256 227
   query46  1089706 693 693
   query47  1676160716011601
   query48  455 360 363 360
   query49  1165332 347 332
   query50  754 375 374 374
   query51  6757662666866626
   query52  105 91  95  91
   query53  341 267 268 267
   query54  326 249 248 248
   query55  85  81  80  80
   query56  245 234 233 233
   query57  1084100510281005
   query58  236 199 208 199
   query59  2774264426342634
   query60  263 256 251 251
   query61  113 112 110 110
   query62  589 392 404 392
   query63  304 325 276 276
   query64  6935381939363819
   query65  3095302930533029
   query66  1402331 324 324
   query67  15119   14840   14582   14582
   query68  9231551 535 535
   query69  666 386 380 380
   query70  1399120712011201
   query71  521 278 269 269
   query72  6585264924872487
   query73  1622328 328 328
   query74  7677658566026585
   query75  9723799680047996
   query76  5615949 977 949
   query77  634 258 263 258
   query78  10444   979497239723
   query79  11625   523 515 515
   query80  2241392 379 379
   query81  485 206 207 206
   query82  333 203 197 197
   query83  244 141 147 141
   query84  280 73  80  73
   query85  1240324 319 319
   query86  351 277 308 277
   query87  3543330632693269
   query88  5283235123652351
   query89  463 357 352 352
   query90  2058174 171 171
   query91  157 129 131 129
   query92  57  47  46  46
   query93  6539515 499 499
   query94  1375173 174 173
   query95  415 325 320 320
   query96  606 269 270 269
   query97  4429425842914258
   query98  224 209 204 204
   query99  973 760 762 760
   Total cold run time: 322807 ms
   Total hot run time: 187409 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to t

Re: [PR] [fix](planner) remove input slot for aggregate slot which is not materialized [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32092:
URL: https://github.com/apache/doris/pull/32092#issuecomment-1990914891

   
   
   ClickBench: Total hot run time: 30.44 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit c325499b1b02c504cf6414c4b1d94da1b0eaf894, 
data reload: false
   
   query1   0.040.030.03
   query2   0.050.020.02
   query3   0.230.060.06
   query4   1.680.100.09
   query5   0.480.520.51
   query6   1.210.620.62
   query7   0.020.010.01
   query8   0.040.030.03
   query9   0.540.500.50
   query10  0.540.550.55
   query11  0.110.090.09
   query12  0.110.090.09
   query13  0.570.570.56
   query14  0.740.740.76
   query15  0.820.780.79
   query16  0.380.400.39
   query17  0.980.951.00
   query18  0.180.280.22
   query19  1.731.661.68
   query20  0.010.000.01
   query21  15.41   0.560.56
   query22  2.692.392.06
   query23  17.18   0.760.78
   query24  3.542.670.86
   query25  0.100.120.16
   query26  0.750.120.14
   query27  0.030.030.03
   query28  10.17   0.930.85
   query29  12.53   3.193.19
   query30  0.680.520.59
   query31  2.800.360.34
   query32  3.440.440.45
   query33  2.792.802.82
   query34  16.00   4.344.31
   query35  4.314.304.37
   query36  1.081.000.99
   query37  0.070.050.06
   query38  0.040.030.03
   query39  0.020.020.02
   query40  0.170.130.14
   query41  0.080.020.02
   query42  0.030.020.02
   query43  0.030.020.02
   Total cold run time: 104.4 s
   Total hot run time: 30.44 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](memory) Fix LRU cache deleter and memory tracking [doris]

2024-03-12 Thread via GitHub


xinyiZzz commented on code in PR #32080:
URL: https://github.com/apache/doris/pull/32080#discussion_r1520949552


##
be/src/runtime/memory/lru_cache_policy.h:
##
@@ -72,7 +81,48 @@ class LRUCachePolicy : public CachePolicy {
 return true;
 }
 
-~LRUCachePolicy() override = default;
+static std::string lru_cache_type_string(LRUCacheType type) {
+switch (type) {
+case LRUCacheType::SIZE:
+return "size";
+case LRUCacheType::NUMBER:
+return "number";
+default:
+LOG(FATAL) << "not match type of lru cache:" << 
static_cast(type);
+}
+}
+
+std::shared_ptr mem_tracker() { return _mem_tracker; }
+
+Cache::Handle* insert(const CacheKey& key, void* value, size_t charge,
+  CachePriority priority = CachePriority::NORMAL, 
size_t bytes = -1) {
+return _insert(true, key, value, charge, priority, bytes);
+}
+
+Cache::Handle* insert_no_tracking(const CacheKey& key, void* value, size_t 
charge,

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](memory) Fix LRU cache deleter and memory tracking [doris]

2024-03-12 Thread via GitHub


xinyiZzz commented on PR #32080:
URL: https://github.com/apache/doris/pull/32080#issuecomment-1990915459

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] (enhance)(Cache) Control whether writing into file cache for query and load [doris]

2024-03-12 Thread via GitHub


ByteYue opened a new pull request, #32099:
URL: https://github.com/apache/doris/pull/32099

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] (enhance)(Cache) Control whether writing into file cache for query and load [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32099:
URL: https://github.com/apache/doris/pull/32099#issuecomment-1990918211

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](routine-load)add rack tag into fe & kafka routine load(#31531) [doris]

2024-03-12 Thread via GitHub


HHoflittlefish777 commented on code in PR #32098:
URL: https://github.com/apache/doris/pull/32098#discussion_r1520951488


##
fe/fe-core/src/main/java/org/apache/doris/load/routineload/KafkaRoutineLoadJob.java:
##
@@ -93,6 +95,7 @@ public class KafkaRoutineLoadJob extends RoutineLoadJob {
 // kafka properties ,property prefix will be mapped to kafka custom 
parameters, which can be extended in the future
 private Map customProperties = Maps.newHashMap();
 private Map convertedCustomProperties = Maps.newHashMap();
+private Map> convertedRackCustomProperties = 
Maps.newHashMap();

Review Comment:
   why do you use a new property "convertedRackCustomProperties" to store rack 
info rather than in convertedCustomProperties?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [fix](lookup)compatible with lookup timestamp types [doris-flink-connector]

2024-03-12 Thread via GitHub


vinlee19 opened a new pull request, #341:
URL: https://github.com/apache/doris-flink-connector/pull/341

   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](planner) remove input slot for aggregate slot which is not materialized [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32092:
URL: https://github.com/apache/doris/pull/32092#issuecomment-1990918396

   
   Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Load test result on commit c325499b1b02c504cf6414c4b1d94da1b0eaf894 with 
default session variables
   Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
   Stream load orc:  59 seconds loaded 1101869774 Bytes, about 17 MB/s
   Stream load parquet:  31 seconds loaded 861443392 Bytes, about 26 MB/s
   Insert into select:   17.9 seconds inserted 1000 Rows, about 558K 
ops/s
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhancement](Nereids) Optimize expression [doris]

2024-03-12 Thread via GitHub


924060929 commented on PR #32067:
URL: https://github.com/apache/doris/pull/32067#issuecomment-1990918902

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [Fix](trino-connector) Fix the bug of query wrongs when using trino-connector [doris]

2024-03-12 Thread via GitHub


BePPPower opened a new pull request, #32100:
URL: https://github.com/apache/doris/pull/32100

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Fix](trino-connector) Fix the bug of query wrongs when using trino-connector [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32100:
URL: https://github.com/apache/doris/pull/32100#issuecomment-1990922257

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](routine-load)add rack tag into fe & kafka routine load(#31531) [doris]

2024-03-12 Thread via GitHub


HHoflittlefish777 commented on PR #32098:
URL: https://github.com/apache/doris/pull/32098#issuecomment-1990922774

   Is rack-aware work when use routine load after this pr? Can you add more 
test to verify it?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature-wip](Cloud) Add property storage vault for OlapTable [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #31628:
URL: https://github.com/apache/doris/pull/31628#issuecomment-1990923837

   
   
   TPC-H: Total hot run time: 36920 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 43c0b6563d19d69d4fb769f3a5bc168dbd33ef6c, 
data reload: false
   
   -- Round 1 --
   q1   17629   428141394139
   q2   2021156 143 143
   q3   10570   926 902 902
   q4   4657684 706 684
   q5   7598288328862883
   q6   190 123 125 123
   q7   1189845 816 816
   q8   9550212520332033
   q9   7348652565086508
   q10  8167252725022502
   q11  422 238 216 216
   q12  788 307 284 284
   q13  17962   303929772977
   q14  286 266 252 252
   q15  504 455 450 450
   q16  490 412 389 389
   q17  943 559 574 559
   q18  6731591358255825
   q19  1592145114851451
   q20  554 288 280 280
   q21  7469319832723198
   q22  811 306 306 306
   Total cold run time: 107471 ms
   Total hot run time: 36920 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4137407140984071
   q2   325 223 222 222
   q3   2947288628492849
   q4   1833155815611558
   q5   5249528852385238
   q6   197 120 117 117
   q7   2247183418501834
   q8   3156330232993299
   q9   8614859886058598
   q10  6205363336423633
   q11  536 441 441 441
   q12  737 554 549 549
   q13  10230   290028482848
   q14  276 250 255 250
   q15  479 449 449 449
   q16  465 409 429 409
   q17  1762149514541454
   q18  7734722170667066
   q19  5615152515371525
   q20  1955171217151712
   q21  4873474246624662
   q22  520 458 467 458
   Total cold run time: 70092 ms
   Total hot run time: 53242 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement](binlog)Support inverted index in CCR (#31743) [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32101:
URL: https://github.com/apache/doris/pull/32101#issuecomment-1990927962

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](ui) fix database cannot be chosen bug [doris]

2024-03-12 Thread via GitHub


morningman commented on PR #32091:
URL: https://github.com/apache/doris/pull/32091#issuecomment-1990928669

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](routine-load)add rack tag into fe & kafka routine load(#31531) [doris]

2024-03-12 Thread via GitHub


flashmouse commented on code in PR #32098:
URL: https://github.com/apache/doris/pull/32098#discussion_r1520961237


##
fe/fe-core/src/main/java/org/apache/doris/load/routineload/KafkaRoutineLoadJob.java:
##
@@ -93,6 +95,7 @@ public class KafkaRoutineLoadJob extends RoutineLoadJob {
 // kafka properties ,property prefix will be mapped to kafka custom 
parameters, which can be extended in the future
 private Map customProperties = Maps.newHashMap();
 private Map convertedCustomProperties = Maps.newHashMap();
+private Map> convertedRackCustomProperties = 
Maps.newHashMap();

Review Comment:
   a new parameter **ConsumerConfig.CLIENT_RACK_CONFIG**(aka client.rack, this 
is a standard KafkaClient consumer config to specify the rack of the 
KafkaConsumer)  introduced into convertedCustomProperties. so the value of each 
BE may not the same.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [improvement](binlog)Support inverted index in CCR (#31743) [doris]

2024-03-12 Thread via GitHub


qidaye opened a new pull request, #32101:
URL: https://github.com/apache/doris/pull/32101

   ## Proposed changes
   
   pick from #31743 
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement](binlog)Support inverted index in CCR (#31743) [doris]

2024-03-12 Thread via GitHub


qidaye commented on PR #32101:
URL: https://github.com/apache/doris/pull/32101#issuecomment-1990928166

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](ui) fix database cannot be chosen bug [doris]

2024-03-12 Thread via GitHub


github-actions[bot] commented on PR #32091:
URL: https://github.com/apache/doris/pull/32091#issuecomment-1990929386

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](ui) fix database cannot be chosen bug [doris]

2024-03-12 Thread via GitHub


github-actions[bot] commented on PR #32091:
URL: https://github.com/apache/doris/pull/32091#issuecomment-1990929430

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] Pick fix mow index compaction and revert comparePostings [doris-thirdparty]

2024-03-12 Thread via GitHub


qidaye merged PR #199:
URL: https://github.com/apache/doris-thirdparty/pull/199


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [fix](merge-cloud) Keep RecyclePartitionInfo compatible with the cloud mode [doris]

2024-03-12 Thread via GitHub


w41ter opened a new pull request, #32102:
URL: https://github.com/apache/doris/pull/32102

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris-thirdparty) branch clucene updated: Pick fix mow index compaction and revert comparePostings (#199)

2024-03-12 Thread jianliangqi
This is an automated email from the ASF dual-hosted git repository.

jianliangqi pushed a commit to branch clucene
in repository https://gitbox.apache.org/repos/asf/doris-thirdparty.git


The following commit(s) were added to refs/heads/clucene by this push:
 new ef95e67ae3 Pick fix mow index compaction and revert comparePostings 
(#199)
ef95e67ae3 is described below

commit ef95e67ae3123409f006072194f742a079603159
Author: qiye 
AuthorDate: Tue Mar 12 15:18:39 2024 +0800

Pick fix mow index compaction and revert comparePostings (#199)

* [fix](write)revert comparePostings due to write core (#195)

revert https://github.com/apache/doris-thirdparty/pull/156

* [fix](index compaction)Remove INT32_MAX out of destPostingQueues (#198)
---
 src/core/CLucene/index/IndexWriter.cpp | 101 +
 src/core/CLucene/index/SDocumentWriter.cpp |  26 +---
 2 files changed, 77 insertions(+), 50 deletions(-)

diff --git a/src/core/CLucene/index/IndexWriter.cpp 
b/src/core/CLucene/index/IndexWriter.cpp
index e30abf5107..0d770182ba 100644
--- a/src/core/CLucene/index/IndexWriter.cpp
+++ b/src/core/CLucene/index/IndexWriter.cpp
@@ -41,6 +41,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #define FINALLY_CLOSE_OUTPUT(x)   \
 try { \
@@ -1667,34 +1668,47 @@ void IndexWriter::mergeTerms(bool hasProx) {
 std::vector> freqBuffers(numDestIndexes);
 auto destPostingQueues = _CLNEW postingQueue(matchSize);
 std::vector destDocs(matchSize);
-for (int i = 0; i < matchSize; ++i) {
-smi = match[i];
-TermPositions *postings = smi->getPositions();
 
-postings->seek(smi->termEnum);
-if (postings->next()) {
+auto processPostings = [&](TermPositions* postings, DestDoc* destDoc, 
int srcIdx) {
+while (postings->next()) {
 int srcDoc = postings->doc();
 std::pair p = 
_trans_vec[smi->readerIndex][srcDoc];
-uint32_t destIdx = p.first;
-uint32_t destDocId = p.second;
-
-destDocs[i].srcIdx = i;
-destDocs[i].destIdx = destIdx;
-destDocs[i].destDocId = destDocId;
+destDoc->destIdx = p.first;
+destDoc->destDocId = p.second;
+destDoc->srcIdx = srcIdx;
+//  indicates current row not exist in 
Doris dest segment.
+// So we ignore this doc here.
+if (destDoc->destIdx == UINT32_MAX || destDoc->destDocId == 
UINT32_MAX) {
+if (infoStream != nullptr) {
+std::stringstream ss;
+ss << "skip UINT32_MAX, srcIdx: " << smi->readerIndex 
<< ", srcDoc: " << srcDoc
+   << ", destIdx: " << destDoc->destIdx << ", 
destDocId: " << destDoc->destDocId;
+message(ss.str());
+}
+continue;
+}
 
 if (hasProx) {
 int32_t freq = postings->freq();
-destDocs[i].destFreq = freq;
-destDocs[i].destPositions.resize(freq);
+destDoc->destFreq = freq;
+destDoc->destPositions.resize(freq);
 
 for (int32_t j = 0; j < freq; j++) {
 int32_t position = postings->nextPosition();
-destDocs[i].destPositions[j] = position;
+destDoc->destPositions[j] = position;
 }
 }
 
-destPostingQueues->put(&destDocs[i]);
+destPostingQueues->put(destDoc);
+break;
 }
+};
+
+for (int i = 0; i < matchSize; ++i) {
+smi = match[i];
+auto* postings = smi->getPositions();
+postings->seek(smi->termEnum);
+processPostings(postings, &destDocs[i], i);
 }
 
 auto encode = [](IndexOutput* out, std::vector& buffer, bool 
isDoc) {
@@ -1719,12 +1733,27 @@ void IndexWriter::mergeTerms(bool hasProx) {
 auto destDocId = destDoc->destDocId;
 auto destFreq = destDoc->destFreq;
 auto& descPositions = destDoc->destPositions;
-//  indicates current row not exist in 
Doris dest segment.
-// So we ignore this doc here.
-if (destIdx == UINT32_MAX || destDocId == UINT32_MAX) {
-continue;
+if (infoStream != nullptr) {
+for (int i = 0; i < _trans_vec.size(); ++i) {
+// find pair < destIdx, destDocId > in _trans_vec[i] 
to get the index of the pair
+auto it = std::find_if(_trans_vec[i].begin(), 
_trans_vec[i].end(),
+   [destIdx, destDocId](const 
std::pai

Re: [PR] [fix](merge-cloud) Keep RecyclePartitionInfo compatible with the cloud mode [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32102:
URL: https://github.com/apache/doris/pull/32102#issuecomment-1990930114

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] (enhance)(Cache) Control whether writing into file cache for query and load [doris]

2024-03-12 Thread via GitHub


gavinchou commented on code in PR #32099:
URL: https://github.com/apache/doris/pull/32099#discussion_r1520962375


##
be/src/olap/rowset/beta_rowset_reader.cpp:
##
@@ -232,6 +232,8 @@ Status 
BetaRowsetReader::get_segment_iterators(RowsetReaderContext* read_context
 _read_options.io_ctx.query_id = 
&_read_context->runtime_state->query_id();
 _read_options.io_ctx.read_file_cache =
 
_read_context->runtime_state->query_options().enable_file_cache;
+_read_options.io_ctx.is_disposable =
+
_read_context->runtime_state->query_options().disable_file_cache;

Review Comment:
   reuse `_read_context->runtime_state->query_options().enable_file_cache`
   because `disable_file_cache` is deprecated.
   
   BTW remove `disable_file_cache` by renaming it to 
`DEPRECATED_disable_file_cache`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](merge-cloud) Keep RecyclePartitionInfo compatible with the cloud mode [doris]

2024-03-12 Thread via GitHub


w41ter commented on PR #32102:
URL: https://github.com/apache/doris/pull/32102#issuecomment-1990930178

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] (enhance)(Cache) Control whether writing into file cache for query and load [doris]

2024-03-12 Thread via GitHub


gavinchou commented on code in PR #32099:
URL: https://github.com/apache/doris/pull/32099#discussion_r1520965317


##
be/src/olap/rowset/beta_rowset_writer.cpp:
##
@@ -207,8 +207,11 @@ Status 
BetaRowsetWriter::_load_noncompacted_segment(segment_v2::SegmentSharedPtr
 }
 auto path = BetaRowset::segment_file_path(_context.rowset_dir, 
_context.rowset_id, segment_id);
 io::FileReaderOptions reader_options {
-.cache_type = config::enable_file_cache ? 
io::FileCachePolicy::FILE_BLOCK_CACHE
-: 
io::FileCachePolicy::NO_CACHE,
+.cache_type =

Review Comment:
   is  enable_file_cache sufficient for the semantic?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](routine-load)add rack tag into fe & kafka routine load(#31531) [doris]

2024-03-12 Thread via GitHub


flashmouse commented on PR #32098:
URL: https://github.com/apache/doris/pull/32098#issuecomment-1990937862

   > Is rack-aware work when using routine load after this pr?
   
   after this pr, [KIP-392: Allow consumers to fetch from closest 
replica](https://cwiki.apache.org/confluence/display/KAFKA/KIP-392%3A+Allow+consumers+to+fetch+from+closest+replica)
 would be effective,  but we still could do some follow-up work to optimize FE 
assign partitions logic to get a better effect(ex [KIP-881: Rack-aware 
Partition Assignment for Kafka 
Consumers](https://cwiki.apache.org/confluence/display/KAFKA/KIP-881%3A+Rack-aware+Partition+Assignment+for+Kafka+Consumers))


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement](binlog)Support inverted index in CCR (#31743) [doris]

2024-03-12 Thread via GitHub


github-actions[bot] commented on PR #32101:
URL: https://github.com/apache/doris/pull/32101#issuecomment-1990937890

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](routine-load)add rack tag into fe & kafka routine load(#31531) [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32098:
URL: https://github.com/apache/doris/pull/32098#issuecomment-1990938496

   
   
   TPC-H: Total hot run time: 36906 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit afef16d0d72644a6eba18bd72ea571b8848721e4, 
data reload: false
   
   -- Round 1 --
   q1   17843   419241004100
   q2   2024157 146 146
   q3   11204   920 953 920
   q4   5267758 703 703
   q5   7877279828292798
   q6   187 129 126 126
   q7   1252854 836 836
   q8   9985213220872087
   q9   7697644964556449
   q10  8160250825542508
   q11  428 214 230 214
   q12  801 295 277 277
   q13  17973   308429912991
   q14  282 255 253 253
   q15  495 454 454 454
   q16  468 397 407 397
   q17  944 533 632 533
   q18  6654590559545905
   q19  1555145113621362
   q20  551 287 282 282
   q21  7374327332613261
   q22  796 308 304 304
   Total cold run time: 109817 ms
   Total hot run time: 36906 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4105405941624059
   q2   321 226 222 222
   q3   3002287128032803
   q4   1853157315671567
   q5   5189527452235223
   q6   193 115 115 115
   q7   2291185618431843
   q8   3163328833043288
   q9   8618855385788553
   q10  6136369036693669
   q11  534 440 441 440
   q12  725 571 571 571
   q13  11983   285228452845
   q14  273 245 241 241
   q15  481 456 438 438
   q16  448 398 424 398
   q17  1720149514831483
   q18  7603728670777077
   q19  1600151215471512
   q20  1885169817481698
   q21  4788477347374737
   q22  515 427 437 427
   Total cold run time: 67426 ms
   Total hot run time: 53209 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Fix](partition) Fix partition id eq 0 impact publish [doris]

2024-03-12 Thread via GitHub


deardeng commented on PR #32072:
URL: https://github.com/apache/doris/pull/32072#issuecomment-1990938875

   这个pr先关了,打桩弄出3副本,2副本partition id = 0的,测试insert。
   
   没用 这个pr修复,insert 之后是commit 状态,然后过5分钟后,变成visble
   
   
   用这个pr修复的,insert之后是publish状态。也就是说pr修复本问题,是可以的。
   
   
   但是感觉是头痛医头。本质上还是partition_id 和tabletinfo 的映射关系出了问题。
   
   以后别人若用这个关系,可能引入别的问题。
   
   pid=0的需要一个治本的方法。
   
   
   所以这个pr先关了
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature-wip](Cloud) Add property storage vault for OlapTable [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #31628:
URL: https://github.com/apache/doris/pull/31628#issuecomment-1990939037

   
   
   TPC-DS: Total hot run time: 186991 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 43c0b6563d19d69d4fb769f3a5bc168dbd33ef6c, 
data reload: false
   
   query1   945 369 341 341
   query2   7411212119131913
   query3   6708217 212 212
   query4   31029   20625   20621   20621
   query5   4351443 394 394
   query6   266 176 171 171
   query7   4625293 290 290
   query8   226 166 165 165
   query9   9959226222472247
   query10  450 236 244 236
   query11  16941   14166   14206   14166
   query12  132 90  82  82
   query13  1636413 415 413
   query14  10844   10941   10442   10442
   query15  252 209 200 200
   query16  8020279 267 267
   query17  1913572 568 568
   query18  2034292 280 280
   query19  212 169 155 155
   query20  92  90  83  83
   query21  204 129 124 124
   query22  4661445343834383
   query23  31879   31160   31145   31145
   query24  10753   284927972797
   query25  627 376 367 367
   query26  1193157 168 157
   query27  2403369 353 353
   query28  7240188518871885
   query29  880 630 649 630
   query30  305 154 151 151
   query31  990 722 734 722
   query32  91  59  59  59
   query33  775 259 263 259
   query34  1093476 479 476
   query35  828 606 619 606
   query36  998 904 870 870
   query37  113 75  75  75
   query38  3280325231363136
   query39  1423136614111366
   query40  216 115 112 112
   query41  49  54  48  48
   query42  100 96  99  96
   query43  505 449 451 449
   query44  1258733 698 698
   query45  279 270 256 256
   query46  1094675 679 675
   query47  1677162315731573
   query48  455 364 353 353
   query49  1124344 333 333
   query50  739 371 371 371
   query51  6696664565326532
   query52  111 98  90  90
   query53  341 276 283 276
   query54  315 241 252 241
   query55  82  77  79  77
   query56  249 256 231 231
   query57  1082100910341009
   query58  230 207 203 203
   query59  2820264926392639
   query60  260 268 257 257
   query61  115 113 110 110
   query62  594 401 391 391
   query63  304 278 283 278
   query64  5683380737673767
   query65  3070302629932993
   query66  852 330 312 312
   query67  15048   14618   14709   14618
   query68  8575511 521 511
   query69  630 383 376 376
   query70  1289113711241124
   query71  521 279 272 272
   query72  6551265625472547
   query73  749 313 317 313
   query74  7813656265606560
   query75  9388795281907952
   query76  5154902 973 902
   query77  675 258 252 252
   query78  10323   966097259660
   query79  13009   532 508 508
   query80  2447383 358 358
   query81  519 219 210 210
   query82  526 204 217 204
   query83  226 140 136 136
   query84  284 80  78  78
   query85  1295328 310 310
   query86  374 298 280 280
   query87  3421325732203220
   query88  5175224622612246
   query89  504 365 364 364
   query90  2013176 171 171
   query91  166 125 142 125
   query92  59  46  47  46
   query93  7286501 480 480
   query94  1310177 173 173
   query95  427 333 326 326
   query96  609 265 261 261
   query97  4406427942704270
   query98  229 214 213 213
   query99  1076759 748 748
   Total cold run time: 320159 ms
   Total hot run time: 186991 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to t

Re: [PR] [improve] Refactor block file cache and add more ut case [doris]

2024-03-12 Thread via GitHub


Lchangliang commented on code in PR #30039:
URL: https://github.com/apache/doris/pull/30039#discussion_r1520969517


##
be/src/common/config.cpp:
##
@@ -984,20 +984,12 @@ DEFINE_Bool(enable_file_cache, "false");
 // format: 
[{"path":"/path/to/file_cache","total_size":21474836480,"query_limit":10737418240}]
 // format: 
[{"path":"/path/to/file_cache","total_size":21474836480,"query_limit":10737418240},{"path":"/path/to/file_cache2","total_size":21474836480,"query_limit":10737418240}]
 DEFINE_String(file_cache_path, "");
-DEFINE_Int64(file_cache_max_file_segment_size, "4194304"); // 4MB
-// 4KB <= file_cache_max_file_segment_size <= 256MB
-DEFINE_Validator(file_cache_max_file_segment_size, [](const int64_t config) -> 
bool {
-return config >= 4096 && config <= 268435456;
-});
-DEFINE_Int64(file_cache_min_file_segment_size, "1048576"); // 1MB
-// 4KB <= file_cache_min_file_segment_size <= 256MB
-DEFINE_Validator(file_cache_min_file_segment_size, [](const int64_t config) -> 
bool {
-return config >= 4096 && config <= 268435456 &&
-   config <= config::file_cache_max_file_segment_size;
-});
+DEFINE_Int64(file_cache_each_block_size, "1048576"); // 1MB

Review Comment:
   Have a check in be/src/runtime/exec_env_init.cpp:init_file_cache_factory



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Fix](partition) Fix partition id eq 0 impact publish [doris]

2024-03-12 Thread via GitHub


deardeng closed pull request #32072: [Fix](partition) Fix partition id eq 0 
impact publish
URL: https://github.com/apache/doris/pull/32072


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [fix](function) incorrect result of eq_for_null [doris]

2024-03-12 Thread via GitHub


mrhhsg opened a new pull request, #32103:
URL: https://github.com/apache/doris/pull/32103

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](function) incorrect result of eq_for_null [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32103:
URL: https://github.com/apache/doris/pull/32103#issuecomment-1990940642

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](function) incorrect result of eq_for_null [doris]

2024-03-12 Thread via GitHub


mrhhsg commented on PR #32103:
URL: https://github.com/apache/doris/pull/32103#issuecomment-1990940852

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feat](Nereids): use table map to construct struct info [doris]

2024-03-12 Thread via GitHub


github-actions[bot] commented on PR #32058:
URL: https://github.com/apache/doris/pull/32058#issuecomment-1990942880

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature-wip](Cloud) Add property storage vault for OlapTable [doris]

2024-03-12 Thread via GitHub


ByteYue commented on PR #31628:
URL: https://github.com/apache/doris/pull/31628#issuecomment-1990945588

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [release](build script) fix build release bug when build multiple version on same machine [doris]

2024-03-12 Thread via GitHub


shuke987 opened a new pull request, #32104:
URL: https://github.com/apache/doris/pull/32104

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [release](build script) fix build release bug when build multiple version on same machine [doris]

2024-03-12 Thread via GitHub


shuke987 commented on PR #32104:
URL: https://github.com/apache/doris/pull/32104#issuecomment-1990945991

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](function) incorrect result of eq_for_null [doris]

2024-03-12 Thread via GitHub


github-actions[bot] commented on PR #32103:
URL: https://github.com/apache/doris/pull/32103#issuecomment-1990946176

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [release](build script) fix build release bug when build multiple version on same machine [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32104:
URL: https://github.com/apache/doris/pull/32104#issuecomment-1990945927

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](function) incorrect result of eq_for_null [doris]

2024-03-12 Thread via GitHub


github-actions[bot] commented on PR #32103:
URL: https://github.com/apache/doris/pull/32103#issuecomment-1990946225

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](fe) Optimize calculate load job num metric in FE [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #31952:
URL: https://github.com/apache/doris/pull/31952#issuecomment-1990946232

   
   
   TPC-H: Total hot run time: 36260 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit b0bd19f3abaf65240db248096cfc2cd1b8dafc80, 
data reload: false
   
   -- Round 1 --
   q1   17695   408540674067
   q2   2012146 145 145
   q3   10579   950 884 884
   q4   4652697 671 671
   q5   7569275327442744
   q6   185 123 125 123
   q7   1147822 802 802
   q8   9479202819651965
   q9   7173644264066406
   q10  8133246024732460
   q11  418 225 211 211
   q12  788 303 294 294
   q13  17963   297229662966
   q14  268 249 249 249
   q15  502 454 465 454
   q16  472 390 399 390
   q17  932 538 523 523
   q18  6509579657835783
   q19  1584141714621417
   q20  527 272 281 272
   q21  7365314432143144
   q22  807 303 290 290
   Total cold run time: 106759 ms
   Total hot run time: 36260 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4093406540904065
   q2   322 226 228 226
   q3   2982277527642764
   q4   1831151114851485
   q5   5201524751925192
   q6   193 121 117 117
   q7   2207184218191819
   q8   3140324132673241
   q9   8474850885418508
   q10  6066367936453645
   q11  529 454 451 451
   q12  726 541 528 528
   q13  11243   286028582858
   q14  273 266 258 258
   q15  487 452 446 446
   q16  455 416 407 407
   q17  1720147214611461
   q18  7632720270827082
   q19  6534149514971495
   q20  1913169617181696
   q21  4873469847844698
   q22  559 463 467 463
   Total cold run time: 71453 ms
   Total hot run time: 52905 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](merge-cloud) Keep PartitionInfo compatible with the cloud mode [doris]

2024-03-12 Thread via GitHub


github-actions[bot] commented on PR #32008:
URL: https://github.com/apache/doris/pull/32008#issuecomment-1990947160

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](merge-cloud) Keep RecyclePartitionInfo compatible with the cloud mode [doris]

2024-03-12 Thread via GitHub


github-actions[bot] commented on PR #32102:
URL: https://github.com/apache/doris/pull/32102#issuecomment-1990947323

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [bugfix](becore) be will core when stop because the map is modified during iterator [doris]

2024-03-12 Thread via GitHub


yiguolei opened a new pull request, #32105:
URL: https://github.com/apache/doris/pull/32105

   ## Proposed changes
   =
   ==9396==ERROR: AddressSanitizer: heap-use-after-free on address 
0x6030052f26e0 at pc 0x563a94549f30 bp 0x7ffd8ec8eb90 sp 0x7ffd8ec8eb88
   READ of size 8 at 0x6030052f26e0 thread T0
   #0 0x563a94549f2f in std::__shared_ptr::get() const 
/var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1291:16
   #1 0x563a94549ee6 in std::__shared_ptr_access::_M_get() const 
/var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:990:66
   #2 0x563a94549166 in std::__shared_ptr_access::operator->() const 
/var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:984:9
   #3 0x563aa975e2f9 in 
doris::vectorized::VDataStreamMgr::~VDataStreamMgr() 
/home/zcp/repo_center/doris_master/doris/be/src/vec/runtime/vdata_stream_mgr.cpp:46:9
   #4 0x563a7b426e4f in doris::ExecEnv::destroy() 
/home/zcp/repo_center/doris_master/doris/be/src/runtime/exec_env_init.cpp:630:5
   #5 0x563a77e67597 in main 
/home/zcp/repo_center/doris_master/doris/be/src/service/doris_main.cpp:597:15
   #6 0x7f7a38f45082 in __libc_start_main 
/build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
   #7 0x563a77d8a029 in _start 
(/mnt/hdd01/ci/master-deploy/be/lib/doris_be+0x25e4f029) (BuildId: 
ad37d7448f997131)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [bugfix](becore) be will core when stop because the map is modified during iterator [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32105:
URL: https://github.com/apache/doris/pull/32105#issuecomment-1990947949

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] (enhance)(Cache) Control whether writing into file cache for query and load [doris]

2024-03-12 Thread via GitHub


ByteYue commented on code in PR #32099:
URL: https://github.com/apache/doris/pull/32099#discussion_r1520974438


##
be/src/olap/rowset/beta_rowset_reader.cpp:
##
@@ -232,6 +232,8 @@ Status 
BetaRowsetReader::get_segment_iterators(RowsetReaderContext* read_context
 _read_options.io_ctx.query_id = 
&_read_context->runtime_state->query_id();
 _read_options.io_ctx.read_file_cache =
 
_read_context->runtime_state->query_options().enable_file_cache;
+_read_options.io_ctx.is_disposable =
+
_read_context->runtime_state->query_options().disable_file_cache;

Review Comment:
   目前disable_file_cache这个名字在社区还不能一下子废除,在导入路径是否写cache的时候会用到



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Fix](partition) Fix partition id eq 0 impact publish [doris]

2024-03-12 Thread via GitHub


deardeng commented on PR #32072:
URL: https://github.com/apache/doris/pull/32072#issuecomment-1990948847

   用此pr修了,是这样的
   
   
![img_v3_028t_3bc3ccfb-04fb-43a7-8f3c-f080eb92b16g](https://github.com/apache/doris/assets/3887565/6f49f67d-5983-4c5c-8d24-9e5ff09a732c)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Fix](partition) Fix partition id eq 0 impact publish [doris]

2024-03-12 Thread via GitHub


deardeng commented on PR #32072:
URL: https://github.com/apache/doris/pull/32072#issuecomment-1990948091

   
![img_v3_028t_79349bbc-cdc2-45aa-8a9f-97f134d117cg](https://github.com/apache/doris/assets/3887565/405930d6-0b0f-4f07-b05d-ad3628520953)
   
   没用那个pr修的时候,3副本2副本(pid=0),insert是这样的


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [bugfix](becore) be will core when stop because the map is modified during iterator [doris]

2024-03-12 Thread via GitHub


yiguolei commented on PR #32105:
URL: https://github.com/apache/doris/pull/32105#issuecomment-1990948530

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature-wip](Cloud) Add property storage vault for OlapTable [doris]

2024-03-12 Thread via GitHub


github-actions[bot] commented on PR #31628:
URL: https://github.com/apache/doris/pull/31628#issuecomment-1990950663

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature-wip](Cloud) Add property storage vault for OlapTable [doris]

2024-03-12 Thread via GitHub


github-actions[bot] commented on PR #31628:
URL: https://github.com/apache/doris/pull/31628#issuecomment-1990950700

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improvement](binlog)Support inverted index in CCR (#31743) [doris]

2024-03-12 Thread via GitHub


yiguolei merged PR #32101:
URL: https://github.com/apache/doris/pull/32101


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch branch-2.1 updated: [improvement](binlog)Support inverted index in CCR (#31743) (#32101)

2024-03-12 Thread yiguolei
This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/branch-2.1 by this push:
 new 5f125bbaaa1 [improvement](binlog)Support inverted index in CCR 
(#31743) (#32101)
5f125bbaaa1 is described below

commit 5f125bbaaa1ddc0b8b98a2043c9c05b70a8c2472
Author: qiye 
AuthorDate: Tue Mar 12 15:34:08 2024 +0800

[improvement](binlog)Support inverted index in CCR (#31743) (#32101)
---
 be/src/http/action/download_binlog_action.cpp |  39 
 be/src/olap/rowset/beta_rowset.cpp|  44 -
 be/src/olap/snapshot_manager.cpp  |  53 ++-
 be/src/olap/tablet.cpp|  26 -
 be/src/olap/tablet.h  |   5 +
 be/src/olap/tablet_manager.cpp|  17 +++-
 be/src/olap/task/engine_clone_task.cpp|  14 ++-
 be/src/service/backend_service.cpp| 132 --
 8 files changed, 308 insertions(+), 22 deletions(-)

diff --git a/be/src/http/action/download_binlog_action.cpp 
b/be/src/http/action/download_binlog_action.cpp
index 697512b2a30..dbe2880d3b4 100644
--- a/be/src/http/action/download_binlog_action.cpp
+++ b/be/src/http/action/download_binlog_action.cpp
@@ -47,6 +47,7 @@ const std::string kTabletIdParameter = "tablet_id";
 const std::string kBinlogVersionParameter = "binlog_version";
 const std::string kRowsetIdParameter = "rowset_id";
 const std::string kSegmentIndexParameter = "segment_index";
+const std::string kSegmentIndexIdParameter = "segment_index_id";
 
 // get http param, if no value throw exception
 const auto& get_http_param(HttpRequest* req, const std::string& param_name) {
@@ -130,6 +131,42 @@ void handle_get_segment_file(HttpRequest* req, 
bufferevent_rate_limit_group* rat
 do_file_response(segment_file_path, req, rate_limit_group);
 }
 
+/// handle get segment index file, need tablet_id, rowset_id, segment_index && 
segment_index_id
+void handle_get_segment_index_file(HttpRequest* req,
+   bufferevent_rate_limit_group* 
rate_limit_group) {
+// Step 1: get download file path
+std::string segment_index_file_path;
+try {
+const auto& tablet_id = get_http_param(req, kTabletIdParameter);
+auto tablet = get_tablet(tablet_id);
+const auto& rowset_id = get_http_param(req, kRowsetIdParameter);
+const auto& segment_index = get_http_param(req, 
kSegmentIndexParameter);
+const auto& segment_index_id = req->param(kSegmentIndexIdParameter);
+segment_index_file_path =
+tablet->get_segment_index_filepath(rowset_id, segment_index, 
segment_index_id);
+} catch (const std::exception& e) {
+HttpChannel::send_reply(req, HttpStatus::INTERNAL_SERVER_ERROR, 
e.what());
+LOG(WARNING) << "get download file path failed, error: " << e.what();
+return;
+}
+
+// Step 2: handle download
+// check file exists
+bool exists = false;
+Status status = 
io::global_local_filesystem()->exists(segment_index_file_path, &exists);
+if (!status.ok()) {
+HttpChannel::send_reply(req, HttpStatus::INTERNAL_SERVER_ERROR, 
status.to_string());
+LOG(WARNING) << "check file exists failed, error: " << 
status.to_string();
+return;
+}
+if (!exists) {
+HttpChannel::send_reply(req, HttpStatus::NOT_FOUND, "file not exist.");
+LOG(WARNING) << "file not exist, file path: " << 
segment_index_file_path;
+return;
+}
+do_file_response(segment_index_file_path, req, rate_limit_group);
+}
+
 void handle_get_rowset_meta(HttpRequest* req) {
 try {
 const auto& tablet_id = get_http_param(req, kTabletIdParameter);
@@ -183,6 +220,8 @@ void DownloadBinlogAction::handle(HttpRequest* req) {
 handle_get_binlog_info(req);
 } else if (method == "get_segment_file") {
 handle_get_segment_file(req, _rate_limit_group.get());
+} else if (method == "get_segment_index_file") {
+handle_get_segment_index_file(req, _rate_limit_group.get());
 } else if (method == "get_rowset_meta") {
 handle_get_rowset_meta(req);
 } else {
diff --git a/be/src/olap/rowset/beta_rowset.cpp 
b/be/src/olap/rowset/beta_rowset.cpp
index e3f28726c4d..58e495be222 100644
--- a/be/src/olap/rowset/beta_rowset.cpp
+++ b/be/src/olap/rowset/beta_rowset.cpp
@@ -439,7 +439,7 @@ Status BetaRowset::add_to_binlog() {
 if (fs->type() != io::FileSystemType::LOCAL) {
 return Status::InternalError("should be local file system");
 }
-io::LocalFileSystem* local_fs = 
static_cast(fs.get());
+auto* local_fs = static_cast(fs.get());
 
 // all segments are in the same directory, so cache binlog_dir without 
multi times check
 std::string binlog_dir;
@@ -447,6 +447,22 @@ Status BetaRowset::add_to_binlog() {
 auto segments

Re: [PR] [feature](routine-load)add rack tag into fe & kafka routine load(#31531) [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32098:
URL: https://github.com/apache/doris/pull/32098#issuecomment-199093

   
   
   TPC-DS: Total hot run time: 187436 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit afef16d0d72644a6eba18bd72ea571b8848721e4, 
data reload: false
   
   query1   912 361 352 352
   query2   7798200519661966
   query3   6701209 207 207
   query4   31120   20711   20707   20707
   query5   4421474 405 405
   query6   255 179 169 169
   query7   4627297 288 288
   query8   229 173 183 173
   query9   9973235723202320
   query10  455 234 242 234
   query11  17313   14129   14167   14129
   query12  130 85  108 85
   query13  1622419 421 419
   query14  11247   11239   10713   10713
   query15  279 218 195 195
   query16  8021264 262 262
   query17  1939559 547 547
   query18  2079298 287 287
   query19  288 154 151 151
   query20  94  85  85  85
   query21  204 132 129 129
   query22  4773450944684468
   query23  31852   30847   31339   30847
   query24  10688   284028292829
   query25  638 372 390 372
   query26  1410161 155 155
   query27  2999353 354 353
   query28  8023187419031874
   query29  888 663 632 632
   query30  303 153 150 150
   query31  967 700 736 700
   query32  95  57  60  57
   query33  774 263 246 246
   query34  1082482 484 482
   query35  829 626 619 619
   query36  997 879 854 854
   query37  124 76  76  76
   query38  3378318732123187
   query39  1438136513641364
   query40  216 110 109 109
   query41  51  47  48  47
   query42  105 94  97  94
   query43  481 437 445 437
   query44  1267740 712 712
   query45  280 255 260 255
   query46  1096679 712 679
   query47  1700161915981598
   query48  441 355 364 355
   query49  1131341 374 341
   query50  766 373 375 373
   query51  6574671965076507
   query52  116 95  90  90
   query53  370 280 276 276
   query54  309 232 245 232
   query55  81  81  80  80
   query56  256 226 236 226
   query57  1105103410191019
   query58  230 204 211 204
   query59  2749264926602649
   query60  266 292 280 280
   query61  122 109 111 109
   query62  612 386 394 386
   query63  296 268 272 268
   query64  6570370337913703
   query65  3089301230173012
   query66  1395314 321 314
   query67  15098   14936   14588   14588
   query68  8649531 548 531
   query69  613 378 370 370
   query70  1269119710721072
   query71  529 289 276 276
   query72  6634261824802480
   query73  730 325 316 316
   query74  8050660066016600
   query75  10018   807379707970
   query76  5021919 970 919
   query77  644 301 265 265
   query78  10456   976597259725
   query79  9449522 523 522
   query80  1570385 433 385
   query81  511 206 206 206
   query82  887 202 197 197
   query83  212 143 146 143
   query84  281 76  75  75
   query85  1453329 310 310
   query86  454 276 289 276
   query87  3381326932293229
   query88  4986234723422342
   query89  511 367 382 367
   query90  1978172 175 172
   query91  155 127 150 127
   query92  64  46  54  46
   query93  6846525 501 501
   query94  177 175 175
   query95  424 337 332 332
   query96  600 274 273 273
   query97  4419425842804258
   query98  235 210 203 203
   query99  1257742 747 742
   Total cold run time: 321134 ms
   Total hot run time: 187436 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to t

Re: [PR] [feature](merge-cloud) implement alter table property of cloud mode [doris]

2024-03-12 Thread via GitHub


dataroaring merged PR #31887:
URL: https://github.com/apache/doris/pull/31887


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch master updated (7f665f381ac -> 5985044be28)

2024-03-12 Thread dataroaring
This is an automated email from the ASF dual-hosted git repository.

dataroaring pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


from 7f665f381ac [docs](docs) Update release note 2.0.6 and readme of 
Master branch (#32087)
 add 5985044be28 [feature](merge-cloud) implement alter table property of 
cloud mode (#31887)

No new revisions were added by this update.

Summary of changes:
 cloud/src/meta-service/meta_service.cpp|   4 +
 .../main/java/org/apache/doris/common/Config.java  |   3 +
 .../main/java/org/apache/doris/alter/Alter.java|  30 +--
 .../apache/doris/alter/SchemaChangeHandler.java|  18 ++
 .../cloud/alter/CloudSchemaChangeHandler.java  | 256 +
 gensrc/proto/cloud.proto   |   2 +
 gensrc/proto/olap_file.proto   |   6 +-
 7 files changed, 289 insertions(+), 30 deletions(-)
 create mode 100644 
fe/fe-core/src/main/java/org/apache/doris/cloud/alter/CloudSchemaChangeHandler.java


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [release](build script) fix build release bug when build multiple version on same machine [doris]

2024-03-12 Thread via GitHub


github-actions[bot] commented on PR #32104:
URL: https://github.com/apache/doris/pull/32104#issuecomment-1990959719

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [release](build script) fix build release bug when build multiple version on same machine [doris]

2024-03-12 Thread via GitHub


github-actions[bot] commented on PR #32104:
URL: https://github.com/apache/doris/pull/32104#issuecomment-1990959750

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [release](build script) fix build release bug when build multiple version on same machine [doris]

2024-03-12 Thread via GitHub


yiguolei merged PR #32104:
URL: https://github.com/apache/doris/pull/32104


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch master updated (5985044be28 -> 6c6466b465d)

2024-03-12 Thread yiguolei
This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


from 5985044be28 [feature](merge-cloud) implement alter table property of 
cloud mode (#31887)
 add 6c6466b465d [release](build script) fix build release bug when build 
multiple version on same machine #32104

No new revisions were added by this update.

Summary of changes:
 build-for-release.sh | 1 +
 1 file changed, 1 insertion(+)


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](fe) Optimize calculate load job num metric in FE [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #31952:
URL: https://github.com/apache/doris/pull/31952#issuecomment-1990960965

   
   
   TPC-DS: Total hot run time: 186493 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit b0bd19f3abaf65240db248096cfc2cd1b8dafc80, 
data reload: false
   
   query1   924 372 346 346
   query2   7410199419241924
   query3   6709213 208 208
   query4   31365   20765   20664   20664
   query5   4420412 390 390
   query6   255 169 162 162
   query7   4619302 284 284
   query8   224 168 169 168
   query9   9298229722832283
   query10  452 235 256 235
   query11  17141   14145   14189   14145
   query12  139 93  81  81
   query13  1628409 415 409
   query14  11381   10584   10438   10438
   query15  260 204 193 193
   query16  7977271 257 257
   query17  1917566 529 529
   query18  2067282 279 279
   query19  225 157 151 151
   query20  89  84  86  84
   query21  201 136 124 124
   query22  4677446044114411
   query23  31656   31182   31266   31182
   query24  10790   288127842784
   query25  624 371 382 371
   query26  1167160 163 160
   query27  2526357 352 352
   query28  7602191518931893
   query29  878 637 621 621
   query30  303 148 149 148
   query31  980 713 729 713
   query32  91  62  54  54
   query33  767 254 254 254
   query34  1083459 462 459
   query35  825 593 595 593
   query36  1000864 863 863
   query37  121 75  76  75
   query38  3266315431513151
   query39  1396138213561356
   query40  208 112 110 110
   query41  50  46  46  46
   query42  101 94  97  94
   query43  485 457 460 457
   query44  1255724 688 688
   query45  266 259 258 258
   query46  1084680 679 679
   query47  1688157416071574
   query48  458 359 349 349
   query49  1137341 326 326
   query50  757 375 373 373
   query51  6704655366216553
   query52  111 89  96  89
   query53  339 269 273 269
   query54  306 235 239 235
   query55  82  78  80  78
   query56  242 248 230 230
   query57  1097100010031000
   query58  237 199 202 199
   query59  2620261625112511
   query60  269 267 245 245
   query61  110 111 110 110
   query62  594 405 388 388
   query63  296 271 279 271
   query64  5551378338473783
   query65  3055301230143012
   query66  871 322 320 320
   query67  14723   14413   14322   14322
   query68  8621520 533 520
   query69  630 378 366 366
   query70  1259108911171089
   query71  518 271 279 271
   query72  6598263524982498
   query73  732 312 318 312
   query74  7989653966186539
   query75  9792808178737873
   query76  5155893 1006893
   query77  689 268 251 251
   query78  10349   964696449644
   query79  9696514 506 506
   query80  1635393 369 369
   query81  521 208 210 208
   query82  493 200 197 197
   query83  220 141 148 141
   query84  284 76  79  76
   query85  1181348 313 313
   query86  361 286 294 286
   query87  3417329632213221
   query88  4787236823522352
   query89  489 373 359 359
   query90  2071174 174 174
   query91  160 130 136 130
   query92  67  47  49  47
   query93  6722509 484 484
   query94  1295185 184 184
   query95  440 333 332 332
   query96  611 276 280 276
   query97  4432426942624262
   query98  241 213 211 211
   query99  1034746 750 746
   Total cold run time: 315638 ms
   Total hot run time: 186493 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to t

Re: [PR] [feat](Nereids): use table map to construct struct info [doris]

2024-03-12 Thread via GitHub


github-actions[bot] commented on PR #32058:
URL: https://github.com/apache/doris/pull/32058#issuecomment-1990961289

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [enhancement](Nereids) refine and speedup analyzer [doris]

2024-03-12 Thread via GitHub


924060929 opened a new pull request, #32106:
URL: https://github.com/apache/doris/pull/32106

   ## Proposed changes
   cherry from #31792
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhancement](Nereids) refine and speedup analyzer [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32106:
URL: https://github.com/apache/doris/pull/32106#issuecomment-1990962008

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](routine-load)add rack tag into fe & kafka routine load(#31531) [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32098:
URL: https://github.com/apache/doris/pull/32098#issuecomment-1990962153

   
   
   ClickBench: Total hot run time: 30 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit afef16d0d72644a6eba18bd72ea571b8848721e4, 
data reload: false
   
   query1   0.040.040.03
   query2   0.060.030.03
   query3   0.230.060.06
   query4   1.670.100.10
   query5   0.500.520.51
   query6   1.180.600.60
   query7   0.020.010.01
   query8   0.030.030.02
   query9   0.560.500.50
   query10  0.540.530.54
   query11  0.110.090.10
   query12  0.110.090.09
   query13  0.560.570.57
   query14  0.760.750.73
   query15  0.800.770.78
   query16  0.390.400.39
   query17  1.031.020.99
   query18  0.190.270.22
   query19  1.841.831.79
   query20  0.020.010.01
   query21  15.45   0.540.55
   query22  2.591.761.61
   query23  17.08   0.730.68
   query24  4.271.631.06
   query25  0.280.180.03
   query26  0.620.130.13
   query27  0.050.030.03
   query28  10.72   0.850.86
   query29  12.66   3.173.03
   query30  0.660.620.54
   query31  2.800.320.33
   query32  3.410.450.45
   query33  2.792.862.77
   query34  16.08   4.304.38
   query35  4.334.344.30
   query36  1.071.021.03
   query37  0.080.050.05
   query38  0.050.030.03
   query39  0.020.020.02
   query40  0.170.140.16
   query41  0.080.020.02
   query42  0.030.020.02
   query43  0.020.020.02
   Total cold run time: 105.95 s
   Total hot run time: 30 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [fix](nereids)EliminateSemiJoin should consider empty table [doris]

2024-03-12 Thread via GitHub


starocean999 opened a new pull request, #32107:
URL: https://github.com/apache/doris/pull/32107

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](nereids)EliminateSemiJoin should consider empty table [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32107:
URL: https://github.com/apache/doris/pull/32107#issuecomment-1990965974

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](nereids)EliminateSemiJoin should consider empty table [doris]

2024-03-12 Thread via GitHub


starocean999 commented on PR #32107:
URL: https://github.com/apache/doris/pull/32107#issuecomment-1990966108

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](routine-load)add rack tag into fe & kafka routine load(#31531) [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32098:
URL: https://github.com/apache/doris/pull/32098#issuecomment-1990966222

   
   Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Load test result on commit afef16d0d72644a6eba18bd72ea571b8848721e4 with 
default session variables
   Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
   Stream load orc:  58 seconds loaded 1101869774 Bytes, about 18 MB/s
   Stream load parquet:  31 seconds loaded 861443392 Bytes, about 26 MB/s
   Insert into select:   17.3 seconds inserted 1000 Rows, about 578K 
ops/s
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [I] [Feature] Doris support Arrow Flight SQL protocol [doris]

2024-03-12 Thread via GitHub


jpohanka commented on issue #25514:
URL: https://github.com/apache/doris/issues/25514#issuecomment-1990966353

   @xinyiZzz Thank you for creating this feature request and for implementing 
Arrow Flight in Doris. Currently (Doris 2.1.0) we have only data retrieval. Are 
you also planning to implement data ingestion via Arrow Flight?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](fe) Optimize calculate load job num metric in FE [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #31952:
URL: https://github.com/apache/doris/pull/31952#issuecomment-1990967928

   
   
   ClickBench: Total hot run time: 29.94 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit b0bd19f3abaf65240db248096cfc2cd1b8dafc80, 
data reload: false
   
   query1   0.040.030.03
   query2   0.060.030.03
   query3   0.230.060.07
   query4   1.650.090.09
   query5   0.510.500.50
   query6   1.220.610.63
   query7   0.020.020.01
   query8   0.040.030.03
   query9   0.560.500.50
   query10  0.540.560.53
   query11  0.110.090.09
   query12  0.110.090.09
   query13  0.570.570.56
   query14  0.740.750.75
   query15  0.800.780.78
   query16  0.400.400.40
   query17  0.981.011.04
   query18  0.190.250.22
   query19  1.881.671.81
   query20  0.020.010.01
   query21  15.40   0.550.55
   query22  2.524.242.02
   query23  17.09   0.850.75
   query24  3.291.920.46
   query25  0.110.200.03
   query26  0.710.130.14
   query27  0.020.030.05
   query28  11.22   0.880.84
   query29  12.53   3.133.21
   query30  0.640.610.60
   query31  2.800.340.36
   query32  3.350.420.44
   query33  2.792.762.85
   query34  16.09   4.324.29
   query35  4.314.324.35
   query36  1.081.021.01
   query37  0.070.060.05
   query38  0.040.030.03
   query39  0.030.020.02
   query40  0.170.130.14
   query41  0.070.020.02
   query42  0.030.020.02
   query43  0.030.030.02
   Total cold run time: 105.06 s
   Total hot run time: 29.94 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](merge-cloud) fix no cluster for common user [doris]

2024-03-12 Thread via GitHub


github-actions[bot] commented on PR #32097:
URL: https://github.com/apache/doris/pull/32097#issuecomment-1990968275

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [enhancement](test) exclude somecases not support in cloud mode [doris]

2024-03-12 Thread via GitHub


dataroaring opened a new pull request, #32108:
URL: https://github.com/apache/doris/pull/32108

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhancement](test) exclude somecases not support in cloud mode [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32108:
URL: https://github.com/apache/doris/pull/32108#issuecomment-1990968573

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhancement](test) exclude somecases not support in cloud mode [doris]

2024-03-12 Thread via GitHub


dataroaring commented on PR #32108:
URL: https://github.com/apache/doris/pull/32108#issuecomment-1990968660

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](merge-cloud) Keep RecyclePartitionInfo compatible with the cloud mode [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32102:
URL: https://github.com/apache/doris/pull/32102#issuecomment-1990968845

   
   
   TPC-H: Total hot run time: 36640 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit acfcd2f0fbc3d5f279a2a9ca26b2a8fdd5538b2c, 
data reload: false
   
   -- Round 1 --
   q1   17648   425841334133
   q2   2057161 153 153
   q3   10635   924 912 912
   q4   4657713 689 689
   q5   7594274127512741
   q6   184 126 126 126
   q7   1222817 804 804
   q8   9500207320532053
   q9   7803650564936493
   q10  8116248124582458
   q11  427 217 214 214
   q12  785 304 298 298
   q13  17959   303530023002
   q14  276 248 264 248
   q15  495 464 451 451
   q16  466 404 392 392
   q17  946 549 537 537
   q18  6749599257895789
   q19  1582134114471341
   q20  550 291 294 291
   q21  7343327332133213
   q22  799 304 302 302
   Total cold run time: 107793 ms
   Total hot run time: 36640 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4116408541074085
   q2   326 226 224 224
   q3   2979284628922846
   q4   1817159715381538
   q5   5236524352525243
   q6   198 115 118 115
   q7   2290187018201820
   q8   3149329032693269
   q9   8558853485328532
   q10  6162367236393639
   q11  538 441 452 441
   q12  739 527 536 527
   q13  13271   283928612839
   q14  280 245 255 245
   q15  481 442 449 442
   q16  453 410 407 407
   q17  1747149014431443
   q18  7786730970037003
   q19  5542153615501536
   q20  1908170717151707
   q21  4888467848164678
   q22  522 461 463 461
   Total cold run time: 72986 ms
   Total hot run time: 53040 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhancement](Nereids) refine and speedup analyzer [doris]

2024-03-12 Thread via GitHub


github-actions[bot] commented on PR #32106:
URL: https://github.com/apache/doris/pull/32106#issuecomment-1990971040

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](function) incorrect result of eq_for_null [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32103:
URL: https://github.com/apache/doris/pull/32103#issuecomment-1990971463

   TeamCity be ut coverage result:
Function Coverage: 34.98% (8556/24459) 
Line Coverage: 26.82% (69528/259264)
Region Coverage: 26.06% (36074/138420)
Branch Coverage: 23.03% (18437/80072)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/1c4fcdcd5f410df88e911f06a40363ede0bbf9fe_1c4fcdcd5f410df88e911f06a40363ede0bbf9fe/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](fe) Optimize calculate load job num metric in FE [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #31952:
URL: https://github.com/apache/doris/pull/31952#issuecomment-1990972154

   
   Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Load test result on commit b0bd19f3abaf65240db248096cfc2cd1b8dafc80 with 
default session variables
   Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
   Stream load orc:  59 seconds loaded 1101869774 Bytes, about 17 MB/s
   Stream load parquet:  33 seconds loaded 861443392 Bytes, about 24 MB/s
   Insert into select:   17.1 seconds inserted 1000 Rows, about 584K 
ops/s
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhancement](Nereids) Optimize expression [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32067:
URL: https://github.com/apache/doris/pull/32067#issuecomment-1990972748

   
   
   TPC-H: Total hot run time: 36333 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 3d75bded1c5c70216bbf403565539c521a234df1, 
data reload: false
   
   -- Round 1 --
   q1   17623   412141124112
   q2   2032151 144 144
   q3   10569   913 883 883
   q4   4650647 683 647
   q5   7608270627412706
   q6   185 127 129 127
   q7   1171814 797 797
   q8   9470198320431983
   q9   7370649564126412
   q10  8271250324712471
   q11  429 227 207 207
   q12  806 300 279 279
   q13  17959   301030043004
   q14  277 275 249 249
   q15  498 446 442 442
   q16  501 385 395 385
   q17  926 571 534 534
   q18  6671582457325732
   q19  1543147713921392
   q20  535 269 275 269
   q21  7342328832593259
   q22  796 299 326 299
   Total cold run time: 107232 ms
   Total hot run time: 36333 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4107404941214049
   q2   318 233 224 224
   q3   2952283528412835
   q4   1821146715251467
   q5   5153519352255193
   q6   192 115 120 115
   q7   2224186118261826
   q8   3135325132823251
   q9   8523850084828482
   q10  6161370336403640
   q11  536 445 435 435
   q12  736 548 562 548
   q13  13564   286128412841
   q14  272 248 251 248
   q15  482 448 437 437
   q16  447 389 397 389
   q17  1714148014701470
   q18  7629718870967096
   q19  5904156215571557
   q20  1904168816971688
   q21  4750458946244589
   q22  534 445 449 445
   Total cold run time: 73058 ms
   Total hot run time: 52825 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature-wip](Cloud) Add property storage vault for OlapTable [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #31628:
URL: https://github.com/apache/doris/pull/31628#issuecomment-1990974753

   TeamCity be ut coverage result:
Function Coverage: 34.97% (8556/24465) 
Line Coverage: 26.81% (69531/259305)
Region Coverage: 26.07% (36087/138436)
Branch Coverage: 23.03% (18443/80084)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/c851ce8263db2a5b7696ee173eaab768d8fd1f02_c851ce8263db2a5b7696ee173eaab768d8fd1f02/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhancement](Nereids) refine and speedup analyzer [doris]

2024-03-12 Thread via GitHub


yiguolei merged PR #32106:
URL: https://github.com/apache/doris/pull/32106


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feat](Nereids): use table map to construct struct info [doris]

2024-03-12 Thread via GitHub


jackwener merged PR #32058:
URL: https://github.com/apache/doris/pull/32058


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch master updated (6c6466b465d -> 5242a80d2bf)

2024-03-12 Thread jakevin
This is an automated email from the ASF dual-hosted git repository.

jakevin pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


from 6c6466b465d [release](build script) fix build release bug when build 
multiple version on same machine #32104
 add 5242a80d2bf [feat](Nereids): use table map to construct struct info 
(#32058)

No new revisions were added by this update.

Summary of changes:
 .../apache/doris/nereids/memo/StructInfoMap.java   | 46 -
 .../doris/nereids/memo/StructInfoMapTest.java  | 48 ++
 .../apache/doris/utframe/TestWithFeService.java| 25 +++
 3 files changed, 108 insertions(+), 11 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [bugfix](becore) be will core when stop because the map is modified during iterator [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32105:
URL: https://github.com/apache/doris/pull/32105#issuecomment-1990975714

   TeamCity be ut coverage result:
Function Coverage: 34.98% (8556/24459) 
Line Coverage: 26.82% (69523/259268)
Region Coverage: 26.06% (36072/138423)
Branch Coverage: 23.03% (18440/80074)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/c900b0973e9cc5ee662613864be5bc97e7297562_c900b0973e9cc5ee662613864be5bc97e7297562/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](routine-load)add rack tag into fe & kafka routine load(#31531) [doris]

2024-03-12 Thread via GitHub


HHoflittlefish777 commented on code in PR #32098:
URL: https://github.com/apache/doris/pull/32098#discussion_r1521001467


##
fe/fe-core/src/main/java/org/apache/doris/load/routineload/KafkaTaskInfo.java:
##
@@ -86,7 +87,11 @@ public TRoutineLoadTask createRoutineLoadTask() throws 
UserException {
 tKafkaLoadInfo.setTopic(routineLoadJob.getTopic());
 tKafkaLoadInfo.setBrokers(routineLoadJob.getBrokerList());
 tKafkaLoadInfo.setPartitionBeginOffset(partitionIdToOffset);
-
tKafkaLoadInfo.setProperties(routineLoadJob.getConvertedCustomProperties());
+Backend backend = Env.getCurrentSystemInfo().getBackend(beId);
+if (backend == null) {
+throw new UserException("failed to get be:" + beId + " rack 
because not exist");

Review Comment:
   If backend is null, can we choose other one?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [build]( build for release ) fix build bug when build multiple release version on same machine [doris]

2024-03-12 Thread via GitHub


shuke987 opened a new pull request, #32109:
URL: https://github.com/apache/doris/pull/32109

   …sion on same machine #32104
   
   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [build]( build for release ) fix build bug when build multiple release version on same machine [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32109:
URL: https://github.com/apache/doris/pull/32109#issuecomment-1990980764

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [build]( build for release ) fix build bug when build multiple release version on same machine [doris]

2024-03-12 Thread via GitHub


shuke987 commented on PR #32109:
URL: https://github.com/apache/doris/pull/32109#issuecomment-1990981165

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](routine-load)add rack tag into fe & kafka routine load(#31531) [doris]

2024-03-12 Thread via GitHub


HHoflittlefish777 commented on code in PR #32098:
URL: https://github.com/apache/doris/pull/32098#discussion_r1521002433


##
fe/fe-core/src/main/java/org/apache/doris/load/routineload/KafkaRoutineLoadJob.java:
##
@@ -741,7 +750,8 @@ public boolean hasMoreDataToConsume(UUID taskId, 
Map partitionIdT
 // all offsets to be consumed are newer than offsets in 
cachedPartitionWithLatestOffsets,
 // maybe the cached offset is out-of-date, fetch from kafka server 
again
 List> tmp = KafkaUtil.getLatestOffsets(id, 
taskId, getBrokerList(),
-getTopic(), getConvertedCustomProperties(), 
Lists.newArrayList(partitionIdToOffset.keySet()));
+getTopic(), 
getConvertedCustomProperties(Tag.VALUE_DEFAULT_TAG),

Review Comment:
   Is always Tag.VALUE_DEFAULT_TAG?



##
fe/fe-core/src/main/java/org/apache/doris/load/routineload/KafkaRoutineLoadJob.java:
##
@@ -741,7 +750,8 @@ public boolean hasMoreDataToConsume(UUID taskId, 
Map partitionIdT
 // all offsets to be consumed are newer than offsets in 
cachedPartitionWithLatestOffsets,
 // maybe the cached offset is out-of-date, fetch from kafka server 
again
 List> tmp = KafkaUtil.getLatestOffsets(id, 
taskId, getBrokerList(),
-getTopic(), getConvertedCustomProperties(), 
Lists.newArrayList(partitionIdToOffset.keySet()));
+getTopic(), 
getConvertedCustomProperties(Tag.VALUE_DEFAULT_TAG),

Review Comment:
   Is it always Tag.VALUE_DEFAULT_TAG?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](merge-cloud) fix no cluster for common user [doris]

2024-03-12 Thread via GitHub


liaoxin01 commented on PR #32097:
URL: https://github.com/apache/doris/pull/32097#issuecomment-1990982753

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch branch-2.1 updated (86bddefe6ba -> 5f125bbaaa1)

2024-03-12 Thread yiguolei
This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a change to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git


 discard 86bddefe6ba [enhancement](Nereids) refine and speedup analyzer 
(#31792) (#32106)

This update removed existing revisions from the reference, leaving the
reference pointing at a previous point in the repository history.

 * -- * -- N   refs/heads/branch-2.1 (5f125bbaaa1)
\
 O -- O -- O   (86bddefe6ba)

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

No new revisions were added by this update.

Summary of changes:
 be/src/clucene |2 +-
 docs/en/docs/admin-manual/config/fe-config.md  | 5471 +---
 docs/en/docs/advanced/variables.md | 2945 ++-
 docs/zh-CN/docs/admin-manual/config/fe-config.md   | 5457 +--
 docs/zh-CN/docs/advanced/variables.md  | 2766 ++
 .../org/apache/doris/catalog/FunctionRegistry.java |   22 +-
 .../apache/doris/catalog/FunctionSignature.java|3 +-
 .../org/apache/doris/nereids/CascadesContext.java  |9 -
 .../doris/nereids/analyzer/ComplexDataType.java|   22 -
 .../apache/doris/nereids/analyzer/MappingSlot.java |  115 -
 .../org/apache/doris/nereids/analyzer/Scope.java   |   37 +-
 .../nereids/jobs/rewrite/PlanTreeRewriteJob.java   |   18 +-
 .../rules/analysis/AvgDistinctToSumDivCount.java   |2 +-
 .../nereids/rules/analysis/BindExpression.java | 1223 ++---
 .../nereids/rules/analysis/BindSlotWithPaths.java  |2 +-
 .../nereids/rules/analysis/CheckAnalysis.java  |   12 +-
 .../rules/analysis/EliminateLogicalSelectHint.java |2 +-
 .../nereids/rules/analysis/ExpressionAnalyzer.java |  807 ---
 .../doris/nereids/rules/analysis/SlotBinder.java   |2 +-
 .../nereids/rules/analysis/SubExprAnalyzer.java|   19 +-
 .../rewrite/PushDownLimitDistinctThroughUnion.java |2 +-
 .../rules/rewrite/PushProjectIntoUnion.java|2 +-
 .../rules/rewrite/PushProjectThroughUnion.java |2 +-
 .../mv/SelectMaterializedIndexWithAggregate.java   |2 +-
 .../doris/nereids/trees/AbstractTreeNode.java  |   10 +-
 .../org/apache/doris/nereids/trees/TreeNode.java   |   17 +-
 .../doris/nereids/trees/expressions/CaseWhen.java  |   31 +-
 .../nereids/trees/expressions/Expression.java  |   65 +-
 .../nereids/trees/expressions/SlotReference.java   |4 +-
 .../functions/ComputeSignatureHelper.java  |   35 +-
 .../functions/ExplicitlyCastableSignature.java |4 -
 .../expressions/functions/IdenticalSignature.java  |4 -
 .../functions/ImplicitlyCastableSignature.java |   10 +-
 .../functions/NullOrIdenticalSignature.java|4 -
 .../visitor/DefaultExpressionRewriter.java |   48 +-
 .../trees/plans/logical/LogicalGenerate.java   |6 +-
 .../nereids/trees/plans/logical/LogicalJoin.java   |6 +-
 .../trees/plans/logical/LogicalOlapScan.java   |   21 +-
 .../trees/plans/logical/LogicalProject.java|   13 +-
 .../trees/plans/logical/LogicalSetOperation.java   |   30 +-
 .../nereids/trees/plans/logical/LogicalUnion.java  |4 +-
 .../trees/plans/visitor/InferPlanOutputAlias.java  |   68 +-
 .../org/apache/doris/nereids/types/ArrayType.java  |3 +-
 .../org/apache/doris/nereids/types/MapType.java|3 +-
 .../org/apache/doris/nereids/types/StructType.java |3 +-
 .../apache/doris/nereids/util/ExpressionUtils.java |  169 +-
 .../org/apache/doris/nereids/util/JoinUtils.java   |8 +-
 .../org/apache/doris/nereids/util/PlanUtils.java   |   18 -
 .../doris/nereids/util/TypeCoercionUtils.java  |   82 +-
 .../java/org/apache/doris/nereids/util/Utils.java  |   69 -
 .../apache/doris/regression/RegressionTest.groovy  |   11 +-
 .../doris/regression/util/OutputUtils.groovy   |2 +-
 52 files changed, 4527 insertions(+), 15165 deletions(-)
 delete mode 100644 
fe/fe-core/src/main/java/org/apache/doris/nereids/analyzer/ComplexDataType.java
 delete mode 100644 
fe/fe-core/src/main/java/org/apache/doris/nereids/analyzer/MappingSlot.java
 delete mode 100644 
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/ExpressionAnalyzer.java


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](memory) Fix LRU cache deleter and memory tracking [doris]

2024-03-12 Thread via GitHub


doris-robot commented on PR #32080:
URL: https://github.com/apache/doris/pull/32080#issuecomment-1990984866

   
   
   TPC-H: Total hot run time: 36643 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 5bd29b5b7bef14d59926a7d60e8076df42469885, 
data reload: false
   
   -- Round 1 --
   q1   17628   422140984098
   q2   2027163 145 145
   q3   10562   910 880 880
   q4   4648660 720 660
   q5   7623281327912791
   q6   181 124 123 123
   q7   1194835 799 799
   q8   9489208120202020
   q9   7376649464606460
   q10  8126246424932464
   q11  417 218 210 210
   q12  762 300 289 289
   q13  17983   301730163016
   q14  280 259 253 253
   q15  493 457 449 449
   q16  484 396 391 391
   q17  958 551 537 537
   q18  6706592758745874
   q19  1577145114661451
   q20  568 267 286 267
   q21  7324317432443174
   q22  793 297 292 292
   Total cold run time: 107199 ms
   Total hot run time: 36643 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4076404940594049
   q2   325 225 226 225
   q3   2978282128362821
   q4   1889154415291529
   q5   5215526452685264
   q6   191 115 115 115
   q7   2249184518721845
   q8   3186329533453295
   q9   8580859585698569
   q10  6112365136283628
   q11  533 455 433 433
   q12  735 563 548 548
   q13  12415   288528482848
   q14  274 246 250 246
   q15  490 451 445 445
   q16  439 425 398 398
   q17  1739149114621462
   q18  7714719870477047
   q19  6827151315381513
   q20  1908169517041695
   q21  4846456047064560
   q22  533 460 441 441
   Total cold run time: 73254 ms
   Total hot run time: 52976 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](planner) remove input slot for aggregate slot which is not materialized [doris]

2024-03-12 Thread via GitHub


github-actions[bot] commented on PR #32092:
URL: https://github.com/apache/doris/pull/32092#issuecomment-1990984953

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](planner) remove input slot for aggregate slot which is not materialized [doris]

2024-03-12 Thread via GitHub


github-actions[bot] commented on PR #32092:
URL: https://github.com/apache/doris/pull/32092#issuecomment-1990984998

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



  1   2   3   4   5   6   7   8   9   10   >