[GitHub] [doris] github-actions[bot] commented on pull request #22238: [fix](compaction) fix time series compaction policy corner case
github-actions[bot] commented on PR #22238: URL: https://github.com/apache/doris/pull/22238#issuecomment-1651092719 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 commented on pull request #22218: [Improvement](shuffle) Accumulate rows in a batch for shuffling
Gabriel39 commented on PR #22218: URL: https://github.com/apache/doris/pull/22218#issuecomment-1651095042 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22241: [Improvement](brpc) adjust brpc_light_work_pool_threads/brpc_heavy_work_pool_threads and …
github-actions[bot] commented on PR #22241: URL: https://github.com/apache/doris/pull/22241#issuecomment-1651095447 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on pull request #21970: [enhancement](nereids) Improve stats preload performance
Kikyou1997 commented on PR #21970: URL: https://github.com/apache/doris/pull/21970#issuecomment-1651096403 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei commented on pull request #22226: [fix](executor)cancel exchange buffer rpc when query is cancelled
yiguolei commented on PR #6: URL: https://github.com/apache/doris/pull/6#issuecomment-1651098495 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #21898: [feature](datetime) Support timezone when insert datetime value
hello-stephen commented on PR #21898: URL: https://github.com/apache/doris/pull/21898#issuecomment-1651099931 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.46 seconds stream load tsv: 513 seconds loaded 74807831229 Bytes, about 139 MB/s stream load json: 30 seconds loaded 2358488459 Bytes, about 74 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 29.5 seconds inserted 1000 Rows, about 338K ops/s storage size: 17167455475 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow opened a new pull request, #22242: [fix](Nereids) cast time should turn nullable on all valid types
morrySnow opened a new pull request, #22242: URL: https://github.com/apache/doris/pull/22242 ## Proposed changes Issue Number: close #xxx valid types to cast to time/timev2: - TINYINT - SMALLINT - INT - BIGINT - LARGEINT - FLOAT - DOUBLE - CHAR - VARCHAR - STRING ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #22242: [fix](Nereids) cast time should turn nullable on all valid types
morrySnow commented on PR #22242: URL: https://github.com/apache/doris/pull/22242#issuecomment-1651103529 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22115: [opt](hive)opt select count(*) stmt push down agg on parquet in hive .
github-actions[bot] commented on PR #22115: URL: https://github.com/apache/doris/pull/22115#issuecomment-1651105152 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22218: [Improvement](shuffle) Accumulate rows in a batch for shuffling
github-actions[bot] commented on PR #22218: URL: https://github.com/apache/doris/pull/22218#issuecomment-1651105184 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #22051: [Regression-test](case) update case of fold_constant on delete
BiteThet commented on PR #22051: URL: https://github.com/apache/doris/pull/22051#issuecomment-1651106345 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #22234: [chore](tools) add icon.svg to .idea
morrySnow commented on PR #22234: URL: https://github.com/apache/doris/pull/22234#issuecomment-1651108723 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22237: [fix](point query) Fix ArrayIndexOutOfBoundsException if close a prep…
hello-stephen commented on PR #22237: URL: https://github.com/apache/doris/pull/22237#issuecomment-1651108810 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.66 seconds stream load tsv: 507 seconds loaded 74807831229 Bytes, about 140 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.3 seconds inserted 1000 Rows, about 341K ops/s storage size: 17166972716 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] ByteYue commented on pull request #22105: [enhance](S3) add s3 bvar metrics for all s3 operation
ByteYue commented on PR #22105: URL: https://github.com/apache/doris/pull/22105#issuecomment-1651112117 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataroaring merged pull request #22174: [fix](binlog) Fix BinlogUtils getExpiredMs overflow
dataroaring merged PR #22174: URL: https://github.com/apache/doris/pull/22174 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jixxiong opened a new pull request, #22243: [fix](Unique-Key) fix version upgrade caused MOR to become MOW
jixxiong opened a new pull request, #22243: URL: https://github.com/apache/doris/pull/22243 ## Proposed changes Issue Number: close #xxx We found that when the unique-key table is upgraded from a version that does not support MOW (e.g. 1.1.5) to a version that supports MOW (e.g. 1.2.1), and then further upgraded to 2.0-beta, an issue occurs where MOR (Merge On Read) changes to MOW (Merge On Write). Therefore, this code is primarily aimed at fixing this problem. When a unique-key table does not have the property "ENABLE_UNIQUE_KEY_MERGE_ON_WRITE," it means that it was upgraded from a version that does not support the Merge On Write feature. Therefore, this table is using Merge On Read, and as a result, the value of "ENABLE_UNIQUE_KEY_MERGE_ON_WRITE" should be false, but in FE, our code mistakenly regarded it as true. ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [fix](binlog) Fix BinlogUtils getExpiredMs overflow (#22174)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new af20d0c521 [fix](binlog) Fix BinlogUtils getExpiredMs overflow (#22174) af20d0c521 is described below commit af20d0c52181402db4680f77fef8aa6e5fb46627 Author: Jack Drogon AuthorDate: Wed Jul 26 15:15:34 2023 +0800 [fix](binlog) Fix BinlogUtils getExpiredMs overflow (#22174) --- fe/fe-core/src/main/java/org/apache/doris/binlog/BinlogUtils.java | 4 1 file changed, 4 insertions(+) diff --git a/fe/fe-core/src/main/java/org/apache/doris/binlog/BinlogUtils.java b/fe/fe-core/src/main/java/org/apache/doris/binlog/BinlogUtils.java index 3af4053ef1..30cbfd0e15 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/binlog/BinlogUtils.java +++ b/fe/fe-core/src/main/java/org/apache/doris/binlog/BinlogUtils.java @@ -119,6 +119,10 @@ public class BinlogUtils { public static long getExpiredMs(long ttlSeconds) { long currentSeconds = System.currentTimeMillis() / 1000; +if (currentSeconds < ttlSeconds) { +return Long.MIN_VALUE; +} + long expireSeconds = currentSeconds - ttlSeconds; return expireSeconds * 1000; } - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22051: [Regression-test](case) update case of fold_constant on delete
github-actions[bot] commented on PR #22051: URL: https://github.com/apache/doris/pull/22051#issuecomment-1651117590 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #21634: [Refactor](load) refactor memtable flush actively
hello-stephen commented on PR #21634: URL: https://github.com/apache/doris/pull/21634#issuecomment-1651119027 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 47.63 seconds stream load tsv: 508 seconds loaded 74807831229 Bytes, about 140 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.3 seconds inserted 1000 Rows, about 341K ops/s storage size: 17162489463 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22115: [opt](hive)opt select count(*) stmt push down agg on parquet in hive .
github-actions[bot] commented on PR #22115: URL: https://github.com/apache/doris/pull/22115#issuecomment-1651120143 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] adonis0147 opened a new pull request, #22244: [chore](third-party) Fix the build order for libunwind
adonis0147 opened a new pull request, #22244: URL: https://github.com/apache/doris/pull/22244 ## Proposed changes ~~Issue Number: close #xxx~~ 1. libunwind depends on lzma 2. Fix the missing headers issues when compiling by GCC-13 ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] adonis0147 commented on pull request #22244: [chore](third-party) Fix the build order for libunwind
adonis0147 commented on PR #22244: URL: https://github.com/apache/doris/pull/22244#issuecomment-1651123239 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22105: [enhance](S3) add s3 bvar metrics for all s3 operation
github-actions[bot] commented on PR #22105: URL: https://github.com/apache/doris/pull/22105#issuecomment-1651123924 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22240: [improvement](scanner) use batch size of session instead of limit to improve performance of reading
hello-stephen commented on PR #22240: URL: https://github.com/apache/doris/pull/22240#issuecomment-1651126982 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.41 seconds stream load tsv: 503 seconds loaded 74807831229 Bytes, about 141 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 29.2 seconds inserted 1000 Rows, about 342K ops/s storage size: 17167233442 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22241: [Improvement](brpc) adjust brpc_light_work_pool_threads/brpc_heavy_work_pool_threads and …
hello-stephen commented on PR #22241: URL: https://github.com/apache/doris/pull/22241#issuecomment-1651127301 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.66 seconds stream load tsv: 503 seconds loaded 74807831229 Bytes, about 141 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 29.4 seconds inserted 1000 Rows, about 340K ops/s storage size: 17162656022 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly commented on pull request #22156: [stats](nereids)in predicate range adjust
englefly commented on PR #22156: URL: https://github.com/apache/doris/pull/22156#issuecomment-1651131477 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22244: [chore](third-party) Fix the build order for libunwind
github-actions[bot] commented on PR #22244: URL: https://github.com/apache/doris/pull/22244#issuecomment-1651132409 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] csun5285 commented on pull request #22159: [improvement](compaction) Configuring compaction policy and options in the properties of a table
csun5285 commented on PR #22159: URL: https://github.com/apache/doris/pull/22159#issuecomment-1651138855 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22218: [Improvement](shuffle) Accumulate rows in a batch for shuffling
hello-stephen commented on PR #22218: URL: https://github.com/apache/doris/pull/22218#issuecomment-1651139406 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.03 seconds stream load tsv: 508 seconds loaded 74807831229 Bytes, about 140 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 29.2 seconds inserted 1000 Rows, about 342K ops/s storage size: 17162759560 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] DongLiang-0 opened a new pull request, #22245: [Improve](avro)s3 TVF reads avro file to adapts MinIO
DongLiang-0 opened a new pull request, #22245: URL: https://github.com/apache/doris/pull/22245 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] DongLiang-0 commented on pull request #22245: [Improve](avro)s3 TVF read avro file to adapts MinIO
DongLiang-0 commented on PR #22245: URL: https://github.com/apache/doris/pull/22245#issuecomment-1651154259 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22242: [fix](Nereids) cast time should turn nullable on all valid types
hello-stephen commented on PR #22242: URL: https://github.com/apache/doris/pull/22242#issuecomment-1651156436 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 44.82 seconds stream load tsv: 542 seconds loaded 74807831229 Bytes, about 131 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 29.1 seconds inserted 1000 Rows, about 343K ops/s storage size: 17168131895 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #22144: [fix](Nereids) Nereids support sql cache
xinyiZzz commented on PR #22144: URL: https://github.com/apache/doris/pull/22144#issuecomment-1651156952 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xinyiZzz commented on a diff in pull request #22144: [fix](Nereids) Nereids support sql cache
xinyiZzz commented on code in PR #22144: URL: https://github.com/apache/doris/pull/22144#discussion_r1274510496 ## fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java: ## @@ -1341,9 +1341,13 @@ private void handleQueryStmt() throws Exception { // Sql and PartitionCache CacheAnalyzer cacheAnalyzer = new CacheAnalyzer(context, parsedStmt, planner); -if (cacheAnalyzer.enableCache() && !isOutfileQuery && queryStmt instanceof SelectStmt) { -handleCacheStmt(cacheAnalyzer, channel, (SelectStmt) queryStmt); -return; +if (cacheAnalyzer.enableCache() && !isOutfileQuery +&& context.getSessionVariable().getSqlSelectLimit() == 9223372036854775807L Review Comment: done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21788: [regression](multi-catalog)add EMR cloud env test tools
github-actions[bot] commented on PR #21788: URL: https://github.com/apache/doris/pull/21788#issuecomment-1651158128 `sh-checker report` To get the full details, please check in the [job]("https://github.com/apache/doris/actions/runs/5666049509";) output. shellcheck errors ``` 'shellcheck ' found no issues. ``` shfmt errors ``` 'shfmt ' returned error 1 finding the following formatting issues: -- --- tools/emr_storage_regression/standard_set/run_standard_set.sh.orig +++ tools/emr_storage_regression/standard_set/run_standard_set.sh @@ -33,7 +33,7 @@ if [[ -z "$5" ]]; then echo "run all test default" -elif [[ "$5" = 'all' ]]; then +elif [[ "$5" = 'all' ]]; then echo "run all test" else case=$5 @@ -64,13 +64,13 @@ # run all sh run_queries.sh "${FE_HOST}" "${USER}" "${PORT}" "${catalog_name}".ssb100_parquet_"${TYPE}" queries/ssb_queries.sql sh run_queries.sh "${FE_HOST}" "${USER}" "${PORT}" "${catalog_name}".ssb100_orc_"${TYPE}" queries/ssb_queries.sql - + sh run_queries.sh "${FE_HOST}" "${USER}" "${PORT}" "${catalog_name}".ssb100_parquet_"${TYPE}" queries/ssb_flat_queries.sql sh run_queries.sh "${FE_HOST}" "${USER}" "${PORT}" "${catalog_name}".ssb100_orc_"${TYPE}" queries/ssb_flat_queries.sql - + sh run_queries.sh "${FE_HOST}" "${USER}" "${PORT}" "${catalog_name}".tpch100_parquet_"${TYPE}" queries/tpch_queries.sql sh run_queries.sh "${FE_HOST}" "${USER}" "${PORT}" "${catalog_name}".tpch100_orc_"${TYPE}" queries/tpch_queries.sql - + sh run_queries.sh "${FE_HOST}" "${USER}" "${PORT}" "${catalog_name}".clickbench_parquet_"${TYPE}" queries/clickbench_queries.sql sh run_queries.sh "${FE_HOST}" "${USER}" "${PORT}" "${catalog_name}".clickbench_orc_"${TYPE}" queries/clickbench_queries.sql fi -- You can reformat the above files to meet shfmt's requirements by typing: shfmt -w filename ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22234: [chore](tools) add icon.svg to .idea
hello-stephen commented on PR #22234: URL: https://github.com/apache/doris/pull/22234#issuecomment-1651157573 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.03 seconds stream load tsv: 507 seconds loaded 74807831229 Bytes, about 140 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.2 seconds inserted 1000 Rows, about 342K ops/s storage size: 17155600804 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22105: [enhance](S3) add s3 bvar metrics for all s3 operation
hello-stephen commented on PR #22105: URL: https://github.com/apache/doris/pull/22105#issuecomment-1651159909 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.63 seconds stream load tsv: 505 seconds loaded 74807831229 Bytes, about 141 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 29.2 seconds inserted 1000 Rows, about 342K ops/s storage size: 17167375363 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wangbo commented on pull request #22226: [fix](executor)cancel exchange buffer rpc when query is cancelled
wangbo commented on PR #6: URL: https://github.com/apache/doris/pull/6#issuecomment-1651159438 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris-thirdparty] BiteTheDDDDt merged pull request #107: [chore](workflow) Increase the build space
BiteThet merged PR #107: URL: https://github.com/apache/doris-thirdparty/pull/107 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-thirdparty] branch main updated: [chore](workflow) Increase the build space (#107)
This is an automated email from the ASF dual-hosted git repository. panxiaolei pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/doris-thirdparty.git The following commit(s) were added to refs/heads/main by this push: new f37abb32 [chore](workflow) Increase the build space (#107) f37abb32 is described below commit f37abb32806f1dc556f03d1cd2728dd1f25ef1ef Author: Adonis Ling AuthorDate: Wed Jul 26 15:49:02 2023 +0800 [chore](workflow) Increase the build space (#107) There is not enough disk space to build the third-party libraries, we should increase the build space. --- .github/workflows/build-2.0.yml | 17 + 1 file changed, 17 insertions(+) diff --git a/.github/workflows/build-2.0.yml b/.github/workflows/build-2.0.yml index 63320754..6833ee3c 100644 --- a/.github/workflows/build-2.0.yml +++ b/.github/workflows/build-2.0.yml @@ -156,6 +156,23 @@ jobs: GH_REPO: ${{ github.repository }} GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} steps: + - name: Checkout easimon/maximize-build-space +if: ${{ matrix.config.name == 'Linux' }} +run: | + git clone -b v7 https://github.com/easimon/maximize-build-space + + - name: Maximize build space +if: ${{ matrix.config.name == 'Linux' }} +uses: ./maximize-build-space +with: +root-reserve-mb: 4096 +swap-size-mb: 8192 +remove-dotnet: 'true' +remove-android: 'true' +remove-haskell: 'true' +remove-codeql: 'true' +remove-docker-images: 'true' + - name: Checkout uses: actions/checkout@v3 with: - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] bobhan1 commented on pull request #22235: [fix](tablets_channel) fix the process when erasing elements from `std::set`
bobhan1 commented on PR #22235: URL: https://github.com/apache/doris/pull/22235#issuecomment-1651165458 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22244: [chore](third-party) Fix the build order for libunwind
hello-stephen commented on PR #22244: URL: https://github.com/apache/doris/pull/22244#issuecomment-1651167510 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.53 seconds stream load tsv: 503 seconds loaded 74807831229 Bytes, about 141 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 29.1 seconds inserted 1000 Rows, about 343K ops/s storage size: 17166897430 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22226: [fix](executor)cancel exchange buffer rpc when query is cancelled
github-actions[bot] commented on PR #6: URL: https://github.com/apache/doris/pull/6#issuecomment-1651169327 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22109: [Enhancement] add soft link for the compile_commands.json
hello-stephen commented on PR #22109: URL: https://github.com/apache/doris/pull/22109#issuecomment-1651175418 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.34 seconds stream load tsv: 508 seconds loaded 74807831229 Bytes, about 140 MB/s stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.2 seconds inserted 1000 Rows, about 342K ops/s storage size: 17167648032 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22156: [stats](nereids)in predicate range adjust
hello-stephen commented on PR #22156: URL: https://github.com/apache/doris/pull/22156#issuecomment-1651180784 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.79 seconds stream load tsv: 506 seconds loaded 74807831229 Bytes, about 140 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 29.4 seconds inserted 1000 Rows, about 340K ops/s storage size: 17161524841 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhannngchen commented on pull request #22243: [fix](Unique-Key) fix version upgrade caused MOR to become MOW
zhannngchen commented on PR #22243: URL: https://github.com/apache/doris/pull/22243#issuecomment-1651190099 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] DarvenDuan opened a new pull request, #22246: [improvement](scan) Optimize the generation rule for scan key range when the number of elements in the IN clause exceeds doris_max_scan_k
DarvenDuan opened a new pull request, #22246: URL: https://github.com/apache/doris/pull/22246 ## Proposed changes Issue Number: close #xxx Before this improvement, Doris generate only one scan_key_range when the number of elements in the IN clause exceeds doris_max_scan_key_num and the column in the IN clause hits prefix index. It's NOT reasonable because data can't be filtered well by prefix index and single scan_key_range limits the scan concurrency. After this improvement, we generate doris_max_scan_key_num scan_key_ranges according to the values in the IN clause. Example: - `select * from tbl where id in (1, 3, 14, 58, 60);`, `id` is prefix index. - doris_max_scan_key_num = 3 before : we get single scan_key_range: [1, 60] now: we get three scan_key_range: [1, 3], [14, 14], [58, 60] In my test case, it can speed up query very obviously : - elements in the IN clause: 100 - doris_max_scan_key_num = 48 before: query cost time: **7s891ms** only one scanKey   after: query cost time : **122ms** 48 scanKeys   ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22243: [fix](Unique-Key) fix version upgrade caused MOR to become MOW
github-actions[bot] commented on PR #22243: URL: https://github.com/apache/doris/pull/22243#issuecomment-1651192736 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22243: [fix](Unique-Key) fix version upgrade caused MOR to become MOW
github-actions[bot] commented on PR #22243: URL: https://github.com/apache/doris/pull/22243#issuecomment-1651192673 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wsjz opened a new pull request, #22247: [improvement](multi-catalog)add scanner isolation class loader
wsjz opened a new pull request, #22247: URL: https://github.com/apache/doris/pull/22247 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] JackDrogon commented on pull request #22173: [Enhancement](binlog) Add binlog enable/disable check && filter in Binlog*
JackDrogon commented on PR #22173: URL: https://github.com/apache/doris/pull/22173#issuecomment-1651196026 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22144: [fix](Nereids) Nereids support sql cache
hello-stephen commented on PR #22144: URL: https://github.com/apache/doris/pull/22144#issuecomment-1651197441 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.98 seconds stream load tsv: 506 seconds loaded 74807831229 Bytes, about 140 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.2 seconds inserted 1000 Rows, about 342K ops/s storage size: 17162119180 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhangstar333 opened a new pull request, #22248: [imporve](flex) support scientific notation(aEb) parser
zhangstar333 opened a new pull request, #22248: URL: https://github.com/apache/doris/pull/22248 ## Proposed changes Issue Number: close #https://github.com/apache/doris/issues/22189 https://github.com/apache/doris/assets/87313068/ec8337f7-060d-471d-9a5c-9ca6c919f711";> ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] starocean999 commented on pull request #21747: [improvement](nereids)dphyper GraphSimplifier should consider missed edges when estimating join cost
starocean999 commented on PR #21747: URL: https://github.com/apache/doris/pull/21747#issuecomment-1651201256 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #22207: [Feature](inverted index) add inverted index tool
github-actions[bot] commented on code in PR #22207: URL: https://github.com/apache/doris/pull/22207#discussion_r1274545116 ## be/src/index-tools/index_tool.cpp: ## @@ -0,0 +1,231 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +#include +#include +#include +#include +#include +#include + +#include Review Comment: warning: 'CLucene.h' file not found [clang-diagnostic-error] ```cpp #include ^ ``` ## be/src/index-tools/index_tool.cpp: ## @@ -0,0 +1,231 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "io/fs/local_file_system.h" +#include "olap/rowset/segment_v2/inverted_index_compound_directory.h" +#include "olap/rowset/segment_v2/inverted_index_compound_reader.h" + +using doris::segment_v2::DorisCompoundReader; +using doris::segment_v2::DorisCompoundDirectory; +using doris::io::FileInfo; +using namespace lucene::analysis; +using namespace lucene::index; +using namespace lucene::util; +using namespace lucene::search; + +DEFINE_string(operation, "", "valid operation: show_nested_files,check_terms,term_query"); + +DEFINE_string(directory, "./", "inverted index file directory"); +DEFINE_string(idx_file_name, "", "inverted index file name"); +DEFINE_string(idx_file_path, "", "inverted index file path"); +DEFINE_string(term, "", "inverted index term to query"); +DEFINE_string(column_name, "", "inverted index column_name to query"); +DEFINE_string(pred_type, "", "inverted index term query predicate, eq/lt/gt/le/ge/match etc."); +DEFINE_bool(print_row_id, false, "print row id when query terms"); + +std::string get_usage(const std::string& progname) { +std::stringstream ss; +ss << progname << " is the Doris inverted index file tool.\n"; +ss << "Stop BE first before use this tool.\n"; +ss << "Usage:\n"; +ss << "./index_tool --operation=show_nested_files --idx_file_path=path/to/file\n"; +ss << "./index_tool --operation=check_terms_stats --idx_file_path=path/to/file\n"; +ss << "./index_tool --operation=term_query --dir=directory " + "--idx_file_name=file --print_row_id --term=term --column_name=column_name --pred_type=eq/lt/gt/le/ge/match etc\n"; +return ss.str(); +} + +void search(lucene::store::Directory* dir, std::string& field, std::string& token, std::string& pred) { +IndexReader* reader = IndexReader::open(dir); + +IndexReader* newreader = reader->reopen(); +if (newreader != reader) { +_CLLDELETE(reader); +reader = newreader; +} +IndexSearcher s(reader); +std::unique_ptr query; + +std::wstring field_ws(field.begin(), field.end()); +std::wstring token_ws(token.begin(), token.end()); +lucene::index::Term* term = _CLNEW lucene::index::Term(field_ws.c_str(), token_ws.c_str()); +if (pred == "eq" || pred == "match") { +query.reset(new lucene::search::TermQuery(term)); +} else if (pred == "lt") { +query.reset(new lucene::search::RangeQuery(nullptr, term, false)); +} else if (pred == "gt") { +query.reset(new lucene::search::RangeQuery(term, nullptr, false)); +} else if (pred == "le") { +query.reset(new lucene::search::RangeQuery(nullptr, term, true)); +} else if (pred == "ge") { +query.reset(new lucene::search::RangeQuery(term, nullptr, true)); +} else { +std::c
[GitHub] [doris] hello-stephen commented on pull request #22245: [Improve](avro)s3 TVF read avro file to adapts MinIO
hello-stephen commented on PR #22245: URL: https://github.com/apache/doris/pull/22245#issuecomment-1651201314 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.28 seconds stream load tsv: 545 seconds loaded 74807831229 Bytes, about 130 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.2 seconds inserted 1000 Rows, about 342K ops/s storage size: 17169239803 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] ByteYue opened a new pull request, #22249: [enhance](ColdHeatSeparation) carry use path style info along with cold heat separation to support using minio
ByteYue opened a new pull request, #22249: URL: https://github.com/apache/doris/pull/22249 ## Proposed changes Issue Number: close #xxx Before this pr when using minio as storage resource in cold heat separation along with use_path_style=true, user would encounter with following error:  ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22216: [Improvement](aggregate) optimization for AggregationMethodKeysFixed::insert_keys_into_columns
github-actions[bot] commented on PR #22216: URL: https://github.com/apache/doris/pull/22216#issuecomment-1651202589 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22216: [Improvement](aggregate) optimization for AggregationMethodKeysFixed::insert_keys_into_columns
github-actions[bot] commented on PR #22216: URL: https://github.com/apache/doris/pull/22216#issuecomment-1651202520 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22246: [improvement](scan) Optimize the generation rule for scan key range when the number of elements in the IN clause exceeds doris_ma
github-actions[bot] commented on PR #22246: URL: https://github.com/apache/doris/pull/22246#issuecomment-1651204621 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt merged pull request #22216: [Improvement](aggregate) optimization for AggregationMethodKeysFixed::insert_keys_into_columns
BiteThet merged PR #22216: URL: https://github.com/apache/doris/pull/22216 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Improvement](aggregate) optimization for AggregationMethodKeysFixed::insert_keys_into_columns (#22216)
This is an automated email from the ASF dual-hosted git repository. panxiaolei pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new 9451382428 [Improvement](aggregate) optimization for AggregationMethodKeysFixed::insert_keys_into_columns (#22216) 9451382428 is described below commit 9451382428cdb6cb55b48b7d2e1941c0fbe72701 Author: Pxl AuthorDate: Wed Jul 26 16:19:15 2023 +0800 [Improvement](aggregate) optimization for AggregationMethodKeysFixed::insert_keys_into_columns (#22216) optimization for AggregationMethodKeysFixed::insert_keys_into_columns --- be/src/vec/columns/column_vector.h | 4 +- be/src/vec/common/aggregation_common.h | 24 +++ be/src/vec/exec/vaggregation_node.h| 111 ++--- 3 files changed, 46 insertions(+), 93 deletions(-) diff --git a/be/src/vec/columns/column_vector.h b/be/src/vec/columns/column_vector.h index b8c119a217..04908d8711 100644 --- a/be/src/vec/columns/column_vector.h +++ b/be/src/vec/columns/column_vector.h @@ -227,9 +227,7 @@ public: use by date, datetime, basic type */ void insert_many_fix_len_data(const char* data_ptr, size_t num) override { -if constexpr (!std::is_same_v) { -insert_many_in_copy_way(data_ptr, num); -} else if (IColumn::is_date) { +if (IColumn::is_date) { insert_date_column(data_ptr, num); } else if (IColumn::is_date_time) { insert_datetime_column(data_ptr, num); diff --git a/be/src/vec/common/aggregation_common.h b/be/src/vec/common/aggregation_common.h index 39beb8b25f..31f19e7418 100644 --- a/be/src/vec/common/aggregation_common.h +++ b/be/src/vec/common/aggregation_common.h @@ -166,17 +166,18 @@ T pack_fixed(size_t i, size_t keys_size, const ColumnRawPtrs& key_columns, const } for (size_t j = 0; j < keys_size; ++j) { -bool is_null; +bool is_null = false; -if (!has_bitmap) -is_null = false; -else { +if (has_bitmap) { size_t bucket = j / 8; size_t off = j % 8; is_null = ((bitmap[bucket] >> off) & 1) == 1; } -if (is_null) continue; +if (is_null) { +offset += key_sizes[j]; +continue; +} switch (key_sizes[j]) { case 1: @@ -184,28 +185,24 @@ T pack_fixed(size_t i, size_t keys_size, const ColumnRawPtrs& key_columns, const static_cast(key_columns[j])->get_raw_data_begin<1>() + i, 1); -offset += 1; break; case 2: memcpy(bytes + offset, static_cast(key_columns[j])->get_raw_data_begin<2>() + i * 2, 2); -offset += 2; break; case 4: memcpy(bytes + offset, static_cast(key_columns[j])->get_raw_data_begin<4>() + i * 4, 4); -offset += 4; break; case 8: memcpy(bytes + offset, static_cast(key_columns[j])->get_raw_data_begin<8>() + i * 8, 8); -offset += 8; break; default: memcpy(bytes + offset, @@ -214,6 +211,8 @@ T pack_fixed(size_t i, size_t keys_size, const ColumnRawPtrs& key_columns, const key_sizes[j]); offset += key_sizes[j]; } + +offset += key_sizes[j]; } return key; @@ -224,7 +223,9 @@ inline UInt128 hash128(size_t i, size_t keys_size, const ColumnRawPtrs& key_colu UInt128 key; SipHash hash; -for (size_t j = 0; j < keys_size; ++j) key_columns[j]->update_hash_with_value(i, hash); +for (size_t j = 0; j < keys_size; ++j) { +key_columns[j]->update_hash_with_value(i, hash); +} hash.get128(key.low, key.high); @@ -253,8 +254,9 @@ inline StringRef serialize_keys_to_pool_contiguous(size_t i, size_t keys_size, const char* begin = nullptr; size_t sum_size = 0; -for (size_t j = 0; j < keys_size; ++j) +for (size_t j = 0; j < keys_size; ++j) { sum_size += key_columns[j]->serialize_value_into_arena(i, pool, begin).size; +} return {begin, sum_size}; } diff --git a/be/src/vec/exec/vaggregation_node.h b/be/src/vec/exec/vaggregation_node.h index 9d6f4c4979..e31240cdbc 100644 --- a/be/src/vec/exec/vaggregation_node.h +++ b/be/src/vec/exec/vaggregation_node.h @@ -154,14 +154,6 @@ struct AggregationMethodSerialized { return max_one_row_byte_size; } -static void insert_key_into_columns(const StringRef& key, MutableColumns& key_columns, -const Sizes&) { -auto pos = key
[GitHub] [doris] DarvenDuan commented on pull request #22246: [improvement](scan) Optimize the generation rule for scan key range when the number of elements in the IN clause exceeds doris_max_scan_ke
DarvenDuan commented on PR #22246: URL: https://github.com/apache/doris/pull/22246#issuecomment-1651211226 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22249: [enhance](ColdHeatSeparation) carry use path style info along with cold heat separation to support using minio
github-actions[bot] commented on PR #22249: URL: https://github.com/apache/doris/pull/22249#issuecomment-1651212267 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zgxme commented on pull request #22231: [chore](debug-ci)Troubleshooting the cause of slow CI execution
zgxme commented on PR #22231: URL: https://github.com/apache/doris/pull/22231#issuecomment-1651212866 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22051: [Regression-test](case) update case of fold_constant on delete
hello-stephen commented on PR #22051: URL: https://github.com/apache/doris/pull/22051#issuecomment-1651213330 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.61 seconds stream load tsv: 506 seconds loaded 74807831229 Bytes, about 140 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 29.0 seconds inserted 1000 Rows, about 344K ops/s storage size: 17167589401 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhannngchen opened a new pull request, #22250: [fix](partial-update) sequence column not updated if using function_column.sequence_type
zhannngchen opened a new pull request, #22250: URL: https://github.com/apache/doris/pull/22250 ## Proposed changes Issue Number: close #xxx If user specify the `function_column.sequence_type` while create table,the __DORIS_SEQUENCE_COL__ is filtered while planning, and the data of __DORIS_SEQUENCE_COL__ is not loaded. ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22173: [Enhancement](binlog) Add binlog enable/disable check && filter in Binlog*
github-actions[bot] commented on PR #22173: URL: https://github.com/apache/doris/pull/22173#issuecomment-1651217343 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] lsy3993 opened a new pull request, #22251: [fix](jdbc_catalog) fix int and bigint in mysql view when use doris catalog
lsy3993 opened a new pull request, #22251: URL: https://github.com/apache/doris/pull/22251 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HappenLee commented on a diff in pull request #22229: [Bug](runtime-filter) fix probe expr prepared twice on minmax runtime filter
HappenLee commented on code in PR #9: URL: https://github.com/apache/doris/pull/9#discussion_r1274560592 ## be/src/exprs/runtime_filter.cpp: ## @@ -1840,14 +1839,16 @@ Status IRuntimeFilter::update_filter(const UpdateRuntimeFilterParamsV2* param, } Status RuntimePredicateWrapper::get_push_exprs(std::vector* container, - const vectorized::VExprContextSPtr& prob_expr) { -DCHECK(container != nullptr); -DCHECK(_pool != nullptr); -DCHECK(prob_expr->root()->type().type == _column_return_type || - (is_string_type(prob_expr->root()->type().type) && + const TExpr& probe_expr) { +vectorized::VExprContextSPtr probe_ctx; +RETURN_IF_ERROR(vectorized::VExpr::create_expr_tree(probe_expr, probe_ctx)); +_probe_ctxs.push_back(probe_ctx); Review Comment: the op need lock else may be cause coredump -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #22207: [Feature](inverted index) add inverted index tool
github-actions[bot] commented on code in PR #22207: URL: https://github.com/apache/doris/pull/22207#discussion_r1274565944 ## be/src/index-tools/index_tool.cpp: ## @@ -0,0 +1,241 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +#include Review Comment: warning: 'CLucene.h' file not found [clang-diagnostic-error] ```cpp #include ^ ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhannngchen commented on pull request #22250: [fix](partial-update) sequence column not updated if using function_column.sequence_type
zhannngchen commented on PR #22250: URL: https://github.com/apache/doris/pull/22250#issuecomment-1651225126 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22251: [fix](jdbc_catalog) fix int and bigint in mysql view when use doris catalog
github-actions[bot] commented on PR #22251: URL: https://github.com/apache/doris/pull/22251#issuecomment-1651229920 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22250: [fix](partial-update) sequence column not updated if using function_column.sequence_type
github-actions[bot] commented on PR #22250: URL: https://github.com/apache/doris/pull/22250#issuecomment-1651231373 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22250: [fix](partial-update) sequence column not updated if using function_column.sequence_type
github-actions[bot] commented on PR #22250: URL: https://github.com/apache/doris/pull/22250#issuecomment-1651231461 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22243: [fix](Unique-Key) fix version upgrade caused MOR to become MOW
hello-stephen commented on PR #22243: URL: https://github.com/apache/doris/pull/22243#issuecomment-1651238571 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 44.93 seconds stream load tsv: 504 seconds loaded 74807831229 Bytes, about 141 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 30.2 seconds inserted 1000 Rows, about 331K ops/s storage size: 17170262411 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] weizhengte closed pull request #19716: [refactor](statistics) refactor the related code of analysis info.
weizhengte closed pull request #19716: [refactor](statistics) refactor the related code of analysis info. URL: https://github.com/apache/doris/pull/19716 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22244: [chore](third-party) Fix the build order for libunwind
github-actions[bot] commented on PR #22244: URL: https://github.com/apache/doris/pull/22244#issuecomment-1651244270 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22249: [enhance](ColdHeatSeparation) carry use path style info along with cold heat separation to support using minio
github-actions[bot] commented on PR #22249: URL: https://github.com/apache/doris/pull/22249#issuecomment-1651251784 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #22229: [Bug](runtime-filter) fix probe expr prepared twice on minmax runtime filter
BiteThet commented on PR #9: URL: https://github.com/apache/doris/pull/9#issuecomment-1651253770 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #21747: [improvement](nereids)dphyper GraphSimplifier should consider missed edges when estimating join cost
hello-stephen commented on PR #21747: URL: https://github.com/apache/doris/pull/21747#issuecomment-1651255575 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.7 seconds stream load tsv: 537 seconds loaded 74807831229 Bytes, about 132 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 28.9 seconds inserted 1000 Rows, about 346K ops/s storage size: 17164527477 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22159: [improvement](compaction) Configuring compaction policy and options in the properties of a table
hello-stephen commented on PR #22159: URL: https://github.com/apache/doris/pull/22159#issuecomment-1651256873 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 44.58 seconds stream load tsv: 505 seconds loaded 74807831229 Bytes, about 141 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.6 seconds inserted 1000 Rows, about 337K ops/s storage size: 17166697669 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] KassieZ opened a new pull request, #22252: [Docs](community) Add Join Community Files of EN & CN Verison
KassieZ opened a new pull request, #22252: URL: https://github.com/apache/doris/pull/22252 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataroaring commented on pull request #22253: [enhencement](binlog) Update table binlog config disable failure when db binlog is enable && modify table binlog more than one property.
dataroaring commented on PR #22253: URL: https://github.com/apache/doris/pull/22253#issuecomment-1651262978 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] JackDrogon opened a new pull request, #22253: [enhencement](binlog) Update table binlog config disable failure when db binlog is enable && modify table binlog more than one property.
JackDrogon opened a new pull request, #22253: URL: https://github.com/apache/doris/pull/22253 ## Proposed change Update table binlog config disable failure when db binlog is enable && modify table binlog more than one property. ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhangstar333 commented on pull request #22169: [improve](agg) support distinct agg node
zhangstar333 commented on PR #22169: URL: https://github.com/apache/doris/pull/22169#issuecomment-1651263001 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22229: [Bug](runtime-filter) fix probe expr prepared twice on minmax runtime filter
github-actions[bot] commented on PR #9: URL: https://github.com/apache/doris/pull/9#issuecomment-1651264655 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22229: [Bug](runtime-filter) fix probe expr prepared twice on minmax runtime filter
github-actions[bot] commented on PR #9: URL: https://github.com/apache/doris/pull/9#issuecomment-1651265454 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22205: [fix](executor) only mysql connect to set GlobalPipelineTask
github-actions[bot] commented on PR #22205: URL: https://github.com/apache/doris/pull/22205#issuecomment-1651266034 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22205: [fix](executor) only mysql connect to set GlobalPipelineTask
github-actions[bot] commented on PR #22205: URL: https://github.com/apache/doris/pull/22205#issuecomment-1651266092 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22253: [enhencement](binlog) Update table binlog config disable failure when db binlog is enable && modify table binlog more than one pr
github-actions[bot] commented on PR #22253: URL: https://github.com/apache/doris/pull/22253#issuecomment-1651265038 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22253: [enhencement](binlog) Update table binlog config disable failure when db binlog is enable && modify table binlog more than one pr
github-actions[bot] commented on PR #22253: URL: https://github.com/apache/doris/pull/22253#issuecomment-1651265094 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22251: [fix](jdbc_catalog) fix int and bigint in mysql view when use doris catalog
github-actions[bot] commented on PR #22251: URL: https://github.com/apache/doris/pull/22251#issuecomment-1651267089 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22249: [enhance](ColdHeatSeparation) carry use path style info along with cold heat separation to support using minio
github-actions[bot] commented on PR #22249: URL: https://github.com/apache/doris/pull/22249#issuecomment-1651270038 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22249: [enhance](ColdHeatSeparation) carry use path style info along with cold heat separation to support using minio
github-actions[bot] commented on PR #22249: URL: https://github.com/apache/doris/pull/22249#issuecomment-1651270104 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xinyiZzz commented on a diff in pull request #22041: [Improve](point query) Improve lookup connection cache from DoubleBuffer to LRU cache for better item pruning
xinyiZzz commented on code in PR #22041: URL: https://github.com/apache/doris/pull/22041#discussion_r1274564679 ## be/src/service/point_query_executor.h: ## @@ -186,76 +189,59 @@ class RowCache { // A cache used for prepare stmt. // One connection per stmt perf uuid -// Use DoublyBufferedData to wrap Cache for performance and thread safe, -// since it's barely modified -class LookupCache { +class LookupConnectionCache : public LRUCachePolicy { public: -// uuid to reusable -using Cache = phmap::flat_hash_map>; -using CacheIter = Cache::iterator; - -LookupCache() = default; -static LookupCache& instance() { -static LookupCache ins; -return ins; -} +static LookupConnectionCache* instance() { return _s_instance; } -void add(uint128 cache_id, std::shared_ptr item) { -assert(item != nullptr); -_double_buffer_cache.Modify(update_cache, std::make_pair(cache_id, item)); -} +static void create_global_instance(size_t capacity); -// find an item, return null if not exist -std::shared_ptr get(uint128 cache_id) { -butil::DoublyBufferedData::ScopedPtr s; -if (_double_buffer_cache.Read(&s) != 0) { -LOG(WARNING) << "failed to get cache from double buffer data"; -return nullptr; -} -auto it = s->find(cache_id); -if (it != s->end()) { -return it->second; -} -return nullptr; +private: +friend class PointQueryExecutor; +LookupConnectionCache(size_t capacity) +: LRUCachePolicy("LookupConnectionCache", capacity, LRUCacheType::SIZE, + config::tablet_lookup_cache_clean_interval) {} + +std::string encode_key(__int128_t cache_id) { +fmt::memory_buffer buffer; +fmt::format_to(buffer, "{}", cache_id); +return std::string(buffer.data(), buffer.size()); } -private: -butil::DoublyBufferedData _double_buffer_cache; -// 30 seconds for expiring an item -int32_t _expir_seconds = config::tablet_lookup_cache_clean_interval; - -static size_t update_cache(Cache& old_cache, - const std::pair>& p) { -old_cache.emplace(p); -return 1; +void add(__int128_t cache_id, std::shared_ptr item) { +std::string key = encode_key(cache_id); +CacheValue* value = new CacheValue; +value->last_visit_time = UnixMillis(); +value->item = item; +auto deleter = [](const doris::CacheKey& key, void* value) { +CacheValue* cache_value = (CacheValue*)value; +delete cache_value; +}; +LOG(INFO) << "Add item mem size " << item->mem_size() Review Comment: Could this log be too frequent Maybe add metrics to `add` and `get` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22041: [Improve](point query) Improve lookup connection cache from DoubleBuffer to LRU cache for better item pruning
github-actions[bot] commented on PR #22041: URL: https://github.com/apache/doris/pull/22041#issuecomment-1651277871 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22041: [Improve](point query) Improve lookup connection cache from DoubleBuffer to LRU cache for better item pruning
github-actions[bot] commented on PR #22041: URL: https://github.com/apache/doris/pull/22041#issuecomment-1651277992 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] adonis0147 commented on pull request #22244: [chore](third-party) Fix the build order for libunwind
adonis0147 commented on PR #22244: URL: https://github.com/apache/doris/pull/22244#issuecomment-1651278610 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22250: [fix](partial-update) sequence column not updated if using function_column.sequence_type
hello-stephen commented on PR #22250: URL: https://github.com/apache/doris/pull/22250#issuecomment-1651279688 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.39 seconds stream load tsv: 506 seconds loaded 74807831229 Bytes, about 140 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.2 seconds inserted 1000 Rows, about 342K ops/s storage size: 17163847761 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org