[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #6219: [Feature] Support SHOW DATA SKEW stmt
EmmyMiao87 commented on a change in pull request #6219: URL: https://github.com/apache/incubator-doris/pull/6219#discussion_r669347621 ## File path: docs/zh-CN/sql-reference/sql-statements/Administration/ADMIN-SHOW-DATA-SKEW.md ## @@ -0,0 +1,50 @@ +--- +{ +"title": "ADMIN SHOW DATA SKEW", +"language": "zh-CN" +} +--- + + + +# ADMIN SHOW DATA SKEW +## description + +该语句用于查看表或某个分区的数据倾斜情况。 + +语法: + +ADMIN SHOW DATA SKEW FROM [db_name.]tbl_name [PARTITION (p1)]; Review comment: 如果 partition 是必填项,那么不用加 [] ## File path: docs/zh-CN/sql-reference/sql-statements/Administration/ADMIN-SHOW-DATA-SKEW.md ## @@ -0,0 +1,50 @@ +--- +{ +"title": "ADMIN SHOW DATA SKEW", +"language": "zh-CN" +} +--- + + + +# ADMIN SHOW DATA SKEW +## description + +该语句用于查看表或某个分区的数据倾斜情况。 + +语法: + +ADMIN SHOW DATA SKEW FROM [db_name.]tbl_name [PARTITION (p1)]; + +说明: + +1. 必须指定且仅指定一个分区。对于非分区表,分区名称同表名。 + 2. 结果将展示指定分区下,各个分桶的数据量,以及每个分桶数据量在总数据量中的占比。 Review comment: 注意格式 ## File path: fe/fe-core/src/main/cup/sql_parser.cup ## @@ -262,6 +262,7 @@ terminal String KW_ADD, KW_ADMIN, KW_AFTER, KW_AGGREGATE, KW_ALL, KW_ALTER, KW_A KW_REPAIR, KW_REPEATABLE, KW_REPOSITORY, KW_REPOSITORIES, KW_REPLACE, KW_REPLACE_IF_NOT_NULL, KW_REPLICA, KW_RESOURCE, KW_RESOURCES, KW_RESTORE, KW_RETURNS, KW_RESUME, KW_REVOKE, KW_RIGHT, KW_ROLE, KW_ROLES, KW_ROLLBACK, KW_ROLLUP, KW_ROUTINE, KW_ROW, KW_ROWS, KW_S3, KW_SCHEMA, KW_SCHEMAS, KW_SECOND, KW_SELECT, KW_SEMI, KW_SERIALIZABLE, KW_SESSION, KW_SET, KW_SETS, KW_SET_VAR, KW_SHOW, KW_SIGNED, + KW_SKEW, Review comment: Pay attention to the format ## File path: fe/fe-core/src/main/java/org/apache/doris/analysis/AdminShowDataSkewStmt.java ## @@ -0,0 +1,99 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.analysis; + +import org.apache.doris.catalog.Catalog; +import org.apache.doris.catalog.Column; +import org.apache.doris.catalog.ScalarType; +import org.apache.doris.cluster.ClusterNamespace; +import org.apache.doris.common.AnalysisException; +import org.apache.doris.common.ErrorCode; +import org.apache.doris.common.ErrorReport; +import org.apache.doris.common.UserException; +import org.apache.doris.mysql.privilege.PrivPredicate; +import org.apache.doris.qe.ConnectContext; +import org.apache.doris.qe.ShowResultSetMetaData; + +import com.google.common.base.Strings; +import com.google.common.collect.ImmutableList; + +// admin show data skew from tbl [partition(p1, p2, ...)] +public class AdminShowDataSkewStmt extends ShowStmt { +public static final ImmutableList TITLE_NAMES = new ImmutableList.Builder() +.add("BucketIdx").add("AvgDataSize") +.add("Graph").add("Percent") +.build(); + +private TableRef tblRef; + +public AdminShowDataSkewStmt(TableRef tblRef) { +this.tblRef = tblRef; +} + +@Override +public void analyze(Analyzer analyzer) throws UserException { +super.analyze(analyzer); + +// check auth +if (!Catalog.getCurrentCatalog().getAuth().checkGlobalPriv(ConnectContext.get(), PrivPredicate.ADMIN)) { + ErrorReport.reportAnalysisException(ErrorCode.ERR_SPECIFIC_ACCESS_DENIED_ERROR, "ADMIN"); +} + +String dbName = null; +if (Strings.isNullOrEmpty(tblRef.getName().getDb())) { +dbName = analyzer.getDefaultDb(); +if (Strings.isNullOrEmpty(dbName)) { +ErrorReport.reportAnalysisException(ErrorCode.ERR_NO_DB_ERROR); +} +} else { +dbName = ClusterNamespace.getFullName(getClusterName(), tblRef.getName().getDb()); +} + +tblRef.getName().setDb(dbName); Review comment: analyze tblRef before check partition -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org ---
[GitHub] [incubator-doris] xinyiZzz commented on pull request #1792: Add ChunkAllocator to accelerate chunk allocation
xinyiZzz commented on pull request #1792: URL: https://github.com/apache/incubator-doris/pull/1792#issuecomment-879684670 ChunkAllocator and BufferAllocator have many similarities. For example, each core has an area, and chunks (buffer) of the same size are in a list. When assigning, it will be first obtained from the free list of the area of the current core, etc. Currently ChunkAllocator has some low utilization problems, such as 1. Memory fragmentation. For example, when a large number of small-size chunks are applied at the beginning, these small chunks may no longer be used in the future, nor will they be merged or recycled, nor will large chunks be split. - BufferAllocator will release the current free buffer after the memory request reaches the upper limit. 2. local_core_alloc_count: other_core_alloc_count = 1:541, that is, there is a high probability of obtaining chunks from areas of other cores, which is locked. - BufferAllocator will apply first from the system when there is no buffer in the current core area, and will get it from other core areas only when the system space is insufficient, so that the effect may be better after running for a period of time. 3. There is no upper and lower limit on the size of the chunk; 4. The 2G reserved space is easy to fill up when high concurrency, although this problem can be avoided by changing to the overall memory usage limit. - BufferAllocator is a limit to the overall memory usage 5 These problems can be solved by BufferAllocator to a certain extent, so why didn't you consider using BufferAllocator to take over all the memory management of Doris, but added a new ChunkAllocator to try to take over? Impala tries to use BufferPool (based on BufferAllocator) to support all runtime memory, instead of MemPools, FreePools, etc. Maybe this is a good choice https://issues.apache.org/jira/browse/IMPALA-3200 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] xinyiZzz edited a comment on pull request #1792: Add ChunkAllocator to accelerate chunk allocation
xinyiZzz edited a comment on pull request #1792: URL: https://github.com/apache/incubator-doris/pull/1792#issuecomment-879684670 @imay ChunkAllocator and BufferAllocator have many similarities. For example, each core has an area, and chunks (buffer) of the same size are in a list. When assigning, it will be first obtained from the free list of the area of the current core, etc. Currently ChunkAllocator has some low utilization problems, such as 1. Memory fragmentation. For example, when a large number of small-size chunks are applied at the beginning, these small chunks may no longer be used in the future, nor will they be merged or recycled, nor will large chunks be split. - BufferAllocator will release the current free buffer after the memory request reaches the upper limit. 2. local_core_alloc_count: other_core_alloc_count = 1:541, that is, there is a high probability of obtaining chunks from areas of other cores, which is locked. - BufferAllocator will apply first from the system when there is no buffer in the current core area, and will get it from other core areas only when the system space is insufficient, so that the effect may be better after running for a period of time. 3. There is no upper and lower limit on the size of the chunk; 4. The 2G reserved space is easy to fill up when high concurrency, although this problem can be avoided by changing to the overall memory usage limit. - BufferAllocator is a limit to the overall memory usage 5 These problems can be solved by BufferAllocator to a certain extent, so why didn't you consider using BufferAllocator to take over all the memory management of Doris, but added a new ChunkAllocator to try to take over? Impala tries to use BufferPool (based on BufferAllocator) to support all runtime memory, instead of MemPools, FreePools, etc. Maybe this is a good choice https://issues.apache.org/jira/browse/IMPALA-3200 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] xinyiZzz edited a comment on pull request #1792: Add ChunkAllocator to accelerate chunk allocation
xinyiZzz edited a comment on pull request #1792: URL: https://github.com/apache/incubator-doris/pull/1792#issuecomment-879684670 @imay ChunkAllocator and BufferAllocator have many similarities. For example, each core has an area, and chunks (buffer) of the same size are in a list. When assigning, it will be first obtained from the free list of the area of the current core, etc. Currently ChunkAllocator has some low utilization problems, such as 1. Memory fragmentation. For example, when a large number of small-size chunks are applied at the beginning, these small chunks may no longer be used in the future, nor will they be merged or recycled, nor will large chunks be split. - BufferAllocator will release the current free buffer after the memory request reaches the upper limit. 2. local_core_alloc_count: other_core_alloc_count = 1:541, that is, there is a high probability of obtaining chunks from areas of other cores, which is locked. - BufferAllocator will apply first from the system when there is no buffer in the current core area, and will get it from other core areas only when the system space is insufficient, so that the effect may be better after running for a period of time. 3. There is no upper and lower limit on the size of the chunk; 4. The 2G reserved space is easy to fill up when high concurrency, although this problem can be avoided by changing to the overall memory usage limit. - BufferAllocator is a limit to the overall memory usage 5. These problems can be solved by BufferAllocator to a certain extent, so why didn't you consider using BufferAllocator to take over all the memory management of Doris, but added a new ChunkAllocator to try to take over? Impala tries to use BufferPool (based on BufferAllocator) to support all runtime memory, instead of MemPools, FreePools, etc. Maybe this is a good choice https://issues.apache.org/jira/browse/IMPALA-3200 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] xinyiZzz edited a comment on pull request #1792: Add ChunkAllocator to accelerate chunk allocation
xinyiZzz edited a comment on pull request #1792: URL: https://github.com/apache/incubator-doris/pull/1792#issuecomment-879684670 @imay At present, there are both ChunkAllocator and BufferAllocator in Doris. They have many similarities. For example, each core has an area, and chunks (buffer) of the same size are in a list. When assigning, it will be first obtained from the free list of the current core area. etc. Currently ChunkAllocator has some low utilization problems, such as 1. Memory fragmentation. For example, when a large number of small-size chunks are applied at the beginning, these small chunks may no longer be used in the future, nor will they be merged or recycled, nor will large chunks be split. - BufferAllocator will release the current free buffer after the memory request reaches the upper limit. 2. local_core_alloc_count: other_core_alloc_count = 1:541, that is, there is a high probability of obtaining chunks from areas of other cores, which is locked. - BufferAllocator will apply first from the system when there is no buffer in the current core area, and will get it from other core areas only when the system space is insufficient, so that the effect may be better after running for a period of time. 3. There is no upper and lower limit on the size of the chunk; 4. The 2G reserved space is easy to fill up when high concurrency, although this problem can be avoided by changing to the overall memory usage limit. - BufferAllocator is a limit to the overall memory usage 5. These problems can be solved by BufferAllocator to a certain extent, so why didn't you consider using BufferAllocator to take over all the memory management of Doris, but added a new ChunkAllocator to try to take over? Impala tries to use BufferPool (based on BufferAllocator) to support all runtime memory, instead of MemPools, FreePools, etc. Maybe this is a good choice https://issues.apache.org/jira/browse/IMPALA-3200 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] tjlygdx opened a new issue #6234: CAST 转 decimal 没有实现预期的精度
tjlygdx opened a new issue #6234: URL: https://github.com/apache/incubator-doris/issues/6234 mysql> select cast(10.1112 as decimal(10,2)) ; ++ | CAST(10.1112 AS DECIMAL(10,2)) | ++ | 10.11120 | ++ 1 row in set (0.05 sec) CAST 转 decimal 没有生效,没有实现预期的精度 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg opened a new issue #6235: The query will slow down when there is data skew
yangzhg opened a new issue #6235: URL: https://github.com/apache/incubator-doris/issues/6235 When be divides the scanner, according to the method of one scanner for each tablet, when the data is skewed, and when one tablet is much larger than other tablets, it will appear that a scan node is far behind other nodes. Therefore, for the DUPLICATE table, the division method can be converted to division according to segment files, and each scanner reads files of the same size -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg opened a new pull request #6236: Optimized the reading performance of Duplicate tables with severe data skew
yangzhg opened a new pull request #6236: URL: https://github.com/apache/incubator-doris/pull/6236 ## Proposed changes When dividing the scanner, there is no longer a scanner for each tablet, but according to the file size of the tablet, the tablet is divided into multiple scanners for reading. By default, each scanner reads 1GB. If the data is stored on a SATA disk, this value can be increased, and if it is an SSD disk, the value can be decreased. ## Types of changes What types of changes does your code introduce to Doris? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) - [ ] Code refactor (Modify the code structure, format the code, etc...) - [x] Optimization. Including functional usability improvements and performance improvements. - [ ] Dependency. Such as changes related to third-party components. - [ ] Other. ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [x] I have created an issue on (Fix #skew) and described the bug/feature there in detail - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If these changes need document changes, I have updated the document - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at d...@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] huangmengbin commented on a change in pull request #6099: [Enhance] improve performance of init_scan_key by sharing the schema
huangmengbin commented on a change in pull request #6099: URL: https://github.com/apache/incubator-doris/pull/6099#discussion_r669419768 ## File path: be/src/olap/reader.cpp ## @@ -548,14 +549,36 @@ OLAPStatus Reader::_init_keys_param(const ReaderParams& read_params) { size_t start_key_size = read_params.start_key.size(); _keys_param.start_keys.resize(start_key_size, nullptr); + +size_t scan_key_size = read_params.start_key.front().size(); +if (scan_key_size > _tablet->tablet_schema().num_columns()) { +LOG(WARNING) +<< "Input param are invalid. Column count is bigger than num_columns of schema. " +<< "column_count=" << scan_key_size +<< ", schema.num_columns=" << _tablet->tablet_schema().num_columns(); +return OLAP_ERR_INPUT_PARAMETER_ERROR; +} + +std::vector columns; +for (size_t i = 0; i < scan_key_size; ++i) { +columns.push_back(i); +} Review comment: - Done. - I used the `std::iota` to replace the for-loop. - Do I need to extra the "if (scan_key_size > _tablet->tablet_schema().num_columns()) { LOG(WARNING)" to a function, and wrap it in the macro "RETURN_NOT_OK()" ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] hf200012 closed issue #5371: Flink Doris Connector
hf200012 closed issue #5371: URL: https://github.com/apache/incubator-doris/issues/5371 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] hf200012 closed issue #3849: Table names are not case sensitive
hf200012 closed issue #3849: URL: https://github.com/apache/incubator-doris/issues/3849 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] huangmengbin commented on a change in pull request #6099: [Enhance] improve performance of init_scan_key by sharing the schema
huangmengbin commented on a change in pull request #6099: URL: https://github.com/apache/incubator-doris/pull/6099#discussion_r669421819 ## File path: be/src/olap/reader.cpp ## @@ -548,14 +549,36 @@ OLAPStatus Reader::_init_keys_param(const ReaderParams& read_params) { size_t start_key_size = read_params.start_key.size(); _keys_param.start_keys.resize(start_key_size, nullptr); + +size_t scan_key_size = read_params.start_key.front().size(); +if (scan_key_size > _tablet->tablet_schema().num_columns()) { +LOG(WARNING) +<< "Input param are invalid. Column count is bigger than num_columns of schema. " +<< "column_count=" << scan_key_size +<< ", schema.num_columns=" << _tablet->tablet_schema().num_columns(); +return OLAP_ERR_INPUT_PARAMETER_ERROR; +} + +std::vector columns; +for (size_t i = 0; i < scan_key_size; ++i) { +columns.push_back(i); +} Review comment: > it is better to extract those to a function, this is duplicted with line 158 - Done. - I used the std::iota to replace the for-loop. - Do I need to extra the "if (scan_key_size > _tablet->tablet_schema().num_columns()) { LOG(WARNING)" to a function, and wrap it in the macro "RETURN_NOT_OK()" ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] huangmengbin commented on a change in pull request #6099: [Enhance] improve performance of init_scan_key by sharing the schema
huangmengbin commented on a change in pull request #6099: URL: https://github.com/apache/incubator-doris/pull/6099#discussion_r669421819 ## File path: be/src/olap/reader.cpp ## @@ -548,14 +549,36 @@ OLAPStatus Reader::_init_keys_param(const ReaderParams& read_params) { size_t start_key_size = read_params.start_key.size(); _keys_param.start_keys.resize(start_key_size, nullptr); + +size_t scan_key_size = read_params.start_key.front().size(); +if (scan_key_size > _tablet->tablet_schema().num_columns()) { +LOG(WARNING) +<< "Input param are invalid. Column count is bigger than num_columns of schema. " +<< "column_count=" << scan_key_size +<< ", schema.num_columns=" << _tablet->tablet_schema().num_columns(); +return OLAP_ERR_INPUT_PARAMETER_ERROR; +} + +std::vector columns; +for (size_t i = 0; i < scan_key_size; ++i) { +columns.push_back(i); +} Review comment: > it is better to extract those to a function, this is duplicted with line 158 - Done. - I used the `std::iota` to replace the for-loop. - Do I need to extra the "if (scan_key_size > _tablet->tablet_schema().num_columns()) { LOG(WARNING)" to a function, and wrap it in the macro `RETURN_NOT_OK()` ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman opened a new pull request #6237: [Feature] Support setting concurrency for thread pool token
morningman opened a new pull request #6237: URL: https://github.com/apache/incubator-doris/pull/6237 ## Proposed changes Now we can submit a group of tasks using thread pool token, and limit the max concurrency of this task group ## Types of changes What types of changes does your code introduce to Doris? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) - [ ] Code refactor (Modify the code structure, format the code, etc...) - [ ] Optimization. Including functional usability improvements and performance improvements. - [ ] Dependency. Such as changes related to third-party components. - [ ] Other. ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on (Fix #ISSUE) and described the bug/feature there in detail - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If these changes need document changes, I have updated the document - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at d...@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #6209: [BDBJE] Add a tool to view the data in BEBJE
EmmyMiao87 commented on a change in pull request #6209: URL: https://github.com/apache/incubator-doris/pull/6209#discussion_r669509134 ## File path: fe/fe-core/src/main/java/org/apache/doris/http/action/SystemAction.java ## @@ -17,10 +17,12 @@ package org.apache.doris.http.action; -import org.apache.commons.validator.routines.UrlValidator; +import io.netty.handler.codec.http.HttpMethod; Review comment: import order -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #6210: [HttpV2] Add more httpv2 APIs
EmmyMiao87 commented on a change in pull request #6210: URL: https://github.com/apache/incubator-doris/pull/6210#discussion_r669509780 ## File path: fe/fe-core/src/main/java/org/apache/doris/httpv2/restv2/ImportAction.java ## @@ -0,0 +1,228 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.httpv2.restv2; + +import lombok.Getter; Review comment: import order -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #6210: [HttpV2] Add more httpv2 APIs
EmmyMiao87 commented on a change in pull request #6210: URL: https://github.com/apache/incubator-doris/pull/6210#discussion_r669510237 ## File path: fe/fe-core/src/test/java/org/apache/doris/common/util/BrokerUtilTest.java ## @@ -17,8 +17,11 @@ package org.apache.doris.common.util; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.fail; +import mockit.Expectations; +import mockit.Injectable; Review comment: same as above -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #6212: [Config] set spark load and odbc table feature enable by default
EmmyMiao87 commented on a change in pull request #6212: URL: https://github.com/apache/incubator-doris/pull/6212#discussion_r669511610 ## File path: fe/fe-core/src/main/java/org/apache/doris/analysis/BrokerDesc.java ## @@ -23,11 +23,11 @@ import org.apache.doris.common.util.PrintableMap; import org.apache.doris.thrift.TFileType; -import com.google.common.collect.Maps; - import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; +import com.google.common.collect.Maps; Review comment: import order。。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on pull request #6215: [Bug][RoutineLoad] Fix bug that routine load thread on BE may be blocked
EmmyMiao87 commented on pull request #6215: URL: https://github.com/apache/incubator-doris/pull/6215#issuecomment-879803065 Please fix ut -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a change in pull request #6237: [Feature] Support setting concurrency for thread pool token
yangzhg commented on a change in pull request #6237: URL: https://github.com/apache/incubator-doris/pull/6237#discussion_r669527260 ## File path: be/src/util/threadpool.h ## @@ -362,6 +362,13 @@ class ThreadPoolToken { // Returns true if all submissions are complete, false otherwise. bool wait_for(const MonoDelta& delta); +bool need_dispatch(); Review comment: need_dispatch is a bit weird. Idle may be better or have other better names -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a change in pull request #6237: [Feature] Support setting concurrency for thread pool token
yangzhg commented on a change in pull request #6237: URL: https://github.com/apache/incubator-doris/pull/6237#discussion_r669527260 ## File path: be/src/util/threadpool.h ## @@ -362,6 +362,13 @@ class ThreadPoolToken { // Returns true if all submissions are complete, false otherwise. bool wait_for(const MonoDelta& delta); +bool need_dispatch(); Review comment: `need_dispatch` is a bit weird. `idle` may be better or have other better names -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee opened a new issue #6238: [Proposal] Vectorization Execution Engine optimization for Doris
HappenLee opened a new issue #6238: URL: https://github.com/apache/incubator-doris/issues/6238 ## Motivation At present, the underlying storage in Doris is column storage. But query execution needs to be transferred to the query layer for execution by row-to-column first. Such an implementation maybe cause the performance problem。 * 1. Column-to-row loss. * 2. Can not get better CPU performance without vectorized execution. Currently, vectorized execution has been commonly adopted in mainstream MPP databases, which has a significant effect on improving CPU utilization. In this paper, we investigate the knowledge of vectorization and give a detailed design for implementing vectorization on columnar storage. ## What is vectorization execution Organizing data in this batched, columnar fashion is the primary prerequisite for using SIMD CPU instructions, which operate on a vector of data at a time. Using SIMD instructions is an eventual goal. Now suppose there is a table **People**, the data content is as follows: |Id|Name|Age| |:: |:: |:: | |101|Ivan|22| 115|Peggy|37 114|Victor|45 113|Eve|25 112|Walter|19 109|Trudy|31 108|Bob|27 105|Zoe|29 104|Charlie|42 102|Alice|35 Execute the following query on the table:`SELECT Id, Name, Age, (Age - 30) * 50 AS Bonus FROM People WHERE Age > 30` Volcano model  For traditional relational databases, the way it returns data is on a per-row basis. The problems are: one function call per line, which interrupts CPU flow and is not conducive to branch prediction; high instruction and data cache misses; unfriendly compiler, not conducive to loop expansion, and not conducive to using CPU acceleration instructions such as SIMD. Vectorization Execution Engine  But for columnar databases, you can actually do it in the form of columns, which can reduce the overall cache failure rate. This is equivalent to a vectorized execution to achieve such an effect, which is a unique optimization technique for columnar databases to reduce CPU consumption and improve the overall CPU utilization. # It can bring us the following benefits: * 1. Reduced dummy function calls for branch prediction. * 2. Reduces interpretation overhead by getting a batch of results at a time. * 3. Easy for compiler to do loop pipelining and SIMD optimization. * 4. Friendly to CPU L1 and L2 Cache. ## How Vectorization Execution Engine For Doris ### The key idea and challenge * Design a new memory structure to replace the original `RowBatch` and `Tuple` structure * Rewrite all operators to support vectorization/columnar computation **ClickHouse is an excellent implementation of the vectorized execution engine database, so here we have borrowed a lot from its excellent implementation in terms of data structure and function implementation. We are based on ClickHouse v19.16.2.2 and would like to thank the ClickHouse community and developers.** ### 1. Data Structure We have implemented all types natively supported by Doris based on **Clickhouse's implementation of Column and Block** including: * tinyint * smallint * int * bigint * largeint * boolean * float * double * decimal * date * datetime * HLL * bitmap ### 2. Function and AggregateFunction We have implemented more than 80% of Doris's native support functions based on Clickhouse's implementation of Function and AggregateFunction interfaces, and have completed a large number of SIMD support. And for a more efficient SIMD optimization, the type of NULLABLE for each function is identified as follows.  The specific function implementation support can be found in the following link:[HERE](https://github.com/doris-vectorized/doris-vectorized/issues/49) ### 3. Operator And Execution System Integration # Operator We already implement * VSortNode * VCrossJoinNode * VAggregateNode * VOlapScanNode After the data is read from the storage layer, it is transformed from `RowCursor` to `Block` structures, i.e., from row storage to column storage. Starting from **VOlapScanNode**, the data is organized in the form of columns.  # Exe
[GitHub] [incubator-doris] morningman opened a new pull request #6239: [Community] Add an github action to auto add 'approved' label
morningman opened a new pull request #6239: URL: https://github.com/apache/incubator-doris/pull/6239 ## Proposed changes 1. If any committer APPROVE a PR, the label 'approved' will be added. 2. If any other reviewed APPROVE a PR, the label 'reviewed' will be added. ## Types of changes What types of changes does your code introduce to Doris? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) - [ ] Code refactor (Modify the code structure, format the code, etc...) - [ ] Optimization. Including functional usability improvements and performance improvements. - [ ] Dependency. Such as changes related to third-party components. - [x] Other. ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on (Fix #ISSUE) and described the bug/feature there in detail - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If these changes need document changes, I have updated the document - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at d...@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] huangmengbin edited a comment on pull request #5369: [Load Parallel][3/3] Support parallel delta writer
huangmengbin edited a comment on pull request #5369: URL: https://github.com/apache/incubator-doris/pull/5369#issuecomment-879117362 > ## Proposed changes > In the previous broker load, multiple OlapTableSinks would send data to the same LoadChannel, > and because of the lock granularity problem, LoadChannel could only process these requests serially, > which made it impossible to make full use of cluster resources. > > This CL modifies the related locks so that LoadChannel can process these requests in parallel. > > In the test, with a size of 20G, the load speed of 334 million rows of data in 3 nodes has been > increased from 9min to 5min, and after enabling 2 concurrency, it can be increased to 3min. > > Also modify the profile of load job. > > ## Types of changes > * [x] Breaking change (fix or feature that would cause existing functionality to not work as expected) > > ## Checklist > * [x] I have created an issue on (Fix [[Performance] Import the load performace #5281](https://github.com/apache/incubator-doris/issues/5281) ) and described the bug/feature there in detail Hi, morningman ! 请问一下可以更详细地描述一下当时测试时的信息吗?比如列的数量、哪种数据模型(Uniq、Aggregate、Duplicate),等等?我在uniq模型上进行测试(9G数据、9台机器、1.3亿行、246列) 没有发现其有较明显的提速(13.5min->10min),不知道是否符合代码的预期。(暂时还没找到原因 Thanks ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] huangmengbin commented on a change in pull request #6099: [Enhance] improve performance of init_scan_key by sharing the schema
huangmengbin commented on a change in pull request #6099: URL: https://github.com/apache/incubator-doris/pull/6099#discussion_r669421819 ## File path: be/src/olap/reader.cpp ## @@ -548,14 +549,36 @@ OLAPStatus Reader::_init_keys_param(const ReaderParams& read_params) { size_t start_key_size = read_params.start_key.size(); _keys_param.start_keys.resize(start_key_size, nullptr); + +size_t scan_key_size = read_params.start_key.front().size(); +if (scan_key_size > _tablet->tablet_schema().num_columns()) { +LOG(WARNING) +<< "Input param are invalid. Column count is bigger than num_columns of schema. " +<< "column_count=" << scan_key_size +<< ", schema.num_columns=" << _tablet->tablet_schema().num_columns(); +return OLAP_ERR_INPUT_PARAMETER_ERROR; +} + +std::vector columns; +for (size_t i = 0; i < scan_key_size; ++i) { +columns.push_back(i); +} Review comment: > it is better to extract those to a function, this is duplicted with line 158 - Done. - I used the `std::iota` to replace the for-loop. - Do I need to extract the "if (scan_key_size > _tablet->tablet_schema().num_columns()) { LOG(WARNING)" to a function, and wrap it in the macro `RETURN_NOT_OK()` ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 merged pull request #6239: [Community] Add an github action to auto add 'approved' label
EmmyMiao87 merged pull request #6239: URL: https://github.com/apache/incubator-doris/pull/6239 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [Community] Add an github action to auto add 'approved' label (#6239)
This is an automated email from the ASF dual-hosted git repository. lingmiao pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new e905dd8 [Community] Add an github action to auto add 'approved' label (#6239) e905dd8 is described below commit e905dd84c8d6066173ae9cdf071095ea839f1e56 Author: Mingyu Chen AuthorDate: Wed Jul 14 21:12:42 2021 +0800 [Community] Add an github action to auto add 'approved' label (#6239) 1. If any committer APPROVE a PR, the label 'approved' will be added. 2. If any other reviewed APPROVE a PR, the label 'reviewed' will be added. --- .github/workflows/approve-label.yml | 29 + 1 file changed, 29 insertions(+) diff --git a/.github/workflows/approve-label.yml b/.github/workflows/approve-label.yml new file mode 100644 index 000..dfde7db --- /dev/null +++ b/.github/workflows/approve-label.yml @@ -0,0 +1,29 @@ +name: Label when approved +on: pull_request_review + +jobs: + + label-when-approved: +name: "Label when approved" +runs-on: ubuntu-latest +outputs: + isApprovedByCommiters: ${{ steps.label-when-approved-by-commiters.outputs.isApproved }} + isApprovedByAnyone: ${{ steps.label-when-approved-by-anyone.outputs.isApproved }} +steps: + - name: Label when approved by commiters +uses: TobKed/label-when-approved-action@v1.3 +id: label-when-approved-by-commiters +with: + token: ${{ secrets.GITHUB_TOKEN }} + label: 'approved' + require_committers_approval: 'true' + remove_label_when_approval_missing: 'true' + comment: 'PR approved by at least one committer and no changes requested.' + - name: Label when approved by anyone +uses: TobKed/label-when-approved-action@v1.3 +id: label-when-approved-by-anyone +with: + token: ${{ secrets.GITHUB_TOKEN }} + label: 'reviewed' + comment: 'PR approved by anyone and no changes requested.' + - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] xy720 opened a new issue #6240: [Feature] support canal-fe (create/resume/pause/stop/show sync job/idempotent)
xy720 opened a new issue #6240: URL: https://github.com/apache/incubator-doris/issues/6240 working. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] boboyunz opened a new issue #6241: my Issue: no valid Basic authorization
boboyunz opened a new issue #6241: URL: https://github.com/apache/incubator-doris/issues/6241 i want to add data with steam load ,but get this error **no valid Basic authorization** this is be log I0715 05:07:46.350615 17205 stream_load.cpp:186] new income streaming load request.id=7045db7eb458d83a-7180372f6e4996a3, job_id=-1, txn_id=-1, label=39c25a5c-7000-496e-a98e-348a264c81dd, db=QHYGJ, tbl=cargis2 W0715 05:07:46.350641 17205 stream_load.cpp:212] parse basic authorization failed.id=7045db7eb458d83a-7180372f6e4996a3, job_id=-1, txn_id=-1, label=39c25a5c-7000-496e-a98e-348a264c81dd my request header: {Content-Type: ;charset=UTF-8 User-Agent: Mozilla/4.0 Content-Length: 175 expect: 100-continue format: json strip_outer_array: true jsonpaths: ["$.cph","$.cpys","$.dt","$.ptbm","$.alarm","$.sta","$.xslc","$.gc","$.sbsd","$.xssd","$.xslc","$.dr","$.lat","$.lon"] label: 39c25a5c-7000-496e-a98e-348a264c81dd Authorization: Basic cm9vdDoxcWF6IVFBWg== } -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] boboyunz commented on issue #6241: my Issue: no valid Basic authorization
boboyunz commented on issue #6241: URL: https://github.com/apache/incubator-doris/issues/6241#issuecomment-880267833 最后我的解决方式如下,供大家参考: 根据官方文档,fe收到stream load 请求后,只是做了负载均衡,随机选择一个BE,转发http请求,问题就出在转发请求会把Basic authorization丢掉,导致访问失败。所以我直接跳过BE,自己维护一个BE列表,随机选择一个来发送请求,这样就把问题解决了。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] boboyunz edited a comment on issue #6241: my Issue: no valid Basic authorization
boboyunz edited a comment on issue #6241: URL: https://github.com/apache/incubator-doris/issues/6241#issuecomment-880267833 最后我的解决方式如下,供大家参考: 根据官方文档,fe收到stream load 请求后,只是做了负载均衡,随机选择一个BE,转发http请求,问题就出在转发请求会把Basic authorization丢掉,导致访问失败。所以我直接跳过FE,自己维护一个BE列表,随机选择一个来发送请求,这样就把问题解决了。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg opened a new issue #6242: Support more feature of array types
yangzhg opened a new issue #6242: URL: https://github.com/apache/incubator-doris/issues/6242 Currently, we have supported ARRAY type of syntax analysis, storage and query, but if the array can be really used, it needs more work. ## Important Feature Assuming that there is a table test, the data is as follows ``` +--++ | name | arr| +--++ | abc | [1, 2, 3]| +--++ 1 row in set (0.01 sec) ``` - [ ] Subscript access: `select arr[0] from test` returns `1` - [ ] array join: `SELECT name, arr from test ARRAY JOIN arr` ``` +--+-+ | name | arr | +--+-+ | abc | 1 | | abc | 2 | | abc | 3 | +--+-+ 3 rows in set (0.01 sec) ``` - [ ] Efficient contains function `select contains(arr, 1) from test` - [ ] size function `select size(arr) from test` - [ ] broker load, stream load, routine load support array type import - [ ] Support importing parquet/orc file array type ## Other Feature - [ ] Support nested array - [ ] Support array as value column of AGG/unique table, support repalce/append aggregation function -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #6211: [s3][bug] Remove log4j1
yangzhg merged pull request #6211: URL: https://github.com/apache/incubator-doris/pull/6211 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [s3][bug] Remove log4j1 (#6211)
This is an automated email from the ASF dual-hosted git repository. yangzhg pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 39945ab [s3][bug] Remove log4j1 (#6211) 39945ab is described below commit 39945aba1e1c76a1bc7a63490eea7f4ef95ed3c2 Author: Mingyu Chen AuthorDate: Thu Jul 15 10:47:59 2021 +0800 [s3][bug] Remove log4j1 (#6211) There are jar package conflict Co-authored-by: chenmingyu --- .../src/main/java/org/apache/doris/analysis/StorageBackend.java | 5 +++-- fe/fe-core/src/main/java/org/apache/doris/analysis/StorageDesc.java | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/StorageBackend.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/StorageBackend.java index e73a90e..933153b 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/analysis/StorageBackend.java +++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/StorageBackend.java @@ -27,12 +27,13 @@ import org.apache.doris.thrift.TStorageBackendType; import com.google.common.base.Strings; import org.apache.commons.lang3.StringUtils; -import org.apache.log4j.Logger; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import java.util.Map; public class StorageBackend extends StorageDesc implements ParseNode { -private static Logger LOG = Logger.getLogger(StorageBackend.class); +private static final Logger LOG = LoggerFactory.getLogger(StorageBackend.class); private String location; private StorageType storageType; diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/StorageDesc.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/StorageDesc.java index 83e5110..914e085 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/analysis/StorageDesc.java +++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/StorageDesc.java @@ -23,14 +23,15 @@ import org.apache.doris.common.Config; import org.apache.commons.collections.map.CaseInsensitiveMap; import org.apache.commons.lang3.StringUtils; import org.apache.http.client.utils.URIBuilder; -import org.apache.log4j.Logger; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import java.net.URI; import java.net.URISyntaxException; import java.util.Map; public abstract class StorageDesc { -private static Logger LOG = Logger.getLogger(StorageBackend.class); +private static final Logger LOG = LoggerFactory.getLogger(StorageBackend.class); // for dfs public static final String HADOOP_JOB_UGI = "hadoop.job.ugi"; public static final String HADOOP_JOB_GROUP_NAME = "hadoop.job.group.name"; - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a change in pull request #6223: [ODBC] Support ODBC external table of SQLServer and revise the doc.
yangzhg commented on a change in pull request #6223: URL: https://github.com/apache/incubator-doris/pull/6223#discussion_r670091008 ## File path: docs/zh-CN/extending-doris/odbc-of-doris.md ## @@ -208,15 +208,32 @@ set enable_odbc_transcation = true; | DATETIME | DATETIME | | | NUMBER | DECIMAL | | +### SQLServer + +| MySQL | Doris | 替换方案 | Review comment: MYSQL ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a change in pull request #6223: [ODBC] Support ODBC external table of SQLServer and revise the doc.
yangzhg commented on a change in pull request #6223: URL: https://github.com/apache/incubator-doris/pull/6223#discussion_r670091188 ## File path: docs/en/extending-doris/odbc-of-doris.md ## @@ -212,15 +212,32 @@ There are different data types among different databases. Here, the types in eac | DATETIME | DATETIME | | | NUMBER | DECIMAL | | +### SQLServer + +| MySQL | Doris | 替换方案 | Review comment: Mysql ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a change in pull request #6223: [ODBC] Support ODBC external table of SQLServer and revise the doc.
yangzhg commented on a change in pull request #6223: URL: https://github.com/apache/incubator-doris/pull/6223#discussion_r670091688 ## File path: docs/en/extending-doris/odbc-of-doris.md ## @@ -212,15 +212,32 @@ There are different data types among different databases. Here, the types in eac | DATETIME | DATETIME | | | NUMBER | DECIMAL | | +### SQLServer + +| MySQL | Doris | 替换方案 | +| :--: | :: | :---: | +| BOOLEAN | BOOLEAN | | +| CHAR | CHAR |Only UTF8 encoding is supported| +| VARCHAR | VARCHAR | Only UTF8 encoding is supported | Review comment: no varchar in sqlserver -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg closed pull request #4655: [array][document] Support array type 4/4. Update array document.
yangzhg closed pull request #4655: URL: https://github.com/apache/incubator-doris/pull/4655 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg closed pull request #4650: [array][be] Support array type 3/4. Be exec scan array, insert array and sink array type.
yangzhg closed pull request #4650: URL: https://github.com/apache/incubator-doris/pull/4650 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg closed pull request #4644: [array][fe] Support array type 2/4. FE SQL syntax, ArrayLiteral, meta support
yangzhg closed pull request #4644: URL: https://github.com/apache/incubator-doris/pull/4644 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg closed pull request #4643: [array][be] Support array type 1/4. add array to tablet_meta, add array expr/function, mysql buffer
yangzhg closed pull request #4643: URL: https://github.com/apache/incubator-doris/pull/4643 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg closed pull request #4623: Support Array type
yangzhg closed pull request #4623: URL: https://github.com/apache/incubator-doris/pull/4623 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg closed pull request #2979: Incomplete support Array type (#2871)
yangzhg closed pull request #2979: URL: https://github.com/apache/incubator-doris/pull/2979 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [Bug-Fix] Fix bug that show view report "Unresolved table reference" error (#6184)
This is an automated email from the ASF dual-hosted git repository. yangzhg pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 7c34dbb [Bug-Fix] Fix bug that show view report "Unresolved table reference" error (#6184) 7c34dbb is described below commit 7c34dbbc5b5d155e08ee912a0a18dff3bfd63901 Author: xy720 <22125576+xy...@users.noreply.github.com> AuthorDate: Thu Jul 15 10:55:15 2021 +0800 [Bug-Fix] Fix bug that show view report "Unresolved table reference" error (#6184) --- .../Data Manipulation/SHOW STREAM LOAD.md | 10 +- .../Data Manipulation/SHOW STREAM LOAD.md | 10 +- .../java/org/apache/doris/analysis/QueryStmt.java | 6 +- .../java/org/apache/doris/analysis/SelectStmt.java | 23 +- .../apache/doris/analysis/SetOperationStmt.java| 6 +- .../org/apache/doris/analysis/ShowViewStmt.java| 8 +- .../java/org/apache/doris/analysis/WithClause.java | 4 +- .../org/apache/doris/analysis/SelectStmtTest.java | 15 +- .../apache/doris/analysis/ShowViewStmtTest.java| 238 + .../java/org/apache/doris/utframe/DorisAssert.java | 14 ++ 10 files changed, 255 insertions(+), 79 deletions(-) diff --git a/docs/en/sql-reference/sql-statements/Data Manipulation/SHOW STREAM LOAD.md b/docs/en/sql-reference/sql-statements/Data Manipulation/SHOW STREAM LOAD.md index fc870e7..b2704e3 100644 --- a/docs/en/sql-reference/sql-statements/Data Manipulation/SHOW STREAM LOAD.md +++ b/docs/en/sql-reference/sql-statements/Data Manipulation/SHOW STREAM LOAD.md @@ -57,12 +57,12 @@ SHOW STREAM LOAD FROM example_db WHERE LABEL LIKE "2014_01_02" LIMIT 10; 3. Show the STREAM LOAD task of the specified db, specify label as "load_example_db_20140102" SHOW STREAM LOAD FROM example_db WHERE LABEL = "load_example_db_20140102"; -4. Show the STREAM LOAD task of the specified db, specify status as "success", and sort it in descending order by LoadStartTime -SHOW STREAM LOAD FROM example_db WHERE STATUS = "success" ORDER BY LoadStartTime DESC; +4. Show the STREAM LOAD task of the specified db, specify status as "success", and sort it in descending order by StartTime +SHOW STREAM LOAD FROM example_db WHERE STATUS = "success" ORDER BY StartTime DESC; -5. Show the STREAM LOAD task of the specified dB and sort it in descending order by LoadStartTime, and display 10 query results starting with offset 5 -SHOW STREAM LOAD FROM example_db ORDER BY LoadStartTime DESC limit 5,10; -SHOW STREAM LOAD FROM example_db ORDER BY LoadStartTime DESC limit 10 offset 5; +5. Show the STREAM LOAD task of the specified dB and sort it in descending order by StartTime, and display 10 query results starting with offset 5 +SHOW STREAM LOAD FROM example_db ORDER BY StartTime DESC limit 5,10; +SHOW STREAM LOAD FROM example_db ORDER BY StartTime DESC limit 10 offset 5; ## keyword SHOW,STREAM LOAD diff --git a/docs/zh-CN/sql-reference/sql-statements/Data Manipulation/SHOW STREAM LOAD.md b/docs/zh-CN/sql-reference/sql-statements/Data Manipulation/SHOW STREAM LOAD.md index ec9f570..53f077d 100644 --- a/docs/zh-CN/sql-reference/sql-statements/Data Manipulation/SHOW STREAM LOAD.md +++ b/docs/zh-CN/sql-reference/sql-statements/Data Manipulation/SHOW STREAM LOAD.md @@ -57,12 +57,12 @@ under the License. 3. 展示指定 db 的Stream Load任务,指定 label 为 "load_example_db_20140102" SHOW STREAM LOAD FROM example_db WHERE LABEL = "load_example_db_20140102"; -4. 展示指定 db 的Stream Load任务,指定 status 为 "success", 并按 LoadStartTime 降序排序 -SHOW STREAM LOAD FROM example_db WHERE STATUS = "success" ORDER BY LoadStartTime DESC; +4. 展示指定 db 的Stream Load任务,指定 status 为 "success", 并按 StartTime 降序排序 +SHOW STREAM LOAD FROM example_db WHERE STATUS = "success" ORDER BY StartTime DESC; -5. 展示指定 db 的导入任务 并按 LoadStartTime 降序排序,并从偏移量5开始显示10条查询结果 -SHOW STREAM LOAD FROM example_db ORDER BY LoadStartTime DESC limit 5,10; -SHOW STREAM LOAD FROM example_db ORDER BY LoadStartTime DESC limit 10 offset 5; +5. 展示指定 db 的导入任务 并按 StartTime 降序排序,并从偏移量5开始显示10条查询结果 +SHOW STREAM LOAD FROM example_db ORDER BY StartTime DESC limit 5,10; +SHOW STREAM LOAD FROM example_db ORDER BY StartTime DESC limit 10 offset 5; ## keyword SHOW,STREAM LOAD diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/QueryStmt.java b/fe/fe-core/src/main/java/org/apache/doris/analysis/QueryStmt.java index 3460cd5..15bc170 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/analysis/QueryStmt.java +++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/QueryStmt.java @@ -429,9 +429,9 @@ public abstract class QueryStmt extends StatementBase { } } -public void getWithClauseTableRefs(List tblRefs, Set parentViewNameSet) { +public void getWithClauseTableRefs(Analyzer analyz
[GitHub] [incubator-doris] yangzhg closed issue #6183: [Bug] Show view report "Unresolved table reference" error
yangzhg closed issue #6183: URL: https://github.com/apache/incubator-doris/issues/6183 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #6184: [Bug-Fix] Fix bug that show view report "Unresolved table reference" error
yangzhg merged pull request #6184: URL: https://github.com/apache/incubator-doris/pull/6184 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] JNSimba opened a new pull request #6243: [Feature]:Flink-connector supports streamload parameters
JNSimba opened a new pull request #6243: URL: https://github.com/apache/incubator-doris/pull/6243 ## Proposed changes Flink-connector supports streamload parameters #6199 ## Types of changes What types of changes does your code introduce to Doris? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) - [ ] Code refactor (Modify the code structure, format the code, etc...) - [ ] Optimization. Including functional usability improvements and performance improvements. - [ ] Dependency. Such as changes related to third-party components. - [ ] Other. ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on (Fix #ISSUE) and described the bug/feature there in detail - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If these changes need document changes, I have updated the document - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at d...@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #6205: [Optimize] Using custom conf dir to save log config of Spring
yangzhg merged pull request #6205: URL: https://github.com/apache/incubator-doris/pull/6205 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [Optimize] Using custom conf dir to save log config of Spring (#6205)
This is an automated email from the ASF dual-hosted git repository. yangzhg pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 7e77b5e [Optimize] Using custom conf dir to save log config of Spring (#6205) 7e77b5e is described below commit 7e77b5ed7f29bf26e66509ba16342a6fa2d98341 Author: Mingyu Chen AuthorDate: Thu Jul 15 11:13:51 2021 +0800 [Optimize] Using custom conf dir to save log config of Spring (#6205) The log4j-config.xml will be generated at startup of FE and also when modifying FE config. But in some deploy environment such as k8s, the conf dir is not writable. So change the dir of log4j-config.xml to Config.custom_conf_dir. Also fix some small bugs: 1. Typo "less then" -> "less than" 2. Duplicated `exec_mem_limit` showed in SHOW ROUTINE LOAD 3. Allow MAXVALUE in single partition column table. 4. Add IP info for "intolerate index channel failure" msg. Change-Id: Ib4e1182084219c41eae44d3a28110c0315fdbd7d Co-authored-by: chenmingyu --- be/src/exec/tablet_sink.cpp| 3 +++ .../src/main/java/org/apache/doris/alter/RollupJobV2.java | 4 ++-- .../java/org/apache/doris/alter/SchemaChangeJobV2.java | 4 ++-- .../apache/doris/analysis/CreateMaterializedViewStmt.java | 2 +- .../main/java/org/apache/doris/analysis/InsertStmt.java| 10 +- .../java/org/apache/doris/analysis/PartitionKeyDesc.java | 14 -- .../src/main/java/org/apache/doris/catalog/Database.java | 2 +- .../src/main/java/org/apache/doris/catalog/OlapTable.java | 2 +- .../src/main/java/org/apache/doris/common/Log4jConfig.java | 5 - .../src/main/java/org/apache/doris/httpv2/HttpServer.java | 3 ++- .../org/apache/doris/httpv2/config/SpringLog4j2Config.java | 2 +- .../java/org/apache/doris/load/loadv2/LoadLoadingTask.java | 2 +- .../org/apache/doris/load/routineload/RoutineLoadJob.java | 7 +-- .../doris/load/routineload/RoutineLoadScheduler.java | 2 +- .../java/org/apache/doris/planner/AssertNumRowsNode.java | 2 +- .../main/java/org/apache/doris/qe/ConnectProcessor.java| 11 +-- .../doris/transaction/TabletQuorumFailedException.java | 2 +- .../test/java/org/apache/doris/utframe/UtFrameUtils.java | 5 + 18 files changed, 53 insertions(+), 29 deletions(-) diff --git a/be/src/exec/tablet_sink.cpp b/be/src/exec/tablet_sink.cpp index d7d24d1..e978f99 100644 --- a/be/src/exec/tablet_sink.cpp +++ b/be/src/exec/tablet_sink.cpp @@ -26,6 +26,7 @@ #include "runtime/row_batch.h" #include "runtime/runtime_state.h" #include "runtime/tuple_row.h" +#include "service/backend_options.h" #include "service/brpc.h" #include "util/brpc_stub_cache.h" #include "util/debug/sanitizer_scopes.h" @@ -473,6 +474,8 @@ Status IndexChannel::add_row(Tuple* tuple, int64_t tablet_id) { } if (has_intolerable_failure()) { +std::stringstream ss; +ss << "index channel has intolerable failure. " << BackendOptions::get_localhost(); return Status::InternalError(ss.str()); } diff --git a/fe/fe-core/src/main/java/org/apache/doris/alter/RollupJobV2.java b/fe/fe-core/src/main/java/org/apache/doris/alter/RollupJobV2.java index 470d4c5..dbd0204 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/alter/RollupJobV2.java +++ b/fe/fe-core/src/main/java/org/apache/doris/alter/RollupJobV2.java @@ -751,8 +751,8 @@ public class RollupJobV2 extends AlterJobV2 implements GsonPostProcessable { } /** - * This method is only used to deserialize the text mate which version is less then 86. - * If the meta version >=86, it will be deserialized by the `read` of AlterJobV2 rather then here. + * This method is only used to deserialize the text mate which version is less than 86. + * If the meta version >=86, it will be deserialized by the `read` of AlterJobV2 rather than here. */ public static RollupJobV2 read(DataInput in) throws IOException { Preconditions.checkState(Catalog.getCurrentCatalogJournalVersion() < FeMetaVersion.VERSION_86); diff --git a/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeJobV2.java b/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeJobV2.java index bd22ba2..3292d86 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeJobV2.java +++ b/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeJobV2.java @@ -979,8 +979,8 @@ public class SchemaChangeJobV2 extends AlterJobV2 { } /** - * This method is only used to deserialize the text mate which version is less then 86. - * If the meta version >=86, it will be deserialized by the `read` of AlterJobV2 rather then here. + * This method is only used to deserialize the text mate which version is less than 86. + * If the
[GitHub] [incubator-doris] yangzhg merged pull request #6208: [Compatibility] Change the response body of load info api in httpv2.
yangzhg merged pull request #6208: URL: https://github.com/apache/incubator-doris/pull/6208 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [Compatibility] Change the response body of load info api in httpv2. (#6208)
This is an automated email from the ASF dual-hosted git repository. yangzhg pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new d5cd3ae [Compatibility] Change the response body of load info api in httpv2. (#6208) d5cd3ae is described below commit d5cd3ae5ee617ff696e19b2b01fc02a6d571362d Author: Mingyu Chen AuthorDate: Thu Jul 15 11:14:45 2021 +0800 [Compatibility] Change the response body of load info api in httpv2. (#6208) 1. To be compatible with response body of GetLoadInfoAction in httpv1. 2. Not drop partition by force in dynamic partition scheduler. Change-Id: I50864ddadf1a1c25efa16a465940a1129f937d3d Co-authored-by: chenmingyu --- .../doris/clone/DynamicPartitionScheduler.java | 2 ++ .../doris/httpv2/rest/GetLoadInfoAction.java | 37 ++ 2 files changed, 33 insertions(+), 6 deletions(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java b/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java index 9d10469..0162bc7 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java +++ b/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java @@ -285,6 +285,8 @@ public class DynamicPartitionScheduler extends MasterDaemon { RangeUtils.checkRangeIntersect(reservePartitionKeyRange, checkDropPartitionKey); if (checkDropPartitionKey.upperEndpoint().compareTo(reservePartitionKeyRange.lowerEndpoint()) <= 0) { String dropPartitionName = olapTable.getPartition(checkDropPartitionId).getName(); +// Do not drop the partition "by force", or the partition will be dropped directly instread of being in +// catalog recycle bin. This is for safe reason. dropPartitionClauses.add(new DropPartitionClause(false, dropPartitionName, false, false)); } } catch (DdlException e) { diff --git a/fe/fe-core/src/main/java/org/apache/doris/httpv2/rest/GetLoadInfoAction.java b/fe/fe-core/src/main/java/org/apache/doris/httpv2/rest/GetLoadInfoAction.java index de83bc1..d79cd5b 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/httpv2/rest/GetLoadInfoAction.java +++ b/fe/fe-core/src/main/java/org/apache/doris/httpv2/rest/GetLoadInfoAction.java @@ -20,7 +20,7 @@ package org.apache.doris.httpv2.rest; import org.apache.doris.catalog.Catalog; import org.apache.doris.common.DdlException; import org.apache.doris.common.MetaNotFoundException; -import org.apache.doris.httpv2.entity.ResponseEntityBuilder; +import org.apache.doris.http.rest.RestBaseResult; import org.apache.doris.load.Load; import org.apache.doris.mysql.privilege.PrivPredicate; import org.apache.doris.qe.ConnectContext; @@ -37,6 +37,20 @@ import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; // Get load information of one load job +// To be compatible with old api, we return like this: +// { +// "status": "OK", +// "msg": "Success", +// "jobInfo": { +// "dbName": "default_cluster:db1", +// "tblNames": ["tbl1"], +// "label": "abc", +// "clusterName": "default_cluster", +// "state": "FINISHED", +// "failMsg": "", +// "trackingUrl": "\\N" +// } +// } @RestController public class GetLoadInfoAction extends RestBaseController { @@ -55,13 +69,13 @@ public class GetLoadInfoAction extends RestBaseController { request.getParameter(LABEL_KEY), ConnectContext.get().getClusterName()); if (Strings.isNullOrEmpty(info.dbName)) { -return ResponseEntityBuilder.badRequest("No database selected"); +return new RestBaseResult("No database selected"); } if (Strings.isNullOrEmpty(info.label)) { -return ResponseEntityBuilder.badRequest("No label selected"); +return new RestBaseResult("No label selected"); } if (Strings.isNullOrEmpty(info.clusterName)) { -return ResponseEntityBuilder.badRequest("No cluster selected"); +return new RestBaseResult("No cluster selected"); } RedirectView redirectView = redirectToMaster(request, response); @@ -83,9 +97,20 @@ public class GetLoadInfoAction extends RestBaseController { try { catalog.getLoadManager().getLoadJobInfo(info); } catch (DdlException e1) { -return ResponseEntityBuilder.okWithCommonError(e1.getMessage()); +return new RestBaseResult(e.getMessage()); } } -return ResponseEntityBuilder.ok(info); +return new Result(info); +} + +// This is just same as Result in http/rest/Ge
[GitHub] [incubator-doris] yangzhg merged pull request #6216: [Optimize] Use flat_hash_set to replace unorderd_set in InPredicate
yangzhg merged pull request #6216: URL: https://github.com/apache/incubator-doris/pull/6216 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated (d5cd3ae -> 68f988b)
This is an automated email from the ASF dual-hosted git repository. yangzhg pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from d5cd3ae [Compatibility] Change the response body of load info api in httpv2. (#6208) add 68f988b [Optimize] Use flat_hash_set to replace unorderd_set in InPredicate (#6216) No new revisions were added by this update. Summary of changes: be/src/olap/in_list_predicate.cpp | 41 -- be/src/olap/in_list_predicate.h| 7 ++-- be/src/olap/reader.cpp | 21 +-- be/test/olap/in_list_predicate_test.cpp| 18 +- be/test/olap/rowset/segment_v2/segment_test.cpp| 4 +-- .../java/org/apache/doris/catalog/Catalog.java | 3 +- 6 files changed, 42 insertions(+), 52 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #6215: [Bug][RoutineLoad] Fix bug that routine load thread on BE may be blocked
yangzhg merged pull request #6215: URL: https://github.com/apache/incubator-doris/pull/6215 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated (68f988b -> 409cee0)
This is an automated email from the ASF dual-hosted git repository. yangzhg pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 68f988b [Optimize] Use flat_hash_set to replace unorderd_set in InPredicate (#6216) add 409cee0 [Bug][RoutineLoad] Fix bug that routine load thread on BE may be blocked (#6215) No new revisions were added by this update. Summary of changes: be/src/util/blocking_priority_queue.hpp | 5 - .../java/org/apache/doris/planner/StreamLoadPlanner.java | 12 ++-- 2 files changed, 14 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] pengxiangyu opened a new issue #6244: Add transaction for the operation of insert
pengxiangyu opened a new issue #6244: URL: https://github.com/apache/incubator-doris/issues/6244 Add transaction for the operation of insert. Only insert is acceptable, nor update or delete. e.g. commit a transaction: begin; insert into Tbl values(11, 22, 33); commit; rollback a transaction: begin; insert into Tbl values(11, 22, 33); rollback; -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] pengxiangyu opened a new pull request #6245: Add transaction for the operation of insert #6244
pengxiangyu opened a new pull request #6245: URL: https://github.com/apache/incubator-doris/pull/6245 ## Proposed changes Add transaction for the operation of insert. It will cost less time than non-transaction(it will cost 1/1000 time) when you want to insert a amount of rows. e.g. commit a transaction: begin; insert into Tbl values(11, 22, 33); commit; rollback a transaction: begin; insert into Tbl values(11, 22, 33); rollback; ## Types of changes What types of changes does your code introduce to Doris? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) - [ ] Code refactor (Modify the code structure, format the code, etc...) - [ ] Optimization. Including functional usability improvements and performance improvements. - [ ] Dependency. Such as changes related to third-party components. - [ ] Other. ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [x] I have created an issue on (Fix #6244) and described the bug/feature there in detail - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If these changes need document changes, I have updated the document - [ ] Any dependent changes have been merged ## Further comments -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #6182: [Docs] Add like, regexp function documents
yangzhg merged pull request #6182: URL: https://github.com/apache/incubator-doris/pull/6182 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [Docs] Add like, regexp function documents (#6182)
This is an automated email from the ASF dual-hosted git repository. yangzhg pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 15c5896 [Docs] Add like, regexp function documents (#6182) 15c5896 is described below commit 15c5896f4110c1c2ac35bdc847b19895b7347f5f Author: EmmyMiao87 <522274...@qq.com> AuthorDate: Thu Jul 15 13:16:21 2021 +0800 [Docs] Add like, regexp function documents (#6182) * [Docs] Add like, regexp function documents * Reconstruct * Fix compile error --- docs/.vuepress/sidebar/en.js | 20 ++- docs/.vuepress/sidebar/zh-CN.js| 20 ++- .../append_trailing_char_if_absent.md | 4 +- .../{regexp_replace.md => like/like.md}| 51 ++-- .../string-functions/like/not_like.md | 68 ++ .../string-functions/regexp/not_regexp.md | 56 ++ .../{regexp_replace.md => regexp/regexp.md}| 42 ++--- .../{ => regexp}/regexp_extract.md | 0 .../{ => regexp}/regexp_replace.md | 0 .../append_trailing_char_if_absent.md | 4 +- .../{regexp_replace.md => like/like.md}| 51 ++-- .../string-functions/like/not_like.md | 68 ++ .../{regexp_replace.md => regexp/not_regexp.md}| 42 ++--- .../{regexp_replace.md => regexp/regexp.md}| 42 ++--- .../{ => regexp}/regexp_extract.md | 3 - .../{ => regexp}/regexp_replace.md | 0 16 files changed, 362 insertions(+), 109 deletions(-) diff --git a/docs/.vuepress/sidebar/en.js b/docs/.vuepress/sidebar/en.js index bf40bd3..ce584d0 100644 --- a/docs/.vuepress/sidebar/en.js +++ b/docs/.vuepress/sidebar/en.js @@ -314,8 +314,6 @@ module.exports = [ "ltrim", "money_format", "null_or_empty", - "regexp_extract", - "regexp_replace", "repeat", "reverse", "right", @@ -324,6 +322,24 @@ module.exports = [ "starts_with", "strleft", "strright", + { +title: "fuzzy match", +directoryPath: "like/", +children: [ + "like", + "not_like", +], + }, + { +title: "regular match", +directoryPath: "regexp/", +children: [ + "regexp", + "regexp_extract", + "regexp_replace", + "not_regexp", +], + }, ], }, { diff --git a/docs/.vuepress/sidebar/zh-CN.js b/docs/.vuepress/sidebar/zh-CN.js index e9ef97b..1e7acc6 100644 --- a/docs/.vuepress/sidebar/zh-CN.js +++ b/docs/.vuepress/sidebar/zh-CN.js @@ -319,8 +319,6 @@ module.exports = [ "ltrim", "money_format", "null_or_empty", - "regexp_extract", - "regexp_replace", "repeat", "reverse", "right", @@ -329,6 +327,24 @@ module.exports = [ "starts_with", "strleft", "strright", + { +title: "模糊匹配", +directoryPath: "like/", +children: [ + "like", + "not_like", +], + }, + { +title: "正则匹配", +directoryPath: "regexp/", +children: [ + "regexp", + "regexp_extract", + "regexp_replace", + "not_regexp", +], + }, ], }, { diff --git a/docs/en/sql-reference/sql-functions/string-functions/append_trailing_char_if_absent.md b/docs/en/sql-reference/sql-functions/string-functions/append_trailing_char_if_absent.md index cd24883..6895c3e 100644 --- a/docs/en/sql-reference/sql-functions/string-functions/append_trailing_char_if_absent.md +++ b/docs/en/sql-reference/sql-functions/string-functions/append_trailing_char_if_absent.md @@ -32,8 +32,8 @@ under the License. `VARCHAR append_trailing_char_if_absent(VARCHAR str, VARCHAR trailing_char)` -If the s string is non-empty and does not contain the c character at the end, it appends the c character to the end. -Trailing_char contains only one character, and it will return NULL if contains more than one character +If the @str string is non-empty and does not contain the @trailing_char character at the end, it appends the @trailing_char character to the end. +@trailing_char contains only one character, and it will retu
[GitHub] [incubator-doris] BiteTheDDDDt closed issue #6195: [Feature] Add trash_used_capacity statistics at 'show backends'
BiteThet closed issue #6195: URL: https://github.com/apache/incubator-doris/issues/6195 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] BiteTheDDDDt closed pull request #6196: [Feature] Add trash_used_capacity statistics at 'show backends'
BiteThet closed pull request #6196: URL: https://github.com/apache/incubator-doris/pull/6196 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] BiteTheDDDDt opened a new issue #6246: [Feature] Support for querying the trash used capacity
BiteThet opened a new issue #6246: URL: https://github.com/apache/incubator-doris/issues/6246 **Is your feature request related to a problem? Please describe.** Sometimes, garbage data will accumulate and take up a lot of space, but users don't know it. **Describe the solution you'd like** Add a query for display trash used capacity. query format: `show trash;` `show trash on ("BackendHost1:BackendHeartBeatPort1", "BackendHost2:BackendHeartBeatPort2", ...);` **Describe alternatives you've considered** Users can manually delete trash after noticing that trash takes up a lot of space **Additional context**   -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] BiteTheDDDDt opened a new pull request #6247: [Feature] Support for querying the trash used capacity
BiteThet opened a new pull request #6247: URL: https://github.com/apache/incubator-doris/pull/6247 ## Proposed changes Now user can proactively scan trash directory. ## Types of changes What types of changes does your code introduce to Doris? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [X] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) - [ ] Code refactor (Modify the code structure, format the code, etc...) - [ ] Optimization. Including functional usability improvements and performance improvements. - [ ] Dependency. Such as changes related to third-party components. - [ ] Other. ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [X] I have created an issue on (Fix #6246) and described the bug/feature there in detail - [X] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If these changes need document changes, I have updated the document - [ ] Any dependent changes have been merged -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a change in pull request #6247: [Feature] Support for querying the trash used capacity
yangzhg commented on a change in pull request #6247: URL: https://github.com/apache/incubator-doris/pull/6247#discussion_r670184683 ## File path: fe/fe-core/src/main/jflex/sql_scanner.flex ## @@ -103,6 +103,7 @@ import org.apache.doris.qe.SqlModeHelper; keywordMap.put("array", new Integer(SqlParserSymbols.KW_ARRAY)); keywordMap.put("backend", new Integer(SqlParserSymbols.KW_BACKEND)); keywordMap.put("backends", new Integer(SqlParserSymbols.KW_BACKENDS)); +keywordMap.put("trash", new Integer(SqlParserSymbols.KW_TRASH)); Review comment: keep the alpha order -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a change in pull request #6247: [Feature] Support for querying the trash used capacity
yangzhg commented on a change in pull request #6247: URL: https://github.com/apache/incubator-doris/pull/6247#discussion_r670187091 ## File path: fe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java ## @@ -1690,6 +1698,46 @@ private void handleShowRoles() { List> infos = Catalog.getCurrentCatalog().getAuth().getRoleInfo(); resultSet = new ShowResultSet(showStmt.getMetaData(), infos); } + +private void handleShowTrash() { +ShowTrashStmt showStmt = (ShowTrashStmt) stmt; +List> results = Lists.newArrayList(); + +for (Backend backend : showStmt.getBackends()) { +BackendService.Client client = null; +TNetworkAddress address = null; +Long trashUsedCapacityB = null; +boolean ok = false; +try { +long start = System.currentTimeMillis(); +address = new TNetworkAddress(backend.getHost(), backend.getBePort()); +client = ClientPool.backendPool.borrowObject(address); +trashUsedCapacityB = client.getTrashUsedCapacity(); +LOG.debug("get trash used capacity from backend: {}, trashUsedCapacity: {}, cost {}", backend.getId(), +trashUsedCapacityB, System.currentTimeMillis() - start); +ok = true; +} catch (Exception e) { +LOG.warn("task exec error. backend[{}]", backend.getId(), e); Review comment: Just a log without further processing? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org