[GitHub] [incubator-doris] jackwener commented on pull request #9686: push predicate to subquery
jackwener commented on PR #9686: URL: https://github.com/apache/incubator-doris/pull/9686#issuecomment-1136861370 There are some checkstyle problems. I think use `idea-checkstyle` to check style problem will be better. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] jackwener opened a new pull request, #9767: [enhancement](style): polish the `checkstyle`
jackwener opened a new pull request, #9767: URL: https://github.com/apache/incubator-doris/pull/9767 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. - cancel the auto-align in Javadoc ```java @ param1 desc @ paramExample desc ``` - polish doc for checkstyle ## Checklist(Required) 1. Does it affect the original behavior: (No) 2. Has unit tests been added: (No) 3. Has document been added or modified: (yes) 4. Does it need to update dependencies: (no) 5. Are there any changes that cannot be rolled back: (No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] englefly opened a new issue, #9768: [Feature] window function: "min() over()" does not support preceding and following in range expression
englefly opened a new issue, #9768: URL: https://github.com/apache/incubator-doris/issues/9768 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description `SELECT nationkey, min(nationkey) OVER (PARTITION BY regionkey ORDER BY nationkey ROWS BETWEEN 2 PRECEDING AND 1 FOLLOWING) AS min FROM tpch_tiny_nation` Doris returns error: java.sql.SQLException: errCode = 2, detailMessage = 'min(`nationkey`)' is only supported with an UNBOUNDED PRECEDING start bound. ### Use case _No response_ ### Related issues _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] englefly opened a new issue, #9769: [Feature] window function rank() does not work with Windowing clause
englefly opened a new issue, #9769: URL: https://github.com/apache/incubator-doris/issues/9769 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description sql: select orderkey, discount, dense_rank() over (order by discount), rank() over (order by discount range between unbounded preceding and current row) from tpch_tiny_lineitem where partkey = 272 Doris returns error java.sql.SQLException: errCode = 2, detailMessage = Windowing clause not allowed with 'rank()' ### Use case _No response_ ### Related issues _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] nextdreamblue commented on pull request #9694: [Bug] Fix create table like when having hidden columns
nextdreamblue commented on PR #9694: URL: https://github.com/apache/incubator-doris/pull/9694#issuecomment-1136942643 @stalary the checkstyle problem, done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] jackwener commented on a diff in pull request #8862: [feature-wip](statistics) step5: show statistics job information
jackwener commented on code in PR #8862: URL: https://github.com/apache/incubator-doris/pull/8862#discussion_r881366908 ## fe/fe-core/src/main/java/org/apache/doris/analysis/ShowAnalyzeStmt.java: ## @@ -0,0 +1,354 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.analysis; + +import org.apache.doris.catalog.Catalog; +import org.apache.doris.catalog.Column; +import org.apache.doris.catalog.Database; +import org.apache.doris.catalog.ScalarType; +import org.apache.doris.catalog.Table; +import org.apache.doris.common.AnalysisException; +import org.apache.doris.common.ErrorCode; +import org.apache.doris.common.ErrorReport; +import org.apache.doris.common.UserException; +import org.apache.doris.common.util.OrderByPair; +import org.apache.doris.mysql.privilege.PaloAuth; +import org.apache.doris.mysql.privilege.PrivPredicate; +import org.apache.doris.qe.ConnectContext; +import org.apache.doris.qe.ShowResultSetMetaData; +import org.apache.doris.statistics.StatisticsJob; + +import com.google.common.base.Preconditions; +import com.google.common.base.Strings; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.Sets; + +import java.util.ArrayList; +import java.util.List; +import java.util.Set; + +/** + * ShowAnalyzeStmt is used to show statistics job info. + * syntax: + *SHOW ANALYZE + *[TABLE | ID] + *[ + *WHERE + *[STATE = ["PENDING"|"SCHEDULING"|"RUNNING"|"FINISHED"|"FAILED"|"CANCELLED"]] + *] + *[ORDER BY ...] + *[LIMIT limit][OFFSET offset]; + */ +public class ShowAnalyzeStmt extends ShowStmt { +private static final String STATE_NAME = "state"; +private static final ImmutableList TITLE_NAMES = new ImmutableList.Builder() +.add("id") +.add("create_time") +.add("start_time") +.add("finish_time") +.add("error_msg") +.add("scope") +.add("progress") +.add("state") +.build(); + +private List jobIds; +private TableName dbTableName; +private Expr whereClause; +private LimitElement limitElement; +private List orderByElements; + +// after analyzed +private long dbId; +private final Set tblIds = Sets.newHashSet(); + +private String stateValue; +private ArrayList orderByPairs; + +public ShowAnalyzeStmt() { +} + +public ShowAnalyzeStmt(List jobIds) { +this.jobIds = jobIds; +} + +public ShowAnalyzeStmt(TableName dbTableName, + Expr whereClause, + List orderByElements, + LimitElement limitElement) { +this.dbTableName = dbTableName; +this.whereClause = whereClause; +this.orderByElements = orderByElements; +this.limitElement = limitElement; +} + +public List getJobIds() { +return jobIds; +} + +public long getDbId() { +Preconditions.checkArgument(isAnalyzed(), +"The dbId must be obtained after the parsing is complete"); +return dbId; +} + +public Set getTblIds() { +Preconditions.checkArgument(isAnalyzed(), +"The dbId must be obtained after the parsing is complete"); +return tblIds; +} + +public String getStateValue() { +Preconditions.checkArgument(isAnalyzed(), +"The tbl name must be obtained after the parsing is complete"); +return stateValue; +} + +public ArrayList getOrderByPairs() { +Preconditions.checkArgument(isAnalyzed(), +"The tbl name must be obtained after the parsing is complete"); +return orderByPairs; +} + +public long getLimit() { +if (limitElement != null && limitElement.hasLimit()) { +return limitElement.getLimit(); +} +return -1L; +} + +public long getOffset() { +if (limitElement != null && limitElement.hasOffset()) { +return limitElement.getOffset(); +} +return -1L; +} + +@Override +public void analyze(Analyzer analyzer) throws AnalysisException, UserEx
[GitHub] [incubator-doris] yangzhg opened a new pull request, #9770: [fix] disabel transfer data large than 2GB
yangzhg opened a new pull request, #9770: URL: https://github.com/apache/incubator-doris/pull/9770 # Proposed changes because of brpc and protobuf cannot transfer data large than 2GB, if large than 2GB will overflow, so add a check before send ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee merged pull request #9688: [fix] group by with two NULL rows after left join
HappenLee merged PR #9688: URL: https://github.com/apache/incubator-doris/pull/9688 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee closed issue #9544: [Bug] After left join, there are two rows of the same key after group by
HappenLee closed issue #9544: [Bug] After left join, there are two rows of the same key after group by URL: https://github.com/apache/incubator-doris/issues/9544 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [fix] group by with two NULL rows after left join (#9688)
This is an automated email from the ASF dual-hosted git repository. lihaopeng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 2725127421 [fix] group by with two NULL rows after left join (#9688) 2725127421 is described below commit 272512742196c8d5d3d8225988d8f26d7851c58d Author: camby <104178...@qq.com> AuthorDate: Wed May 25 16:43:55 2022 +0800 [fix] group by with two NULL rows after left join (#9688) Co-authored-by: cambyzju --- be/src/vec/columns/column_nullable.cpp | 5 +- .../query/aggregate/aggregate_groupby_null.out | 7 +++ .../query/aggregate/aggregate_groupby_null.groovy | 58 ++ 3 files changed, 69 insertions(+), 1 deletion(-) diff --git a/be/src/vec/columns/column_nullable.cpp b/be/src/vec/columns/column_nullable.cpp index f7cf9e9d3e..215ced2383 100644 --- a/be/src/vec/columns/column_nullable.cpp +++ b/be/src/vec/columns/column_nullable.cpp @@ -102,7 +102,10 @@ StringRef ColumnNullable::serialize_value_into_arena(size_t n, Arena& arena, static constexpr auto s = sizeof(arr[0]); auto pos = arena.alloc_continue(s, begin); -memcpy(pos, &arr[n], s); +// Value of `NULL` may be 1 or JOIN_NULL_HINT, we serialize both to 1. +// Because we need same key for both `NULL` values while processing `group by`. +UInt8* val = reinterpret_cast(pos); +*val = (arr[n] ? 1 : 0); if (arr[n]) return StringRef(pos, s); diff --git a/regression-test/data/query/aggregate/aggregate_groupby_null.out b/regression-test/data/query/aggregate/aggregate_groupby_null.out new file mode 100644 index 00..9c44613df2 --- /dev/null +++ b/regression-test/data/query/aggregate/aggregate_groupby_null.out @@ -0,0 +1,7 @@ +-- This file is automatically generated. You should know what you did if you want to edit this +-- !groupby_null -- +\N 3 + +-- !groupby_null -- +\N 3 + diff --git a/regression-test/suites/query/aggregate/aggregate_groupby_null.groovy b/regression-test/suites/query/aggregate/aggregate_groupby_null.groovy new file mode 100644 index 00..dc3742c5fe --- /dev/null +++ b/regression-test/suites/query/aggregate/aggregate_groupby_null.groovy @@ -0,0 +1,58 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +suite("aggregate_groupby_null") { +def leftTable = "agg_groupby_null_left" +sql """ DROP TABLE IF EXISTS ${leftTable} """ +sql """ +CREATE TABLE IF NOT EXISTS ${leftTable} ( +id INT NULL, +device_id STRING NULL +) +UNIQUE KEY(id) +DISTRIBUTED BY HASH(id) BUCKETS 1 +PROPERTIES ( + "replication_num" = "1" +) +""" +sql """ INSERT INTO ${leftTable} VALUES (1,'1'),(2,'2'),(3,'3'),(4,'4') """ + +def rightTable = "agg_groupby_null_right" +sql """ DROP TABLE IF EXISTS ${rightTable} """ +sql """ +CREATE TABLE IF NOT EXISTS ${rightTable} ( +id INT NULL, +device_name STRING NULL +) +UNIQUE KEY(id) +DISTRIBUTED BY HASH(id) BUCKETS 1 +PROPERTIES ( + "replication_num" = "1" +) +""" +sql """ INSERT INTO ${rightTable} VALUES (1,'name'),(3,null) """ + +sql """ set enable_vectorized_engine=false """ +qt_groupby_null """ SELECT rt.device_name, COUNT(${leftTable}.id) FROM ${leftTable} +LEFT JOIN ${rightTable} rt ON ${leftTable}.id = rt.id + WHERE rt.device_name is NULL group by rt.device_name """ + +sql """ set enable_vectorized_engine=true """ +qt_groupby_null """ SELECT rt.device_name, COUNT(${leftTable}.id) FROM ${leftTable} +LEFT JOIN ${rightTable} rt ON ${leftTable}.id = rt.id + WHERE rt.device_name is NULL group by rt.device_name """ +} - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee merged pull request #9700: [doc] Add manual for Array data type and functions
HappenLee merged PR #9700: URL: https://github.com/apache/incubator-doris/pull/9700 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [doc] Add manual for Array data type and functions (#9700)
This is an automated email from the ASF dual-hosted git repository. lihaopeng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 2ad691edf7 [doc] Add manual for Array data type and functions (#9700) 2ad691edf7 is described below commit 2ad691edf72a76d6d0967f0a878724046ebc3dad Author: camby <104178...@qq.com> AuthorDate: Wed May 25 16:44:20 2022 +0800 [doc] Add manual for Array data type and functions (#9700) Co-authored-by: cambyzju --- docs/.vuepress/sidebar/en.js | 11 +++ docs/.vuepress/sidebar/zh-CN.js| 11 +++ .../array-functions/array_contains.md | 65 + .../array-functions/array_position.md | 65 + .../sql-functions/array-functions/element_at.md| 81 + .../sql-manual/sql-reference/Data-Types/ARRAY.md | 84 ++ .../array-functions/array_contains.md | 65 + .../array-functions/array_position.md | 65 + .../sql-functions/array-functions/element_at.md| 81 + .../sql-manual/sql-reference/Data-Types/ARRAY.md | 84 ++ 10 files changed, 612 insertions(+) diff --git a/docs/.vuepress/sidebar/en.js b/docs/.vuepress/sidebar/en.js index cb677e32d3..a05ad55e7c 100644 --- a/docs/.vuepress/sidebar/en.js +++ b/docs/.vuepress/sidebar/en.js @@ -547,6 +547,16 @@ module.exports = [ "WINDOW-FUNCTION-ROW-NUMBER", ], }, + { +title: "Array Functions", +directoryPath: "array-functions/", +initialOpenGroupIndex: -1, +children: [ + "array_contains", + "array_position", + "element_at", +], + }, "cast", "digital-masking", ], @@ -812,6 +822,7 @@ module.exports = [ "STRING", "TINYINT", "VARCHAR", + "ARRAY", ], }, { diff --git a/docs/.vuepress/sidebar/zh-CN.js b/docs/.vuepress/sidebar/zh-CN.js index e1fad83642..65a972ca13 100644 --- a/docs/.vuepress/sidebar/zh-CN.js +++ b/docs/.vuepress/sidebar/zh-CN.js @@ -547,6 +547,16 @@ module.exports = [ "WINDOW-FUNCTION-ROW-NUMBER", ], }, + { +title: "Array函数", +directoryPath: "array-functions/", +initialOpenGroupIndex: -1, +children: [ + "array_contains", + "array_position", + "element_at", +], + }, "cast", "digital-masking", ], @@ -812,6 +822,7 @@ module.exports = [ "STRING", "TINYINT", "VARCHAR", + "ARRAY", ], }, { diff --git a/docs/en/sql-manual/sql-functions/array-functions/array_contains.md b/docs/en/sql-manual/sql-functions/array-functions/array_contains.md new file mode 100644 index 00..57091a57ed --- /dev/null +++ b/docs/en/sql-manual/sql-functions/array-functions/array_contains.md @@ -0,0 +1,65 @@ +--- +{ +"title": "array_contains", +"language": "en" +} +--- + + + +## array_contains + +### description + + Syntax + +`BOOLEAN array_contains(ARRAY arr, T value)` + +Check if a value presents in an array column. Return below values: + +``` +1- if value presents in an array; +0- if value does not present in an array; +NULL - when array is NULL; +``` + +### notice + +`Only supported in vectorized engine` + +### example + +``` +mysql> set enable_vectorized_engine=true; + +mysql> SELECT id,c_array,array_contains(c_array, 5) FROM `array_test`; ++--+-+--+ +| id | c_array | array_contains(`c_array`, 5) | ++--+-+--+ +|1 | [1, 2, 3, 4, 5] |1 | +|2 | [6, 7, 8] |0 | +|3 | [] |0 | +|4 | NULL| NULL | ++--+-+--+ +``` + +### keywords + +ARRAY_CONTAINS diff --git a/docs/en/sql-manual/sql-functions/array-functions/array_position.md b/docs/en/sql-manual/sql-functions/array-functions/array_position.md new file mode 100644 index 00..dd47628c27 --- /dev/null +++ b/docs/en/sql-manual/sql-functions/array-functions/array_position.md @@ -0,0 +1,65 @@ +--- +{ +"title": "array_position", +"language": "en" +} +--- + + + +## array_position + +### description + + Syntax + +`BIGINT array_position(ARRAY arr, T value)` + +Returns a position/index of first occurrence of the `value` in the given array. + +``` +position - value
[GitHub] [incubator-doris] jacktengg opened a new pull request, #9771: [improvement] Show detail status code string for BE http api
jacktengg opened a new pull request, #9771: URL: https://github.com/apache/incubator-doris/pull/9771 # Proposed changes Issue Number: close #xxx ## Problem Summary: Failure message for BE http api does not show detail status code: ``` "status": "Fail", "msg": "" } ``` Improved to: ``` { "status": "Not found", "msg": "Tablet not found. tablet_id=1202" } ``` ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] Gabriel39 opened a new pull request, #9773: [BUG] fix correctness for vectorized compaction
Gabriel39 opened a new pull request, #9773: URL: https://github.com/apache/incubator-doris/pull/9773 # Proposed changes Issue Number: close #9766 ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #9770: [fix] disabel transfer data large than 2GB
github-actions[bot] commented on PR #9770: URL: https://github.com/apache/incubator-doris/pull/9770#issuecomment-1136976048 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee opened a new pull request, #9772: [Opt][VecLoad] Opt the vec stream load performance
HappenLee opened a new pull request, #9772: URL: https://github.com/apache/incubator-doris/pull/9772 # Proposed changes Issue Number: close #xxx ## Problem Summary: Reduce unless virtual function call and switch case to improve load performance of stream load. 1. Test table: ``` CREATE TABLE `test1` ( `id` int(11) NOT NULL COMMENT "", `k1` varchar(32) NOT NULL COMMENT "", `k2` varchar(128) NOT NULL COMMENT "", `m` decimal(10, 3) NULL DEFAULT "10.5" COMMENT "", `d` date NULL DEFAULT "2014-02-04" COMMENT "", `e` datetime NULL DEFAULT "2014-02-04 15:36:00" COMMENT "", `f` boolean NOT NULL COMMENT "", INDEX index_id (`id`) USING BITMAP COMMENT '' ) ENGINE=OLAP DUPLICATE KEY(`id`, `k1`) COMMENT "OLAP" DISTRIBUTED BY HASH(`id`) BUCKETS 16 PROPERTIES ( "replication_allocation" = "tag.location.default: 1", "in_memory" = "false", "storage_format" = "V2" ) ``` 2. Test data (1kw) ``` 1,VOtWRdM0vqgMRD4g,QuSHANt7mA0hcsE8O6ggzOMOSLWtmfDs,99.6441922075,1982-11-10,1982-11-10 02:57:53,1 ,82N10yffLWGu6WM9,zDm0ekBEoYyfFfGGovRgp2hySHw3r31Y,3.9179255308,1992-06-13,1992-06-13 21:54:32,1 9998,To1UUHYi5uy0m0g7,HGihOdi9OyH5LawdgbcgOsGxTaH0du1H,24.4223120552,2007-04-01,2007-04-01 05:15:26,0 9997,zGaXQadwfnchbvhc,XtkA3KuKgK0BuGQONGx4mkZnbm54grSP,47.5443607568,2010-04-26,2010-04-26 03:54:28,0 ``` # 3. opt resutl | origin | opt | | | | | 25s | 19s | ## Checklist(Required) 1. Does it affect the original behavior: (No/) 2. Has unit tests been added: (No Need) 3. Has document been added or modified: (No Need) 4. Does it need to update dependencies: (No) 5. Are there any changes that cannot be rolled back: (Yes) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #9770: [fix] disabel transfer data large than 2GB
github-actions[bot] commented on PR #9770: URL: https://github.com/apache/incubator-doris/pull/9770#issuecomment-1136975953 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei commented on a diff in pull request #9663: [Enhancement]Add prefetch in join node
yiguolei commented on code in PR #9663: URL: https://github.com/apache/incubator-doris/pull/9663#discussion_r881422957 ## be/src/vec/exec/join/vhash_join_node.cpp: ## @@ -77,18 +81,44 @@ struct ProcessHashTableBuild { inserted_rows.reserve(_batch_size); } +std::vector cached_hash_values(CACHE_NUM); +std::vector cached_key_holders(CACHE_NUM); + +size_t cached_count = 0; for (size_t k = 0; k < _rows; ++k) { +if (UNLIKELY(k % CACHE_NUM == 0)) { +cached_count = std::min(CACHE_NUM, _rows - k); +for (size_t i = k; i < _rows && i < (k + CACHE_NUM); ++i) { Review Comment: for (size_t i = 0; i < cache_count; ++i) { if constexpr (ignore_null) { if ((*null_map)[i+k]) { if ((*null_map)[i+k]) { continue; } } cached_key_holders[i ] = key_getter.get_key_holder(i + k, _join_node->_arena); cached_hash_values[i ] = key_getter.get_hash( hash_table_ctx.hash_table, cached_key_holders[i ]); } -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei commented on a diff in pull request #9663: [Enhancement]Add prefetch in join node
yiguolei commented on code in PR #9663: URL: https://github.com/apache/incubator-doris/pull/9663#discussion_r881422957 ## be/src/vec/exec/join/vhash_join_node.cpp: ## @@ -77,18 +81,44 @@ struct ProcessHashTableBuild { inserted_rows.reserve(_batch_size); } +std::vector cached_hash_values(CACHE_NUM); +std::vector cached_key_holders(CACHE_NUM); + +size_t cached_count = 0; for (size_t k = 0; k < _rows; ++k) { +if (UNLIKELY(k % CACHE_NUM == 0)) { +cached_count = std::min(CACHE_NUM, _rows - k); +for (size_t i = k; i < _rows && i < (k + CACHE_NUM); ++i) { Review Comment: for (size_t i = 0; i < cache_count; ++i) { if constexpr (ignore_null) { if ((*null_map)[i+k]) { if ((*null_map)[i+k]) { continue; } } cached_key_holders[i ] = key_getter.get_key_holder(i + k, _join_node->_arena); cached_hash_values[i ] = key_getter.get_hash( hash_table_ctx.hash_table, cached_key_holders[i ]); } -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei commented on a diff in pull request #9663: [Enhancement]Add prefetch in join node
yiguolei commented on code in PR #9663: URL: https://github.com/apache/incubator-doris/pull/9663#discussion_r881422957 ## be/src/vec/exec/join/vhash_join_node.cpp: ## @@ -77,18 +81,44 @@ struct ProcessHashTableBuild { inserted_rows.reserve(_batch_size); } +std::vector cached_hash_values(CACHE_NUM); +std::vector cached_key_holders(CACHE_NUM); + +size_t cached_count = 0; for (size_t k = 0; k < _rows; ++k) { +if (UNLIKELY(k % CACHE_NUM == 0)) { +cached_count = std::min(CACHE_NUM, _rows - k); +for (size_t i = k; i < _rows && i < (k + CACHE_NUM); ++i) { Review Comment: for (size_t i = 0; i < cache_count; ++i) { if constexpr (ignore_null) { if ((*null_map)[i+k]) { if ((*null_map)[i+k]) { continue; } } cached_key_holders[i ] = key_getter.get_key_holder(i + k, _join_node->_arena); cached_hash_values[i ] = key_getter.get_hash( hash_table_ctx.hash_table, cached_key_holders[i ]); } -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] Gabriel39 opened a new pull request, #9774: [Improvement] remove unused code in vectorized compaction
Gabriel39 opened a new pull request, #9774: URL: https://github.com/apache/incubator-doris/pull/9774 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] xiepengcheng01 commented on a diff in pull request #9772: [Opt][VecLoad] Opt the vec stream load performance
xiepengcheng01 commented on code in PR #9772: URL: https://github.com/apache/incubator-doris/pull/9772#discussion_r881432756 ## be/src/vec/exec/vbroker_scanner.cpp: ## @@ -20,11 +20,11 @@ #include #include -#include #include "exec/exec_node.h" #include "exec/plain_text_line_reader.h" #include "exec/text_converter.h" +#include "exec/text_converter.hpp" Review Comment: can remove `exec/text_converter.h` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #9763: [security] update canal version to fix fastjson security issue
github-actions[bot] commented on PR #9763: URL: https://github.com/apache/incubator-doris/pull/9763#issuecomment-1137055444 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #9763: [security] update canal version to fix fastjson security issue
github-actions[bot] commented on PR #9763: URL: https://github.com/apache/incubator-doris/pull/9763#issuecomment-1137055487 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #9763: [security] update canal version to fix fastjson security issue
yangzhg merged PR #9763: URL: https://github.com/apache/incubator-doris/pull/9763 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [security] update canal version to fix fastjson security issue (#9763)
This is an automated email from the ASF dual-hosted git repository. yangzhg pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new be026addde [security] update canal version to fix fastjson security issue (#9763) be026addde is described below commit be026adddee8ad9154bd81e34141f3f075739edc Author: Zhengguo Yang AuthorDate: Wed May 25 18:22:37 2022 +0800 [security] update canal version to fix fastjson security issue (#9763) --- fe/fe-core/pom.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fe/fe-core/pom.xml b/fe/fe-core/pom.xml index 009cf9df90..b975bdafdd 100644 --- a/fe/fe-core/pom.xml +++ b/fe/fe-core/pom.xml @@ -431,7 +431,7 @@ under the License. com.alibaba.otter canal.client -1.1.4 +1.1.6 ch.qos.logback @@ -447,7 +447,7 @@ under the License. com.alibaba.otter canal.protocol -1.1.4 +1.1.6 ch.qos.logback - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] jackwener opened a new issue, #9775: [Enhancement] Remove the redundant `this.`
jackwener opened a new issue, #9775: URL: https://github.com/apache/incubator-doris/issues/9775 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description Remove uncessary `this.` Most scenario, it just use in `constructor`. ### Solution _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] stalary commented on pull request #9694: [Bug] Fix create table like when having hidden columns
stalary commented on PR #9694: URL: https://github.com/apache/incubator-doris/pull/9694#issuecomment-1137075942 LGTM @yiguolei -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #9770: [fix] disabel transfer data large than 2GB
yangzhg merged PR #9770: URL: https://github.com/apache/incubator-doris/pull/9770 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [fix] disable transfer data large than 2GB by brpc (#9770)
This is an automated email from the ASF dual-hosted git repository. yangzhg pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new f5bef328fe [fix] disable transfer data large than 2GB by brpc (#9770) f5bef328fe is described below commit f5bef328fe4c4701bb3eccfb02f992d68c6fa5ae Author: Zhengguo Yang AuthorDate: Wed May 25 18:41:13 2022 +0800 [fix] disable transfer data large than 2GB by brpc (#9770) because of brpc and protobuf cannot transfer data large than 2GB, if large than 2GB will overflow, so add a check before send --- be/src/runtime/row_batch.cpp | 11 ++- be/src/vec/core/block.cpp| 35 --- 2 files changed, 30 insertions(+), 16 deletions(-) diff --git a/be/src/runtime/row_batch.cpp b/be/src/runtime/row_batch.cpp index 520a5642f9..5acf7634de 100644 --- a/be/src/runtime/row_batch.cpp +++ b/be/src/runtime/row_batch.cpp @@ -273,10 +273,6 @@ Status RowBatch::serialize(PRowBatch* output_batch, size_t* uncompressed_size, try { // Allocation of extra-long contiguous memory may fail, and data compression cannot be used if it fails _compression_scratch.resize(max_compressed_size); -} catch (const std::bad_alloc& e) { -can_compress = false; -LOG(WARNING) << "Try to alloc " << max_compressed_size - << " bytes for compression scratch failed. " << e.what(); } catch (...) { can_compress = false; std::exception_ptr p = std::current_exception(); @@ -309,11 +305,8 @@ Status RowBatch::serialize(PRowBatch* output_batch, size_t* uncompressed_size, *compressed_size = pb_size; if (pb_size > std::numeric_limits::max()) { // the protobuf has a hard limit of 2GB for serialized data. -return Status::InternalError( -fmt::format("The rowbatch is large than 2GB({}), can not send by Protobuf. " -"please set BE config 'transfer_data_by_brpc_attachment' to true " -"and restart BE.", -pb_size)); +return Status::InternalError(fmt::format( +"The rowbatch is large than 2GB({}), can not send by Protobuf.", pb_size)); } } else { *uncompressed_size = pb_size + tuple_byte_size; diff --git a/be/src/vec/core/block.cpp b/be/src/vec/core/block.cpp index aa482fcfbf..4866a1129e 100644 --- a/be/src/vec/core/block.cpp +++ b/be/src/vec/core/block.cpp @@ -669,7 +669,16 @@ Status Block::serialize(PBlock* pblock, size_t* uncompressed_bytes, size_t* comp // serialize data values // when data type is HLL, content_uncompressed_size maybe larger than real size. -allocated_buf->resize(content_uncompressed_size); +try { +allocated_buf->resize(content_uncompressed_size); +} catch (...) { +std::exception_ptr p = std::current_exception(); +std::string msg = fmt::format("Try to alloc {} bytes for allocated_buf failed. reason {}", + content_uncompressed_size, + p ? p.__cxa_exception_type()->name() : "null"); +LOG(WARNING) << msg; +return Status::BufferAllocFailed(msg); +} char* buf = allocated_buf->data(); for (const auto& c : *this) { buf = c.type->serialize(*(c.column), buf); @@ -678,12 +687,21 @@ Status Block::serialize(PBlock* pblock, size_t* uncompressed_bytes, size_t* comp // compress if (config::compress_rowbatches && content_uncompressed_size > 0) { -// Try compressing the content to compression_scratch, -// swap if compressed data is smaller +size_t max_compressed_size = snappy::MaxCompressedLength(content_uncompressed_size); std::string compression_scratch; -uint32_t max_compressed_size = snappy::MaxCompressedLength(content_uncompressed_size); -compression_scratch.resize(max_compressed_size); - +try { +// Try compressing the content to compression_scratch, +// swap if compressed data is smaller +// Allocation of extra-long contiguous memory may fail, and data compression cannot be used if it fails +compression_scratch.resize(max_compressed_size); +} catch (...) { +std::exception_ptr p = std::current_exception(); +std::string msg = +fmt::format("Try to alloc {} bytes for compression scratch failed. reason {}", +max_compressed_size, p ? p.__cxa_exception_type()->name() : "null"); +LOG(WARNING) << msg; +return Status::BufferAllocFailed(msg); +} size_t compressed_size = 0; char* compressed_output = compression_
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #9774: [Improvement] remove unused code in vectorized compaction
github-actions[bot] commented on PR #9774: URL: https://github.com/apache/incubator-doris/pull/9774#issuecomment-1137086006 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] LOVEGISER opened a new pull request, #9776: website document update
LOVEGISER opened a new pull request, #9776: URL: https://github.com/apache/incubator-doris/pull/9776 # Proposed changes website document update ## Checklist(Required) 1. Does it affect the original behavior: (No) 2. Has unit tests been added: (No Need) 3. Has document been added or modified: (Yes) 4. Does it need to update dependencies: (No) 5. Are there any changes that cannot be rolled back: (Yes) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] pengxiangyu opened a new issue, #9777: [Feature] Add cancel export operation
pengxiangyu opened a new issue, #9777: URL: https://github.com/apache/incubator-doris/issues/9777 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description _No response_ ### Use case _No response_ ### Related issues _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #9773: [BUG] fix correctness for vectorized compaction
github-actions[bot] commented on PR #9773: URL: https://github.com/apache/incubator-doris/pull/9773#issuecomment-1137119322 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei merged pull request #9743: [Improvement] fix typo
yiguolei merged PR #9743: URL: https://github.com/apache/incubator-doris/pull/9743 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [Improvement] fix typo (#9743)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 8470543144 [Improvement] fix typo (#9743) 8470543144 is described below commit 8470543144f46ff92fe7ef4364291a9709b06fe7 Author: Gabriel AuthorDate: Wed May 25 19:29:01 2022 +0800 [Improvement] fix typo (#9743) --- be/src/vec/core/types.h| 2 +- .../vec/function/function_array_element_test.cpp | 20 ++--- be/test/vec/function/function_array_index_test.cpp | 16 ++-- be/test/vec/function/function_ifnull_test.cpp | 4 +- be/test/vec/function/function_nullif_test.cpp | 4 +- be/test/vec/function/function_string_test.cpp | 8 +- be/test/vec/function/function_test_util.cpp| 2 +- be/test/vec/function/function_test_util.h | 2 +- be/test/vec/function/function_time_test.cpp| 94 +++--- 9 files changed, 76 insertions(+), 76 deletions(-) diff --git a/be/src/vec/core/types.h b/be/src/vec/core/types.h index 0d70204e14..3f4d4317de 100644 --- a/be/src/vec/core/types.h +++ b/be/src/vec/core/types.h @@ -138,7 +138,7 @@ struct TypeName { }; template <> struct TypeName { -static const char* get() { return "SringValue"; } +static const char* get() { return "StringValue"; } }; template <> diff --git a/be/test/vec/function/function_array_element_test.cpp b/be/test/vec/function/function_array_element_test.cpp index 39d9c23cab..1c1be231f6 100644 --- a/be/test/vec/function/function_array_element_test.cpp +++ b/be/test/vec/function/function_array_element_test.cpp @@ -90,13 +90,13 @@ TEST(function_array_element_test, element_at) { { InputTypeSet input_types = {TypeIndex::Array, TypeIndex::DateTime, TypeIndex::Int64}; -Array vec = {str_to_data_time("2022-01-02 01:00:00"), str_to_data_time(""), - str_to_data_time("2022-07-08 03:00:00")}; +Array vec = {str_to_date_time("2022-01-02 01:00:00"), str_to_date_time(""), + str_to_date_time("2022-07-08 03:00:00")}; DataSet data_set = {{{vec, Int64(0)}, Null()}, -{{vec, Int64(1)}, str_to_data_time("2022-01-02 01:00:00")}, +{{vec, Int64(1)}, str_to_date_time("2022-01-02 01:00:00")}, {{vec, Int64(4)}, Null()}, -{{vec, Int64(-1)}, str_to_data_time("2022-07-08 03:00:00")}, -{{vec, Int64(-2)}, str_to_data_time("")}, +{{vec, Int64(-1)}, str_to_date_time("2022-07-08 03:00:00")}, +{{vec, Int64(-2)}, str_to_date_time("")}, {{vec, Int64(-4)}, Null()}, {{Null(), Int64(1)}, Null()}, {{empty_arr, Int64(0)}, Null()}, @@ -109,13 +109,13 @@ TEST(function_array_element_test, element_at) { { InputTypeSet input_types = {TypeIndex::Array, TypeIndex::Date, TypeIndex::Int64}; -Array vec = {str_to_data_time("2022-01-02"), str_to_data_time(""), - str_to_data_time("2022-07-08")}; +Array vec = {str_to_date_time("2022-01-02"), str_to_date_time(""), + str_to_date_time("2022-07-08")}; DataSet data_set = {{{vec, Int64(0)}, Null()}, -{{vec, Int64(1)}, str_to_data_time("2022-01-02")}, +{{vec, Int64(1)}, str_to_date_time("2022-01-02")}, {{vec, Int64(4)}, Null()}, -{{vec, Int64(-1)}, str_to_data_time("2022-07-08")}, -{{vec, Int64(-2)}, str_to_data_time("")}, +{{vec, Int64(-1)}, str_to_date_time("2022-07-08")}, +{{vec, Int64(-2)}, str_to_date_time("")}, {{vec, Int64(-4)}, Null()}, {{Null(), Int64(1)}, Null()}, {{empty_arr, Int64(0)}, Null()}, diff --git a/be/test/vec/function/function_array_index_test.cpp b/be/test/vec/function/function_array_index_test.cpp index c1a0231a7d..6584c69a75 100644 --- a/be/test/vec/function/function_array_index_test.cpp +++ b/be/test/vec/function/function_array_index_test.cpp @@ -113,8 +113,8 @@ TEST(function_array_index_test, array_contains) { { InputTypeSet input_types = {TypeIndex::Array, TypeIndex::Date, TypeIndex::Date}; -Array vec = {str_to_data_time("2022-01-02", false), str_to_data_time("", false), - str_to_data_time("2022-07-08", false)}; +Array vec = {str_to_date_time("2022-01-02", false), str_to_date_time("", false), + str_to_date_time("2022-07-08", false)}; DataSet data_set = {{{vec, std::string("2022-01-02")
[GitHub] [incubator-doris] Gabriel39 commented on a diff in pull request #9582: [refactor](decimalv3) optimize decimal performance and precision
Gabriel39 commented on code in PR #9582: URL: https://github.com/apache/incubator-doris/pull/9582#discussion_r881540625 ## be/src/vec/functions/function_binary_arithmetic.h: ## @@ -449,6 +469,18 @@ struct DecimalBinaryOperation { private: /// there's implicit type convertion here static NativeResultType apply(NativeResultType a, NativeResultType b) { +if (config::enable_execution_decimalv3) { +if constexpr (OpTraits::can_overflow && check_overflow) { +NativeResultType res; +if (Op::template apply(a, b, res)) { +LOG(FATAL) << "Decimal math overflow"; Review Comment: how about to return null when overflow? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] stalary opened a new pull request, #9778: [Bug] sql block rule npe
stalary opened a new pull request, #9778: URL: https://github.com/apache/incubator-doris/pull/9778 # Proposed changes Issue Number: close https://github.com/apache/incubator-doris/issues/9749 ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei commented on a diff in pull request #9771: [improvement] Show detail status code string for BE http api
yiguolei commented on code in PR #9771: URL: https://github.com/apache/incubator-doris/pull/9771#discussion_r881545905 ## be/src/util/json_util.cpp: ## @@ -32,11 +32,7 @@ std::string to_json(const Status& status) { writer.StartObject(); // status writer.Key("status"); -if (status.ok()) { Review Comment: Hi jack, I think we could move to_json(Status& status) method to common/status.h since there is only one method in this file and it is related with common/status. And also you could remove json_util_test.h because we could not ensure that the error message always the same with the expected string. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] liaoxin01 commented on a diff in pull request #9582: [refactor](decimalv3) optimize decimal performance and precision
liaoxin01 commented on code in PR #9582: URL: https://github.com/apache/incubator-doris/pull/9582#discussion_r881554793 ## be/src/vec/functions/function_binary_arithmetic.h: ## @@ -449,6 +469,18 @@ struct DecimalBinaryOperation { private: /// there's implicit type convertion here static NativeResultType apply(NativeResultType a, NativeResultType b) { +if (config::enable_execution_decimalv3) { +if constexpr (OpTraits::can_overflow && check_overflow) { +NativeResultType res; +if (Op::template apply(a, b, res)) { +LOG(FATAL) << "Decimal math overflow"; Review Comment: The column may not be ColumnNullable, so we can't return null. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] pengxiangyu opened a new pull request, #9779: Add backend_rpc_timeout_second in AdminConfig
pengxiangyu opened a new pull request, #9779: URL: https://github.com/apache/incubator-doris/pull/9779 # Proposed changes Issue Number: close #xxx ## Problem Summary: Add backend_rpc_timeout_second in AdminConfig ## Checklist(Required) 1. Does it affect the original behavior: (No) 2. Has unit tests been added: (No) 3. Has document been added or modified: (No) 4. Does it need to update dependencies: (No) 5. Are there any changes that cannot be rolled back: (No) ## Further comments -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] xxiao2018 commented on pull request #8730: [Enhancement] enhancement for regression-test framework
xxiao2018 commented on PR #8730: URL: https://github.com/apache/incubator-doris/pull/8730#issuecomment-1137199570 fix in #8930 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #9753: [doc]Add Doris join optimization documentation
github-actions[bot] commented on PR #9753: URL: https://github.com/apache/incubator-doris/pull/9753#issuecomment-1137213800 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #9753: [doc]Add Doris join optimization documentation
github-actions[bot] commented on PR #9753: URL: https://github.com/apache/incubator-doris/pull/9753#issuecomment-1137213841 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #9761: [fix](help) fix bug of help command
github-actions[bot] commented on PR #9761: URL: https://github.com/apache/incubator-doris/pull/9761#issuecomment-1137217678 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #9761: [fix](help) fix bug of help command
github-actions[bot] commented on PR #9761: URL: https://github.com/apache/incubator-doris/pull/9761#issuecomment-1137217599 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee merged pull request #9459: [stream-load-vec]: memtable flush only if necessary after aggregated
HappenLee merged PR #9459: URL: https://github.com/apache/incubator-doris/pull/9459 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee closed issue #8656: [Enhancement] [Vectorized] optimize pre-agg in stream load memtable
HappenLee closed issue #8656: [Enhancement] [Vectorized] optimize pre-agg in stream load memtable URL: https://github.com/apache/incubator-doris/issues/8656 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [stream-load-vec]: memtable flush only if necessary after aggregated (#9459)
This is an automated email from the ASF dual-hosted git repository. lihaopeng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 73e31a2179 [stream-load-vec]: memtable flush only if necessary after aggregated (#9459) 73e31a2179 is described below commit 73e31a21799c15abde903502ad5b73138b01c336 Author: spaces-x AuthorDate: Wed May 25 21:12:24 2022 +0800 [stream-load-vec]: memtable flush only if necessary after aggregated (#9459) Co-authored-by: weixiang --- be/src/common/config.h | 3 +++ be/src/olap/delta_writer.cpp | 9 --- be/src/olap/memtable.cpp | 56 +++- be/src/olap/memtable.h | 10 +++- be/src/vec/core/block.cpp| 19 +++ be/src/vec/core/block.h | 7 ++ 6 files changed, 89 insertions(+), 15 deletions(-) diff --git a/be/src/common/config.h b/be/src/common/config.h index fcfc2e3f78..a73efcf24d 100644 --- a/be/src/common/config.h +++ b/be/src/common/config.h @@ -464,6 +464,9 @@ CONF_Int32(memory_max_alignment, "16"); // write buffer size before flush CONF_mInt64(write_buffer_size, "209715200"); +// max buffer size used in memtable for the aggregated table +CONF_mInt64(memtable_max_buffer_size, "419430400"); + // following 2 configs limit the memory consumption of load process on a Backend. // eg: memory limit to 80% of mem limit config but up to 100GB(default) // NOTICE(cmy): set these default values very large because we don't want to diff --git a/be/src/olap/delta_writer.cpp b/be/src/olap/delta_writer.cpp index 11ee242449..fe0ffeddbb 100644 --- a/be/src/olap/delta_writer.cpp +++ b/be/src/olap/delta_writer.cpp @@ -203,9 +203,12 @@ Status DeltaWriter::write(const vectorized::Block* block, const std::vector } } -if (_mem_table->memory_usage() >= config::write_buffer_size) { -RETURN_NOT_OK(_flush_memtable_async()); -_reset_mem_table(); +if (_mem_table->need_to_agg()) { +_mem_table->shrink_memtable_by_agg(); +if (_mem_table->is_flush()) { +RETURN_NOT_OK(_flush_memtable_async()); +_reset_mem_table(); +} } return Status::OK(); diff --git a/be/src/olap/memtable.cpp b/be/src/olap/memtable.cpp index c7f94ead4f..cf58f7e4ba 100644 --- a/be/src/olap/memtable.cpp +++ b/be/src/olap/memtable.cpp @@ -126,11 +126,10 @@ void MemTable::insert(const vectorized::Block* block, size_t row_pos, size_t num } } size_t cursor_in_mutableblock = _input_mutable_block.rows(); -size_t oldsize = _input_mutable_block.allocated_bytes(); _input_mutable_block.add_rows(block, row_pos, num_rows); -size_t newsize = _input_mutable_block.allocated_bytes(); -_mem_usage += newsize - oldsize; -_mem_tracker->consume(newsize - oldsize); +size_t input_size = block->allocated_bytes() * num_rows / block->rows(); +_mem_usage += input_size; +_mem_tracker->consume(input_size); for (int i = 0; i < num_rows; i++) { _row_in_blocks.emplace_back(new RowInBlock {cursor_in_mutableblock + i}); @@ -242,7 +241,8 @@ void MemTable::_aggregate_two_row_in_block(RowInBlock* new_row, RowInBlock* row_ new_row->_row_pos, nullptr); } } -vectorized::Block MemTable::_collect_vskiplist_results() { +template +void MemTable::_collect_vskiplist_results() { VecTable::Iterator it(_vec_skip_list.get()); vectorized::Block in_block = _input_mutable_block.to_block(); // TODO: should try to insert data by column, not by row. to opt the code @@ -251,6 +251,7 @@ vectorized::Block MemTable::_collect_vskiplist_results() { _output_mutable_block.add_row(&in_block, it.key()->_row_pos); } } else { +size_t idx = 0; for (it.SeekToFirst(); it.Valid(); it.Next()) { auto& block_data = in_block.get_columns_with_type_and_name(); // move key columns @@ -263,11 +264,46 @@ vectorized::Block MemTable::_collect_vskiplist_results() { auto function = _agg_functions[i]; function->insert_result_into(it.key()->_agg_places[i], *(_output_mutable_block.get_column_by_position(i))); -function->destroy(it.key()->_agg_places[i]); +if constexpr (is_final) { +function->destroy(it.key()->_agg_places[i]); +} } +if constexpr (!is_final) { +// re-index the row_pos in VSkipList +it.key()->_row_pos = idx; +idx++; +} +} +if constexpr (!is_final) { +// if is not final, we collect the agg results to input_block and then continue to insert +size_t shrunked_after_agg = _output_mutable_block.allocated_bytes(); +
[GitHub] [incubator-doris] stalary opened a new issue, #9780: [Feature] Support Col Policy
stalary opened a new issue, #9780: URL: https://github.com/apache/incubator-doris/issues/9780 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description Support Col Policy, similar to row policy. ### Use case _No response_ ### Related issues _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #9776: [optimiz] website document update
github-actions[bot] commented on PR #9776: URL: https://github.com/apache/incubator-doris/pull/9776#issuecomment-1137277358 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #9776: [optimiz] website document update
github-actions[bot] commented on PR #9776: URL: https://github.com/apache/incubator-doris/pull/9776#issuecomment-1137277475 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a diff in pull request #9779: Add backend_rpc_timeout_second in AdminConfig
morningman commented on code in PR #9779: URL: https://github.com/apache/incubator-doris/pull/9779#discussion_r881765585 ## fe/fe-core/src/main/java/org/apache/doris/common/Config.java: ## @@ -1651,4 +1651,8 @@ public class Config extends ConfigBase { @ConfField public static boolean enable_vectorized_load = false; + +@ConfField(mutable = false, masterOnly = true) +public static int backend_rpc_timeout_ms = 6; // 1 min Review Comment: Please add comment for this config and also modify the config document `admin-manual/config/fe-config.md` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #9728: [Bug] If function adds type inference
github-actions[bot] commented on PR #9728: URL: https://github.com/apache/incubator-doris/pull/9728#issuecomment-1137433042 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] JNSimba commented on issue #9713: [Bug] doris-flink 连接器不能正常运行
JNSimba commented on issue #9713: URL: https://github.com/apache/incubator-doris/issues/9713#issuecomment-1137435681 Does doris-flink-1.0-SNAPSHOT.jar also depend on flink1.13? Then confirm whether the scala versions on both sides are the same? Also, you can use the jar package of the flink doris connector in the maven repository -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #9465: [feature-wip][array-type] Support more sub types.
morningman closed issue #9465: [feature-wip][array-type] Support more sub types. URL: https://github.com/apache/incubator-doris/issues/9465 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #9466: [feature-wip][array-type] Support more sub types.
morningman merged PR #9466: URL: https://github.com/apache/incubator-doris/pull/9466 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [feature-wip][array-type] Support more sub types. (#9466)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 2a11a4ab99 [feature-wip][array-type] Support more sub types. (#9466) 2a11a4ab99 is described below commit 2a11a4ab99206ebf9cf6c62c885088b8ead5bc05 Author: Adonis Ling AuthorDate: Thu May 26 08:41:34 2022 +0800 [feature-wip][array-type] Support more sub types. (#9466) Please refer to #9465 --- be/src/exprs/array_functions.cpp | 5 +- be/src/exprs/literal.cpp | 13 +- be/src/olap/aggregate_func.cpp | 16 +- be/src/olap/rowset/segment_v2/segment_writer.cpp | 2 +- be/src/olap/types.h| 15 +- be/src/runtime/collection_value.cpp| 636 - be/src/runtime/collection_value.h | 176 -- be/src/runtime/mysql_result_writer.cpp | 22 +- be/src/runtime/raw_value.cpp | 9 +- be/src/runtime/row_batch.cpp | 8 +- be/src/runtime/tuple.h | 2 +- be/src/udf/udf.h | 2 +- be/src/util/array_parser.h | 65 ++- be/src/vec/core/block.cpp | 28 +- be/src/vec/olap/olap_data_convertor.cpp| 266 - be/src/vec/olap/olap_data_convertor.h | 85 +-- be/src/vec/sink/mysql_result_writer.cpp| 12 + be/test/exprs/array_functions_test.cpp | 2 +- be/test/runtime/array_test.cpp | 417 ++ be/test/runtime/collection_value_test.cpp | 13 +- be/test/testutil/array_utils.cpp | 26 + be/test/testutil/array_utils.h | 2 +- be/test/util/array_parser_test.cpp | 54 ++ .../java/org/apache/doris/analysis/TypeDef.java| 4 - .../java/org/apache/doris/catalog/ArrayType.java | 3 + 25 files changed, 1207 insertions(+), 676 deletions(-) diff --git a/be/src/exprs/array_functions.cpp b/be/src/exprs/array_functions.cpp index b9b7fcf8a4..be9f151b22 100644 --- a/be/src/exprs/array_functions.cpp +++ b/be/src/exprs/array_functions.cpp @@ -30,8 +30,9 @@ void ArrayFunctions::init() {} DCHECK_EQ(context->get_return_type().children.size(), 1); \ CollectionValue v; \ CollectionValue::init_collection(context, num_children, PRIMARY_TYPE, &v); \ -for (int i = 0; i < num_children; ++i) { \ -v.set(i, PRIMARY_TYPE, values + i); \ +auto iterator = v.iterator(PRIMARY_TYPE); \ +for (int i = 0; i < num_children; ++i, iterator.next()) { \ +iterator.set(values + i); \ } \ CollectionVal ret; \ v.to_collection_val(&ret); \ diff --git a/be/src/exprs/literal.cpp b/be/src/exprs/literal.cpp index 5e1bbb9846..64401b7fe2 100644 --- a/be/src/exprs/literal.cpp +++ b/be/src/exprs/literal.cpp @@ -183,13 +183,14 @@ Status Literal::prepare(RuntimeState* state, const RowDescriptor& row_desc, Expr if (type().type == TYPE_ARRAY) { DCHECK_EQ(type().children.size(), 1) << "array children type not 1"; // init array value -auto td = type().children.at(0).type; -RETURN_IF_ERROR(CollectionValue::init_collection(state->obj_pool(), get_num_children(), td, - &_value.array_val)); +auto child_type = type().children.at(0).type; +RETURN_IF_ERROR(CollectionValue::init_collection(state->obj_pool(), get_num_children(), + child_type, &_value.array_val)); +auto iterator = _value.array_val.iterator(child_type); // init every item -for (int i = 0; i < get_num_children(); ++i) { -Expr* children = get_child(i); -RETURN_IF_ERROR(_value.array_val.set(i, td, children->get_const_val(context))); +for (int i = 0; i < get_num_children() && iterator.has_next(); ++i, iterator.next()) { +Expr* child = get_child(i); +iterator.set(child->get_const_val(context)); } } diff --git a/be/src/olap/aggregate_func.cpp b/be/src/olap/aggregate_func.cpp index 4d583d476b..781ab39082 100644 --- a/be/src/olap/aggregate_func.cpp +++ b/be/src/olap/aggregate_func.cpp @@ -105,6 +105,8
[GitHub] [incubator-doris] morningman merged pull request #9492: [fix](resource-tag) Consider resource tags when assigning tasks for broker & routine load
morningman merged PR #9492: URL: https://github.com/apache/incubator-doris/pull/9492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [fix](resource-tag) Consider resource tags when assigning tasks for broker & routine load (#9492)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 0c70359404 [fix](resource-tag) Consider resource tags when assigning tasks for broker & routine load (#9492) 0c70359404 is described below commit 0c70359404f7b4ecfb53ec17c994d0a3a6e5c08a Author: Mingyu Chen AuthorDate: Thu May 26 08:42:09 2022 +0800 [fix](resource-tag) Consider resource tags when assigning tasks for broker & routine load (#9492) This CL mainly changes: 1. Broker Load When assigning backends, use user level resource tag to find available backends. If user level resource tag is not set, broker load task can be assigned to any BE node, otherwise, task can only be assigned to BE node which match the user level tags. 2. Routine Load The current routine load job does not have user info, so it can not get user level tag when assigning tasks. So there are 2 ways: 1. For old routine load job, use tags of replica allocation info to select BE nodes. 2. For new routine load job, the user info will be added and persisted in routine load job. --- docs/en/admin-manual/multi-tenant.md | 12 docs/zh-CN/admin-manual/multi-tenant.md| 14 +++- .../org/apache/doris/common/FeMetaVersion.java | 6 +- .../doris/load/loadv2/LoadingTaskPlanner.java | 2 +- .../load/routineload/KafkaRoutineLoadJob.java | 8 ++- .../doris/load/routineload/RoutineLoadJob.java | 27 +++- .../doris/load/routineload/RoutineLoadManager.java | 74 -- .../load/routineload/RoutineLoadTaskScheduler.java | 3 +- .../mysql/privilege/UserResourceProperty.java | 31 - .../org/apache/doris/planner/BrokerScanNode.java | 45 + .../load/routineload/KafkaRoutineLoadJobTest.java | 26 .../doris/load/routineload/RoutineLoadJobTest.java | 16 +++-- .../load/routineload/RoutineLoadManagerTest.java | 32 ++ .../load/routineload/RoutineLoadSchedulerTest.java | 15 +++-- .../transaction/GlobalTransactionMgrTest.java | 13 ++-- 15 files changed, 223 insertions(+), 101 deletions(-) diff --git a/docs/en/admin-manual/multi-tenant.md b/docs/en/admin-manual/multi-tenant.md index fb121fa191..41663a1e4f 100644 --- a/docs/en/admin-manual/multi-tenant.md +++ b/docs/en/admin-manual/multi-tenant.md @@ -133,6 +133,18 @@ Node resource division refers to setting tags for BE nodes in a Doris cluster, a In this way, we have achieved physical resource isolation for different user queries by dividing nodes and restricting user resource usage. Furthermore, we can create different users for different business departments and restrict each user from using different resource groups. In order to avoid the use of resource interference between different business parts. For example, there is a business table in the cluster that needs to be shared by all 9 business departments, but it is hoped [...] On the other hand, for the isolation of online and offline tasks. We can use resource groups to achieve this. For example, we can divide nodes into two resource groups, Online and Offline. The table data is still stored in 3 copies, of which 2 copies are stored in the Online resource group, and 1 copy is stored in the Offline resource group. The Online resource group is mainly used for online data services with high concurrency and low latency. Some large queries or offline ETL opera [...] + +4. Resource group assignments for load job + +The resource usage of load jobs (including insert, broker load, routine load, stream load, etc.) can be divided into two parts: +1. Computing resources: responsible for reading data sources, data transformation and distribution. +2. Write resource: responsible for data encoding, compression and writing to disk. + +The write resource must be the node where the replica is located, and the computing resource can theoretically select any node to complete. Therefore, the allocation of resource groups for load jobs is divided into two steps: +1. Use user-level resource tags to limit the resource groups that computing resources can use. +2. Use the resource tag of the replica to limit the resource group that the write resource can use. + +So if you want all the resources used by the load operation to be limited to the resource group where the data is located, you only need to set the resource tag of the user level to the same as the resource tag of the replica. ## Single query resource limit diff --git a/docs/zh-CN/admin-manual/multi-tenant.md b/docs/zh-CN/admin-manual/multi-tenant.md index dadabd1a5a..acc1775c0a 100644 --- a/docs/zh-CN/admin-manual/multi-tenant.md +++ b/docs/zh-CN/admin-manual/multi-tenant.md
[GitHub] [incubator-doris] morningman commented on pull request #9708: [refactor][be] Optimize headers
morningman commented on PR #9708: URL: https://github.com/apache/incubator-doris/pull/9708#issuecomment-1138019571 Hi @adonis0147 please rebase to solve the conflict -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #9761: [fix](help) fix bug of help command
morningman merged PR #9761: URL: https://github.com/apache/incubator-doris/pull/9761 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [fix](help) fix bug of help command (#9761)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 32a210f426 [fix](help) fix bug of help command (#9761) 32a210f426 is described below commit 32a210f426f9422a971a1e0157a349c6f88c9dcd Author: Mingyu Chen AuthorDate: Thu May 26 08:44:00 2022 +0800 [fix](help) fix bug of help command (#9761) This bug is introduced from #9306, that user need to execute "help stream-load" to show the help doc. But actually, it should be "help stream load". --- fe/fe-core/src/main/java/org/apache/doris/common/MarkDownParser.java | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/common/MarkDownParser.java b/fe/fe-core/src/main/java/org/apache/doris/common/MarkDownParser.java index eced29b736..200c106003 100755 --- a/fe/fe-core/src/main/java/org/apache/doris/common/MarkDownParser.java +++ b/fe/fe-core/src/main/java/org/apache/doris/common/MarkDownParser.java @@ -166,7 +166,9 @@ public class MarkDownParser { } // Note that multiple line breaks at content's end will be merged to be one, // and other whitespace characters will be deleted. -return Maps.immutableEntry(key.substring(headLevel).trim(), +// Also, the header in md file is like "## STREAM-LOAD", we need to convert it to "STREAM LOAD", +// so that we can execute "help stream load" to show the help doc. +return Maps.immutableEntry(key.substring(headLevel).trim().replaceAll("-", " "), sb.toString().replaceAll("\\s+$", "\n")); } } - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] hf200012 commented on issue #9594: [Bug] PAUSE SYNC JOB 卡死
hf200012 commented on issue #9594: URL: https://github.com/apache/incubator-doris/issues/9594#issuecomment-1138036514 Judging from your prompt information, it is the connection between your client and the server that is disconnected. You don't need to restart fe, just reconnect it. I made more than a dozen tables, inserted data, loaded it through binlog, and repeated it. Pause , resume , stop did not reproduce your problem @yiyulinfeng -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] mrhhsg opened a new issue, #9781: [Enhancement] Need to ignore generated files in tools by updating .gitignore
mrhhsg opened a new issue, #9781: URL: https://github.com/apache/incubator-doris/issues/9781 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description Some generated files should be add ignored. ```bash $ git status On branch git_ignore Untracked files: (use "git add ..." to include in what will be committed) tools/ssb-tools/ssb-data/ tools/ssb-tools/ssb-dbgen/ tools/tpch-tools/TPC-H_Tools_v3.0.0.zip tools/tpch-tools/TPC-H_Tools_v3.0.0/ tools/tpch-tools/tpc-h_v3.0.0.docx tools/tpch-tools/tpc-h_v3.0.0.pdf tools/tpch-tools/tpch-data/ ``` ### Solution _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] mrhhsg opened a new pull request, #9782: Update .gitignore to ignore generated files in tools
mrhhsg opened a new pull request, #9782: URL: https://github.com/apache/incubator-doris/pull/9782 # Proposed changes Issue Number: close #9781 ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a diff in pull request #9753: [doc]Add Doris join optimization documentation
yangzhg commented on code in PR #9753: URL: https://github.com/apache/incubator-doris/pull/9753#discussion_r882257343 ## docs/zh-CN/advanced/join-optimization/doris-join-optimization.md: ## @@ -0,0 +1,226 @@ +--- +{ +"title": "Doris Join 优化原理", +"language": "zh-CN" +} + + +--- + + + +# Doris Join 优化原理 + +Doris 支持两种物理算子,一类是 **Hash Join**,另一类是 **Nest Loop Join**。 + Review Comment: doris doesn't really support `Nest Loop Join`, it just uses an Nest-Loop-Join-like algorithm for non-equivalent joins, so I think this will cause some insurmountable problems for join optimization -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a diff in pull request #9753: [doc]Add Doris join optimization documentation
yangzhg commented on code in PR #9753: URL: https://github.com/apache/incubator-doris/pull/9753#discussion_r882257444 ## docs/zh-CN/advanced/join-optimization/doris-join-optimization.md: ## @@ -0,0 +1,226 @@ +--- +{ +"title": "Doris Join 优化原理", +"language": "zh-CN" +} + + +--- + + + +# Doris Join 优化原理 + +Doris 支持两种物理算子,一类是 **Hash Join**,另一类是 **Nest Loop Join**。 + +- Hash Join:在右表上根据等值 Join 列建立哈希表,左表流式的利用哈希表进行 Join 计算,它的限制是只能适用于等值 Join。 +- Nest Loop Join:通过两个 for 循环,很直观。然后它适用的场景就是不等值的 Join,例如:大于小于或者是需要求笛卡尔积的场景。它是一个通用的 Join 算子,但是性能表现差。 + +作为分布式的 MPP 数据库, 在 Join 的过程中是需要进行数据的 Shuffle。数据需要进行拆分调度,才能保证最终的 Join 结果是正确的。举个简单的例子,假设关系S 和 R 进行Join,N 表示参与 Join 计算的节点的数量;T 则表示关系的 Tuple 数目。 + + + +## Doris Shuffle 方式 + +Doris 支持 4 种 Shuffle 方式 + +1. BroadCast Join Review Comment: ```suggestion 1. Broadcast Join ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a diff in pull request #9753: [doc]Add Doris join optimization documentation
yangzhg commented on code in PR #9753: URL: https://github.com/apache/incubator-doris/pull/9753#discussion_r882258297 ## docs/zh-CN/advanced/join-optimization/doris-join-optimization.md: ## @@ -0,0 +1,226 @@ +--- +{ +"title": "Doris Join 优化原理", +"language": "zh-CN" +} + + +--- + + + +# Doris Join 优化原理 + +Doris 支持两种物理算子,一类是 **Hash Join**,另一类是 **Nest Loop Join**。 + +- Hash Join:在右表上根据等值 Join 列建立哈希表,左表流式的利用哈希表进行 Join 计算,它的限制是只能适用于等值 Join。 +- Nest Loop Join:通过两个 for 循环,很直观。然后它适用的场景就是不等值的 Join,例如:大于小于或者是需要求笛卡尔积的场景。它是一个通用的 Join 算子,但是性能表现差。 + +作为分布式的 MPP 数据库, 在 Join 的过程中是需要进行数据的 Shuffle。数据需要进行拆分调度,才能保证最终的 Join 结果是正确的。举个简单的例子,假设关系S 和 R 进行Join,N 表示参与 Join 计算的节点的数量;T 则表示关系的 Tuple 数目。 + + + +## Doris Shuffle 方式 + +Doris 支持 4 种 Shuffle 方式 + +1. BroadCast Join + + 它要求把右表全量的数据都发送到左表上,即每一个参与 Join 的节点,它都拥有右表全量的数据,也就是 T(R)。 + + 它适用的场景是比较通用的,同时能够支持 Hash Join 和 Nest loop Join,它的网络开销 N * T(R)。 + +  + + 左表数据不移动,右表数据发送到左表数据的扫描节点。 + +2. Shuffle Join + + 当进行 Hash Join 时候,可以通过 Join 列计算对应的 Hash 值,并进行 Hash 分桶。 + + 它的网络开销则是:T(R) + T(N),但它只能支持 Hash Join,因为它是根据 Join 的条件也去做计算分桶的。 + +  + + 左右表数据根据分区,计算的记过发送到不同的分区节点上。 + +3. Bucket Shuffle Join + + Doris 的表数据本身是通过 Hash 计算分桶的,所以就可以利用表本身的分桶列的性质来进行 Join 数据的 Shuffle。假如两张表需要做 Join,并且 Join 列是左表的分桶列,那么左表的数据其实可以不用去移动右表通过左表的数据分桶发送数据就可以完成 Join 的计算。 + + 它的网络开销则是:T(R)相当于只 Shuffle 右表的数据就可以了。 + +  + + 左表数据不移动,右表数据根据分区计算的结果发送到左表扫表的节点 + +4. Colocate + + 它与 Bucket Shuffle Join 相似,相当于在数据导入的时候,根据预设的 Join 列的场景已经做好了数据的 Shuffle。那么实际查询的时候就可以直接进行 Join 计算而不需要考虑数据的 Shuffle 问题了。 + +  + + 数据已经预先分区,直接在本地进行 Join 计算 + +### 四种 Shuffle 方式对比 + +| Shuffle方式| 网络开销| 物理算子 | 适用场景 | +| -- | --- | -- | | +| BroadCast | N * T(R)| Hash Join / Nest Loop Join | 通用 | +| Shuffle| T(S) + T(R) | Hash Join | 通用 | +| Bucket Shuffle | T(R)| Hash Join | Join条件中存在左表的分布式列,且左表执行时为单分区 | +| Colocate | 0 | Hash Join | Join条件中存在左表的分布式列,切左右表同属于一个Colocate Group | + +N : 参与 Join 计算的 Instance 个数 + +T(关系) : 关系的 Tuple 数目 + +上面这 4 种方式灵活度是从高到低的,它对这个数据分布的要求是越来越严格,但 Join 计算的性能也是越来越好的。 + +## Runtime Filter Join 优化 + +Doris 在进行 Hash Join 计算时会在右表构建一个哈希表,左表流式的通过右表的哈希表从而得出 Join 结果。而 RuntimeFilter 就是充分利用了右表的 Hash 表,在右表生成哈希表的时,同时生成一个基于哈希表数据的一个过滤条件,然后下推到左表的数据扫描节点。通过这样的方式,Doris 可以在运行时进行数据过滤。 + +假如左表是一张大表,右表是一张小表,那么利用左表生成的过滤条件就可以把绝大多数在 Join 层要过滤的数据在数据读取时就提前过滤,这样就能大幅度的提升 Join 查询的性能。 + +当前 Doris 支持三种类型 RuntimeFilter + +- 一种是 IN— IN,很好理解,将一个 hashset 下推到数据扫描节点。 Review Comment: ```suggestion - 一种是 IN,很好理解,将一个 hashset 下推到数据扫描节点。 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] carlvinhust2012 commented on a diff in pull request #9778: [Bug] sql block rule npe
carlvinhust2012 commented on code in PR #9778: URL: https://github.com/apache/incubator-doris/pull/9778#discussion_r882261551 ## regression-test/suites/sql_block_rule/test_sql_block_rule.groovy: ## @@ -0,0 +1,55 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +suite("test_sql_block_rule", "sql_block_rule") { +sql """ +USE regression_test + """ + +sql """ +CREATE SQL_BLOCK_RULE test_rule_sql +PROPERTIES("sql"="SELECT * FROM table_2", "global"= "true", "enable"= "true") + """ + +test { +sql "SELECT * FROM table_2" +exception "sql match regex sql block rule: test_rule_sql" +} + +sql """ +DROP SQL_BLOCK_RULE test_rule_sql + """ + +sql """ +CREATE SQL_BLOCK_RULE test_rule_num +PROPERTIES("tablet_num"="1", "global"= "true", "enable"= "true") + """ + +test { +sql "SELECT * FROM table_2" +exception "sql hits sql block rule: test_rule_num, reach tablet_num : 1" +} + +qt_select """ +SHOW SQL_BLOCK_RULE + """ + +sql """ +DROP SQL_BLOCK_RULE test_rule_num + """ + Review Comment: Add a normal case maybe better. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] carlvinhust2012 commented on a diff in pull request #9778: [Bug] sql block rule npe
carlvinhust2012 commented on code in PR #9778: URL: https://github.com/apache/incubator-doris/pull/9778#discussion_r882262763 ## fe/fe-core/src/main/java/org/apache/doris/blockrule/SqlBlockRule.java: ## @@ -33,12 +34,13 @@ import java.util.List; import java.util.regex.Pattern; +/** + * Use for block some sql by rule. + **/ public class SqlBlockRule implements Writable { public static final String NAME_TYPE = "SQL BLOCK RULE NAME"; -public static final String DEFAULT_USER = "default"; Review Comment: why delete this line? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morrySnow opened a new pull request, #9783: [enhancement] (community) update IntelliJ IDEA style file
morrySnow opened a new pull request, #9783: URL: https://github.com/apache/incubator-doris/pull/9783 # Proposed changes update three warp rules to adapt check style. ## Problem Summary: current IntelliJ IDEA style file could not move TERNARY_OPERATION_SIGNS, PLACE_ASSIGNMENT_SIGN and ASSERT_STATEMENT_COLON to next line automatically. ## Checklist(Required) 1. Does it affect the original behavior: No 2. Has unit tests been added: No Need 3. Has document been added or modified: No Need 4. Does it need to update dependencies: No 5. Are there any changes that cannot be rolled back: No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] stalary commented on a diff in pull request #9778: [Bug] sql block rule npe
stalary commented on code in PR #9778: URL: https://github.com/apache/incubator-doris/pull/9778#discussion_r882263977 ## fe/fe-core/src/main/java/org/apache/doris/blockrule/SqlBlockRule.java: ## @@ -33,12 +34,13 @@ import java.util.List; import java.util.regex.Pattern; +/** + * Use for block some sql by rule. + **/ public class SqlBlockRule implements Writable { public static final String NAME_TYPE = "SQL BLOCK RULE NAME"; -public static final String DEFAULT_USER = "default"; Review Comment: DEFAULT_USER is used only when the first designed, and then defined with global, it is no longer needed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] stalary commented on a diff in pull request #9778: [Bug] sql block rule npe
stalary commented on code in PR #9778: URL: https://github.com/apache/incubator-doris/pull/9778#discussion_r882264423 ## regression-test/suites/sql_block_rule/test_sql_block_rule.groovy: ## @@ -0,0 +1,55 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +suite("test_sql_block_rule", "sql_block_rule") { +sql """ +USE regression_test + """ + +sql """ +CREATE SQL_BLOCK_RULE test_rule_sql +PROPERTIES("sql"="SELECT * FROM table_2", "global"= "true", "enable"= "true") + """ + +test { +sql "SELECT * FROM table_2" +exception "sql match regex sql block rule: test_rule_sql" +} + +sql """ +DROP SQL_BLOCK_RULE test_rule_sql + """ + +sql """ +CREATE SQL_BLOCK_RULE test_rule_num +PROPERTIES("tablet_num"="1", "global"= "true", "enable"= "true") + """ + +test { +sql "SELECT * FROM table_2" +exception "sql hits sql block rule: test_rule_num, reach tablet_num : 1" +} + +qt_select """ +SHOW SQL_BLOCK_RULE + """ + +sql """ +DROP SQL_BLOCK_RULE test_rule_num + """ + Review Comment: Ok, I'm going to add it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] adonis0147 opened a new issue, #9784: [Enhancement][doc] Add a note about the version of checkstyle
adonis0147 opened a new issue, #9784: URL: https://github.com/apache/incubator-doris/issues/9784 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description [UnusedLocalVariable](https://github.com/apache/incubator-doris/blob/32a210f426f9422a971a1e0157a349c6f88c9dcd/fe/check/checkstyle/checkstyle.xml#L167) check was introduced by [checkstyle 9.3](https://checkstyle.sourceforge.io/config_coding.html#UnusedLocalVariable), we should remind users of the version. ### Solution Add a note about the version of checkstyle. ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] adonis0147 opened a new pull request, #9785: [doc] Add a note about the version of checkstyle
adonis0147 opened a new pull request, #9785: URL: https://github.com/apache/incubator-doris/pull/9785 # Proposed changes Issue Number: close #9784 ## Problem Summary: Please refer to #9784 ## Checklist(Required) 1. Does it affect the original behavior: No 2. Has unit tests been added: No Need 3. Has document been added or modified: Yes 4. Does it need to update dependencies: No 5. Are there any changes that cannot be rolled back: No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] adonis0147 commented on pull request #9785: [doc] Add a note about the version of checkstyle
adonis0147 commented on PR #9785: URL: https://github.com/apache/incubator-doris/pull/9785#issuecomment-1138123079 CC @morrySnow -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a diff in pull request #9764: [style] (fe) wrap and whitespace rules
morningman commented on code in PR #9764: URL: https://github.com/apache/incubator-doris/pull/9764#discussion_r882264236 ## fe/fe-core/src/main/java/org/apache/doris/analysis/BinaryPredicate.java: ## @@ -134,11 +134,21 @@ public Operator converse() { } } -public boolean isEquivalence() { return this == EQ || this == EQ_FOR_NULL; }; +public boolean isEquivalence() { +return this == EQ || this == EQ_FOR_NULL; +} + +; Review Comment: remove ## fe/fe-core/src/main/java/org/apache/doris/analysis/BinaryPredicate.java: ## @@ -134,11 +134,21 @@ public Operator converse() { } } -public boolean isEquivalence() { return this == EQ || this == EQ_FOR_NULL; }; +public boolean isEquivalence() { +return this == EQ || this == EQ_FOR_NULL; +} + +; + +public boolean isUnNullSafeEquivalence() { +return this == EQ; +} -public boolean isUnNullSafeEquivalence() { return this == EQ; }; +; Review Comment: remove ## fe/fe-core/src/main/java/org/apache/doris/analysis/BinaryPredicate.java: ## @@ -134,11 +134,21 @@ public Operator converse() { } } -public boolean isEquivalence() { return this == EQ || this == EQ_FOR_NULL; }; +public boolean isEquivalence() { +return this == EQ || this == EQ_FOR_NULL; +} + +; Review Comment: remove -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] pengxiangyu opened a new pull request, #9786: Change policy
pengxiangyu opened a new pull request, #9786: URL: https://github.com/apache/incubator-doris/pull/9786 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] adonis0147 commented on pull request #9708: [refactor][be] Optimize headers
adonis0147 commented on PR #9708: URL: https://github.com/apache/incubator-doris/pull/9708#issuecomment-1138142482 Hi @morningman , I have resolved all conflicts. PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #9734: [Bug] [Vectorized] add padding when load char type data
github-actions[bot] commented on PR #9734: URL: https://github.com/apache/incubator-doris/pull/9734#issuecomment-1138156724 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #9734: [Bug] [Vectorized] add padding when load char type data
github-actions[bot] commented on PR #9734: URL: https://github.com/apache/incubator-doris/pull/9734#issuecomment-1138156751 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] hf200012 closed pull request #9697: [fix]Documentation fixes
hf200012 closed pull request #9697: [fix]Documentation fixes URL: https://github.com/apache/incubator-doris/pull/9697 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] hf200012 opened a new pull request, #9787: [fix]Documentation fixes
hf200012 opened a new pull request, #9787: URL: https://github.com/apache/incubator-doris/pull/9787 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #9787: [fix]Documentation fixes
github-actions[bot] commented on PR #9787: URL: https://github.com/apache/incubator-doris/pull/9787#issuecomment-1138169415 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #9787: [fix]Documentation fixes
github-actions[bot] commented on PR #9787: URL: https://github.com/apache/incubator-doris/pull/9787#issuecomment-1138169396 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] Vikasss7663 commented on issue #9775: [Enhancement] Remove the redundant `this.`
Vikasss7663 commented on issue #9775: URL: https://github.com/apache/incubator-doris/issues/9775#issuecomment-1138177476 Hey @jackwener I am a beginner and never contributed to any open source. I want to work on this, can you help me with this issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] Gabriel39 commented on pull request #9582: [refactor](decimalv3) optimize decimal performance and precision
Gabriel39 commented on PR #9582: URL: https://github.com/apache/incubator-doris/pull/9582#issuecomment-1138183867 1. Add UTs in BE 2. rebase master to resolve conflicts -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] jackwener commented on issue #9775: [Enhancement] Remove the redundant `this.`
jackwener commented on issue #9775: URL: https://github.com/apache/incubator-doris/issues/9775#issuecomment-1138183981 > Hey @jackwener I am a beginner and never contributed to any open source. I want to work on this, can you help me with this issue. It's ok😀, let's me explain this issue. In Java, we can use `this.` to access `class member variables` like `if (this.name == jackwener)`. But, in most scenario, it's redundant. just use in constructor like: ```java class Person { public string name; Person(name) { this.name = name; } boolean isJack() { this.name == "jack"; // It's redundant; } } ``` So, we can remove the redundant like `isJack()`. example in #8862 `fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsJobManager.java` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] jackwener commented on issue #9775: [Enhancement] Remove the redundant `this.`
jackwener commented on issue #9775: URL: https://github.com/apache/incubator-doris/issues/9775#issuecomment-1138184514 You can find code in `fe/`. If there are redundant `this.`, you can remove them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] stalary commented on pull request #9786: Change policy
stalary commented on PR #9786: URL: https://github.com/apache/incubator-doris/pull/9786#issuecomment-1138189958 Can you submit an issue describing what you are going to do? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] BiteTheDDDDt commented on issue #9596: [Bug] [Load-Vectorized] core dump on KeyCoder::encode_ascending
BiteThet commented on issue #9596: URL: https://github.com/apache/incubator-doris/issues/9596#issuecomment-1138201343 fixed on https://github.com/apache/incubator-doris/pull/9734 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] englefly opened a new issue, #9789: [Bug] variance calculate error
englefly opened a new issue, #9789: URL: https://github.com/apache/incubator-doris/issues/9789 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version Dev-1.0.1 ### What's Wrong? for column A =[34, 78, 23] select variance(A) Expect is: 564.7 But real is: 564.66788 ### What You Expected? for column A =[34, 78, 23] select variance(A) Expect is: 564.7 But real is: 564.66788 ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] hf200012 opened a new pull request, #9790: [doc][fix]Sync job doc
hf200012 opened a new pull request, #9790: URL: https://github.com/apache/incubator-doris/pull/9790 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org