Re: [PR] [docs](help) remove sql manual [doris]
github-actions[bot] commented on PR #33081: URL: https://github.com/apache/doris/pull/33081#issuecomment-2031218262 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [docs](help) remove sql manual [doris]
github-actions[bot] commented on PR #33081: URL: https://github.com/apache/doris/pull/33081#issuecomment-2031220926 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix] (vectorization) regexp all_pass string #32515 [doris]
AcKing-Sam commented on PR #33142: URL: https://github.com/apache/doris/pull/33142#issuecomment-2031220439 @xiaokang Hi, what can I help to cherry-pick this issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [log](move-memtable) improve log in vtablet writer v2 and load stream [doris]
kaijchen commented on PR #33103: URL: https://github.com/apache/doris/pull/33103#issuecomment-2031221982 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [enhancement](function truncate) truncate can use column as scale argument [doris]
zhiqiang- opened a new pull request, #33157: URL: https://github.com/apache/doris/pull/33157 pick #32746 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](function truncate) truncate can use column as scale argument [doris]
doris-robot commented on PR #33157: URL: https://github.com/apache/doris/pull/33157#issuecomment-2031222992 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](function truncate) truncate can use column as scale argument [doris]
zhiqiang- commented on PR #33157: URL: https://github.com/apache/doris/pull/33157#issuecomment-2031223146 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [log](move-memtable) improve log in vtablet writer v2 and load stream [doris]
github-actions[bot] commented on PR #33103: URL: https://github.com/apache/doris/pull/33103#issuecomment-2031224455 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [docs](help) remove sql manual [doris]
yiguolei merged PR #33081: URL: https://github.com/apache/doris/pull/33081 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](backup) ignore table that not support type when backup, and… [doris]
doris-robot commented on PR #33158: URL: https://github.com/apache/doris/pull/33158#issuecomment-2031229091 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](backup) ignore table that not support type when backup, and… [doris]
nextdreamblue commented on PR #33158: URL: https://github.com/apache/doris/pull/33158#issuecomment-2031229511 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [log](move-memtable) improve log in vtablet writer v2 and load stream [doris]
github-actions[bot] commented on PR #33103: URL: https://github.com/apache/doris/pull/33103#issuecomment-2031230179 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](Mysql) Refactoring the process of using external component… [doris]
github-actions[bot] commented on PR #32875: URL: https://github.com/apache/doris/pull/32875#issuecomment-2031231056 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](Mysql) Refactoring the process of using external component… [doris]
github-actions[bot] commented on PR #32875: URL: https://github.com/apache/doris/pull/32875#issuecomment-2031231109 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [feature](backup) ignore table that not support type when backup, and… [doris]
nextdreamblue opened a new pull request, #33158: URL: https://github.com/apache/doris/pull/33158 … not report exception ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](backup) ignore table that not support type when backup, and… [doris]
nextdreamblue commented on PR #33158: URL: https://github.com/apache/doris/pull/33158#issuecomment-2031229267 run buidlall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](compile) fix gcc compile failed (#30596) [doris]
github-actions[bot] commented on PR #33150: URL: https://github.com/apache/doris/pull/33150#issuecomment-2031232004 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](compile) fix gcc compile failed (#30596) [doris]
github-actions[bot] commented on PR #33150: URL: https://github.com/apache/doris/pull/33150#issuecomment-2031231947 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](function truncate) truncate can use column as scale argument [doris]
github-actions[bot] commented on code in PR #33157:
URL: https://github.com/apache/doris/pull/33157#discussion_r1547254634
##
be/src/vec/functions/function_truncate.h:
##
@@ -0,0 +1,245 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
+#include
+#include
+#include
+
+#include "common/exception.h"
+#include "common/status.h"
+#include "olap/olap_common.h"
+#include "round.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_const.h"
+#include "vec/columns/column_decimal.h"
+#include "vec/columns/column_vector.h"
+#include "vec/common/assert_cast.h"
+#include "vec/core/call_on_type_index.h"
+#include "vec/core/field.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_decimal.h"
+#include "vec/data_types/data_type_number.h"
+
+namespace doris::vectorized {
+
+struct TruncateFloatOneArgImpl {
+static constexpr auto name = "truncate";
+static DataTypes get_variadic_argument_types() { return
{std::make_shared()}; }
+};
+
+struct TruncateFloatTwoArgImpl {
+static constexpr auto name = "truncate";
+static DataTypes get_variadic_argument_types() {
+return {std::make_shared(),
std::make_shared()};
+}
+};
+
+struct TruncateDecimalOneArgImpl {
+static constexpr auto name = "truncate";
+static DataTypes get_variadic_argument_types() {
+// All Decimal types are named Decimal, and real scale will be passed
as type argument for execute function
+// So we can just register Decimal32 here
+return {std::make_shared>(9, 0)};
+}
+};
+
+struct TruncateDecimalTwoArgImpl {
+static constexpr auto name = "truncate";
+static DataTypes get_variadic_argument_types() {
+return {std::make_shared>(9, 0),
+std::make_shared()};
+}
+};
+
+template
+class FunctionTruncate : public FunctionRounding {
+public:
+static FunctionPtr create() { return std::make_shared();
}
+
+ColumnNumbers get_arguments_that_are_always_constant() const override {
return {}; }
Review Comment:
warning: function 'get_arguments_that_are_always_constant' should be marked
[[nodiscard]] [modernize-use-nodiscard]
```suggestion
[[nodiscard]] ColumnNumbers get_arguments_that_are_always_constant()
const override { return {}; }
```
##
be/test/vec/function/function_truncate_decimal_test.cpp:
##
@@ -0,0 +1,370 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
Review Comment:
warning: 'gtest/gtest-message.h' file not found [clang-diagnostic-error]
```cpp
#include
^
```
##
be/src/vec/functions/function_truncate.h:
##
@@ -0,0 +1,245 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License fo
Re: [PR] [refactor](Mysql) Refactoring the process of using external component… [doris]
morningman merged PR #32875: URL: https://github.com/apache/doris/pull/32875 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Refactor] (Mysql) Refactoring the process of using external components to authenticate in MySQL connections [doris]
morningman closed issue #32795: [Refactor] (Mysql) Refactoring the process of using external components to authenticate in MySQL connections URL: https://github.com/apache/doris/issues/32795 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch master updated: [Doc](Flink-Connect)update flink connect 1.6.0 doc (#490)
This is an automated email from the ASF dual-hosted git repository. luzhijing pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 83cfd18a4139 [Doc](Flink-Connect)update flink connect 1.6.0 doc (#490) 83cfd18a4139 is described below commit 83cfd18a41391d66657a9c9bb7d34954feb0e70a Author: wudi <676366...@qq.com> AuthorDate: Tue Apr 2 15:13:01 2024 +0800 [Doc](Flink-Connect)update flink connect 1.6.0 doc (#490) --- docs/ecosystem/flink-doris-connector.md| 34 +- .../current/ecosystem/flink-doris-connector.md | 34 +- 2 files changed, 42 insertions(+), 26 deletions(-) diff --git a/docs/ecosystem/flink-doris-connector.md b/docs/ecosystem/flink-doris-connector.md index 52738e1238c1..bcc61b637ba2 100644 --- a/docs/ecosystem/flink-doris-connector.md +++ b/docs/ecosystem/flink-doris-connector.md @@ -45,6 +45,7 @@ under the License. | 1.3.0 | 1.16| 1.0+ | 8| - | | 1.4.0 | 1.15,1.16,1.17 | 1.0+ | 8 |- | | 1.5.2 | 1.15,1.16,1.17,1.18 | 1.0+ | 8 |- | +| 1.6.0 | 1.15,1.16,1.17,1.18,1.19 | 1.0+ | 8 |- | ## USE @@ -57,7 +58,7 @@ Add flink-doris-connector org.apache.doris flink-doris-connector-1.16 - 1.5.2 + 1.6.0 ``` @@ -320,19 +321,24 @@ ON a.city = c.city | password | --| Y| Password to access Doris | | auto-redirect| true | N| Whether to redirect StreamLoad requests. After being turned on, StreamLoad will be written through FE, and BE information will no longer be displayed. | | doris.request.retries| 3 | N| Number of retries to send requests to Doris | -| doris.request.connect.timeout.ms | 3 | N| Connection timeout for sending requests to Doris | -| doris.request.read.timeout.ms| 3 | N| Read timeout for sending requests to Doris | +| doris.request.connect.timeout | 30s | N| Connection timeout for sending requests to Doris | +| doris.request.read.timeout| 30s | N| Read timeout for sending requests to Doris | ### Source configuration item | Key | Default Value | Required | Comment | | - | -- | | | -| doris.request.query.timeout.s | 3600 | N| The timeout time for querying Doris, the default value is 1 hour, -1 means no timeout limit | -| doris.request.tablet.size | Integer. MAX_VALUE | N| The number of Doris Tablets corresponding to a Partition. The smaller this value is set, the more Partitions will be generated. This improves the parallelism on the Flink side, but at the same time puts more pressure on Doris. | +| doris.request.query.timeout | 21600s | N| The timeout time for querying Doris, the default value is 6 hour | +| doris.request.tablet.size | 1 | N| The number of Doris Tablets corresponding to a Partition. The smaller this value is set, the more Partitions will be generated. This improves the parallelism on the Flink side, but at the same time puts more pressure on Doris. | | doris.batch.size | 1024 | N| The maximum number of rows to read data from BE at a time. Increasing this value reduces the number of connections established between Flink and Doris. Thereby reducing the additional time overhead caused by network delay. | -| doris.exec.mem.limit | 2147483648 | N| Memory limit for a single query. The default is 2GB, in bytes | +| doris.exec.mem.limit | 8192mb | N| Memory limit for a single query. The default is 8GB, in bytes | | doris.deserialize.arrow.async | FALSE | N| Whether to support asynchronous conversion of Arrow format to RowBatch needed for flink-doris-connector iterations | | doris.deserialize.queue.size | 64 | N| Asynchronous conversion of internal processing queue in Arrow format, effective when doris.deserialize.arrow.async is true | + + DataStream proprietary configuration items + +| Key | Default Value | Required | Comment | +| - | -- | | | | doris.read.field | -- | N| Read the list of column names of the Doris table, s
Re: [PR] [typo](docs) update/delete supports CTE [doris-website]
luzhijing merged PR #492: URL: https://github.com/apache/doris-website/pull/492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [refactor](Mysql) Refactoring the process of using external components to authenticate in MySQL connections (#32875)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new e8648411fdd [refactor](Mysql) Refactoring the process of using
external components to authenticate in MySQL connections (#32875)
e8648411fdd is described below
commit e8648411fdddfcc0ed8b1ee898b2bc0217e10bdd
Author: LompleZ Liu <47652868+lomp...@users.noreply.github.com>
AuthorDate: Tue Apr 2 15:12:03 2024 +0800
[refactor](Mysql) Refactoring the process of using external components to
authenticate in MySQL connections (#32875)
Add a separate processing method for external permission verification, and
LDAP is now just one of its many switch cases. It will be easier to add other
external authentication systems in the future.
This change did not change the original execution logic of the code.
---
.../main/java/org/apache/doris/common/Config.java | 5 +
.../java/org/apache/doris/common/LdapConfig.java | 7 -
.../org/apache/doris/analysis/CreateUserStmt.java | 5 +-
.../org/apache/doris/analysis/DropUserStmt.java| 5 +-
.../java/org/apache/doris/mysql/MysqlProto.java| 117 +---
.../apache/doris/mysql/authenticate/MysqlAuth.java | 205 +
.../doris/mysql/authenticate/MysqlAuthType.java| 38
.../authenticate}/ldap/LdapAuthenticate.java | 2 +-
.../{ => mysql/authenticate}/ldap/LdapClient.java | 2 +-
.../{ => mysql/authenticate}/ldap/LdapManager.java | 13 +-
.../authenticate}/ldap/LdapUserInfo.java | 2 +-
.../org/apache/doris/mysql/privilege/Auth.java | 8 +-
.../org/apache/doris/mysql/MysqlProtoTest.java | 14 +-
.../authenticate}/ldap/LdapAuthenticateTest.java | 2 +-
.../authenticate}/ldap/LdapClientTest.java | 5 +-
.../authenticate}/ldap/LdapManagerTest.java| 6 +-
16 files changed, 286 insertions(+), 150 deletions(-)
diff --git a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
index 73059c35a03..d2e463f55ac 100644
--- a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
+++ b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
@@ -2513,6 +2513,11 @@ public class Config extends ConfigBase {
options = {"default", "ranger-doris"})
public static String access_controller_type = "default";
+@ConfField(description = {"指定 mysql登录身份认证类型",
+"Specifies the authentication type"},
+options = {"default", "ldap"})
+public static String authentication_type = "default";
+
@ConfField(mutable = true, masterOnly = false, description = {"指定
trino-connector catalog 的插件默认加载路径",
"Specify the default plugins loading path for the trino-connector
catalog"})
public static String trino_connector_plugin_dir = EnvUtils.getDorisHome()
+ "/connectors";
diff --git a/fe/fe-common/src/main/java/org/apache/doris/common/LdapConfig.java
b/fe/fe-common/src/main/java/org/apache/doris/common/LdapConfig.java
index 569c43b71ff..ef35484cbca 100644
--- a/fe/fe-common/src/main/java/org/apache/doris/common/LdapConfig.java
+++ b/fe/fe-common/src/main/java/org/apache/doris/common/LdapConfig.java
@@ -21,13 +21,6 @@ package org.apache.doris.common;
* LDAP configuration
*/
public class LdapConfig extends ConfigBase {
-
-/**
- * Flag to enable LDAP authentication.
- */
-@ConfigBase.ConfField
-public static boolean ldap_authentication_enabled = false;
-
/**
* LDAP server ip.
*/
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateUserStmt.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateUserStmt.java
index e64a5e2e7f8..8757a9e088d 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateUserStmt.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateUserStmt.java
@@ -24,8 +24,8 @@ import org.apache.doris.common.Config;
import org.apache.doris.common.ErrorCode;
import org.apache.doris.common.ErrorReport;
import org.apache.doris.common.FeNameFormat;
-import org.apache.doris.common.LdapConfig;
import org.apache.doris.common.UserException;
+import org.apache.doris.mysql.authenticate.MysqlAuthType;
import org.apache.doris.mysql.privilege.PrivPredicate;
import org.apache.doris.mysql.privilege.Role;
import org.apache.doris.qe.ConnectContext;
@@ -146,7 +146,8 @@ public class CreateUserStmt extends DdlStmt {
public void analyze(Analyzer analyzer) throws UserException {
super.analyze(analyzer);
-if (Config.access_controller_type.equalsIgnoreCase("ranger-doris") &&
LdapConfig.ldap_authentication_enabled) {
+if (Config.access_controller_type.equalsIgnoreCase("ranger-doris")
+&& MysqlAuthType.getAuthTypeConfig() == MysqlAuthType.LDAP) {
Re: [PR] [Doc] Update show data skew doc [doris-website]
luzhijing merged PR #496: URL: https://github.com/apache/doris-website/pull/496 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [doc](mtmv)partition limit [doris-website]
luzhijing merged PR #485: URL: https://github.com/apache/doris-website/pull/485 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](backup) ignore table that not support type when backup, and… [doris]
nextdreamblue commented on PR #33158: URL: https://github.com/apache/doris/pull/33158#issuecomment-2031237721 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch master updated: [doc](mtmv)partition limit (#485)
This is an automated email from the ASF dual-hosted git repository.
luzhijing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 45ee42639afc [doc](mtmv)partition limit (#485)
45ee42639afc is described below
commit 45ee42639afc9ce45e2fe45239b92d97c5498d4b
Author: zhangdong <493738...@qq.com>
AuthorDate: Tue Apr 2 15:12:49 2024 +0800
[doc](mtmv)partition limit (#485)
---
.../async-materialized-view.md | 37 +
.../Create/CREATE-ASYNC-MATERIALIZED-VIEW.md | 10 ++
.../async-materialized-view.md | 38 ++
.../Create/CREATE-ASYNC-MATERIALIZED-VIEW.md | 8 +
4 files changed, 93 insertions(+)
diff --git
a/docs/query-acceleration/async-materialized-view/async-materialized-view.md
b/docs/query-acceleration/async-materialized-view/async-materialized-view.md
index e55ff51414bc..71b59c8082d8 100644
--- a/docs/query-acceleration/async-materialized-view/async-materialized-view.md
+++ b/docs/query-acceleration/async-materialized-view/async-materialized-view.md
@@ -203,6 +203,43 @@ The materialized view has a dedicated deletion syntax and
cannot be deleted thro
Specific syntax can be viewed [DROP ASYNC MATERIALIZED
VIEW](../../sql-manual/sql-reference/Data-Definition-Statements/Drop/DROP-ASYNC-MATERIALIZED-VIEW.md)
+## Best Practice
+### When there are excessive partitions in the base table, the materialized
view should only focus on the data from the most recent period.
+create table with 3 partitions
+```sql
+CREATE TABLE t1 (
+`k1` INT,
+`k2` DATE NOT NULL
+) ENGINE=OLAP
+DUPLICATE KEY(`k1`)
+COMMENT 'OLAP'
+PARTITION BY range(`k2`)
+(
+PARTITION p26 VALUES [("2024-03-26"),("2024-03-27")),
+PARTITION p27 VALUES [("2024-03-27"),("2024-03-28")),
+PARTITION p28 VALUES [("2024-03-28"),("2024-03-29"))
+)
+DISTRIBUTED BY HASH(`k1`) BUCKETS 2
+PROPERTIES (
+'replication_num' = '1'
+);
+```
+To create a materialized view that focuses only on the data from the most
recent day, assuming the current time is 2024-03-28 xx:xx:xx, the materialized
view would include only the partition ranging from ["2024-03-28") to
("2024-03-29"). This ensures that the materialized view contains data only for
the latest day.
+```sql
+CREATE MATERIALIZED VIEW mv1
+BUILD DEFERRED REFRESH AUTO ON MANUAL
+partition by(`k2`)
+DISTRIBUTED BY RANDOM BUCKETS 2
+PROPERTIES (
+'replication_num' = '1',
+'partition_sync_limit'='1',
+'partition_sync_time_unit'='DAY'
+)
+AS
+SELECT * FROM t1;
+```
+As another day passes and the current time becomes 2024-03-29 xx:xx:xx, a new
partition is added to t1, ranging from ["2024-03-29") to ("2024-03-30"). If the
materialized view is refreshed at this point, upon completion of the refresh,
the materialized view will contain only this new partition, ranging from
["2024-03-29") to ("2024-03-30").
+
## The use of materialized views
can be viewed [Query async materialized
view](./query-async-materialized-view.md)
diff --git
a/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-ASYNC-MATERIALIZED-VIEW.md
b/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-ASYNC-MATERIALIZED-VIEW.md
index f1fd88afd572..83ce5c02905a 100644
---
a/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-ASYNC-MATERIALIZED-VIEW.md
+++
b/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-ASYNC-MATERIALIZED-VIEW.md
@@ -192,6 +192,16 @@ The properties unique to materialized views include:
`workload_group`: The name of the workload_group used by the materialized view
when performing refresh tasks. This is used to limit the resources used for
refreshing data in the materialized view, in order to avoid affecting the
operation of other business processes. For details on how to create and use
workload_group, refer to
[WORKLOAD-GROUP](../../../../admin-manual/workload-group.md)
+`partition_sync_limit`: When the partition field of the base table is time
(and if it's a string-type time, `partition_date_format` can be set), this
attribute can be used to configure the partition range for synchronizing the
base table. It works in conjunction with `partition_sync_time_unit`.
+
+For instance, if it's set to 2 and `partition_sync_time_unit` is set to
`MONTH`, it means only the partitions and data of the last 2 months in the base
table will be synchronized. The minimum value is 1.
+
+As time passes, the materialized view will automatically add and delete
partitions during each refresh. For example, if the materialized view currently
has data for February and March, next month it will automatically delete the
data for February and add data for April.
+
+`partition_sync_time_unit`: time unit, support DAY/MONTH/YEAR（default: DAY）
+
+`partition_date_format`: date format of partition column，for example:
"%Y-%m
Re: [PR] [Doc](Flink-Connect)update flink connect 1.6.0 doc [doris-website]
luzhijing merged PR #490: URL: https://github.com/apache/doris-website/pull/490 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch master updated: [typo](docs) update/delete supports CTE (#492)
This is an automated email from the ASF dual-hosted git repository.
luzhijing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new f8a4c95af267 [typo](docs) update/delete supports CTE (#492)
f8a4c95af267 is described below
commit f8a4c95af26749fbffd2b889257c1cce4916f38b
Author: yagagagaga
AuthorDate: Tue Apr 2 15:13:24 2024 +0800
[typo](docs) update/delete supports CTE (#492)
---
.../Manipulation/DELETE.md | 60 +--
.../Manipulation/UPDATE.md | 68 +-
.../Manipulation/DELETE.md | 60 +--
.../Manipulation/UPDATE.md | 68 +-
.../Manipulation/DELETE.md | 60 +--
.../Manipulation/UPDATE.md | 68 +-
.../Manipulation/DELETE.md | 60 +--
.../Manipulation/UPDATE.md | 68 +-
8 files changed, 336 insertions(+), 176 deletions(-)
diff --git
a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/DELETE.md
b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/DELETE.md
index 1a34bc2dede4..eb39ddac6978 100644
---
a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/DELETE.md
+++
b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/DELETE.md
@@ -46,44 +46,31 @@ WHERE
column_name op { value | value_list } [ AND column_name op { value |
value_list } ...];
```
-
-
Syntax 2：This syntax can only used on UNIQUE KEY model
```sql
+[cte]
DELETE FROM table_name
[PARTITION partition_name | PARTITIONS (partition_name [, partition_name])]
[USING additional_tables]
WHERE condition
```
-
-
Required Parameters
+ table_name: Specifies the table from which rows are removed.
+ column_name: column belong to table_name
+ op: Logical comparison operator, The optional types of op include: =, >, <,
>=, <=, !=, in, not in
+ value | value_list: value or value list used for logial comparison
-
-
-
+ WHERE condition: Specifies a condition to use to select rows for removal
-
-
-
Optional Parameters
++ cte: Common Table Expression, e.g. 'WITH a AS SELECT * FROM tbl'.
+ PARTITION partition_name | PARTITIONS (partition_name [, partition_name]):
Specifies the partition or partitions to select rows for removal
-
-
-
+ table_alias: alias of table
+ USING additional_tables: If you need to refer to additional tables in the
WHERE clause to help identify the rows to be removed, then specify those table
names in the USING clause. You can also use the USING clause to specify
subqueries that identify the rows to be removed.
-
-
Note
1. Only conditions on the key column can be specified when using AGGREGATE
(UNIQUE) model.
@@ -115,8 +102,6 @@ DELETE FROM table_name
WHERE k1 >= 3 AND k2 = "abc";
-
-
4. use the result of `t2` join `t3` to romve rows from `t1`,delete table only
support unique key model
```sql
@@ -172,7 +157,46 @@ DELETE FROM table_name
++++++
```
-
+5. using cte
+
+ ```sql
+ create table orders(
+o_orderkey bigint,
+o_totalprice decimal(15, 2)
+ ) unique key(o_orderkey)
+ distributed by hash(o_orderkey) buckets 1
+ properties (
+ "replication_num" = "1"
+ );
+
+ insert into orders values
+ (1, 34.1),
+ (2, 432.8);
+
+ create table lineitem(
+ l_linenumber int,
+ o_orderkey bigint,
+ l_discount decimal(15, 2)
+ ) unique key(l_linenumber)
+ distributed by hash(l_linenumber) buckets 1
+ properties (
+ "replication_num" = "1"
+ );
+
+ insert into lineitem values
+ (1, 1, 1.23),
+ (2, 1, 3.21),
+ (3, 2, 18.08),
+ (4, 2, 23.48);
+
+ with discount_orders as (
+ select * from orders
+ where o_totalprice > 100
+ )
+ delete from lineitem
+ using discount_orders
+ where lineitem.o_orderkey = discount_orders.o_orderkey;
+ ```
### Keywords
diff --git
a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/UPDATE.md
b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/UPDATE.md
index 050e59d393c9..b7c3fc5933fc 100644
---
a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/UPDATE.md
+++
b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/UPDATE.md
@@ -38,31 +38,13 @@ The UPDATE operation currently only supports updating the
Value column. The upda
Syntax
```sql
-UPDATE target_table [table_alias]
-SET assignment_list
-WHERE condition
-
-assignment_list:
-assignment [, assignment] ...
-
-assignment:
-col_name = value
-
-value:
-{expr | DEFAULT}
-```
-
-
-
-```sql
+[cte]
UPDATE ta
Re: [PR] [doc](k8s)k8s debug doc modify port [doris-website]
luzhijing merged PR #497: URL: https://github.com/apache/doris-website/pull/497 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch master updated: update show data skew doc (#496)
This is an automated email from the ASF dual-hosted git repository.
luzhijing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new da74ce58b80c update show data skew doc (#496)
da74ce58b80c is described below
commit da74ce58b80c67dfb32c976a66213cd56e21b482
Author: dongsilun <70750474+dongsi...@users.noreply.github.com>
AuthorDate: Tue Apr 2 15:13:55 2024 +0800
update show data skew doc (#496)
---
.../Show-Statements/SHOW-DATA-SKEW.md | 123 +++--
.../Show-Statements/SHOW-DATA-SKEW.md | 119 ++--
2 files changed, 224 insertions(+), 18 deletions(-)
diff --git a/docs/sql-manual/sql-reference/Show-Statements/SHOW-DATA-SKEW.md
b/docs/sql-manual/sql-reference/Show-Statements/SHOW-DATA-SKEW.md
index c54587eb7283..e8938478e113 100644
--- a/docs/sql-manual/sql-reference/Show-Statements/SHOW-DATA-SKEW.md
+++ b/docs/sql-manual/sql-reference/Show-Statements/SHOW-DATA-SKEW.md
@@ -32,25 +32,128 @@ SHOW DATA SKEW
### Description
-This statement is used to view the data skew of a table or a partition.
+This statement is used to view the data skew of a table or the specified
partitions.
grammar:
-SHOW DATA SKEW FROM [db_name.]tbl_name [PARTITION (p1)];
+SHOW DATA SKEW FROM [db_name.]tbl_name [PARTITION (partition_name,
...)];
- Description:
+Description:
- 1. Only one partition must be specified. For non-partitioned
tables, the partition name is the same as the table name.
- 2. The result will show row count and data volume of each
bucket under the specified partition, and the proportion of the data volume of
each bucket in the total data volume.
+1. The result will show row count and data volume of each bucket under
the specified partition, and the proportion of the data volume of each bucket
in the total data volume.
+2. For non-partitioned tables, the partition name in result is the
same as the table name.
### Example
-
-1. View the data skew of the table
-
-SHOW DATA SKEW FROM db1.test PARTITION(p1);
+1. For partitioned tables
+
+* CREATE-TABLE
+```sql
+CREATE TABLE test_show_data_skew
+(
+ id int,
+ name string,
+ pdate date
+)
+PARTITION BY RANGE(pdate)
+(
+ FROM ("2023-04-16") TO ("2023-04-20") INTERVAL 1 DAY
+)
+DISTRIBUTED BY HASH(id) BUCKETS 5
+PROPERTIES (
+ "replication_num" = "1"
+);
+```
+* View the data skew of the table
+ ```sql
+mysql> SHOW DATA SKEW FROM test_show_data_skew;
+
+---+---+-+-+--+-+
+| PartitionName | BucketIdx | AvgRowCount | AvgDataSize | Graph
| Percent |
+
+---+---+-+-+--+-+
+| p_20230416| 0 | 1 | 648 |
>
| 49.77 % |
+| p_20230416| 1 | 2 | 654 |
>>
| 50.23 % |
+| p_20230416| 2 | 0 | 0 |
| 00.00 % |
+| p_20230416| 3 | 0 | 0 |
| 00.00 % |
+| p_20230416| 4 | 0 | 0 |
| 00.00 % |
+| p_20230417| 0 | 0 | 0 |
| 00.00 % |
+| p_20230417| 1 | 0 | 0 |
| 00.00 % |
+| p_20230417| 2 | 0 | 0 |
| 00.00 % |
+| p_20230417| 3 | 0 | 0 |
| 00.00 % |
+| p_20230417| 4 | 2 | 656 |
(doris-website) branch master updated: [doc](k8s)k8s debug doc modify port (#497)
This is an automated email from the ASF dual-hosted git repository.
luzhijing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 03e6571eb2e5 [doc](k8s)k8s debug doc modify port (#497)
03e6571eb2e5 is described below
commit 03e6571eb2e53dee062e426544fce5392003f931
Author: catpineapple <42031973+catpineap...@users.noreply.github.com>
AuthorDate: Tue Apr 2 15:14:10 2024 +0800
[doc](k8s)k8s debug doc modify port (#497)
---
docs/install/k8s-deploy/debug-crash.md | 4 ++--
docs/install/k8s-deploy/install-doris-cluster.md| 2 ++
.../current/install/k8s-deploy/debug-crash.md | 5 ++---
.../current/install/k8s-deploy/install-doris-cluster.md | 2 ++
.../install/cluster-deployment/k8s-deploy/debug-crash.md| 6 ++
.../install/cluster-deployment/k8s-deploy/install-doris-cluster.md | 2 ++
.../version-2.1/install/k8s-deploy/debug-crash.md | 4 ++--
.../version-2.1/install/k8s-deploy/install-doris-cluster.md | 2 ++
.../install/cluster-deployment/k8s-deploy/debug-crash.md| 4 ++--
.../install/cluster-deployment/k8s-deploy/install-doris-cluster.md | 2 ++
versioned_docs/version-2.1/install/k8s-deploy/debug-crash.md| 4 ++--
.../version-2.1/install/k8s-deploy/install-doris-cluster.md | 2 ++
12 files changed, 24 insertions(+), 15 deletions(-)
diff --git a/docs/install/k8s-deploy/debug-crash.md
b/docs/install/k8s-deploy/debug-crash.md
index 911a65603cb6..5e8de55808b5 100644
--- a/docs/install/k8s-deploy/debug-crash.md
+++ b/docs/install/k8s-deploy/debug-crash.md
@@ -68,6 +68,6 @@ $ kubectl delete pod ${pod_name} --namespace ${namespace}
**After entering the pod, you need to modify the port information of the
configuration file before you can manually start the corresponding Doris
component.**
-- FE needs to modify the `http_port=8030` configuration with the default path:
`/opt/apache-doris/fe/conf/fe.conf`.
-- BE needs to modify the `webserver_port=8040` configuration with the default
path: `/opt/apache-doris/be/conf/be.conf`.
+- FE needs to modify the `query_port=9030` configuration with the default
path: `/opt/apache-doris/fe/conf/fe.conf`.
+- BE needs to modify the `heartbeat_service_port=9050` configuration with the
default path: `/opt/apache-doris/be/conf/be.conf`.
diff --git a/docs/install/k8s-deploy/install-doris-cluster.md
b/docs/install/k8s-deploy/install-doris-cluster.md
index ccaada2ecdf4..b425c4b95409 100644
--- a/docs/install/k8s-deploy/install-doris-cluster.md
+++ b/docs/install/k8s-deploy/install-doris-cluster.md
@@ -95,6 +95,8 @@ kubectl apply -f ./${cluster_sample}.yaml -n ${namespace}
**Online Deployment**
+Before installation, you need to add a deployment warehouse. If it has been
added, you can directly install Doris Cluster. Otherwise, please refer to the
operation of **Add Deployment Warehouse** when adding [Deploy Doris
Operator](<./install-operator.md#Option 3: Helm deploys Doris Operator>)
+
1. Install Doris Cluster
Install [doriscluster](https://artifacthub.io/packages/helm/doris/doris),
using the default configuration this deployment only deploys 3 FE and 3 BE
components, using the default `storageClass` to implement PV dynamic
provisioning.
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/install/k8s-deploy/debug-crash.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/install/k8s-deploy/debug-crash.md
index f135f761ab53..63b64fb574ff 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/install/k8s-deploy/debug-crash.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/install/k8s-deploy/debug-crash.md
@@ -68,6 +68,5 @@ $ kubectl delete pod ${pod_name} --namespace ${namespace}
**进入 pod 内部后，需要修改配置文件的端口信息，才能手动启动 相应的 Doris 组件**
-- FE 需要修改默认路径为：`/opt/apache-doris/fe/conf/fe.conf` 的 `http_port=8030` 配置。
-- BE 需要修改默认路径为：`/opt/apache-doris/be/conf/be.conf` 的 `webserver_port=8040` 配置。
-
+- FE 需要修改默认路径为：`/opt/apache-doris/fe/conf/fe.conf` 的 `query_port=9030` 配置。
+- BE 需要修改默认路径为：`/opt/apache-doris/be/conf/be.conf` 的
`heartbeat_service_port=9050` 配置。
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/install/k8s-deploy/install-doris-cluster.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/install/k8s-deploy/install-doris-cluster.md
index 9241c58be099..343c2a5335ad 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/install/k8s-deploy/install-doris-cluster.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/install/k8s-deploy/install-doris-cluster.md
@@ -95,6 +95,8 @@ kubectl apply -f ./${cluster_sample}.yaml -n ${namespace}
**在线部署**
+在安装开始前，需要添加部署仓库，若已经添加则可直接进行 Doris Cluster 的安装，否则请参考添加[部署 Doris
Operator](./install-operator.md#%E6%96%B9%E6%A1%88%E4%B8%89helm-%E9%83%A8%E7%BD%B2-doris-operator)
时
Re: [PR] [Fix](core) Fix wal mgr heap use after free when stop doris [doris]
Yukang-Lian commented on PR #33131: URL: https://github.com/apache/doris/pull/33131#issuecomment-2031242065 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](hive)support `ExternalTransaction` for writing exteral table. [doris]
github-actions[bot] commented on PR #32726: URL: https://github.com/apache/doris/pull/32726#issuecomment-2031242950 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](hive)support `ExternalTransaction` for writing exteral table. [doris]
github-actions[bot] commented on PR #32726: URL: https://github.com/apache/doris/pull/32726#issuecomment-2031243000 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](spill) avoid small spill write and improve spill timers [doris]
jacktengg commented on PR #33156: URL: https://github.com/apache/doris/pull/33156#issuecomment-2031246365 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](insert)process tbl/db exist when create/drop db/tbl [doris]
morningman commented on code in PR #33119:
URL: https://github.com/apache/doris/pull/33119#discussion_r1547271992
##
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergMetadataOps.java:
##
@@ -109,6 +132,14 @@ public void createTable(CreateTableStmt stmt) throws
UserException {
throw new UserException("Failed to get database: '" + dbName + "'
in catalog: " + dorisCatalog.getName());
}
String tableName = stmt.getTableName();
+if (db.getTable(tableName).isPresent()) {
Review Comment:
Same comment as hive catalog
##
fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveMetadataOps.java:
##
@@ -186,24 +212,22 @@ public void createTable(CreateTableStmt stmt) throws
UserException {
}
}
-private static List parsePartitionKeys(Map
props) {
-List parsedKeys = new ArrayList<>();
-String pkStr = props.getOrDefault("partition_keys", "");
-if (pkStr.isEmpty()) {
-return parsedKeys;
-} else {
-// TODO: parse string to partition keys list
-return parsedKeys;
-}
-}
-
@Override
public void dropTable(DropTableStmt stmt) throws DdlException {
String dbName = stmt.getDbName();
ExternalDatabase db = catalog.getDbNullable(stmt.getDbName());
if (db == null) {
throw new DdlException("Failed to get database: '" + dbName + "'
in catalog: " + catalog.getName());
}
+ExternalTable table = db.getTableNullable(stmt.getTableName());
Review Comment:
I think we should check whether table exist in hms, not just in Doris.
Same as `databaseExist()`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](core) Fix wal mgr heap use after free when stop doris [doris]
github-actions[bot] commented on PR #33131: URL: https://github.com/apache/doris/pull/33131#issuecomment-2031250424 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](spill) avoid small spill write and improve spill timers [doris]
github-actions[bot] commented on PR #33156: URL: https://github.com/apache/doris/pull/33156#issuecomment-2031255190 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](spill) avoid small spill write and improve spill timers [doris]
github-actions[bot] commented on PR #33156: URL: https://github.com/apache/doris/pull/33156#issuecomment-2031256489 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch master updated: [docs](BI & IDE) add DBeaver and DataGrip connect Doris doc (#505)
This is an automated email from the ASF dual-hosted git repository.
zykkk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 42ba42cba7ef [docs](BI & IDE) add DBeaver and DataGrip connect Doris
doc (#505)
42ba42cba7ef is described below
commit 42ba42cba7ef98c1bb5173cb07ad5a42f3adc4bf
Author: zy-kkk
AuthorDate: Tue Apr 2 15:27:34 2024 +0800
[docs](BI & IDE) add DBeaver and DataGrip connect Doris doc (#505)
---
.../{bi-integration => bi}/apache-superset.md | 2 +-
docs/ecosystem/bi/datagrip.md | 89 +++
docs/ecosystem/bi/dbeaver.md | 122
.../docusaurus-plugin-content-docs/current.json| 4 +
.../ecosystem/bi-integration/apache-superset.md| 58 --
.../ecosystem/bi}/apache-superset.md | 0
.../current/ecosystem/bi/datagrip.md | 89 +++
.../current/ecosystem/bi/dbeaver.md| 127 +
.../version-2.0.json | 4 +
.../ecosystem/{ => bi}/apache-superset.md | 0
.../version-2.0/ecosystem/bi/datagrip.md | 89 +++
.../version-2.0/ecosystem/bi/dbeaver.md| 127 +
.../version-2.1.json | 4 +
.../ecosystem/bi-integration/apache-superset.md| 58 --
.../ecosystem/bi}/apache-superset.md | 0
.../version-2.1/ecosystem/bi/datagrip.md | 89 +++
.../version-2.1/ecosystem/bi/dbeaver.md| 127 +
sidebars.json | 10 +-
static/images/datagrip1.png| Bin 0 -> 230855 bytes
static/images/datagrip2.png| Bin 0 -> 189679 bytes
static/images/datagrip3.png| Bin 0 -> 192801 bytes
static/images/datagrip4.png| Bin 0 -> 265627 bytes
static/images/datagrip5.png| Bin 0 -> 64529 bytes
static/images/dbeaver1.png | Bin 0 -> 80879 bytes
static/images/dbeaver2.png | Bin 0 -> 165711 bytes
static/images/dbeaver3.png | Bin 0 -> 392716 bytes
static/images/dbeaver4.png | Bin 0 -> 230780 bytes
static/images/dbeaver5.png | Bin 0 -> 234243 bytes
static/images/dbeaver6.png | Bin 0 -> 394581 bytes
static/images/dbeaver7.png | Bin 0 -> 123712 bytes
.../ecosystem/{ => bi}/apache-superset.md | 2 +-
.../version-2.0/ecosystem/bi/datagrip.md | 89 +++
versioned_docs/version-2.0/ecosystem/bi/dbeaver.md | 122
.../{bi-integration => bi}/apache-superset.md | 2 +-
.../version-2.1/ecosystem/bi/datagrip.md | 89 +++
versioned_docs/version-2.1/ecosystem/bi/dbeaver.md | 122
versioned_sidebars/version-2.0-sidebars.json | 12 +-
versioned_sidebars/version-2.1-sidebars.json | 10 +-
38 files changed, 1324 insertions(+), 123 deletions(-)
diff --git a/docs/ecosystem/bi-integration/apache-superset.md
b/docs/ecosystem/bi/apache-superset.md
similarity index 98%
rename from docs/ecosystem/bi-integration/apache-superset.md
rename to docs/ecosystem/bi/apache-superset.md
index 0ada613da752..6c172c17ec1c 100644
--- a/docs/ecosystem/bi-integration/apache-superset.md
+++ b/docs/ecosystem/bi/apache-superset.md
@@ -1,6 +1,6 @@
---
{
-"title": "Apache Superset Integrate",
+"title": "Apache Superset",
"language": "en"
}
---
diff --git a/docs/ecosystem/bi/datagrip.md b/docs/ecosystem/bi/datagrip.md
new file mode 100644
index ..f92c70a9bdb3
--- /dev/null
+++ b/docs/ecosystem/bi/datagrip.md
@@ -0,0 +1,89 @@
+---
+{
+"title": "DataGrip",
+"language": "en"
+}
+---
+
+
+
+## introduce
+
+DataGrip is a powerful cross-platform database tool for relational and NoSQL
databases from JetBrains.
+
+Apache Doris is highly compatible with the MySQL protocol. You can use
DataGrip's MySQL data source to connect to Apache Doris and query data in the
internal catalog and external catalog.
+
+## Preconditions
+
+DataGrip installed
+You can visit www.jetbrains.com/datagrip/ to download and install DataGrip
+
+## Add data source
+
+:::info Note
+Currently verified using DataGrip version 2023.3.4
+:::
+
+1. Start DataGrip
+2. Click the plus sign (**+**) icon in the upper left corner of the DataGrip
window and select the MySQL data source
+
+
+
+3. Configure Doris connection
+
+On the General tab of the Data Sources and Drivers window, configure the
following connection information:
+
+ - Host: FE host IP address of the Doris cluster.
+ - Port: FE query port of D
Re: [PR] [Chore](build) add -Wconversion and remove some unused code [doris]
doris-robot commented on PR #33127: URL: https://github.com/apache/doris/pull/33127#issuecomment-2031261906 TeamCity be ut coverage result: Function Coverage: 35.59% (8852/24869) Line Coverage: 27.30% (72540/265762) Region Coverage: 26.49% (37546/141719) Branch Coverage: 23.29% (19135/82160) Coverage Report: http://coverage.selectdb-in.cc/coverage/1aab889d614902e7199bf70a6c60436aa48531fe_1aab889d614902e7199bf70a6c60436aa48531fe/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [docs](jdbc) modify prompt box level [doris-website]
zy-kkk merged PR #506: URL: https://github.com/apache/doris-website/pull/506 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch master updated (03e6571eb2e5 -> 43d9fd9df273)
This is an automated email from the ASF dual-hosted git repository. zykkk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git from 03e6571eb2e5 [doc](k8s)k8s debug doc modify port (#497) add 43d9fd9df273 [docs](jdbc) modify prompt box level (#506) No new revisions were added by this update. Summary of changes: docs/lakehouse/multi-catalog/jdbc.md| 2 +- .../current/lakehouse/multi-catalog/jdbc.md | 2 +- .../version-2.1/lakehouse/multi-catalog/jdbc.md | 2 +- versioned_docs/version-2.1/lakehouse/multi-catalog/jdbc.md | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [docs](BI & IDE) add DBeaver and DataGrip connect Doris doc [doris-website]
zy-kkk merged PR #505: URL: https://github.com/apache/doris-website/pull/505 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](nereids) support create view in nereids [doris]
924060929 commented on PR #32743: URL: https://github.com/apache/doris/pull/32743#issuecomment-2031268185 add a udf test case? e.g. ```sql use current_db; create view test_view as select other_db.my_udf(col) from other_db.tbl ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](nereids) forward the user define variables to master [doris]
feiniaofeiafei commented on PR #33013: URL: https://github.com/apache/doris/pull/33013#issuecomment-2031271720 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](pipeline) avoid unnecessary copying during projection. [doris]
Mryange commented on PR #33126: URL: https://github.com/apache/doris/pull/33126#issuecomment-2031273701 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](nereids) forward the user define variables to master [doris]
github-actions[bot] commented on PR #33013: URL: https://github.com/apache/doris/pull/33013#issuecomment-2031280563 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](pipeline) avoid unnecessary copying during projection. [doris]
github-actions[bot] commented on code in PR #33126:
URL: https://github.com/apache/doris/pull/33126#discussion_r1547306672
##
be/src/vec/core/block.cpp:
##
@@ -728,6 +728,64 @@ void Block::shuffle_columns(const std::vector&
result_column_ids) {
swap(Block {tmp_data});
}
+void Block::copy_block_from_result_column_id(Block& input_block,
Review Comment:
warning: method 'copy_block_from_result_column_id' can be made const
[readability-make-member-function-const]
be/src/vec/core/block.cpp:732:
```diff
- bool keep_origin) {
+ bool keep_origin) const {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Chore](build) add -Wconversion and remove some unused code [doris]
doris-robot commented on PR #33127: URL: https://github.com/apache/doris/pull/33127#issuecomment-2031282519 TPC-H: Total hot run time: 38841 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 1aab889d614902e7199bf70a6c60436aa48531fe, data reload: false -- Round 1 -- q1 17750 425040794079 q2 2019196 187 187 q3 10445 124214071242 q4 10204 884 1056884 q5 7466294329402940 q6 218 135 133 133 q7 1106661 620 620 q8 9420209520472047 q9 6668618361196119 q10 8475350335093503 q11 421 243 250 243 q12 393 222 221 221 q13 17776 292229412922 q14 270 241 249 241 q15 526 485 475 475 q16 527 390 381 381 q17 982 930 896 896 q18 7313660564416441 q19 1598154615321532 q20 590 309 307 307 q21 3598312531713125 q22 377 303 311 303 Total cold run time: 108142 ms Total hot run time: 38841 ms - Round 2, with runtime_filter_mode=off - q1 4089405040374037 q2 332 219 221 219 q3 2965295629612956 q4 1915189819041898 q5 5225524352475243 q6 213 127 130 127 q7 2245179318381793 q8 3236328032953280 q9 8488846584708465 q10 3770501840424042 q11 556 470 464 464 q12 821 646 633 633 q13 14138 313231193119 q14 310 295 259 259 q15 520 480 497 480 q16 500 431 439 431 q17 1773175317191719 q18 8179769275727572 q19 3262172416741674 q20 1960178118501781 q21 5346505050995050 q22 501 434 461 434 Total cold run time: 70344 ms Total hot run time: 55676 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [docs][fix]revise the max_filter_ration to max_filter_ratio [doris-website]
echo-hhj opened a new pull request, #507: URL: https://github.com/apache/doris-website/pull/507 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](mysql compatible) add user and procs_priv tables to mysql db in all catalogs [doris]
zy-kkk commented on code in PR #33058:
URL: https://github.com/apache/doris/pull/33058#discussion_r1547312219
##
fe/fe-core/src/main/java/org/apache/doris/analysis/SchemaTableType.java:
##
@@ -72,7 +72,9 @@ public enum SchemaTableType {
SCH_PROFILING("PROFILING", "PROFILING", TSchemaTableType.SCH_PROFILING),
SCH_BACKEND_ACTIVE_TASKS("BACKEND_ACTIVE_TASKS", "BACKEND_ACTIVE_TASKS",
TSchemaTableType.SCH_BACKEND_ACTIVE_TASKS),
SCH_ACTIVE_QUERIES("ACTIVE_QUERIES", "ACTIVE_QUERIES",
TSchemaTableType.SCH_ACTIVE_QUERIES),
-SCH_WORKLOAD_GROUPS("WORKLOAD_GROUPS", "WORKLOAD_GROUPS",
TSchemaTableType.SCH_WORKLOAD_GROUPS);
+SCH_WORKLOAD_GROUPS("WORKLOAD_GROUPS", "WORKLOAD_GROUPS",
TSchemaTableType.SCH_WORKLOAD_GROUPS),
+SCHE_USER("user", "user", TSchemaTableType.SCH_USER),
+SCH_PROCS_PRIV("procs_priv", "procs_priv",
TSchemaTableType.SCH_PROCS_PRIV);
Review Comment:
procs_priv does not need a scanner for the time being, just use
SchemaDummyScanner to read empty data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](insert)support catalog name when create/drop db [doris]
wsjz commented on PR #33116: URL: https://github.com/apache/doris/pull/33116#issuecomment-2031289242 run fe ut -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] Sankarshan mishra patch 1 [doris]
SankarshanMishra opened a new pull request, #33159: URL: https://github.com/apache/doris/pull/33159 ## added yaml file in .github/workflows Issue Number: close #8721 used Include what you use github action to remove unused headers. ## Further comments Kindly guide me and help me out because It is my first opensource contribution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Sankarshan mishra patch 1 [doris]
doris-robot commented on PR #33159: URL: https://github.com/apache/doris/pull/33159#issuecomment-2031290807 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](mysql compatible) add user and procs_priv tables to mysql db in all catalogs [doris]
zy-kkk commented on code in PR #33058:
URL: https://github.com/apache/doris/pull/33058#discussion_r1547313730
##
fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/client/JdbcClient.java:
##
@@ -409,6 +409,8 @@ protected List filterDatabaseNames(List
remoteDbNames) {
protected Set getFilterInternalDatabases() {
return ImmutableSet.builder()
.add("information_schema")
+.add("performance_schema")
Review Comment:
We do not need to obtain MySQL's performance_schema in Doris, which will
increase the burden of obtaining metadata, so filtering
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](insert)support catalog name when create/drop db [doris]
wsjz commented on PR #33116: URL: https://github.com/apache/doris/pull/33116#issuecomment-2031292650 run feut -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](arrow-flight) BE upgrade arrow to 15.0.2 [doris]
xinyiZzz merged PR #32805: URL: https://github.com/apache/doris/pull/32805 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](insert)support catalog name when create/drop db [doris]
github-actions[bot] commented on PR #33116: URL: https://github.com/apache/doris/pull/33116#issuecomment-2031294141 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](insert)support catalog name when create/drop db [doris]
morningman commented on code in PR #33116: URL: https://github.com/apache/doris/pull/33116#discussion_r1547310900 ## fe/fe-core/src/main/java/org/apache/doris/analysis/DbName.java: ## @@ -0,0 +1,86 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. +// This file is copied from +// https://github.com/apache/impala/blob/branch-2.9.0/fe/src/main/java/org/apache/impala/TableName.java Review Comment: Remove it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (e8648411fdd -> fde3fbf4fd9)
This is an automated email from the ASF dual-hosted git repository. zouxinyi pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from e8648411fdd [refactor](Mysql) Refactoring the process of using external components to authenticate in MySQL connections (#32875) add fde3fbf4fd9 [fix](arrow-flight) BE upgrade arrow to 15.0.2 (#32805) No new revisions were added by this update. Summary of changes: thirdparty/CHANGELOG.md | 4 thirdparty/vars.sh | 10 +- 2 files changed, 9 insertions(+), 5 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](insert)support catalog name when create/drop db [doris]
github-actions[bot] commented on PR #33116: URL: https://github.com/apache/doris/pull/33116#issuecomment-2031294145 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](mysql compatible) add user and procs_priv tables to mysql db in all catalogs [doris]
zy-kkk commented on code in PR #33058:
URL: https://github.com/apache/doris/pull/33058#discussion_r1547318530
##
fe/fe-core/src/main/java/org/apache/doris/datasource/infoschema/ExternalMysqlDatabase.java:
##
@@ -0,0 +1,55 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.datasource.infoschema;
+
+import org.apache.doris.catalog.MysqlDBTable;
+import org.apache.doris.catalog.MysqlDb;
+import org.apache.doris.datasource.ExternalCatalog;
+import org.apache.doris.datasource.ExternalDatabase;
+import org.apache.doris.datasource.ExternalTable;
+import org.apache.doris.datasource.InitDatabaseLog.Type;
+
+import com.google.common.collect.Lists;
+
+import java.util.List;
+
+public class ExternalMysqlDatabase extends ExternalDatabase {
Review Comment:
When creating ExternalCatalog, a database named mysql will be automatically
created. This is to be compatible with some BI tools. It is used in
ExternalCatalog.java. Unit tests and regression tests have been added to it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](mysql compatible) add user and procs_priv tables to mysql db in all catalogs [doris]
zy-kkk commented on PR #33058: URL: https://github.com/apache/doris/pull/33058#issuecomment-2031299460 > Please add regression test to select data from user/procs_priv, and show their schema. This part of the test will be added later. I will obtain the statements sent when connecting to Doris from various BI or Database IDE tools and add them to the regression test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](mysql compatible) add user and procs_priv tables to mysql db in all catalogs [doris]
zy-kkk commented on code in PR #33058:
URL: https://github.com/apache/doris/pull/33058#discussion_r1547321512
##
fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/Auth.java:
##
@@ -1937,4 +1938,97 @@ public String getDefaultCloudCluster(String user) {
return cluster;
}
// == END CLOUD ==
+
+// for mysql.user table
+public List> getAllUserInfo() {
Review Comment:
Because the data in userinfo requires permissions and password policies, FE
can better directly determine the data that needs to be sent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] test [doris]
wangbo commented on PR #32557: URL: https://github.com/apache/doris/pull/32557#issuecomment-2031305276 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Chore](build) add -Wconversion and remove some unused code [doris]
doris-robot commented on PR #33127: URL: https://github.com/apache/doris/pull/33127#issuecomment-2031307023 TPC-DS: Total hot run time: 182677 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 1aab889d614902e7199bf70a6c60436aa48531fe, data reload: false query1 1243113311071107 query2 6189187318261826 query3 6675207 214 207 query4 24906 21806 21560 21560 query5 4149406 406 406 query6 273 189 181 181 query7 4599304 303 303 query8 233 175 175 175 query9 8465225622452245 query10 444 252 274 252 query11 14957 14470 14409 14409 query12 145 99 94 94 query13 1649386 391 386 query14 8539694470566944 query15 210 178 180 178 query16 6727275 269 269 query17 988 592 592 592 query18 1845291 286 286 query19 207 164 162 162 query20 98 96 94 94 query21 193 134 129 129 query22 4987481848164816 query23 33587 32839 32856 32839 query24 13124 316132363161 query25 723 438 431 431 query26 1895170 177 170 query27 3294377 415 377 query28 7034190718851885 query29 1317617 606 606 query30 314 176 178 176 query31 1048764 749 749 query32 97 63 59 59 query33 689 244 281 244 query34 1299511 521 511 query35 838 747 769 747 query36 1043886 872 872 query37 277 89 85 85 query38 3691350035753500 query39 1616160616111606 query40 240 150 149 149 query41 46 43 48 43 query42 112 115 106 106 query43 462 411 401 401 query44 1132730 746 730 query45 271 262 262 262 query46 1118847 833 833 query47 1945187519071875 query48 381 321 303 303 query49 936 365 371 365 query50 820 399 430 399 query51 7054687069836870 query52 107 97 104 97 query53 383 308 302 302 query54 304 236 245 236 query55 97 81 80 80 query56 253 237 225 225 query57 1260120711711171 query58 292 245 240 240 query59 2689246224072407 query60 264 242 248 242 query61 111 106 109 106 query62 667 469 461 461 query63 321 288 288 288 query64 5865315431853154 query65 3067304430333033 query66 1312336 321 321 query67 15720 15135 14962 14962 query68 7927583 584 583 query69 557 338 339 338 query70 1261111510721072 query71 499 285 282 282 query72 6347258124202420 query73 821 320 323 320 query74 6648627262766272 query75 3413232322842284 query76 5097107011701070 query77 566 262 253 253 query78 10949 10209 10136 10136 query79 10082 528 543 528 query80 1620417 427 417 query81 534 244 230 230 query82 725 106 99 99 query83 207 160 162 160 query84 267 87 86 86 query85 1369303 286 286 query86 474 268 261 261 query87 3628348034593459 query88 4355226522702265 query89 547 361 363 361 query90 1960171 183 171 query91 133 108 106 106 query92 62 50 53 50 query93 7199536 534 534 query94 1134192 196 192 query95 443 340 336 336 query96 607 277 267 267 query97 2655252624892489 query98 232 224 217 217 query99 1276848 833 833 Total cold run time: 300430 ms Total hot run time: 182677 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [improvement](partial-update) add more regression-test for mow partial update [doris]
doris-robot commented on PR #33160: URL: https://github.com/apache/doris/pull/33160#issuecomment-2031308148 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](partial-update) add more regression-test for mow partial update [doris]
hust-hhb commented on PR #33160: URL: https://github.com/apache/doris/pull/33160#issuecomment-2031308447 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](insert)process tbl/db exist when create/drop db/tbl [doris]
wsjz commented on PR #33119: URL: https://github.com/apache/doris/pull/33119#issuecomment-2031309775 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [improvement](partial-update) add more regression-test for mow partial update [doris]
hust-hhb opened a new pull request, #33160: URL: https://github.com/apache/doris/pull/33160 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](partial-update) make partial-update on agg table use less memory [doris]
doris-robot commented on PR #32200: URL: https://github.com/apache/doris/pull/32200#issuecomment-2031311338 TPC-H: Total hot run time: 38826 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit e80c8cd7e4e8ca2f5a78349f7faa8564c78dc4e0, data reload: false -- Round 1 -- q1 17608 415041334133 q2 2013187 188 187 q3 10481 124614151246 q4 10210 872 978 872 q5 7987303029242924 q6 219 138 133 133 q7 1117618 609 609 q8 9418205720422042 q9 6704622561756175 q10 8452354535013501 q11 424 252 252 252 q12 389 219 213 213 q13 1 291029072907 q14 269 246 243 243 q15 545 489 479 479 q16 502 405 379 379 q17 971 919 924 919 q18 7343657163996399 q19 1587153415371534 q20 592 319 307 307 q21 3523310830653065 q22 388 320 307 307 Total cold run time: 108519 ms Total hot run time: 38826 ms - Round 2, with runtime_filter_mode=off - q1 4105417740954095 q2 325 214 220 214 q3 2948297329622962 q4 1883186918441844 q5 5230523652325232 q6 206 125 125 125 q7 2259182418071807 q8 3227336232973297 q9 8522848984908489 q10 3768403140154015 q11 574 477 485 477 q12 769 626 649 626 q13 16880 316931433143 q14 324 301 282 282 q15 531 484 498 484 q16 501 436 442 436 q17 1770176517781765 q18 8196774277417741 q19 1694168716671667 q20 1970180218491802 q21 5214492950104929 q22 515 438 452 438 Total cold run time: 71411 ms Total hot run time: 55870 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](core) Fix wal mgr heap use after free when stop doris [doris]
doris-robot commented on PR #33131: URL: https://github.com/apache/doris/pull/33131#issuecomment-2031315468 TPC-H: Total hot run time: 39193 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit da2877e439d53ef0bda0080cdcf530aaf01e43a6, data reload: false -- Round 1 -- q1 19552 426341994199 q2 2470196 191 191 q3 11601 127214691272 q4 10760 903 1050903 q5 8376312330463046 q6 225 141 139 139 q7 1191698 644 644 q8 9418202120612021 q9 6690622261856185 q10 8478352735053505 q11 429 241 229 229 q12 387 217 218 217 q13 1 290229182902 q14 281 243 242 242 q15 531 484 489 484 q16 521 380 373 373 q17 988 885 918 885 q18 7387654665046504 q19 1625156715491549 q20 623 324 321 321 q21 3635313530723072 q22 357 310 312 310 Total cold run time: 113302 ms Total hot run time: 39193 ms - Round 2, with runtime_filter_mode=off - q1 4129403840554038 q2 328 216 221 216 q3 2988297429662966 q4 1903182618281826 q5 5254525452405240 q6 209 126 126 126 q7 2256180318291803 q8 3226330332873287 q9 8677846984768469 q10 3764383238443832 q11 561 452 449 449 q12 744 563 534 534 q13 11601 289529212895 q14 286 262 261 261 q15 520 476 461 461 q16 449 404 399 399 q17 1727168716861686 q18 7631741372387238 q19 1649164516641645 q20 1925171417141714 q21 5039480048444800 q22 504 421 421 421 Total cold run time: 65370 ms Total hot run time: 54306 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Chore](build) add -Wconversion and remove some unused code [doris]
doris-robot commented on PR #33127: URL: https://github.com/apache/doris/pull/33127#issuecomment-2031317154 ClickBench: Total hot run time: 29.95 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 1aab889d614902e7199bf70a6c60436aa48531fe, data reload: false query1 0.040.030.03 query2 0.080.040.04 query3 0.240.050.05 query4 1.670.070.07 query5 0.470.470.49 query6 1.140.650.66 query7 0.020.010.00 query8 0.050.040.05 query9 0.560.520.51 query10 0.580.550.56 query11 0.150.110.11 query12 0.140.120.11 query13 0.600.600.59 query14 0.780.780.79 query15 0.870.840.83 query16 0.350.350.36 query17 1.041.010.99 query18 0.260.240.25 query19 1.841.701.72 query20 0.020.010.01 query21 15.40 0.770.67 query22 3.345.651.59 query23 17.38 1.301.06 query24 1.500.370.22 query25 0.130.090.08 query26 0.300.170.18 query27 0.080.090.08 query28 13.43 0.980.98 query29 12.63 3.413.40 query30 0.260.060.05 query31 2.870.390.38 query32 3.260.470.46 query33 2.892.902.84 query34 15.47 4.364.37 query35 4.374.384.39 query36 0.660.460.47 query37 0.210.180.17 query38 0.180.170.16 query39 0.050.040.04 query40 0.190.150.14 query41 0.090.050.04 query42 0.060.050.06 query43 0.050.050.05 Total cold run time: 105.7 s Total hot run time: 29.95 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](grace-exit) Fix threadpool hang on grace exit [doris]
github-actions[bot] commented on PR #33132: URL: https://github.com/apache/doris/pull/33132#issuecomment-2031319080 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](grace-exit) Fix threadpool hang on grace exit [doris]
zhiqiang- commented on code in PR #33132:
URL: https://github.com/apache/doris/pull/33132#discussion_r1547336369
##
be/src/runtime/exec_env_init.cpp:
##
@@ -597,6 +593,7 @@ void ExecEnv::destroy() {
_delta_writer_v2_pool.reset();
_load_stream_stub_pool.reset();
_file_cache_open_fd_cache.reset();
+_write_cooldown_meta_executors.reset();
Review Comment:
call `_write_cooldown_meta_executors.stop();` before it is destoried. and
the calling of this function should be move forward.
##
be/src/olap/tablet.h:
##
@@ -84,6 +80,27 @@ enum TabletStorageType { STORAGE_TYPE_LOCAL,
STORAGE_TYPE_REMOTE, STORAGE_TYPE_R
static inline constexpr auto TRACE_TABLET_LOCK_THRESHOLD =
std::chrono::seconds(1);
+struct WriteCooldownMetaExecutors {
+WriteCooldownMetaExecutors(size_t executor_nums = 5);
+
+~WriteCooldownMetaExecutors();
+
+void submit(TabletSharedPtr tablet);
Review Comment:
add a method `void stop()`, which stops all the executors.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](hive)support `ExternalTransaction` for writing exteral table. [doris]
wuwenchi commented on PR #32726: URL: https://github.com/apache/doris/pull/32726#issuecomment-2031321783 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Chore](build) add -Wconversion and remove some unused code [doris]
doris-robot commented on PR #33127: URL: https://github.com/apache/doris/pull/33127#issuecomment-2031323819 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit 1aab889d614902e7199bf70a6c60436aa48531fe with default session variables Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s Insert into select: 15.2 seconds inserted 1000 Rows, about 657K ops/s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [log](move-memtable) improve log in vtablet writer v2 and load stream [doris]
doris-robot commented on PR #33103: URL: https://github.com/apache/doris/pull/33103#issuecomment-2031327118 TPC-H: Total hot run time: 38964 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 61cffca91b07a36ae7fb7a3ae1084361467d7abe, data reload: false -- Round 1 -- q1 17731 410940674067 q2 2009186 186 186 q3 10478 123114201231 q4 10208 895 1008895 q5 7492301529572957 q6 220 132 130 130 q7 1101650 615 615 q8 9405209520522052 q9 6691618561586158 q10 8469351635093509 q11 427 247 256 247 q12 397 225 221 221 q13 17786 288829262888 q14 273 247 249 247 q15 537 493 492 492 q16 506 388 375 375 q17 1000936 932 932 q18 7342654364466446 q19 2651155415411541 q20 624 314 304 304 q21 3591315731823157 q22 374 314 314 314 Total cold run time: 109312 ms Total hot run time: 38964 ms - Round 2, with runtime_filter_mode=off - q1 4092404740294029 q2 324 217 217 217 q3 2994294929562949 q4 1864187818571857 q5 5233522252165216 q6 208 122 123 122 q7 2229180518201805 q8 3208329133313291 q9 8534848084918480 q10 3772403039983998 q11 560 459 479 459 q12 755 614 612 612 q13 16871 319031063106 q14 301 267 283 267 q15 522 478 480 478 q16 518 415 434 415 q17 1781178017661766 q18 8212792576787678 q19 1666165216611652 q20 2034184118101810 q21 5130497449644964 q22 522 441 454 441 Total cold run time: 71330 ms Total hot run time: 55612 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](backup) ignore table that not support type when backup, and… [doris]
doris-robot commented on PR #33158: URL: https://github.com/apache/doris/pull/33158#issuecomment-2031327265 TPC-H: Total hot run time: 38743 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 1712af489575a8b482c782b026e765009053f561, data reload: false -- Round 1 -- q1 17601 417740834083 q2 2013191 195 191 q3 10467 121514161215 q4 10192 875 1027875 q5 7485300129442944 q6 219 139 136 136 q7 1106616 610 610 q8 9411203520462035 q9 6699620561566156 q10 8462352535863525 q11 420 244 236 236 q12 386 217 216 216 q13 17774 287028832870 q14 277 247 245 245 q15 528 511 478 478 q16 518 379 367 367 q17 968 924 924 924 q18 7246648163956395 q19 1610154415351535 q20 610 309 317 309 q21 3484313730853085 q22 361 321 313 313 Total cold run time: 107837 ms Total hot run time: 38743 ms - Round 2, with runtime_filter_mode=off - q1 4069414440624062 q2 328 219 222 219 q3 2941292629402926 q4 1874182718671827 q5 5209520952155209 q6 208 127 131 127 q7 2236179718201797 q8 3215331032763276 q9 8479847185058471 q10 3771395140213951 q11 567 469 460 460 q12 756 613 594 594 q13 16770 307231183072 q14 308 277 278 277 q15 561 505 497 497 q16 509 441 445 441 q17 1796173517491735 q18 8225781176497649 q19 1672169016821682 q20 2017185418651854 q21 5042491549444915 q22 516 440 432 432 Total cold run time: 71069 ms Total hot run time: 55473 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](spill) avoid small spill write and improve spill timers [doris]
doris-robot commented on PR #33156: URL: https://github.com/apache/doris/pull/33156#issuecomment-2031329718 TPC-H: Total hot run time: 38933 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit efc7cdcf464e40c3438bce71d81a51d58dc52cf9, data reload: false -- Round 1 -- q1 17687 424140704070 q2 2009193 186 186 q3 10534 132514521325 q4 10376 871 1019871 q5 7496302029532953 q6 218 134 135 134 q7 1097635 625 625 q8 9414204820332033 q9 6637618761566156 q10 8476352835353528 q11 426 250 237 237 q12 383 220 216 216 q13 17785 289229282892 q14 280 249 247 247 q15 536 494 497 494 q16 512 402 384 384 q17 971 934 901 901 q18 7428641264776412 q19 1588153615441536 q20 616 315 320 315 q21 3569313931013101 q22 387 317 319 317 Total cold run time: 108425 ms Total hot run time: 38933 ms - Round 2, with runtime_filter_mode=off - q1 4103402240744022 q2 331 216 217 216 q3 2999294229612942 q4 1870187418551855 q5 5218522152355221 q6 211 125 127 125 q7 2227180118101801 q8 3207328932643264 q9 8474851285198512 q10 3771488340624062 q11 561 489 464 464 q12 771 651 592 592 q13 16836 317531143114 q14 324 279 288 279 q15 528 479 499 479 q16 504 434 427 427 q17 1760171617741716 q18 8311767877747678 q19 1726165116661651 q20 1988183618601836 q21 5203497649634963 q22 532 448 447 447 Total cold run time: 71455 ms Total hot run time: 55666 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch high-priority-column updated: Fix ut (#33161)
This is an automated email from the ASF dual-hosted git repository.
lijibing pushed a commit to branch high-priority-column
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/high-priority-column by this
push:
new f8ad6b9c864 Fix ut (#33161)
f8ad6b9c864 is described below
commit f8ad6b9c8646e7024587f310b6c163aa65e29fff
Author: Jibing-Li <64681310+jibing...@users.noreply.github.com>
AuthorDate: Tue Apr 2 16:02:28 2024 +0800
Fix ut (#33161)
---
.../org/apache/doris/statistics/util/StatisticsUtilTest.java | 10 +-
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/statistics/util/StatisticsUtilTest.java
b/fe/fe-core/src/test/java/org/apache/doris/statistics/util/StatisticsUtilTest.java
index 3be11b4d931..275471a6698 100644
---
a/fe/fe-core/src/test/java/org/apache/doris/statistics/util/StatisticsUtilTest.java
+++
b/fe/fe-core/src/test/java/org/apache/doris/statistics/util/StatisticsUtilTest.java
@@ -195,7 +195,7 @@ class StatisticsUtilTest {
new MockUp() {
@Mock
-public ColStatsMeta findColumnStatsMeta(String colName) {
+public ColStatsMeta findColumnStatsMeta(String indexName, String
colName) {
return new ColStatsMeta(0, null, null, null, 0, 0, 0);
}
};
@@ -243,7 +243,7 @@ class StatisticsUtilTest {
};
new MockUp() {
@Mock
-public ColStatsMeta findColumnStatsMeta(String colName) {
+public ColStatsMeta findColumnStatsMeta(String indexName, String
colName) {
return new ColStatsMeta(0, null, null, null, 0, 100, 0);
}
};
@@ -253,7 +253,7 @@ class StatisticsUtilTest {
// Test table still empty.
new MockUp() {
@Mock
-public ColStatsMeta findColumnStatsMeta(String colName) {
+public ColStatsMeta findColumnStatsMeta(String indexName, String
colName) {
return new ColStatsMeta(0, null, null, null, 0, 0, 0);
}
};
@@ -269,7 +269,7 @@ class StatisticsUtilTest {
};
new MockUp() {
@Mock
-public ColStatsMeta findColumnStatsMeta(String colName) {
+public ColStatsMeta findColumnStatsMeta(String indexName, String
colName) {
return new ColStatsMeta(0, null, null, null, 0, 500, 0);
}
};
@@ -285,7 +285,7 @@ class StatisticsUtilTest {
};
new MockUp() {
@Mock
-public ColStatsMeta findColumnStatsMeta(String colName) {
+public ColStatsMeta findColumnStatsMeta(String indexName, String
colName) {
return new ColStatsMeta(0, null, null, null, 0, 100, 80);
}
};
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[PR] Fix ut [doris]
Jibing-Li opened a new pull request, #33161: URL: https://github.com/apache/doris/pull/33161 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Fix ut [doris]
doris-robot commented on PR #33161: URL: https://github.com/apache/doris/pull/33161#issuecomment-2031333085 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Fix ut [doris]
Jibing-Li merged PR #33161: URL: https://github.com/apache/doris/pull/33161 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](partial-update) make partial-update on agg table use less memory [doris]
doris-robot commented on PR #32200: URL: https://github.com/apache/doris/pull/32200#issuecomment-2031334266 TPC-DS: Total hot run time: 183192 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit e80c8cd7e4e8ca2f5a78349f7faa8564c78dc4e0, data reload: false query1 1204112610971097 query2 6204195919161916 query3 214 208 208 query4 24686 21508 21472 21472 query5 4211400 406 400 query6 276 200 189 189 query7 4594308 308 308 query8 232 186 180 180 query9 8454229522672267 query10 456 255 259 255 query11 15040 14435 14437 14435 query12 140 95 92 92 query13 1651390 391 390 query14 8417713269306930 query15 206 178 180 178 query16 6428284 284 284 query17 951 600 578 578 query18 1756300 279 279 query19 207 165 164 164 query20 101 98 94 94 query21 198 131 134 131 query22 5042486648094809 query23 33511 32823 32787 32787 query24 13210 330831963196 query25 724 443 436 436 query26 1867176 167 167 query27 3250380 392 380 query28 7031195219281928 query29 1155620 602 602 query30 306 174 176 174 query31 1066756 754 754 query32 103 60 60 60 query33 688 263 261 261 query34 1258509 545 509 query35 829 740 739 739 query36 1000889 875 875 query37 230 91 85 85 query38 3772364236673642 query39 1636157916451579 query40 237 139 138 138 query41 49 44 42 42 query42 125 101 107 101 query43 459 403 419 403 query44 1128734 724 724 query45 287 257 267 257 query46 1137837 809 809 query47 2006193519521935 query48 397 327 323 323 query49 943 353 388 353 query50 827 423 421 421 query51 6936702069576957 query52 108 100 102 100 query53 379 305 291 291 query54 295 241 237 237 query55 90 76 74 74 query56 251 239 237 237 query57 1310120312191203 query58 252 223 233 223 query59 2756243923912391 query60 269 248 249 248 query61 110 108 109 108 query62 686 455 458 455 query63 306 280 285 280 query64 6034358832813281 query65 3107301530163015 query66 1308320 322 320 query67 15351 14810 14973 14810 query68 8139600 600 600 query69 561 332 336 332 query70 1204108310771077 query71 497 282 275 275 query72 6400255724142414 query73 802 332 326 326 query74 6759628763446287 query75 3451228723292287 query76 4834113612251136 query77 566 244 251 244 query78 10909 10184 10168 10168 query79 9023541 538 538 query80 1443424 416 416 query81 503 230 241 230 query82 726 115 104 104 query83 203 160 163 160 query84 271 90 87 87 query85 1369290 278 278 query86 444 278 309 278 query87 3671349234723472 query88 4455235523752355 query89 556 381 376 376 query90 1902177 178 177 query91 133 107 107 107 query92 64 50 52 50 query93 7126535 536 535 query94 1042197 190 190 query95 440 328 331 328 query96 613 284 272 272 query97 2670249225492492 query98 229 221 210 210 query99 1313825 841 825 Total cold run time: 298217 ms Total hot run time: 183192 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
(doris) branch high-priority-column updated (f8ad6b9c864 -> f6ac3af46d3)
This is an automated email from the ASF dual-hosted git repository. lijibing pushed a change to branch high-priority-column in repository https://gitbox.apache.org/repos/asf/doris.git discard f8ad6b9c864 Fix ut (#33161) omit 60bb7cdd687 Support high priority column stats auto collection. * fix visible column (#33023) * Collect high priority columns. (#31235) * High priority queue and map. (#31509) * Support column level health value. (#31794) * Support follower sync query columns to master. (#31859) * Support show auto analyze pending jobs. (#31926) * Check column health value earlier, show job priority. (#32064) * support window (#32094) * Refactor. (#32273) * refactor2 (#32278) * Unit test (#32398) [...] new f6ac3af46d3 Support high priority column stats auto collection. * fix visible column (#33023) * Collect high priority columns. (#31235) * High priority queue and map. (#31509) * Support column level health value. (#31794) * Support follower sync query columns to master. (#31859) * Support show auto analyze pending jobs. (#31926) * Check column health value earlier, show job priority. (#32064) * support window (#32094) * Refactor. (#32273) * refactor2 (#32278) * Unit test (#32398) [...] This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (f8ad6b9c864) \ N -- N -- N refs/heads/high-priority-column (f6ac3af46d3) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](core) Fix wal mgr heap use after free when stop doris [doris]
doris-robot commented on PR #33131: URL: https://github.com/apache/doris/pull/33131#issuecomment-2031338587 TPC-DS: Total hot run time: 181459 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit da2877e439d53ef0bda0080cdcf530aaf01e43a6, data reload: false query1 1218374 1123374 query2 6497187918161816 query3 6669218 219 218 query4 24922 21627 21385 21385 query5 4158402 409 402 query6 283 185 185 185 query7 4611316 310 310 query8 230 179 176 176 query9 8496227722632263 query10 570 258 276 258 query11 14864 14567 14572 14567 query12 146 96 100 96 query13 1634379 372 372 query14 8632691169066906 query15 214 178 186 178 query16 7160289 272 272 query17 991 612 578 578 query18 1910282 277 277 query19 203 161 161 161 query20 99 93 94 93 query21 198 128 124 124 query22 4974485647544754 query23 33676 32660 32498 32498 query24 12676 323531663166 query25 722 403 415 403 query26 1915169 161 161 query27 3003323 339 323 query28 6699186818511851 query29 1390595 594 594 query30 307 163 167 163 query31 1012738 727 727 query32 98 62 57 57 query33 715 264 252 252 query34 1056489 512 489 query35 832 692 705 692 query36 997 843 872 843 query37 283 77 80 77 query38 3525341733963396 query39 1569153015151515 query40 299 133 137 133 query41 50 47 45 45 query42 109 104 104 104 query43 433 403 398 398 query44 1110712 700 700 query45 277 255 253 253 query46 1069790 782 782 query47 1901177217921772 query48 368 306 304 304 query49 1168366 360 360 query50 793 401 391 391 query51 6828681068256810 query52 117 95 98 95 query53 359 290 286 286 query54 338 237 247 237 query55 88 77 81 77 query56 256 231 226 226 query57 1221113211261126 query58 255 225 230 225 query59 2546250524462446 query60 260 255 252 252 query61 114 109 109 109 query62 693 471 452 452 query63 308 287 289 287 query64 6464355033113311 query65 3080302130033003 query66 1444334 333 333 query67 15614 15308 14912 14912 query68 7405564 554 554 query69 541 325 315 315 query70 1205113410921092 query71 508 272 263 263 query72 6315256824242424 query73 791 323 332 323 query74 6677632262086208 query75 3520231622822282 query76 5053111611971116 query77 615 250 249 249 query78 10777 10233 10016 10016 query79 8743568 518 518 query80 1354426 417 417 query81 513 247 238 238 query82 732 102 108 102 query83 205 159 163 159 query84 282 88 93 88 query85 1459299 287 287 query86 433 298 287 287 query87 3684349734853485 query88 4342235923622359 query89 560 368 374 368 query90 2025175 191 175 query91 132 103 103 103 query92 66 53 53 53 query93 6891542 529 529 query94 1133195 197 195 query95 1092109310981093 query96 610 274 276 274 query97 2678244724582447 query98 238 221 212 212 query99 1325872 827 827 Total cold run time: 298620 ms Total hot run time: 181459 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [feature](statistics)Support high priority column stats auto collection. [doris]
Jibing-Li closed pull request #33155: [feature](statistics)Support high priority column stats auto collection. URL: https://github.com/apache/doris/pull/33155 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](partial-update) make partial-update on agg table use less memory [doris]
doris-robot commented on PR #32200: URL: https://github.com/apache/doris/pull/32200#issuecomment-2031345661 ClickBench: Total hot run time: 30.42 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit e80c8cd7e4e8ca2f5a78349f7faa8564c78dc4e0, data reload: false query1 0.040.030.03 query2 0.080.040.04 query3 0.240.040.04 query4 1.700.060.06 query5 0.490.480.50 query6 1.130.650.66 query7 0.020.010.01 query8 0.060.050.05 query9 0.560.520.51 query10 0.560.580.56 query11 0.150.110.11 query12 0.140.110.11 query13 0.620.600.60 query14 0.760.790.80 query15 0.860.840.85 query16 0.360.360.36 query17 0.990.970.96 query18 0.260.250.25 query19 1.801.731.72 query20 0.010.010.01 query21 15.41 0.770.68 query22 3.255.081.94 query23 17.61 1.361.15 query24 1.440.230.23 query25 0.150.090.08 query26 0.270.180.19 query27 0.080.080.09 query28 13.76 0.980.95 query29 12.70 3.443.37 query30 0.250.060.05 query31 2.870.390.38 query32 3.270.480.47 query33 2.832.902.88 query34 15.49 4.324.32 query35 4.424.384.35 query36 0.670.470.48 query37 0.200.170.17 query38 0.160.170.16 query39 0.040.040.04 query40 0.180.150.15 query41 0.100.050.06 query42 0.060.050.05 query43 0.050.040.04 Total cold run time: 106.09 s Total hot run time: 30.42 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](memory) Fix memory log compile by ASAN [doris]
xinyiZzz opened a new pull request, #33162: URL: https://github.com/apache/doris/pull/33162 ## Proposed changes ASAN compiles BE, add markers in memory logs ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](memory) Fix memory log compile by ASAN [doris]
doris-robot commented on PR #33162: URL: https://github.com/apache/doris/pull/33162#issuecomment-2031348034 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](core) Fix wal mgr heap use after free when stop doris [doris]
doris-robot commented on PR #33131: URL: https://github.com/apache/doris/pull/33131#issuecomment-2031349977 ClickBench: Total hot run time: 30.16 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit da2877e439d53ef0bda0080cdcf530aaf01e43a6, data reload: false query1 0.040.030.03 query2 0.080.040.05 query3 0.240.050.05 query4 1.670.070.07 query5 0.470.490.49 query6 1.150.650.65 query7 0.020.010.01 query8 0.050.040.04 query9 0.550.510.51 query10 0.580.560.57 query11 0.150.110.11 query12 0.130.110.12 query13 0.600.590.59 query14 0.770.780.80 query15 0.860.830.86 query16 0.350.360.36 query17 1.020.971.00 query18 0.260.240.25 query19 1.841.751.79 query20 0.020.010.01 query21 15.41 0.780.75 query22 3.164.661.46 query23 17.63 1.241.13 query24 1.530.260.30 query25 0.140.080.07 query26 0.270.170.18 query27 0.080.080.08 query28 13.65 1.030.96 query29 12.64 3.503.50 query30 0.260.060.06 query31 2.860.370.39 query32 3.270.480.47 query33 2.832.832.86 query34 15.51 4.354.35 query35 4.394.384.42 query36 0.660.470.47 query37 0.200.170.17 query38 0.170.160.16 query39 0.050.040.04 query40 0.180.150.17 query41 0.100.040.04 query42 0.060.060.05 query43 0.050.040.04 Total cold run time: 105.95 s Total hot run time: 30.16 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](memory) Fix memory log compile by ASAN [doris]
xinyiZzz commented on PR #33162: URL: https://github.com/apache/doris/pull/33162#issuecomment-2031348129 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](grace-exit) Fix threadpool hang on grace exit [doris]
zclllyybb commented on PR #33132: URL: https://github.com/apache/doris/pull/33132#issuecomment-2031349365 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](backup) ignore table that not support type when backup, and… [doris]
doris-robot commented on PR #33158: URL: https://github.com/apache/doris/pull/33158#issuecomment-2031350844 TPC-DS: Total hot run time: 182093 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 1712af489575a8b482c782b026e765009053f561, data reload: false query1 882 113311071107 query2 6320194017581758 query3 6904216 209 209 query4 24314 21568 21477 21477 query5 4233406 407 406 query6 266 191 194 191 query7 4604314 310 310 query8 233 186 183 183 query9 8476227722692269 query10 465 262 279 262 query11 14908 14417 14427 14417 query12 145 93 94 93 query13 1637378 372 372 query14 8458688869266888 query15 209 173 183 173 query16 6805285 280 280 query17 972 609 582 582 query18 1860285 284 284 query19 211 166 173 166 query20 97 96 104 96 query21 208 137 135 135 query22 4961477348334773 query23 33583 32939 32606 32606 query24 13293 324632273227 query25 740 452 439 439 query26 1893186 180 180 query27 3267382 430 382 query28 7484192019081908 query29 1311619 629 619 query30 323 172 174 172 query31 1086769 764 764 query32 102 65 63 63 query33 706 259 253 253 query34 1336527 536 527 query35 879 721 738 721 query36 1002862 860 860 query37 268 91 86 86 query38 3660364735523552 query39 1690159015941590 query40 266 141 140 140 query41 49 49 47 47 query42 111 117 119 117 query43 465 413 411 411 query44 1184728 736 728 query45 276 277 270 270 query46 1109817 793 793 query47 1984187918881879 query48 377 320 326 320 query49 966 378 384 378 query50 851 422 431 422 query51 6965682569926825 query52 115 106 103 103 query53 383 305 310 305 query54 331 259 264 259 query55 100 80 88 80 query56 255 244 254 244 query57 1296118511601160 query58 253 246 243 243 query59 2759243023802380 query60 259 242 243 242 query61 94 89 91 89 query62 652 451 446 446 query63 312 283 282 282 query64 5802305733483057 query65 3084300230113002 query66 1303320 320 320 query67 15619 14728 14733 14728 query68 8971569 587 569 query69 566 333 326 326 query70 1313108811511088 query71 488 274 280 274 query72 6199253824342434 query73 1452329 333 329 query74 6761621962906219 query75 3428228023042280 query76 4949112212161122 query77 558 244 250 244 query78 10881 10144 10050 10050 query79 9861547 529 529 query80 1822427 429 427 query81 532 238 242 238 query82 725 109 106 106 query83 215 171 166 166 query84 264 89 86 86 query85 1321307 282 282 query86 409 264 269 264 query87 3657349834623462 query88 3819236723792367 query89 556 369 368 368 query90 1935181 183 181 query91 134 109 107 107 query92 63 51 55 51 query93 6730542 541 541 query94 1050191 193 191 query95 445 334 339 334 query96 607 276 278 276 query97 2677247925082479 query98 235 215 215 215 query99 1318838 840 838 Total cold run time: 300952 ms Total hot run time: 182093 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [log](move-memtable) improve log in vtablet writer v2 and load stream [doris]
doris-robot commented on PR #33103: URL: https://github.com/apache/doris/pull/33103#issuecomment-2031350722 TPC-DS: Total hot run time: 181584 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 61cffca91b07a36ae7fb7a3ae1084361467d7abe, data reload: false query1 1209371 1110371 query2 6209207518461846 query3 6658217 206 206 query4 24843 21533 21331 21331 query5 4147391 399 391 query6 279 190 187 187 query7 4593313 300 300 query8 234 180 178 178 query9 8470228222932282 query10 436 258 267 258 query11 14884 14408 14533 14408 query12 144 96 100 96 query13 1629369 384 369 query14 8461694368766876 query15 208 170 178 170 query16 6629283 279 279 query17 949 598 588 588 query18 1819288 275 275 query19 208 165 165 165 query20 97 96 95 95 query21 196 129 126 126 query22 5043484748744847 query23 33725 32575 32786 32575 query24 13199 323931833183 query25 718 436 440 436 query26 1832183 171 171 query27 3316387 409 387 query28 7266192118921892 query29 1307635 628 628 query30 317 174 185 174 query31 1100737 752 737 query32 105 74 61 61 query33 702 259 281 259 query34 1256522 565 522 query35 863 732 780 732 query36 1029876 875 875 query37 262 82 87 82 query38 3702357435253525 query39 1651159515891589 query40 239 147 147 147 query41 50 47 48 47 query42 110 116 106 106 query43 458 415 405 405 query44 1166729 751 729 query45 298 279 276 276 query46 1117844 810 810 query47 1977188119061881 query48 397 322 323 322 query49 945 385 376 376 query50 839 418 433 418 query51 7064691669916916 query52 117 114 108 108 query53 387 313 301 301 query54 328 261 268 261 query55 89 84 76 76 query56 255 241 230 230 query57 1275117611881176 query58 255 226 228 226 query59 2708246224442444 query60 245 228 241 228 query61 98 86 81 81 query62 676 460 443 443 query63 308 282 281 281 query64 5823329531533153 query65 3059302330343023 query66 1331315 300 300 query67 15603 15023 14874 14874 query68 9016583 591 583 query69 551 325 322 322 query70 1320110310921092 query71 497 277 284 277 query72 6257253224092409 query73 1446334 328 328 query74 6662628162476247 query75 3478232322642264 query76 5232114512181145 query77 573 247 248 247 query78 10819 10146 10135 10135 query79 9911532 570 532 query80 2048420 423 420 query81 513 246 241 241 query82 693 106 103 103 query83 205 163 161 161 query84 264 90 89 89 query85 1372288 275 275 query86 405 299 269 269 query87 3770347434433443 query88 3772235223672352 query89 553 382 368 368 query90 1972174 178 174 query91 131 106 105 105 query92 67 50 50 50 query93 6729521 531 521 query94 1203203 194 194 query95 443 337 332 332 query96 612 280 277 277 query97 2670250424882488 query98 253 231 214 214 query99 1281845 817 817 Total cold run time: 301630 ms Total hot run time: 181584 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [improvement](partial-update) make partial-update on agg table use less memory [doris]
doris-robot commented on PR #32200: URL: https://github.com/apache/doris/pull/32200#issuecomment-2031351419 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit e80c8cd7e4e8ca2f5a78349f7faa8564c78dc4e0 with default session variables Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s Insert into select: 16.1 seconds inserted 1000 Rows, about 621K ops/s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
