[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8567: [api-change] add soft limit of String type length

[GitBox]

github-actions[bot] commented on pull request #8567:
URL: https://github.com/apache/incubator-doris/pull/8567#issuecomment-1077299811






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] EmmyMiao87 commented on issue #7901: [Feature] [Vectorized] Some Join opt in vec exec engine

[GitBox]

EmmyMiao87 commented on issue #7901:
URL: 
https://github.com/apache/incubator-doris/issues/7901#issuecomment-1077323522


   >  shuffling profit most from column pruning so we only do projection 
(including column pruning, some computing operations) in each datastreamsender 
instead of each node (join, agg, etc.).
   
   When be is implemented, it is possible to put the project logic in the 
sender uniformly. 
   However, in the case of vectorization, if the project logic is pushed down 
to the hash join node, the time-consuming of **memory copying can also be 
reduced during node calculation**. Therefore, the project implementation of 
hash join is not uniformly placed in the sender.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] whutpencil commented on pull request #8603: fix string default value bug

[GitBox]

whutpencil commented on pull request #8603:
URL: https://github.com/apache/incubator-doris/pull/8603#issuecomment-1077324046


   > 
   
   @morningman  My branch is master. Please look at the description in my 
issue. It is Load instead of insert stmt.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris-manager] branch master updated: fix module delete and config bug && update agent code (#16)

[diwu]

This is an automated email from the ASF dual-hosted git repository.

diwu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris-manager.git


The following commit(s) were added to refs/heads/master by this push:
 new 8ba0737  fix module delete and config bug && update agent code (#16)
8ba0737 is described below

commit 8ba0737589967c8a3b2f5537adcfd3af73d75a5f
Author: LiRui <1176867...@qq.com>
AuthorDate: Thu Mar 24 15:37:29 2022 +0800

fix module delete and config bug && update agent code (#16)

* add dm-agent resource includes

* fix module delete/config bug

* export agent config and start module at module root dir

* check agent port before starting agent

* add cache for heartbeat event

Co-authored-by: lirui40 
---
 build.sh   |  4 ++
 .../service/heartbeat/DorisInstanceOperator.java   | 10 ++--
 .../component/ModelControlRequestComponent.java|  1 -
 .../control/manager/DorisClusterModuleManager.java |  4 ++
 .../manager/ResourceNodeAndAgentManager.java   | 53 ++
 .../org/apache/doris/stack/shell/BaseCommand.java  | 22 +++--
 .../org/apache/doris/stack/util/Constants.java |  1 +
 .../src/main/resources/cache/ehcache.xml   | 25 ++
 .../doris/stack/dao/ClusterInstanceRepository.java | 14 ++
 .../doris/stack/dao/HeartBeatEventRepository.java  | 20 
 .../doris/stack/dao/ResourceNodeRepository.java| 21 +
 11 files changed, 166 insertions(+), 9 deletions(-)

diff --git a/build.sh b/build.sh
index 75ad587..4dfb601 100644
--- a/build.sh
+++ b/build.sh
@@ -50,6 +50,10 @@ mv output/manager-bin/agent output/
 mv output/manager-bin output/server/bin
 mkdir -p output/agent/lib
 mv manager/dm-agent/target/dm-agent-1.0.0.jar output/agent/lib/dm-agent.jar
+
+mkdir -p output/agent/config
+cp manager/dm-agent/src/main/resources/application.properties 
output/agent/config
+
 cp -r manager/manager-server/src/main/resources/web-resource output/server/
 tar -zcvf doris-manager-1.0.0.tar.gz output/
 echo "copy to output package end"
\ No newline at end of file
diff --git 
a/manager/dm-agent/src/main/java/org/apache/doris/manager/agent/service/heartbeat/DorisInstanceOperator.java
 
b/manager/dm-agent/src/main/java/org/apache/doris/manager/agent/service/heartbeat/DorisInstanceOperator.java
index c9195b5..961f4a4 100644
--- 
a/manager/dm-agent/src/main/java/org/apache/doris/manager/agent/service/heartbeat/DorisInstanceOperator.java
+++ 
b/manager/dm-agent/src/main/java/org/apache/doris/manager/agent/service/heartbeat/DorisInstanceOperator.java
@@ -306,9 +306,13 @@ public class DorisInstanceOperator {
 
 private void executePkgShellScriptWithBash(String scriptName, String 
runningDir,
String moduleName, Map 
environment) throws Exception {
-String scripts = Paths.get(runningDir, moduleName, "bin", 
scriptName).toFile().getAbsolutePath();
-final String shellCmd = "sh " + scripts;
-log.info("begin to execute: `" + shellCmd + "`");
+String mouduleRootDir = runningDir + File.separator + moduleName;
+String script = "bin" + File.separator + scriptName;
+
+String cmdFormat = "cd %s && sh %s";
+final String shellCmd = String.format(cmdFormat, mouduleRootDir, 
script);
+
+log.info("begin to execute with bash: `" + shellCmd + "`");
 ShellUtil.cmdExecute(shellCmd);
 }
 
diff --git 
a/manager/dm-server/src/main/java/org/apache/doris/stack/component/ModelControlRequestComponent.java
 
b/manager/dm-server/src/main/java/org/apache/doris/stack/component/ModelControlRequestComponent.java
index e49e71e..8c72032 100644
--- 
a/manager/dm-server/src/main/java/org/apache/doris/stack/component/ModelControlRequestComponent.java
+++ 
b/manager/dm-server/src/main/java/org/apache/doris/stack/component/ModelControlRequestComponent.java
@@ -51,7 +51,6 @@ public class ModelControlRequestComponent {
 requestEntity.setCurrentEventType(eventType);
 requestEntity.setModelId(modelId);
 requestEntity.setRequestInfo(requestInfo);
-requestRepository.save(requestEntity);
 return requestRepository.save(requestEntity);
 }
 
diff --git 
a/manager/dm-server/src/main/java/org/apache/doris/stack/control/manager/DorisClusterModuleManager.java
 
b/manager/dm-server/src/main/java/org/apache/doris/stack/control/manager/DorisClusterModuleManager.java
index 3c3ae9f..30205de 100644
--- 
a/manager/dm-server/src/main/java/org/apache/doris/stack/control/manager/DorisClusterModuleManager.java
+++ 
b/manager/dm-server/src/main/java/org/apache/doris/stack/control/manager/DorisClusterModuleManager.java
@@ -149,6 +149,7 @@ public class DorisClusterModuleManager {
 serviceCreateOperation(moduleEntity, serviceNamePorts, accessInfo);
 }
 
+clusterModuleRepository.save(moduleEntity);

[GitHub] [incubator-doris-manager] JNSimba merged pull request #16: fix module delete and config bug && update agent code

[GitBox]

JNSimba merged pull request #16:
URL: https://github.com/apache/incubator-doris-manager/pull/16


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] HappenLee commented on a change in pull request #8572: [Vectorized][Bug] fix percentile_approx function to return always nullable

[GitBox]

HappenLee commented on a change in pull request #8572:
URL: https://github.com/apache/incubator-doris/pull/8572#discussion_r834002852



##
File path: be/src/vec/aggregate_functions/aggregate_function_percentile_approx.h
##
@@ -152,32 +151,57 @@ class AggregateFunctionPercentileApproxMerge : public 
AggregateFunctionPercentil
 }
 };
 
+template 
 class AggregateFunctionPercentileApproxTwoParams : public 
AggregateFunctionPercentileApprox {
 public:
 AggregateFunctionPercentileApproxTwoParams(const DataTypes& 
argument_types_)
 : AggregateFunctionPercentileApprox(argument_types_) {}
 void add(AggregateDataPtr __restrict place, const IColumn** columns, 
size_t row_num,
  Arena*) const override {
-const auto& sources = static_cast&>(*columns[0]);
-const auto& quantile = static_cast&>(*columns[1]);
+if constexpr (is_nullable) {
+const auto* nullable_column = 
check_and_get_column(columns[0]);
+const auto& sources =
+static_cast&>(nullable_column->get_nested_column());
+//requires second parameter must be a constant
+const auto& quantile = static_cast&>(*columns[1]);
+this->data(place).init();
+this->data(place).add(sources.get_float64(row_num), 
quantile.get_float64(row_num));
 
-this->data(place).init();
-this->data(place).add(sources.get_float64(row_num), 
quantile.get_float64(row_num));
+} else {
+const auto& sources = static_cast&>(*columns[0]);
+const auto& quantile = static_cast&>(*columns[1]);
+
+this->data(place).init();
+this->data(place).add(sources.get_float64(row_num), 
quantile.get_float64(row_num));
+}
 }
 };
 
+template 
 class AggregateFunctionPercentileApproxThreeParams : public 
AggregateFunctionPercentileApprox {
 public:
 AggregateFunctionPercentileApproxThreeParams(const DataTypes& 
argument_types_)
 : AggregateFunctionPercentileApprox(argument_types_) {}
 void add(AggregateDataPtr __restrict place, const IColumn** columns, 
size_t row_num,
  Arena*) const override {
-const auto& sources = static_cast&>(*columns[0]);
-const auto& quantile = static_cast&>(*columns[1]);
-const auto& compression = static_cast&>(*columns[2]);
+if constexpr (is_nullable) {
+const auto* nullable_column = 
check_and_get_column(columns[0]);

Review comment:
   same to the up

##
File path: be/src/vec/aggregate_functions/aggregate_function_percentile_approx.h
##
@@ -152,32 +151,57 @@ class AggregateFunctionPercentileApproxMerge : public 
AggregateFunctionPercentil
 }
 };
 
+template 
 class AggregateFunctionPercentileApproxTwoParams : public 
AggregateFunctionPercentileApprox {
 public:
 AggregateFunctionPercentileApproxTwoParams(const DataTypes& 
argument_types_)
 : AggregateFunctionPercentileApprox(argument_types_) {}
 void add(AggregateDataPtr __restrict place, const IColumn** columns, 
size_t row_num,
  Arena*) const override {
-const auto& sources = static_cast&>(*columns[0]);
-const auto& quantile = static_cast&>(*columns[1]);
+if constexpr (is_nullable) {
+const auto* nullable_column = 
check_and_get_column(columns[0]);

Review comment:
   need check null in nullable column first！！！




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #8618: Enable hash join project

[GitBox]

EmmyMiao87 commented on a change in pull request #8618:
URL: https://github.com/apache/incubator-doris/pull/8618#discussion_r834003125



##
File path: fe/fe-core/src/main/java/org/apache/doris/planner/PlanNode.java
##
@@ -870,4 +864,22 @@ public void convertToVectoriezd() {
 child.convertToVectoriezd();
 }
 }
+
+// The slotIdSet could be null
+public void initOutputSlotIds(Set slotIdSet, Analyzer analyzer) 
throws NotImplementedException {

Review comment:
   fixed




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #8618: Enable hash join project

[GitBox]

EmmyMiao87 commented on a change in pull request #8618:
URL: https://github.com/apache/incubator-doris/pull/8618#discussion_r834003318



##
File path: be/src/vec/exec/join/vhash_join_node.cpp
##
@@ -210,12 +210,24 @@ struct ProcessHashTableProbe {
 }
 
 // output probe side result column
-void probe_side_output_column(MutableColumns& mcol, int column_length, int 
size) {
-for (int i = 0; i < column_length; ++i) {
-auto& column = _probe_block.get_by_position(i).column;
-column->replicate(&_items_counts[0], size, *mcol[i]);
+//void probe_side_output_column(MutableColumns& mcol, int column_length, 
int size) {
+//for (int i = 0; i < column_length; ++i) {
+//auto& column = _probe_block.get_by_position(i).column;
+//column->replicate(&_items_counts[0], size, *mcol[i]);
+//}
+//}
+
+void probe_side_output_column(MutableColumns& mcol, std::vector 
_left_output_slot_idx, int size) {

Review comment:
   fixed




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #8618: Enable hash join project

[GitBox]

EmmyMiao87 commented on a change in pull request #8618:
URL: https://github.com/apache/incubator-doris/pull/8618#discussion_r834004203



##
File path: be/src/vec/exec/join/vhash_join_node.h
##
@@ -232,6 +232,10 @@ class HashJoinNode : public ::doris::ExecNode {
 std::vector _build_block_rows;
 
 std::shared_ptr _hash_table_mem_tracker;
+
+std::vector _output_slot_ids;

Review comment:
   It may be changed in later PR




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] whutpencil opened a new issue #8624: [Bug] An error is reported in the non-existent field of load

[GitBox]

whutpencil opened a new issue #8624:
URL: https://github.com/apache/incubator-doris/issues/8624


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Version
   
   master branch
   
   ### What's Wrong?
   
   `CREATE TABLE test_db( 
 aaa date,
 bbb string , 
 ccc string,
   )
   `
   
   `
   LOAD x 
   set (
 aaa='_col1',
 bbb='_col2',
 ccc='_col3',
 =default_value("good")
   )
   `
   
   ``  is a field that does not exist in the Doris table,  This often 
happens. We don't need to import all the fields, so the Doris table has a few 
fewer fields than the corresponding hive table, which is easy to make mistakes 
when loading.
   
   The following error messages will appear:
   
![image](https://user-images.githubusercontent.com/24907215/159867942-d3367e0e-8faf-4d12-a13b-e3b02a1af528.png)
   
   
   
   ### What You Expected?
   
   The error content is consistent with the actual situation.
   
   ### How to Reproduce?
   
   _No response_
   
   ### Anything Else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris-manager] hf200012 merged pull request #17: Mybranch

[GitBox]

hf200012 merged pull request #17:
URL: https://github.com/apache/incubator-doris-manager/pull/17


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris-manager] hf200012 merged pull request #15: [fix] update sidebar styles

[GitBox]

hf200012 merged pull request #15:
URL: https://github.com/apache/incubator-doris-manager/pull/15


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris-manager] branch master updated (8ba0737 -> 4160c4c)

[jiafengzheng]

This is an automated email from the ASF dual-hosted git repository.

jiafengzheng pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris-manager.git.


from 8ba0737  fix module delete and config bug && update agent code (#16)
 add 4160c4c  Mybranch (#17)

No new revisions were added by this update.

Summary of changes:
 .../src/main/resources/application.properties  | 17 +++
 .../config/InstanceDeployCheckEventConfigInfo.java | 17 +++
 .../config/InstanceInstallEventConfigInfo.java | 17 +++
 .../config/InstanceRestartEventConfigInfo.java | 17 +++
 .../config/InstanceStartEventConfigInfo.java   | 17 +++
 .../config/InstanceStopEventConfigInfo.java| 17 +++
 .../doris/manager/common/util/ConfigDefault.java   | 17 +++
 .../doris/stack/component/MailComponent.java   | 11 +-
 .../apache/doris/stack/constant/ConstantDef.java   |  6 +++---
 .../src/main/resources/cache/ehcache.xml   | 17 +++
 .../resources/locale/exception_message.properties  | 17 +++
 .../locale/exception_message_en.properties | 17 +++
 .../locale/exception_message_zh_CN.properties  | 17 +++
 .../src/main/resources/mail/_footer.mustache   | 17 +++
 .../src/main/resources/mail/_footer_pulse.mustache | 19 +++-
 .../src/main/resources/mail/_header.mustache   | 17 +++
 .../src/main/resources/mail/_logo.mustache | 17 +++
 .../resources/mail/alert_new_confirmation.mustache | 17 +++
 .../resources/mail/alert_unsubscribed.mustache | 17 +++
 .../main/resources/mail/alert_was_deleted.mustache | 21 --
 .../resources/mail/alert_you_were_added.mustache   | 17 +++
 .../main/resources/mail/new_user_invite.mustache   | 25 ++
 .../main/resources/mail/password_reset.mustache| 17 +++
 .../mail/user_joined_notification.mustache | 17 +++
 .../service/construct/DataManageServiceTest.java   |  2 +-
 .../service/construct/NativeQueryServiceTest.java  |  2 +-
 manager/pom.xml| 16 ++
 27 files changed, 412 insertions(+), 13 deletions(-)

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris-manager] branch master updated (4160c4c -> 13b78db)

[jiafengzheng]

This is an automated email from the ASF dual-hosted git repository.

jiafengzheng pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris-manager.git.


from 4160c4c  Mybranch (#17)
 add 13b78db  [fix] update sidebar styles (#15)

No new revisions were added by this update.

Summary of changes:
 .github/workflows/build-extension.yml |  7 +-
 frontend/src/components/header/header.tsx |  1 +
 frontend/src/components/header/index.module.less  |  4 +++
 frontend/src/components/sidebar/sidebar.less  |  7 --
 frontend/src/index.css| 30 +++
 frontend/src/routes/admin/admin.module.less   |  2 +-
 frontend/src/routes/container.less|  4 +--
 frontend/src/routes/settings/settings.module.less |  2 +-
 8 files changed, 50 insertions(+), 7 deletions(-)

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] whutpencil opened a new pull request #8625: fix wrong error content on Load

[GitBox]

whutpencil opened a new pull request #8625:
URL: https://github.com/apache/incubator-doris/pull/8625


   # Proposed changes
   
   Issue Number: close #8624 
   
   ## Problem Summary:
   
   The reason is that the field is null, and the subsequent logic only deals 
with AnalysisException, so there is no timely error report in the analysis 
stage.
   
   see the issue content. 
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (No)
   2. Has unit tests been added: (No)
   3. Has document been added or modified: (No)
   4. Does it need to update dependencies: (No)
   5. Are there any changes that cannot be rolled back: (No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] wangbo opened a new pull request #8626: [Bug](vectorization) Vectorization decimal arithmetic inconsistent

[GitBox]

wangbo opened a new pull request #8626:
URL: https://github.com/apache/incubator-doris/pull/8626


   # Proposed changes
   
   Issue Number: close #8588
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8626: [Bug](vectorization) Vectorization decimal arithmetic inconsistent

[GitBox]

github-actions[bot] commented on pull request #8626:
URL: https://github.com/apache/incubator-doris/pull/8626#issuecomment-1077350780






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] zenoyang opened a new pull request #8627: [refactor][optimize] Code optimization and refactoring for low-cardinality columns in storage layer

[GitBox]

zenoyang opened a new pull request #8627:
URL: https://github.com/apache/incubator-doris/pull/8627


   # Proposed changes
   
   Code optimization and refactoring for low-cardinality columns in storage 
layer.
   
   ## Problem Summary:
   
   Optimization:
   - Add `dict_code` member variable to the predicate. Before the predicate is 
calculated, the conversion of the predicate constant into the dictionary code 
only needs to be done once
   
   
   Refactor:
   - Add `PredicateType` enum, add `type` method to the predicate, which is 
used to represent the predicate type, instead of the original less flexible 
way, for example: `is_in_predicate()` ...
   - Added the following interface to `IColumn` to simplify the code of the 
storage layer and avoid exposing the specific type of column:
- `convert_to_predicate_column_if_dictionary`
- `convert_dict_codes_if_necessary`
- `set_predicate_dict_code_if_necessary`
   - Class member variables start with an underscore
   - Specify the `int` type only supported by the specialized 
`ColumnDictionary` template
   
   In addition to this, there are some priorities that I will provide in the 
coming period.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] dataroaring opened a new pull request #8628: row num is more accurate than column num in data_types

[GitBox]

dataroaring opened a new pull request #8628:
URL: https://github.com/apache/incubator-doris/pull/8628


   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (No)
   2. Has unit tests been added: (No Need)
   3. Has document been added or modified: (No Need)
   4. Does it need to update dependencies: (No)
   5. Are there any changes that cannot be rolled back: (No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris-manager] colordove opened a new pull request #19: [build] add github-cli change pipeline

[GitBox]

colordove opened a new pull request #19:
URL: https://github.com/apache/incubator-doris-manager/pull/19


   # Proposed changes
   
   Add Steps
   - Install Nodejs
   - Run build.sh
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (No)
   2. Has unit tests been added: (No)
   3. Has document been added or modified: (No)
   4. Does it need to update dependencies: (No)
   5. Are there any changes that cannot be rolled back: (No)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris-manager] colordove commented on pull request #19: [build] add github-cli change pipeline

[GitBox]

colordove commented on pull request #19:
URL: 
https://github.com/apache/incubator-doris-manager/pull/19#issuecomment-1077387411


   The Commit has been merged in the previous Pull Request #15


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris-manager] colordove closed pull request #19: [build] add github-cli change pipeline

[GitBox]

colordove closed pull request #19:
URL: https://github.com/apache/incubator-doris-manager/pull/19


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8628: row num is more accurate than column num in data_types

[GitBox]

github-actions[bot] commented on pull request #8628:
URL: https://github.com/apache/incubator-doris/pull/8628#issuecomment-1077389268






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] morningman commented on a change in pull request #8553: [feature](balance) Support balance between disks on a single BE

[GitBox]

morningman commented on a change in pull request #8553:
URL: https://github.com/apache/incubator-doris/pull/8553#discussion_r834047621



##
File path: be/src/olap/task/engine_storage_migration_task.cpp
##
@@ -32,91 +32,147 @@ OLAPStatus EngineStorageMigrationTask::execute() {
 return _migrate();
 }
 
-OLAPStatus EngineStorageMigrationTask::_migrate() {
-int64_t tablet_id = _tablet->tablet_id();
-int32_t schema_hash = _tablet->schema_hash();
-LOG(INFO) << "begin to process tablet migrate. "
-  << "tablet_id=" << tablet_id << ", dest_store=" << 
_dest_store->path();
-
-DorisMetrics::instance()->storage_migrate_requests_total->increment(1);
+OLAPStatus EngineStorageMigrationTask::_get_versions(int32_t start_version, 
int32_t* end_version,
+std::vector 
*consistent_rowsets) {
+ReadLock rdlock(_tablet->get_header_lock());
+const RowsetSharedPtr last_version = _tablet->rowset_with_max_version();
+if (last_version == nullptr) {
+LOG(WARNING) << "failed to get rowset with max version, tablet="
+<< _tablet->full_name();
+return OLAP_ERR_VERSION_NOT_EXIST;
+}
 
-// try hold migration lock first
-OLAPStatus res = OLAP_SUCCESS;
-UniqueWriteLock migration_wlock(_tablet->get_migration_lock(), 
std::try_to_lock);
-if (!migration_wlock.owns_lock()) {
-return OLAP_ERR_RWLOCK_ERROR;
+*end_version = last_version->end_version();
+if (*end_version < start_version) {
+// rowsets are empty
+VLOG_DEBUG << "consistent rowsets empty. tablet=" << 
_tablet->full_name()
+<< ", start_version=" << start_version << ", 
end_version=" << *end_version;
+return OLAP_SUCCESS;
 }
+_tablet->capture_consistent_rowsets(Version(start_version, *end_version), 
consistent_rowsets);
+if (consistent_rowsets->empty()) {
+LOG(WARNING) << "fail to capture consistent rowsets. tablet=" << 
_tablet->full_name()
+<< ", version=" << *end_version;
+return OLAP_ERR_VERSION_NOT_EXIST;
+}
+return OLAP_SUCCESS;
+}
 
-// check if this tablet has related running txns. if yes, can not do 
migration.
+OLAPStatus EngineStorageMigrationTask::_check_running_txns() {
+// need hold migration lock outside
 int64_t partition_id;
 std::set transaction_ids;
+// check if this tablet has related running txns. if yes, can not do 
migration.
 StorageEngine::instance()->txn_manager()->get_tablet_related_txns(
-tablet_id, schema_hash, _tablet->tablet_uid(), &partition_id, 
&transaction_ids);
+_tablet->tablet_id(), _tablet->schema_hash(), 
_tablet->tablet_uid(), &partition_id, &transaction_ids);
 if (transaction_ids.size() > 0) {
 LOG(WARNING) << "could not migration because has unfinished txns, "
- << " tablet=" << _tablet->full_name();
+<< " tablet=" << _tablet->full_name();
 return OLAP_ERR_HEADER_HAS_PENDING_DATA;
 }
+return OLAP_SUCCESS;
+}
 
-_tablet->obtain_push_lock();
+OLAPStatus EngineStorageMigrationTask::_migrate() {
+int64_t tablet_id = _tablet->tablet_id();
+int32_t schema_hash = _tablet->schema_hash();
+LOG(INFO) << "begin to process tablet migrate. "
+  << "tablet_id=" << tablet_id << ", dest_store=" << 
_dest_store->path();
 
-// TODO(ygl): the tablet should not under schema change or rollup or load
-do {
-std::vector consistent_rowsets;
-{
-ReadLock rdlock(_tablet->get_header_lock());
-// get all versions to be migrate
-const RowsetSharedPtr last_version = 
_tablet->rowset_with_max_version();
-if (last_version == nullptr) {
-res = OLAP_ERR_VERSION_NOT_EXIST;
-LOG(WARNING) << "failed to get rowset with max version, 
tablet="
- << _tablet->full_name();
-break;
-}
-int32_t end_version = last_version->end_version();
-res = _tablet->capture_consistent_rowsets(Version(0, end_version), 
&consistent_rowsets);
-if (consistent_rowsets.empty()) {
-res = OLAP_ERR_VERSION_NOT_EXIST;
-LOG(WARNING) << "fail to capture consistent rowsets. tablet=" 
<< _tablet->full_name()
- << ", version=" << end_version;
-break;
-}
+DorisMetrics::instance()->storage_migrate_requests_total->increment(1);
+int32_t start_version = 0;
+int32_t end_version = 0;
+std::vector consistent_rowsets;
+
+// try hold migration lock first
+OLAPStatus res = OLAP_SUCCESS;
+{
+UniqueWriteLock migration_wlock(_tablet->get_migration_lock(), 
std::try_to_lock);
+if (!migration_wlock.owns_lock()) {
+return OLAP_ERR_RWLOCK_ERROR;
 }
 
-uint64_t shar

[GitHub] [incubator-doris] DarvenDuan opened a new issue #8629: [Bug] [sql_block_rule] the limitations of sql_block_rule can be negative

[GitBox]

DarvenDuan opened a new issue #8629:
URL: https://github.com/apache/incubator-doris/issues/8629


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Version
   
   master branch
   
   ### What's Wrong?
   
   When I create or alter a sql_block_rule like : `create sql_block_rule 
test_rule properties("partition_num"="-100","tablet_num"="-100");`,it can be 
executed correctly. And I exec `show sql_block_rule for test_rule` got the 
result like this: 
   
++--+-+--+---+-+++
   | Name   | Sql  | SqlHash | PartitionNum | TabletNum | Cardinality | 
Global | Enable |
   
++--+-+--+---+-+++
   | test_rule2 | NULL | NULL| -100| -100 | 0   
| false  | true   |
   
++--+-+--+---+-+++
   
   Obviously, this is unreasonable
   
   ### What You Expected?
   
   Value of properties can't be negative
   
   ### How to Reproduce?
   
   _No response_
   
   ### Anything Else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] pengxiangyu commented on a change in pull request #8532: [feature](cold-hot) support remote storage

[GitBox]

pengxiangyu commented on a change in pull request #8532:
URL: https://github.com/apache/incubator-doris/pull/8532#discussion_r834070609



##
File path: 
fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
##
@@ -130,22 +139,50 @@ public static DataProperty 
analyzeDataProperty(Map properties, D
 hasCooldown = true;
 DateLiteral dateLiteral = new DateLiteral(value, 
Type.DATETIME);
 coolDownTimeStamp = 
dateLiteral.unixTimestamp(TimeUtils.getTimeZone());
+} else if (!hasColdMedium && 
key.equalsIgnoreCase(PROPERTIES_STORAGE_COLD_MEDIUM)) {
+hasColdMedium = true;

Review comment:
   coldMedium有可能是HDD。

##
File path: 
fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
##
@@ -130,22 +139,50 @@ public static DataProperty 
analyzeDataProperty(Map properties, D
 hasCooldown = true;
 DateLiteral dateLiteral = new DateLiteral(value, 
Type.DATETIME);
 coolDownTimeStamp = 
dateLiteral.unixTimestamp(TimeUtils.getTimeZone());
+} else if (!hasColdMedium && 
key.equalsIgnoreCase(PROPERTIES_STORAGE_COLD_MEDIUM)) {
+hasColdMedium = true;
+if (value.equalsIgnoreCase(TStorageMedium.S3.name())) {
+coldStorageMedium = TStorageMedium.S3;
+} else {
+throw new AnalysisException("Invalid cold storage medium: 
" + value);
+}
+} else if (!hasRemoteStorage && 
key.equalsIgnoreCase(PROPERTIES_REMOTE_STORAGE)) {
+hasRemoteStorage = true;
+remoteStorageName = value;
 }
 } // end for properties
 
-if (!hasCooldown && !hasMedium) {
+// remote_storage_medium may be empty, when data moves SSD to HDD.
+if (!hasCooldown && !hasMedium && !hasColdMedium) {
 return oldDataProperty;
 }
 
 properties.remove(PROPERTIES_STORAGE_MEDIUM);
 properties.remove(PROPERTIES_STORAGE_COLDOWN_TIME);
+properties.remove(PROPERTIES_STORAGE_COLD_MEDIUM);
+properties.remove(PROPERTIES_REMOTE_STORAGE);
+
+if (hasRemoteStorage && !hasColdMedium) {
+throw new AnalysisException("Invalid data property, " +
+"`remote_storage` must be used with 
`storage_cold_medium`.");
+}
+
+if (hasColdMedium && hasRemoteStorage) {
+remotestorageProperty = Catalog.getCurrentCatalog()
+
.getRemoteStorageMgr().getRemoteStorageByName(remoteStorageName);
+
+if 
(!coldStorageMedium.name().equalsIgnoreCase(remotestorageProperty.getStorageType().name()))
 {
+throw new AnalysisException("Invalid data property, " +
+"`storage_cold_medium` is inconsistent with 
`remote_storage`.");
+}
+}
 
 if (hasCooldown && !hasMedium) {
 throw new AnalysisException("Invalid data property. storage medium 
property is not found");
 }
 
-if (storageMedium == TStorageMedium.HDD && hasCooldown) {
-throw new AnalysisException("Can not assign cooldown timestamp to 
HDD storage medium");
+if ((storageMedium == TStorageMedium.HDD && hasCooldown) && 
!(hasColdMedium && hasRemoteStorage)) {
+throw new AnalysisException("Can not assign cooldown timestamp to 
HDD storage medium without remote storage");

Review comment:
   if (storageMedium == TStorageMedium.S3 && !(hasColdMedium && 
hasRemoteStorage)) {

##
File path: 
fe/fe-core/src/main/java/org/apache/doris/catalog/RemoteStorageMgr.java
##
@@ -0,0 +1,282 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog;
+
+import org.apache.doris.analysis.AddRemoteStorageClause;
+import org.apache.doris.analysis.DropRemoteStorageClause;
+import org.apache.doris.analysis.ModifyRemoteStorageClause;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.DdlException;
+import org.apache.doris.common.io.Text;
+import org.apache.doris.common.io.Writable;
+import org.apac

[incubator-doris-manager] branch master updated: [fix] fix space administrator cannot remove or add administrators to the space and fix login page form label (#18)

[jiafengzheng]

This is an automated email from the ASF dual-hosted git repository.

jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris-manager.git


The following commit(s) were added to refs/heads/master by this push:
 new 004c5e4  [fix] fix space administrator cannot remove or add 
administrators to the space and fix login page form label (#18)
004c5e4 is described below

commit 004c5e4a5963b8bff4c26a8b4a40cbc684868785
Author: zhengbowen <101616192+zhengbowe...@users.noreply.github.com>
AuthorDate: Thu Mar 24 17:26:22 2022 +0800

[fix] fix space administrator cannot remove or add administrators to the 
space and fix login page form label (#18)

 fix space administrator cannot remove or add administrators to the space 
and fix login page form label
---
 frontend/public/locales/en-us.json  |  3 +++
 frontend/public/locales/zh-cn.json  |  3 +++
 frontend/src/components/sidebar/sidebar.tsx |  4 ++--
 frontend/src/routes/admin/people/role/list/list.tsx |  4 ++--
 .../src/routes/admin/people/role/member/member.tsx  |  4 ++--
 frontend/src/routes/admin/people/user/user.tsx  |  4 ++--
 frontend/src/routes/container.tsx   |  2 +-
 frontend/src/routes/passport/login.tsx  |  6 +++---
 frontend/src/routes/space/detail/space-detail.tsx   | 21 +
 frontend/src/routes/space/list/list.tsx |  6 --
 10 files changed, 35 insertions(+), 22 deletions(-)

diff --git a/frontend/public/locales/en-us.json 
b/frontend/public/locales/en-us.json
index 418dd52..48eb538 100644
--- a/frontend/public/locales/en-us.json
+++ b/frontend/public/locales/en-us.json
@@ -1,8 +1,11 @@
 {
 "username": "Username",
 "password": "Password",
+"Username or Mail": "Username or Mail",
 "OldPassword":"Old Password",
 "PleaseEnterTheOldPassword":"Please enter the old password",
+"Please input the username or email": "Please input the username or email",
+"Please input the password": "Please input the password",
 "NewPassword":"New Password",
 "ContainAtLeast2Types":"Contain at least 2 types of English uppercase, 
lowercase English, numbers and special symbols",
 "PleaseEnterTheNewPassword":"Please enter the new password",
diff --git a/frontend/public/locales/zh-cn.json 
b/frontend/public/locales/zh-cn.json
index a4b3a83..40b8383 100644
--- a/frontend/public/locales/zh-cn.json
+++ b/frontend/public/locales/zh-cn.json
@@ -1,8 +1,11 @@
 {
 "username": "用户名",
 "password": "密码",
+"Username or Mail": "用户名或邮箱",
 "OldPassword":"旧密码",
 "PleaseEnterTheOldPassword":"请输入旧密码",
+"Please input the username or email": "请输入用户名或邮箱",
+"Please input the password": "请输入密码",
 "NewPassword":"新密码",
 "ContainAtLeast2Types":"至少包含英文大写、英文小写、数字和特殊符号中的2种",
 "PleaseEnterTheNewPassword":"请输入新密码",
diff --git a/frontend/src/components/sidebar/sidebar.tsx 
b/frontend/src/components/sidebar/sidebar.tsx
index 6cc564c..3a4ae11 100644
--- a/frontend/src/components/sidebar/sidebar.tsx
+++ b/frontend/src/components/sidebar/sidebar.tsx
@@ -147,9 +147,9 @@ export function Sidebar(props: any) {
 }>
 {t`data`}
 
-}>
+{/* }>
 {t`Query`}
-
+ */}
 {(isSuperAdmin || isSpaceAdmin) && (
 }>
 {t`Space 
Manager`}
diff --git a/frontend/src/routes/admin/people/role/list/list.tsx 
b/frontend/src/routes/admin/people/role/list/list.tsx
index f5c62c9..c623923 100644
--- a/frontend/src/routes/admin/people/role/list/list.tsx
+++ b/frontend/src/routes/admin/people/role/list/list.tsx
@@ -116,7 +116,7 @@ export function RoleList(props: any) {
 
 {t`roleTopMessage`}
 
- {
@@ -125,7 +125,7 @@ export function RoleList(props: any) {
 }}
 >
 {t`create`}
-
+ */}
 
 
 
 
 {showName(members?.name)}
- {
@@ -133,7 +133,7 @@ export function RoleMembers(props: any) {
 }}
 >
 {t`addMembers`}
-
+ */}
 
 
-
+{/* 
  setModalVisible(true)}>
 {t`addMembers`}
 
-
+ */}
 
 
 
-
+{/*  */}
 
 

[GitHub] [incubator-doris-manager] hf200012 merged pull request #18: [fix] fix space administrator cannot remove or add administrators to …

[GitBox]

hf200012 merged pull request #18:
URL: https://github.com/apache/incubator-doris-manager/pull/18


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] morningman commented on a change in pull request #8618: Enable hash join project

[GitBox]

morningman commented on a change in pull request #8618:
URL: https://github.com/apache/incubator-doris/pull/8618#discussion_r834093523



##
File path: fe/fe-core/src/main/java/org/apache/doris/planner/PlanNode.java
##
@@ -870,4 +869,23 @@ public void convertToVectoriezd() {
 child.convertToVectoriezd();
 }
 }
+
+// The slotIdSet could be null
+// The output slots is subset of required slots when there are more than 
one child
+public void initOutputSlotIds(Set requiredSlotIdSet, Analyzer 
analyzer) throws NotImplementedException {
+throw new NotImplementedException("The `initOutputSlotIds` hasn't been 
implemented in " + planNodeName);
+}
+
+public Set computeInputSlotIds() throws NotImplementedException {

Review comment:
   Add comment for these 2 new interface.

##
File path: fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java
##
@@ -176,6 +176,9 @@
 
 public static final String BLOCK_ENCRYPTION_MODE = "block_encryption_mode";
 
+// TODO(ml): change to all node project
+public static final String ENABLE_HASH_PROJECT = "enable_hash_project";

Review comment:
   ```suggestion
   public static final String ENABLE_PROJECTION = "enable_projection";
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris-flink-connector] JNSimba opened a new pull request #22: [Bug] Inconsistent id when deserializing

[GitBox]

JNSimba opened a new pull request #22:
URL: https://github.com/apache/incubator-doris-flink-connector/pull/22


   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem Summary:
   
   add serialVersionUID 
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] adonis0147 opened a new issue #8630: [Bug] Fix compilation for test code.

[GitBox]

adonis0147 opened a new issue #8630:
URL: https://github.com/apache/incubator-doris/issues/8630


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Version
   
   master
   
   ### What's Wrong?
   
   Run `run-be-ut.sh` and errors occur due to `quantile_state_test.cpp` doesn't 
have `main` function.
   
   ### What You Expected?
   
   Fix compilation for test code and run `run-be-ut.sh` successfully.
   
   ### How to Reproduce?
   
   Run `run-be-ut.sh` in terminal.
   
   ### Anything Else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] adonis0147 opened a new pull request #8631: [chore][be-test] Link gtest_main to provide default main function definition.

[GitBox]

adonis0147 opened a new pull request #8631:
URL: https://github.com/apache/incubator-doris/pull/8631


   # Proposed changes
   
   Issue Number: close #8630
   
   ## Problem Summary:
   
   Please refer to #8630 .
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: No
   2. Has unit tests been added: No Need
   3. Has document been added or modified: No Need
   4. Does it need to update dependencies: No
   5. Are there any changes that cannot be rolled back: No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] 924060929 opened a new pull request #8632: [test] support a lot of actions

[GitBox]

924060929 opened a new pull request #8632:
URL: https://github.com/apache/incubator-doris/pull/8632


   # Proposed changes
   
   Support a lot of actions for regression testing framework.
   e.g. `thread`, `lazyCheck`, `onSuccess`, `connect`, `selectUnionAll`, `timer`
   
   ## Problem Summary:
   
   For easy to open source p0 test case
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: No
   2. Has unit tests been added: Yes
   3. Has document been added or modified: Yes
   4. Does it need to update dependencies: No
   5. Are there any changes that cannot be rolled back: No
   
   @EmmyMiao87 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] zbtzbtzbt opened a new pull request #8633: [Opt] Optimize hotspot functions in compaction

[GitBox]

zbtzbtzbt opened a new pull request #8633:
URL: https://github.com/apache/incubator-doris/pull/8633


   ## Proposed changes
   **Compaction** is very cpu consuming, 
   after test, I found partial bottleneck is in function 
`rowset_with_max_version` 
   so made following targeted performance tuning:
   
   - better time cost O(1) function`max_version()` rather than previous heavy 
and hotspot O(n)
   - use faster `phmap::flat_hash_map` replace `rs_version_map` and 
`_stale_rs_version_map` data structure
   
   currently, these changes seems do not affect the speed of data load.
   
   1FE 3BE load 5GB data for agg table:
   
   Previous:
   test1: start | end  ==> 14:38:22|14:40:16 ==> 114 s
   test2: start | end   ==> 14:47:46|14:49:38 ==> 112 s
   
   Enhancement:
   test1: start | end  ==> 15:33:02|15:34:58 ==> 116 s
   test2: start | end ==> 16:54:20|16:56:22 ==> 122s
   
   
   ### Previous: 
   `Tablet::can_do_compaction` 40%, like Health Kit in China, red means 
unhealth.
   
   https://user-images.githubusercontent.com/35688959/159883132-ce6f2409-b9b3-4f68-b486-8996f5c54106.png";>
   
   ### Enhancement:
   `Tablet::can_do_compaction` 20%, like Health Kit in China, green means 
health.
   
   https://user-images.githubusercontent.com/35688959/159883088-19b9c97c-3ecd-49f5-9524-a662b40c0f29.png";>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] DarvenDuan opened a new pull request #8634: [bug] (sql_block_rule) Fix sql block rule bug

[GitBox]

DarvenDuan opened a new pull request #8634:
URL: https://github.com/apache/incubator-doris/pull/8634


   # Proposed changes
   
   Issue Number: #8629 & #8621
   
   ## Problem Summary:
   
   1. Check properties'  effectiveness of a sql_block_rule, can't set 
limitations of sql_block_rule to be negative.
   2. Optimize the judgment conditions when a query hits a sql_block_rule
   3. Check if sql_block_rule has already exist when exec `set property for 
"user" "sql_block_rule" = "xxx"`
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes)
   4. Has unit tests been added: (No Need)
   5. Has document been added or modified: (No)
   6. Does it need to update dependencies: (No)
   7. Are there any changes that cannot be rolled back: (No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] emerkfu commented on pull request #8299: [feature] Window funnel

[GitBox]

emerkfu commented on pull request #8299:
URL: https://github.com/apache/incubator-doris/pull/8299#issuecomment-1077483872


   Can window be in milliseconds? If the window is in seconds, events that 
occur in the same second but different milliseconds may be miscalculated. 
ClickHouse's windoFunnel has such a problem.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] emerkfu edited a comment on pull request #8299: [feature] Window funnel

[GitBox]

emerkfu edited a comment on pull request #8299:
URL: https://github.com/apache/incubator-doris/pull/8299#issuecomment-1077483872


   Can timestamp be in milliseconds? If the timestamp is in seconds, events 
that occur in the same second but different milliseconds may be miscalculated. 
ClickHouse's windoFunnel has such a problem.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] yiguolei opened a new pull request #8635: [Refactor] Remove etl mgr and etl job pool from be

[GitBox]

yiguolei opened a new pull request #8635:
URL: https://github.com/apache/incubator-doris/pull/8635


   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] wangbo closed issue #8300: [Bug] GroupingSet return error result with mismatch partition type

[GitBox]

wangbo closed issue #8300:
URL: https://github.com/apache/incubator-doris/issues/8300


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] zhuxueming169473 opened a new pull request #8636: Document update

[GitBox]

zhuxueming169473 opened a new pull request #8636:
URL: https://github.com/apache/incubator-doris/pull/8636


   # contend update
   
   "where" should be in front of "to".


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] hf200012 commented on pull request #8636: Document update

[GitBox]

hf200012 commented on pull request #8636:
URL: https://github.com/apache/incubator-doris/pull/8636#issuecomment-1077527226


   modify sidebar


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8632: [test] support a lot of actions

[GitBox]

github-actions[bot] commented on pull request #8632:
URL: https://github.com/apache/incubator-doris/pull/8632#issuecomment-1077528501






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] yangzhg opened a new pull request #8637: [chore] optimize aws thirdparty package download.

[GitBox]

yangzhg opened a new pull request #8637:
URL: https://github.com/apache/incubator-doris/pull/8637


   # Proposed changes
   
   optimize aws thirdparty package download. try prefetch_crt_dependency after 
download failed
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] emerkfu opened a new pull request #8638: Document Update compilation.md

[GitBox]

emerkfu opened a new pull request #8638:
URL: https://github.com/apache/incubator-doris/pull/8638


   # Proposed changes
   
   Added solutions to the "fatal error: Killed signal terminated program ..." 
problem encountered when compiling with Docker to the FAQ
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] 924060929 merged pull request #8632: [test] support a lot of actions

[GitBox]

924060929 merged pull request #8632:
URL: https://github.com/apache/incubator-doris/pull/8632


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris] branch master updated: [test] support a lot of actions (#8632)

[huajianlan]

This is an automated email from the ASF dual-hosted git repository.

huajianlan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 9db2a96  [test] support a lot of actions (#8632)
9db2a96 is described below

commit 9db2a96af19477c1269a86dc26ab2b647815cd3b
Author: 924060929 <924060...@qq.com>
AuthorDate: Thu Mar 24 20:22:24 2022 +0800

[test] support a lot of actions (#8632)

Support a lot of actions for regression testing framework.
e.g. thread, lazyCheck, onSuccess, connect, selectUnionAll, timer

Demo exists in ${DORIS_HOME}/regression-test/suites/demo
---
 docs/zh-CN/developer-guide/regression-testing.md   |   9 +-
 regression-test/data/demo/qt_action.out|   2 +-
 .../{qt_action.out => select_union_all_action.out} |  17 +-
 .../data/demo/{qt_action.out => thread_action.out} |  16 +-
 .../org/apache/doris/regression/Config.groovy  |  13 +-
 .../apache/doris/regression/ConfigOptions.groovy   |  10 ++
 .../apache/doris/regression/RegressionTest.groovy  |  42 +++--
 .../doris/regression/action/ExplainAction.groovy   |   6 +-
 .../regression/action/StreamLoadAction.groovy  |   4 +-
 .../doris/regression/action/TestAction.groovy  |   6 +-
 .../org/apache/doris/regression/suite/Suite.groovy | 198 +
 .../doris/regression/suite/SuiteContext.groovy |  67 +--
 .../doris/regression/util/OutputUtils.groovy   |  73 ++--
 .../apache/doris/regression/util/Recorder.groovy   |   4 +-
 .../framework/src/main/groovy/suite.gdsl   |  83 +
 regression-test/suites/demo/connect_action.groovy  |  16 ++
 regression-test/suites/demo/event_action.groovy|  40 +
 .../suites/demo/lazyCheck_action.groovy|  33 
 .../suites/demo/select_union_all_action.groovy |  19 ++
 regression-test/suites/demo/sql_action.groovy  |   2 +
 .../suites/demo/streamLoad_action.groovy   |   4 +-
 regression-test/suites/demo/thread_action.groovy   |  48 +
 regression-test/suites/demo/timer_action.groovy|   7 +
 23 files changed, 600 insertions(+), 119 deletions(-)

diff --git a/docs/zh-CN/developer-guide/regression-testing.md 
b/docs/zh-CN/developer-guide/regression-testing.md
index eff0cda..a962502 100644
--- a/docs/zh-CN/developer-guide/regression-testing.md
+++ b/docs/zh-CN/developer-guide/regression-testing.md
@@ -36,7 +36,7 @@ under the License.
 1. 需要预先安装好集群
 2. 修改配置文件`${DORIS_HOME}/conf/regression-conf.groovy`，设置jdbc url、用户等配置项
 3. 创建测试用例文件并编写用例
-4. 如果用例文件包含`qt` 
Action，则需要创建关联的的data文件，比如`suites/demo/qt_action.groovy`这个例子，需要用到`data/demo/qt_action.out`这个TSV文件来校验输出是否一致
+4. 如果用例文件包含`qt` 
Action，则需要创建关联的data文件，比如`suites/demo/qt_action.groovy`这个例子，需要用到`data/demo/qt_action.out`这个TSV文件来校验输出是否一致
 5. 
运行`${DORIS_HOME}/run-regression-test.sh`测试全部用例,或运行`${DORIS_HOME}/run-regression-test.sh
 --run ` 测试若干用例，更多例子见"启动脚本例子"章节
 
 ## 目录结构
@@ -114,7 +114,7 @@ customConf1 = "test_custom_conf_value"
 ## 编写用例的步骤
 1. 进入`${DORIS_HOME}/regression-test`目录
 2. 根据测试的目的来选择用例的目录，正确性测试存在`suites/correctness`，而性能测试存在`suites/performance`
-3. 新建一个groovy用例文件，增加若干`Action`用于测试，Action讲在后续章节具体说明
+3. 新建一个groovy用例文件，增加若干`Action`用于测试，Action将在后续章节具体说明
 
 ## Action
 Action是一个测试框架默认提供的测试行为，使用DSL来定义。
@@ -178,6 +178,7 @@ try {
  *return xxx(args)
  * } catch (Throwable t) {
  * // do nothing
+ * return null
  * }
  */
 try_sql("DROP TABLE IF EXISTS ${testTable}")
@@ -447,6 +448,10 @@ streamLoad {
 }
 ```
 
+### 其他Action
+thread, lazyCheck, events, connect, selectUnionAll
+具体可以在这个目录找到例子: `${DORIS_HOME}/regression-test/suites/demo`
+
 ## 启动脚本例子
 ```shell
 # 查看脚本参数说明
diff --git a/regression-test/data/demo/qt_action.out 
b/regression-test/data/demo/qt_action.out
index ea3e7b4..c34a713 100644
--- a/regression-test/data/demo/qt_action.out
+++ b/regression-test/data/demo/qt_action.out
@@ -6,7 +6,7 @@
 -- !select2 --
 2
 
--- !union --
+-- !union_all --
 \N
 1
 15
diff --git a/regression-test/data/demo/qt_action.out 
b/regression-test/data/demo/select_union_all_action.out
similarity index 57%
copy from regression-test/data/demo/qt_action.out
copy to regression-test/data/demo/select_union_all_action.out
index ea3e7b4..7cdd5df 100644
--- a/regression-test/data/demo/qt_action.out
+++ b/regression-test/data/demo/select_union_all_action.out
@@ -1,15 +1,10 @@
 -- This file is automatically generated. You should know what you did if you 
want to edit this
--- !select --
-1  beijing
-2  shanghai
-
--- !select2 --
-2
-
--- !union --
-\N
+-- !select_union_all1 --
 1
-15
-2
+10
 3
 
+-- !select_union_all2 --
+0  abc
+1  123
+2  \N
diff --git a/regression-test/data/demo/qt_action.out 
b/regression-test/data/demo/thread_action.out
similarity index 55%
copy from regression-test/data/demo/qt_action.out
copy to regression-test/data/demo/thread_action.out
index ea3e7b4..4f14b39 100644
--- 

[GitHub] [incubator-doris] airborne12 opened a new issue #8639: [Bug] be core when running yandex_metrica case

[GitBox]

airborne12 opened a new issue #8639:
URL: https://github.com/apache/incubator-doris/issues/8639


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Version
   
   dev-1.0.0
   
   ### What's Wrong?
   
   be coredump when running yandex_metrica case if set enable_vectorized_engine 
true.
   
   ### What You Expected?
   
   no core
   
   ### How to Reproduce?
   
   _No response_
   
   ### Anything Else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] Gabriel39 commented on a change in pull request #8597: [feature-wip](array-type)Add element_at and subscript functions

[GitBox]

Gabriel39 commented on a change in pull request #8597:
URL: https://github.com/apache/incubator-doris/pull/8597#discussion_r834254309



##
File path: be/src/vec/functions/array/function_array_element.h
##
@@ -0,0 +1,247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+// This file is copied from
+// 
https://github.com/ClickHouse/ClickHouse/blob/master/src/Functions/array/arrayElement.cpp
+// and modified by Doris
+#pragma once
+
+#include 
+
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_const.h"
+#include "vec/columns/column_string.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_helpers.h"
+
+namespace doris::vectorized {
+
+class FunctionArrayElement : public IFunction {
+public:
+static constexpr auto name = "element_at";
+static FunctionPtr create() { return 
std::make_shared(); }
+
+/// Get function name.
+String get_name() const override { return name; }
+
+bool is_variadic() const override { return false; }
+
+size_t get_number_of_arguments() const override { return 2; }
+
+DataTypePtr get_return_type_impl(const DataTypes& arguments) const 
override {
+DCHECK(is_array(arguments[0]))
+<< "first argument for function: " << name << " should be 
DataTypeArray";
+DCHECK(is_integer(arguments[1]))
+<< "second argument for function: " << name << " should be 
Integer";
+return make_nullable(
+
check_and_get_data_type(arguments[0].get())->get_nested_type());
+}
+
+Status execute_impl(FunctionContext* context, Block& block, const 
ColumnNumbers& arguments,
+size_t result, size_t input_rows_count) override {
+auto dst_null_column = ColumnUInt8::create(input_rows_count);
+UInt8* dst_null_map = dst_null_column->get_data().data();
+const UInt8* src_null_map = nullptr;
+ColumnsWithTypeAndName args;
+auto col_left = block.get_by_position(arguments[0]);
+if (col_left.column->is_nullable()) {
+auto null_col = 
check_and_get_column(*col_left.column);
+src_null_map = null_col->get_null_map_column().get_data().data();
+args = {{null_col->get_nested_column_ptr(), 
remove_nullable(col_left.type),
+ col_left.name},
+block.get_by_position(arguments[1])};
+} else {
+args = {col_left, block.get_by_position(arguments[1])};
+}
+
+auto result_type = remove_nullable(
+
check_and_get_data_type(args[0].type.get())->get_nested_type());
+
+auto res_column = _perform(args, result_type, input_rows_count, 
src_null_map, dst_null_map);
+if (!res_column) {
+return Status::RuntimeError(
+fmt::format("unsupported types for function {}({}, {})", 
get_name(),
+
block.get_by_position(arguments[0]).type->get_name(),
+
block.get_by_position(arguments[1]).type->get_name()));
+}
+block.replace_by_position(
+result, ColumnNullable::create(std::move(res_column), 
std::move(dst_null_column)));
+return Status::OK();
+}
+
+private:
+ColumnPtr _perform(const ColumnsWithTypeAndName& arguments, const 
DataTypePtr& result_type,
+   size_t input_rows_count, const UInt8* src_null_map, 
UInt8* dst_null_map) {
+ColumnPtr res;
+if (!((res = _execute_number(arguments, result_type, 
input_rows_count, src_null_map,
+   dst_null_map)) ||
+  (res = _execute_number(arguments, result_type, 
input_rows_count, src_null_map,
+dst_null_map)) ||
+  (res = _execute_number(arguments, result_type, 
input_rows_count, src_null_map,
+dst_null_map)) ||
+  (res = _execute_number(arguments, result_type, 
input_rows_count, src_null_map,
+dst_null_map)) ||
+  (r

[GitHub] [incubator-doris] xinyiZzz opened a new issue #8640: [Bug] Memory leak in VDataStreamRecvr::SenderQueue

[GitBox]

xinyiZzz opened a new issue #8640:
URL: https://github.com/apache/incubator-doris/issues/8640


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Version
   
   trunk
   
   ### What's Wrong?
   
   
![image](https://user-images.githubusercontent.com/13197424/159917022-83b82dc4-3ebb-4265-bd57-dcd8e4d6a8c5.png)
   
   
   ### What You Expected?
   
   When mem tracker is destructed, consumption is equal to 0
   
   ### How to Reproduce?
   
   _No response_
   
   ### Anything Else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] BiteTheDDDDt closed pull request #8470: [Enhancement] add clang-tidy config file && fix some clang-tidy diagnostic warnings and code style errors

[GitBox]

BiteThet closed pull request #8470:
URL: https://github.com/apache/incubator-doris/pull/8470


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] airborne12 opened a new pull request #8641: [bug](vec) fix coredump for aggregate function

[GitBox]

airborne12 opened a new pull request #8641:
URL: https://github.com/apache/incubator-doris/pull/8641


   [bug](vec) fix coredump for aggregate function when delete large_data, due 
to alloc-dealloc-mismatch
   
   # Proposed changes
   
   Issue Number: close #8639
   
   ## Problem Summary:
   
   BE coredump when running yundex metrica case
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (No)
   2. Has unit tests been added: (No Need)
   3. Has document been added or modified: (No Need)
   4. Does it need to update dependencies: (No)
   5. Are there any changes that cannot be rolled back: (No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] BiteTheDDDDt opened a new pull request #8642: [Enhancement] add clang-tidy config && add C++ Code Diagnostic document

[GitBox]

BiteThet opened a new pull request #8642:
URL: https://github.com/apache/incubator-doris/pull/8642


   # Proposed changes
   add clang-tidy config && add C++ Code Diagnostic document
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] xinyiZzz opened a new pull request #8643: [Bug] fix memory leak in VDataStreamRecvr::SenderQueue

[GitBox]

xinyiZzz opened a new pull request #8643:
URL: https://github.com/apache/incubator-doris/pull/8643


   # Proposed changes
   
   Issue Number: close #8640
   
   ## Problem Summary:
   
   After `VDataStreamRecvr::SenderQueue::close` clears `_block_queue`, calling 
`VDataStreamRecvr::SenderQueue::add_block` again will cause a memory leak.
   
   So, change the lock position, like the other add_block and add_batch.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] xinyiZzz commented on pull request #8643: [Bug] fix memory leak in VDataStreamRecvr::SenderQueue

[GitBox]

xinyiZzz commented on pull request #8643:
URL: https://github.com/apache/incubator-doris/pull/8643#issuecomment-1077593568


   cc @dataroaring  


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8641: [bug](vec) fix coredump for aggregate function

[GitBox]

github-actions[bot] commented on pull request #8641:
URL: https://github.com/apache/incubator-doris/pull/8641#issuecomment-1077594647


   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8643: [Bug] fix memory leak in VDataStreamRecvr::SenderQueue

[GitBox]

github-actions[bot] commented on pull request #8643:
URL: https://github.com/apache/incubator-doris/pull/8643#issuecomment-1077596134


   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] dataroaring commented on pull request #8642: [Enhancement] add clang-tidy config && add C++ Code Diagnostic document

[GitBox]

dataroaring commented on pull request #8642:
URL: https://github.com/apache/incubator-doris/pull/8642#issuecomment-1077596864


   Could tidy find the bug like below?
   https://github.com/apache/incubator-doris/pull/8641


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] dataroaring commented on a change in pull request #8638: Document Update compilation.md

[GitBox]

dataroaring commented on a change in pull request #8638:
URL: https://github.com/apache/incubator-doris/pull/8638#discussion_r834277455



##
File path: docs/zh-CN/installing/compilation.md
##
@@ -207,6 +207,12 @@ under the License.
 
 REPOSITORY_URL 中包含所有第三方库源码包和他们的历史版本。
 
+3. `fatal error: Killed signal terminated program ...`
+
+使用 Docker 镜像编译时如遇到上述报错，可能是分配给镜像的内存不足（Docker 默认分配的内存大小为 2GB，编译过程中内存占用的峰值大于 
2GB）
+
+尝试适当调大镜像的分配内存，推荐 4GB ~ 8GB。
+

Review comment:
   Could you write english counter part?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] emerkfu commented on a change in pull request #8638: Document Update compilation.md

[GitBox]

emerkfu commented on a change in pull request #8638:
URL: https://github.com/apache/incubator-doris/pull/8638#discussion_r834281524



##
File path: docs/zh-CN/installing/compilation.md
##
@@ -207,6 +207,12 @@ under the License.
 
 REPOSITORY_URL 中包含所有第三方库源码包和他们的历史版本。
 
+3. `fatal error: Killed signal terminated program ...`
+
+使用 Docker 镜像编译时如遇到上述报错，可能是分配给镜像的内存不足（Docker 默认分配的内存大小为 2GB，编译过程中内存占用的峰值大于 
2GB）
+
+尝试适当调大镜像的分配内存，推荐 4GB ~ 8GB。
+

Review comment:
   ok




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8500: [Opt] speed up data load for duplicate table no-vec-load

[GitBox]

github-actions[bot] commented on pull request #8500:
URL: https://github.com/apache/incubator-doris/pull/8500#issuecomment-1077610052






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] emerkfu opened a new pull request #8644: Document Update compilation.md(en)

[GitBox]

emerkfu opened a new pull request #8644:
URL: https://github.com/apache/incubator-doris/pull/8644


   Added solutions to the "fatal error: Killed signal terminated program ..." 
problem encountered when compiling with Docker to the FAQ.
   
   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] hf200012 merged pull request #8609: [Doris Manager][Doc]Basic User Documents for Doris Manager

[GitBox]

hf200012 merged pull request #8609:
URL: https://github.com/apache/incubator-doris/pull/8609


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris] branch master updated: [Doris Manager][Doc]Basic User Documents for Doris Manager (#8609)

[jiafengzheng]

This is an automated email from the ASF dual-hosted git repository.

jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 5511d43  [Doris Manager][Doc]Basic User Documents for Doris Manager 
(#8609)
5511d43 is described below

commit 5511d435dee869c225d10cdd4cf56bb90a075419
Author: Justice Gong 
AuthorDate: Thu Mar 24 21:34:49 2022 +0800

[Doris Manager][Doc]Basic User Documents for Doris Manager (#8609)
---
 .../images/doris-manager/clustermanagenent-1.png   | Bin 0 -> 811985 bytes
 .../images/doris-manager/clustermanagenent-2.png   | Bin 0 -> 303574 bytes
 .../images/doris-manager/clustermanagenent-3.png   | Bin 0 -> 72 bytes
 .../images/doris-manager/clustermanagenent-4.png   | Bin 0 -> 277173 bytes
 .../images/doris-manager/clustermanagenent-5.png   | Bin 0 -> 396001 bytes
 .../public/images/doris-manager/initializing-1.png | Bin 0 -> 72708 bytes
 .../images/doris-manager/initializing-10.png   | Bin 0 -> 651452 bytes
 .../images/doris-manager/initializing-11.png   | Bin 0 -> 456628 bytes
 .../images/doris-manager/initializing-12.png   | Bin 0 -> 611762 bytes
 .../images/doris-manager/initializing-13.png   | Bin 0 -> 426572 bytes
 .../images/doris-manager/initializing-14.png   | Bin 0 -> 422438 bytes
 .../images/doris-manager/initializing-15.png   | Bin 0 -> 44133 bytes
 .../images/doris-manager/initializing-16.png   | Bin 0 -> 380961 bytes
 .../images/doris-manager/initializing-17.png   | Bin 0 -> 128710 bytes
 .../images/doris-manager/initializing-18.png   | Bin 0 -> 323719 bytes
 .../images/doris-manager/initializing-19.png   | Bin 0 -> 589860 bytes
 .../public/images/doris-manager/initializing-2.png | Bin 0 -> 328269 bytes
 .../public/images/doris-manager/initializing-3.png | Bin 0 -> 651452 bytes
 .../public/images/doris-manager/initializing-4.png | Bin 0 -> 809366 bytes
 .../public/images/doris-manager/initializing-5.png | Bin 0 -> 704258 bytes
 .../public/images/doris-manager/initializing-6.png | Bin 0 -> 565328 bytes
 .../public/images/doris-manager/initializing-7.png | Bin 0 -> 602699 bytes
 .../public/images/doris-manager/initializing-8.png | Bin 0 -> 564646 bytes
 .../public/images/doris-manager/initializing-9.png | Bin 0 -> 609950 bytes
 .../images/doris-manager/spacemanagement-1.png | Bin 0 -> 459251 bytes
 .../images/doris-manager/spacemanagement-10.png| Bin 0 -> 391499 bytes
 .../images/doris-manager/spacemanagement-2.png | Bin 0 -> 329131 bytes
 .../images/doris-manager/spacemanagement-3.png | Bin 0 -> 315978 bytes
 .../images/doris-manager/spacemanagement-4.png | Bin 0 -> 319956 bytes
 .../images/doris-manager/spacemanagement-5.png | Bin 0 -> 361250 bytes
 .../images/doris-manager/spacemanagement-6.png | Bin 0 -> 303275 bytes
 .../images/doris-manager/spacemanagement-7.png | Bin 0 -> 403391 bytes
 .../images/doris-manager/spacemanagement-8.png | Bin 0 -> 351331 bytes
 .../images/doris-manager/spacemanagement-9.png | Bin 0 -> 330859 bytes
 .../images/doris-manager/systemsettings-1.png  | Bin 0 -> 561497 bytes
 .../images/doris-manager/systemsettings-10.png | Bin 0 -> 551044 bytes
 .../images/doris-manager/systemsettings-11.png | Bin 0 -> 353040 bytes
 .../images/doris-manager/systemsettings-2.png  | Bin 0 -> 449987 bytes
 .../images/doris-manager/systemsettings-3.png  | Bin 0 -> 2219701 bytes
 .../images/doris-manager/systemsettings-4.png  | Bin 0 -> 2196473 bytes
 .../images/doris-manager/systemsettings-5.png  | Bin 0 -> 432792 bytes
 .../images/doris-manager/systemsettings-6.png  | Bin 0 -> 2192074 bytes
 .../images/doris-manager/systemsettings-7.png  | Bin 0 -> 381154 bytes
 .../images/doris-manager/systemsettings-8.png  | Bin 0 -> 2189352 bytes
 .../images/doris-manager/systemsettings-9.png  | Bin 0 -> 347617 bytes
 docs/.vuepress/sidebar/zh-CN.js|  11 +
 .../doris-manager/cluster-managenent.md|  69 +
 .../doris-manager/compiling-deploying.md   | 102 
 .../extending-doris/doris-manager/initializing.md  | 277 +
 .../doris-manager/space-management.md  | 101 
 .../doris-manager/system-settings.md   |  82 ++
 51 files changed, 642 insertions(+)

diff --git a/docs/.vuepress/public/images/doris-manager/clustermanagenent-1.png 
b/docs/.vuepress/public/images/doris-manager/clustermanagenent-1.png
new file mode 100644
index 000..50185ef
Binary files /dev/null and 
b/docs/.vuepress/public/images/doris-manager/clustermanagenent-1.png differ
diff --git a/docs/.vuepress/public/images/doris-manager/clustermanagenent-2.png 
b/docs/.vuepress/public/images/doris-manager/clustermanagenent-2.png
new file mode 100644
index 000..e7e0383
Binary files /dev/null and 
b/docs/.vuepress/public/images/doris-manager/clustermana

[GitHub] [incubator-doris] caiconghui commented on pull request #8644: Document Update compilation.md(en)

[GitBox]

caiconghui commented on pull request #8644:
URL: https://github.com/apache/incubator-doris/pull/8644#issuecomment-1077644961


   suggest merge in one PR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] daikon12 opened a new pull request #8645: [doc][fix][optimize some doc expression]

[GitBox]

daikon12 opened a new pull request #8645:
URL: https://github.com/apache/incubator-doris/pull/8645


   # Proposed changes
   
   Update confusing descriptions;
   Modify long and difficult sentences into short and easy sentences
   
   ## Problem Summary:
   
   Some long sentences in the doc burden the user's comprehension, I suggest 
optimize .
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (No)
   2. Has unit tests been added: (No)
   3. Has document been added or modified: (Yes)
   4. Does it need to update dependencies: (No)
   5. Are there any changes that cannot be rolled back: (No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] emerkfu opened a new pull request #8646: Document Update compilation.md

[GitBox]

emerkfu opened a new pull request #8646:
URL: https://github.com/apache/incubator-doris/pull/8646


   # Proposed changes
   
   Added solutions to the "fatal error: Killed signal terminated program ..." 
problem encountered when compiling with Docker to the FAQ.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] hf200012 closed pull request #8644: Document Update compilation.md(en)

[GitBox]

hf200012 closed pull request #8644:
URL: https://github.com/apache/incubator-doris/pull/8644


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] hf200012 closed pull request #8638: Document Update compilation.md

[GitBox]

hf200012 closed pull request #8638:
URL: https://github.com/apache/incubator-doris/pull/8638


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8641: [bug](vec) fix coredump for aggregate function

[GitBox]

github-actions[bot] commented on pull request #8641:
URL: https://github.com/apache/incubator-doris/pull/8641#issuecomment-1077705797


   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] daikon12 commented on pull request #8645: [doc][fix][optimize some doc expression]

[GitBox]

daikon12 commented on pull request #8645:
URL: https://github.com/apache/incubator-doris/pull/8645#issuecomment-1077717606


   
   
   
   > Please update the English documentation
   @hf200012 Thank you for reminding me, I noticed that the corresponding 
English doc are fine. 
   At your suggestion, I also submitted the places that I thought could be 
optimized.
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] dataroaring opened a new pull request #8647: add -rtlib=compiler-rt for UBSAN under clang

[GitBox]

dataroaring opened a new pull request #8647:
URL: https://github.com/apache/incubator-doris/pull/8647


   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (No)
   2. Has unit tests been added: (No Need)
   3. Has document been added or modified: (No Need)
   4. Does it need to update dependencies: (No)
   5. Are there any changes that cannot be rolled back: (No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] morningman commented on a change in pull request #8643: [Bug] fix memory leak in VDataStreamRecvr::SenderQueue

[GitBox]

morningman commented on a change in pull request #8643:
URL: https://github.com/apache/incubator-doris/pull/8643#discussion_r834473251



##
File path: be/src/vec/runtime/vdata_stream_recvr.cpp
##
@@ -140,6 +140,7 @@ void VDataStreamRecvr::SenderQueue::add_block(const PBlock& 
pblock, int be_numbe
 }
 
 void VDataStreamRecvr::SenderQueue::add_block(Block* block, bool use_move) {
+std::unique_lock l(_lock);

Review comment:
   ```suggestion
   std::lock_guard l(_lock);
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] morningman opened a new issue #8648: [Bug] There are more than xxx unfinished load jobs, please retry later.

[GitBox]

morningman opened a new issue #8648:
URL: https://github.com/apache/incubator-doris/issues/8648


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Version
   
   trunk
   
   ### What's Wrong?
   
   When submitting a load job, error returns:
   ```
   There are more than 100 unfinished load jobs, please retry later. You can 
use `SHOW LOAD` to view submitted jobs
   ```
   
   But `show load` return no job's running.
   
   ### What You Expected?
   
   Can submit load job
   
   ### How to Reproduce?
   
   _No response_
   
   ### Anything Else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] morningman opened a new pull request #8649: [fix](mini-load) Remove mini load in LOADING and PENDING state

[GitBox]

morningman opened a new pull request #8649:
URL: https://github.com/apache/incubator-doris/pull/8649


   # Proposed changes
   
   Issue Number: close #8648 
   
   ## Problem Summary:
   
   1. Remove some unused code.
   2. handle mini load with wrong state
   1. For some historical reasons, some mini load jobs in LOADING state 
have not been cleared.
   As a result, new load jobs cannot be committed.
   2. If a mini load job is created right before FE restart, the mini load 
job will be in PENDING state forever.
   But it should be removed finally.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (No)
   4. Has unit tests been added: (No Need)
   5. Has document been added or modified: (No Need)
   6. Does it need to update dependencies: (No)
   7. Are there any changes that cannot be rolled back: (No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] yangzhg merged pull request #8567: [api-change] add soft limit of String type length

[GitBox]

yangzhg merged pull request #8567:
URL: https://github.com/apache/incubator-doris/pull/8567


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris] branch master updated (5511d43 -> cfb57be)

[yangzhg]

This is an automated email from the ASF dual-hosted git repository.

yangzhg pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git.


from 5511d43  [Doris Manager][Doc]Basic User Documents for Doris Manager 
(#8609)
 add cfb57be  [api-change] add soft limit of String type length (#8567)

No new revisions were added by this update.

Summary of changes:
 be/src/common/config.h |  7 +++
 be/src/exec/tablet_sink.cpp| 55 +-
 be/src/olap/delete_handler.cpp |  4 +-
 be/src/olap/olap_define.h  |  3 --
 be/src/olap/row_block2.cpp |  9 ++--
 be/src/olap/types.h|  4 +-
 be/src/olap/wrapper_field.cpp  |  5 +-
 be/src/runtime/types.h |  3 +-
 be/src/vec/sink/vtablet_sink.cpp   | 29 ++--
 docs/en/administrator-guide/config/be_config.md|  5 ++
 .../sql-statements/Data Types/STRING.md|  2 +-
 docs/zh-CN/administrator-guide/config/be_config.md |  5 ++
 .../sql-statements/Data Types/STRING.md|  2 +-
 fe/fe-core/src/main/cup/sql_parser.cup |  2 -
 .../java/org/apache/doris/analysis/ColumnDef.java  |  5 +-
 .../org/apache/doris/analysis/CreateTableStmt.java | 12 +++--
 .../apache/doris/analysis/DistributionDesc.java|  2 +-
 .../doris/analysis/HashDistributionDesc.java   | 18 ++-
 .../org/apache/doris/analysis/PartitionDesc.java   |  5 ++
 .../doris/analysis/RandomDistributionDesc.java |  2 +-
 .../java/org/apache/doris/catalog/ScalarType.java  |  1 -
 .../java/org/apache/doris/planner/PlannerTest.java | 15 ++
 22 files changed, 129 insertions(+), 66 deletions(-)

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] cambyzju commented on a change in pull request #8597: [feature-wip](array-type)Add element_at and subscript functions

[GitBox]

cambyzju commented on a change in pull request #8597:
URL: https://github.com/apache/incubator-doris/pull/8597#discussion_r834881751



##
File path: be/src/vec/functions/array/function_array_element.h
##
@@ -0,0 +1,247 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+// This file is copied from
+// 
https://github.com/ClickHouse/ClickHouse/blob/master/src/Functions/array/arrayElement.cpp
+// and modified by Doris
+#pragma once
+
+#include 
+
+#include "vec/columns/column_array.h"
+#include "vec/columns/column_const.h"
+#include "vec/columns/column_string.h"
+#include "vec/data_types/data_type_array.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/functions/function.h"
+#include "vec/functions/function_helpers.h"
+
+namespace doris::vectorized {
+
+class FunctionArrayElement : public IFunction {
+public:
+static constexpr auto name = "element_at";
+static FunctionPtr create() { return 
std::make_shared(); }
+
+/// Get function name.
+String get_name() const override { return name; }
+
+bool is_variadic() const override { return false; }
+
+size_t get_number_of_arguments() const override { return 2; }
+
+DataTypePtr get_return_type_impl(const DataTypes& arguments) const 
override {
+DCHECK(is_array(arguments[0]))
+<< "first argument for function: " << name << " should be 
DataTypeArray";
+DCHECK(is_integer(arguments[1]))
+<< "second argument for function: " << name << " should be 
Integer";
+return make_nullable(
+
check_and_get_data_type(arguments[0].get())->get_nested_type());
+}
+
+Status execute_impl(FunctionContext* context, Block& block, const 
ColumnNumbers& arguments,
+size_t result, size_t input_rows_count) override {
+auto dst_null_column = ColumnUInt8::create(input_rows_count);
+UInt8* dst_null_map = dst_null_column->get_data().data();
+const UInt8* src_null_map = nullptr;
+ColumnsWithTypeAndName args;
+auto col_left = block.get_by_position(arguments[0]);
+if (col_left.column->is_nullable()) {
+auto null_col = 
check_and_get_column(*col_left.column);
+src_null_map = null_col->get_null_map_column().get_data().data();
+args = {{null_col->get_nested_column_ptr(), 
remove_nullable(col_left.type),
+ col_left.name},
+block.get_by_position(arguments[1])};
+} else {
+args = {col_left, block.get_by_position(arguments[1])};
+}
+
+auto result_type = remove_nullable(
+
check_and_get_data_type(args[0].type.get())->get_nested_type());
+
+auto res_column = _perform(args, result_type, input_rows_count, 
src_null_map, dst_null_map);
+if (!res_column) {
+return Status::RuntimeError(
+fmt::format("unsupported types for function {}({}, {})", 
get_name(),
+
block.get_by_position(arguments[0]).type->get_name(),
+
block.get_by_position(arguments[1]).type->get_name()));
+}
+block.replace_by_position(
+result, ColumnNullable::create(std::move(res_column), 
std::move(dst_null_column)));
+return Status::OK();
+}
+
+private:
+ColumnPtr _perform(const ColumnsWithTypeAndName& arguments, const 
DataTypePtr& result_type,
+   size_t input_rows_count, const UInt8* src_null_map, 
UInt8* dst_null_map) {
+ColumnPtr res;
+if (!((res = _execute_number(arguments, result_type, 
input_rows_count, src_null_map,
+   dst_null_map)) ||
+  (res = _execute_number(arguments, result_type, 
input_rows_count, src_null_map,
+dst_null_map)) ||
+  (res = _execute_number(arguments, result_type, 
input_rows_count, src_null_map,
+dst_null_map)) ||
+  (res = _execute_number(arguments, result_type, 
input_rows_count, src_null_map,
+dst_null_map)) ||
+  (re

[GitHub] [incubator-doris] xinyiZzz commented on a change in pull request #8643: [Bug] fix memory leak in VDataStreamRecvr::SenderQueue

[GitBox]

xinyiZzz commented on a change in pull request #8643:
URL: https://github.com/apache/incubator-doris/pull/8643#discussion_r834899290



##
File path: be/src/vec/runtime/vdata_stream_recvr.cpp
##
@@ -140,6 +140,7 @@ void VDataStreamRecvr::SenderQueue::add_block(const PBlock& 
pblock, int be_numbe
 }
 
 void VDataStreamRecvr::SenderQueue::add_block(Block* block, bool use_move) {
+std::unique_lock l(_lock);

Review comment:
   fix, great




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] luozenglin commented on pull request #8649: [fix](mini-load) Remove mini load in LOADING and PENDING state

[GitBox]

luozenglin commented on pull request #8649:
URL: https://github.com/apache/incubator-doris/pull/8649#issuecomment-1078583574


   LGTM


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] yiguolei commented on pull request #8649: [fix](mini-load) Remove mini load in LOADING and PENDING state

[GitBox]

yiguolei commented on pull request #8649:
URL: https://github.com/apache/incubator-doris/pull/8649#issuecomment-1078584367


   Could we reimplement the miniload api by using stream load? And remove all 
these mini load code?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] adonis0147 commented on a change in pull request #8642: [Enhancement] add clang-tidy config && add C++ Code Diagnostic document

[GitBox]

adonis0147 commented on a change in pull request #8642:
URL: https://github.com/apache/incubator-doris/pull/8642#discussion_r834903719



##
File path: docs/en/developer-guide/cpp-diagnostic-code.md
##
@@ -0,0 +1,53 @@
+---
+{
+"title": "C++ Code Diagnostic",
+"language": "en"
+}
+---
+
+
+
+# C++ Code Diagnostic
+
+Doris support to use [Clangd](https://clangd.llvm.org/) and 
[Clang-Tidy](https://clang.llvm.org/extra/clang-tidy/) to diagnostic code. 
Clangd and Clang-Tidy already has in 
[LDB-toolchain](https://doris.apache.org/zh-CN/installing/compilation-with-ldb-toolchain)，also
 can install by self.
+
+### Clang-Tidy
+Clang-Tidy can do some diagnostic cofig, config file `.clang-tidy` is in Doris 
root path.
+
+### Enable clangd on VSCODE
+
+First we should install clangd plugin, then edit `settings.json` or just 
change config on gui.
+
+```json
+"clangd.path": "ldb_toolchain/bin/clangd", //clangd path
+"clangd.arguments": [
+"--background-index",
+"--clang-tidy", //enable clang-tidy
+"--compile-commands-dir=doris/be/build_RELEASE/", //clangd should read 
compile_commands.json create by cmake, so you should compile once
+"--completion-style=detailed",
+"-j=5", //clangd diagnostic parallelism
+"--all-scopes-completion",
+"--pch-storage=memory",
+"--pretty",
+"-log=verbose",
+"--query-driver=ldb_toolchain/bin/g++" //path of g++ or clang++

Review comment:
   ```suggestion
   "--query-driver=ldb_toolchain/bin/*" // path of compiler
   ```
   Using wildcard to match is better.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] adonis0147 commented on a change in pull request #8642: [Enhancement] add clang-tidy config && add C++ Code Diagnostic document

[GitBox]

adonis0147 commented on a change in pull request #8642:
URL: https://github.com/apache/incubator-doris/pull/8642#discussion_r834903770



##
File path: docs/zh-CN/developer-guide/cpp-diagnostic-code.md
##
@@ -0,0 +1,53 @@
+---
+{
+"title": "C++ 代码分析",
+"language": "zh-CN"
+}
+---
+
+
+
+# C++ 代码分析
+
+Doris支持使用[Clangd](https://clangd.llvm.org/)和[Clang-Tidy](https://clang.llvm.org/extra/clang-tidy/)进行代码静态分析。Clangd和Clang-Tidy在[LDB-toolchain](https://doris.apache.org/zh-CN/installing/compilation-with-ldb-toolchain)中已经内置，另外也可以自己安装或者编译。
+
+### Clang-Tidy
+Clang-Tidy中可以做一些代码分析的配置,配置文件`.clang-tidy`在Doris根目录下。
+
+### 在VSCODE中配置Clangd
+
+首先需要安装clangd插件，然后在`settings.json`中编辑或者直接在首选项中更改插件配置。
+
+```json
+"clangd.path": "ldb_toolchain/bin/clangd", //clangd的路径
+"clangd.arguments": [
+"--background-index",
+"--clang-tidy", //开启clang-tidy
+"--compile-commands-dir=doris/be/build_RELEASE/", 
//会用到cmake生成的compile_commands.json,所以需要先编译一次生成该文件
+"--completion-style=detailed",
+"-j=5", //clangd分析文件的并行数
+"--all-scopes-completion",
+"--pch-storage=memory",
+"--pretty",
+"-log=verbose",
+"--query-driver=ldb_toolchain/bin/g++" //g++或者clang++的路径

Review comment:
   ```suggestion
   "--query-driver=ldb_toolchain/bin/*" // 编译器的路径
   ```
   Using wildcard to match is better.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8631: [chore][be-test] Link gtest_main to provide default main function definition.

[GitBox]

github-actions[bot] commented on pull request #8631:
URL: https://github.com/apache/incubator-doris/pull/8631#issuecomment-1078603071


   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] zxm17863626761 opened a new pull request #8650: Document update

[GitBox]

zxm17863626761 opened a new pull request #8650:
URL: https://github.com/apache/incubator-doris/pull/8650


   "where" should be in front of "to".


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8650: Document update

[GitBox]

github-actions[bot] commented on pull request #8650:
URL: https://github.com/apache/incubator-doris/pull/8650#issuecomment-1078603991






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] dataroaring opened a new pull request #8651: limit memory used by regression test framework

[GitBox]

dataroaring opened a new pull request #8651:
URL: https://github.com/apache/incubator-doris/pull/8651


   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] adonis0147 opened a new issue #8652: [Enhancement] Optimize build_lz4 in build-thirdparty.sh

[GitBox]

adonis0147 opened a new issue #8652:
URL: https://github.com/apache/incubator-doris/issues/8652


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Description
   
   For now, when we can't rerun `build-thirdparty.sh` because `build_lz4` will 
fail.
   
   
![Snipaste_2022-03-25_10-42-19](https://user-images.githubusercontent.com/1443003/160046034-a634f05d-a923-49fa-92b3-c7486ca023b2.jpg)
   
   ### Solution
   
   Clean old symbolic links before build lz4.
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] adonis0147 opened a new pull request #8653: [chore] Optimize build_lz4 in build-thirdparty.sh

[GitBox]

adonis0147 opened a new pull request #8653:
URL: https://github.com/apache/incubator-doris/pull/8653


   # Proposed changes
   
   Issue Number: close #8652
   
   ## Problem Summary:
   
   Please refer to #8652 
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: No
   2. Has unit tests been added: No Need
   3. Has document been added or modified: No Need
   4. Does it need to update dependencies: No
   5. Are there any changes that cannot be rolled back: No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris-spark-connector] smallhibiscus opened a new pull request #15: [improvement] stream load data is converted to json format

[GitBox]

smallhibiscus opened a new pull request #15:
URL: https://github.com/apache/incubator-doris-spark-connector/pull/15


   # Proposed changes
   
   Issue Number: close #14
   
   ## Problem Summary:
   
   Stream load data is converted to json format.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] wangbo commented on a change in pull request #8627: [refactor][optimize] Code optimization and refactoring for low-cardinality columns in storage layer

[GitBox]

wangbo commented on a change in pull request #8627:
URL: https://github.com/apache/incubator-doris/pull/8627#discussion_r834937056



##
File path: be/src/vec/columns/column_dictionary.h
##
@@ -242,18 +213,73 @@ class ColumnDictionary final : public COWHelper> {
const StringRef* dict_array, size_t data_num,
uint32_t dict_num) override {
 if (!is_dict_inited()) {
-dict.reserve(dict_num);
+_dict.reserve(dict_num);
 for (uint32_t i = 0; i < dict_num; ++i) {
 auto value = StringValue(dict_array[i].data, 
dict_array[i].size);
-dict.insert_value(value);
+_dict.insert_value(value);
 }
 _dict_inited = true;
 }
 
-char* end_ptr = (char*)codes.get_end_ptr();
+char* end_ptr = (char*)_codes.get_end_ptr();
 memcpy(end_ptr, data_array + start_index, data_num * sizeof(T));
 end_ptr += data_num * sizeof(T);
-codes.set_end_ptr(end_ptr);
+_codes.set_end_ptr(end_ptr);
+}
+
+void convert_dict_codes_if_necessary() override {
+if (!is_dict_sorted()) {
+_dict.sort();
+_dict_sorted = true;
+}
+
+if (!is_dict_code_converted()) {
+for (size_t i = 0; i < size(); ++i) {
+_codes[i] = _dict.convert_code(_codes[i]);
+}
+_dict_code_converted = true;
+}
+}
+
+void set_predicate_dict_code_if_necessary(doris::ColumnPredicate* 
predicate) override {
+switch (predicate->type()) {
+case PredicateType::EQ:
+case PredicateType::NE: {
+// cast to EqualPredicate, just to get value, may not be 
EqualPredicate
+auto* comp_pred = 
reinterpret_cast*>(predicate);
+auto pred_value = comp_pred->get_value();
+auto code = _dict.find_code(pred_value);
+comp_pred->set_dict_code(code);
+break;
+}
+case PredicateType::LT:
+case PredicateType::LE:
+case PredicateType::GT:
+case PredicateType::GE: {
+// cast to LessPredicate, just to get value, may not be 
LessPredicate
+auto* comp_pred = 
reinterpret_cast*>(predicate);
+auto pred_value = comp_pred->get_value();
+auto less = predicate->type() == PredicateType::LT ||
+predicate->type() == PredicateType::LE;
+auto eq = predicate->type() == PredicateType::LE ||
+  predicate->type() == PredicateType::GE;
+auto code = _dict.find_bound_code(pred_value, less, eq);
+comp_pred->set_dict_code(code);
+break;
+}
+case PredicateType::InList:
+case PredicateType::NotInList: {
+auto* in_pred = 
reinterpret_cast*>(predicate);
+auto pred_values = in_pred->get_values();
+auto code_set = _dict.find_codes(pred_values);
+in_pred->set_dict_codes(code_set);
+break;
+}
+default:
+LOG(FATAL) << "PredicateType: " << 
static_cast(predicate->type())
+   <<  " not supported in ColumnDictionary";
+break;

Review comment:
   ```LOG(FATAL)``` could core directly， so break is useless.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] dataroaring opened a new pull request #8654: regression framework use RollingFileAppender by default

[GitBox]

dataroaring opened a new pull request #8654:
URL: https://github.com/apache/incubator-doris/pull/8654


   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris-flink-connector] branch branch-for-flink-before-1.13 updated: add serialVersionUID (#22)

[jiafengzheng]

This is an automated email from the ASF dual-hosted git repository.

jiafengzheng pushed a commit to branch branch-for-flink-before-1.13
in repository 
https://gitbox.apache.org/repos/asf/incubator-doris-flink-connector.git


The following commit(s) were added to refs/heads/branch-for-flink-before-1.13 
by this push:
 new 52c431b  add serialVersionUID (#22)
52c431b is described below

commit 52c431b16a52517de9fc7c4263718e9959e404d3
Author: wudi <676366...@qq.com>
AuthorDate: Fri Mar 25 12:11:48 2022 +0800

add serialVersionUID (#22)

add serialVersionUID
---
 .../main/java/org/apache/doris/flink/cfg/GenericDorisSinkFunction.java   | 1 +
 .../main/java/org/apache/doris/flink/datastream/DorisSourceFunction.java | 1 +
 .../main/java/org/apache/doris/flink/table/DorisDynamicOutputFormat.java | 1 +
 .../src/main/java/org/apache/doris/flink/table/DorisStreamLoad.java  | 1 +
 .../src/main/java/org/apache/doris/flink/table/DorisTableInputSplit.java | 1 +
 5 files changed, 5 insertions(+)

diff --git 
a/flink-doris-connector/src/main/java/org/apache/doris/flink/cfg/GenericDorisSinkFunction.java
 
b/flink-doris-connector/src/main/java/org/apache/doris/flink/cfg/GenericDorisSinkFunction.java
index 6be6aa4..0ea11d0 100644
--- 
a/flink-doris-connector/src/main/java/org/apache/doris/flink/cfg/GenericDorisSinkFunction.java
+++ 
b/flink-doris-connector/src/main/java/org/apache/doris/flink/cfg/GenericDorisSinkFunction.java
@@ -31,6 +31,7 @@ import javax.annotation.Nonnull;
 public class GenericDorisSinkFunction extends RichSinkFunction
 implements CheckpointedFunction {
 
+private static final long serialVersionUID = 1L;
 private final DorisDynamicOutputFormat outputFormat;
 
 public GenericDorisSinkFunction(@Nonnull DorisDynamicOutputFormat 
outputFormat) {
diff --git 
a/flink-doris-connector/src/main/java/org/apache/doris/flink/datastream/DorisSourceFunction.java
 
b/flink-doris-connector/src/main/java/org/apache/doris/flink/datastream/DorisSourceFunction.java
index edde953..df2a576 100644
--- 
a/flink-doris-connector/src/main/java/org/apache/doris/flink/datastream/DorisSourceFunction.java
+++ 
b/flink-doris-connector/src/main/java/org/apache/doris/flink/datastream/DorisSourceFunction.java
@@ -40,6 +40,7 @@ import java.util.List;
 
 public class DorisSourceFunction extends RichParallelSourceFunction> 
implements ResultTypeQueryable> {
 
+private static final long serialVersionUID = 1L;
 private static final Logger logger = 
LoggerFactory.getLogger(DorisSourceFunction.class);
 
 private final DorisDeserializationSchema> deserializer;
diff --git 
a/flink-doris-connector/src/main/java/org/apache/doris/flink/table/DorisDynamicOutputFormat.java
 
b/flink-doris-connector/src/main/java/org/apache/doris/flink/table/DorisDynamicOutputFormat.java
index cd8dee0..3f51df6 100644
--- 
a/flink-doris-connector/src/main/java/org/apache/doris/flink/table/DorisDynamicOutputFormat.java
+++ 
b/flink-doris-connector/src/main/java/org/apache/doris/flink/table/DorisDynamicOutputFormat.java
@@ -59,6 +59,7 @@ import static 
org.apache.flink.table.data.RowData.createFieldGetter;
  **/
 public class DorisDynamicOutputFormat extends RichOutputFormat {
 
+private static final long serialVersionUID = 1L;
 private static final Logger LOG = 
LoggerFactory.getLogger(DorisDynamicOutputFormat.class);
 private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
 
diff --git 
a/flink-doris-connector/src/main/java/org/apache/doris/flink/table/DorisStreamLoad.java
 
b/flink-doris-connector/src/main/java/org/apache/doris/flink/table/DorisStreamLoad.java
index 9c05b83..0d5ba7f 100644
--- 
a/flink-doris-connector/src/main/java/org/apache/doris/flink/table/DorisStreamLoad.java
+++ 
b/flink-doris-connector/src/main/java/org/apache/doris/flink/table/DorisStreamLoad.java
@@ -51,6 +51,7 @@ import java.util.UUID;
  **/
 public class DorisStreamLoad implements Serializable {
 
+private static final long serialVersionUID = 1L;
 private static final Logger LOG = 
LoggerFactory.getLogger(DorisStreamLoad.class);
 private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
 
diff --git 
a/flink-doris-connector/src/main/java/org/apache/doris/flink/table/DorisTableInputSplit.java
 
b/flink-doris-connector/src/main/java/org/apache/doris/flink/table/DorisTableInputSplit.java
index 5e81cc1..c9db66e 100644
--- 
a/flink-doris-connector/src/main/java/org/apache/doris/flink/table/DorisTableInputSplit.java
+++ 
b/flink-doris-connector/src/main/java/org/apache/doris/flink/table/DorisTableInputSplit.java
@@ -24,6 +24,7 @@ import org.apache.flink.core.io.InputSplit;
  **/
 public class DorisTableInputSplit implements InputSplit, java.io.Serializable {
 
+private static final long serialVersionUID = 1L;
 /**
  * The number of the split.
  */

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For add

[GitHub] [incubator-doris-flink-connector] hf200012 merged pull request #22: [Bug] Inconsistent id when deserializing

[GitBox]

hf200012 merged pull request #22:
URL: https://github.com/apache/incubator-doris-flink-connector/pull/22


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris-flink-connector] branch master updated: Update README.md (#19)

[jiafengzheng]

This is an automated email from the ASF dual-hosted git repository.

jiafengzheng pushed a commit to branch master
in repository 
https://gitbox.apache.org/repos/asf/incubator-doris-flink-connector.git


The following commit(s) were added to refs/heads/master by this push:
 new 03ea974  Update README.md (#19)
03ea974 is described below

commit 03ea974e1690e4702e9616be69cf8cabc1d2ba83
Author: BAKEZQ 
AuthorDate: Fri Mar 25 12:12:46 2022 +0800

Update README.md (#19)

* Update README.md
---
 README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/README.md b/README.md
index c7ef5a8..078805a 100644
--- a/README.md
+++ b/README.md
@@ -24,7 +24,7 @@ under the License.
 
 ## Flink Doris Connector
 
-Flink Doris Connector now support flink version from 0.11 to 0.14.
+Flink Doris Connector now support flink version from 1.11 to 1.14.
 
 If you wish to contribute or use a connector from flink 1.13 (and earlier), 
please use the 
[branch-for-flink-before-1.13](https://github.com/apache/incubator-doris-flink-connector/tree/branch-for-flink-before-1.13)
 

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris-flink-connector] hf200012 merged pull request #19: Update README.md

[GitBox]

hf200012 merged pull request #19:
URL: https://github.com/apache/incubator-doris-flink-connector/pull/19


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] yangzhg closed pull request #8579: [deps] remove mysql-server dependency use mariadb-connector-c instead

[GitBox]

yangzhg closed pull request #8579:
URL: https://github.com/apache/incubator-doris/pull/8579


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] wangbo commented on pull request #8627: [refactor][optimize] Code optimization and refactoring for low-cardinality columns in storage layer

[GitBox]

wangbo commented on pull request #8627:
URL: https://github.com/apache/incubator-doris/pull/8627#issuecomment-1078638500


   Is there a  performance drop after code refactoring?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] zbtzbtzbt opened a new pull request #8655: [Doc] update doc of vec-execution-engine

[GitBox]

zbtzbtzbt opened a new pull request #8655:
URL: https://github.com/apache/incubator-doris/pull/8655


   # Proposed changes
   
   Issue Number: close #xxx
   
   @HappenLee 
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] spaces-X opened a new issue #8656: [Enhancement] [Vectorized] optimize pre-agg in stream load memtable

[GitBox]

spaces-X opened a new issue #8656:
URL: https://github.com/apache/incubator-doris/issues/8656


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Description
   
   The current version( #8561 ) of vectorized stream load is using skip list to 
aggregate values, which is a row-structured constant ordered data result.
   
   Considering that we just do pre-aggregation in memtable and queries will not 
go to memtable, this implementation of maintaining constant order is too 
expensive.
   
   I plan to refactor this part of the code and replace the existing skip list 
with other solutions based on pr #8561. 
   
   
   
   
   ### Solution
   
   Solution 1:
 First,  sort of incoming-data-block.
 Then, merge the sorted-data-block.
 Then, append merged-data-block to final-block.
 At last, do a finalize(sort + merge) of the final-block to flush.
   
   Solution 2:
 First, aggregate the incoming-data-block by hash table.
 At last, sort the whole aggregated-block.
   
   
   
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] wangbo commented on a change in pull request #8627: [refactor][optimize] Code optimization and refactoring for low-cardinality columns in storage layer

[GitBox]

wangbo commented on a change in pull request #8627:
URL: https://github.com/apache/incubator-doris/pull/8627#discussion_r834945195



##
File path: be/src/olap/rowset/segment_v2/segment_iterator.cpp
##
@@ -830,23 +831,17 @@ void 
SegmentIterator::_evaluate_short_circuit_predicate(uint16_t* vec_sel_rowid_
 return;
 }
 
-for (auto column_predicate : _short_cir_eval_predicate) {
-auto column_id = column_predicate->column_id();
+for (auto predicate : _short_cir_eval_predicate) {
+auto column_id = predicate->column_id();
 auto& short_cir_column = _current_return_columns[column_id];
 auto* col_ptr = short_cir_column.get();
-// todo(zeno) define convert_dict_codes_if_dictionary interface in 
IColumn
-if (short_cir_column->is_nullable()) {
-auto nullable_col =
-
reinterpret_cast(short_cir_column.get());
-col_ptr = nullable_col->get_nested_column_ptr().get();
+// range comparison predicate needs to sort the dict and convert the 
encoding
+if (predicate->type() == PredicateType::LT || predicate->type() == 
PredicateType::LE ||
+predicate->type() == PredicateType::GT || predicate->type() == 
PredicateType::GE) {
+col_ptr->convert_dict_codes_if_necessary();
 }
-
-if (col_ptr->is_column_dictionary() && 
column_predicate->is_range_comparison_predicate()) {
-auto& dict_col =
-
reinterpret_cast&>(*col_ptr);
-dict_col.convert_dict_codes();
-}
-column_predicate->evaluate(*short_cir_column, vec_sel_rowid_idx, 
selected_size_ptr);
+col_ptr->set_predicate_dict_code_if_necessary(predicate);

Review comment:
   First, It seems that you do ```set_predicate_dict_code_if_necessary``` 
after read one block.
   Maybe we can do ```set_predicate_dict_code_if_necessary``` in SegmentIter's 
init method.
   Because I think dict is immutable after segment is loaded, and the same to 
Predicate.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org