[incubator-doris-website] branch master updated: 修改文档错误别字

[jiafengzheng]

This is an automated email from the ASF dual-hosted git repository.

jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git


The following commit(s) were added to refs/heads/master by this push:
 new 4970454  修改文档错误别字
 new c7826fe  Merge pull request #8 from coderjiang/bugfix/flink_cdc_doc
4970454 is described below

commit 4970454c305889025e9b990608773214517b87e1
Author: coderjiang 
AuthorDate: Mon Feb 7 10:08:03 2022 +0800

修改文档错误别字
---
 blogs/zh-CN/flink-cdc-to-doris.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/blogs/zh-CN/flink-cdc-to-doris.md 
b/blogs/zh-CN/flink-cdc-to-doris.md
index d6627ea..f5828fc 100644
--- a/blogs/zh-CN/flink-cdc-to-doris.md
+++ b/blogs/zh-CN/flink-cdc-to-doris.md
@@ -90,9 +90,9 @@ Flink Doris Connector 是 doris 社区为了方便用户使用 Flink 读写Doris
 目前Flink doris connector目前控制入库通过两个参数：
 
 1. sink.batch.size ：每多少条写入一次，默认100条
-2. sink.batch.interval ：每个多少秒写入一下，默认1秒
+2. sink.batch.interval ：每隔多少秒写入一下，默认1秒
 
-这两参数同时起作用，那个条件先到就触发写doris表操作，
+这两参数同时起作用，哪个条件先到就触发写doris表操作，
 
 **注意：**
 

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris-flink-connector] hf200012 edited a comment on pull request #13: Feature/20220305 support flush when batch bytes reach threshold

[GitBox]

hf200012 edited a comment on pull request #13:
URL: 
https://github.com/apache/incubator-doris-flink-connector/pull/13#issuecomment-1059714657


   Update documentation configuration parameter description @bridgeDream 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] hf200012 merged pull request #8346: [docs]modify docs

[GitBox]

hf200012 merged pull request #8346:
URL: https://github.com/apache/incubator-doris/pull/8346


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris] branch master updated: format fe config title , add link for tablet_rebalancer_type (#8346)

[jiafengzheng]

This is an automated email from the ASF dual-hosted git repository.

jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 2519f8b  format fe config title , add link for tablet_rebalancer_type 
(#8346)
2519f8b is described below

commit 2519f8b9a437ddafff2810b035f4adb1bcd9ad63
Author: kylinmac 
AuthorDate: Sun Mar 6 16:54:26 2022 +0800

format fe config title , add link for tablet_rebalancer_type (#8346)

* format fe config title , add link for tablet_rebalancer_type (#8346)
---
 docs/zh-CN/administrator-guide/config/fe_config.md | 636 ++---
 .../operation/tablet-repair-and-balance.md |   2 +-
 2 files changed, 319 insertions(+), 319 deletions(-)

diff --git a/docs/zh-CN/administrator-guide/config/fe_config.md 
b/docs/zh-CN/administrator-guide/config/fe_config.md
index 6b9b123..35c5471 100644
--- a/docs/zh-CN/administrator-guide/config/fe_config.md
+++ b/docs/zh-CN/administrator-guide/config/fe_config.md
@@ -121,7 +121,7 @@ FE 的配置项有两种方式进行配置：
 
 ## 配置项列表
 
-### max_dynamic_partition_num
+### `max_dynamic_partition_num`
 
 默认值：500
 
@@ -129,32 +129,32 @@ FE 的配置项有两种方式进行配置：
 
 是否为 Master FE 节点独有的配置项：true
 
-用于限制创建动态分区表时可以创建的最大分区数，避免一次创建过多分区。 数量由动态分区参数中的“开始”和“结束”决定。 
+用于限制创建动态分区表时可以创建的最大分区数，避免一次创建过多分区。 数量由动态分区参数中的“开始”和“结束”决定。
 
-### grpc_max_message_size_bytes
+### `grpc_max_message_size_bytes`
 
 默认值：1G
 
 用于设置 GRPC 客户端通道的初始流窗口大小，也用于设置最大消息大小。当结果集较大时，可能需要增大该值。
 
-### min_replication_num_per_tablet
+### `min_replication_num_per_tablet`
 
 默认值：1
 
 用于设置单个tablet的最小replication数量。
 
-### max_replication_num_per_tablet
+### `max_replication_num_per_tablet`
 
 默认值：32767
 
 用于设置单个tablet的最大replication数量。
 
-### enable_outfile_to_local
+### `enable_outfile_to_local`
 
 默认值：false
 是否允许outfile函数将结果导出到本地磁盘
 
-### enable_access_file_without_broker
+### `enable_access_file_without_broker`
 
 默认值：false
 
@@ -164,33 +164,33 @@ FE 的配置项有两种方式进行配置：
 
 此配置用于在通过代理访问 bos 或其他云存储时尝试跳过代理
 
-### enable_bdbje_debug_mode
+### `enable_bdbje_debug_mode`
 
 默认值：false
 
 如果设置为 true，FE 将在 BDBJE 调试模式下启动，在 Web 页面 `System->bdbje` 可以查看相关信息，否则不可以查看
 
-### enable_alpha_rowset
+### `enable_alpha_rowset`
 
 默认值：false
 
 是否支持创建 alpha rowset。默认为 false，只应在紧急情况下使用，此配置应在未来的某个版本中删除
 
-### enable_http_server_v2
+### `enable_http_server_v2`
 
 默认值：从官方 0.14.0 release 版之后默认是 true，之前默认 false
 
 HTTP Server V2 由 SpringBoot 实现。它采用前后端分离的架构。只有启用 httpv2 才能用户使用新的前端 UI 界面
 
-### jetty_server_acceptors
+### `jetty_server_acceptors`
 
 默认值：2
 
-### jetty_server_selectors
+### `jetty_server_selectors`
 
 默认值：4
 
-### jetty_server_workers
+### `jetty_server_workers`
 
 默认值：0
 
@@ -200,13 +200,13 @@ HTTP Server V2 由 SpringBoot 实现。它采用前后端分离的架构。只
 
 workers 线程池默认不做设置，根据自己需要进行设置
 
-### jetty_server_max_http_post_size
+### `jetty_server_max_http_post_size`
 
 默认值：100 * 1024 * 1024  （100MB）
 
 这个是put或post方法上传文件的最大字节数，默认值：100MB
 
-### default_max_filter_ratio
+### `default_max_filter_ratio`
 
 默认值：0
 
@@ -216,7 +216,7 @@ workers 线程池默认不做设置，根据自己需要进行设置
 
 可过滤数据的最大百分比（由于数据不规则等原因）默认值为0。表示严格模式，只要数据有一条被过滤掉整个导入失败
 
-### default_db_data_quota_bytes
+### `default_db_data_quota_bytes`
 
 默认值：1PB
 
@@ -233,7 +233,7 @@ ALTER DATABASE db_name SET DATA QUOTA quota;
 show data （其他用法：HELP SHOW DATA）
 ```
 
-### default_db_replica_quota_size
+### `default_db_replica_quota_size`
 
 默认值：1073741824
 
@@ -250,7 +250,7 @@ ALTER DATABASE db_name SET REPLICA QUOTA quota;
 show data （其他用法：HELP SHOW DATA）
 ```
 
-### enable_batch_delete_by_default
+### `enable_batch_delete_by_default`
 
 默认值：false
 
@@ -260,7 +260,7 @@ show data （其他用法：HELP SHOW DATA）
 
 创建唯一表时是否添加删除标志列，具体原理参照官方文档：操作手册->数据导入->批量删除
 
-### recover_with_empty_tablet
+### `recover_with_empty_tablet`
 
 默认值：false
 
@@ -270,7 +270,7 @@ show data （其他用法：HELP SHOW DATA）
 
 在某些情况下，某些 tablet 可能会损坏或丢失所有副本。 此时数据已经丢失，损坏的tablet会导致整个查询失败，无法查询剩余的健康 tablet。 
在这种情况下，您可以将此配置设置为 true。 系统会将损坏的药片替换为空药片，以确保查询可以执行。 （但此时数据已经丢失，所以查询结果可能不准确）
 
-### max_allowed_in_element_num_of_delete
+### `max_allowed_in_element_num_of_delete`
 
 默认值：1024
 
@@ -280,7 +280,7 @@ show data （其他用法：HELP SHOW DATA）
 
 用于限制 delete 语句中 Predicate 的元素个数
 
-### cache_result_max_row_count
+### `cache_result_max_row_count`
 
 默认值：3000
 
@@ -290,7 +290,7 @@ show data （其他用法：HELP SHOW DATA）
 
 设置可以缓存的最大行数，详细的原理可以参考官方文档：操作手册->分区缓存
 
-### cache_last_version_interval_second
+### `cache_last_version_interval_second`
 
 默认值：900
 
@@ -300,7 +300,7 @@ show data （其他用法：HELP SHOW DATA）
 
 缓存结果时上一版本的最小间隔，该参数区分离线更新和实时更新
 
-### cache_enable_partition_mode
+### `cache_enable_partition_mode`
 
 默认值：true
 
@@ -310,7 +310,7 @@ show data （其他用法：HELP SHOW DATA）
 
 如果设置为 true，fe 将从 be cache 中获取数据，该选项适用于部分分区的实时更新。
 
-### cache_enable_sql_mode
+### `cache_enable_sql_mode`
 
 默认值：true
 
@@ -325,7 +325,7 @@ show data （其他用法：HELP SHOW DATA）
 | enable_sql_cache   | false | true  | true  | false |
 | enable_partition_cache | false | false | true  | tru

[GitHub] [incubator-doris] hf200012 merged pull request #8343: [docs]update http port doc to be more intuitive

[GitBox]

hf200012 merged pull request #8343:
URL: https://github.com/apache/incubator-doris/pull/8343


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris] branch master updated (2519f8b -> 1879623)

[jiafengzheng]

This is an automated email from the ASF dual-hosted git repository.

jiafengzheng pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git.


from 2519f8b  format fe config title , add link for tablet_rebalancer_type 
(#8346)
 add 1879623  [docs]update http port doc to be more intuitive (#8343)

No new revisions were added by this update.

Summary of changes:
 docs/en/installing/install-deploy.md| 36 ++---
 docs/zh-CN/installing/install-deploy.md | 24 +++---
 2 files changed, 32 insertions(+), 28 deletions(-)

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] stalary opened a new issue #8353: [Feature] Supports row-level permissions

[GitBox]

stalary opened a new issue #8353:
URL: https://github.com/apache/incubator-doris/issues/8353


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Description
   
   Restrict certain users to seeing certain data
   
   ### Use case
   
   data id1=1~100, id2=2 or 5
   userA can select id1=1~50
   userB can select id1=50~100 and id2=2
   userC can select id2=5
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8352: [improvement]support build with parallel parameter only

[GitBox]

github-actions[bot] commented on pull request #8352:
URL: https://github.com/apache/incubator-doris/pull/8352#issuecomment-1059925774






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] morningman commented on a change in pull request #8322: [refactor] Impl of MemTracker, and related use

[GitBox]

morningman commented on a change in pull request #8322:
URL: https://github.com/apache/incubator-doris/pull/8322#discussion_r820197198



##
File path: be/src/exprs/expr_context.cpp
##
@@ -371,10 +376,11 @@ Status ExprContext::get_const_value(RuntimeState* state, 
Expr& expr, AnyVal** co
 StringVal* sv = reinterpret_cast(*const_val);
 if (!sv->is_null && sv->len > 0) {
 // Make sure the memory is owned by this evaluator.
-char* ptr_copy = 
reinterpret_cast(_pool->try_allocate(sv->len));
+Status rst;
+char* ptr_copy = 
reinterpret_cast(_pool->try_allocate(sv->len, &rst));
 if (ptr_copy == nullptr) {
-return _pool->mem_tracker()->MemLimitExceeded(
-state, "Could not allocate constant string value", 
sv->len);
+RETURN_ALLOC_LIMIT_EXCEEDED(_pool->mem_tracker(), state,

Review comment:
   what is the difference between `RETURN_LIMIT_EXCEEDED` and 
`RETURN_ALLOC_LIMIT_EXCEEDED`

##
File path: be/src/exec/es/es_scroll_parser.cpp
##
@@ -410,11 +411,12 @@ Status ScrollParser::fill_tuple(const TupleDescriptor* 
tuple_desc, Tuple* tuple,
 }
 }
 size_t val_size = val.length();
-char* buffer = 
reinterpret_cast(tuple_pool->try_allocate_unaligned(val_size));
+Status rst;
+char* buffer = 
reinterpret_cast(tuple_pool->try_allocate_unaligned(val_size, &rst));

Review comment:
   why not return Status and use `buffer` as parameter?

##
File path: be/src/olap/task/engine_alter_tablet_task.cpp
##
@@ -18,13 +18,21 @@
 #include "olap/task/engine_alter_tablet_task.h"
 
 #include "olap/schema_change.h"
+#include "runtime/mem_tracker.h"
 
 namespace doris {
 
 using std::to_string;
 
 EngineAlterTabletTask::EngineAlterTabletTask(const TAlterTabletReqV2& request)
-: _alter_tablet_req(request) {}
+: _alter_tablet_req(request) {
+_mem_tracker = MemTracker::create_tracker(

Review comment:
   This memtracker is not used

##
File path: be/src/olap/task/engine_batch_load_task.h
##
@@ -76,6 +76,8 @@ class EngineBatchLoadTask : public EngineTask {
 AgentStatus* _res_status;
 std::string _remote_file_path;
 std::string _local_file_path;
+
+std::shared_ptr _mem_tracker;

Review comment:
   not used?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris-manager] branch master updated: [Bug-Fix][Manager] fix swapping space name and user name when creating space bug. (#2)

[jiafengzheng]

This is an automated email from the ASF dual-hosted git repository.

jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris-manager.git


The following commit(s) were added to refs/heads/master by this push:
 new 1dd72e0  [Bug-Fix][Manager] fix swapping space name and user name when 
creating space bug. (#2)
1dd72e0 is described below

commit 1dd72e007ae1167afe5e8b4620b7616124e3b20d
Author: Hao Wang <1727386...@qq.com>
AuthorDate: Sun Mar 6 17:30:00 2022 +0800

[Bug-Fix][Manager] fix swapping space name and user name when creating 
space bug. (#2)

fix swapping space name and user name when creating space bug. (#2)
---
 .../src/routes/super-admin/space/new-space/new-space.tsx  | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git 
a/manager/doris-manager/src/routes/super-admin/space/new-space/new-space.tsx 
b/manager/doris-manager/src/routes/super-admin/space/new-space/new-space.tsx
index 5cfdb1c..789f6d5 100644
--- a/manager/doris-manager/src/routes/super-admin/space/new-space/new-space.tsx
+++ b/manager/doris-manager/src/routes/super-admin/space/new-space/new-space.tsx
@@ -45,11 +45,11 @@ const SpaceNew = () => {
 queryPort: cluValues.queryPort,
 user: cluValues.user,
 },
-name: values.userName,
+name: values.name,
 describe: values.describe,
 user: {
 email: values.email,
-name: values.name,
+name: values.userName,
 password: values.password,
 },
 }).then(res => {

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris-manager] hf200012 merged pull request #2: [Bug-Fix][Manager] fix swapping space name and user name bug.

[GitBox]

hf200012 merged pull request #2:
URL: https://github.com/apache/incubator-doris-manager/pull/2


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] stalary closed issue #8353: [Feature] Supports row-level permissions

[GitBox]

stalary closed issue #8353:
URL: https://github.com/apache/incubator-doris/issues/8353


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] stalary commented on issue #7540: 行过滤和列脱敏[Feature]

[GitBox]

stalary commented on issue #7540:
URL: 
https://github.com/apache/incubator-doris/issues/7540#issuecomment-1059931541


   I think we should refer to pg's design and support both OR and AND by 
configuration, while ignoring `where` in configuration


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] xinyiZzz commented on a change in pull request #8322: [refactor] Impl of MemTracker, and related use

[GitBox]

xinyiZzz commented on a change in pull request #8322:
URL: https://github.com/apache/incubator-doris/pull/8322#discussion_r820210300



##
File path: be/src/exec/hash_table.cpp
##
@@ -244,7 +244,7 @@ void HashTable::grow_node_array() {
 _alloc_list.push_back(_current_nodes);
 _end_list.push_back(_current_nodes + alloc_size);
 
-_mem_tracker->Consume(alloc_size);
+_mem_tracker->consume(alloc_size);

Review comment:
   > There are too many places call memory tracker and we may lost to add 
mem_tracker.consume in some node or thread. If we are following clickhouse's 
method, I think it do not need to call memory tracker everywhere and call mem 
tracker.consume so many times.
   > 
   > We could attach the running thread to a conext, the context maybe a query 
context or a compaction context or olap scanner. And rewrite the new and delete 
method to update the memory tracker automatically.
   
   You are right, I have already implemented it like this. In #7198 mentioned 
above, 
   you can see the original design document last year: 
https://shimo.im/docs/DT6JXDRkdTvdyV3G




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] xinyiZzz commented on a change in pull request #8322: [refactor] Impl of MemTracker, and related use

[GitBox]

xinyiZzz commented on a change in pull request #8322:
URL: https://github.com/apache/incubator-doris/pull/8322#discussion_r820210536



##
File path: be/src/exec/es/es_scroll_parser.cpp
##
@@ -410,11 +411,12 @@ Status ScrollParser::fill_tuple(const TupleDescriptor* 
tuple_desc, Tuple* tuple,
 }
 }
 size_t val_size = val.length();
-char* buffer = 
reinterpret_cast(tuple_pool->try_allocate_unaligned(val_size));
+Status rst;
+char* buffer = 
reinterpret_cast(tuple_pool->try_allocate_unaligned(val_size, &rst));

Review comment:
   is to be consistent with the previous interface,
   It seems that what you said is more common, I will change it.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] xinyiZzz commented on a change in pull request #8322: [refactor] Impl of MemTracker, and related use

[GitBox]

xinyiZzz commented on a change in pull request #8322:
URL: https://github.com/apache/incubator-doris/pull/8322#discussion_r820212845



##
File path: be/src/exprs/expr_context.cpp
##
@@ -371,10 +376,11 @@ Status ExprContext::get_const_value(RuntimeState* state, 
Expr& expr, AnyVal** co
 StringVal* sv = reinterpret_cast(*const_val);
 if (!sv->is_null && sv->len > 0) {
 // Make sure the memory is owned by this evaluator.
-char* ptr_copy = 
reinterpret_cast(_pool->try_allocate(sv->len));
+Status rst;
+char* ptr_copy = 
reinterpret_cast(_pool->try_allocate(sv->len, &rst));
 if (ptr_copy == nullptr) {
-return _pool->mem_tracker()->MemLimitExceeded(
-state, "Could not allocate constant string value", 
sv->len);
+RETURN_ALLOC_LIMIT_EXCEEDED(_pool->mem_tracker(), state,

Review comment:
   The difference is the number of parameters, which is not very well named.
   I try to unify with variadic macros.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] xinyiZzz commented on a change in pull request #8322: [refactor] Impl of MemTracker, and related use

[GitBox]

xinyiZzz commented on a change in pull request #8322:
URL: https://github.com/apache/incubator-doris/pull/8322#discussion_r820212977



##
File path: be/src/olap/task/engine_alter_tablet_task.cpp
##
@@ -18,13 +18,21 @@
 #include "olap/task/engine_alter_tablet_task.h"
 
 #include "olap/schema_change.h"
+#include "runtime/mem_tracker.h"
 
 namespace doris {
 
 using std::to_string;
 
 EngineAlterTabletTask::EngineAlterTabletTask(const TAlterTabletReqV2& request)
-: _alter_tablet_req(request) {}
+: _alter_tablet_req(request) {
+_mem_tracker = MemTracker::create_tracker(

Review comment:
   Yes, the mem_tracker in xxx_task will be used in Thread Local after 
waiting.
   Do you think you should delete it first?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] xinyiZzz commented on a change in pull request #8322: [refactor] Impl of MemTracker, and related use

[GitBox]

xinyiZzz commented on a change in pull request #8322:
URL: https://github.com/apache/incubator-doris/pull/8322#discussion_r820210536



##
File path: be/src/exec/es/es_scroll_parser.cpp
##
@@ -410,11 +411,12 @@ Status ScrollParser::fill_tuple(const TupleDescriptor* 
tuple_desc, Tuple* tuple,
 }
 }
 size_t val_size = val.length();
-char* buffer = 
reinterpret_cast(tuple_pool->try_allocate_unaligned(val_size));
+Status rst;
+char* buffer = 
reinterpret_cast(tuple_pool->try_allocate_unaligned(val_size, &rst));

Review comment:
   is to be compatible with the previous interface,
   It seems that what you said is more common, I will change it.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris-spark-connector] branch master updated: Docs: Change http to https (#11)

[jiafengzheng]

This is an automated email from the ASF dual-hosted git repository.

jiafengzheng pushed a commit to branch master
in repository 
https://gitbox.apache.org/repos/asf/incubator-doris-spark-connector.git


The following commit(s) were added to refs/heads/master by this push:
 new 869cbcc  Docs: Change http to https (#11)
869cbcc is described below

commit 869cbcc5ce193969760b7ee41e3dc61a14d9edf3
Author: Cheng Pan <379377...@qq.com>
AuthorDate: Sun Mar 6 22:00:44 2022 +0800

Docs: Change http to https (#11)

Change http to https (#11)
---
 README.md | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/README.md b/README.md
index f4e9246..8d9b82e 100644
--- a/README.md
+++ b/README.md
@@ -24,11 +24,11 @@ under the License.
 
 ### Spark Doris Connector
 
-More information about compilation and usage, please visit [Spark Doris 
Connector](http://doris.apache.org/extending-doris/spark-doris-connector.html)
+More information about compilation and usage, please visit [Spark Doris 
Connector](https://doris.apache.org/extending-doris/spark-doris-connector.html)
 
 ## License
 
-[Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0)
+[Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0)
 
 ## Report issues or submit pull request
 
@@ -40,10 +40,10 @@ Contact us through the following mailing list.
 
 | Name 
 | Scope   |
 |  
   |
  |
 
|:--|:|:|:|:-|
-| [d...@doris.apache.org](mailto:d...@doris.apache.org) | 
Development-related discussions | 
[Subscribe](mailto:dev-subscr...@doris.apache.org)   | 
[Unsubscribe](mailto:dev-unsubscr...@doris.apache.org)   | 
[Archives](http://mail-archives.apache.org/mod_mbox/doris-dev/)   |
+| [d...@doris.apache.org](mailto:d...@doris.apache.org) | 
Development-related discussions | 
[Subscribe](mailto:dev-subscr...@doris.apache.org)   | 
[Unsubscribe](mailto:dev-unsubscr...@doris.apache.org)   | 
[Archives](https://mail-archives.apache.org/mod_mbox/doris-dev/)   |
 
 ## Links
 
-* Doris official site - 
+* Doris official site - 
 * Developer Mailing list - . Mail to 
, follow the reply to subscribe the mail list.
 * Slack channel - [Join the 
Slack](https://join.slack.com/t/apachedoriscommunity/shared_invite/zt-11jb8gesh-7IukzSrdea6mqoG0HB4gZg)

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris-spark-connector] hf200012 merged pull request #11: Docs: Change http to https

[GitBox]

hf200012 merged pull request #11:
URL: https://github.com/apache/incubator-doris-spark-connector/pull/11


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] stalary commented on issue #7540: 行过滤和列脱敏[Feature]

[GitBox]

stalary commented on issue #7540:
URL: 
https://github.com/apache/incubator-doris/issues/7540#issuecomment-1059969632


   I'll implement it this way `grant row_priv  on db.tbl AS RESTRICTIVE to 
'root' USING(a = xxx)`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] morningman opened a new issue #8354: [Bug]

[GitBox]

morningman opened a new issue #8354:
URL: https://github.com/apache/incubator-doris/issues/8354


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Version
   
   trunk
   
   ### What's Wrong?
   
   When starting FE, error in fe.out:
   
   ```
   java.io.IOException: failed to read session variable: java.lang.Long cannot 
be cast to java.lang.Integer
   at 
org.apache.doris.qe.SessionVariable.readFromJson(SessionVariable.java:1025)
   at 
org.apache.doris.qe.SessionVariable.readFields(SessionVariable.java:981)
   at org.apache.doris.qe.VariableMgr.read(VariableMgr.java:312)
   at 
org.apache.doris.catalog.Catalog.loadGlobalVariable(Catalog.java:1902)
   at org.apache.doris.common.MetaReader.read(MetaReader.java:89)
   at org.apache.doris.catalog.Catalog.loadImage(Catalog.java:1618)
   at org.apache.doris.catalog.Catalog.initialize(Catalog.java:848)
   at org.apache.doris.PaloFe.start(PaloFe.java:125)
   at org.apache.doris.PaloFe.main(PaloFe.java:63)
   ```
   
   ### What You Expected?
   
   works well
   
   ### How to Reproduce?
   
   This bug is introduced from #8319
   
   ### Anything Else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] morningman opened a new pull request #8355: [improvement][fix](insert)(replay) support SHOW LAST INSERT stmt and fix json replay bug

[GitBox]

morningman opened a new pull request #8355:
URL: https://github.com/apache/incubator-doris/pull/8355


   # Proposed changes
   
   Issue Number: close #8354
   
   ## Problem Summary:
   
   1. support SHOW LAST INSERT
   In the current implementation, the insert operation returns a json 
string to describe the result information
   of the insert. But this information is in the session track field of the 
mysql protocol,
   and it is difficult to obtain programmatically.
   
   Therefore, I provide a new syntax `show last insert` to explicitly 
obtain the result of the latest insert operation,
   and return a normal query result set to facilitate the user to obtain 
the result information of the insert.
   
   2. Fix a bug described in #8354 
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (No)
   3. Has unit tests been added: (No Need)
   4. Has document been added or modified: (Yes)
   5. Does it need to update dependencies: (No)
   6. Are there any changes that cannot be rolled back: (No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] stalary removed a comment on issue #7540: 行过滤和列脱敏[Feature]

[GitBox]

stalary removed a comment on issue #7540:
URL: 
https://github.com/apache/incubator-doris/issues/7540#issuecomment-1059969632


   I'll implement it this way `grant row_priv  on db.tbl AS RESTRICTIVE to 
'root' USING(a = xxx)`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] stalary commented on issue #7540: 行过滤和列脱敏[Feature]

[GitBox]

stalary commented on issue #7540:
URL: 
https://github.com/apache/incubator-doris/issues/7540#issuecomment-1060069844


   It seems that we had better devise a separate grammar，reference
   gaussDB：https://support.huaweicloud.com/devg2-dws/dws_0402_0041.html
   
clickhouse：https://www.bookstack.cn/read/clickhouse-21.9-zh/90412fd4e39f7333.md
   postgres：http://www.postgres.cn/docs/10/ddl-rowsecurity.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] emmataobao commented on issue #8331: [Bug] doris on es 中 连接es ip问题

[GitBox]

emmataobao commented on issue #8331:
URL: 
https://github.com/apache/incubator-doris/issues/8331#issuecomment-1060109317


   > 这个报错是refused不是timed out，所以先确认下172.17.0.2上9200是否打开了并且是否防火墙允许访问？
   
   我在配置上的ip是 "hosts" = "http://192.168.5.242:9200"; , 但是doris 连接的时候将ip转成docker 
ip172.17.0.2 
，为什么出现refused，是因为doris测试节点上有其他容器的ip是172.17.0.2，所以出现了refused。我现在的困惑是我的配置esip是192，结果连接转成172，导致连接不上目标es，我的es节点是防火墙端口开放的，我已经配置了
 "nodes_discovery" = "false"。


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] hf200012 merged pull request #8330: [improvement]support doriswriter build in macos

[GitBox]

hf200012 merged pull request #8330:
URL: https://github.com/apache/incubator-doris/pull/8330


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris] branch master updated: support doriswriter build in macos (#8330)

[jiafengzheng]

This is an automated email from the ASF dual-hosted git repository.

jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 3b159a9  support doriswriter build in macos (#8330)
3b159a9 is described below

commit 3b159a98202a94599e659c9fa0b2ba4d568c52b4
Author: wunan1210 
AuthorDate: Mon Mar 7 09:53:16 2022 +0800

support doriswriter build in macos (#8330)

support doriswriter build in macos (#8330)
---
 extension/DataX/init-env.sh | 34 --
 1 file changed, 28 insertions(+), 6 deletions(-)

diff --git a/extension/DataX/init-env.sh b/extension/DataX/init-env.sh
index b43756f..83eeb8e 100755
--- a/extension/DataX/init-env.sh
+++ b/extension/DataX/init-env.sh
@@ -35,6 +35,15 @@ DATAX_POM=$DATAX_EXTENSION_HOME/DataX/pom.xml
 DATAX_PACKAGE=$DATAX_EXTENSION_HOME/DataX/package.xml
 DATAX_CORE_POM=$DATAX_EXTENSION_HOME/DataX/core/pom.xml
 
+
+USE_MACOS=0
+# check OS type
+if [[ ! -z "$OSTYPE" ]]; then
+if [[ ${OSTYPE:0:6} == "darwin" ]]; then
+USE_MACOS=1
+fi
+fi
+
 if [ ! -d $DATAX_GIT_DIR ]; then
 echo "Clone DataX from $DATAX_GITHUB"
 git clone $DATAX_GITHUB $DATAX_GIT_DIR
@@ -50,24 +59,37 @@ fi
 
 if [ `grep -c "doriswriter" $DATAX_POM` -eq 0 ]; then
 echo "No doriswriter module in $DATAX_POM, add it"
-cp $DATAX_POM ${DATAX_POM}.orig
-sed -i "s/<\/modules>/doriswriter<\/module>\n
<\/modules>/g"  $DATAX_POM 
+if [[ USE_MACOS -eq 0 ]];then
+cp $DATAX_POM ${DATAX_POM}.orig
+sed -i "s/<\/modules>/doriswriter<\/module>\n
<\/modules>/g"  $DATAX_POM
+else
+sed -i '.orig' "s/<\/modules>/doriswriter<\/module>\n
<\/modules>/g"  $DATAX_POM
+fi
 else
 echo "doriswriter module exists in $DATAX_POM"  
 fi
 
 if [ `grep -c "doriswriter" $DATAX_PACKAGE` -eq 0 ]; then
 echo "No doriswriter module in $DATAX_PACKAGE, add it"
-cp $DATAX_PACKAGE ${DATAX_PACKAGE}.orig
-sed -i "s/<\/fileSets>/\n
doriswriter\/target\/datax\/<\/directory>\n\n  
  **\/*.*<\/include>\n<\/includes>\n   
 datax<\/outputDirectory>\n<\/fileSet>\n
<\/fileSets>/g"  $DATAX_PACKAGE
+if [[ USE_MACOS -eq 0 ]];then
+cp $DATAX_PACKAGE ${DATAX_PACKAGE}.orig
+sed -i "s/<\/fileSets>/\n
doriswriter\/target\/datax\/<\/directory>\n\n  
  **\/*.*<\/include>\n<\/includes>\n   
 datax<\/outputDirectory>\n<\/fileSet>\n
<\/fileSets>/g"  $DATAX_PACKAGE
+else
+sed -i '.orig' "s/<\/fileSets>/\n
doriswriter\/target\/datax\/<\/directory>\n\n  
  **\/*.*<\/include>\n<\/includes>\n   
 datax<\/outputDirectory>\n<\/fileSet>\n
<\/fileSets>/g"  $DATAX_PACKAGE
+fi
 else
 echo "doriswriter module exists in $DATAX_PACKAGE"  
 fi
 
 if [ `grep -c "4.5.13" $DATAX_CORE_POM` -eq 0 ]; then
 echo "No httpclient 4.5.13 in $DATAX_CORE_POM, add it"
-cp $DATAX_CORE_POM ${DATAX_CORE_POM}.orig
-sed -i ":a;N;s/httpclient<\/artifactId>\n
4.5<\/version>/httpclient<\/artifactId>\n
4.5.13<\/version>/g" $DATAX_CORE_POM
+if [[ USE_MACOS -eq 0 ]];then
+cp $DATAX_CORE_POM ${DATAX_CORE_POM}.orig
+sed -i ":a;N;s/httpclient<\/artifactId>\n
4.5<\/version>/httpclient<\/artifactId>\n
4.5.13<\/version>/g" $DATAX_CORE_POM
+else
+sed -i '.orig' -e "N" -e "s/httpclient<\/artifactId>\n 
   4.5<\/version>/httpclient<\/artifactId>\n   
 4.5.13<\/version>/g" $DATAX_CORE_POM
+fi
+
 else
 echo "httpclient 4.5.13 exists in $DATAX_CORE_POM"  
 fi

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] emmataobao commented on issue #8331: [Bug] doris on es 中 连接es ip问题

[GitBox]

emmataobao commented on issue #8331:
URL: 
https://github.com/apache/incubator-doris/issues/8331#issuecomment-1060110295


   
如果我的es集群部署方式是物理机部署，或者be安装在es部署节点上，则不会出现上述问题。如果我的es集群为docker部署，且宿主机无be部署则会出现上述问题。
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] zhaomin1423 opened a new issue #8356: [Bug] broker load can't set not to import a part of columns when the file format is parquet

[GitBox]

zhaomin1423 opened a new issue #8356:
URL: https://github.com/apache/incubator-doris/issues/8356


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Version
   
   0.14.13
   
   ### What's Wrong?
   
   broker load can't set not to import a part of columns when the file format 
is parquet.
   
   ### What You Expected?
   
   The columns that be add set conf only can loaded.
   
   ### How to Reproduce?
   
   _No response_
   
   ### Anything Else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] zhangstar333 commented on a change in pull request #8299: [feature] Window funnel

[GitBox]

zhangstar333 commented on a change in pull request #8299:
URL: https://github.com/apache/incubator-doris/pull/8299#discussion_r820330598



##
File path: be/src/vec/aggregate_functions/aggregate_function_window_funnel.h
##
@@ -0,0 +1,210 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "common/logging.h"
+#include "vec/aggregate_functions/aggregate_function.h"
+#include "vec/columns/columns_number.h"
+#include "vec/data_types/data_type_decimal.h"
+#include "vec/io/var_int.h"
+
+namespace doris::vectorized {
+
+struct WindowFunnelState {
+std::vector> events;
+int max_event_level;
+bool sorted;
+int64_t window;
+
+WindowFunnelState() {
+sorted = true;
+max_event_level = 0;
+window = 0;
+}
+
+void reset() {
+sorted = true;
+max_event_level = 0;
+window = 0;
+std::vector> tmp;
+events.swap(tmp);
+}
+
+void add(const VecDateTimeValue& timestamp, int event_idx, int event_num, 
int64_t win) {
+window = win;
+max_event_level = event_num;
+if (sorted && events.size() > 0) {
+if (events.back().first == timestamp) {
+sorted = events.back().second <= event_idx;
+} else {
+sorted = events.back().first < timestamp;
+}
+}
+events.emplace_back(timestamp, event_idx);
+}
+
+void sort() {
+if (sorted) {
+return;
+}
+std::stable_sort(events.begin(), events.end());
+}
+
+int get() const {
+std::vector> 
events_timestamp(max_event_level);
+for (int64_t i = 0; i < events.size(); i++) {
+const int& event_idx = events[i].second;
+const VecDateTimeValue& timestamp = events[i].first;
+if (event_idx == 0) {
+events_timestamp[0] = timestamp;
+continue;
+}
+if (events_timestamp[event_idx - 1].has_value()) {
+const VecDateTimeValue& first_timestamp = 
events_timestamp[event_idx - 1].value();
+VecDateTimeValue last_timestamp = first_timestamp;
+TimeInterval interval(SECOND, window, false);
+last_timestamp.date_add_interval(interval, SECOND);
+
+if (timestamp <= last_timestamp) {
+events_timestamp[event_idx] = first_timestamp;
+if (event_idx + 1 == max_event_level) {
+// Usually, max event level is small.
+return max_event_level;
+}
+}
+}
+}
+
+for (int64_t i = events_timestamp.size() - 1; i >= 0; i--) {
+if (events_timestamp[i].has_value()) {
+return i + 1;
+}
+}
+
+return 0;
+}
+
+void merge(const WindowFunnelState& other) {
+if (other.events.empty()) {
+return;
+}
+
+int64_t orig_size = events.size();
+events.insert(std::end(events), std::begin(other.events), 
std::end(other.events));
+const auto begin = std::begin(events);
+const auto middle = std::next(events.begin(), orig_size);
+const auto end = std::end(events);
+if (!other.sorted) {
+std::stable_sort(middle, end);
+}
+
+if (!sorted) {
+std::stable_sort(begin, middle);
+}
+std::inplace_merge(begin, middle, end);
+max_event_level = max_event_level > 0 ? max_event_level : 
other.max_event_level;
+window = window > 0 ? window : other.window;
+
+sorted = true;
+}
+
+void write(BufferWritable &out) const {
+write_var_int(max_event_level, out);
+write_var_int(window, out);
+write_var_int(events.size(), out);
+
+for (int64_t i = 0; i < events.size(); i++) {
+int64_t timestamp = events[i].first;
+int event_idx = events[i].second;
+write_var_int(timestamp, out);
+write_var_int(event_idx, out);
+}
+}
+
+void read(BufferReadable& in) {
+int64_t event_level;
+read_var_

[GitHub] [incubator-doris] zhaomin1423 commented on issue #8356: [Bug] broker load can't set not to import a part of columns when the file format is parquet

[GitBox]

zhaomin1423 commented on issue #8356:
URL: 
https://github.com/apache/incubator-doris/issues/8356#issuecomment-1060111973


   When the column name of parquet file and doris column name is same, the 
column must be loaded, otherwise, I don't expect
   the column be loaded, because I don't set it in 'set'  config.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] emmataobao commented on issue #8331: [Bug] doris on es 中 连接es ip问题

[GitBox]

emmataobao commented on issue #8331:
URL: 
https://github.com/apache/incubator-doris/issues/8331#issuecomment-1060112459


   这个问题同样出现使用docker在doris部署上，我使用编译后自己制作镜像进行部署。但是我部署好be和fe，在ALTER SYSTEM ADD 
BACKEND "host:port"，我的host使用的是宿主机ip的话，SHOW PROC '/backends'; ， 
isAlive就是false，我不得不使用docker容器和宿主机ip一致才没有出现问题。


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #8299: [feature] Window funnel

[GitBox]

EmmyMiao87 commented on a change in pull request #8299:
URL: https://github.com/apache/incubator-doris/pull/8299#discussion_r820344685



##
File path: fe/fe-core/src/main/java/org/apache/doris/catalog/Function.java
##
@@ -102,6 +102,8 @@
 private Type retType;
 // Array of parameter types.  empty array if this function does not have 
parameters.
 private Type[] argTypes;
+// Only used by window_funnel()()
+private Type[] leftParamsTypes;

Review comment:
   Remove unused code ~




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] zbtzbtzbt commented on issue #8266: [Enhancement]

[GitBox]

zbtzbtzbt commented on issue #8266:
URL: 
https://github.com/apache/incubator-doris/issues/8266#issuecomment-1060143056


   can you discribe your issue in more detail？


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris-flink-connector] annotated tag 1.14_2.12-1.0.3 updated (c076349 -> 5d5f1ac)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a change to annotated tag 1.14_2.12-1.0.3
in repository 
https://gitbox.apache.org/repos/asf/incubator-doris-flink-connector.git.


*** WARNING: tag 1.14_2.12-1.0.3 was modified! ***

from c076349  (commit)
  to 5d5f1ac  (tag)
 tagging c076349559bfa1a691f7ff2980e502c9e26c171e (commit)
  by morningman
  on Mon Mar 7 11:25:34 2022 +0800

- Log -
[maven-release-plugin] copy for tag 1.14_2.12-1.0.3
---


No new revisions were added by this update.

Summary of changes:

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] zhangstar333 opened a new issue #8357: [Refactor] to_bitmap(-1) return NULL instead of return parse failed error_message

[GitBox]

zhangstar333 opened a new issue #8357:
URL: https://github.com/apache/incubator-doris/issues/8357


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Description
   
   to_bitmap(NULL).  return NULL;
   
   to_bitmap(-1),return NULL;
   
   ### Solution
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris-flink-connector] annotated tag 1.13_2.12-1.0.3 updated (b8828af -> 832ca30)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a change to annotated tag 1.13_2.12-1.0.3
in repository 
https://gitbox.apache.org/repos/asf/incubator-doris-flink-connector.git.


*** WARNING: tag 1.13_2.12-1.0.3 was modified! ***

from b8828af  (commit)
  to 832ca30  (tag)
 tagging b8828af96d5685cd79c2542d392b60099dbcea85 (commit)
  by morningman
  on Mon Mar 7 11:31:02 2022 +0800

- Log -
[maven-release-plugin] copy for tag 1.13_2.12-1.0.3
---


No new revisions were added by this update.

Summary of changes:

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] HappenLee opened a new pull request #8358: [Doc][Vectorized] Add vectorized execution engine docs

[GitBox]

HappenLee opened a new pull request #8358:
URL: https://github.com/apache/incubator-doris/pull/8358


   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (No)
   2. Has unit tests been added: No Need)
   3. Has document been added or modified: (Yes)
   4. Does it need to update dependencies: (No)
   5. Are there any changes that cannot be rolled back: (Yes)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris-flink-connector] annotated tag 1.12_2.12-1.0.3 updated (661b068 -> caad5ea)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a change to annotated tag 1.12_2.12-1.0.3
in repository 
https://gitbox.apache.org/repos/asf/incubator-doris-flink-connector.git.


*** WARNING: tag 1.12_2.12-1.0.3 was modified! ***

from 661b068  (commit)
  to caad5ea  (tag)
 tagging 661b068e4fbd256a695134f18bfebd2fa8093709 (commit)
  by morningman
  on Mon Mar 7 11:33:58 2022 +0800

- Log -
[maven-release-plugin] copy for tag 1.12_2.12-1.0.3
---


No new revisions were added by this update.

Summary of changes:

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris-flink-connector] annotated tag 1.11_2.12-1.0.3 updated (4961f44 -> cd83b15)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a change to annotated tag 1.11_2.12-1.0.3
in repository 
https://gitbox.apache.org/repos/asf/incubator-doris-flink-connector.git.


*** WARNING: tag 1.11_2.12-1.0.3 was modified! ***

from 4961f44  (commit)
  to cd83b15  (tag)
 tagging 4961f445e129e5ed6d37ccd5c97eb77cff8a9def (commit)
  by morningman
  on Mon Mar 7 11:37:26 2022 +0800

- Log -
[maven-release-plugin] copy for tag 1.11_2.12-1.0.3
---


No new revisions were added by this update.

Summary of changes:

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris-spark-connector] annotated tag 3.1_2.12-1.0.1 updated (c99448d -> 40b78c2)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a change to annotated tag 3.1_2.12-1.0.1
in repository 
https://gitbox.apache.org/repos/asf/incubator-doris-spark-connector.git.


*** WARNING: tag 3.1_2.12-1.0.1 was modified! ***

from c99448d  (commit)
  to 40b78c2  (tag)
 tagging c99448d6e7756e0f67e2feaa66385e5adac3d131 (commit)
  by morningman
  on Mon Mar 7 11:42:35 2022 +0800

- Log -
[maven-release-plugin] copy for tag 3.1_2.12-1.0.1
---


No new revisions were added by this update.

Summary of changes:

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris-spark-connector] annotated tag 2.3_2.11-1.0.1 updated (0d79b1c -> 42f261e)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a change to annotated tag 2.3_2.11-1.0.1
in repository 
https://gitbox.apache.org/repos/asf/incubator-doris-spark-connector.git.


*** WARNING: tag 2.3_2.11-1.0.1 was modified! ***

from 0d79b1c  (commit)
  to 42f261e  (tag)
 tagging 0d79b1c7875014a1b82860065c704bbe8154b291 (commit)
  by morningman
  on Mon Mar 7 11:47:01 2022 +0800

- Log -
[maven-release-plugin] copy for tag 2.3_2.11-1.0.1
---


No new revisions were added by this update.

Summary of changes:

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] Userwhite opened a new issue #8360: [Enhancement] support disable slow node's execution ability to ensure query performance

[GitBox]

Userwhite opened a new issue #8360:
URL: https://github.com/apache/incubator-doris/issues/8360


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Description
   
    背景
   * 高并发场景下可能出现偶发的10min超时(集群10s超时)
   
![image](https://user-images.githubusercontent.com/49226823/156964880-6f5b5359-04c1-4b1f-ac90-d5dc1c81be7c.png)
   
   
   ### Solution
   
   * 高并发场景集群可能因为某些后台因素(导入/compaction等 出现慢节点
   * 能够确认是慢节点上的OlapScanNode变慢并且导致后续scanner都在排队
   
   所以对于这种情况，我们可以通过scanner调度时间&scanner调度队列&集群超时空情况及时检查出慢节点，通过disable 
query暂时关闭其查询功能，并在5min(或者可配置)后自动恢复。
   
   **目前这是一个提议。**
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] caiconghui commented on issue #8360: [Enhancement] support disable slow node's execution ability to ensure query performance

[GitBox]

caiconghui commented on issue #8360:
URL: 
https://github.com/apache/incubator-doris/issues/8360#issuecomment-1060163253


   you can type `help alter system` for more detail info, now doris support 
disable query or load for backend manually;


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris-spark-connector] annotated tag 2.3_2.11-1.0.1 updated (258e037 -> 0fab007)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a change to annotated tag 2.3_2.11-1.0.1
in repository 
https://gitbox.apache.org/repos/asf/incubator-doris-spark-connector.git.


*** WARNING: tag 2.3_2.11-1.0.1 was modified! ***

from 258e037  (commit)
  to 0fab007  (tag)
 tagging 258e037a109308734c14736cdcec4f9a010971e9 (commit)
  by morningman
  on Mon Mar 7 12:01:06 2022 +0800

- Log -
2.3_2.11-1.0.1
---


No new revisions were added by this update.

Summary of changes:

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] Userwhite commented on issue #8360: [Enhancement] support disable slow node's execution ability to ensure query performance

[GitBox]

Userwhite commented on issue #8360:
URL: 
https://github.com/apache/incubator-doris/issues/8360#issuecomment-1060165392


   > you can type `help alter system` for more detail info, now doris support 
disable query or load for backend manually;
   
   是的，我这个提议是基于disable query这个功能的。希望FE能够自动发现慢节点并disable其query能力；
   因为根据上图，往往这个时间会很短(10min)，等到发现时去恢复可能已经晚了，开启一个后台进程有点冗余，最好能集成到FE中自动进行比较好


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris-spark-connector] annotated tag 3.1_2.12-1.0.1 updated (c1ca48f -> 556df83)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a change to annotated tag 3.1_2.12-1.0.1
in repository 
https://gitbox.apache.org/repos/asf/incubator-doris-spark-connector.git.


*** WARNING: tag 3.1_2.12-1.0.1 was modified! ***

from c1ca48f  (commit)
  to 556df83  (tag)
 tagging c1ca48f865a71d342497450ca944c7faacae20dc (commit)
  by morningman
  on Mon Mar 7 12:03:12 2022 +0800

- Log -
3.1_2.12-1.0.1
---


No new revisions were added by this update.

Summary of changes:

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris-spark-connector] morningman opened a new pull request #13: [chore] fix name bug in build.sh

[GitBox]

morningman opened a new pull request #13:
URL: https://github.com/apache/incubator-doris-spark-connector/pull/13


   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] kangshisen opened a new pull request #8361: fix doc

[GitBox]

kangshisen opened a new pull request #8361:
URL: https://github.com/apache/incubator-doris/pull/8361


   the doc of help broker load is wrong
   fix complete


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] zenoyang commented on a change in pull request #8318: [improvement](storage) Low cardinality string optimization in storage layer

[GitBox]

zenoyang commented on a change in pull request #8318:
URL: https://github.com/apache/incubator-doris/pull/8318#discussion_r820376200



##
File path: be/src/olap/rowset/segment_v2/binary_dict_page.cpp
##
@@ -132,7 +134,6 @@ void BinaryDictPageBuilder::reset() {
 } else {
 _data_page_builder->reset();
 }
-_finished = false;

Review comment:
   `_finished = false;` appears twice here




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] zenoyang commented on a change in pull request #8318: [improvement](storage) Low cardinality string optimization in storage layer

[GitBox]

zenoyang commented on a change in pull request #8318:
URL: https://github.com/apache/incubator-doris/pull/8318#discussion_r820376317



##
File path: be/src/vec/columns/column_dictionary.h
##
@@ -0,0 +1,381 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include 
+#include 
+
+#include "gutil/hash/string_hash.h"
+#include "olap/decimal12.h"
+#include "olap/uint24.h"
+#include "runtime/string_value.h"
+#include "util/slice.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_decimal.h"
+#include "vec/columns/column_impl.h"
+#include "vec/columns/column_string.h"
+#include "vec/columns/column_vector.h"
+#include "vec/columns/predicate_column.h"
+#include "vec/core/types.h"
+
+namespace doris::vectorized {
+
+/**
+ * For low cardinality string columns, using ColumnDictionary can reducememory
+ * usage and improve query efficiency.
+ * For equal predicate comparisons, convert the predicate constant to encodings
+ * according to the dictionary, so that encoding comparisons are used instead
+ * of string comparisons to improve performance.
+ * For range comparison predicates, it is necessary to sort the dictionary
+ * contents, convert the encoding column, and then compare the encoding 
directly.
+ * If the read data page contains plain-encoded data pages, the dictionary
+ * columns are converted into PredicateColumn for processing.
+ * Currently ColumnDictionary is only used for storage layer.
+ */
+template 
+class ColumnDictionary final : public COWHelper> {
+private:
+friend class COWHelper;
+
+ColumnDictionary() {}
+ColumnDictionary(const size_t n) : codes(n) {}
+ColumnDictionary(const ColumnDictionary& src) : codes(src.codes.begin(), 
src.codes.end()) {}
+
+public:
+using Self = ColumnDictionary;
+using value_type = T;
+using Container = PaddedPODArray;
+using DictContainer = PaddedPODArray;
+
+bool is_numeric() const override { return false; }
+
+bool is_predicate_column() const override { return false; }
+
+bool is_column_dictionary() const override { return true; }
+
+size_t size() const override { return codes.size(); }
+
+[[noreturn]] StringRef get_data_at(size_t n) const override {
+LOG(FATAL) << "get_data_at not supported in ColumnDictionary";
+}
+
+void insert_from(const IColumn& src, size_t n) override {
+LOG(FATAL) << "insert_from not supported in ColumnDictionary";
+}
+
+void insert_range_from(const IColumn& src, size_t start, size_t length) 
override {
+LOG(FATAL) << "insert_range_from not supported in ColumnDictionary";
+}
+
+void insert_indices_from(const IColumn& src, const int* indices_begin,
+ const int* indices_end) override {
+LOG(FATAL) << "insert_indices_from not supported in ColumnDictionary";
+}
+
+void pop_back(size_t n) override { LOG(FATAL) << "pop_back not supported 
in ColumnDictionary"; }
+
+void update_hash_with_value(size_t n, SipHash& hash) const override {
+LOG(FATAL) << "update_hash_with_value not supported in 
ColumnDictionary";
+}
+
+void insert_data(const char* pos, size_t /*length*/) override {
+codes.push_back(unaligned_load(pos));
+}
+
+void insert_data(const T value) { codes.push_back(value); }
+
+void insert_default() override { codes.push_back(T()); }
+
+void clear() override { codes.clear(); }
+
+// TODO: Make dict memory usage more precise
+size_t byte_size() const override { return codes.size() * 
sizeof(codes[0]); }
+
+size_t allocated_bytes() const override { return byte_size(); }
+
+void protect() override {}
+
+void get_permutation(bool reverse, size_t limit, int nan_direction_hint,
+ IColumn::Permutation& res) const override {
+LOG(FATAL) << "get_permutation not supported in ColumnDictionary";
+}
+
+void reserve(size_t n) override { codes.reserve(n); }
+
+[[noreturn]] const char* get_family_name() const override {
+LOG(FATAL) << "get_family_name not supported in ColumnDictionary";
+}
+
+[[noreturn]] MutableColumnPtr clone_resized(size_t size) const override {
+LOG(FATAL) << "clo

[GitHub] [incubator-doris] zenoyang commented on a change in pull request #8318: [improvement](storage) Low cardinality string optimization in storage layer

[GitBox]

zenoyang commented on a change in pull request #8318:
URL: https://github.com/apache/incubator-doris/pull/8318#discussion_r820376504



##
File path: be/src/olap/rowset/segment_v2/binary_dict_page.cpp
##
@@ -240,6 +241,16 @@ void BinaryDictPageDecoder::set_dict_decoder(PageDecoder* 
dict_decoder, StringRe
 
 Status BinaryDictPageDecoder::next_batch(size_t* n, 
vectorized::MutableColumnPtr &dst) {
 if (_encoding_type == PLAIN_ENCODING) {
+auto* col_ptr = dst.get();
+if (dst->is_nullable()) {
+auto nullable_col = 
reinterpret_cast(dst.get());
+col_ptr = nullable_col->get_nested_column_ptr().get();
+}

Review comment:
   ok




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] zenoyang commented on a change in pull request #8318: [improvement](storage) Low cardinality string optimization in storage layer

[GitBox]

zenoyang commented on a change in pull request #8318:
URL: https://github.com/apache/incubator-doris/pull/8318#discussion_r820376713



##
File path: be/src/vec/columns/column_dictionary.h
##
@@ -0,0 +1,381 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include 
+#include 
+
+#include "gutil/hash/string_hash.h"
+#include "olap/decimal12.h"
+#include "olap/uint24.h"
+#include "runtime/string_value.h"
+#include "util/slice.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_decimal.h"
+#include "vec/columns/column_impl.h"
+#include "vec/columns/column_string.h"
+#include "vec/columns/column_vector.h"
+#include "vec/columns/predicate_column.h"
+#include "vec/core/types.h"
+
+namespace doris::vectorized {
+
+/**
+ * For low cardinality string columns, using ColumnDictionary can reducememory
+ * usage and improve query efficiency.
+ * For equal predicate comparisons, convert the predicate constant to encodings
+ * according to the dictionary, so that encoding comparisons are used instead
+ * of string comparisons to improve performance.
+ * For range comparison predicates, it is necessary to sort the dictionary
+ * contents, convert the encoding column, and then compare the encoding 
directly.
+ * If the read data page contains plain-encoded data pages, the dictionary
+ * columns are converted into PredicateColumn for processing.
+ * Currently ColumnDictionary is only used for storage layer.
+ */
+template 
+class ColumnDictionary final : public COWHelper> {
+private:
+friend class COWHelper;
+
+ColumnDictionary() {}
+ColumnDictionary(const size_t n) : codes(n) {}
+ColumnDictionary(const ColumnDictionary& src) : codes(src.codes.begin(), 
src.codes.end()) {}
+
+public:
+using Self = ColumnDictionary;
+using value_type = T;
+using Container = PaddedPODArray;
+using DictContainer = PaddedPODArray;
+
+bool is_numeric() const override { return false; }
+
+bool is_predicate_column() const override { return false; }
+
+bool is_column_dictionary() const override { return true; }
+
+size_t size() const override { return codes.size(); }
+
+[[noreturn]] StringRef get_data_at(size_t n) const override {
+LOG(FATAL) << "get_data_at not supported in ColumnDictionary";
+}
+
+void insert_from(const IColumn& src, size_t n) override {
+LOG(FATAL) << "insert_from not supported in ColumnDictionary";
+}
+
+void insert_range_from(const IColumn& src, size_t start, size_t length) 
override {
+LOG(FATAL) << "insert_range_from not supported in ColumnDictionary";
+}
+
+void insert_indices_from(const IColumn& src, const int* indices_begin,
+ const int* indices_end) override {
+LOG(FATAL) << "insert_indices_from not supported in ColumnDictionary";
+}
+
+void pop_back(size_t n) override { LOG(FATAL) << "pop_back not supported 
in ColumnDictionary"; }
+
+void update_hash_with_value(size_t n, SipHash& hash) const override {
+LOG(FATAL) << "update_hash_with_value not supported in 
ColumnDictionary";
+}
+
+void insert_data(const char* pos, size_t /*length*/) override {
+codes.push_back(unaligned_load(pos));
+}
+
+void insert_data(const T value) { codes.push_back(value); }
+
+void insert_default() override { codes.push_back(T()); }
+
+void clear() override { codes.clear(); }
+
+// TODO: Make dict memory usage more precise
+size_t byte_size() const override { return codes.size() * 
sizeof(codes[0]); }
+
+size_t allocated_bytes() const override { return byte_size(); }
+
+void protect() override {}
+
+void get_permutation(bool reverse, size_t limit, int nan_direction_hint,
+ IColumn::Permutation& res) const override {
+LOG(FATAL) << "get_permutation not supported in ColumnDictionary";
+}
+
+void reserve(size_t n) override { codes.reserve(n); }
+
+[[noreturn]] const char* get_family_name() const override {
+LOG(FATAL) << "get_family_name not supported in ColumnDictionary";
+}
+
+[[noreturn]] MutableColumnPtr clone_resized(size_t size) const override {
+LOG(FATAL) << "clo

[GitHub] [incubator-doris] morningman opened a new issue #8362: [Bug] Broker Load job's state is LOADING while the job is actually CANCELLED

[GitBox]

morningman opened a new issue #8362:
URL: https://github.com/apache/incubator-doris/issues/8362


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Version
   
   0.15
   
   ### What's Wrong?
   
   ```
   mysql> show load where state="loading"\G
   *** 1. row ***
JobId: 2752441
Label: load_content_20220302
State: LOADING
 Progress: ETL:100%; LOAD:0%
 Type: BROKER
  EtlInfo: NULL
 TaskInfo: cluster:N/A; timeout(s):10800; max_filter_ratio:0.0
 ErrorMsg: type:LOAD_RUN_FAIL; msg:timeout by txn manager
   CreateTime: 2022-03-04 18:20:26
 EtlStartTime: 2022-03-04 21:23:48
EtlFinishTime: 2022-03-04 21:23:48
LoadStartTime: 2022-03-04 21:23:48
   LoadFinishTime: 2022-03-04 21:20:43
  URL: NULL
   JobDetails: {"Unfinished 
backends":{},"ScannedRows":0,"TaskNumber":0,"LoadBytes":0,"All 
backends":{},"FileNumber":xxx,"FileSize":xxx}
TransactionId: 28988388
 ErrorTablets: {}
   ```
   
   The above job is cancelled due to `timeout by txn manager`, but state is 
LOADING.
   
   
   
   ### What You Expected?
   
   The state should be "CANCELLED"
   
   ### How to Reproduce?
   
   Not very common.
   
   ### Anything Else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] morningman opened a new pull request #8363: [fix](broker-load) fix bug that a cancelled job's state is LOADING

[GitBox]

morningman opened a new pull request #8363:
URL: https://github.com/apache/incubator-doris/pull/8363


   # Proposed changes
   
   Issue Number: close #8362 
   
   ## Problem Summary:
   
   Before executing LoadLoadingTask of a broker load, we should check if the job
   is cancelled.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (No)
   2. Has unit tests been added: (No Need)
   3. Has document been added or modified: (No Need)
   4. Does it need to update dependencies: (No)
   5. Are there any changes that cannot be rolled back: (No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] carlvinhust2012 opened a new pull request #8364: [Function] Add compare_version function and ut

[GitBox]

carlvinhust2012 opened a new pull request #8364:
URL: https://github.com/apache/incubator-doris/pull/8364


   # Proposed changes
   Add compare_version function and ut. the function will return the compared 
result of two version.
   Issue Number: close #xxx
   
   ## Problem Summary:
   
   Describe the overview of changes.
   1.Add compare_version function and ut.
   2.add md files for this compare_version function.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: No
   2. Has unit tests been added: Yes
   3. Has document been added or modified: Yes
   4. Does it need to update dependencies:No
   5. Are there any changes that cannot be rolled back: No
   
   ## Further comments
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] morningman opened a new pull request #8365: [feature](user-property) Support user level exec_mem_limit and load_mem_limit

[GitBox]

morningman opened a new pull request #8365:
URL: https://github.com/apache/incubator-doris/pull/8365


   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem Summary:
   
   ```
   SET PROPERTY FOR 'jack' 'exec_mem_limit' = '2147483648';
   SET PROPERTY FOR 'jack' 'load_mem_limit' = '2147483648';
   ```
   
   The user level property will overwrite the value in session variables.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] morningman opened a new issue #8366: [Feature] Support user level memory limit settings

[GitBox]

morningman opened a new issue #8366:
URL: https://github.com/apache/incubator-doris/issues/8366


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Description
   
   Doris currently only supports setting the memory limit for querying and 
loading at the session level.
   It is hoped that user-level memory limits will also be supported.
   
   ### Use case
   
   _No response_
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] yiguolei commented on pull request #8352: [improvement]support build with parallel parameter only

[GitBox]

yiguolei commented on pull request #8352:
URL: https://github.com/apache/incubator-doris/pull/8352#issuecomment-1060222308


   Hi everybody, I think we should think about this feature.
   I think we should set the parallel parameter to half of the build machine's 
CPU core. Because many user feel build process is too slow and we have to told 
them that there is parameter to speed up. But many user will not ask for help 
and they will treat the slow build process as normal.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] morningman merged pull request #8296: [Feature] Support update stmt in vec query engine

[GitBox]

morningman merged pull request #8296:
URL: https://github.com/apache/incubator-doris/pull/8296


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] morningman closed issue #8295: [Feature] Support update query in vec query engine

[GitBox]

morningman closed issue #8295:
URL: https://github.com/apache/incubator-doris/issues/8295


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris] branch master updated: [feature](vec) Support update stmt in vec query engine (#8296)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 477b87c  [feature](vec) Support update stmt in vec query engine (#8296)
477b87c is described below

commit 477b87cb28042cfc8ee725a9694f99165d02426a
Author: HappenLee 
AuthorDate: Mon Mar 7 14:03:55 2022 +0800

[feature](vec) Support update stmt in vec query engine (#8296)
---
 .../src/main/java/org/apache/doris/load/update/UpdatePlanner.java | 4 
 .../main/java/org/apache/doris/load/update/UpdateStmtExecutor.java| 2 ++
 fe/fe-core/src/main/java/org/apache/doris/planner/HashJoinNode.java   | 2 +-
 fe/fe-core/src/main/java/org/apache/doris/planner/PlanNode.java   | 2 +-
 fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java | 4 
 5 files changed, 12 insertions(+), 2 deletions(-)

diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/load/update/UpdatePlanner.java 
b/fe/fe-core/src/main/java/org/apache/doris/load/update/UpdatePlanner.java
index aaadd11..a452730 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/load/update/UpdatePlanner.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/load/update/UpdatePlanner.java
@@ -30,6 +30,7 @@ import org.apache.doris.catalog.OlapTable;
 import org.apache.doris.common.AnalysisException;
 import org.apache.doris.common.IdGenerator;
 import org.apache.doris.common.UserException;
+import org.apache.doris.common.util.VectorizedUtil;
 import org.apache.doris.planner.DataPartition;
 import org.apache.doris.planner.OlapScanNode;
 import org.apache.doris.planner.OlapTableSink;
@@ -86,6 +87,9 @@ public class UpdatePlanner extends Planner {
 /* END */
 olapScanNode.init(analyzer);
 olapScanNode.finalize(analyzer);
+if (VectorizedUtil.isVectorized()) {
+olapScanNode.convertToVectoriezd();
+}
 scanNodeList.add(olapScanNode);
 // 2. gen olap table sink
 OlapTableSink olapTableSink = new OlapTableSink(targetTable, 
computeTargetTupleDesc(), null);
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/load/update/UpdateStmtExecutor.java 
b/fe/fe-core/src/main/java/org/apache/doris/load/update/UpdateStmtExecutor.java
index e1ec792..f6a3d49 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/load/update/UpdateStmtExecutor.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/load/update/UpdateStmtExecutor.java
@@ -34,6 +34,7 @@ import org.apache.doris.common.QuotaExceedException;
 import org.apache.doris.common.UserException;
 import org.apache.doris.common.util.DebugUtil;
 import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
 import org.apache.doris.metric.MetricRepo;
 import org.apache.doris.qe.Coordinator;
 import org.apache.doris.qe.QeProcessorImpl;
@@ -141,6 +142,7 @@ public class UpdateStmtExecutor {
 coordinator = new Coordinator(Catalog.getCurrentCatalog().getNextId(), 
queryId, analyzer.getDescTbl(),
 updatePlanner.getFragments(), updatePlanner.getScanNodes(), 
TimeUtils.DEFAULT_TIME_ZONE, false);
 coordinator.setQueryType(TQueryType.LOAD);
+coordinator.setExecVecEngine(VectorizedUtil.isVectorized());
 QeProcessorImpl.INSTANCE.registerQuery(queryId, coordinator);
 analyzer.getContext().getExecutor().setCoord(coordinator);
 
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/planner/HashJoinNode.java 
b/fe/fe-core/src/main/java/org/apache/doris/planner/HashJoinNode.java
index 2ee2618..8015fb7 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/planner/HashJoinNode.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/planner/HashJoinNode.java
@@ -669,7 +669,7 @@ public class HashJoinNode extends PlanNode {
 }
 
 @Override
-void convertToVectoriezd() {
+public void convertToVectoriezd() {
 if (!otherJoinConjuncts.isEmpty()) {
 votherJoinConjunct = 
convertConjunctsToAndCompoundPredicate(otherJoinConjuncts);
 initCompoundPredicate(votherJoinConjunct);
diff --git a/fe/fe-core/src/main/java/org/apache/doris/planner/PlanNode.java 
b/fe/fe-core/src/main/java/org/apache/doris/planner/PlanNode.java
index a6003f3..e076021 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/planner/PlanNode.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/planner/PlanNode.java
@@ -860,7 +860,7 @@ abstract public class PlanNode extends TreeNode {
 return sb.toString();
 }
 
-void convertToVectoriezd() {
+public void convertToVectoriezd() {
 if (!conjuncts.isEmpty()) {
 vconjunct = convertConjunctsToAndCompoundPredicate(conjuncts);
 initCompoundPredicate(vconjunct);
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java 
b/fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java
ind

[GitHub] [incubator-doris] morningman commented on pull request #8352: [improvement]support build with parallel parameter only

[GitBox]

morningman commented on pull request #8352:
URL: https://github.com/apache/incubator-doris/pull/8352#issuecomment-1060225204


   > Hi everybody, I think we should think about this feature. I think we 
should set the parallel parameter to half of the build machine's CPU core. 
Because many user feel build process is too slow and we have to told them that 
there is parameter to speed up. But many user will not ask for help and they 
will treat the slow build process as normal.
   
   Currently, the default value is 1/4 of cpu core. Using 1/2 may cause OOM 
exception(I am not really sure about that).
   It there any "best practice" that we can refer to?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris] 03/03: [feature](vec) Support update stmt in vec query engine (#8296)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch dev-1.0.0
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git

commit d7f8a5d79e804f44b02323170e2cf5d1026627ba
Author: HappenLee 
AuthorDate: Mon Mar 7 14:03:55 2022 +0800

[feature](vec) Support update stmt in vec query engine (#8296)
---
 .../src/main/java/org/apache/doris/load/update/UpdatePlanner.java | 4 
 .../main/java/org/apache/doris/load/update/UpdateStmtExecutor.java| 2 ++
 fe/fe-core/src/main/java/org/apache/doris/planner/HashJoinNode.java   | 2 +-
 fe/fe-core/src/main/java/org/apache/doris/planner/PlanNode.java   | 2 +-
 fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java | 4 
 5 files changed, 12 insertions(+), 2 deletions(-)

diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/load/update/UpdatePlanner.java 
b/fe/fe-core/src/main/java/org/apache/doris/load/update/UpdatePlanner.java
index a686d26..12d9c16 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/load/update/UpdatePlanner.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/load/update/UpdatePlanner.java
@@ -30,6 +30,7 @@ import org.apache.doris.catalog.OlapTable;
 import org.apache.doris.common.AnalysisException;
 import org.apache.doris.common.IdGenerator;
 import org.apache.doris.common.UserException;
+import org.apache.doris.common.util.VectorizedUtil;
 import org.apache.doris.planner.DataPartition;
 import org.apache.doris.planner.OlapScanNode;
 import org.apache.doris.planner.OlapTableSink;
@@ -86,6 +87,9 @@ public class UpdatePlanner extends Planner {
 /* END */
 olapScanNode.init(analyzer);
 olapScanNode.finalize(analyzer);
+if (VectorizedUtil.isVectorized()) {
+olapScanNode.convertToVectoriezd();
+}
 scanNodeList.add(olapScanNode);
 // 2. gen olap table sink
 OlapTableSink olapTableSink = new OlapTableSink(targetTable, 
computeTargetTupleDesc(), null);
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/load/update/UpdateStmtExecutor.java 
b/fe/fe-core/src/main/java/org/apache/doris/load/update/UpdateStmtExecutor.java
index e1ec792..f6a3d49 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/load/update/UpdateStmtExecutor.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/load/update/UpdateStmtExecutor.java
@@ -34,6 +34,7 @@ import org.apache.doris.common.QuotaExceedException;
 import org.apache.doris.common.UserException;
 import org.apache.doris.common.util.DebugUtil;
 import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
 import org.apache.doris.metric.MetricRepo;
 import org.apache.doris.qe.Coordinator;
 import org.apache.doris.qe.QeProcessorImpl;
@@ -141,6 +142,7 @@ public class UpdateStmtExecutor {
 coordinator = new Coordinator(Catalog.getCurrentCatalog().getNextId(), 
queryId, analyzer.getDescTbl(),
 updatePlanner.getFragments(), updatePlanner.getScanNodes(), 
TimeUtils.DEFAULT_TIME_ZONE, false);
 coordinator.setQueryType(TQueryType.LOAD);
+coordinator.setExecVecEngine(VectorizedUtil.isVectorized());
 QeProcessorImpl.INSTANCE.registerQuery(queryId, coordinator);
 analyzer.getContext().getExecutor().setCoord(coordinator);
 
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/planner/HashJoinNode.java 
b/fe/fe-core/src/main/java/org/apache/doris/planner/HashJoinNode.java
index 2ee2618..8015fb7 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/planner/HashJoinNode.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/planner/HashJoinNode.java
@@ -669,7 +669,7 @@ public class HashJoinNode extends PlanNode {
 }
 
 @Override
-void convertToVectoriezd() {
+public void convertToVectoriezd() {
 if (!otherJoinConjuncts.isEmpty()) {
 votherJoinConjunct = 
convertConjunctsToAndCompoundPredicate(otherJoinConjuncts);
 initCompoundPredicate(votherJoinConjunct);
diff --git a/fe/fe-core/src/main/java/org/apache/doris/planner/PlanNode.java 
b/fe/fe-core/src/main/java/org/apache/doris/planner/PlanNode.java
index a6003f3..e076021 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/planner/PlanNode.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/planner/PlanNode.java
@@ -860,7 +860,7 @@ abstract public class PlanNode extends TreeNode {
 return sb.toString();
 }
 
-void convertToVectoriezd() {
+public void convertToVectoriezd() {
 if (!conjuncts.isEmpty()) {
 vconjunct = convertConjunctsToAndCompoundPredicate(conjuncts);
 initCompoundPredicate(vconjunct);
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java 
b/fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java
index 6be03cb..a64853a 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java
@@ -300,6 +300

[incubator-doris] branch dev-1.0.0 updated (f4e0971 -> d7f8a5d)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a change to branch dev-1.0.0
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git.


from f4e0971  [fix][vectorized] coredump about const compare in exchange 
node (#8302)
 new 2028994  [fix] Use fmt::to_string replace memory buffer::data() (#8311)
 new 15b09d1  [fix][vectorized] Fix error cast to boolean (#8345)
 new d7f8a5d  [feature](vec) Support update stmt in vec query engine (#8296)

The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 be/src/exec/base_scanner.cpp   |  4 ++--
 be/src/exec/broker_scanner.cpp |  6 +++---
 be/src/exec/json_scanner.cpp   | 22 +++---
 be/src/exec/odbc_connector.cpp | 22 +++---
 be/src/exec/tablet_sink.cpp|  6 +++---
 be/src/runtime/runtime_state.cpp   |  4 ++--
 be/src/vec/functions/function_cast.h   | 12 
 be/src/vec/io/io_helper.h  | 17 +
 be/src/vec/sink/vmysql_table_writer.cpp|  8 
 be/src/vec/sink/vtablet_sink.cpp   | 16 +++-
 .../apache/doris/load/update/UpdatePlanner.java|  4 
 .../doris/load/update/UpdateStmtExecutor.java  |  2 ++
 .../org/apache/doris/planner/HashJoinNode.java |  2 +-
 .../java/org/apache/doris/planner/PlanNode.java|  2 +-
 .../main/java/org/apache/doris/qe/Coordinator.java |  4 
 15 files changed, 84 insertions(+), 47 deletions(-)

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris] 02/03: [fix][vectorized] Fix error cast to boolean (#8345)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch dev-1.0.0
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git

commit 15b09d1692c10a58225569d2d2e34ffd41ec4d42
Author: HappenLee 
AuthorDate: Sun Mar 6 13:47:46 2022 +0800

[fix][vectorized] Fix error cast to boolean (#8345)
---
 be/src/vec/functions/function_cast.h | 12 
 be/src/vec/io/io_helper.h| 17 +
 2 files changed, 29 insertions(+)

diff --git a/be/src/vec/functions/function_cast.h 
b/be/src/vec/functions/function_cast.h
index 053d44c..144b782 100644
--- a/be/src/vec/functions/function_cast.h
+++ b/be/src/vec/functions/function_cast.h
@@ -138,6 +138,13 @@ struct ConvertImpl {
 vec_to[i] = static_cast(vec_from[i]);
 }
 
+// TODO: support boolean cast more reasonable
+if constexpr (std::is_same_v) {
+for (int i = 0; i < size; ++i) {
+vec_to[i] = static_cast(vec_to[i]);
+}
+}
+
 block.replace_by_position(result, std::move(col_to));
 } else {
 return Status::RuntimeError(
@@ -315,6 +322,11 @@ bool try_parse_impl(typename DataType::FieldType& x, 
ReadBuffer& rb, const DateL
 return try_read_float_text(x, rb);
 }
 
+// uint8_t now use as boolean in doris
+if constexpr (std::is_same_v) {
+return try_read_bool_text(x, rb);
+}
+
 if constexpr (std::is_integral_v) {
 return try_read_int_text(x, rb);
 }
diff --git a/be/src/vec/io/io_helper.h b/be/src/vec/io/io_helper.h
index 99253c9..dafbed6 100644
--- a/be/src/vec/io/io_helper.h
+++ b/be/src/vec/io/io_helper.h
@@ -295,6 +295,23 @@ bool read_decimal_text_impl(T& x, ReadBuffer& buf) {
 }
 
 template 
+bool try_read_bool_text(T& x, ReadBuffer& buf) {
+if (read_int_text_impl(x, buf)) {
+return x == 0 || x == 1;
+}
+
+StringParser::ParseResult result;
+x = StringParser::string_to_bool(buf.position(), buf.count(), &result);
+if (UNLIKELY(result != StringParser::PARSE_SUCCESS)) {
+return false;
+}
+
+// only to match the is_all_read() check to prevent return null
+buf.position() = buf.end();
+return true;
+}
+
+template 
 bool try_read_int_text(T& x, ReadBuffer& buf) {
 return read_int_text_impl(x, buf);
 }

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris] 01/03: [fix] Use fmt::to_string replace memory buffer::data() (#8311)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch dev-1.0.0
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git

commit 2028994543fe1c5a85109ad3bde60101c8f76cfa
Author: HappenLee 
AuthorDate: Sun Mar 6 13:44:11 2022 +0800

[fix] Use fmt::to_string replace memory buffer::data() (#8311)
---
 be/src/exec/base_scanner.cpp|  4 ++--
 be/src/exec/broker_scanner.cpp  |  6 +++---
 be/src/exec/json_scanner.cpp| 22 +++---
 be/src/exec/odbc_connector.cpp  | 22 +++---
 be/src/exec/tablet_sink.cpp |  6 +++---
 be/src/runtime/runtime_state.cpp|  4 ++--
 be/src/vec/sink/vmysql_table_writer.cpp |  8 
 be/src/vec/sink/vtablet_sink.cpp| 16 +++-
 8 files changed, 43 insertions(+), 45 deletions(-)

diff --git a/be/src/exec/base_scanner.cpp b/be/src/exec/base_scanner.cpp
index f7c0a1b..ab004b1 100644
--- a/be/src/exec/base_scanner.cpp
+++ b/be/src/exec/base_scanner.cpp
@@ -209,7 +209,7 @@ Status BaseScanner::fill_dest_tuple(Tuple* dest_tuple, 
MemPool* mem_pool) {
"column({}) value is incorrect 
while strict mode is {}, "
"src value is {}",
slot_desc->col_name(), 
_strict_mode, raw_string);
-return error_msg.data();
+return fmt::to_string(error_msg);
 },
 &_scanner_eof));
 _counter->num_rows_filtered++;
@@ -227,7 +227,7 @@ Status BaseScanner::fill_dest_tuple(Tuple* dest_tuple, 
MemPool* mem_pool) {
 error_msg,
 "column({}) values is null while columns 
is not nullable",
 slot_desc->col_name());
-return error_msg.data();
+return fmt::to_string(error_msg);
 },
 &_scanner_eof));
 _counter->num_rows_filtered++;
diff --git a/be/src/exec/broker_scanner.cpp b/be/src/exec/broker_scanner.cpp
index af44b8f..96e9e31 100644
--- a/be/src/exec/broker_scanner.cpp
+++ b/be/src/exec/broker_scanner.cpp
@@ -479,7 +479,7 @@ Status BrokerScanner::_line_to_src_tuple(const Slice& line) 
{
 []() -> std::string {
 fmt::memory_buffer error_msg;
 fmt::format_to(error_msg, "{}", "Unable to display");
-return error_msg.data();
+return fmt::to_string(error_msg);
 }, &_scanner_eof));
 _counter->num_rows_filtered++;
 _success = false;
@@ -514,7 +514,7 @@ Status BrokerScanner::_line_to_src_tuple(const Slice& line) 
{
 fmt::format_to(error_msg, "{}", "actual column number 
is less than schema column number.");
 fmt::format_to(error_msg, "actual number: {}, column 
separator: [{}], ", _split_values.size(), _value_separator);
 fmt::format_to(error_msg, "line delimiter: [{}], 
schema number: {}; ", _line_delimiter, _src_slot_descs.size());
-return error_msg.data();
+return fmt::to_string(error_msg);
 }, &_scanner_eof));
 _counter->num_rows_filtered++;
 _success = false;
@@ -527,7 +527,7 @@ Status BrokerScanner::_line_to_src_tuple(const Slice& line) 
{
 fmt::format_to(error_msg, "{}", "actual column number 
is more than schema column number.");
 fmt::format_to(error_msg, "actual number: {}, column 
separator: [{}], ", _split_values.size(), _value_separator);
 fmt::format_to(error_msg, "line delimiter: [{}], 
schema number: {}; ", _line_delimiter, _src_slot_descs.size());
-return error_msg.data();
+return fmt::to_string(error_msg);
 }, &_scanner_eof));
 _counter->num_rows_filtered++;
 _success = false;
diff --git a/be/src/exec/json_scanner.cpp b/be/src/exec/json_scanner.cpp
index eaff43f..67a58c4 100644
--- a/be/src/exec/json_scanner.cpp
+++ b/be/src/exec/json_scanner.cpp
@@ -406,7 +406,7 @@ Status JsonReader::_parse_json_doc(size_t* size, bool* eof) 
{
 fmt::format_to(error_msg, "Parse json data for JsonDoc failed. code: 
{}, error info: {}",
 _origin_json_doc.GetParseError(), 
rapidjson::GetParseError_En(_origin_json_doc.GetParseError()));
 RETURN_IF_ERROR(_state->append_error_msg_to_file([&]() -> std::string 
{ return std::string((char*)json_str, *size); },
-[&]() -> std::string { return error_msg.data(); }, 
_scanner_eof));
+[&]() -> std::string { return fmt::to_string(error_msg)

[GitHub] [incubator-doris] yiguolei commented on pull request #8352: [improvement]support build with parallel parameter only

[GitBox]

yiguolei commented on pull request #8352:
URL: https://github.com/apache/incubator-doris/pull/8352#issuecomment-1060231896


   > 
   If too many build thread will cause OOM, then it is ok for me. And I will 
track this issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] steadyBoy commented on a change in pull request #8339: [docs] add document conditional-functions

[GitBox]

steadyBoy commented on a change in pull request #8339:
URL: https://github.com/apache/incubator-doris/pull/8339#discussion_r820409012



##
File path: docs/en/sql-reference/sql-functions/conditional-functions/ifnull.md
##
@@ -0,0 +1,54 @@
+---
+{
+"title": "ifnull",
+"language": "en"
+}
+---
+
+
+
+# ifnull
+## description
+### Syntax
+
+`ifnull(expr1, expr2)`
+
+
+If the value of expr1 is not null, V1 is returned, otherwise expr2 is returned
+

Review comment:
   Yes, I'll fix it

##
File path: docs/zh-CN/sql-reference/sql-functions/conditional-functions/case.md
##
@@ -0,0 +1,59 @@
+---
+{
+"title": "case",
+"language": "zh-CN"
+}
+---
+
+
+
+# case
+## description
+### Syntax
+
+`CASE a WHEN b THEN c [WHEN d THEN e]... [ELSE f] END`
+
+OR
+
+`CASE WHEN a THEN b [WHEN c THEN d]* [ELSE e] END`
+

Review comment:
   OK




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] steadyBoy commented on a change in pull request #8339: [docs] add document conditional-functions

[GitBox]

steadyBoy commented on a change in pull request #8339:
URL: https://github.com/apache/incubator-doris/pull/8339#discussion_r820409945



##
File path: docs/en/sql-reference/sql-functions/conditional-functions/if.md
##
@@ -0,0 +1,48 @@
+---
+{
+"title": "if",
+"language": "en"
+}
+---
+
+
+
+# if
+## description
+### Syntax
+
+`if(boolean condition, type valueTrue, type valueFalseOrNull)`
+

Review comment:
   I added in the description




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] adonis0147 opened a new issue #8367: [feature-wip][array-type] Array data can be loaded in stream load.

[GitBox]

adonis0147 opened a new issue #8367:
URL: https://github.com/apache/incubator-doris/issues/8367


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Description
   
   This issue is a sub task of #7570 .
   
   Array data can be loaded in stream load.
   
   ### Use case
   
   ```shell
   mysql> create table array_type_table(k1 INT, k2 Array) duplicate key 
(k1)
   -> distributed by hash(k1) buckets 1 properties('replication_num' = '1');
   ```
   
   data.json
   ```json
   [{"k1": 1, "v1": [1, 2, 3]}]
   ```
   
   stream load:
   ```shell
   curl -v --location-trusted -u root: -H "format: json" -T data.json 
http://:/api/test/array_type_table/_stream_load
   ```
   
   ### Related issues
   
   #7570 
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] adonis0147 opened a new pull request #8368: [feature-wip][array-type] Array data can be loaded in stream load.

[GitBox]

adonis0147 opened a new pull request #8368:
URL: https://github.com/apache/incubator-doris/pull/8368


   # Proposed changes
   
   Issue Number: close #8367 
   
   ## Problem Summary:
   
   Please refer to #8367 .
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: No
   2. Has unit tests been added: Yes
   3. Has document been added or modified: No
   4. Does it need to update dependencies: No
   5. Are there any changes that cannot be rolled back: No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] bridgeDream opened a new pull request #8369: [docs] Update documentation configuration parameter `sink.batch.bytes…

[GitBox]

bridgeDream opened a new pull request #8369:
URL: https://github.com/apache/incubator-doris/pull/8369


   …` in flink-doris-connector
   
   # Proposed changes
   
   
   
   ## Problem Summary:
   
   Please refer to: 
https://github.com/apache/incubator-doris-flink-connector/pull/13
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (No)
   2. Has unit tests been added: (No Need)
   3. Has document been added or modified: (Yes)
   4. Does it need to update dependencies: (No)
   5. Are there any changes that cannot be rolled back: (No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris-flink-connector] bridgeDream commented on pull request #13: Feature/20220305 support flush when batch bytes reach threshold

[GitBox]

bridgeDream commented on pull request #13:
URL: 
https://github.com/apache/incubator-doris-flink-connector/pull/13#issuecomment-1060243759


   > Update documentation configuration parameter description @bridgeDream
   
   Hi, I had already submitted a PR to update the document in 
https://github.com/apache/incubator-doris/pull/8369 @hf200012 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] Userwhite opened a new pull request #8370: Support get metric by name

[GitBox]

Userwhite opened a new pull request #8370:
URL: https://github.com/apache/incubator-doris/pull/8370


   # Proposed changes
   
   add api parameter for metric name
   
   ## Problem Summary:
   
   support get metric by name(url)
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] hf200012 opened a new pull request #8371: Update hive support version，hive：3.1.2

[GitBox]

hf200012 opened a new pull request #8371:
URL: https://github.com/apache/incubator-doris/pull/8371


   Update hive support version：3.1.2
   
   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris] branch array-type updated (04ad3d2 -> 28ea295)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a change to branch array-type
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git.


omit 04ad3d2  [feature-wip][array-type] Support ArrayLiteral in SQL. (#8089)
omit ac6048a  [feature-wip](array-type) Create table with nested array 
type. (#8003)
 add 246ac4e  [fix] fix a bug of encryption function with iv  may return 
wrong result (#8277)
 add 114eb19  [fix](ut) query stmt test error  (#8303)
 add 09bfb8b  [fix] (rpc-udf) Fixed the problem that the query could not be 
interrupted (#8248)
 add 18098c5  [fix](fe-ut) Fix FE unit test (#8293)
 add 8be71b6  [refactor] remove pusher.cpp and related mock test code 
(#8288)
 add f622ce0  [refactor] remove types_test (#8289)
 add f5ab055  [chore] remove some ut temp files and add some file to 
.gitignore (#8309)
 add c56a372  [improvement][fix](grouping-set)(tablet-repair) optimize 
compaction too slow replica process, (#8123)
 add e7c4175  [fix] fix hash table insert() may be failed but not handle 
this error (#8207)
 add 80e8815  [improvement](restore) allow query on part of partitions when 
others are in RESTORE (#8245)
 add 538df28  [improvement](routine-load) Support routine load task succeed 
with empty data consumed (#8256)
 add 0ee53be  [fix][improvement](runtime-filter) fix string type length 
limit error && add runtime filter decimal support (#8282)
 add 46ca23f  [Feature] Support Changing the bucketing mode of the table 
from Hash Distribution to Random Distribution (#8259)
 add 0383001  [Enhancement] Support Skipping compaction lower replica where 
select queryable replica for better scan performance (#8146)
 add 450273c  [typo]update spark build doc (#8333)
 add 777c9a4  [typo](comment) Translate the code comments of gensrc (#8308)
 add 61455ec  [docs] fix invalid links in docker-dev document (#8313)
 add 11c9c4f  [improvement][website] The expansion of sidebar is off by 
default (#8314)
 add 9cf2798  [typo] fix error for PushTask (#8316)
 add 9961b2c  [refactor] Remove mysql-connector and replace org.json with 
com.googlecode.json-simple (#8319)
 add 9becb5a  [typo] translate the comments of schema_change.h (#8321)
 add f52d479  [fix](ut) fix be ut fragment_mgr_test compile failed (#8344)
 add f57f02b  [improvement] Support show tablets stmt (#7970)
 add 22a0011  [fix](planner) Convert format in RewriteFromUnixTimeRule 
(#8235)
 add c18717d  [fix](vectorized) Fix core dump of mutable block different of 
block (#8280)
 add 910a140  [fix][vectorized] coredump about const compare in exchange 
node (#8302)
 add baa3b14  [fix] Use fmt::to_string replace memory buffer::data() (#8311)
 add be4dd0c  [fix][vectorized] Fix error cast to boolean (#8345)
 add ea4ad1a  [doc] Add sync job fe configuration item description (#8349)
 add 2519f8b  format fe config title , add link for tablet_rebalancer_type 
(#8346)
 add 1879623  [docs]update http port doc to be more intuitive (#8343)
 add 3b159a9  support doriswriter build in macos (#8330)
 add 477b87c  [feature](vec) Support update stmt in vec query engine (#8296)
 new 3e2d90d  [feature-wip](array-type) Create table with nested array 
type. (#8003)
 new 28ea295  [feature-wip][array-type] Support ArrayLiteral in SQL. (#8089)

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (04ad3d2)
\
 N -- N -- N   refs/heads/array-type (28ea295)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .gitignore |  11 +-
 be/CMakeLists.txt  |   2 +-
 be/src/agent/pusher.cpp| 215 ---
 be/src/common/config.h |   3 -
 be/src/exec/base_scanner.cpp   |   4 +-
 be/src/exec/broker_scanner.cpp |   6 +-
 be/src/exec/except_node.cpp|  12 +-
 be/src/exec/hash_join_node.cpp |   8 +-
 be/src/exec/hash_table.h   |  32 +-
 be/src/exec/hash_table.hpp |   4 +-
 be/src/e

[incubator-doris] 01/02: [feature-wip](array-type) Create table with nested array type. (#8003)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch array-type
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git

commit 3e2d90d60be35b30d087e7d04e635ba75932c7dd
Author: Adonis Ling 
AuthorDate: Wed Feb 16 12:44:30 2022 +0800

[feature-wip](array-type) Create table with nested array type. (#8003)

```
create table array_type_table(k1 INT, k2 Array>) duplicate key 
(k1)
distributed by hash(k1) buckets 1 properties('replication_num' = '1');
```
---
 be/src/olap/tablet_meta.cpp  |  9 -
 .../org/apache/doris/analysis/CreateTableStmt.java   | 16 
 .../main/java/org/apache/doris/analysis/TypeDef.java | 10 --
 .../java/org/apache/doris/catalog/ArrayType.java |  9 +
 .../main/java/org/apache/doris/catalog/Column.java   | 10 --
 .../src/main/java/org/apache/doris/catalog/Type.java | 20 +++-
 .../org/apache/doris/catalog/CreateTableTest.java| 13 +
 7 files changed, 41 insertions(+), 46 deletions(-)

diff --git a/be/src/olap/tablet_meta.cpp b/be/src/olap/tablet_meta.cpp
index 6e6195d..73632a1 100644
--- a/be/src/olap/tablet_meta.cpp
+++ b/be/src/olap/tablet_meta.cpp
@@ -128,11 +128,6 @@ TabletMeta::TabletMeta(int64_t table_id, int64_t 
partition_id, int64_t tablet_id
 }
 }
 }
-
-if (tcolumn.column_type.type == TPrimitiveType::ARRAY) {
-ColumnPB* children_column = column->add_children_columns();
-_init_column_from_tcolumn(0, tcolumn.children_column[0], 
children_column);
-}
 }
 
 schema->set_next_column_unique_id(next_unique_id);
@@ -210,6 +205,10 @@ void TabletMeta::_init_column_from_tcolumn(uint32_t 
unique_id, const TColumn& tc
 if (tcolumn.__isset.is_bloom_filter_column) {
 column->set_is_bf_column(tcolumn.is_bloom_filter_column);
 }
+if (tcolumn.column_type.type == TPrimitiveType::ARRAY) {
+ColumnPB* children_column = column->add_children_columns();
+_init_column_from_tcolumn(0, tcolumn.children_column[0], 
children_column);
+}
 }
 
 OLAPStatus TabletMeta::create_from_file(const string& file_path) {
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateTableStmt.java 
b/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateTableStmt.java
index ed2b689..5fb937e 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateTableStmt.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/CreateTableStmt.java
@@ -17,8 +17,11 @@
 
 package org.apache.doris.analysis;
 
+import com.google.common.base.Strings;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Sets;
+import org.apache.commons.collections.CollectionUtils;
 import org.apache.doris.catalog.AggregateType;
-import org.apache.doris.catalog.ArrayType;
 import org.apache.doris.catalog.Catalog;
 import org.apache.doris.catalog.Column;
 import org.apache.doris.catalog.Index;
@@ -35,12 +38,6 @@ import org.apache.doris.common.util.PrintableMap;
 import org.apache.doris.external.elasticsearch.EsUtil;
 import org.apache.doris.mysql.privilege.PrivPredicate;
 import org.apache.doris.qe.ConnectContext;
-
-import com.google.common.base.Strings;
-import com.google.common.collect.Lists;
-import com.google.common.collect.Sets;
-
-import org.apache.commons.collections.CollectionUtils;
 import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
 
@@ -372,11 +369,6 @@ public class CreateTableStmt extends DdlStmt {
 columnDef.analyze(engineName.equals("olap"));
 
 if (columnDef.getType().isArrayType()) {
-ArrayType tp = (ArrayType) columnDef.getType();
-if (!tp.getItemType().getPrimitiveType().isIntegerType() &&
-!tp.getItemType().getPrimitiveType().isCharFamily()) {
-throw new AnalysisException("Array column just support 
INT/VARCHAR sub-type");
-}
 if (columnDef.getAggregateType() != null && 
columnDef.getAggregateType() != AggregateType.NONE) {
 throw new AnalysisException("Array column can't support 
aggregation " + columnDef.getAggregateType());
 }
diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/TypeDef.java 
b/fe/fe-core/src/main/java/org/apache/doris/analysis/TypeDef.java
index 6bce9b5..d81d633 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/TypeDef.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/TypeDef.java
@@ -92,8 +92,10 @@ public class TypeDef implements ParseNode {
 throw new AnalysisException("Unsupported data type: " + type.toSql());
   }
   if (type.isArrayType()) {
-ScalarType itemType = (ScalarType) ((ArrayType) type).getItemType();
-analyzeNestedType(itemType);
+Type itemType = ((ArrayType) type).getItem

[incubator-doris] 02/02: [feature-wip][array-type] Support ArrayLiteral in SQL. (#8089)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch array-type
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git

commit 28ea295ec8318c88258e2644df8d8701c2bd4180
Author: Adonis Ling 
AuthorDate: Fri Feb 18 10:59:57 2022 +0800

[feature-wip][array-type] Support ArrayLiteral in SQL. (#8089)

Please refer to #8074
---
 be/src/olap/column_vector.cpp  |  21 ++--
 be/src/olap/rowset/segment_v2/column_reader.cpp|   8 +-
 be/src/olap/rowset/segment_v2/column_reader.h  |  13 +++
 be/src/olap/rowset/segment_v2/column_writer.cpp|  10 +-
 be/src/olap/rowset/segment_v2/column_writer.h  |   5 +-
 be/src/runtime/collection_value.cpp|  16 ++-
 be/src/runtime/mysql_result_writer.cpp |   6 +-
 be/src/runtime/raw_value.cpp   |   1 +
 be/src/runtime/row_batch.cpp   |  10 +-
 be/src/runtime/tuple.cpp   |  29 +++---
 .../segment_v2/column_reader_writer_test.cpp   |  24 +
 fe/fe-core/src/main/cup/sql_parser.cup |  14 +++
 .../org/apache/doris/analysis/ArrayLiteral.java|  28 +++--
 .../org/apache/doris/analysis/CreateTableStmt.java |  10 +-
 .../main/java/org/apache/doris/analysis/Expr.java  |   3 +-
 .../java/org/apache/doris/catalog/ArrayType.java   |   4 +
 .../main/java/org/apache/doris/catalog/Column.java |  13 ++-
 .../main/java/org/apache/doris/catalog/Type.java   |   4 +-
 .../org/apache/doris/analysis/ColumnDefTest.java   |  18 +++-
 .../apache/doris/analysis/InsertArrayStmtTest.java | 114 +
 .../org/apache/doris/utframe/UtFrameUtils.java |   9 ++
 21 files changed, 306 insertions(+), 54 deletions(-)

diff --git a/be/src/olap/column_vector.cpp b/be/src/olap/column_vector.cpp
index 3340628..0237b62 100644
--- a/be/src/olap/column_vector.cpp
+++ b/be/src/olap/column_vector.cpp
@@ -210,14 +210,19 @@ void ArrayColumnVectorBatch::prepare_for_read(size_t 
start_idx, size_t size, boo
 DCHECK(start_idx + size <= capacity());
 for (size_t i = 0; i < size; ++i) {
 if (!is_null_at(start_idx + i)) {
-_data[start_idx + i] = CollectionValue(
-
_elements->mutable_cell_ptr(*(_offsets->scalar_cell_ptr(start_idx + i))),
-*(_offsets->scalar_cell_ptr(start_idx + i + 1)) -
-*(_offsets->scalar_cell_ptr(start_idx + i)),
-item_has_null,
-_elements->is_nullable() ? 
const_cast(&_elements->null_signs()[*(
-   
_offsets->scalar_cell_ptr(start_idx + i))])
- : nullptr);
+auto next_offset = *(_offsets->scalar_cell_ptr(start_idx + i + 1));
+auto offset = *(_offsets->scalar_cell_ptr(start_idx + i));
+uint32_t length = next_offset - offset;
+if (length == 0) {
+_data[start_idx + i] = CollectionValue(length);
+} else {
+_data[start_idx + i] = CollectionValue(
+_elements->mutable_cell_ptr(offset),
+length,
+item_has_null,
+_elements->is_nullable() ? 
const_cast(&_elements->null_signs()[offset])
+ : nullptr);
+}
 }
 }
 }
diff --git a/be/src/olap/rowset/segment_v2/column_reader.cpp 
b/be/src/olap/rowset/segment_v2/column_reader.cpp
index 20d2918..9530d80 100644
--- a/be/src/olap/rowset/segment_v2/column_reader.cpp
+++ b/be/src/olap/rowset/segment_v2/column_reader.cpp
@@ -127,7 +127,7 @@ Status ColumnReader::init() {
 "Bad file $0: invalid column index type $1", 
_path_desc.filepath, index_meta.type()));
 }
 }
-if (_ordinal_index_meta == nullptr) {
+if (!is_empty() && _ordinal_index_meta == nullptr) {
 return Status::Corruption(strings::Substitute(
 "Bad file $0: missing ordinal index for column $1", 
_path_desc.filepath, _meta.column_id()));
 }
@@ -339,6 +339,10 @@ Status ColumnReader::seek_at_or_before(ordinal_t ordinal, 
OrdinalPageIndexIterat
 }
 
 Status ColumnReader::new_iterator(ColumnIterator** iterator) {
+if (is_empty()) {
+*iterator = new EmptyFileColumnIterator();
+return Status::OK();
+}
 if (is_scalar_type((FieldType)_meta.type())) {
 *iterator = new FileColumnIterator(this);
 return Status::OK();
@@ -427,7 +431,7 @@ Status ArrayFileColumnIterator::next_batch(size_t* n, 
ColumnBlockView* dst, bool
 
 // read item
 size_t item_size = array_batch->get_item_size(dst->current_offset(), *n);
-if (item_size > 0) {
+if (item_size >= 0) {
 bool item_has_null = false;
 ColumnVectorBatch* item_vector_batch = array_batch->elements();
 
diff --git a/be/src/olap/rowset/segment

[GitHub] [incubator-doris] zbtzbtzbt commented on pull request #8193: [refactor](storage)(olap) improve pageDecoder using final keyword and reuse BinaryDictPageDecoder's members

[GitBox]

zbtzbtzbt commented on pull request #8193:
URL: https://github.com/apache/incubator-doris/pull/8193#issuecomment-1060263635


   I add a run-able link for `final` speed test. https://godbolt.org/z/v3MbqTrKf


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #8355: [improvement][fix](insert)(replay) support SHOW LAST INSERT stmt and fix json replay bug

[GitBox]

EmmyMiao87 commented on a change in pull request #8355:
URL: https://github.com/apache/incubator-doris/pull/8355#discussion_r820435667



##
File path: 
fe/fe-core/src/main/java/org/apache/doris/analysis/ShowLastInsertStmt.java
##
@@ -0,0 +1,55 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.analysis;
+
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.ScalarType;
+import org.apache.doris.qe.ShowResultSetMetaData;
+
+// SHOW LAST INSERT
+public class ShowLastInsertStmt extends ShowStmt {
+private static final ShowResultSetMetaData META_DATA =
+ShowResultSetMetaData.builder()
+.addColumn(new Column("TransactionId", 
ScalarType.createVarchar(128)))
+.addColumn(new Column("Label", 
ScalarType.createVarchar(128)))
+.addColumn(new Column("Database", 
ScalarType.createVarchar(128)))
+.addColumn(new Column("Table", 
ScalarType.createVarchar(128)))
+.addColumn(new Column("TransactionStatus", 
ScalarType.createVarchar(64)))
+.addColumn(new Column("LoadedRows", 
ScalarType.createVarchar(128)))
+.addColumn(new Column("FilteredRows", 
ScalarType.createVarchar(128)))
+.build();
+
+@Override
+public void analyze(Analyzer analyzer) {

Review comment:
   check priv ？




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris] 01/01: Revert "[refactor] remove unused new_in_predicate code (#8263)"

[yangzhg]

This is an automated email from the ASF dual-hosted git repository.

yangzhg pushed a commit to branch revert-8263-remove_code
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git

commit 4ce2b4a1f75f7aee14a03f691cf1ddf103f211ba
Author: Zhengguo Yang <780531...@qq.com>
AuthorDate: Mon Mar 7 15:44:25 2022 +0800

Revert "[refactor] remove unused new_in_predicate code (#8263)"

This reverts commit 757e35744d4f6319e936fca84b4be13cf043a578.
---
 be/src/common/daemon.cpp  |   2 +
 be/src/exprs/CMakeLists.txt   |   1 +
 be/src/exprs/new_in_predicate.cpp | 185 
 be/src/exprs/new_in_predicate.h   | 348 ++
 4 files changed, 536 insertions(+)

diff --git a/be/src/common/daemon.cpp b/be/src/common/daemon.cpp
index 04625e3..044feda 100644
--- a/be/src/common/daemon.cpp
+++ b/be/src/common/daemon.cpp
@@ -37,6 +37,7 @@
 #include "exprs/json_functions.h"
 #include "exprs/like_predicate.h"
 #include "exprs/math_functions.h"
+#include "exprs/new_in_predicate.h"
 #include "exprs/operators.h"
 #include "exprs/string_functions.h"
 #include "exprs/table_function/dummy_table_functions.h"
@@ -249,6 +250,7 @@ void Daemon::init(int argc, char** argv, const 
std::vector& paths) {
 StringFunctions::init();
 ArrayFunctions::init();
 CastFunctions::init();
+InPredicate::init();
 MathFunctions::init();
 EncryptionFunctions::init();
 TimestampFunctions::init();
diff --git a/be/src/exprs/CMakeLists.txt b/be/src/exprs/CMakeLists.txt
index f266fd9..c71f23e 100644
--- a/be/src/exprs/CMakeLists.txt
+++ b/be/src/exprs/CMakeLists.txt
@@ -41,6 +41,7 @@ add_library(Exprs
   expr.cpp
   expr_context.cpp
   in_predicate.cpp
+  new_in_predicate.cpp
   bloomfilter_predicate.cpp
   block_bloom_filter_avx_impl.cc
   block_bloom_filter_impl.cc
diff --git a/be/src/exprs/new_in_predicate.cpp 
b/be/src/exprs/new_in_predicate.cpp
new file mode 100644
index 000..b0a6390
--- /dev/null
+++ b/be/src/exprs/new_in_predicate.cpp
@@ -0,0 +1,185 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "exprs/new_in_predicate.h"
+
+#include 
+
+#include "exprs/anyval_util.h"
+#include "runtime/string_value.hpp"
+
+namespace doris {
+
+void InPredicate::init() {}
+
+// Templated getter functions for extracting 'SetType' values from AnyVals
+template 
+SetType get_val(const FunctionContext::TypeDesc* type, const T& x) {
+DCHECK(!x.is_null);
+return x.val;
+}
+
+template <>
+StringValue get_val(const FunctionContext::TypeDesc* type, const StringVal& x) 
{
+DCHECK(!x.is_null);
+return StringValue::from_string_val(x);
+}
+
+template <>
+DateTimeValue get_val(const FunctionContext::TypeDesc* type, const 
DateTimeVal& x) {
+return DateTimeValue::from_datetime_val(x);
+}
+
+template <>
+DecimalV2Value get_val(const FunctionContext::TypeDesc* type, const 
DecimalV2Val& x) {
+return DecimalV2Value::from_decimal_val(x);
+}
+
+template 
+void InPredicate::set_lookup_prepare(FunctionContext* ctx,
+ FunctionContext::FunctionStateScope 
scope) {
+if (scope != FunctionContext::FRAGMENT_LOCAL) {
+return;
+}
+
+SetLookupState* state = new SetLookupState;
+state->type = ctx->get_arg_type(0);
+state->contains_null = false;
+for (int i = 1; i < ctx->get_num_args(); ++i) {
+DCHECK(ctx->is_arg_constant(i));
+T* arg = reinterpret_cast(ctx->get_constant_arg(i));
+if (arg->is_null) {
+state->contains_null = true;
+} else {
+state->val_set.insert(get_val(state->type, *arg));
+}
+}
+ctx->set_function_state(scope, state);
+}
+
+template 
+void InPredicate::set_lookup_close(FunctionContext* ctx,
+   FunctionContext::FunctionStateScope scope) {
+if (scope != FunctionContext::FRAGMENT_LOCAL) {
+return;
+}
+SetLookupState* state =
+
reinterpret_cast*>(ctx->get_function_state(scope));
+delete state;
+}
+
+template 
+BooleanVal InPredicate::templated_in(FunctionContext* ctx, const T& val, int 
num_args,
+ const T* args) {
+if (val.is_null) {
+ 

[incubator-doris] branch revert-8263-remove_code created (now 4ce2b4a)

[yangzhg]

This is an automated email from the ASF dual-hosted git repository.

yangzhg pushed a change to branch revert-8263-remove_code
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git.


  at 4ce2b4a  Revert "[refactor] remove unused new_in_predicate code 
(#8263)"

This branch includes the following new commits:

 new 4ce2b4a  Revert "[refactor] remove unused new_in_predicate code 
(#8263)"

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] yangzhg opened a new pull request #8372: [revert] Revert "[refactor] remove unused new_in_predicate code"

[GitBox]

yangzhg opened a new pull request #8372:
URL: https://github.com/apache/incubator-doris/pull/8372


   Reverts apache/incubator-doris#8263


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] zhangstar333 opened a new pull request #8373: [Refactor] to_bitmap(-1) return NULL instead of return parse failed error_message

[GitBox]

zhangstar333 opened a new pull request #8373:
URL: https://github.com/apache/incubator-doris/pull/8373


   # Proposed changes
   
   Issue Number: close #8357 
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8372: [revert] Revert "[refactor] remove unused new_in_predicate code"

[GitBox]

github-actions[bot] commented on pull request #8372:
URL: https://github.com/apache/incubator-doris/pull/8372#issuecomment-1060283749






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] yangzhg commented on pull request #8372: [revert] Revert "[refactor] remove unused new_in_predicate code"

[GitBox]

yangzhg commented on pull request #8372:
URL: https://github.com/apache/incubator-doris/pull/8372#issuecomment-1060286755


   merge it to fix p0 test error


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] yangzhg merged pull request #8372: [revert] Revert "[refactor] remove unused new_in_predicate code"

[GitBox]

yangzhg merged pull request #8372:
URL: https://github.com/apache/incubator-doris/pull/8372


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [incubator-doris] zz-jason opened a new issue #8374: [Feature] support TiKV as an option for metadata storage

[GitBox]

zz-jason opened a new issue #8374:
URL: https://github.com/apache/incubator-doris/issues/8374


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Description
   
   TiKV is a low latency, strongly consistent, highly available key-value 
database which supports ACID transactions. It has been used to store metadata 
for some infrastructures and services, for example:
   
   1. Metadata of the Object Storage Service in JD Cloud: 
https://www.cncf.io/blog/2019/11/26/case-study-tikv-in-jd-cloud/
   2. Metadata of the JuiceFS, which is a POSIX file system designed for 
cloud-native environment: 
https://juicefs.com/blog/en/posts/juicefs-release-v016/
   
   TiKV is also the storage of TiDB, a distributed HTAP database used by 1500+ 
users in production. Which gives it the following strengths:
   1. Low maintenance overhead for existing TiDB users, the SOPs, KBs of TiDB 
storage layer can be reused
   2. No need to learn how to maintain another system for metadata storage
   
   ### Use case
   
   By adopting TiKV, it enables us to solve the following problems:
   1. Doris stores metadata in local disk, caches it in memory in a specific 
format. It might be inconvenient to view or modify it
   2. As the cluster and data grow, there might be more metadata needed to be 
stored, the current metadata storage mechanism might be a bottleneck
   
   ### Related issues
   
   N/A
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[incubator-doris] branch master updated (477b87c -> d9c2c2c)

[yangzhg]

This is an automated email from the ASF dual-hosted git repository.

yangzhg pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git.


from 477b87c  [feature](vec) Support update stmt in vec query engine (#8296)
 add d9c2c2c  Revert "[refactor] remove unused new_in_predicate code 
(#8263)" (#8372)

No new revisions were added by this update.

Summary of changes:
 be/src/common/daemon.cpp  |   2 +
 be/src/exprs/CMakeLists.txt   |   1 +
 be/src/exprs/new_in_predicate.cpp | 185 
 be/src/exprs/new_in_predicate.h   | 348 ++
 4 files changed, 536 insertions(+)
 create mode 100644 be/src/exprs/new_in_predicate.cpp
 create mode 100644 be/src/exprs/new_in_predicate.h

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org