[GitHub] [incubator-doris] morningman commented on issue #7129: [Feature] Limit cluster resource usage in user granularity

2021-11-15 Thread GitBox
morningman commented on issue #7129: URL: https://github.com/apache/incubator-doris/issues/7129#issuecomment-969974125 What if the request exceed the limit? return error or slow down? And is there any other system we can refer to? -- This is an automated message from the Apache Git Se

[GitHub] [incubator-doris] tianhui5 commented on issue #7129: [Feature] Limit cluster resource usage in user granularity

2021-11-15 Thread GitBox
tianhui5 commented on issue #7129: URL: https://github.com/apache/incubator-doris/issues/7129#issuecomment-969971912 I'm prefer to add syntax `LIMITER` on behalf of the frequency limiter. This limiter is an abstract concept, it can have many kinds of types like query or stream load. So

[GitHub] [incubator-doris] tianhui5 opened a new issue #7129: [Feature] Limit cluster resource usage in user granularity

2021-11-15 Thread GitBox
tianhui5 opened a new issue #7129: URL: https://github.com/apache/incubator-doris/issues/7129 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description I

[GitHub] [incubator-doris] morningman commented on issue #6806: Release note 0.15.0

2021-11-15 Thread GitBox
morningman commented on issue #6806: URL: https://github.com/apache/incubator-doris/issues/6806#issuecomment-969915020 > @morningman Will vectorization query optimization show up in this release? No, it is still WIP. But you can try part of this feature by using Palo(powered by Dori

svn commit: r50959 - in /dev/incubator/doris: ./ 0.15/ 0.15/0.15.0-rc03/

2021-11-15 Thread morningman
Author: morningman Date: Tue Nov 16 06:28:49 2021 New Revision: 50959 Log: Add 0.15.0-rc03 Added: dev/incubator/doris/0.15/ dev/incubator/doris/0.15/0.15.0-rc03/ dev/incubator/doris/0.15/0.15.0-rc03/apache-doris-0.15.0-incubating-src.tar.gz (with props) dev/incubator/doris/0.

[GitHub] [incubator-doris] yangzhg commented on pull request #7109: [Doc]Modify the wrong comment of the ScanTime

2021-11-15 Thread GitBox
yangzhg commented on pull request #7109: URL: https://github.com/apache/incubator-doris/pull/7109#issuecomment-969900806 Please also modify the English docs in `docs/en/administrator-guide/running-profile.md` -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [incubator-doris] wuyunfeng edited a comment on pull request #6990: Doris on es 聚合下推

2021-11-15 Thread GitBox
wuyunfeng edited a comment on pull request #6990: URL: https://github.com/apache/incubator-doris/pull/6990#issuecomment-969890986 @Userwhite As discussed in our online meeting,we should move the predicate-transform from BE (backends) to FE (frontends) which would contribute significant ef

[GitHub] [incubator-doris] wuyunfeng commented on pull request #6990: Doris on es 聚合下推

2021-11-15 Thread GitBox
wuyunfeng commented on pull request #6990: URL: https://github.com/apache/incubator-doris/pull/6990#issuecomment-969890986 @Userwhite As discussed in our online meeting,we should move the predicate-transform from BE (backends) to FE (frontends) which would contribute significant effect f

[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7114: [Routine load] Support show offset lag in show routine load stmt

2021-11-15 Thread GitBox
github-actions[bot] commented on pull request #7114: URL: https://github.com/apache/incubator-doris/pull/7114#issuecomment-969872557 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [incubator-doris] morningman commented on a change in pull request #7032: [Heartbeat] Support cancel outdated query after FE restart

2021-11-15 Thread GitBox
morningman commented on a change in pull request #7032: URL: https://github.com/apache/incubator-doris/pull/7032#discussion_r749890518 ## File path: be/src/agent/heartbeat_server.cpp ## @@ -78,6 +78,32 @@ void HeartbeatServer::heartbeat(THeartbeatResult& heartbeat_result, Sta

[GitHub] [incubator-doris] github-actions[bot] commented on pull request #6856: [refactor] replace boost smart ptr with stl

2021-11-15 Thread GitBox
github-actions[bot] commented on pull request #6856: URL: https://github.com/apache/incubator-doris/pull/6856#issuecomment-969846134 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7108: [Bug] Fix the incompatibility of sql mode between Doris and MySQL

2021-11-15 Thread GitBox
github-actions[bot] commented on pull request #7108: URL: https://github.com/apache/incubator-doris/pull/7108#issuecomment-969845938 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[incubator-doris] branch master updated: [Blog] Example of binlog load usage (#7080)

2021-11-15 Thread morningman
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 5710cf8 [Blog] Example of binlog l

[GitHub] [incubator-doris] morningman merged pull request #7080: [Blog]Example of binlog load usage

2021-11-15 Thread GitBox
morningman merged pull request #7080: URL: https://github.com/apache/incubator-doris/pull/7080 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: com

[GitHub] [incubator-doris] morningman commented on a change in pull request #7126: [enhancement] Optimize the fail logic of loading data when some backend failed

2021-11-15 Thread GitBox
morningman commented on a change in pull request #7126: URL: https://github.com/apache/incubator-doris/pull/7126#discussion_r749882831 ## File path: be/src/exec/tablet_sink.h ## @@ -295,8 +305,10 @@ class IndexChannel { std::unordered_map _node_channels; // from table

[incubator-doris] branch master updated: [Feature] Support query hive table (#6569)

2021-11-15 Thread lingmiao
This is an automated email from the ASF dual-hosted git repository. lingmiao pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 5b01f7b [Feature] Support query hive

[GitHub] [incubator-doris] EmmyMiao87 merged pull request #6569: [Feature] Support query hive table

2021-11-15 Thread GitBox
EmmyMiao87 merged pull request #6569: URL: https://github.com/apache/incubator-doris/pull/6569 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: com

[GitHub] [incubator-doris] EmmyMiao87 closed issue #6568: [Feature] Support query hive table

2021-11-15 Thread GitBox
EmmyMiao87 closed issue #6568: URL: https://github.com/apache/incubator-doris/issues/6568 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-

[GitHub] [incubator-doris] github-actions[bot] commented on pull request #6515: [Memory] BitShufflePageDecoder use memory allocated by ChunkAllocator instead of Faststring

2021-11-15 Thread GitBox
github-actions[bot] commented on pull request #6515: URL: https://github.com/apache/incubator-doris/pull/6515#issuecomment-969835258 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7105: [BUG] fix profile not working with sql_cache enabled

2021-11-15 Thread GitBox
github-actions[bot] commented on pull request #7105: URL: https://github.com/apache/incubator-doris/pull/7105#issuecomment-969830587 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [incubator-doris] morningman opened a new pull request #7128: [Dependency] Add breakpad for minidump

2021-11-15 Thread GitBox
morningman opened a new pull request #7128: URL: https://github.com/apache/incubator-doris/pull/7128 ## Proposed changes Related #7123 ## Types of changes What types of changes does your code introduce to Doris? _Put an `x` in the boxes that apply_ - [ ] Bugfi

[GitHub] [incubator-doris] yangzhg opened a new pull request #7126: [enhancement] Optimize the fail logic of loading data when some backend failed

2021-11-15 Thread GitBox
yangzhg opened a new pull request #7126: URL: https://github.com/apache/incubator-doris/pull/7126 mark the load job fail when more than a half of replica write failed of a tablet, the code before is counting all replica has more than a half write failed. ## Proposed changes

[GitHub] [incubator-doris] drgnchan opened a new pull request #7127: correct getLogger argument

2021-11-15 Thread GitBox
drgnchan opened a new pull request #7127: URL: https://github.com/apache/incubator-doris/pull/7127 ## Proposed changes correct getLogger argument ## Types of changes What types of changes does your code introduce to Doris? _Put an `x` in the boxes that apply_ -

[GitHub] [incubator-doris] yangzhg opened a new issue #7125: [enhancement] Optimize the fail logic of loading data when some backend failed

2021-11-15 Thread GitBox
yangzhg opened a new issue #7125: URL: https://github.com/apache/incubator-doris/issues/7125 When loading data, when a tablet sink node writes data to other be, the logic for judging the write failure is that the total number of failed nodes exceeds half of the number of replicas, but a s

[GitHub] [incubator-doris] morningman commented on a change in pull request #7124: [Feature] Add minidump support

2021-11-15 Thread GitBox
morningman commented on a change in pull request #7124: URL: https://github.com/apache/incubator-doris/pull/7124#discussion_r749852913 ## File path: be/src/service/doris_main.cpp ## @@ -57,6 +58,8 @@ #include "util/thrift_server.h" #include "util/uid_util.h" +#include "clie

[GitHub] [incubator-doris] yangzhg commented on a change in pull request #7124: [Feature] Add minidump support

2021-11-15 Thread GitBox
yangzhg commented on a change in pull request #7124: URL: https://github.com/apache/incubator-doris/pull/7124#discussion_r749852606 ## File path: be/src/service/doris_main.cpp ## @@ -57,6 +58,8 @@ #include "util/thrift_server.h" #include "util/uid_util.h" +#include "client/

[GitHub] [incubator-doris] tianhui5 commented on pull request #7087: add replica num limit per tablet

2021-11-15 Thread GitBox
tianhui5 commented on pull request #7087: URL: https://github.com/apache/incubator-doris/pull/7087#issuecomment-969707666 > hi @tianhui5 the FE unit test failed at DynamicPartitionTableTest.testSetDynamicPartitionReplicationNum. You can see details here: ![image](https://user-images.githu

[incubator-doris] branch master updated (5aaf24b -> ccb1ea8)

2021-11-15 Thread morningman
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 5aaf24b [Compile] Remove unused import (#7112) add ccb1ea8 [Refactor] logger error in BDBStateCha

[incubator-doris] branch master updated (ccb1ea8 -> e476e15)

2021-11-15 Thread morningman
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from ccb1ea8 [Refactor] logger error in BDBStateChangeListener.java (#7101) add e476e15 [Bug] Fix inap

[GitHub] [incubator-doris] morningman merged pull request #7111: [Bug] Fix inappropriate Cmake option used to build ZSTD

2021-11-15 Thread GitBox
morningman merged pull request #7111: URL: https://github.com/apache/incubator-doris/pull/7111 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: com

[GitHub] [incubator-doris] morningman merged pull request #7101: logger error in BDBStateChangeListener.java

2021-11-15 Thread GitBox
morningman merged pull request #7101: URL: https://github.com/apache/incubator-doris/pull/7101 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: com

[GitHub] [incubator-doris] morningman commented on pull request #7087: add replica num limit per tablet

2021-11-15 Thread GitBox
morningman commented on pull request #7087: URL: https://github.com/apache/incubator-doris/pull/7087#issuecomment-969674663 hi @tianhui5 the FE unit test failed at DynamicPartitionTableTest.testSetDynamicPartitionReplicationNum. You can see details here: ![image](https://user-images.

[GitHub] [incubator-doris] morningman opened a new pull request #7124: [Feature] Add minidump support

2021-11-15 Thread GitBox
morningman opened a new pull request #7124: URL: https://github.com/apache/incubator-doris/pull/7124 ## Proposed changes Add minidump support. Now minidump file will be created when BE crashes. And user can manually trigger a minidump by sending SIGUSR1 to BE process. Mor

[GitHub] [incubator-doris] morningman opened a new issue #7123: [Feature] Add minidump support

2021-11-15 Thread GitBox
morningman opened a new issue #7123: URL: https://github.com/apache/incubator-doris/issues/7123 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description

[GitHub] [incubator-doris] github-actions[bot] commented on pull request #6569: [Feature] Support query hive table

2021-11-15 Thread GitBox
github-actions[bot] commented on pull request #6569: URL: https://github.com/apache/incubator-doris/pull/6569#issuecomment-968763546 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [incubator-doris] EmmyMiao87 commented on issue #7008: [Enhance] use predicate if in bitmap_union_count() can not hit rollup.

2021-11-15 Thread GitBox
EmmyMiao87 commented on issue #7008: URL: https://github.com/apache/incubator-doris/issues/7008#issuecomment-968694069 > 还有另一个问题也反馈一下,sum(case when)这种写法,也属于聚合函数里面用表达式 当存在隐式转换时,也可能不会命中rollup,比如从int转化为bigint就命中不了了 猜测转换的原因是怕数据溢出吧 > > ``` > explain > select dt, > sum(arri

[GitHub] [incubator-doris] zbtzbtzbt edited a comment on issue #7008: [Enhance] use predicate if in bitmap_union_count() can not hit rollup.

2021-11-15 Thread GitBox
zbtzbtzbt edited a comment on issue #7008: URL: https://github.com/apache/incubator-doris/issues/7008#issuecomment-968689508 还有另一个问题也反馈一下,sum(case when)这种写法,也属于聚合函数里面用表达式 当存在隐式转换时,也可能不会命中rollup,比如从int转化为bigint就命中不了了 猜测转换的原因是怕数据溢出吧 ``` explain select dt, sum(arrived_w

[GitHub] [incubator-doris] zbtzbtzbt commented on issue #7008: [Enhance] use predicate if in bitmap_union_count() can not hit rollup.

2021-11-15 Thread GitBox
zbtzbtzbt commented on issue #7008: URL: https://github.com/apache/incubator-doris/issues/7008#issuecomment-968689508 还有另一个问题也反馈一下,sum(case when)这种写法 当存在隐式转换时,也可能不会命中rollup,比如从int转化为bigint就命中不了了 猜测转换的原因是怕数据溢出吧 ``` explain select dt, sum(arrived_waybill_cnt) as arrive

[GitHub] [incubator-doris] zbtzbtzbt commented on issue #7008: [Enhance] use predicate if in bitmap_union_count() can not hit rollup.

2021-11-15 Thread GitBox
zbtzbtzbt commented on issue #7008: URL: https://github.com/apache/incubator-doris/issues/7008#issuecomment-968686655 这里是用来计算每日新客数目 ``` select dt ,bitmap_union_count(if(is_day_new_user=1,buy_user_id,null)) as newuser_num -- 新客数 ,sum(sale_amt) as sale_amt -- 销售额

[GitHub] [incubator-doris] EmmyMiao87 commented on issue #7008: [Enhance] use predicate if in bitmap_union_count() can not hit rollup.

2021-11-15 Thread GitBox
EmmyMiao87 commented on issue #7008: URL: https://github.com/apache/incubator-doris/issues/7008#issuecomment-968676882 我有一个偏业务的小问题,什么类型的业务查询会有这种 bitmap_union_count(IF(k2=0,v1,NULL)) AS v1_num 需求呢? 能举个具体的业务例子嘛? -- This is an automated message from the Apache Git Service. To respond

[GitHub] [incubator-doris] EmmyMiao87 edited a comment on issue #7102: [Feature] Reduce useless cast

2021-11-15 Thread GitBox
EmmyMiao87 edited a comment on issue #7102: URL: https://github.com/apache/incubator-doris/issues/7102#issuecomment-968508822 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [incubator-doris] EmmyMiao87 edited a comment on issue #7102: [Feature] Reduce useless cast

2021-11-15 Thread GitBox
EmmyMiao87 edited a comment on issue #7102: URL: https://github.com/apache/incubator-doris/issues/7102#issuecomment-968508822 # 如何梳理所有 Expr 中各个类型转换是否存在无意义的 cast 1. 梳理各个不同表达式,及对应的cast 规则并记录。 2. 判断 cast 规则是否合理。 3. 是否能统一 cast 规则。 ## 梳理 由于不同的表达式对多参兼容性的处理规则不太相同,所以首先需要整理各个表达式都使

[GitHub] [incubator-doris] EmmyMiao87 edited a comment on issue #7102: [Feature] Reduce useless cast

2021-11-15 Thread GitBox
EmmyMiao87 edited a comment on issue #7102: URL: https://github.com/apache/incubator-doris/issues/7102#issuecomment-968508822 # 如何梳理所有 Expr 中各个类型转换是否存在无意义的 cast 1. 梳理各个不同表达式,及对应的cast 规则并记录。 2. 判断 cast 规则是否合理。 3. 是否能统一 cast 规则。 ## 梳理 由于不同的表达式对多参兼容性的处理规则不太相同,所以首先需要整理各个表达式都使

[GitHub] [incubator-doris] EmmyMiao87 edited a comment on issue #7102: [Feature] Reduce useless cast

2021-11-15 Thread GitBox
EmmyMiao87 edited a comment on issue #7102: URL: https://github.com/apache/incubator-doris/issues/7102#issuecomment-968508822 # 如何梳理所有 Expr 中各个类型转换是否存在无意义的 cast 1. 梳理各个不同表达式,及对应的cast 规则并记录。 2. 判断 cast 规则是否合理。 3. 是否能统一 cast 规则。 ## 梳理 由于不同的表达式对多参兼容性的处理规则不太相同,所以首先需要整理各个表达式都使

[GitHub] [incubator-doris] EmmyMiao87 edited a comment on issue #7102: [Feature] Reduce useless cast

2021-11-15 Thread GitBox
EmmyMiao87 edited a comment on issue #7102: URL: https://github.com/apache/incubator-doris/issues/7102#issuecomment-968508822 # 如何梳理所有 Expr 中各个类型转换是否存在无意义的 cast 1. 梳理各个不同表达式,及对应的cast 规则并记录。 2. 判断 cast 规则是否合理。 3. 是否能统一 cast 规则。 ## 梳理 由于不同的表达式对多参兼容性的处理规则不太相同,所以首先需要整理各个表达式都使

[GitHub] [incubator-doris] EmmyMiao87 edited a comment on issue #7102: [Feature] Reduce useless cast

2021-11-15 Thread GitBox
EmmyMiao87 edited a comment on issue #7102: URL: https://github.com/apache/incubator-doris/issues/7102#issuecomment-968508822 # 如何梳理所有 Expr 中各个类型转换是否存在无意义的 cast 1. 梳理各个不同表达式,及对应的cast 规则并记录。 2. 判断 cast 规则是否合理。 3. 是否能统一 cast 规则。 ## 梳理 由于不同的表达式对多参兼容性的处理规则不太相同,所以首先需要整理各个表达式都使

[GitHub] [incubator-doris] EmmyMiao87 edited a comment on issue #7102: [Feature] Reduce useless cast

2021-11-15 Thread GitBox
EmmyMiao87 edited a comment on issue #7102: URL: https://github.com/apache/incubator-doris/issues/7102#issuecomment-968508822 # 如何梳理所有 Expr 中各个类型转换是否存在无意义的 cast 1. 梳理各个不同表达式,及对应的cast 规则并记录。 2. 判断 cast 规则是否合理。 3. 是否能统一 cast 规则。 ## 梳理 由于不同的表达式对多参兼容性的处理规则不太相同,所以首先需要整理各个表达式都使

[GitHub] [incubator-doris] EmmyMiao87 edited a comment on issue #7102: [Feature] Reduce useless cast

2021-11-15 Thread GitBox
EmmyMiao87 edited a comment on issue #7102: URL: https://github.com/apache/incubator-doris/issues/7102#issuecomment-968508822 # 如何梳理所有 Expr 中各个类型转换是否存在无意义的 cast 1. 梳理各个不同表达式,及对应的cast 规则并记录。 2. 判断 cast 规则是否合理。 3. 是否能统一 cast 规则。 ## 梳理 由于不同的表达式对多参兼容性的处理规则不太相同,所以首先需要整理各个表达式都使