[GitHub] [incubator-doris] EmmyMiao87 edited a comment on issue #7102: [Feature] Reduce useless cast
EmmyMiao87 edited a comment on issue #7102: URL: https://github.com/apache/incubator-doris/issues/7102#issuecomment-968508822 # 如何梳理所有 Expr 中各个类型转换是否存在无意义的 cast 1. 梳理各个不同表达式，及对应的cast 规则并记录。 2. 判断 cast 规则是否合理。 3. 是否能统一 cast 规则。 ## 梳理 由于不同的表达式对多参兼容性的处理规则不太相同，所以首先需要整理各个表达式都使用了哪些 cast 规则。 比如对于 BetweenPredicate 来说，c1 between a and b 。 则需要找到 c1, a, b 三个参数都能兼容的类型。 ```c1(int) between a (bigint) and b (varchar)``` 则会转化为 ```cast(c1 as varchar) between cast(a as varchar) and b(varchar)``` 但对于 BinaryPredicate 来说，a = b 。当 a(int) = b(string) 的时候，他们的兼容类型是 int，则会转化为 ``` a = cast(b as int) ``` 下表为不同表达式，以及他们对应的 cast 规则记录。 | Expr | cast 规则 | 是否正确 | |---|---| ---| | BetweenPredicate | Analyzer.castAllToCompatibleType()| | | InPredicate | | | | BinaryPredicate | | | | ### 规则正确性判断 1. 对照 Mysql 行为，尽量保持一致 2. 查看 be 函数实现，是否存在该类型的函数。 3. 那种兼容策略性能更好，比如 int 和 date 对齐方式不同。 ## 规则明细 Analyzer.castAllToCompatibleType(List exprs) 转化规则 将所有 Expr 转成一个大家都可兼容的类型 | Type1 | Type2 | CompatibleType | After swapping left and right | | --- | --- | --- | ---| | int | bigint | bigint| bigint | ## 是否能精简统一各个cast 规则 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 edited a comment on issue #7102: [Feature] Reduce useless cast
EmmyMiao87 edited a comment on issue #7102: URL: https://github.com/apache/incubator-doris/issues/7102#issuecomment-968508822 # 如何梳理所有 Expr 中各个类型转换是否存在无意义的 cast 1. 梳理各个不同表达式，及对应的cast 规则并记录。 2. 判断 cast 规则是否合理。 3. 是否能统一 cast 规则。 ## 梳理 由于不同的表达式对多参兼容性的处理规则不太相同，所以首先需要整理各个表达式都使用了哪些 cast 规则。 比如对于 BetweenPredicate 来说，c1 between a and b 。 则需要找到 c1, a, b 三个参数都能兼容的类型。 ```c1(int) between a (bigint) and b (varchar)``` 则会转化为 ```cast(c1 as varchar) between cast(a as varchar) and b(varchar)``` 但对于 BinaryPredicate 来说，a = b 。当 a(int) = b(string) 的时候，他们的兼容类型是 int，则会转化为 ``` a = cast(b as int) ``` 下表为不同表达式，以及他们对应的 cast 规则记录。 | Expr | cast 规则 | 是否正确 | |---|---| ---| | BetweenPredicate | Analyzer.castAllToCompatibleType()| | | InPredicate | | | | BinaryPredicate | | | ### 规则正确性判断 1. 对照 Mysql 行为，尽量保持一致 2. 查看 be 函数实现，是否存在该类型的函数。 3. 那种兼容策略性能更好，比如 int 和 date 对齐方式不同。 ## 规则明细 Analyzer.castAllToCompatibleType(List exprs) 转化规则 将所有 Expr 转成一个大家都可兼容的类型 | Type1 | Type2 | CompatibleType | After swapping left and right | | --- | --- | --- | ---| | int | bigint | bigint| bigint | ## 是否能精简统一各个cast 规则 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 edited a comment on issue #7102: [Feature] Reduce useless cast
EmmyMiao87 edited a comment on issue #7102: URL: https://github.com/apache/incubator-doris/issues/7102#issuecomment-968508822 # 如何梳理所有 Expr 中各个类型转换是否存在无意义的 cast 1. 梳理各个不同表达式，及对应的cast 规则并记录。 2. 判断 cast 规则是否合理。 3. 是否能统一 cast 规则。 ## 梳理 由于不同的表达式对多参兼容性的处理规则不太相同，所以首先需要整理各个表达式都使用了哪些 cast 规则。 比如对于 BetweenPredicate 来说，c1 between a and b 。 则需要找到 c1, a, b 三个参数都能兼容的类型。 ```c1(int) between a (bigint) and b (varchar)``` 则会转化为 ```cast(c1 as varchar) between cast(a as varchar) and b(varchar)``` 但对于 BinaryPredicate 来说，a = b 。当 a(int) = b(string) 的时候，他们的兼容类型是 int，则会转化为 ``` a = cast(b as int) ``` 下表为不同表达式，以及他们对应的 cast 规则记录。 | Expr | cast 规则 | |---|---| ---| | BetweenPredicate | Analyzer.castAllToCompatibleType()| | InPredicate | | | BinaryPredicate | | ### 规则正确性判断 1. 对照 Mysql 行为，尽量保持一致 2. 查看 be 函数实现，是否存在该类型的函数。 3. 那种兼容策略性能更好，比如 int 和 date 对齐方式不同。 ## 规则明细 Analyzer.castAllToCompatibleType(List exprs) 转化规则 将所有 Expr 转成一个大家都可兼容的类型 | Type1 | Type2 | CompatibleType | After swapping left and right | 是否正确或可以优化 | | --- | --- | --- | ---| --- | | int | bigint | bigint| bigint | ---| ## 是否能精简统一各个cast 规则 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 edited a comment on issue #7102: [Feature] Reduce useless cast
EmmyMiao87 edited a comment on issue #7102: URL: https://github.com/apache/incubator-doris/issues/7102#issuecomment-968508822 # 如何梳理所有 Expr 中各个类型转换是否存在无意义的 cast 1. 梳理各个不同表达式，及对应的cast 规则并记录。 2. 判断 cast 规则是否合理。 3. 是否能统一 cast 规则。 ## 梳理 由于不同的表达式对多参兼容性的处理规则不太相同，所以首先需要整理各个表达式都使用了哪些 cast 规则。 比如对于 BetweenPredicate 来说，c1 between a and b 。 则需要找到 c1, a, b 三个参数都能兼容的类型。 ```c1(int) between a (bigint) and b (varchar)``` 则会转化为 ```cast(c1 as varchar) between cast(a as varchar) and b(varchar)``` 但对于 BinaryPredicate 来说，a = b 。当 a(int) = b(string) 的时候，他们的兼容类型是 int，则会转化为 ``` a = cast(b as int) ``` 下表为不同表达式，以及他们对应的 cast 规则记录。 | Expr | cast 规则 | |---|---| ---| | BetweenPredicate | Analyzer.castAllToCompatibleType()| | InPredicate | | | BinaryPredicate | | ### 规则正确性判断 1. 对照 Mysql 行为，尽量保持一致 2. 查看 be 函数实现，是否存在该类型的函数。 3. 那种兼容策略性能更好，比如 int 和 date 对齐方式不同。 ## 规则明细 Analyzer.castAllToCompatibleType(List exprs) 转化规则 将所有 Expr 转成一个大家都可兼容的类型 | Type1 | Type2 | CompatibleType | After swapping left and right | 是否正确或可以优化 | | --- | --- | --- | ---| --- | | int | bigint | bigint| bigint | 正确| ## 是否能精简统一各个cast 规则 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 edited a comment on issue #7102: [Feature] Reduce useless cast
EmmyMiao87 edited a comment on issue #7102: URL: https://github.com/apache/incubator-doris/issues/7102#issuecomment-968508822 # 如何梳理所有 Expr 中各个类型转换是否存在无意义的 cast 1. 梳理各个不同表达式，及对应的cast 规则并记录。 2. 判断 cast 规则是否合理。 3. 是否能统一 cast 规则。 ## 梳理 由于不同的表达式对多参兼容性的处理规则不太相同，所以首先需要整理各个表达式都使用了哪些 cast 规则。 比如对于 BetweenPredicate 来说，c1 between a and b 。 则需要找到 c1, a, b 三个参数都能兼容的类型。 ```c1(int) between a (bigint) and b (varchar)``` 则会转化为 ```cast(c1 as varchar) between cast(a as varchar) and b(varchar)``` 但对于 BinaryPredicate 来说，a = b 。当 a(int) = b(string) 的时候，他们的兼容类型是 int，则会转化为 ``` a = cast(b as int) ``` 下表为不同表达式，以及他们对应的 cast 规则记录。 | Expr | cast 规则 | |---|---| | BetweenPredicate | Analyzer.castAllToCompatibleType()| | InPredicate | | | BinaryPredicate | | ### 规则正确性判断 1. 对照 Mysql 行为，尽量保持一致 2. 查看 be 函数实现，是否存在该类型的函数。 3. 那种兼容策略性能更好，比如 int 和 date 对齐方式不同。 ## 规则明细 Analyzer.castAllToCompatibleType(List exprs) 转化规则 将所有 Expr 转成一个大家都可兼容的类型 | Type1 | Type2 | CompatibleType | After swapping left and right | 是否正确或可以优化 | | --- | --- | --- | ---| --- | | int | bigint | bigint| bigint | 正确| ## 是否能精简统一各个cast 规则 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 edited a comment on issue #7102: [Feature] Reduce useless cast
EmmyMiao87 edited a comment on issue #7102: URL: https://github.com/apache/incubator-doris/issues/7102#issuecomment-968508822 # 如何梳理所有 Expr 中各个类型转换是否存在无意义的 cast 1. 梳理各个不同表达式，及对应的cast 规则并记录。 2. 判断 cast 规则是否合理。 3. 是否能统一 cast 规则。 ## 梳理 由于不同的表达式对多参兼容性的处理规则不太相同，所以首先需要整理各个表达式都使用了哪些 cast 规则。 比如对于 BetweenPredicate 来说，c1 between a and b 。 则需要找到 c1, a, b 三个参数都能兼容的类型。 ```c1(int) between a (bigint) and b (varchar)``` 则会转化为 ```cast(c1 as varchar) between cast(a as varchar) and b(varchar)``` 但对于 BinaryPredicate 来说，a = b 。当 a(int) = b(string) 的时候，他们的兼容类型是 int，则会转化为 ``` a = cast(b as int) ``` 下表为不同表达式，以及他们对应的 cast 规则记录。 | Expr | cast 规则 | |---|---| | BetweenPredicate | Analyzer.castAllToCompatibleType()| | InPredicate | | | BinaryPredicate | | ### 规则正确性判断 1. 对照 Mysql 行为，尽量保持一致 2. 查看 be 函数实现，是否存在该类型的函数。 3. 那种兼容策略性能更好，比如 int 和 date 对齐方式不同。 ## 规则明细 Analyzer.castAllToCompatibleType(List exprs) 转化规则 将所有 Expr 转成一个大家都可兼容的类型 | Type1 | Type2 | CompatibleType | After swapping left and right | 是否正确或可以优化 | | --- | --- | --- | ---| --- | | int | bigint | bigint| bigint | 正确| ## 精简代码逻辑 Type.getCmpType(Type t1, Type t2) 精简为从一个二维数组直接取值。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 edited a comment on issue #7102: [Feature] Reduce useless cast
EmmyMiao87 edited a comment on issue #7102: URL: https://github.com/apache/incubator-doris/issues/7102#issuecomment-968508822 # 如何梳理所有 Expr 中各个类型转换是否存在无意义的 cast 1. 梳理各个不同表达式，及对应的cast 规则并记录。 2. 判断 cast 规则是否合理。 3. 是否能统一 cast 规则。 ## 梳理 由于不同的表达式对多参兼容性的处理规则不太相同，所以首先需要整理各个表达式都使用了哪些 cast 规则。 比如对于 BetweenPredicate 来说，c1 between a and b 。 则需要找到 c1, a, b 三个参数都能兼容的类型。 ```c1(int) between a (bigint) and b (varchar)``` 则会转化为 ```cast(c1 as varchar) between cast(a as varchar) and b(varchar)``` 但对于 BinaryPredicate 来说，a = b 。当 a(int) = b(string) 的时候，他们的兼容类型是 int，则会转化为 ``` a = cast(b as int) ``` 下表为不同表达式，以及他们对应的 cast 规则记录。 | Expr | cast 规则 | |---|---| | BetweenPredicate | Analyzer.castAllToCompatibleType()| | InPredicate | | | BinaryPredicate | | ### 规则正确性判断 1. 对照 Mysql 行为，尽量保持一致 2. 查看 be 函数实现，是否存在该类型的函数。 3. 那种兼容策略性能更好，比如 int 和 date 对齐方式不同。 ## 规则明细 Analyzer.castAllToCompatibleType(List exprs) 转化规则 将所有 Expr 转成一个大家都可兼容的类型 | Type1 | Type2 | CompatibleType | After swapping left and right | 是否正确或可以优化 | | --- | --- | --- | ---| --- | | int | bigint | bigint| bigint | 正确| # 精简代码逻辑 Type.getCmpType(Type t1, Type t2) 精简为从一个二维数组直接取值。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 edited a comment on issue #7102: [Feature] Reduce useless cast
EmmyMiao87 edited a comment on issue #7102: URL: https://github.com/apache/incubator-doris/issues/7102#issuecomment-968508822 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on issue #7008: [Enhance] use predicate if in bitmap_union_count() can not hit rollup.
EmmyMiao87 commented on issue #7008: URL: https://github.com/apache/incubator-doris/issues/7008#issuecomment-968676882 我有一个偏业务的小问题，什么类型的业务查询会有这种 bitmap_union_count(IF(k2=0,v1,NULL)) AS v1_num 需求呢？ 能举个具体的业务例子嘛？ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zbtzbtzbt commented on issue #7008: [Enhance] use predicate if in bitmap_union_count() can not hit rollup.
zbtzbtzbt commented on issue #7008: URL: https://github.com/apache/incubator-doris/issues/7008#issuecomment-968686655 这里是用来计算每日新客数目 ``` select dt ,bitmap_union_count(if(is_day_new_user=1,buy_user_id,null)) as newuser_num -- 新客数 ,sum(sale_amt) as sale_amt -- 销售额 ,sum(sale_amt)/bitmap_union_count(buy_user_id) as ARPU -- APRU ,sum(sale_amt)/sum(sale_num) as pcs_avg_price -- 件均价 ,sum(sale_num)/bitmap_union_count(buy_user_id) as user_avg_sale_num -- 人均销售件数 ,bitmap_union_count(buy_user_id) as buy_user_cnt -- 购买用户数 ,sum(sale_num) as sale_num -- 销售件数 ,bitmap_union_count(sale_main_order_id) as main_order_num -- 订单量 from grocery_doris_pdt.topic_pdt_pro_coupon_ord_sku_dt_period_v1 where dt=20211109 -- 区域筛选 and net_region_id = 350 -- 业务模式 group by dt ``` 下面这个sql很丑，看样子是根据用户类型分别计算什么指标 ``` SELECT IF(GROUPING_ID(c_chain)=1,2,c_chain) AS plat_type,-- 0:MT，1:DP,2:ALL bitmap_union_count(IF(user_type=0 AND is_jm_user=1,jm_user_pk,NULL)) AS jm_trade_num, bitmap_union_count(IF(user_type=1,jm_user_pk,NULL)) AS jm_uv FROM dm_aggr_brandka_jmc_user_d WHERE partition_date BETWEEN '2021-01-01' and '2021-11-01' AND pdc_brand_id = 184707 GROUP BY GROUPING SETS ((c_chain),()); ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zbtzbtzbt commented on issue #7008: [Enhance] use predicate if in bitmap_union_count() can not hit rollup.
zbtzbtzbt commented on issue #7008: URL: https://github.com/apache/incubator-doris/issues/7008#issuecomment-968689508 还有另一个问题也反馈一下，sum(case when)这种写法 当存在隐式转换时，也可能不会命中rollup，比如从int转化为bigint就命中不了了 猜测转换的原因是怕数据溢出吧 ``` explain select dt, sum(arrived_waybill_cnt) as arrived_waybill_cnt , sum(case when delivery_type_second_level_code=2 then arrived_waybill_cnt end) as arrived_waybill_cnt_jiameng from bi_peisong.app_rider_model_ctl_sa_view_day where dt > 20211022 and dt<= 20211109 and delivery_type_second_level_code in (2,4) group by dt ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zbtzbtzbt edited a comment on issue #7008: [Enhance] use predicate if in bitmap_union_count() can not hit rollup.
zbtzbtzbt edited a comment on issue #7008: URL: https://github.com/apache/incubator-doris/issues/7008#issuecomment-968689508 还有另一个问题也反馈一下，sum(case when)这种写法，也属于聚合函数里面用表达式 当存在隐式转换时，也可能不会命中rollup，比如从int转化为bigint就命中不了了 猜测转换的原因是怕数据溢出吧 ``` explain select dt, sum(arrived_waybill_cnt) as arrived_waybill_cnt , sum(case when delivery_type_second_level_code=2 then arrived_waybill_cnt end) as arrived_waybill_cnt_jiameng from bi_peisong.app_rider_model_ctl_sa_view_day where dt > 20211022 and dt<= 20211109 and delivery_type_second_level_code in (2,4) group by dt ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on issue #7008: [Enhance] use predicate if in bitmap_union_count() can not hit rollup.
EmmyMiao87 commented on issue #7008: URL: https://github.com/apache/incubator-doris/issues/7008#issuecomment-968694069 > 还有另一个问题也反馈一下，sum(case when)这种写法，也属于聚合函数里面用表达式 当存在隐式转换时，也可能不会命中rollup，比如从int转化为bigint就命中不了了 猜测转换的原因是怕数据溢出吧 > > ``` > explain > select dt, > sum(arrived_waybill_cnt) as arrived_waybill_cnt > , sum(case when delivery_type_second_level_code=2 then arrived_waybill_cnt end) as arrived_waybill_cnt_jiameng > from bi_peisong.app_rider_model_ctl_sa_view_day > where dt > 20211022 and dt<= 20211109 > and delivery_type_second_level_code in (2,4) > group by dt > ``` 是这样的，目前物化视图选择器不支持 sum(expr) 其中 expr 不是一个列的情况。 最主要的原因是，无法判断 expr 中的列物化视图是否满足要求。 比如 bitmap_union_count(if(is_day_new_user=1,buy_user_id,null)) 其中需要检测，is_day_new_user， buy_user_id 是否能正确的从物化视图中读取出。 由于 expr 千变万化，需要有表达式等价这类框架支持才能判断。顾目前都不支持聚合函数中出现表达式的物化视图选择。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #6569: [Feature] Support query hive table
github-actions[bot] commented on pull request #6569: URL: https://github.com/apache/incubator-doris/pull/6569#issuecomment-968763546 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman opened a new issue #7123: [Feature] Add minidump support
morningman opened a new issue #7123: URL: https://github.com/apache/incubator-doris/issues/7123 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description C language program may cause the process to hang due to various reasons. Usually we need to locate the error through the coredump file. However, coredump files are usually relatively large, which is not conducive to transmission, and most online environments are closed by default. Minidump is a file format defined by Microsoft for reporting errors after program crashes. It includes thread information, register information, call stack information, etc. at the time of the crash, which helps developers quickly locate the problem. Unlike [Coredump](https://en.wikipedia.org/wiki/Core_dump), Minidump files are smaller and easier to report and network transmission. Coredump file will contain a complete memory image, so the volume may be dozens or hundreds of GB. The Minidump file only contains the call stack and register information of the key thread, so the size is usually only MB level. [Breakpad](https://github.com/google/breakpad) is a cross-platform crash dump and analysis framework and tool collection. Users can use Breakpad to conduct self-service analysis of Minidump files. You can also collect Minidump files and report them to Doris cluster operation and maintenance or developers. Many systems use minidump to assist in locating and reporting problems, such as impala, kudu and so on. Doris can also introduce this feature to help users troubleshoot and locate problems. ### Use case _No response_ ### Related issues _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman opened a new pull request #7124: [Feature] Add minidump support
morningman opened a new pull request #7124: URL: https://github.com/apache/incubator-doris/pull/7124 ## Proposed changes Add minidump support. Now minidump file will be created when BE crashes. And user can manually trigger a minidump by sending SIGUSR1 to BE process. More details can be found in `minidump.md` documents ## Types of changes What types of changes does your code introduce to Doris? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) - [ ] Code refactor (Modify the code structure, format the code, etc...) - [ ] Optimization. Including functional usability improvements and performance improvements. - [ ] Dependency. Such as changes related to third-party components. - [ ] Other. ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [x] I have created an issue on (Fix #7123 ) and described the bug/feature there in detail - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If these changes need document changes, I have updated the document - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at d...@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on pull request #7087: add replica num limit per tablet
morningman commented on pull request #7087: URL: https://github.com/apache/incubator-doris/pull/7087#issuecomment-969674663 hi @tianhui5 the FE unit test failed at DynamicPartitionTableTest.testSetDynamicPartitionReplicationNum. You can see details here:  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #7101: logger error in BDBStateChangeListener.java
morningman merged pull request #7101: URL: https://github.com/apache/incubator-doris/pull/7101 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #7111: [Bug] Fix inappropriate Cmake option used to build ZSTD
morningman merged pull request #7111: URL: https://github.com/apache/incubator-doris/pull/7111 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated (ccb1ea8 -> e476e15)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from ccb1ea8 [Refactor] logger error in BDBStateChangeListener.java (#7101) add e476e15 [Bug] Fix inappropriate Cmake option used to build ZSTD (#7111) No new revisions were added by this update. Summary of changes: thirdparty/build-thirdparty.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated (5aaf24b -> ccb1ea8)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 5aaf24b [Compile] Remove unused import (#7112) add ccb1ea8 [Refactor] logger error in BDBStateChangeListener.java (#7101) No new revisions were added by this update. Summary of changes: .../src/main/java/org/apache/doris/ha/BDBStateChangeListener.java | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] tianhui5 commented on pull request #7087: add replica num limit per tablet
tianhui5 commented on pull request #7087: URL: https://github.com/apache/incubator-doris/pull/7087#issuecomment-969707666 > hi @tianhui5 the FE unit test failed at DynamicPartitionTableTest.testSetDynamicPartitionReplicationNum. You can see details here:  I've fix it. Sorry I didn't see the button of log in as guest. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a change in pull request #7124: [Feature] Add minidump support
yangzhg commented on a change in pull request #7124: URL: https://github.com/apache/incubator-doris/pull/7124#discussion_r749852606 ## File path: be/src/service/doris_main.cpp ## @@ -57,6 +58,8 @@ #include "util/thrift_server.h" #include "util/uid_util.h" +#include "client/linux/handler/exception_handler.h" Review comment: Is this include necessary ## File path: be/src/service/doris_main.cpp ## @@ -57,6 +58,8 @@ #include "util/thrift_server.h" #include "util/uid_util.h" +#include "client/linux/handler/exception_handler.h" Review comment: Is this include necessary? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request #7124: [Feature] Add minidump support
morningman commented on a change in pull request #7124: URL: https://github.com/apache/incubator-doris/pull/7124#discussion_r749852913 ## File path: be/src/service/doris_main.cpp ## @@ -57,6 +58,8 @@ #include "util/thrift_server.h" #include "util/uid_util.h" +#include "client/linux/handler/exception_handler.h" Review comment: No, I will remove it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg opened a new issue #7125: [enhancement] Optimize the fail logic of loading data when some backend failed
yangzhg opened a new issue #7125: URL: https://github.com/apache/incubator-doris/issues/7125 When loading data, when a tablet sink node writes data to other be, the logic for judging the write failure is that the total number of failed nodes exceeds half of the number of replicas, but a sink node may write multiple tablets, and the correct judgment is written The logic of entry failure is that the number of failed copies of each tablet exceeds half of the number of copies of this tablet -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] drgnchan opened a new pull request #7127: correct getLogger argument
drgnchan opened a new pull request #7127: URL: https://github.com/apache/incubator-doris/pull/7127 ## Proposed changes correct getLogger argument ## Types of changes What types of changes does your code introduce to Doris? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) - [ ] Code refactor (Modify the code structure, format the code, etc...) - [x] Optimization. Including functional usability improvements and performance improvements. - [ ] Dependency. Such as changes related to third-party components. - [ ] Other. ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on (Fix #ISSUE) and described the bug/feature there in detail - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If these changes need document changes, I have updated the document - [x] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at d...@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg opened a new pull request #7126: [enhancement] Optimize the fail logic of loading data when some backend failed
yangzhg opened a new pull request #7126: URL: https://github.com/apache/incubator-doris/pull/7126 mark the load job fail when more than a half of replica write failed of a tablet, the code before is counting all replica has more than a half write failed. ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Types of changes What types of changes does your code introduce to Doris? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) - [ ] Code refactor (Modify the code structure, format the code, etc...) - [x] Optimization. Including functional usability improvements and performance improvements. - [ ] Dependency. Such as changes related to third-party components. - [ ] Other. ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [x] I have created an issue on (Fix #7125) and described the bug/feature there in detail - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If these changes need document changes, I have updated the document - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at d...@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman opened a new pull request #7128: [Dependency] Add breakpad for minidump
morningman opened a new pull request #7128: URL: https://github.com/apache/incubator-doris/pull/7128 ## Proposed changes Related #7123 ## Types of changes What types of changes does your code introduce to Doris? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) - [ ] Code refactor (Modify the code structure, format the code, etc...) - [ ] Optimization. Including functional usability improvements and performance improvements. - [ ] Dependency. Such as changes related to third-party components. - [ ] Other. ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on (Fix #ISSUE) and described the bug/feature there in detail - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If these changes need document changes, I have updated the document - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at d...@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7105: [BUG] fix profile not working with sql_cache enabled
github-actions[bot] commented on pull request #7105: URL: https://github.com/apache/incubator-doris/pull/7105#issuecomment-969830587 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #6515: [Memory] BitShufflePageDecoder use memory allocated by ChunkAllocator instead of Faststring
github-actions[bot] commented on pull request #6515: URL: https://github.com/apache/incubator-doris/pull/6515#issuecomment-969835258 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 closed issue #6568: [Feature] Support query hive table
EmmyMiao87 closed issue #6568: URL: https://github.com/apache/incubator-doris/issues/6568 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [Feature] Support query hive table (#6569)
This is an automated email from the ASF dual-hosted git repository.
lingmiao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 5b01f7b [Feature] Support query hive table (#6569)
5b01f7b is described below
commit 5b01f7bba2462daf818fc956f650527d51e8be97
Author: qiye
AuthorDate: Tue Nov 16 11:59:07 2021 +0800
[Feature] Support query hive table (#6569)
Users can directly query the data in the hive table in Doris, and can use
join to perform complex queries without laboriously importing data from hive.
Main changes list below:
FE:
Extend HiveScanNode from BrokerScanNode
HiveMetaStoreClientHelper communicate with HIVE and HDFS.
BE:
Treate HiveScanNode as BrokerScanNode, treate HiveTable as BrokerTable.
broker_scanner.cpp: suppot read column from HDFS path.
orc_scanner.cpp: support read hdfs file.
POM:
Add hive.version=2.3.7, hive-metastore and hive-exec
Add hadoop.version=2.8.0, hadoop-hdfs
Upgrade commons-lang to fix incompatiblity of Java 9 and later.
Thrift:
Add THiveTable
Add read_by_column_def in TBrokerRangeDesc
---
be/src/exec/broker_scanner.cpp | 68 ++-
be/src/exec/orc_scanner.cpp| 13 +
docs/.vuepress/sidebar/en.js | 1 +
docs/.vuepress/sidebar/zh-CN.js| 1 +
docs/en/extending-doris/hive-of-doris.md | 117 +
.../sql-statements/Data Definition/CREATE TABLE.md | 2 +-
docs/zh-CN/extending-doris/hive-of-doris.md| 117 +
.../sql-statements/Data Definition/CREATE TABLE.md | 1 -
fe/fe-core/pom.xml | 35 +-
.../java/org/apache/doris/analysis/FromClause.java | 3 -
.../java/org/apache/doris/catalog/Catalog.java | 8 +
.../doris/catalog/HiveMetaStoreClientHelper.java | 552 +
.../java/org/apache/doris/catalog/HiveTable.java | 14 +-
.../org/apache/doris/load/BrokerFileGroup.java | 18 +
.../org/apache/doris/planner/BrokerScanNode.java | 108 ++--
.../org/apache/doris/planner/HiveScanNode.java | 210
.../apache/doris/planner/SingleNodePlanner.java| 6 +-
fe/pom.xml | 84 +++-
gensrc/thrift/Descriptors.thrift | 7 +
gensrc/thrift/PlanNodes.thrift | 2 +
20 files changed, 1278 insertions(+), 89 deletions(-)
diff --git a/be/src/exec/broker_scanner.cpp b/be/src/exec/broker_scanner.cpp
index a260035..e99bf7e 100644
--- a/be/src/exec/broker_scanner.cpp
+++ b/be/src/exec/broker_scanner.cpp
@@ -478,35 +478,51 @@ bool BrokerScanner::line_to_src_tuple(const Slice& line) {
// range of current file
const TBrokerRangeDesc& range = _ranges.at(_next_range - 1);
+bool read_by_column_def = false;
+if (range.__isset.read_by_column_def) {
+read_by_column_def = range.read_by_column_def;
+}
const std::vector& columns_from_path =
range.columns_from_path;
-if (_split_values.size() + columns_from_path.size() <
_src_slot_descs.size()) {
-std::stringstream error_msg;
-error_msg << "actual column number is less than schema column number. "
- << "actual number: " << _split_values.size() << " column
separator: ["
- << _value_separator << "], "
- << "line delimiter: [" << _line_delimiter << "], "
- << "schema number: " << _src_slot_descs.size() << "; ";
-if (_file_format_type == TFileFormatType::FORMAT_PROTO) {
-_state->append_error_msg_to_file("", error_msg.str());
-} else {
-_state->append_error_msg_to_file(std::string(line.data,
line.size), error_msg.str());
+// read data by column defination, resize _split_values to _src_solt_size
+if (read_by_column_def) {
+// fill solts by NULL
+while (_split_values.size() + columns_from_path.size() <
_src_slot_descs.size()) {
+_split_values.emplace_back(_split_values.back().get_data(), 0);
}
-_counter->num_rows_filtered++;
-return false;
-} else if (_split_values.size() + columns_from_path.size() >
_src_slot_descs.size()) {
-std::stringstream error_msg;
-error_msg << "actual column number is more than schema column number. "
- << "actual number: " << _split_values.size() << " column
separator: ["
- << _value_separator << "], "
- << "line delimiter: [" << _line_delimiter << "], "
- << "schema number: " << _src_slot_descs.size() << "; ";
-if (_file_format_type == TFileFormatType::FORMAT_PROTO) {
-_state->append_error_msg_to_file("", error_msg.str());
-} else {
-_state->append_error_msg_to_file(std::string(line.data
[GitHub] [incubator-doris] EmmyMiao87 merged pull request #6569: [Feature] Support query hive table
EmmyMiao87 merged pull request #6569: URL: https://github.com/apache/incubator-doris/pull/6569 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request #7126: [enhancement] Optimize the fail logic of loading data when some backend failed
morningman commented on a change in pull request #7126:
URL: https://github.com/apache/incubator-doris/pull/7126#discussion_r749882831
##
File path: be/src/exec/tablet_sink.h
##
@@ -295,8 +305,10 @@ class IndexChannel {
std::unordered_map _node_channels;
// from tablet_id to backend channel
std::unordered_map> _channels_by_tablet;
+// from backend channel to tablet_id
+std::unordered_map> _tablets_by_channel;
Review comment:
```suggestion
std::unordered_map>
_tablets_by_channel;
```
##
File path: be/src/exec/tablet_sink.h
##
@@ -295,8 +305,10 @@ class IndexChannel {
std::unordered_map _node_channels;
// from tablet_id to backend channel
std::unordered_map> _channels_by_tablet;
+// from backend channel to tablet_id
+std::unordered_map> _tablets_by_channel;
// BeId
Review comment:
modify comment
##
File path: be/src/exec/tablet_sink.h
##
@@ -295,8 +305,10 @@ class IndexChannel {
std::unordered_map _node_channels;
// from tablet_id to backend channel
std::unordered_map> _channels_by_tablet;
+// from backend channel to tablet_id
+std::unordered_map> _tablets_by_channel;
// BeId
-std::set _failed_channels;
+std::unordered_map> _failed_channels;
Review comment:
```suggestion
std::unordered_map>
_failed_channels;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #7080: [Blog]Example of binlog load usage
morningman merged pull request #7080: URL: https://github.com/apache/incubator-doris/pull/7080 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [Blog] Example of binlog load usage (#7080)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 5710cf8 [Blog] Example of binlog load usage (#7080)
5710cf8 is described below
commit 5710cf8febece396e1e4bff19ca3687e349b7b08
Author: jiafeng.zhang
AuthorDate: Tue Nov 16 12:12:44 2021 +0800
[Blog] Example of binlog load usage (#7080)
Example of binlog load usage
---
.../images/binlog/image-2020145044815.png | Bin 0 -> 146900 bytes
.../images/binlog/image-2020160106602.png | Bin 0 -> 52259 bytes
.../images/binlog/image-2020160331479.png | Bin 0 -> 67624 bytes
.../images/binlog/image-2020160710709.png | Bin 0 -> 56885 bytes
docs/en/article/articles/doris-binlog-load.md | 418
docs/zh-CN/article/articles/doris-binlog-load.md | 421 +
6 files changed, 839 insertions(+)
diff --git a/docs/.vuepress/public/images/binlog/image-2020145044815.png
b/docs/.vuepress/public/images/binlog/image-2020145044815.png
new file mode 100644
index 000..c701de1
Binary files /dev/null and
b/docs/.vuepress/public/images/binlog/image-2020145044815.png differ
diff --git a/docs/.vuepress/public/images/binlog/image-2020160106602.png
b/docs/.vuepress/public/images/binlog/image-2020160106602.png
new file mode 100644
index 000..0a3b9a7
Binary files /dev/null and
b/docs/.vuepress/public/images/binlog/image-2020160106602.png differ
diff --git a/docs/.vuepress/public/images/binlog/image-2020160331479.png
b/docs/.vuepress/public/images/binlog/image-2020160331479.png
new file mode 100644
index 000..6adbe59
Binary files /dev/null and
b/docs/.vuepress/public/images/binlog/image-2020160331479.png differ
diff --git a/docs/.vuepress/public/images/binlog/image-2020160710709.png
b/docs/.vuepress/public/images/binlog/image-2020160710709.png
new file mode 100644
index 000..c17c211
Binary files /dev/null and
b/docs/.vuepress/public/images/binlog/image-2020160710709.png differ
diff --git a/docs/en/article/articles/doris-binlog-load.md
b/docs/en/article/articles/doris-binlog-load.md
new file mode 100644
index 000..6511551
--- /dev/null
+++ b/docs/en/article/articles/doris-binlog-load.md
@@ -0,0 +1,418 @@
+---
+{
+"title": "How to use Apache Doris Binlog Load and examples",
+"description": "Binlog Load provides a CDC (Change Data Capture) function
that enables Doris to incrementally synchronize the user's data update
operation in the Mysql database, making it more convenient for users to
complete the import of Mysql data.",
+"date": "2021-11-10",
+"metaTitle": "How to use Apache Doris Binlog Load and examples",
+"language": "en",
+"isArticle": true,
+"author": "张家锋",
+"layout": "Article",
+"sidebar": false
+}
+---
+
+
+Binlog Load provides a CDC (Change Data Capture) function that enables Doris
to incrementally synchronize the user's data update operation in the Mysql
database, making it more convenient for users to complete the import of Mysql
data.
+
+>Note:
+>
+>This function needs to be used in 0.15 and later versions
+
+## 1. Install and configure Mysql
+
+1. Install Mysql
+
+ Quickly use Docker to install and configure Mysql, refer to the following
link for details
+
+ https://segmentfault.com/a/119021523570
+
+ If it is installed on a physical machine, please refer to the following
connection:
+
+ [在 CentOS 7 中安装 MySQL 8
的教程详解](https://cloud.tencent.com/developer/article/1721575)
+
+2. enable Mysql binlog
+
+ Enter the Docker container or modify the `/etc/my.cnf` file on the physical
machine, and add the following content under [mysqld],
+
+ ```
+ log_bin=mysql_bin
+ binlog-format=Row
+ server-id=1
+ ```
+
+ Then restart Mysql
+
+ ```
+ systemctl restart mysqld
+ ```
+
+3. Create Mysql table
+
+ ```sql
+ create database demo;
+
+CREATE TABLE `test_cdc` (
+ `id` int NOT NULL AUTO_INCREMENT,
+ `sex` TINYINT(1) DEFAULT NULL,
+ `name` varchar(20) DEFAULT NULL,
+ `address` varchar(255) DEFAULT NULL,
+ PRIMARY KEY (`id`)
+) ENGINE=InnoDB
+ ```
+
+
+## 2. Install and configure Canal
+
+Download canal-1.1.5:
https://github.com/alibaba/canal/releases/download/canal-1.1.5/canal.deployer-1.1.5.tar.gz
+
+1. Unzip Canal to the specified directory:
+
+ ```shell
+ tar zxvf canal.deployer-1.1.5.tar.gz -C ./canal
+ ```
+
+2. Create a new directory in the conf folder and rename it as the root
directory of the instance. You can name the directory for easy identification.
+
+ For example, my name here is consistent with the name of my database
library: demo
+
+ ```shell
+ vi conf/demo/instance.properties
+ ```
+
+ Given below is a sample configuration of mine:
+
+ For the parameter description,
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7108: [Bug] Fix the incompatibility of sql mode between Doris and MySQL
github-actions[bot] commented on pull request #7108: URL: https://github.com/apache/incubator-doris/pull/7108#issuecomment-969845938 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #6856: [refactor] replace boost smart ptr with stl
github-actions[bot] commented on pull request #6856: URL: https://github.com/apache/incubator-doris/pull/6856#issuecomment-969846134 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request #7032: [Heartbeat] Support cancel outdated query after FE restart
morningman commented on a change in pull request #7032:
URL: https://github.com/apache/incubator-doris/pull/7032#discussion_r749890518
##
File path: be/src/agent/heartbeat_server.cpp
##
@@ -78,6 +78,32 @@ void HeartbeatServer::heartbeat(THeartbeatResult&
heartbeat_result,
Status HeartbeatServer::_heartbeat(const TMasterInfo& master_info) {
std::lock_guard lk(_hb_mtx);
+if (master_info.__isset.frontends_info) {
+std::stringstream ss;
+ss << "Heartbeat frontends info len: " <<
master_info.frontends_info.size();
+for (auto info: master_info.frontends_info) {
Review comment:
```suggestion
for (auto& info: master_info.frontends_info) {
```
##
File path: be/src/agent/heartbeat_server.h
##
@@ -34,9 +35,22 @@ class StorageEngine;
class Status;
class ThriftServer;
+struct FrontendStartInfo {
+int64_t start_time;
+bool is_alive;
+DateTimeValue* last_heartbeat; // Invalid time of Info
Review comment:
why not just use int64?
##
File path: be/src/agent/heartbeat_server.cpp
##
@@ -78,6 +78,32 @@ void HeartbeatServer::heartbeat(THeartbeatResult&
heartbeat_result,
Status HeartbeatServer::_heartbeat(const TMasterInfo& master_info) {
std::lock_guard lk(_hb_mtx);
+if (master_info.__isset.frontends_info) {
+std::stringstream ss;
+ss << "Heartbeat frontends info len: " <<
master_info.frontends_info.size();
+for (auto info: master_info.frontends_info) {
+ss << "; host:" << info.network_address.hostname
+<< ", port:" << info.network_address.port
+<< ", fe_start_time:" << info.fe_start_time
+<< ", is_alive:" << info.is_alive;
+std::string coord_addr_str = info.network_address.hostname + ":" +
std::to_string(info.network_address.port);
+auto fsi_ptr =
_exec_env->frontends_start_time().find(coord_addr_str);
+if (LIKELY(fsi_ptr != _exec_env->frontends_start_time().end())) {
+fsi_ptr->second->is_alive = info.is_alive;
Review comment:
Need lock to update
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7114: [Routine load] Support show offset lag in show routine load stmt
github-actions[bot] commented on pull request #7114: URL: https://github.com/apache/incubator-doris/pull/7114#issuecomment-969872557 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] wuyunfeng commented on pull request #6990: Doris on es 聚合下推
wuyunfeng commented on pull request #6990: URL: https://github.com/apache/incubator-doris/pull/6990#issuecomment-969890986 @Userwhite As discussed in our online meeting，we should move the predicate-transform from BE (backends) to FE (frontends) which would contribute significant effect for subsequent planning and extension. We will continue to pay more attention and participate in this PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] wuyunfeng edited a comment on pull request #6990: Doris on es 聚合下推
wuyunfeng edited a comment on pull request #6990: URL: https://github.com/apache/incubator-doris/pull/6990#issuecomment-969890986 @Userwhite As discussed in our online meeting，we should move the predicate-transform from BE (backends) to FE (frontends) which would contribute significant effect for subsequent planning and extension. We will continue to pay more attention and participation in this PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on pull request #7109: [Doc]Modify the wrong comment of the ScanTime
yangzhg commented on pull request #7109: URL: https://github.com/apache/incubator-doris/pull/7109#issuecomment-969900806 Please also modify the English docs in `docs/en/administrator-guide/running-profile.md` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
svn commit: r50959 - in /dev/incubator/doris: ./ 0.15/ 0.15/0.15.0-rc03/
Author: morningman Date: Tue Nov 16 06:28:49 2021 New Revision: 50959 Log: Add 0.15.0-rc03 Added: dev/incubator/doris/0.15/ dev/incubator/doris/0.15/0.15.0-rc03/ dev/incubator/doris/0.15/0.15.0-rc03/apache-doris-0.15.0-incubating-src.tar.gz (with props) dev/incubator/doris/0.15/0.15.0-rc03/apache-doris-0.15.0-incubating-src.tar.gz.asc dev/incubator/doris/0.15/0.15.0-rc03/apache-doris-0.15.0-incubating-src.tar.gz.sha512 Modified: dev/incubator/doris/KEYS Added: dev/incubator/doris/0.15/0.15.0-rc03/apache-doris-0.15.0-incubating-src.tar.gz == Binary file - no diff available. Propchange: dev/incubator/doris/0.15/0.15.0-rc03/apache-doris-0.15.0-incubating-src.tar.gz -- svn:mime-type = application/octet-stream Added: dev/incubator/doris/0.15/0.15.0-rc03/apache-doris-0.15.0-incubating-src.tar.gz.asc == --- dev/incubator/doris/0.15/0.15.0-rc03/apache-doris-0.15.0-incubating-src.tar.gz.asc (added) +++ dev/incubator/doris/0.15/0.15.0-rc03/apache-doris-0.15.0-incubating-src.tar.gz.asc Tue Nov 16 06:28:49 2021 @@ -0,0 +1,14 @@ +-BEGIN PGP SIGNATURE- + +iQHKBAABCAA0FiEEfDkqHpxmyczPOIYY8QYm42JFg8sFAmGTTVwWHGNoZW5taW5n +eXVAYXBhY2hlLm9yZwAKCRDxBibjYkWDyyvJC/9ZR/0lCSrrR8VNxoQMSPC9bk2O +NToqK2eVmF0ycTzOmJP8O1sHm496renscymsJzMHAmJg+pWZ76JWQGCbIhnuBkgf +Me/ZITVof+A53SDIih9GR/QTUKMwO2rn2i5mabCoDuAi7btLr6Pu0D5h7SeM7Eqk +51o+hfLwyhdppVtvQsVJsnLXx9WZ5JLilXoGdPKVUq57WJ8XWlZ2KOIaPsAr+Nli +FJ171K3HIk2hDAdcmdvsYZCei93L3LIuvsO0+Rr+Xi1Q5yJY7OQewtjHjbjOieAM +u5AsbYSBHiQNlXMzHA5d8QJcoFVl6xDGWj7tYUqEL/sV7uvyb0xoc2IUMzpqJMqa +cAnJaTyzwF8ooctpkzhan8ZSZRgt54jDyhS/R5PGFt45K9QQsv4JqgAfVK5BTR91 +CTz7Eovu/qK0DfIQPQbRxC1EqKLROcTuINHu99odiTDy2owDJscv28Bv8PJFNJLW +FvMlpXLwLb5OL56aKSWQizMdnJm9BrKMi0siuh8= +=twCN +-END PGP SIGNATURE- Added: dev/incubator/doris/0.15/0.15.0-rc03/apache-doris-0.15.0-incubating-src.tar.gz.sha512 == --- dev/incubator/doris/0.15/0.15.0-rc03/apache-doris-0.15.0-incubating-src.tar.gz.sha512 (added) +++ dev/incubator/doris/0.15/0.15.0-rc03/apache-doris-0.15.0-incubating-src.tar.gz.sha512 Tue Nov 16 06:28:49 2021 @@ -0,0 +1 @@ +412a07538f4b5b1a36f22eda1f8c7614b83b7db02f764861d17403a4c94a1ed04d59f377789fa732b9161c13311257f0af0e31215af84ec9b7a813ff1679481d apache-doris-0.15.0-incubating-src.tar.gz Modified: dev/incubator/doris/KEYS == --- dev/incubator/doris/KEYS (original) +++ dev/incubator/doris/KEYS Tue Nov 16 06:28:49 2021 @@ -301,3 +301,51 @@ yfLX9EGMpInL7VxeVxUf3vxJBPk/Vknw3X1DTIMw bTEsSwUVNLld2iXN19cUgv8FQqq76jQ8sgpjCsZqyc4= =zRd8 -END PGP PUBLIC KEY BLOCK- +pub rsa3072 2021-11-16 [SC] [ææè³ï¼2023-11-16] + 7C392A1E9C66C9CCCF388618F10626E3624583CB +uid [ ç»å¯¹ ] morningman +sig 3F10626E3624583CB 2021-11-16 morningman +sub rsa3072 2021-11-16 [E] [ææè³ï¼2023-11-16] +sig F10626E3624583CB 2021-11-16 morningman + +-BEGIN PGP PUBLIC KEY BLOCK- + +mQGNBGGTS/YBDADUlkw1urh3DH2K1OWslkg4Gzq2ONWUIcPijaFB2JLwlENnr3MJ +EvC3HGs1XK0ziVTJ7OlCKgG1HG+RqqNUbe0d7jja6TDB71neh3P1j//VBMwPhLVx +i2qIluaNT/zOl9zBmKJaNp4Yn58j3MeY76/9w/cvAO7jOako0ddW2Izs2wg000RH +wlv7duFPBqDiDv86mo48gNeWk9wH1aEbVb9bbnAjgF+VwU/76SUr6ifgGPFgP7yU +SXYhYSq/zNwPDRj0wUN++UXoZW2+npg1YoJZmoY92WpfsANLqhOdFS7vfZnjgxL2 +yCBVdFtNAefP3UI443L0x9R4Qwh+QiauY2NijLmx2GYJGWFjt1kOe9R5JAPBxGoQ +KWWI5kvYCj5V9by48MhweU+iaeXivID2YCubMK9BytcyUXJfwjL8YIxRpk8XhCYE +eZAF3I8MPp2fborJM6jxGcK4J8WebdLi3CSVh7Qwh3dLJQvPgF7WO/V2vmc/gsRi +UV+8+p4asHlItlsAEQEAAbQibW9ybmluZ21hbiA8Y2hlbm1pbmd5dUBhcGFjaGUu +b3JnPokB1AQTAQgAPhYhBHw5Kh6cZsnMzziGGPEGJuNiRYPLBQJhk0v2AhsDBQkD +wmcABQsJCAcCBhUKCQgLAgQWAgMBAh4BAheAAAoJEPEGJuNiRYPL90gL/1RZJDTL +GXsZNh4vYUsnMqIMcmV095mdqsEbK0XZ5lcSjtcMZijA4SVl75f9qKGE+P9fg3rL +UAHvKRc92OXNtPD0Hr+sThJub4i/Jb7hSo9j5wRd61KShmwoAgYiV5xb0UtlI3bE +WKAiiE5rvcQCcSYsJ3XmcdkLYYO8x3lzUnx0fXuwjTkieDJXB8lbinffk8JBmudG +RjVdpjyjDWUiZSOlfO9OgasnFVVQdeJywsWgJoQh//1chYMpjgF6EF42OgA2ZJ5B +bn5y0B4rhE2TINJgo7Bjgy7xPvHE07s+a7EgkT907m//aoC21KQrytPTEDj2S0gz +uscY5deyKMpjbQ8ms8/w1N1TRX39cVjX6z+FYznzl/O98w0cV8RWMBZaOV/zERe2 +ORif6vFAuv5oNJPl8CnsHQ3fELY1XpQTNtmAEj5G+XOGCoEVDijZgVrLT8DvxN56 +87cgUPj5mjUP8OmewKQgl7CA2adby4Vpjs86JJ/WUiM/iJQQ0syPAWYq1rkBjQRh +k0v2AQwAxL5NnYu36ddpIkMtxwS0+N2h1mXNXl+nUtuf8mfLXFB5lY0pd3DoSHMN +nJKTuZhv7DK5UFKlUh8Xt4mDYr2PUaWnlSAEFYlXmzbcWfqAZq5kUCaSvFpXXDwO +tK+ehqLJViVSeMCHbHpE28zHr2DhQe3eFfE0VaAuhV8Yz9njLzHoiGbEDNElHSzl +8vrowl8SNpZb1L4GqJH9MaDIE60eDmx5/lO/MqEunEiXIriMBoP369giaIiypCKK +2tpQ+WQDq6uLyRmNmQa0rOf1rOumR5IXRyc+Jz5OhUB2DRsn1MTncV8Mgn6YplRq +qXCCO7iqCImSQIjWeLMUWzaWUl0/Du96WJC2XBKGz6pzrGaJ38o3N9zxd/9vlQrU +TRWS7XZLyfkuD3vQ1CwrSsjm9mIFddW7ryiIduxS43CpAZTMG
[GitHub] [incubator-doris] morningman commented on issue #6806: Release note 0.15.0
morningman commented on issue #6806: URL: https://github.com/apache/incubator-doris/issues/6806#issuecomment-969915020 > @morningman Will vectorization query optimization show up in this release? No, it is still WIP. But you can try part of this feature by using Palo(powered by Doris) here: palo.baidu.com, version 0.15.1-rc09 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] tianhui5 opened a new issue #7129: [Feature] Limit cluster resource usage in user granularity
tianhui5 opened a new issue #7129: URL: https://github.com/apache/incubator-doris/issues/7129 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description In productive environment, the Doris cluster is often facing pressure from many aspects (mainly from stream load and query), cause many resource shortage problem like OOM, especially in shared cluster. I think it's better to have a way to limit the usage of each user. Maybe limit the usage frequency is a proper way. ### Use case _No response_ ### Related issues _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] tianhui5 commented on issue #7129: [Feature] Limit cluster resource usage in user granularity
tianhui5 commented on issue #7129:
URL:
https://github.com/apache/incubator-doris/issues/7129#issuecomment-969971912
I'm prefer to add syntax `LIMITER` on behalf of the frequency limiter. This
limiter is an abstract concept, it can have many kinds of types like query or
stream load.
So the SQL to monipulate `LIMITER` is like below:
`CREATE LIMITER name PORPERTIES("key1"="value1", "key2"="value2");`
`DROP LIMITER name;`
`SHOW LIMITER;`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #7129: [Feature] Limit cluster resource usage in user granularity
morningman commented on issue #7129: URL: https://github.com/apache/incubator-doris/issues/7129#issuecomment-969974125 What if the request exceed the limit? return error or slow down? And is there any other system we can refer to? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
