[GitHub] [incubator-doris] yangzhg merged pull request #4234: Update support batch delete storage design document

2020-08-18 Thread GitBox
yangzhg merged pull request #4234: URL: https://github.com/apache/incubator-doris/pull/4234 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[incubator-doris] branch master updated: Update support batch delete storage design document (#4234)

2020-08-18 Thread yangzhg
This is an automated email from the ASF dual-hosted git repository. yangzhg pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 8a3eaee Update support batch delete s

[GitHub] [incubator-doris] yangzhg commented on a change in pull request #4212: Compaction rules optimization

2020-08-18 Thread GitBox
yangzhg commented on a change in pull request #4212: URL: https://github.com/apache/incubator-doris/pull/4212#discussion_r471988812 ## File path: be/src/common/config.h ## @@ -265,6 +265,27 @@ namespace config { CONF_mInt64(base_compaction_interval_seconds_since_last_opera

[GitHub] [incubator-doris] yangzhg commented on a change in pull request #4212: Compaction rules optimization

2020-08-18 Thread GitBox
yangzhg commented on a change in pull request #4212: URL: https://github.com/apache/incubator-doris/pull/4212#discussion_r471990440 ## File path: be/src/olap/cumulative_compaction_policy.h ## @@ -0,0 +1,263 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// o

[GitHub] [incubator-doris] yangzhg commented on a change in pull request #4212: Compaction rules optimization

2020-08-18 Thread GitBox
yangzhg commented on a change in pull request #4212: URL: https://github.com/apache/incubator-doris/pull/4212#discussion_r471991211 ## File path: be/src/olap/cumulative_compaction_policy.h ## @@ -0,0 +1,263 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// o

[GitHub] [incubator-doris] EmmyMiao87 edited a comment on issue #4370: Release Nodes 0.13.0

2020-08-18 Thread GitBox
EmmyMiao87 edited a comment on issue #4370: URL: https://github.com/apache/incubator-doris/issues/4370#issuecomment-674835876 # Credits @ZhangYu0123 @wfjcmcb @Fullstop000 @sduzh @Stalary @worker24h

[GitHub] [incubator-doris] yangzhg commented on a change in pull request #4212: Compaction rules optimization

2020-08-18 Thread GitBox
yangzhg commented on a change in pull request #4212: URL: https://github.com/apache/incubator-doris/pull/4212#discussion_r471992521 ## File path: be/src/olap/cumulative_compaction_policy.h ## @@ -0,0 +1,263 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// o

[GitHub] [incubator-doris] yangzhg commented on a change in pull request #4212: Compaction rules optimization

2020-08-18 Thread GitBox
yangzhg commented on a change in pull request #4212: URL: https://github.com/apache/incubator-doris/pull/4212#discussion_r471993152 ## File path: be/src/olap/cumulative_compaction_policy.h ## @@ -0,0 +1,263 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// o

[GitHub] [incubator-doris] xy720 opened a new pull request #4383: [SparkLoad]Use the yarn command to get status and kill the application

2020-08-18 Thread GitBox
xy720 opened a new pull request #4383: URL: https://github.com/apache/incubator-doris/pull/4383 ## Proposed changes #4346 #4203 This cl will use yarn command as follows to kill or get status of application running on YARN. ``` yarn --config confdir application <-kill |

[GitHub] [incubator-doris] yangzhg commented on a change in pull request #4212: Compaction rules optimization

2020-08-18 Thread GitBox
yangzhg commented on a change in pull request #4212: URL: https://github.com/apache/incubator-doris/pull/4212#discussion_r471994484 ## File path: be/src/olap/cumulative_compaction_policy.h ## @@ -0,0 +1,263 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// o

[GitHub] [incubator-doris] yangzhg commented on a change in pull request #4212: Compaction rules optimization

2020-08-18 Thread GitBox
yangzhg commented on a change in pull request #4212: URL: https://github.com/apache/incubator-doris/pull/4212#discussion_r471996935 ## File path: be/src/olap/olap_server.cpp ## @@ -277,6 +281,27 @@ void* StorageEngine::_disk_stat_monitor_thread_callback(void* arg) { retur

[incubator-doris] branch master updated (8a3eaee -> 38a2a7a)

2020-08-18 Thread morningman
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 8a3eaee Update support batch delete storage design document (#4234) add 38a2a7a [Bug] Fix bug th

[GitHub] [incubator-doris] morningman closed issue #4323: [Bug] Modification of global variables is not correctly persisted.

2020-08-18 Thread GitBox
morningman closed issue #4323: URL: https://github.com/apache/incubator-doris/issues/4323 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [incubator-doris] morningman merged pull request #4324: [Bug] Fix bug that modification of global variable can not be persisted.

2020-08-18 Thread GitBox
morningman merged pull request #4324: URL: https://github.com/apache/incubator-doris/pull/4324 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [incubator-doris] morningman closed issue #4344: [Bug]BE crash when doing LOADING phase of spark load

2020-08-18 Thread GitBox
morningman closed issue #4344: URL: https://github.com/apache/incubator-doris/issues/4344 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [incubator-doris] morningman merged pull request #4345: [Bug][MemTracker] Cleanup the mem tracker's constructor to avoid wrong usage

2020-08-18 Thread GitBox
morningman merged pull request #4345: URL: https://github.com/apache/incubator-doris/pull/4345 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[incubator-doris] branch master updated: [Bug][MemTracker] Cleanup the mem tracker's constructor to avoid wrong usage (#4345)

2020-08-18 Thread morningman
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new e251080 [Bug][MemTracker] Cleanup

[GitHub] [incubator-doris] morningman merged pull request #4327: [Metrics] Support tablet level metrics

2020-08-18 Thread GitBox
morningman merged pull request #4327: URL: https://github.com/apache/incubator-doris/pull/4327 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[incubator-doris] branch master updated (e251080 -> 56260a6)

2020-08-18 Thread morningman
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from e251080 [Bug][MemTracker] Cleanup the mem tracker's constructor to avoid wrong usage (#4345) add

[GitHub] [incubator-doris] morningman opened a new issue #4384: [Bug][SparkLoad] Spark load will create rowset with incorrect rowset type

2020-08-18 Thread GitBox
morningman opened a new issue #4384: URL: https://github.com/apache/incubator-doris/issues/4384 **Describe the bug** 1. create a table with segment v2 format 2. load data with spark load 3. the rowset with version [2-2] is with storage format SegmentV1, which is expected to be V2.

[GitHub] [incubator-doris] morningman opened a new issue #4385: [Bug] tablet type in tablet meta is wrong

2020-08-18 Thread GitBox
morningman opened a new issue #4385: URL: https://github.com/apache/incubator-doris/issues/4385 **Describe the bug** The `tablet_type` in tablet meta should be `TABLET_TYPE_DISK`. But it is set to `TABLET_TYPE_MEMORY` T

[GitHub] [incubator-doris] morningman opened a new issue #4386: [SegmentV2] Change the default storage format to SegmentV2

2020-08-18 Thread GitBox
morningman opened a new issue #4386: URL: https://github.com/apache/incubator-doris/issues/4386 **Is your feature request related to a problem? Please describe.** SInce the Segment V2 has been released for a long time, we should make it as default storage format for newly created tab

[GitHub] [incubator-doris] morningman opened a new pull request #4387: [SegmentV2] Change the default storage format to SegmentV2

2020-08-18 Thread GitBox
morningman opened a new pull request #4387: URL: https://github.com/apache/incubator-doris/pull/4387 ## Proposed changes SInce the Segment V2 has been released for a long time, we should make it as default storage format for newly created table. This CL mainly changes: 1. F

[GitHub] [incubator-doris] HangyuanLiu opened a new pull request #4388: Add OLAP_ERR_DATE_QUALITY_ERR error status to display schema change failure

2020-08-18 Thread GitBox
HangyuanLiu opened a new pull request #4388: URL: https://github.com/apache/incubator-doris/pull/4388 ## Proposed changes In the process of historical data transformation of materialized views, it may occur that the transformation fails due to data quality. Add an error status co

[GitHub] [incubator-doris] imay commented on pull request #4366: Optimise coding bit operation in BE

2020-08-18 Thread GitBox
imay commented on pull request #4366: URL: https://github.com/apache/incubator-doris/pull/4366#issuecomment-675370532 > > Not intended to interfere, just curious about how many improvements can be achieved from this PR, are there any benchmarks? > > In my develop computure, when enco

[GitHub] [incubator-doris] HappenLee opened a new pull request #4389: [ODBC SCAN NODE] 2/4 Add Thrift Interface and Meta of ODBC_Scan_Node

2020-08-18 Thread GitBox
HappenLee opened a new pull request #4389: URL: https://github.com/apache/incubator-doris/pull/4389 issue:#4376 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or res

[GitHub] [incubator-doris] EmmyMiao87 commented on pull request #4253: Support more materialized view syntax

2020-08-18 Thread GitBox
EmmyMiao87 commented on pull request #4253: URL: https://github.com/apache/incubator-doris/pull/4253#issuecomment-675379372 `To compatibility with the old rollup logic, the syntax "DROP MATERIALIZED VIEW [ IF EXISTS ] [db_name].< mv_name > FROM [db].[table]" has been added` Both FROM and

[GitHub] [incubator-doris] EmmyMiao87 commented on pull request #4253: Support more materialized view syntax

2020-08-18 Thread GitBox
EmmyMiao87 commented on pull request #4253: URL: https://github.com/apache/incubator-doris/pull/4253#issuecomment-675380536 The syntax of `SHOW`, `ALTER`, `DROP` should be consistent This is an automated message from the Apac

[GitHub] [incubator-doris] shuaijinchao opened a new issue #4390: [Document] increase mailing list subscription method.

2020-08-18 Thread GitBox
shuaijinchao opened a new issue #4390: URL: https://github.com/apache/incubator-doris/issues/4390 `Email` is an important communication method in the `Apache` project. I think the way to subscribe to the developer mailing list should be put in the README so that more people can see and par

[GitHub] [incubator-doris] shuaijinchao opened a new pull request #4391: doc: increase mailing list subscription method.

2020-08-18 Thread GitBox
shuaijinchao opened a new pull request #4391: URL: https://github.com/apache/incubator-doris/pull/4391 ## Proposed changes FIX #4390 ## Types of changes - [x] Documentation Update (increase mailing list subscription method) ## Checklist - [x] I have create

[GitHub] [incubator-doris] morningman commented on a change in pull request #4378: FIX: fix dynamic partition replicationNum error

2020-08-18 Thread GitBox
morningman commented on a change in pull request #4378: URL: https://github.com/apache/incubator-doris/pull/4378#discussion_r472084694 ## File path: fe/fe-core/src/main/java/org/apache/doris/catalog/Catalog.java ## @@ -3973,43 +3974,55 @@ public static void getDdlStmt(Table tab

[GitHub] [incubator-doris] ZhangYu0123 commented on pull request #4366: Optimise coding bit operation in BE

2020-08-18 Thread GitBox
ZhangYu0123 commented on pull request #4366: URL: https://github.com/apache/incubator-doris/pull/4366#issuecomment-675405126 > > > Not intended to interfere, just curious about how many improvements can be achieved from this PR, are there any benchmarks? > > > > > > In my develop

[GitHub] [incubator-doris] stalary commented on a change in pull request #4378: FIX: fix dynamic partition replicationNum error

2020-08-18 Thread GitBox
stalary commented on a change in pull request #4378: URL: https://github.com/apache/incubator-doris/pull/4378#discussion_r472088500 ## File path: fe/fe-core/src/main/java/org/apache/doris/catalog/Catalog.java ## @@ -3973,43 +3974,55 @@ public static void getDdlStmt(Table table,

[GitHub] [incubator-doris] morningman commented on a change in pull request #4383: [SparkLoad]Use the yarn command to get status and kill the application

2020-08-18 Thread GitBox
morningman commented on a change in pull request #4383: URL: https://github.com/apache/incubator-doris/pull/4383#discussion_r472090745 ## File path: fe/fe-core/src/main/java/org/apache/doris/load/loadv2/YarnApplicationReport.java ## @@ -0,0 +1,121 @@ +// Licensed to the Apache

[GitHub] [incubator-doris] yangzhg commented on a change in pull request #4212: Compaction rules optimization

2020-08-18 Thread GitBox
yangzhg commented on a change in pull request #4212: URL: https://github.com/apache/incubator-doris/pull/4212#discussion_r472006650 ## File path: be/src/olap/cumulative_compaction_policy.h ## @@ -0,0 +1,263 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// o

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #4253: Support more materialized view syntax

2020-08-18 Thread GitBox
EmmyMiao87 commented on a change in pull request #4253: URL: https://github.com/apache/incubator-doris/pull/4253#discussion_r472123863 ## File path: fe/fe-core/src/main/java/org/apache/doris/analysis/DropMaterializedViewStmt.java ## @@ -38,38 +43,91 @@ */ public class DropM

[GitHub] [incubator-doris] morningman opened a new pull request #4392: [Bug] Remove RECOVER_TABLET worker pool to make ASAN compile happy

2020-08-18 Thread GitBox
morningman opened a new pull request #4392: URL: https://github.com/apache/incubator-doris/pull/4392 In PR #4255, I missed to remove some code This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [incubator-doris] marising commented on a change in pull request #4330: [Feature][Cache] Sql cache and partition cache #2581

2020-08-18 Thread GitBox
marising commented on a change in pull request #4330: URL: https://github.com/apache/incubator-doris/pull/4330#discussion_r472246168 ## File path: fe/fe-core/src/main/java/org/apache/doris/qe/cache/PartitionCache.java ## @@ -0,0 +1,215 @@ +// Licensed to the Apache Software Fo

[GitHub] [incubator-doris] marising commented on a change in pull request #4330: [Feature][Cache] Sql cache and partition cache #2581

2020-08-18 Thread GitBox
marising commented on a change in pull request #4330: URL: https://github.com/apache/incubator-doris/pull/4330#discussion_r472259742 ## File path: fe/fe-core/src/main/java/org/apache/doris/qe/cache/CacheAnalyzer.java ## @@ -0,0 +1,450 @@ +// Licensed to the Apache Software Foun

[GitHub] [incubator-doris] marising commented on a change in pull request #4330: [Feature][Cache] Sql cache and partition cache #2581

2020-08-18 Thread GitBox
marising commented on a change in pull request #4330: URL: https://github.com/apache/incubator-doris/pull/4330#discussion_r472266869 ## File path: fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java ## @@ -575,6 +583,78 @@ private void handleSetStmt() { cont

[GitHub] [incubator-doris] marising commented on a change in pull request #4330: [Feature][Cache] Sql cache and partition cache #2581

2020-08-18 Thread GitBox
marising commented on a change in pull request #4330: URL: https://github.com/apache/incubator-doris/pull/4330#discussion_r472277065 ## File path: fe/fe-core/src/main/java/org/apache/doris/qe/cache/PartitionRange.java ## @@ -0,0 +1,596 @@ +// Licensed to the Apache Software Fo

[GitHub] [incubator-doris] morningman merged pull request #4212: Compaction rules optimization

2020-08-18 Thread GitBox
morningman merged pull request #4212: URL: https://github.com/apache/incubator-doris/pull/4212 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[incubator-doris] branch master updated: [Compaction]Compaction rules optimization (#4212)

2020-08-18 Thread morningman
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new dc3ed1c [Compaction]Compaction rul

[GitHub] [incubator-doris] HangyuanLiu commented on a change in pull request #4253: Support more materialized view syntax

2020-08-18 Thread GitBox
HangyuanLiu commented on a change in pull request #4253: URL: https://github.com/apache/incubator-doris/pull/4253#discussion_r472590847 ## File path: fe/fe-core/src/main/java/org/apache/doris/analysis/DropMaterializedViewStmt.java ## @@ -38,38 +43,91 @@ */ public class Drop

[incubator-doris] branch master updated: Support udaf_orthogonal_bitmap (#4198)

2020-08-18 Thread lingmiao
This is an automated email from the ASF dual-hosted git repository. lingmiao pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new f924282 Support udaf_orthogonal_bitm

[GitHub] [incubator-doris] EmmyMiao87 merged pull request #4198: Add bitmap longitudinal cutting udaf

2020-08-18 Thread GitBox
EmmyMiao87 merged pull request #4198: URL: https://github.com/apache/incubator-doris/pull/4198 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [incubator-doris] marising commented on issue #4370: Release Nodes 0.13.0

2020-08-18 Thread GitBox
marising commented on issue #4370: URL: https://github.com/apache/incubator-doris/issues/4370#issuecomment-675815496 Please merge the feature: [Feature][Cache] Doris caches query results based on partition #2581 LiHaibo 2020-8-19

[GitHub] [incubator-doris] marising commented on a change in pull request #4330: [Feature][Cache] Sql cache and partition cache #2581

2020-08-18 Thread GitBox
marising commented on a change in pull request #4330: URL: https://github.com/apache/incubator-doris/pull/4330#discussion_r472609245 ## File path: fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java ## @@ -575,6 +583,78 @@ private void handleSetStmt() { cont

[GitHub] [incubator-doris] marising commented on a change in pull request #4330: [Feature][Cache] Sql cache and partition cache #2581

2020-08-18 Thread GitBox
marising commented on a change in pull request #4330: URL: https://github.com/apache/incubator-doris/pull/4330#discussion_r472611199 ## File path: fe/fe-core/src/main/java/org/apache/doris/qe/cache/PartitionRange.java ## @@ -0,0 +1,596 @@ +// Licensed to the Apache Software Fo

[GitHub] [incubator-doris] stalary closed pull request #4378: FIX: fix dynamic partition replicationNum error

2020-08-18 Thread GitBox
stalary closed pull request #4378: URL: https://github.com/apache/incubator-doris/pull/4378 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [incubator-doris] stalary commented on pull request #4393: FIX: fix dynamic partition replicationNum error

2020-08-18 Thread GitBox
stalary commented on pull request #4393: URL: https://github.com/apache/incubator-doris/pull/4393#issuecomment-675820778 The previous branch did not work properly, so I recreated PR. @morningman This is an automated message

[GitHub] [incubator-doris] stalary opened a new pull request #4393: FIX: fix dynamic partition replicationNum error

2020-08-18 Thread GitBox
stalary opened a new pull request #4393: URL: https://github.com/apache/incubator-doris/pull/4393 ## Proposed changes dynamic_partition.replication_num default is replication_num, but show create table show -1 ## Types of changes What types of changes does your code introdu

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #4375: Fix errors when alter materialized view which based on dup table

2020-08-18 Thread GitBox
EmmyMiao87 commented on a change in pull request #4375: URL: https://github.com/apache/incubator-doris/pull/4375#discussion_r472626637 ## File path: fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java ## @@ -556,6 +556,10 @@ private void addColumnInternal(

[GitHub] [incubator-doris] HappenLee opened a new issue #4394: [Proposal] Support Bucket Shuffle Join for Doris

2020-08-18 Thread GitBox
HappenLee opened a new issue #4394: URL: https://github.com/apache/incubator-doris/issues/4394 ## Motivation At present, Doris support 3 type join: **shuffle join**, **broadcast join**, **colocate join**. Except colocate join,another join will lead to a lot of network consumption.