[GitHub] [incubator-doris] yangzhg opened a new pull request #5018: [BUG] fix core when schema change

2020-12-02 Thread GitBox
yangzhg opened a new pull request #5018: URL: https://github.com/apache/incubator-doris/pull/5018 ## Proposed changes Fix #5017 ## Types of changes What types of changes does your code introduce to Doris? _Put an `x` in the boxes that apply_ - [x] Bugfix (non-b

[GitHub] [incubator-doris] yangzhg opened a new issue #5017: [BUG] schema change may be core dump

2020-12-02 Thread GitBox
yangzhg opened a new issue #5017: URL: https://github.com/apache/incubator-doris/issues/5017 **Describe the bug** schema change may be core dump , because of _next_row_func in schema change job may be wrong type This is a

[GitHub] [incubator-doris] acelyc111 opened a new issue #5016: [Bug] All TaskWorkerPools have the same incorrect name

2020-12-02 Thread GitBox
acelyc111 opened a new issue #5016: URL: https://github.com/apache/incubator-doris/issues/5016 **Describe the bug** On BE website '/threadz?group=thread%20pool' ![image](https://user-images.githubusercontent.com/10775040/100964665-188fa180-3564-11eb-8536-8f6b240a14d0.png) --

[GitHub] [incubator-doris] acelyc111 opened a new pull request #5015: [Buf] Fix incorrect name of TaskWorkerPool

2020-12-02 Thread GitBox
acelyc111 opened a new pull request #5015: URL: https://github.com/apache/incubator-doris/pull/5015 ## Proposed changes '_task_worker_type' is not well initialized when use it to init '_name', then '_name' is always 'TaskWorkerPool.CREATE_TABLE', this patch fix this bug.

[GitHub] [incubator-doris] josh7 opened a new issue #5014: build spark-dpp fail

2020-12-02 Thread GitBox
josh7 opened a new issue #5014: URL: https://github.com/apache/incubator-doris/issues/5014 [INFO] [INFO] Reactor Summary: [INFO] [INFO] fe-common 1.0.0 SUCCESS [06:58 min

[GitHub] [incubator-doris] morningman merged pull request #4991: [Enhance] Add "statistics" meta table and fix some mysql compatibility problem

2020-12-02 Thread GitBox
morningman merged pull request #4991: URL: https://github.com/apache/incubator-doris/pull/4991 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [incubator-doris] morningman merged pull request #5004: [Function] Let "str_to_date" return correct type

2020-12-02 Thread GitBox
morningman merged pull request #5004: URL: https://github.com/apache/incubator-doris/pull/5004 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [incubator-doris] morningman merged pull request #4994: [Docs]adding instructions for converting dynamic and manual partition tables to each other

2020-12-02 Thread GitBox
morningman merged pull request #4994: URL: https://github.com/apache/incubator-doris/pull/4994 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [incubator-doris] ZhaoBQ commented on issue #5011: Does doris have an official performance evaluation tool?

2020-12-02 Thread GitBox
ZhaoBQ commented on issue #5011: URL: https://github.com/apache/incubator-doris/issues/5011#issuecomment-737631599 I use UNIQUE KEY instead of primary key. I run 12 queries, and the performance of the query is not ideal, half of queries is timeout(600 sec). I don't know if this is normal.

[GitHub] [incubator-doris] ZhaoBQ commented on issue #5011: Does doris have an official performance evaluation tool?

2020-12-02 Thread GitBox
ZhaoBQ commented on issue #5011: URL: https://github.com/apache/incubator-doris/issues/5011#issuecomment-737629242 Thank you @chaoyli . Yes, I changed the primary key order, and delete the other table's last '|'. Although I did not convert the empty string to NULL, the data can also be l

[incubator-doris] branch master updated (bd558f1 -> 1ae6de7)

2020-12-02 Thread morningman
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from bd558f1 [Doris][Doris On ES] support prefix @ symbol for column name (#5006) add 1ae6de7 [Enhanc

[incubator-doris] branch master updated (4fa47bc -> 92db00b)

2020-12-02 Thread morningman
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 4fa47bc [Docs]adding instructions for converting dynamic and manual partition tables to each other (#4

[incubator-doris] branch master updated (5215727 -> bd558f1)

2020-12-02 Thread morningman
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 5215727 [Function] Let "str_to_date" return correct type (#5004) add bd558f1 [Doris][Doris On ES

[incubator-doris] branch master updated (204c151 -> 5215727)

2020-12-02 Thread morningman
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 204c151 [Bug] ConcurrentModificationException when finish transaction (#5003) add 5215727 [Funct

[incubator-doris] branch master updated (92db00b -> 204c151)

2020-12-02 Thread morningman
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 92db00b [Bug] Fix concurrent access of _tablets_under_clone in TabletManager (#5000) add 204c151

[incubator-doris] branch master updated (b4c1eab -> 4fa47bc)

2020-12-02 Thread morningman
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from b4c1eab [Bug] fix finished load jobs cost too much heap (#4993) add 4fa47bc [Docs]adding instruc

[GitHub] [incubator-doris] morningman closed issue #5005: [Doris][Doris On ES] [Proposal] Support `@` leading column name

2020-12-02 Thread GitBox
morningman closed issue #5005: URL: https://github.com/apache/incubator-doris/issues/5005 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [incubator-doris] morningman merged pull request #5006: [Doris][Doris On ES] support `@` leading column name

2020-12-02 Thread GitBox
morningman merged pull request #5006: URL: https://github.com/apache/incubator-doris/pull/5006 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [incubator-doris] morningman merged pull request #5003: (#5002) ConcurrentModificationException when finish transaction

2020-12-02 Thread GitBox
morningman merged pull request #5003: URL: https://github.com/apache/incubator-doris/pull/5003 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [incubator-doris] morningman closed issue #5002: [BUG]ConcurrentModificationException when finish transaction

2020-12-02 Thread GitBox
morningman closed issue #5002: URL: https://github.com/apache/incubator-doris/issues/5002 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [incubator-doris] morningman closed issue #5001: [Bug] Concurrent access _tablets_under_clone/restore should also sharding

2020-12-02 Thread GitBox
morningman closed issue #5001: URL: https://github.com/apache/incubator-doris/issues/5001 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [incubator-doris] morningman merged pull request #5000: [Bug] Fix concurrent access of _tablets_under_clone in TabletManager

2020-12-02 Thread GitBox
morningman merged pull request #5000: URL: https://github.com/apache/incubator-doris/pull/5000 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [incubator-doris] hyytt commented on pull request #4636: [Compile] Add pluginRepository for java-cup-plugins

2020-12-02 Thread GitBox
hyytt commented on pull request #4636: URL: https://github.com/apache/incubator-doris/pull/4636#issuecomment-737322759 > > I've found this package in: https://repository.cloudera.com/artifactory/ext-release-local/net/sourceforge/czt/dev/cup-maven-plugin/1.6-cdh/cup-maven-plugin-1.6-cdh.jar

[GitHub] [incubator-doris] morningman commented on pull request #4636: [Compile] Add pluginRepository for java-cup-plugins

2020-12-02 Thread GitBox
morningman commented on pull request #4636: URL: https://github.com/apache/incubator-doris/pull/4636#issuecomment-737311120 > I've found this package in: https://repository.cloudera.com/artifactory/ext-release-local/net/sourceforge/czt/dev/cup-maven-plugin/1.6-cdh/cup-maven-plugin-1.6-cdh.j

[GitHub] [incubator-doris] hyytt commented on pull request #4636: [Compile] Add pluginRepository for java-cup-plugins

2020-12-02 Thread GitBox
hyytt commented on pull request #4636: URL: https://github.com/apache/incubator-doris/pull/4636#issuecomment-73730 I've found this package in: https://repository.cloudera.com/artifactory/ext-release-local/net/sourceforge/czt/dev/cup-maven-plugin/1.6-cdh/cup-maven-plugin-1.6-cdh.jar , a

[GitHub] [incubator-doris] bingfeng2004 commented on issue #4967: ALL BE node is down

2020-12-02 Thread GitBox
bingfeng2004 commented on issue #4967: URL: https://github.com/apache/incubator-doris/issues/4967#issuecomment-737232399 No.It may be caused by inserting data too frequently. This is an automated message from the Apache Git

[GitHub] [incubator-doris] xinyiZzz opened a new pull request #5013: [Repair] Add an option whether to allow the partition column to be NULL

2020-12-02 Thread GitBox
xinyiZzz opened a new pull request #5013: URL: https://github.com/apache/incubator-doris/pull/5013 ## Proposed changes Add an option `allow_partition_column_nullable`, which means whether to allow the partition column to be NULL. When the option is enabled, the partition column mu

[GitHub] [incubator-doris] caoyang10 commented on issue #5009: [Proposal] Add column to all index

2020-12-02 Thread GitBox
caoyang10 commented on issue #5009: URL: https://github.com/apache/incubator-doris/issues/5009#issuecomment-737207097 I think I have found the way to solve the problem: I can use multi alter stmt like: **alter table test_table add column another_col BIGINT SUM NULL default "0" to ro

[GitHub] [incubator-doris] xinyiZzz opened a new issue #5012: [Proposal] Add an option whether to allow the partition column to be NULL

2020-12-02 Thread GitBox
xinyiZzz opened a new issue #5012: URL: https://github.com/apache/incubator-doris/issues/5012 **Is your feature request related to a problem? Please describe.** Strict data consistency is required in similar financial scenarios, when the partition column in the data is NULL, the import n

[GitHub] [incubator-doris] chaoyli commented on issue #5011: Does doris have an official performance evaluation tool?

2020-12-02 Thread GitBox
chaoyli commented on issue #5011: URL: https://github.com/apache/incubator-doris/issues/5011#issuecomment-737187685 ``` curl --location-trusted -u root: -H "max_filter_ratio:0.2" -H "column_separator:|" -T customer_address.dat http://172.26.92.139:8530/api/tpcds/customer_address/_strea

[GitHub] [incubator-doris] chaoyli commented on issue #5011: Does doris have an official performance evaluation tool?

2020-12-02 Thread GitBox
chaoyli commented on issue #5011: URL: https://github.com/apache/incubator-doris/issues/5011#issuecomment-737187300 And also, you should convert empty string to NULL, because the data generate TPCDS will not "\N" format. ``` sed -i 's/^|/\\N|/g;s/|||/|\\N|\\N|/g;s/||/|\\N|/g;s/||/|\

[GitHub] [incubator-doris] chaoyli commented on issue #5011: Does doris have an official performance evaluation tool?

2020-12-02 Thread GitBox
chaoyli commented on issue #5011: URL: https://github.com/apache/incubator-doris/issues/5011#issuecomment-737186828 I use tpcds to test doris, but it may be change the primary key order in sales and returns. This is my create table SQL. ``` create table customer_address (

[GitHub] [incubator-doris] sunzhangbin commented on issue #4917: Routine load丢失数据

2020-12-02 Thread GitBox
sunzhangbin commented on issue #4917: URL: https://github.com/apache/incubator-doris/issues/4917#issuecomment-737185426 > > 如果你用unique 模型的话可能是因为不保证顺序,导致数据被replace 了。 > > routine load如果并发的话不同任务之间的执行是无序的,也就是说你kafka 中的 offset 靠前的数据并不一定先导入。 > > 又由于是unique 模型,后面导入成功的数据会覆盖前面的数据,所以会产生你说的丢数据的

[GitHub] [incubator-doris] stalary commented on issue #4917: Routine load丢失数据

2020-12-02 Thread GitBox
stalary commented on issue #4917: URL: https://github.com/apache/incubator-doris/issues/4917#issuecomment-737182844 > 如果你用unique 模型的话可能是因为不保证顺序,导致数据被replace 了。 > routine load如果并发的话不同任务之间的执行是无序的,也就是说你kafka 中的 offset 靠前的数据并不一定先导入。 > 又由于是unique 模型,后面导入成功的数据会覆盖前面的数据,所以会产生你说的丢数据的错觉。 >

[GitHub] [incubator-doris] EmmyMiao87 commented on issue #4917: Routine load丢失数据

2020-12-02 Thread GitBox
EmmyMiao87 commented on issue #4917: URL: https://github.com/apache/incubator-doris/issues/4917#issuecomment-737180128 如果你用unique 模型的话可能是因为不保证顺序,导致数据被replace 了。 routine load如果并发的话不同任务之间的执行是无序的,也就是说你kafka 中的 offset 靠前的数据并不一定先导入。 又由于是unique 模型,后面导入成功的数据会覆盖前面的数据,所以会产生你说的丢数据的错觉。

[GitHub] [incubator-doris] sunzhangbin commented on issue #4917: Routine load丢失数据

2020-12-02 Thread GitBox
sunzhangbin commented on issue #4917: URL: https://github.com/apache/incubator-doris/issues/4917#issuecomment-737178214 同时4个routine load job往同一张表同步数据偶尔会丢数据,改成1个routine load job往一个表同步数据没有再发现丢数据的情况?具体原因查不到,日志也看不出来异常 This

[GitHub] [incubator-doris] ZhaoBQ opened a new issue #5011: Does doris have an official performance evaluation tool?

2020-12-02 Thread GitBox
ZhaoBQ opened a new issue #5011: URL: https://github.com/apache/incubator-doris/issues/5011 We recently built a doris cluster, 3 fe, 10 be, all use default configuration. But the performance is not very good. We use TPC-DS to test the perofrmance. After setting up the test environ

[GitHub] [incubator-doris] vagetablechicken opened a new pull request #5010: [Rebalancer] support partition rebalancer

2020-12-02 Thread GitBox
vagetablechicken opened a new pull request #5010: URL: https://github.com/apache/incubator-doris/pull/5010 ## Proposed changes PartitionRebalancer based on TwoDimensionalGreedyAlgo. Two dims of Doris should be cluster & partition. And we only consider about the replica count, do n

[GitHub] [incubator-doris] caoyang10 opened a new issue #5009: [Proposal] Add column to all index

2020-12-02 Thread GitBox
caoyang10 opened a new issue #5009: URL: https://github.com/apache/incubator-doris/issues/5009 In such case, I have a base table A and two different rollup index B and C. I couldn't add column D to all index A B C while doris just support add column to A & B or A & C, using "TO B" or "TO C

[GitHub] [incubator-doris] Skysheepwang opened a new pull request #5008: [Optimize][Cache]Implementation of Separated Page Cache

2020-12-02 Thread GitBox
Skysheepwang opened a new pull request #5008: URL: https://github.com/apache/incubator-doris/pull/5008 ## Proposed changes #4995 **Implementation of Separated Page Cache** - Add config "index_page_cache_ratio" to set the ratio of capacity of index page cache - Change the member

[GitHub] [incubator-doris] EmmyMiao87 closed issue #4992: [bug] finished load jobs cost too much heap memory

2020-12-02 Thread GitBox
EmmyMiao87 closed issue #4992: URL: https://github.com/apache/incubator-doris/issues/4992 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [incubator-doris] EmmyMiao87 merged pull request #4993: [Bug] fix finished load jobs cost too much heap

2020-12-02 Thread GitBox
EmmyMiao87 merged pull request #4993: URL: https://github.com/apache/incubator-doris/pull/4993 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[incubator-doris] branch master updated: [Bug] fix finished load jobs cost too much heap (#4993)

2020-12-02 Thread lingmiao
This is an automated email from the ASF dual-hosted git repository. lingmiao pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new b4c1eab [Bug] fix finished load jobs

[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #4994: [Docs]adding instructions for converting dynamic and manual partition tables to each other

2020-12-02 Thread GitBox
EmmyMiao87 commented on a change in pull request #4994: URL: https://github.com/apache/incubator-doris/pull/4994#discussion_r533999509 ## File path: docs/zh-CN/administrator-guide/dynamic-partition.md ## @@ -321,3 +321,20 @@ mysql> SHOW DYNAMIC PARTITION TABLES; HTTP 协议:

[GitHub] [incubator-doris] jxxfxkp opened a new issue #5007: Presto query doris view error (Invalid VARCHAR length -1)

2020-12-02 Thread GitBox
jxxfxkp opened a new issue #5007: URL: https://github.com/apache/incubator-doris/issues/5007 The latest master branch supports presto query doris table.But doris view is not supported if this view has an varchar type. presto err(mysql jdbc): ``` java.lang.IllegalArgumentException: