[GitHub] [incubator-doris] imay commented on a change in pull request #1154: Add logic of cancel job
imay commented on a change in pull request #1154: Add logic of cancel job
URL: https://github.com/apache/incubator-doris/pull/1154#discussion_r283659159
##
File path: fe/src/main/java/org/apache/doris/load/loadv2/LoadManager.java
##
@@ -113,10 +114,11 @@ public void cancelLoadJob(CancelLoadStmt stmt) throws
DdlException {
if (!dbIdToLabelToLoadJobs.containsKey(db.getId())) {
throw new DdlException("Load job does not exist");
}
-if
(!dbIdToLabelToLoadJobs.get(db.getId()).containsKey(stmt.getLabel())) {
+Map> labelToLoadJobs =
dbIdToLabelToLoadJobs.get(db.getId());
+if (!labelToLoadJobs.containsKey(stmt.getLabel())) {
throw new DdlException("Load job does not exist");
}
-List loadJobList =
dbIdToLabelToLoadJobs.get(db.getId()).get(stmt.getLabel());
+List loadJobList = labelToLoadJobs.get(stmt.getLabel());
Review comment:
Hash the key is the first operation, `containsKey` will also check if two
keys are equal, because hash of two keys may equal. And these operation's cost
is less than a `get` operation.
Yes, in this case you add a readLock. But what I want to say is that it is
good habit to check get's value
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] imay commented on issue #1157: Use inexistent key (in prefix-index) to filter rows will cost too much time to return.
imay commented on issue #1157: Use inexistent key (in prefix-index) to filter rows will cost too much time to return. URL: https://github.com/apache/incubator-doris/issues/1157#issuecomment-492115378 It seems like a bad case for Doris, We will look into it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #1145: BE process exits because of failure of loading index
morningman closed issue #1145: BE process exits because of failure of loading index URL: https://github.com/apache/incubator-doris/issues/1145 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #1099: Broker load is not cancelled even if some instances already failed
morningman closed issue #1099: Broker load is not cancelled even if some instances already failed URL: https://github.com/apache/incubator-doris/issues/1099 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #1111: NullpointerException encountered in transaction process
morningman closed issue #: NullpointerException encountered in transaction process URL: https://github.com/apache/incubator-doris/issues/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #903: Modify the method to obtain some metrics
morningman closed issue #903: Modify the method to obtain some metrics URL: https://github.com/apache/incubator-doris/issues/903 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #794: Use funciton (count and HLL_UNION_AGG) in one column cause doris return wrong value
morningman closed issue #794: Use funciton (count and HLL_UNION_AGG) in one column cause doris return wrong value URL: https://github.com/apache/incubator-doris/issues/794 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #1148: Partition pruning support more function
morningman closed issue #1148: Partition pruning support more function URL: https://github.com/apache/incubator-doris/issues/1148 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #1115: Add str_to_date function in fe
morningman closed issue #1115: Add str_to_date function in fe URL: https://github.com/apache/incubator-doris/issues/1115 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #1090: Bugs when using function in both stream load request and routine load job
morningman closed issue #1090: Bugs when using function in both stream load request and routine load job URL: https://github.com/apache/incubator-doris/issues/1090 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #681: BE can't start
morningman commented on issue #681: BE can't start URL: https://github.com/apache/incubator-doris/issues/681#issuecomment-492119644 related to #1145 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #681: BE can't start
morningman closed issue #681: BE can't start URL: https://github.com/apache/incubator-doris/issues/681 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #677: Running partition mark does not remove when delete operation encounter errors
morningman closed issue #677: Running partition mark does not remove when delete operation encounter errors URL: https://github.com/apache/incubator-doris/issues/677 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #679: Did not clear dpp temp file after load finished
morningman closed issue #679: Did not clear dpp temp file after load finished URL: https://github.com/apache/incubator-doris/issues/679 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #768: Add HLL colum by alter table result in doris all be core dump
morningman closed issue #768: Add HLL colum by alter table result in doris all be core dump URL: https://github.com/apache/incubator-doris/issues/768 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #774: BE core when build in debug mode for es table
morningman closed issue #774: BE core when build in debug mode for es table URL: https://github.com/apache/incubator-doris/issues/774 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #1154: Add logic of cancel job
EmmyMiao87 commented on a change in pull request #1154: Add logic of cancel job
URL: https://github.com/apache/incubator-doris/pull/1154#discussion_r283668614
##
File path: fe/src/main/java/org/apache/doris/load/loadv2/LoadManager.java
##
@@ -113,10 +114,11 @@ public void cancelLoadJob(CancelLoadStmt stmt) throws
DdlException {
if (!dbIdToLabelToLoadJobs.containsKey(db.getId())) {
throw new DdlException("Load job does not exist");
}
-if
(!dbIdToLabelToLoadJobs.get(db.getId()).containsKey(stmt.getLabel())) {
+Map> labelToLoadJobs =
dbIdToLabelToLoadJobs.get(db.getId());
+if (!labelToLoadJobs.containsKey(stmt.getLabel())) {
throw new DdlException("Load job does not exist");
}
-List loadJobList =
dbIdToLabelToLoadJobs.get(db.getId()).get(stmt.getLabel());
+List loadJobList = labelToLoadJobs.get(stmt.getLabel());
Review comment:
Yes, you are right. If I only check the key, the containsKey is better. If I
need to check and get in concurrent situation, the get and check null is better.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #936: Fail to load indices due to balance when adding be
morningman closed issue #936: Fail to load indices due to balance when adding be URL: https://github.com/apache/incubator-doris/issues/936 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #936: Fail to load indices due to balance when adding be
morningman commented on issue #936: Fail to load indices due to balance when adding be URL: https://github.com/apache/incubator-doris/issues/936#issuecomment-492120776 related to ISSUE #1145 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #955: Constant predicate related bug
morningman closed issue #955: Constant predicate related bug URL: https://github.com/apache/incubator-doris/issues/955 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #1073: No rule to make target
morningman closed issue #1073: No rule to make target URL: https://github.com/apache/incubator-doris/issues/1073 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #1073: No rule to make target
morningman commented on issue #1073: No rule to make target URL: https://github.com/apache/incubator-doris/issues/1073#issuecomment-492121399 It does not reproduce in my environment, may be rebuild thrift can solve this problem. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #28: Fedora编译be模块失败
morningman closed issue #28: Fedora编译be模块失败 URL: https://github.com/apache/incubator-doris/issues/28 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #14: 编译be出错，重复定义struct std::hash<__int128>
morningman commented on issue #14: 编译be出错，重复定义struct std::hash<__int128> URL: https://github.com/apache/incubator-doris/issues/14#issuecomment-492121797 Not a problem anymore, using docker dev environment instead This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #28: Fedora编译be模块失败
morningman commented on issue #28: Fedora编译be模块失败 URL: https://github.com/apache/incubator-doris/issues/28#issuecomment-492121864 Not a problem anymore, using docker dev environment instead This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #14: 编译be出错，重复定义struct std::hash<__int128>
morningman closed issue #14: 编译be出错，重复定义struct std::hash<__int128> URL: https://github.com/apache/incubator-doris/issues/14 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #97: ubuntu16.04下编译palo报错
morningman closed issue #97: ubuntu16.04下编译palo报错 URL: https://github.com/apache/incubator-doris/issues/97 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #97: ubuntu16.04下编译palo报错
morningman commented on issue #97: ubuntu16.04下编译palo报错 URL: https://github.com/apache/incubator-doris/issues/97#issuecomment-492122007 Not a problem anymore, using docker dev environment instead This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #109: ERROR 5036 (HY000): exec rpc errorUNKONWN
morningman commented on issue #109: ERROR 5036 (HY000): exec rpc errorUNKONWN URL: https://github.com/apache/incubator-doris/issues/109#issuecomment-492122188 Reopen it if happening again This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #116: Readme文档写得真不错，3个问题，希望能够回答
morningman closed issue #116: Readme文档写得真不错，3个问题，希望能够回答 URL: https://github.com/apache/incubator-doris/issues/116 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] chaoyli commented on issue #1157: Use inexistent key (in prefix-index) to filter rows will cost too much time to return.
chaoyli commented on issue #1157: Use inexistent key (in prefix-index) to filter rows will cost too much time to return. URL: https://github.com/apache/incubator-doris/issues/1157#issuecomment-492122337 There may be has something with VARCHAR type. You can try it in another type. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #109: ERROR 5036 (HY000): exec rpc errorUNKONWN
morningman closed issue #109: ERROR 5036 (HY000): exec rpc errorUNKONWN URL: https://github.com/apache/incubator-doris/issues/109 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #485: Backend start fail cause unable to find tablet
morningman closed issue #485: Backend start fail cause unable to find tablet URL: https://github.com/apache/incubator-doris/issues/485 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #485: Backend start fail cause unable to find tablet
morningman commented on issue #485: Backend start fail cause unable to find tablet URL: https://github.com/apache/incubator-doris/issues/485#issuecomment-492123237 related to #1145 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #522: Index id in Partition instance is different from index id in OlapTable instance
morningman closed issue #522: Index id in Partition instance is different from index id in OlapTable instance URL: https://github.com/apache/incubator-doris/issues/522 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #444: Sometimes insert operation does not generate rollup's data file
morningman closed issue #444: Sometimes insert operation does not generate rollup's data file URL: https://github.com/apache/incubator-doris/issues/444 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #455: Failed to build doris
morningman closed issue #455: Failed to build doris URL: https://github.com/apache/incubator-doris/issues/455 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #406: Doris 0.9.0-incubating release notes
morningman closed issue #406: Doris 0.9.0-incubating release notes URL: https://github.com/apache/incubator-doris/issues/406 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #405: The Release of Doris 0.9.0-incubating
morningman closed issue #405: The Release of Doris 0.9.0-incubating URL: https://github.com/apache/incubator-doris/issues/405 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #365: Failed to start Doris BE on centos 7
morningman closed issue #365: Failed to start Doris BE on centos 7 URL: https://github.com/apache/incubator-doris/issues/365 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #346: BE olap_common_test couldn't run
morningman closed issue #346: BE olap_common_test couldn't run URL: https://github.com/apache/incubator-doris/issues/346 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #353: Support loading data from Kafka
morningman closed issue #353: Support loading data from Kafka URL: https://github.com/apache/incubator-doris/issues/353 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #307: The BE storage_root_path is set to a soft symlink, causing the load task to stay in the QUORUM_FINIHSED state
morningman closed issue #307: The BE storage_root_path is set to a soft symlink, causing the load task to stay in the QUORUM_FINIHSED state URL: https://github.com/apache/incubator-doris/issues/307 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #297: Support for custom gcc paths
morningman closed issue #297: Support for custom gcc paths URL: https://github.com/apache/incubator-doris/issues/297 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #517: build local source code with docker met error "mkdir: cannot create directory 'build': File exists"
morningman commented on issue #517: build local source code with docker met error "mkdir: cannot create directory 'build': File exists" URL: https://github.com/apache/incubator-doris/issues/517#issuecomment-492127678 plz use docker dev env to build from source This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #517: build local source code with docker met error "mkdir: cannot create directory 'build': File exists"
morningman closed issue #517: build local source code with docker met error "mkdir: cannot create directory 'build': File exists" URL: https://github.com/apache/incubator-doris/issues/517 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #242: Poor query performance under high concurrence
morningman commented on issue #242: Poor query performance under high concurrence URL: https://github.com/apache/incubator-doris/issues/242#issuecomment-492129186 Reopen it if it still a problem This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #242: Poor query performance under high concurrence
morningman closed issue #242: Poor query performance under high concurrence URL: https://github.com/apache/incubator-doris/issues/242 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #211: Improve exception safety with smart pointers
morningman closed issue #211: Improve exception safety with smart pointers URL: https://github.com/apache/incubator-doris/issues/211 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #211: Improve exception safety with smart pointers
morningman commented on issue #211: Improve exception safety with smart pointers URL: https://github.com/apache/incubator-doris/issues/211#issuecomment-492129837 Yes, we already change some ptr to smart pointer like shared_ptr or unique_ptr. Just close this issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #199: Palo导入数据问题
morningman closed issue #199: Palo导入数据问题 URL: https://github.com/apache/incubator-doris/issues/199 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #199: Palo导入数据问题
morningman commented on issue #199: Palo导入数据问题 URL: https://github.com/apache/incubator-doris/issues/199#issuecomment-492130804 目前我们提供了流式导入的接口，外层需要适配。详细情况可以见wiki This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] elfring commented on issue #211: Improve exception safety with smart pointers
elfring commented on issue #211: Improve exception safety with smart pointers URL: https://github.com/apache/incubator-doris/issues/211#issuecomment-492132819 How are the chances to [fix the class “TaskWorkerPool”](https://github.com/apache/incubator-doris/blob/a08170fd501c9a41aa0ce5a7811bbf2f64483308/be/src/agent/task_worker_pool.cpp#L91 "Update candidate")? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] imay merged pull request #1154: Add logic of cancel job
imay merged pull request #1154: Add logic of cancel job URL: https://github.com/apache/incubator-doris/pull/1154 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman opened a new pull request #1158: Fix bug that bad tablet blocking compaction of other tablets
morningman opened a new pull request #1158: Fix bug that bad tablet blocking compaction of other tablets URL: https://github.com/apache/incubator-doris/pull/1158 A bad tablet is always be chosen to do compaction, and failed again and again, which may block compaction of other tablets. Add a BE config 'min_compaction_failure_interval_ms' to avoid choosing bad tablet again at a certain interval, so that other tablets have chance to do the compaction. Also fix a bug that using avg() function on varchar column return unexpected exception. ISSUE: #363 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei opened a new pull request #1159: Check _empty when convert old files or make snapshots
yiguolei opened a new pull request #1159: Check _empty when convert old files or make snapshots URL: https://github.com/apache/incubator-doris/pull/1159 1. check _empty when convert old files or make snapshots 2. change segment group id from -1 to 0 when convert old files. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] kangpinghuang commented on a change in pull request #1159: Check _empty when convert old files or make snapshots
kangpinghuang commented on a change in pull request #1159: Check _empty when
convert old files or make snapshots
URL: https://github.com/apache/incubator-doris/pull/1159#discussion_r284051285
##
File path: be/src/olap/olap_snapshot_converter.cpp
##
@@ -497,4 +502,21 @@ OLAPStatus OlapSnapshotConverter::save(const string&
file_path, const OLAPHeader
return OLAP_SUCCESS;
}
+void OlapSnapshotConverter::_modify_old_segment_group_id(RowsetMetaPB&
rowset_meta) {
+if (!rowset_meta.has_alpha_rowset_extra_meta_pb()) {
+return;
+}
+AlphaRowsetExtraMetaPB* alpha_rowset_extra_meta_pb =
rowset_meta.mutable_alpha_rowset_extra_meta_pb();
+for (auto& segment_group_pb :
alpha_rowset_extra_meta_pb->segment_groups()) {
+if (segment_group_pb.segment_group_id() == -1) {
+// check if segment groups size == 1
+if (alpha_rowset_extra_meta_pb->segment_groups().size() != 1) {
+LOG(FATAL) << "the rowset has a segment group's id == -1 but
it contains more than one segment group"
+ << " it should not happen";
+}
+
(const_cast(segment_group_pb)).set_segment_group_id(0);
Review comment:
do you rename the old file name to new file name when segment group id is -1?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] chaoyli opened a new pull request #1160: Skip tablet under compaction when choose candidate
chaoyli opened a new pull request #1160: Skip tablet under compaction when choose candidate URL: https://github.com/apache/incubator-doris/pull/1160 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] imay commented on a change in pull request #1160: Skip tablet under compaction when choose candidate
imay commented on a change in pull request #1160: Skip tablet under compaction when choose candidate URL: https://github.com/apache/incubator-doris/pull/1160#discussion_r284083165 ## File path: be/src/olap/segment_group.cpp ## @@ -568,6 +568,7 @@ OLAPStatus SegmentGroup::add_short_key(const RowCursor& short_key, const uint32_ _check_io_error(res); return res; } +_file_created=true; Review comment: ```suggestion _file_created = true; ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] chaoyli commented on a change in pull request #1158: Fix bug that bad tablet blocking compaction of other tablets
chaoyli commented on a change in pull request #1158: Fix bug that bad tablet
blocking compaction of other tablets
URL: https://github.com/apache/incubator-doris/pull/1158#discussion_r284084857
##
File path: be/src/common/config.h
##
@@ -248,6 +248,10 @@ namespace config {
CONF_Int64(cumulative_compaction_budgeted_bytes, "104857600");
CONF_Int32(cumulative_compaction_write_mbytes_per_sec, "100");
+// if compaction of a tablet failed, this tablet should not be chosen to
+// compaction until this interval passes.
+CONF_Int64(min_compaction_failure_interval_ms, "60") // 10 min
Review comment:
Unify to seconds.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] chaoyli merged pull request #1158: Fix bug that bad tablet blocking compaction of other tablets
chaoyli merged pull request #1158: Fix bug that bad tablet blocking compaction of other tablets URL: https://github.com/apache/incubator-doris/pull/1158 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] chaoyli commented on a change in pull request #1158: Fix bug that bad tablet blocking compaction of other tablets
chaoyli commented on a change in pull request #1158: Fix bug that bad tablet
blocking compaction of other tablets
URL: https://github.com/apache/incubator-doris/pull/1158#discussion_r284083155
##
File path: be/src/common/config.h
##
@@ -248,6 +248,10 @@ namespace config {
CONF_Int64(cumulative_compaction_budgeted_bytes, "104857600");
CONF_Int32(cumulative_compaction_write_mbytes_per_sec, "100");
+// if compaction of a tablet failed, this tablet should not be chosen to
+// compaction until this interval passes.
+CONF_Int64(min_compaction_failure_interval_ms, "60") // 10 min
Review comment:
cumulative compaction should have another interval?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] chaoyli merged pull request #1160: Skip tablet under compaction when choose candidate
chaoyli merged pull request #1160: Skip tablet under compaction when choose candidate URL: https://github.com/apache/incubator-doris/pull/1160 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei opened a new pull request #1161: One thread per disk in compaction
yiguolei opened a new pull request #1161: One thread per disk in compaction URL: https://github.com/apache/incubator-doris/pull/1161 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request #1161: One thread per disk in compaction
morningman commented on a change in pull request #1161: One thread per disk in
compaction
URL: https://github.com/apache/incubator-doris/pull/1161#discussion_r284097282
##
File path: be/src/olap/olap_server.cpp
##
@@ -57,23 +57,30 @@ OLAPStatus OLAPEngine::_start_bg_worker() {
[this] {
_unused_index_thread_callback(nullptr);
});
-
+
+uint32_t file_system_num = get_file_system_count();
Review comment:
rename to get_store_num() is better
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #1074: whether there is an QQ group or wechat group?
morningman commented on issue #1074: whether there is an QQ group or wechat group? URL: https://github.com/apache/incubator-doris/issues/1074#issuecomment-492516894 Currently Wechat is enough, because there isn't much people of our age using QQ ... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #1074: whether there is an QQ group or wechat group?
morningman closed issue #1074: whether there is an QQ group or wechat group? URL: https://github.com/apache/incubator-doris/issues/1074 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #581: install document issue
morningman closed issue #581: install document issue URL: https://github.com/apache/incubator-doris/issues/581 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #639: LICENSE is missing MIT licensed Sizzle.js
morningman closed issue #639: LICENSE is missing MIT licensed Sizzle.js URL: https://github.com/apache/incubator-doris/issues/639 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] chaoyli closed issue #593: TabletMeta in RocksDB is not trashed
chaoyli closed issue #593: TabletMeta in RocksDB is not trashed URL: https://github.com/apache/incubator-doris/issues/593 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #558: Add column to multi rollup index in one ALTER TABLE stmt
morningman closed issue #558: Add column to multi rollup index in one ALTER TABLE stmt URL: https://github.com/apache/incubator-doris/issues/558 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #555: Meaningless saving tablet_meta leads to too many sync
morningman closed issue #555: Meaningless saving tablet_meta leads to too many sync URL: https://github.com/apache/incubator-doris/issues/555 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #555: Meaningless saving tablet_meta leads to too many sync
morningman commented on issue #555: Meaningless saving tablet_meta leads to too many sync URL: https://github.com/apache/incubator-doris/issues/555#issuecomment-492517479 This is already fixed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #661: delete data from table is not user-friendly
morningman closed issue #661: delete data from table is not user-friendly URL: https://github.com/apache/incubator-doris/issues/661 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #554: ERROR 1064 (HY000): convert_tzcan't support
morningman closed issue #554: ERROR 1064 (HY000): convert_tzcan't support URL: https://github.com/apache/incubator-doris/issues/554 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #1161: One thread per disk in compaction
morningman merged pull request #1161: One thread per disk in compaction URL: https://github.com/apache/incubator-doris/pull/1161 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
Re:Re: [DISCUSS] Propose to release Apache Doris(incubating) 0.10.0
Yes, there are still some pending issues, we will fix them later. You can check list here: https://github.com/apache/incubator-doris/issues?q=is%3Aissue+is%3Aopen+label%3Abug+milestone%3A0.11 And I forgot to paste the document of Doris on ES, which can be found here: https://github.com/apache/incubator-doris/blob/master/docs/documentation/cn/extending-doris/doris-on-es.md -- 此致！Best Regards 陈明雨 Mingyu Chen Email: chenmin...@apache.org At 2019-05-14 14:47:25, "Zhao Chun" wrote: >We have some pending issues which are belong to 0.10.0. Before we release >we need to handle these issues. > >Thanks, >ZHAO Chun > > > >Zhao Chun 于2019年5月14日周二 下午2:13写道： > >> Yes, It has been a long time since we release 0.9.0. And we have developed >> some interesting features. >> >> give my +1 for releasing our new version. >> >> Thanks for your job Mingyu. >> >> Thanks, >> ZHAO Chun >> >> >> >> 陈明雨 于2019年5月14日周二 下午1:59写道： >> >>> Hi all, >>> Somethings delayed our 0.10.0 release. >>> I think it is time to release the version 0.10.0 as some exciting feature >>> are fully implements. >>> The major features show as bellow: >>> >>> >>> New Features: >>> >>> >>> * Support Routine Load >>> >>> >>> Doris now support routine load job, which allow user to create a routine >>> load job with simple instruction, and the data will be consumed and loaded >>> into Doris automatically. More information and guide can be found in >>> [例行导入使用手册]( >>> https://github.com/apache/incubator-doris/blob/master/docs/documentation/cn/administrator-guide/load-data/routine-load-manual.md >>> ). >>> >>> >>> * Support Doris on ES >>> >>> >>> Doris now support querying Elasticsearch. User can create an extern table >>> with engine type `ES`, and query it as a noraml table. Doris also support a >>> new built-in function `es_query()`, which allow user to write special >>> Elasticsearch query language in SQL. More information and guide can be >>> found in []() >>> >>> >>> * Support UDF and UDAF >>> >>> >>> Now user can write UDF and UDAF in C++ language. >>> >>> >>> * New documentation framework >>> >>> >>> All documents of Doris will be moved to the directory >>> `docs/documentation/`. You can see [README]( >>> https://github.com/apache/incubator-doris/blob/master/docs/readme.md) to >>> get more information, and help us complete the documentations. Any >>> documents are welcome. >>> There is also a new directory `samples/` which contains some samples of >>> using Doris. >>> >>> >>> Enhancement: >>> >>> >>> * High performance Decimal type implementation.(#727) >>> * Parallel execution of fragment instance (#851) >>> * Support hll_union_agg in Analytic Function (#819) >>> * Support hll_raw_agg in Aggregate Function (#832) >>> * Support adding columns to multiple rollup indexes in one ALTER TABLE >>> stmt (#931) >>> >>> >>> Usability: >>> >>> >>> * Add --daemon option to all start scripts (#642) >>> * Remove the restrict that delete stmt must specify partition even for >>> unpartitioned table (#668) >>> * Remove restrict of that Analytic function must have partition by clause >>> (#659) >>> * Support `enable_insert_strict` session variable to control the insert >>> request (#1013) >>> * Support NEGATIVE keyword in Broker Load (#1101) >>> * Support more functions which can be calculated in Frontends: >>> unix_timestamp(), str_to_date(), current_user(), user() >>> * Support more built-in functions: money_format(), left(), right() >>> * New metrics: disk_state, tablet_distribution, scheduled_tablet_num >>> * Forward some requests to Master Frontend to get accurate results (#944) >>> >>> >>> Bug fixed: >>> >>> >>> * A lot of bugs fixed. Please see [ISSUES]( >>> https://github.com/apache/incubator-doris/issues) to get more >>> information. >>> >>> >>> -- >>> 此致！Best Regards >>> 陈明雨 Mingyu Chen >>> >>> Email: >>> chenmin...@apache.org >>> >>> >>> >>> At 2019-04-19 12:01:51, "德 李" wrote: >>> >Hi all, >>> > >>> >I propose to discuss the release of Doris 0.10.0. >>> > >>> >It will include some important features, and you can also add other >>> feature >>> >to below list: >>> > >>> >1. Unified Loading Framework >>> >You can refer to the email "[DISCUSS] Unify the load framework in Doris". >>> >a) Improve the performance of broker and mini load by using the >>> framework of >>> >stream load. >>> >b) The loading of large file is supported by the unified framework. >>> > >>> >2. Routine Load Job >>> >a) Introduce Build-in routine load manager >>> >b) Automatically Data Consuming and Loading >>> > >>> >3. Doris on ElasticSearch >>> >It enable us to query ElasticSearch on Doris by SQL way. >>> > >>> >4. Performance Optimization >>> >a) Introduce high-efficient Decimal >>> >b) Parallel Fragment Instance >>> > >>> >5. Usability >>> >a) Introduce UDF >>> >b) More HLL functions >>> >c) More metrics to monitor >>> >d) Bug fixed >>> > >>> >Best Regards, >>> >Reed >>> > >>> > >>> >>
[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #1154: Add logic of cancel job
EmmyMiao87 commented on a change in pull request #1154: Add logic of cancel job
URL: https://github.com/apache/incubator-doris/pull/1154#discussion_r283653171
##
File path: fe/src/main/java/org/apache/doris/load/loadv2/LoadManager.java
##
@@ -113,10 +114,11 @@ public void cancelLoadJob(CancelLoadStmt stmt) throws
DdlException {
if (!dbIdToLabelToLoadJobs.containsKey(db.getId())) {
throw new DdlException("Load job does not exist");
}
-if
(!dbIdToLabelToLoadJobs.get(db.getId()).containsKey(stmt.getLabel())) {
+Map> labelToLoadJobs =
dbIdToLabelToLoadJobs.get(db.getId());
+if (!labelToLoadJobs.containsKey(stmt.getLabel())) {
throw new DdlException("Load job does not exist");
}
-List loadJobList =
dbIdToLabelToLoadJobs.get(db.getId()).get(stmt.getLabel());
+List loadJobList = labelToLoadJobs.get(stmt.getLabel());
Review comment:
Because I think `containsKey` is more graceful then checking the result is
null is more
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] qdore commented on issue #1157: Use inexistent key (in prefix-index) to filter rows will cost too much time to return.
qdore commented on issue #1157: Use inexistent key (in prefix-index) to filter
rows will cost too much time to return.
URL:
https://github.com/apache/incubator-doris/issues/1157#issuecomment-492107186
slow query:
```
+--+
| Explain String
|
+--+
| PLAN FRAGMENT 0
|
| OUTPUT EXPRS:`event_day` | `cuid` |
|
| PARTITION: UNPARTITIONED
|
|
|
| RESULT SINK
|
|
|
| 1:EXCHANGE
|
| tuple ids: 0
|
|
|
| PLAN FRAGMENT 1
|
| OUTPUT EXPRS:
|
| PARTITION: RANDOM
|
|
|
| STREAM DATA SINK
|
| EXCHANGE ID: 01
|
| UNPARTITIONED
|
|
|
| 0:OlapScanNode
|
| TABLE: visd_dws_topic_novel_wide_table2
|
| PREAGGREGATION: OFF. Reason: No AggregateInfo
|
| PREDICATES: `event_day` = '2019-05-14 00:00:00', `cuid` IN
('5799A449415E6C8EA84B0FED6F3C|643611430939568',
'5799A449415E6C8EA84B0FED6F3C|643611430939567') |
| partitions=1/58
|
| rollup: visd_dws_topic_novel_wide_table2
|
| buckets=2/32
[GitHub] [incubator-doris] qdore commented on issue #1157: Use inexistent key (in prefix-index) to filter rows will cost too much time to return.
qdore commented on issue #1157: Use inexistent key (in prefix-index) to filter
rows will cost too much time to return.
URL:
https://github.com/apache/incubator-doris/issues/1157#issuecomment-492107502
normal query:
+-+
| Explain String
|
+-+
| PLAN FRAGMENT 0
|
| OUTPUT EXPRS:`event_day` | `cuid` |
|
| PARTITION: UNPARTITIONED
|
|
|
| RESULT SINK
|
|
|
| 1:EXCHANGE
|
| tuple ids: 0
|
|
|
| PLAN FRAGMENT 1
|
| OUTPUT EXPRS:
|
| PARTITION: RANDOM
|
|
|
| STREAM DATA SINK
|
| EXCHANGE ID: 01
|
| UNPARTITIONED
|
|
|
| 0:OlapScanNode
|
| TABLE: visd_dws_topic_novel_wide_table2
|
| PREAGGREGATION: OFF. Reason: No AggregateInfo
|
| PREDICATES: `event_day` = '2019-05-14 00:00:00', `cuid` IN
('0556EE50B09F7F9E378E7262CF0B86E9B5EF3FHOKRADALC',
'5799A449415E6C8EA84B0FED6F3C|643611430939568') |
| partitions=1/58
|
| rollup: visd_dws_topic_novel_wide_table2
|
| buckets=2/32
[GitHub] [incubator-doris] qdore edited a comment on issue #1157: Use inexistent key (in prefix-index) to filter rows will cost too much time to return.
qdore edited a comment on issue #1157: Use inexistent key (in prefix-index) to
filter rows will cost too much time to return.
URL:
https://github.com/apache/incubator-doris/issues/1157#issuecomment-492107502
normal query:
```
+-+
| Explain String
|
+-+
| PLAN FRAGMENT 0
|
| OUTPUT EXPRS:`event_day` | `cuid` |
|
| PARTITION: UNPARTITIONED
|
|
|
| RESULT SINK
|
|
|
| 1:EXCHANGE
|
| tuple ids: 0
|
|
|
| PLAN FRAGMENT 1
|
| OUTPUT EXPRS:
|
| PARTITION: RANDOM
|
|
|
| STREAM DATA SINK
|
| EXCHANGE ID: 01
|
| UNPARTITIONED
|
|
|
| 0:OlapScanNode
|
| TABLE: visd_dws_topic_novel_wide_table2
|
| PREAGGREGATION: OFF. Reason: No AggregateInfo
|
| PREDICATES: `event_day` = '2019-05-14 00:00:00', `cuid` IN
('0556EE50B09F7F9E378E7262CF0B86E9B5EF3FHOKRADALC',
'5799A449415E6C8EA84B0FED6F3C|643611430939568') |
| partitions=1/58
|
| rollup: visd_dws_topic_novel_wide_table2
|
| buckets=2/32
[GitHub] [incubator-doris] imay commented on a change in pull request #1154: Add logic of cancel job
imay commented on a change in pull request #1154: Add logic of cancel job
URL: https://github.com/apache/incubator-doris/pull/1154#discussion_r283654086
##
File path: fe/src/main/java/org/apache/doris/load/loadv2/LoadManager.java
##
@@ -113,10 +114,11 @@ public void cancelLoadJob(CancelLoadStmt stmt) throws
DdlException {
if (!dbIdToLabelToLoadJobs.containsKey(db.getId())) {
throw new DdlException("Load job does not exist");
}
-if
(!dbIdToLabelToLoadJobs.get(db.getId()).containsKey(stmt.getLabel())) {
+Map> labelToLoadJobs =
dbIdToLabelToLoadJobs.get(db.getId());
+if (!labelToLoadJobs.containsKey(stmt.getLabel())) {
throw new DdlException("Load job does not exist");
}
-List loadJobList =
dbIdToLabelToLoadJobs.get(db.getId()).get(stmt.getLabel());
+List loadJobList = labelToLoadJobs.get(stmt.getLabel());
Review comment:
I don't agree with you, you get it twice. And in some cases, If another
thread modify this map, your check can't make sure you can get a not null value
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] qdore commented on issue #1157: Use inexistent key (in prefix-index) to filter rows will cost too much time to return.
qdore commented on issue #1157: Use inexistent key (in prefix-index) to filter rows will cost too much time to return. URL: https://github.com/apache/incubator-doris/issues/1157#issuecomment-492108376 I faked the value of cuid by modify the last character, then the query become slow:  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] qdore edited a comment on issue #1157: Use inexistent key (in prefix-index) to filter rows will cost too much time to return.
qdore edited a comment on issue #1157: Use inexistent key (in prefix-index) to filter rows will cost too much time to return. URL: https://github.com/apache/incubator-doris/issues/1157#issuecomment-492108376 I faked the value of `cuid` by modify the last character, then the query become slow:  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] imay commented on issue #1157: Use inexistent key (in prefix-index) to filter rows will cost too much time to return.
imay commented on issue #1157: Use inexistent key (in prefix-index) to filter rows will cost too much time to return. URL: https://github.com/apache/incubator-doris/issues/1157#issuecomment-492108852 can you `set is_report_success = true`, and reexcute slow query, and see query's profile in frontend's web? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #1154: Add logic of cancel job
EmmyMiao87 commented on a change in pull request #1154: Add logic of cancel job
URL: https://github.com/apache/incubator-doris/pull/1154#discussion_r283656157
##
File path: fe/src/main/java/org/apache/doris/load/loadv2/LoadManager.java
##
@@ -113,10 +114,11 @@ public void cancelLoadJob(CancelLoadStmt stmt) throws
DdlException {
if (!dbIdToLabelToLoadJobs.containsKey(db.getId())) {
throw new DdlException("Load job does not exist");
}
-if
(!dbIdToLabelToLoadJobs.get(db.getId()).containsKey(stmt.getLabel())) {
+Map> labelToLoadJobs =
dbIdToLabelToLoadJobs.get(db.getId());
+if (!labelToLoadJobs.containsKey(stmt.getLabel())) {
throw new DdlException("Load job does not exist");
}
-List loadJobList =
dbIdToLabelToLoadJobs.get(db.getId()).get(stmt.getLabel());
+List loadJobList = labelToLoadJobs.get(stmt.getLabel());
Review comment:
Why you two hold opposite opinions /(ㄒoㄒ)/~~ @morningman
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #1154: Add logic of cancel job
EmmyMiao87 commented on a change in pull request #1154: Add logic of cancel job
URL: https://github.com/apache/incubator-doris/pull/1154#discussion_r283656604
##
File path: fe/src/main/java/org/apache/doris/load/loadv2/LoadManager.java
##
@@ -113,10 +114,11 @@ public void cancelLoadJob(CancelLoadStmt stmt) throws
DdlException {
if (!dbIdToLabelToLoadJobs.containsKey(db.getId())) {
throw new DdlException("Load job does not exist");
}
-if
(!dbIdToLabelToLoadJobs.get(db.getId()).containsKey(stmt.getLabel())) {
+Map> labelToLoadJobs =
dbIdToLabelToLoadJobs.get(db.getId());
+if (!labelToLoadJobs.containsKey(stmt.getLabel())) {
throw new DdlException("Load job does not exist");
}
-List loadJobList =
dbIdToLabelToLoadJobs.get(db.getId()).get(stmt.getLabel());
+List loadJobList = labelToLoadJobs.get(stmt.getLabel());
Review comment:
I do not get it twice. The containsKey does not get the value, it just hash
the key. Also, I add the read lock whether get and check null or containsKey
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] [incubator-doris] qdore commented on issue #1157: Use inexistent key (in prefix-index) to filter rows will cost too much time to return.
qdore commented on issue #1157: Use inexistent key (in prefix-index) to filter
rows will cost too much time to return.
URL:
https://github.com/apache/incubator-doris/issues/1157#issuecomment-492112331
slow profile:
```
Query:
Summary:
Query ID: a2e2f1f8a7b64d2e:9f65f45daad76595
Start Time: 2019-05-14 15:24:28
End Time: 2019-05-14 15:26:39
Total: 2m
Query Type: Query
Query State: EOF
Doris Version: 0.9.0
User: root
Default Db: default_cluster:novel
Sql Statement: select event_day,cuid from
visd_dws_topic_novel_wide_table2 where event_day = '2019-05-14' and cuid in
('5799A449415E6C8EA84B0FED6F3C|643611430939568',
'5799A449415E6C8EA84B0FED6F3C|643611430939567')
Execution Profile a2e2f1f8a7b64d2e:9f65f45daad76595:(Active: 2m, %
non-child: 0.00%)
Fragment 0:
Instance a2e2f1f8a7b64d2e:9f65f45daad76598
(host=TNetworkAddress(hostname:10.90.239.31, port:9073)):(Active: 2m, %
non-child: 0.00%)
- BytesReceived: 577.00 B
- PeakUsedReservation: 0.00
- SendersBlockedTimer: 0ns
- DeserializeRowBatchTimer: 26.305us
- RowsProduced: 41
- SendersBlockedTotalTimer(*): 0ns
- AverageThreadTokens: 4607182418800017400.00
- PeakReservation: 0.00
- FirstBatchArrivalWaitTime: 0ns
BlockMgr:
- BlocksCreated: 0
- MemoryLimit: 30.00 GB
- BlockWritesOutstanding: 0
- BytesWritten: 0.00
- TotalEncryptionTime: 0ns
- BufferedPins: 0
- TotalReadBlockTime: 0ns
- TotalBufferWaitTime: 0ns
- BlocksRecycled: 0
- TotalIntegrityCheckTime: 0ns
- MaxBlockSize: 8.00 MB
DataBufferSender
(dst_fragment_instance_id=a2e2f1f8a7b64d2e:9f65f45daad76598):
EXCHANGE_NODE (id=1):(Active: 2m, % non-child: 0.00%)
- ConvertRowBatchTime: 2.809us
- RowsReturned: 41
- MemoryUsed: 0.00
- RowsReturnedRate: 0
Fragment 1:
Instance a2e2f1f8a7b64d2e:9f65f45daad76597
(host=TNetworkAddress(hostname:10.90.239.28, port:9083)):(Active: 621.184ms, %
non-child: 0.00%)
- PeakUsedReservation: 0.00
- RowsProduced: 41
- AverageThreadTokens: 4607182418800017400.00
- PeakReservation: 0.00
BlockMgr:
- BlocksCreated: 0
- MemoryLimit: 30.00 GB
- BlockWritesOutstanding: 0
- BytesWritten: 0.00
- TotalEncryptionTime: 0ns
- BufferedPins: 0
- TotalReadBlockTime: 0ns
- TotalBufferWaitTime: 0ns
- BlocksRecycled: 0
- TotalIntegrityCheckTime: 0ns
- MaxBlockSize: 8.00 MB
DataStreamSender (dst_id=1):(Active: 641.540us, % non-child: 0.00%)
- OverallThroughput: 878.318359375 KB/sec
- SerializeBatchTime: 531.374us
- IgnoreRows: 0
- BytesSent: 577.00 B
- NetworkThroughput(*): 0.0 /sec
- ThriftTransmitTime(*): 0ns
- UncompressedRowBatchSize: 3.37 KB
OLAP_SCAN_NODE (id=0):(Active: 620.219ms, % non-child: 0.00%)
- NumThread: 0
- TotalRawReadTime(*): 0ns
- CompressedBytesRead: 12.81 MB
- RowsPushedCondFiltered: 0
- ScanRangesComplete: 0
- ScanTime: 260.732us
- RowsStatsFiltered: 1.213828M (1213828)
- VectorPredEvalTime: 43.146us
- BlockSeekTime: 885.459ms
- RawRowsRead: 34.967K (34967)
- ScannerThreadsVoluntaryContextSwitches: 0
- RowsDelFiltered: 0
- IndexLoadTime: 231.724ms
- ScannerThreadsTotalWallClockTime: 0ns
- MaterializeTupleTime(*): 0ns
- ScannerThreadsUserTime: 0ns
- ScannerThreadsSysTime: 0ns
- MemoryUsed: 0.00
- RowsReturnedRate: 66
- BlockLoadTime: 42.22ms
- BlocksLoad: 153
- UncompressedBytesRead: 24.33 MB
- TabletCount : 1
- RowsReturned: 41
- ScannerThreadsInvoluntaryContextSwitches: 0
- DecompressorTimer: 27.432ms
- RowsVectorPredFiltered: 0
- ReaderInitTime: 1s208ms
- RowsRead: 41
- PerReadThreadRawHdfsThroughput: 0.0 /sec
- BlockFetchTime: 1s204ms
- ShowHintsTime: 92.321us
- TotalReadThroughput: 0.0 /sec
- IOTimer: 1s80ms
- BytesRead: 5.85 MB
- BlockConvertTime: 27.510ms
Instance a2e2f1f8a7b64d2e:9f65f45daad76596
(host=TNetworkAddress(hostname:10.90.239.25, port
