Re: [PR] [feature](merge-cloud) Support specify meta service endpoints separated by comma [doris]
github-actions[bot] commented on PR #32230: URL: https://github.com/apache/doris/pull/32230#issuecomment-1996680528 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](merge-cloud) enable merge on write by default [doris]
liaoxin01 commented on PR #32175: URL: https://github.com/apache/doris/pull/32175#issuecomment-1996681739 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bugfix]json_length() BE crash fix [doris]
yiguolei merged PR #32145: URL: https://github.com/apache/doris/pull/32145 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (26369306009 -> ee0a461f966)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 26369306009 [feature](Nereids) support agg state type in create table (#32171) add ee0a461f966 [bugfix]json_length() BE crash fix (#32145) No new revisions were added by this update. Summary of changes: be/src/vec/functions/function_jsonb.cpp | 5 +++-- regression-test/data/jsonb_p0/test_jsonb_load_and_function.out | 3 +++ regression-test/suites/jsonb_p0/test_jsonb_load_and_function.groovy | 1 + 3 files changed, 7 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] json_length() function causes BE crash [doris]
yiguolei closed issue #32124: [Bug] json_length() function causes BE crash URL: https://github.com/apache/doris/issues/32124 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [draft][improvement](partial-update) make partial-update on agg table use less memory [doris]
morrySnow commented on code in PR #32200:
URL: https://github.com/apache/doris/pull/32200#discussion_r1524331219
##
fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java:
##
@@ -1486,6 +1488,9 @@ public void setEnableLeftZigZag(boolean enableLeftZigZag)
{
@VariableMgr.VarAttr(name = ENABLE_UNIQUE_KEY_PARTIAL_UPDATE, needForward
= true)
public boolean enableUniqueKeyPartialUpdate = false;
+@VariableMgr.VarAttr(name = ENABLE_AGG_KEY_PARTIAL_UPDATE, needForward =
true)
+public boolean enableAggregateKeyPartialUpdate = false;
Review Comment:
what's different between `enableUniqueKeyPartialUpdate` and
`enableAggregateKeyPartialUpdate`? why not use one variable to control them?
In `LogicalPlanBuilder` and `BindSink`, we cannot distinguish them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](invert index) use lowercase by default [doris]
github-actions[bot] commented on PR #32232: URL: https://github.com/apache/doris/pull/32232#issuecomment-1996695637 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Enhancement](compatible) display dateV2/datetimeV2/decimalv3 as date/datetime/decimal when show column [doris]
weizuo93 opened a new pull request, #32233: URL: https://github.com/apache/doris/pull/32233 ## Proposed changes Issue Number: close #xxx There is column with type `date/datetime/decimal`when create table，but the column type returned from `SHOW FULL COLUMNS FROM xxx`is `dateV2/datetimeV2/decimalv3`. For example: Create table: ``` CREATE TABLE `test_table` ( `a` DATETIME NULL, `b` DATE NULL, `c` DECIMAL(5, 1) NULL, `d` INT NULL ) ENGINE=OLAP DUPLICATE KEY(`a`, `b`) COMMENT 'OLAP' DISTRIBUTED BY HASH(`a`) BUCKETS 4 PROPERTIES ( "replication_allocation" = "tag.location.default: 1", "storage_format" = "V2" ); ``` Show columns: ``` mysql> SHOW FULL COLUMNS FROM test_table; +---+---+--+--+--+---+-+--+--+ | Field | Type| Collation | Null | Key | Default | Extra | Privileges | Comment | +---+---+--+--+--+---+-+--+--+ | a | datetimev2(0)| | YES | YES | NULL | || | | b | datev2 | | YES | YES | NULL | || | | c | decimalv3(5, 1) | | YES | NO | NULL | NONE || | | d | int| | YES | NO | NULL | NONE || | +---+---+--+--+--+---++---+--+ 4 rows in set (0.00 sec) ``` It is necessary to display `dateV2/datetimeV2/decimalv3` as `date/datetime/decimal` in `ShowColumnStmt`. As follows： ``` mysql> SHOW FULL COLUMNS FROM test_table; +---+---+--+--+--+---+-+--+--+ | Field | Type| Collation | Null | Key | Default | Extra | Privileges | Comment | +---+---+--+--+--+---+-+--+--+ | a | datetime | | YES | YES | NULL | || | | b | date| | YES | YES | NULL | || | | c | decimal(5, 1) | | YES | NO | NULL | NONE || | | d | int| | YES | NO | NULL | NONE || | +---+---+--+--+--+---++---+--+ 4 rows in set (0.00 sec) ``` Related PR：#18358 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](compatible) display dateV2/datetimeV2/decimalv3 as date/datetime/decimal when show column [doris]
doris-robot commented on PR #32233: URL: https://github.com/apache/doris/pull/32233#issuecomment-1996702701 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] SELECT * FROM test1 INTO OUTFILE 任务完成 但是卡住 不退出 [doris]
wyz0130 closed issue #32231: [Bug] SELECT * FROM test1 INTO OUTFILE 任务完成 但是卡住 不退出 URL: https://github.com/apache/doris/issues/32231 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](spill) optimize the spilling logic of hash join operator [doris]
yiguolei commented on code in PR #32202:
URL: https://github.com/apache/doris/pull/32202#discussion_r1524337865
##
be/src/pipeline/pipeline_x/dependency.h:
##
@@ -578,6 +578,7 @@ struct HashJoinSharedState : public JoinSharedState {
struct PartitionedHashJoinSharedState : public HashJoinSharedState {
std::vector>
partitioned_build_blocks;
std::vector spilled_streams;
+bool need_to_spill {false};
Review Comment:
= false we should unify the init operator
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[I] [Bug] Bug summary for doris 2.0.4 [doris]
yongjinhou opened a new issue, #32234: URL: https://github.com/apache/doris/issues/32234 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version 目前2.0.4版本全量失败case有258个，目前针对失败case进行分类，先将各个分类的case全部修复完成后，再跑一轮回归测试，查看case的修复情况。 case修复流程： 1. 先查看case在master上是否有问题，若master已修复，则找到master上修复case的pr，cherry-pick到社区2.0分支 2. 若master上也存在问题，则将case修复并合入到master上，再cherry-pick 到2.0分支。 (注：修复的case需要增加regression-test的p0 case中，这样后续这类case可以作为代码合入的准入) 注意去对应的 issue 下认领完 assign 下，防止重复投入人力。 [https://github.com/apache/doris/issues/31524](https://github.com/apache/doris/issues/31524) 已修复 [https://github.com/apache/doris/issues/31516](https://github.com/apache/doris/issues/31516) 未修复 [https://github.com/apache/doris/issues/31552](https://github.com/apache/doris/issues/31552) 未修复 [https://github.com/apache/doris/issues/31517](https://github.com/apache/doris/issues/31517) 未修复 [https://github.com/apache/doris/issues/31514](https://github.com/apache/doris/issues/31514) 已修复 [https://github.com/apache/doris/issues/31518](https://github.com/apache/doris/issues/31518) 已修复 [https://github.com/apache/doris/issues/31938](https://github.com/apache/doris/issues/31938) 未修复 [https://github.com/apache/doris/issues/31957](https://github.com/apache/doris/issues/31957) 未修复 ### What's Wrong? 无 ### What You Expected? 无 ### How to Reproduce? _No response_ ### Anything Else? 无 ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](spill) optimize the spilling logic of hash join operator [doris]
yiguolei commented on code in PR #32202:
URL: https://github.com/apache/doris/pull/32202#discussion_r1524338964
##
be/src/pipeline/exec/partitioned_hash_join_sink_operator.cpp:
##
@@ -207,7 +229,7 @@ size_t
PartitionedHashJoinSinkOperatorX::revocable_mem_size(RuntimeState* state)
size_t mem_size = 0;
for (uint32_t i = 0; i != _partition_count; ++i) {
auto& block = partitioned_blocks[i];
-if (block) {
+if (block && block->rows() >= state->batch_size()) {
mem_size += block->allocated_bytes();
Review Comment:
This method is not right for bitmap column, we should consider replace this
logic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](merge-cloud) Add multi cloud cluster [doris]
github-actions[bot] commented on PR #31749: URL: https://github.com/apache/doris/pull/31749#issuecomment-1996713300 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Variant) support view for accessing variant subcolumns and temp… [doris]
morrySnow commented on code in PR #32225:
URL: https://github.com/apache/doris/pull/32225#discussion_r1524339901
##
fe/fe-core/src/main/java/org/apache/doris/nereids/CascadesContext.java:
##
@@ -127,6 +128,11 @@ public class CascadesContext implements ScheduleContext {
private boolean isLeadingDisableJoinReorder = false;
+// keep track of which element_at function's level
+// e.g. element_at(element_at(v, 'repo'), 'name') level 1
+// element_at(v, 'repo') level 2
+private int currentElementAtLevel = 0;
Review Comment:
how to support more than one element at in plan?
##
fe/fe-core/src/main/java/org/apache/doris/analysis/BinaryPredicate.java:
##
@@ -492,17 +492,27 @@ private Type getCmpType() throws AnalysisException {
}
}
-if ((t1.isDecimalV3Type() && !t2.isStringType() &&
!t2.isFloatingPointType())
-|| (t2.isDecimalV3Type() && !t1.isStringType() &&
!t1.isFloatingPointType())) {
+if ((t1.isDecimalV3Type() && !t2.isStringType() &&
!t2.isFloatingPointType() && !t2.isVariantType())
+|| (t2.isDecimalV3Type() && !t1.isStringType() &&
!t1.isFloatingPointType() && !t1.isVariantType())) {
return Type.getAssignmentCompatibleType(getChild(0).getType(),
getChild(1).getType(), false,
SessionVariable.getEnableDecimal256());
}
// Variant can be implicit cast to numeric type and string type at
present
if (t1.isVariantType() && (t2.isNumericType() || t2.isStringType())) {
+if (t2.isDecimalV2Type() || t2.isDecimalV3Type()) {
+// TODO support decimal
+throw new
org.apache.doris.nereids.exceptions.AnalysisException("Not support cast to
decimal type"
++ " at present, maybe explicitly cast variant to
double");
+}
return Type.fromPrimitiveType(t2);
}
if (t2.isVariantType() && (t1.isNumericType() || t1.isStringType())) {
+if (t1.isDecimalV2Type() || t1.isDecimalV3Type()) {
+// TODO support decimal
+throw new
org.apache.doris.nereids.exceptions.AnalysisException("Not support cast to
decimal type"
++ " at present, maybe explicitly cast variant to
double");
+}
Review Comment:
implicit cast to double is better?
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/FunctionBinder.java:
##
@@ -188,6 +191,15 @@ public Expression visitUnboundFunction(UnboundFunction
unboundFunction, Expressi
// so wrap COUNT with Nvl to ensure it's result is 0 instead
of null to get the correct result
boundFunction = new Nvl(boundFunction, new BigIntLiteral(0));
}
+if (context.cascadesContext.currentElementAtLevel() == 1
Review Comment:
it is hard to understand, please add enough comment to explain the if
condition
##
fe/fe-core/src/main/java/org/apache/doris/analysis/BinaryPredicate.java:
##
@@ -492,17 +492,27 @@ private Type getCmpType() throws AnalysisException {
}
}
-if ((t1.isDecimalV3Type() && !t2.isStringType() &&
!t2.isFloatingPointType())
-|| (t2.isDecimalV3Type() && !t1.isStringType() &&
!t1.isFloatingPointType())) {
+if ((t1.isDecimalV3Type() && !t2.isStringType() &&
!t2.isFloatingPointType() && !t2.isVariantType())
+|| (t2.isDecimalV3Type() && !t1.isStringType() &&
!t1.isFloatingPointType() && !t1.isVariantType())) {
return Type.getAssignmentCompatibleType(getChild(0).getType(),
getChild(1).getType(), false,
SessionVariable.getEnableDecimal256());
}
// Variant can be implicit cast to numeric type and string type at
present
if (t1.isVariantType() && (t2.isNumericType() || t2.isStringType())) {
+if (t2.isDecimalV2Type() || t2.isDecimalV3Type()) {
+// TODO support decimal
+throw new
org.apache.doris.nereids.exceptions.AnalysisException("Not support cast to
decimal type"
Review Comment:
use `org.apache.doris.common.AnalysisException` here
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/FunctionBinder.java:
##
@@ -141,6 +141,9 @@ private UnboundFunction
bindHighOrderFunction(UnboundFunction unboundFunction, E
@Override
public Expression visitUnboundFunction(UnboundFunction unboundFunction,
ExpressionRewriteContext context) {
+if (unboundFunction.getName().equalsIgnoreCase("element_at")) {
Review Comment:
it could not work correctly when user craate a UDF named element_at under
database.
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/FunctionBinder.java:
##
@@ -188,6 +191,15 @@ public Expression visitUnboundFun
Re: [PR] [feature](merge-cloud) Add multi cloud cluster [doris]
yujun777 commented on PR #31749: URL: https://github.com/apache/doris/pull/31749#issuecomment-1996728508 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](nereids)Fix Bug when grouping has same grouping set [doris]
doris-robot commented on PR #32235: URL: https://github.com/apache/doris/pull/32235#issuecomment-1996731214 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Fix](nereids)Fix Bug when grouping has same grouping set [doris]
feiniaofeiafei opened a new pull request, #32235: URL: https://github.com/apache/doris/pull/32235 fix sql like select a from t1 group by grouping sets((a),(a)); -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](fs) Eliminate the reference of FileReader/FileWriter to FS [doris]
platoneko commented on code in PR #31716:
URL: https://github.com/apache/doris/pull/31716#discussion_r1524357299
##
be/src/olap/rowset/segment_v2/inverted_index_compound_directory.cpp:
##
@@ -512,11 +511,7 @@ void DorisCompoundDirectory::FSIndexOutput::close() {
int64_t DorisCompoundDirectory::FSIndexOutput::length() const {
CND_PRECONDITION(_writer != nullptr, "file is not open");
-int64_t ret;
-if (!_writer->fs()->file_size(_writer->path(), &ret).ok()) {
-return -1;
-}
-return ret;
+return _writer->bytes_appended();
Review Comment:
Seems that there is no code within clucene that checks if the return value
of `length` is -1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](merge-cloud) Add multi cloud cluster [doris]
github-actions[bot] commented on PR #31749: URL: https://github.com/apache/doris/pull/31749#issuecomment-1996734881 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](routine-load) self-adaption adjustment of timeout time [doris]
HHoflittlefish777 commented on PR #32227: URL: https://github.com/apache/doris/pull/32227#issuecomment-1996735714 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](nereids)Fix Bug when grouping has same grouping set [doris]
feiniaofeiafei commented on PR #32235: URL: https://github.com/apache/doris/pull/32235#issuecomment-1996742998 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](neredis) Add tpch test for query rewrite by materialized view [doris]
seawinde commented on PR #30870: URL: https://github.com/apache/doris/pull/30870#issuecomment-1996748400 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](spill) optimize the spilling logic of hash join operator [doris]
yiguolei commented on code in PR #32202:
URL: https://github.com/apache/doris/pull/32202#discussion_r1524367616
##
be/src/pipeline/exec/partitioned_hash_join_probe_operator.cpp:
##
@@ -657,6 +703,14 @@ Status
PartitionedHashJoinProbeOperatorX::_revoke_memory(RuntimeState* state, bo
}
bool PartitionedHashJoinProbeOperatorX::_should_revoke_memory(RuntimeState*
state) const {
+auto& local_state = get_local_state(state);
+
+if (local_state._shared_state->need_to_spill) {
Review Comment:
build: 1 2 3 4
partition 1 --> memory
probe --> block ---> partition 4 ---> probe partition 1, left probe
partition 2,3,4
memtable spill stream 的buffer
build --> sort
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](group commit) Fix group commit connect to observer fe [doris]
mymeiyi commented on PR #3: URL: https://github.com/apache/doris/pull/3#issuecomment-1996749087 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Lambda [doris]
doris-robot commented on PR #32210: URL: https://github.com/apache/doris/pull/32210#issuecomment-1996753470 TPC-H: Total hot run time: 38398 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit e1997e2f1819a052ff6f1389b81215baa23f, data reload: false -- Round 1 -- q1 17618 475241214121 q2 2021150 145 145 q3 10600 1102898 898 q4 7783743 745 743 q5 7494275327382738 q6 184 125 125 125 q7 1183842 806 806 q8 9344202619931993 q9 7187645364456445 q10 8489350936013509 q11 421 220 215 215 q12 722 308 289 289 q13 17998 285928332833 q14 273 248 251 248 q15 501 448 447 447 q16 498 389 392 389 q17 974 533 579 533 q18 7150654564336433 q19 3709142414761424 q20 557 292 278 278 q21 6103350834943494 q22 375 292 326 292 Total cold run time: 84 ms Total hot run time: 38398 ms - Round 2, with runtime_filter_mode=off - q1 4133408640644064 q2 322 226 225 225 q3 2939285028522850 q4 1866156015361536 q5 5208521852055205 q6 209 115 116 115 q7 2245187218621862 q8 3132328732713271 q9 8539850885988508 q10 3721366936613661 q11 545 442 435 435 q12 725 556 571 556 q13 16939 285528982855 q14 275 247 253 247 q15 473 439 451 439 q16 445 420 383 383 q17 1748148314541454 q18 7437716370867086 q19 1601150315501503 q20 1899169917001699 q21 4915483547714771 q22 534 466 438 438 Total cold run time: 69850 ms Total hot run time: 53163 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [opt](Nereids) opt distinct agg without group by key plan [doris]
morrySnow opened a new pull request, #32236: URL: https://github.com/apache/doris/pull/32236 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](Nereids) opt distinct agg without group by key plan [doris]
morrySnow commented on PR #32236: URL: https://github.com/apache/doris/pull/32236#issuecomment-1996758049 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](Nereids) opt distinct agg without group by key plan [doris]
doris-robot commented on PR #32236: URL: https://github.com/apache/doris/pull/32236#issuecomment-1996757979 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](regression) prepare_insert failed when connect to observer fe [doris]
github-actions[bot] commented on PR #32223: URL: https://github.com/apache/doris/pull/32223#issuecomment-1996765535 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](regression) prepare_insert failed when connect to observer fe [doris]
github-actions[bot] commented on PR #32223: URL: https://github.com/apache/doris/pull/32223#issuecomment-1996765571 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](ci) fix performance [doris]
hello-stephen opened a new pull request, #32237: URL: https://github.com/apache/doris/pull/32237 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](ci) fix performance [doris]
doris-robot commented on PR #32237: URL: https://github.com/apache/doris/pull/32237#issuecomment-1996766316 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](ci) fix tpcds [doris]
hello-stephen commented on PR #32237: URL: https://github.com/apache/doris/pull/32237#issuecomment-1996766551 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Lambda [doris]
doris-robot commented on PR #32210: URL: https://github.com/apache/doris/pull/32210#issuecomment-1996769221 TeamCity be ut coverage result: Function Coverage: 34.99% (8572/24500) Line Coverage: 26.74% (69457/259758) Region Coverage: 26.01% (36071/138689) Branch Coverage: 22.96% (18421/80214) Coverage Report: http://coverage.selectdb-in.cc/coverage/e1997e2f1819a052ff6f1389b81215baa23f_e1997e2f1819a052ff6f1389b81215baa23f/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](Nereids) opt distinct agg without group by key plan [doris]
github-actions[bot] commented on PR #32236: URL: https://github.com/apache/doris/pull/32236#issuecomment-1996773501 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](Nereids) opt distinct agg without group by key plan [doris]
github-actions[bot] commented on PR #32236: URL: https://github.com/apache/doris/pull/32236#issuecomment-1996773573 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](nereids)unify outputTupleDesc and projection [doris]
doris-robot commented on PR #32128: URL: https://github.com/apache/doris/pull/32128#issuecomment-1996775446 TPC-H: Total hot run time: 38777 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 8cfac990622988546333b77bffcaf379a16a50cb, data reload: false -- Round 1 -- q1 17675 420941374137 q2 2020148 140 140 q3 10616 1066913 913 q4 7430737 743 737 q5 7469277527702770 q6 197 119 124 119 q7 1162840 816 816 q8 9415203519791979 q9 7426660965766576 q10 8558356936873569 q11 434 225 221 221 q12 620 293 293 293 q13 17796 288728882887 q14 273 242 241 241 q15 500 452 461 452 q16 502 394 393 393 q17 957 547 562 547 q18 7185655764126412 q19 3409146013971397 q20 543 286 289 286 q21 6217358736353587 q22 373 309 305 305 Total cold run time: 110777 ms Total hot run time: 38777 ms - Round 2, with runtime_filter_mode=off - q1 4086407340424042 q2 319 234 218 218 q3 2970288728042804 q4 1906156415271527 q5 5297532953285328 q6 193 115 116 115 q7 2232184618521846 q8 3128329232683268 q9 8711868686538653 q10 3766373338293733 q11 529 451 434 434 q12 718 544 531 531 q13 17135 287528692869 q14 278 238 252 238 q15 476 432 445 432 q16 439 398 392 392 q17 1722150814671467 q18 7383721870777077 q19 1618156315441544 q20 1898169116881688 q21 4771470547624705 q22 525 435 463 435 Total cold run time: 70100 ms Total hot run time: 53346 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](nereids)unify outputTupleDesc and projection [doris]
doris-robot commented on PR #32128: URL: https://github.com/apache/doris/pull/32128#issuecomment-1996776058 TeamCity be ut coverage result: Function Coverage: 34.99% (8572/24500) Line Coverage: 26.73% (69448/259857) Region Coverage: 26.00% (36065/138719) Branch Coverage: 22.95% (18414/80232) Coverage Report: http://coverage.selectdb-in.cc/coverage/8cfac990622988546333b77bffcaf379a16a50cb_8cfac990622988546333b77bffcaf379a16a50cb/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](merge-cloud) enable merge on write by default [doris]
doris-robot commented on PR #32175: URL: https://github.com/apache/doris/pull/32175#issuecomment-1996776507 TPC-H: Total hot run time: 38320 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 5b798b4e176740c9efac33a27dece22c7a0a7886, data reload: false -- Round 1 -- q1 17653 431741174117 q2 2016158 146 146 q3 10598 1094905 905 q4 7420737 754 737 q5 7530274127052705 q6 184 123 121 121 q7 1190844 814 814 q8 9324207120112011 q9 7211648464296429 q10 8522350136003501 q11 437 227 213 213 q12 877 300 293 293 q13 18168 287228592859 q14 280 253 248 248 q15 506 445 454 445 q16 498 389 385 385 q17 962 534 631 534 q18 7160652662856285 q19 1614146114121412 q20 544 280 280 280 q21 6065364935833583 q22 347 297 298 297 Total cold run time: 109106 ms Total hot run time: 38320 ms - Round 2, with runtime_filter_mode=off - q1 4103406840644064 q2 320 224 230 224 q3 3000288528832883 q4 1850159215511551 q5 5246525652245224 q6 197 114 119 114 q7 181618491816 q8 3155327832933278 q9 8525858685168516 q10 3714375436613661 q11 556 435 435 435 q12 734 538 557 538 q13 16950 286128542854 q14 274 252 269 252 q15 487 436 441 436 q16 448 416 418 416 q17 1739147614761476 q18 7562730571307130 q19 3376156915291529 q20 1892170616881688 q21 4900471146114611 q22 535 436 461 436 Total cold run time: 71785 ms Total hot run time: 53132 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](merge-cloud) Support specify meta service endpoints separated by comma [doris]
doris-robot commented on PR #32230: URL: https://github.com/apache/doris/pull/32230#issuecomment-1996776108 TPC-H: Total hot run time: 38381 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 8a5e55e1fec166a860ab9dd9b3c70428ae3d37fb, data reload: false -- Round 1 -- q1 17632 440741784178 q2 2023149 145 145 q3 10598 1147903 903 q4 6728738 730 730 q5 7471264226732642 q6 182 122 120 120 q7 1208817 804 804 q8 9329206820142014 q9 7143645864696458 q10 8534351336193513 q11 433 236 232 232 q12 672 305 294 294 q13 17801 286528332833 q14 279 243 252 243 q15 519 481 450 450 q16 506 393 396 393 q17 971 510 539 510 q18 7377656963786378 q19 4082145714061406 q20 548 284 282 282 q21 6487355435903554 q22 365 299 328 299 Total cold run time: 110888 ms Total hot run time: 38381 ms - Round 2, with runtime_filter_mode=off - q1 4150408240774077 q2 329 225 228 225 q3 2999287228552855 q4 1878151916301519 q5 5235524152915241 q6 192 122 115 115 q7 2223185918771859 q8 3167326832623262 q9 8606859685838583 q10 3782372336403640 q11 543 446 447 446 q12 699 553 545 545 q13 16915 286528762865 q14 281 237 254 237 q15 493 448 448 448 q16 451 425 411 411 q17 1802149814641464 q18 7650728971157115 q19 1622150515011501 q20 1925173017161716 q21 5065460548024605 q22 532 443 433 433 Total cold run time: 70539 ms Total hot run time: 53162 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](jdbc catalog) fix jdbc-connector coredump as get env return nullptr [doris]
yiguolei merged PR #32217: URL: https://github.com/apache/doris/pull/32217 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [fix](jdbc catalog) fix jdbc-connector coredump as get env return nullptr (#32217)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 60f97abb1f9 [fix](jdbc catalog) fix jdbc-connector coredump as get env
return nullptr (#32217)
60f97abb1f9 is described below
commit 60f97abb1f954c36bd8431c701e86a38b1422260
Author: zy-kkk
AuthorDate: Thu Mar 14 16:04:06 2024 +0800
[fix](jdbc catalog) fix jdbc-connector coredump as get env return nullptr
(#32217)
---
be/src/util/jni-util.cpp | 4 ++--
be/src/util/jni-util.h| 11 +--
be/src/vec/aggregate_functions/aggregate_function_java_udaf.h | 2 +-
be/src/vec/exec/vjdbc_connector.cpp | 2 +-
4 files changed, 13 insertions(+), 6 deletions(-)
diff --git a/be/src/util/jni-util.cpp b/be/src/util/jni-util.cpp
index 764fc812cba..6e66f97ddae 100644
--- a/be/src/util/jni-util.cpp
+++ b/be/src/util/jni-util.cpp
@@ -141,7 +141,7 @@ const std::string GetDorisJNIClasspathOption() {
jvm_options[i] = {const_cast(options[i].c_str()), nullptr};
}
-JNIEnv* env;
+JNIEnv* env = nullptr;
JavaVMInitArgs vm_args;
vm_args.version = JNI_VERSION_1_8;
vm_args.options = jvm_options.get();
@@ -407,7 +407,7 @@ Status JniUtil::Init() {
RETURN_IF_ERROR(LibJVMLoader::instance().load());
// Get the JNIEnv* corresponding to current thread.
-JNIEnv* env;
+JNIEnv* env = nullptr;
RETURN_IF_ERROR(JniUtil::GetJNIEnv(&env));
if (env == NULL) return Status::InternalError("Failed to get/create JVM");
diff --git a/be/src/util/jni-util.h b/be/src/util/jni-util.h
index 317ba48d3d1..ca305c32bf1 100644
--- a/be/src/util/jni-util.h
+++ b/be/src/util/jni-util.h
@@ -52,9 +52,16 @@ public:
static Status GetJNIEnv(JNIEnv** env) {
if (tls_env_) {
*env = tls_env_;
-return Status::OK();
+} else {
+Status status = GetJNIEnvSlowPath(env);
+if (!status.ok()) {
+return status;
+}
}
-return GetJNIEnvSlowPath(env);
+if (*env == nullptr) {
+return Status::RuntimeError("Failed to get JNIEnv: it is
nullptr.");
+}
+return Status::OK();
}
static Status GetGlobalClassRef(JNIEnv* env, const char* class_str,
diff --git a/be/src/vec/aggregate_functions/aggregate_function_java_udaf.h
b/be/src/vec/aggregate_functions/aggregate_function_java_udaf.h
index 4ef64aae558..7a367623eaf 100644
--- a/be/src/vec/aggregate_functions/aggregate_function_java_udaf.h
+++ b/be/src/vec/aggregate_functions/aggregate_function_java_udaf.h
@@ -60,7 +60,7 @@ public:
AggregateJavaUdafData(int64_t num_args) { argument_size = num_args; }
~AggregateJavaUdafData() {
-JNIEnv* env;
+JNIEnv* env = nullptr;
if (!JniUtil::GetJNIEnv(&env).ok()) {
LOG(WARNING) << "Failed to get JNIEnv";
}
diff --git a/be/src/vec/exec/vjdbc_connector.cpp
b/be/src/vec/exec/vjdbc_connector.cpp
index 41acaf00fb8..6a5aa0fdc38 100644
--- a/be/src/vec/exec/vjdbc_connector.cpp
+++ b/be/src/vec/exec/vjdbc_connector.cpp
@@ -83,7 +83,7 @@ Status JdbcConnector::close(Status /*unused*/) {
if (_is_in_transaction) {
RETURN_IF_ERROR(abort_trans());
}
-JNIEnv* env;
+JNIEnv* env = nullptr;
RETURN_IF_ERROR(JniUtil::GetJNIEnv(&env));
env->DeleteGlobalRef(_executor_factory_clazz);
env->DeleteGlobalRef(_executor_clazz);
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [fix](jdbc catalog) fix jdbc-connector coredump as get env return nullptr (#32217)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new bbdce3eb5ee [fix](jdbc catalog) fix jdbc-connector coredump as get env
return nullptr (#32217)
bbdce3eb5ee is described below
commit bbdce3eb5ee50c4d97b6a476d85459c88ba3c76b
Author: zy-kkk
AuthorDate: Thu Mar 14 16:04:06 2024 +0800
[fix](jdbc catalog) fix jdbc-connector coredump as get env return nullptr
(#32217)
---
be/src/util/jni-util.cpp | 4 ++--
be/src/util/jni-util.h| 11 +--
be/src/vec/aggregate_functions/aggregate_function_java_udaf.h | 2 +-
be/src/vec/exec/vjdbc_connector.cpp | 2 +-
4 files changed, 13 insertions(+), 6 deletions(-)
diff --git a/be/src/util/jni-util.cpp b/be/src/util/jni-util.cpp
index 15344a22a8d..c626ee021ed 100644
--- a/be/src/util/jni-util.cpp
+++ b/be/src/util/jni-util.cpp
@@ -140,7 +140,7 @@ const std::string GetDorisJNIClasspathOption() {
jvm_options[i] = {const_cast(options[i].c_str()), nullptr};
}
-JNIEnv* env;
+JNIEnv* env = nullptr;
JavaVMInitArgs vm_args;
vm_args.version = JNI_VERSION_1_8;
vm_args.options = jvm_options.get();
@@ -406,7 +406,7 @@ Status JniUtil::Init() {
RETURN_IF_ERROR(LibJVMLoader::instance().load());
// Get the JNIEnv* corresponding to current thread.
-JNIEnv* env;
+JNIEnv* env = nullptr;
RETURN_IF_ERROR(JniUtil::GetJNIEnv(&env));
if (env == NULL) return Status::InternalError("Failed to get/create JVM");
diff --git a/be/src/util/jni-util.h b/be/src/util/jni-util.h
index 317ba48d3d1..ca305c32bf1 100644
--- a/be/src/util/jni-util.h
+++ b/be/src/util/jni-util.h
@@ -52,9 +52,16 @@ public:
static Status GetJNIEnv(JNIEnv** env) {
if (tls_env_) {
*env = tls_env_;
-return Status::OK();
+} else {
+Status status = GetJNIEnvSlowPath(env);
+if (!status.ok()) {
+return status;
+}
}
-return GetJNIEnvSlowPath(env);
+if (*env == nullptr) {
+return Status::RuntimeError("Failed to get JNIEnv: it is
nullptr.");
+}
+return Status::OK();
}
static Status GetGlobalClassRef(JNIEnv* env, const char* class_str,
diff --git a/be/src/vec/aggregate_functions/aggregate_function_java_udaf.h
b/be/src/vec/aggregate_functions/aggregate_function_java_udaf.h
index 4ef64aae558..7a367623eaf 100644
--- a/be/src/vec/aggregate_functions/aggregate_function_java_udaf.h
+++ b/be/src/vec/aggregate_functions/aggregate_function_java_udaf.h
@@ -60,7 +60,7 @@ public:
AggregateJavaUdafData(int64_t num_args) { argument_size = num_args; }
~AggregateJavaUdafData() {
-JNIEnv* env;
+JNIEnv* env = nullptr;
if (!JniUtil::GetJNIEnv(&env).ok()) {
LOG(WARNING) << "Failed to get JNIEnv";
}
diff --git a/be/src/vec/exec/vjdbc_connector.cpp
b/be/src/vec/exec/vjdbc_connector.cpp
index 4020278e84d..318651ea885 100644
--- a/be/src/vec/exec/vjdbc_connector.cpp
+++ b/be/src/vec/exec/vjdbc_connector.cpp
@@ -83,7 +83,7 @@ Status JdbcConnector::close(Status /*unused*/) {
if (_is_in_transaction) {
RETURN_IF_ERROR(abort_trans());
}
-JNIEnv* env;
+JNIEnv* env = nullptr;
RETURN_IF_ERROR(JniUtil::GetJNIEnv(&env));
env->DeleteGlobalRef(_executor_factory_clazz);
env->DeleteGlobalRef(_executor_clazz);
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids): don't pushdown project when project contains both side of join [doris]
XieJiann commented on PR #32214: URL: https://github.com/apache/doris/pull/32214#issuecomment-1996782519 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Variant) support view for accessing variant subcolumns and temp… [doris]
924060929 commented on PR #32225: URL: https://github.com/apache/doris/pull/32225#issuecomment-1996785108 This fix is too hard to understand. Why `element_to` is unbound function when bind sink? In the short term, I think this PR can be merge because of the introduction of new tests, it can help we test when we refactor the analyze of variant -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [nereids] enable gby key eliminate by default [doris]
xzj7019 commented on PR #32238: URL: https://github.com/apache/doris/pull/32238#issuecomment-1996789494 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [nereids] enable gby key eliminate by default [doris]
xzj7019 opened a new pull request, #32238: URL: https://github.com/apache/doris/pull/32238 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [nereids] enable gby key eliminate by default [doris]
doris-robot commented on PR #32238: URL: https://github.com/apache/doris/pull/32238#issuecomment-1996788842 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](nereids)Fix Bug when grouping has same grouping set [doris]
doris-robot commented on PR #32235: URL: https://github.com/apache/doris/pull/32235#issuecomment-1996791293 TPC-H: Total hot run time: 38078 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 526dc184a1dd5c5329053142ed578f8951a9814d, data reload: false -- Round 1 -- q1 17637 420440944094 q2 2017140 141 140 q3 10614 1063881 881 q4 7083706 676 676 q5 7459266225852585 q6 185 124 123 123 q7 1195800 791 791 q8 9387202120032003 q9 7271642764256425 q10 8548350935933509 q11 433 227 214 214 q12 770 303 289 289 q13 18154 283528272827 q14 278 242 243 242 q15 507 451 450 450 q16 499 389 389 389 q17 940 528 519 519 q18 7089645063776377 q19 1770139413691369 q20 555 285 282 282 q21 6266357835763576 q22 348 330 317 317 Total cold run time: 109005 ms Total hot run time: 38078 ms - Round 2, with runtime_filter_mode=off - q1 4054406440334033 q2 320 227 218 218 q3 2912285927952795 q4 1859153815661538 q5 5147522751695169 q6 196 122 116 116 q7 2240186018331833 q8 3122326532513251 q9 8473848084768476 q10 3663367236643664 q11 529 451 432 432 q12 745 579 536 536 q13 16923 285228612852 q14 275 247 254 247 q15 486 453 450 450 q16 453 413 404 404 q17 1704148214691469 q18 7413702170647021 q19 1610153815161516 q20 1895169516901690 q21 4765471247604712 q22 495 450 468 450 Total cold run time: 69279 ms Total hot run time: 52872 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](routine-load) support auto resume when get partition failed [doris]
doris-robot commented on PR #32239: URL: https://github.com/apache/doris/pull/32239#issuecomment-1996793219 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](spark load) update spark version for spark load to resolve cve problem [doris]
CalvinKirs commented on PR #30368: URL: https://github.com/apache/doris/pull/30368#issuecomment-1996795170 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [opt](routine-load) support auto resume when get partition failed [doris]
HHoflittlefish777 opened a new pull request, #32239: URL: https://github.com/apache/doris/pull/32239 ## Proposed changes Auto resume when get partition failed, can be It can solve some unstable factors and keep the routing load stable. some unstable factors: 1. be restart or shutdown upgrade 2. RPC timeout 3. The topic in Kafka has not been established yet. Automatically pulling up in these situations can avoid users manually pulling up, improving the user experience. ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](spark load) update spark version for spark load to resolve cve problem [doris]
gnehil commented on PR #30368: URL: https://github.com/apache/doris/pull/30368#issuecomment-1996795942 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](Nereids) opt distinct agg without group by key plan [doris]
doris-robot commented on PR #32236: URL: https://github.com/apache/doris/pull/32236#issuecomment-1996798316 TPC-H: Total hot run time: 38338 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 03452fbc50b18ff9b78dc324f6c9881e8681ce75, data reload: false -- Round 1 -- q1 17648 432940914091 q2 2036150 143 143 q3 10718 1083905 905 q4 7136716 748 716 q5 7493261526232615 q6 186 121 120 120 q7 1212825 812 812 q8 9404199920291999 q9 7019643064216421 q10 8479354336023543 q11 419 231 220 220 q12 670 309 313 309 q13 17806 288428552855 q14 272 259 255 255 q15 498 452 454 452 q16 480 399 390 390 q17 946 528 541 528 q18 7382648164516451 q19 1848146513941394 q20 552 293 292 292 q21 6310358235273527 q22 372 300 304 300 Total cold run time: 108886 ms Total hot run time: 38338 ms - Round 2, with runtime_filter_mode=off - q1 4096406340194019 q2 327 227 215 215 q3 2976285828722858 q4 1850154315991543 q5 5239525352385238 q6 204 116 123 116 q7 2187181618471816 q8 3131326532953265 q9 8567854185918541 q10 3730372036903690 q11 547 444 440 440 q12 732 556 542 542 q13 16917 283728692837 q14 286 250 245 245 q15 487 436 452 436 q16 452 419 420 419 q17 1740147614731473 q18 7597731870707070 q19 1599157014921492 q20 1910173417361734 q21 4981472445274527 q22 515 458 450 450 Total cold run time: 70070 ms Total hot run time: 52966 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](ip) conversion of ipv4 or ipv6 to arrow type [doris]
vinlee19 opened a new pull request, #32240: URL: https://github.com/apache/doris/pull/32240 ## Proposed changes When utilizing the flink-connector-doris to read IPv6 addresses, we encounter the following errors: internal failure, with a status code of [INVALID_ARGUMENT] and an error message stating [([xxx.xxx.xxx.xxx](http://xxx.xxx.xxx.xxx/))[INVALID_ARGUMENT]Unknown primitive type(0) converted to Arrow type]. This issue arises due to the incompatible conversion between IPv6 and arrow strings. Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](ip) conversion of ipv4 or ipv6 to arrow type [doris]
doris-robot commented on PR #32240: URL: https://github.com/apache/doris/pull/32240#issuecomment-1996806822 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [docs]Modify the string range, the sample range is wrong [doris]
xyfsjq opened a new pull request, #32241: URL: https://github.com/apache/doris/pull/32241 …ta insertion fails ## Proposed changes Issue Number: close #xxx ## Further comments Modify the string range, the sample range is wrong, and the sample data insertion fails If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [docs]Modify the string range, the sample range is wrong [doris]
doris-robot commented on PR #32241: URL: https://github.com/apache/doris/pull/32241#issuecomment-1996821286 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](merge-cloud) Add multi cloud cluster [doris]
doris-robot commented on PR #31749: URL: https://github.com/apache/doris/pull/31749#issuecomment-1996821612 TPC-H: Total hot run time: 38415 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 1d5e92ef42044125ea2da90c3db019f97ef9cba2, data reload: false -- Round 1 -- q1 17641 421141224122 q2 2037150 150 150 q3 10588 1110898 898 q4 8126815 742 742 q5 7490261826662618 q6 184 124 122 122 q7 1192830 798 798 q8 9343201220172012 q9 7226645064256425 q10 8535351936493519 q11 435 234 227 227 q12 711 307 310 307 q13 17796 286028422842 q14 286 259 246 246 q15 505 459 454 454 q16 501 398 389 389 q17 957 563 569 563 q18 7235654664636463 q19 4183137914461379 q20 544 284 284 284 q21 6335359635653565 q22 363 293 290 290 Total cold run time: 112213 ms Total hot run time: 38415 ms - Round 2, with runtime_filter_mode=off - q1 4159407140804071 q2 321 228 233 228 q3 2979283927912791 q4 1857153715781537 q5 5270525952605259 q6 208 114 116 114 q7 2212188218211821 q8 3162328432773277 q9 8592857286048572 q10 3705371236523652 q11 558 441 437 437 q12 719 557 554 554 q13 16905 284928682849 q14 272 244 245 244 q15 480 461 446 446 q16 463 419 404 404 q17 1743145814671458 q18 7414718672717186 q19 1625145515251455 q20 1913169517081695 q21 4830469147174691 q22 535 425 439 425 Total cold run time: 69922 ms Total hot run time: 53166 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](neredis) Add tpch test for query rewrite by materialized view [doris]
doris-robot commented on PR #30870: URL: https://github.com/apache/doris/pull/30870#issuecomment-1996830794 TPC-H: Total hot run time: 37897 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit fa45998b1429a78180ce8d36886b4747f97d11a3, data reload: false -- Round 1 -- q1 17622 419540874087 q2 2015148 140 140 q3 10615 1062872 872 q4 6380714 718 714 q5 7484255526552555 q6 181 123 124 123 q7 1229808 792 792 q8 9361195020361950 q9 7087639464336394 q10 8459349435983494 q11 429 227 219 219 q12 703 301 311 301 q13 17784 283528542835 q14 267 250 245 245 q15 506 452 444 444 q16 512 392 385 385 q17 943 563 561 561 q18 7121638263386338 q19 1694148213781378 q20 538 294 267 267 q21 6599354935083508 q22 350 295 309 295 Total cold run time: 107879 ms Total hot run time: 37897 ms - Round 2, with runtime_filter_mode=off - q1 4064403840984038 q2 322 219 220 219 q3 2907283827392739 q4 1827152015321520 q5 5193518952185189 q6 196 114 118 114 q7 2224181018711810 q8 3137325732853257 q9 8505851984908490 q10 3671365236423642 q11 541 443 446 443 q12 722 548 535 535 q13 16926 284428282828 q14 272 241 256 241 q15 483 446 441 441 q16 456 429 416 416 q17 1720148614711471 q18 7493721869976997 q19 1620155415191519 q20 1887169116961691 q21 4848480146474647 q22 531 447 479 447 Total cold run time: 69545 ms Total hot run time: 52694 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](routine-load) support auto resume when get partition failed [doris]
HHoflittlefish777 commented on PR #32239: URL: https://github.com/apache/doris/pull/32239#issuecomment-1996832559 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](ip) conversion of ipv4 or ipv6 to arrow type [doris]
vinlee19 commented on PR #32240: URL: https://github.com/apache/doris/pull/32240#issuecomment-1996837180 @sjyango -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[I] Doris support OpenTelemetry data store [Feature] [doris]
young138120 opened a new issue, #32242: URL: https://github.com/apache/doris/issues/32242 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description _No response_ ### Use case _No response_ ### Related issues _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [improve](load) retry 3 times if failed to get base_migration_lock in rowset builder [doris]
kaijchen opened a new pull request, #32243: URL: https://github.com/apache/doris/pull/32243 ## Proposed changes Retry 3 times if failed to get base_migration_lock in rowset builder, with 10ms interval. ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](load) retry 3 times if failed to get base_migration_lock in rowset builder [doris]
doris-robot commented on PR #32243: URL: https://github.com/apache/doris/pull/32243#issuecomment-1996843375 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](load) retry 3 times if failed to get base_migration_lock in rowset builder [doris]
kaijchen commented on PR #32243: URL: https://github.com/apache/doris/pull/32243#issuecomment-1996843458 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids)AssertNumRow node's output should be nullable [doris]
starocean999 commented on PR #32136: URL: https://github.com/apache/doris/pull/32136#issuecomment-1996845792 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](ci) fix tpcds [doris]
doris-robot commented on PR #32237: URL: https://github.com/apache/doris/pull/32237#issuecomment-1996848125 TPC-H: Total hot run time: 38134 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 04d59ff1fe9a5d2f4b7f4fe084d88cf41d34df43, data reload: false -- Round 1 -- q1 17654 416840974097 q2 2014148 142 142 q3 10602 1061901 901 q4 8115794 706 706 q5 7468264825962596 q6 186 121 123 121 q7 1188825 807 807 q8 9342200920062006 q9 7161642063836383 q10 8512350935943509 q11 429 228 215 215 q12 612 303 312 303 q13 17798 283628452836 q14 273 253 248 248 q15 497 446 452 446 q16 497 392 389 389 q17 938 559 532 532 q18 7176650364166416 q19 1596146413811381 q20 539 283 274 274 q21 6432353235743532 q22 358 335 294 294 Total cold run time: 109387 ms Total hot run time: 38134 ms - Round 2, with runtime_filter_mode=off - q1 4113406041554060 q2 319 226 220 220 q3 2967283328262826 q4 1833157015801570 q5 5200522452285224 q6 191 117 118 117 q7 2227182418531824 q8 3128325432663254 q9 8499853385538533 q10 3654368336903683 q11 534 435 438 435 q12 731 549 538 538 q13 16919 285728412841 q14 280 244 254 244 q15 483 454 446 446 q16 457 412 405 405 q17 1708148614451445 q18 7575726370587058 q19 1625149515571495 q20 1937171517311715 q21 4878473647984736 q22 537 449 441 441 Total cold run time: 69795 ms Total hot run time: 53110 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](nereids)Fix Bug when grouping has same grouping set [doris]
morrySnow commented on code in PR #32235: URL: https://github.com/apache/doris/pull/32235#discussion_r1524445530 ## docs/zh-CN/community/design/grouping_sets_design.md: ## @@ -458,7 +458,7 @@ SELECT k1, k2, GROUPING_ID(k1,k2), SUM(k3) FROM t GROUP BY GROUPING SETS ((k1, k ``` -可以看到，其结果与对 GROUPING SETS 子句后每个子集进行 GROUP BY 后再进行 UNION 的结果一致。 +可以看到，其结果与对 GROUPING SETS 子句后每个子集进行 GROUP BY 后再进行 UNION ALL的结果一致。 Review Comment: ```suggestion 可以看到，其结果与对 GROUPING SETS 子句后每个子集进行 GROUP BY 后再进行 UNION ALL 的结果一致。 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](nereids)Fix Bug when grouping has same grouping set [doris]
github-actions[bot] commented on PR #32235: URL: https://github.com/apache/doris/pull/32235#issuecomment-1996851041 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](nereids)Fix Bug when grouping has same grouping set [doris]
github-actions[bot] commented on PR #32235: URL: https://github.com/apache/doris/pull/32235#issuecomment-1996850915 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] feature](insert)implement hive table sink plan [doris]
github-actions[bot] commented on PR #31765: URL: https://github.com/apache/doris/pull/31765#issuecomment-1996857098 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](load) retry 3 times if failed to get base_migration_lock in rowset builder [doris]
github-actions[bot] commented on PR #32243: URL: https://github.com/apache/doris/pull/32243#issuecomment-1996858112 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](load) retry 3 times if failed to get base_migration_lock in rowset builder [doris]
kaijchen commented on PR #32243: URL: https://github.com/apache/doris/pull/32243#issuecomment-1996860901 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids)AssertNumRow node's output should be nullable [doris]
github-actions[bot] commented on PR #32136: URL: https://github.com/apache/doris/pull/32136#issuecomment-1996862984 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](load) retry 3 times if failed to get base_migration_lock in rowset builder [doris]
github-actions[bot] commented on PR #32243: URL: https://github.com/apache/doris/pull/32243#issuecomment-1996871247 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](load) retry 3 times if failed to get base_migration_lock in rowset builder [doris]
github-actions[bot] commented on PR #32243: URL: https://github.com/apache/doris/pull/32243#issuecomment-1996871364 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](routine-load) support auto resume when get partition failed [doris]
doris-robot commented on PR #32239: URL: https://github.com/apache/doris/pull/32239#issuecomment-1996873894 TPC-H: Total hot run time: 38480 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 5b985eb4d2d409f25c6ee9136dc84dbcf8a65cbd, data reload: false -- Round 1 -- q1 17737 426241574157 q2 2021157 155 155 q3 10660 1127907 907 q4 744 730 730 q5 7491275527192719 q6 186 126 127 126 q7 1148828 815 815 q8 9414201819901990 q9 7097643763506350 q10 8514352236413522 q11 438 221 217 217 q12 627 313 300 300 q13 17808 285528452845 q14 275 243 256 243 q15 493 458 449 449 q16 508 390 384 384 q17 955 538 604 538 q18 7324655064796479 q19 3828139514311395 q20 544 285 276 276 q21 6240357136073571 q22 346 312 331 312 Total cold run time: 111431 ms Total hot run time: 38480 ms - Round 2, with runtime_filter_mode=off - q1 4166413440894089 q2 325 224 223 223 q3 2972279628122796 q4 1849153415831534 q5 5219521752475217 q6 192 123 116 116 q7 2190186518651865 q8 3183329832583258 q9 8620859485648564 q10 3791377837323732 q11 545 465 464 464 q12 749 562 552 552 q13 16921 283728632837 q14 269 245 255 245 q15 486 446 449 446 q16 448 418 406 406 q17 1747148914451445 q18 7600714671017101 q19 1591154315281528 q20 1897168417151684 q21 4940471247904712 q22 530 468 456 456 Total cold run time: 70230 ms Total hot run time: 53270 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](load) retry 3 times if failed to get base_migration_lock in rowset builder [doris]
github-actions[bot] commented on PR #32243: URL: https://github.com/apache/doris/pull/32243#issuecomment-1996875325 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](spark load) update spark version for spark load to resolve cve problem [doris]
gnehil commented on PR #30368: URL: https://github.com/apache/doris/pull/30368#issuecomment-1996878580 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Variant) support view for accessing variant subcolumns and temp… [doris]
eldenmoon commented on PR #32225: URL: https://github.com/apache/doris/pull/32225#issuecomment-1996882310 @morrySnow @924060929 i will modify some of the code and add more comments to explain -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](Nereids): rewrite sum literal to sum and count [doris]
doris-robot commented on PR #32244: URL: https://github.com/apache/doris/pull/32244#issuecomment-1996884352 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [feat](Nereids): rewrite sum literal to sum and count [doris]
XieJiann opened a new pull request, #32244: URL: https://github.com/apache/doris/pull/32244 ## Proposed changes 1. sum(v + 2) => sum(v) + 2*count(v) 2. sum(v - 2) => sum(v) - 2*count(v) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Variant) support view for accessing variant subcolumns and temp… [doris]
eldenmoon commented on code in PR #32225:
URL: https://github.com/apache/doris/pull/32225#discussion_r1524465387
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/FunctionBinder.java:
##
@@ -188,6 +191,15 @@ public Expression visitUnboundFunction(UnboundFunction
unboundFunction, Expressi
// so wrap COUNT with Nvl to ensure it's result is 0 instead
of null to get the correct result
boundFunction = new Nvl(boundFunction, new BigIntLiteral(0));
}
+if (context.cascadesContext.currentElementAtLevel() == 1
+&&
PushDownToProjectionFunction.validToPushDown(boundFunction)) {
+// Only rewrite the top level of PushDownToProjectionFunction,
otherwise invalid slot will be generated
+context.cascadesContext.resetCurrentElementAtLevel();
Review Comment:
this function can not be pushed down now， so this pr has no effect for such
expression
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](Nereids): rewrite sum literal to sum and count [doris]
XieJiann commented on PR #32244: URL: https://github.com/apache/doris/pull/32244#issuecomment-1996885980 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Variant) support view for accessing variant subcolumns and temp… [doris]
eldenmoon commented on code in PR #32225:
URL: https://github.com/apache/doris/pull/32225#discussion_r1524465901
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/FunctionBinder.java:
##
@@ -141,6 +141,9 @@ private UnboundFunction
bindHighOrderFunction(UnboundFunction unboundFunction, E
@Override
public Expression visitUnboundFunction(UnboundFunction unboundFunction,
ExpressionRewriteContext context) {
+if (unboundFunction.getName().equalsIgnoreCase("element_at")) {
Review Comment:
`currentElementAtLevel` this variable will not affect udf function
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[I] [Bug] [INTERNAL_ERROR]Unsupported exec type in pipelineX: ODBC_SCAN_NODE [doris]
wanqiang2046 opened a new issue, #32245: URL: https://github.com/apache/doris/issues/32245 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version 2.1.0 ### What's Wrong? 用客户端或者sql查询之前的odbc外链表，全部报错，报错如下： Caused by: java.sql.SQLException: errCode = 2, detailMessage = (172.19.176.193)[INTERNAL_ERROR]Unsupported exec type in pipelineX: ODBC_SCAN_NODE 建表语句例子： CREATE EXTERNAL TABLE `dim_basic_ts_staff` ( `id` BIGINT NOT NULL COMMENT '主键', `code` VARCHAR(30) NOT NULL DEFAULT "" COMMENT '员工编号', `name` VARCHAR(100) NOT NULL DEFAULT "" COMMENT '员工名称', `belong_site` VARCHAR(64) NOT NULL DEFAULT "" COMMENT '网点编号', `belong_department` VARCHAR(64) NOT NULL DEFAULT "1" COMMENT '所属部门(运营部DT01,销售部DT02,IT部DT03,客服部DT04,财务部DT05)', `staff_type` VARCHAR(32) NOT NULL DEFAULT "1" COMMENT '员工类型（骑手ET01，操作员ET02，客服ET03，财务ET04，主管ET05，司机ET06）', `telephone` VARCHAR(32) NOT NULL DEFAULT "" COMMENT '手机号', `email` VARCHAR(50) NOT NULL DEFAULT "" COMMENT '邮箱', `sex` VARCHAR(32) NOT NULL DEFAULT "2" COMMENT '性别（0男,1女,2未知）', `induction_time` DATETIME NOT NULL COMMENT '入职日期', `address` VARCHAR(128) NOT NULL DEFAULT "" COMMENT '地址信息', `bank_account` VARCHAR(64) NOT NULL DEFAULT "" COMMENT '银行账号', `remark` VARCHAR(128) NOT NULL DEFAULT "" COMMENT '备注', `status` VARCHAR(32) NOT NULL DEFAULT "1" COMMENT '有效状态（0禁用，1启用）', `country_code` CHAR(2) NOT NULL DEFAULT "" COMMENT '所属国家二字码', `international_code` VARCHAR(8) NULL DEFAULT "" COMMENT '所属国家国际区号', `batch_operation_status` TINYINT NOT NULL DEFAULT "0" COMMENT '批量操作零担状态 0:否 1:是', `create_by` VARCHAR(64) NOT NULL DEFAULT "" COMMENT '创建者id', `create_time` DATETIME NOT NULL COMMENT '创建时间', `update_by` VARCHAR(64) NOT NULL DEFAULT "" COMMENT '更新者id', `update_time` DATETIME NOT NULL COMMENT '更新时间', `id_card` VARCHAR(64) NOT NULL DEFAULT "" COMMENT '员工身份证号码', `social_account` VARCHAR(64) NOT NULL DEFAULT "" COMMENT '员工社保帐号' ) ENGINE=ODBC COMMENT 'ODBC' PROPERTIES ( "host" = "***", "port" = "3306", "user" = "*", "password" = "", "driver" = "MySQL ODBC 5.3 Unicode Driver", "odbc_type" = "mysql", "charest" = "utf8", "database" = "speedaf_cheetah_basic", "table" = "ts_staff" ); ### What You Expected? 希望2.1.0能继续支持odbc外链表，目前我们生产上面还有很多odbc外链表。 ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](load) retry 3 times if failed to get base_migration_lock in rowset builder [doris]
doris-robot commented on PR #32243: URL: https://github.com/apache/doris/pull/32243#issuecomment-1996919029 TeamCity be ut coverage result: Function Coverage: 34.99% (8573/24498) Line Coverage: 26.74% (69451/259773) Region Coverage: 26.02% (36076/138671) Branch Coverage: 22.96% (18417/80196) Coverage Report: http://coverage.selectdb-in.cc/coverage/c9980a733142c17046290ef818680730aa860935_c9980a733142c17046290ef818680730aa860935/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](spark load) update spark version for spark load to resolve cve problem [doris]
doris-robot commented on PR #30368: URL: https://github.com/apache/doris/pull/30368#issuecomment-1996919713 TPC-H: Total hot run time: 38360 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 7ad6689c8b6b4eb5c12094982deb72fb7f80c4c2, data reload: false -- Round 1 -- q1 17699 557241524152 q2 2039153 149 149 q3 10752 1051903 903 q4 7878780 724 724 q5 7467257726232577 q6 184 125 123 123 q7 1231830 793 793 q8 9349203420322032 q9 7097643064226422 q10 8491349936353499 q11 426 218 221 218 q12 573 310 305 305 q13 17806 281528742815 q14 273 252 245 245 q15 505 460 465 460 q16 488 413 389 389 q17 947 597 519 519 q18 7231655364376437 q19 2828149414211421 q20 566 285 281 281 q21 6221359136563591 q22 363 305 307 305 Total cold run time: 110414 ms Total hot run time: 38360 ms - Round 2, with runtime_filter_mode=off - q1 4144411940614061 q2 316 224 229 224 q3 2928282628422826 q4 1893158116021581 q5 5214524652755246 q6 195 116 118 116 q7 2262181718741817 q8 3154326832833268 q9 8616856485388538 q10 3753366836883668 q11 535 455 439 439 q12 718 544 539 539 q13 16915 287028572857 q14 297 251 242 242 q15 481 452 454 452 q16 450 437 413 413 q17 1734146314871463 q18 7531726671677167 q19 1617154214971497 q20 1897170217041702 q21 4705474646284628 q22 533 452 463 452 Total cold run time: 69888 ms Total hot run time: 53196 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [nereids] enable gby key eliminate by default [doris]
xzj7019 commented on PR #32238: URL: https://github.com/apache/doris/pull/32238#issuecomment-1996921837 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](decimal) fix Arithmetic Overflow error of converting string to decimal [doris]
jacktengg opened a new pull request, #32246: URL: https://github.com/apache/doris/pull/32246 ## Proposed changes Issue Number: close #31552 ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](decimal) fix Arithmetic Overflow error of converting string to decimal [doris]
doris-robot commented on PR #32246: URL: https://github.com/apache/doris/pull/32246#issuecomment-1996941885 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](decimal) fix Arithmetic Overflow error of converting string to decimal [doris]
jacktengg commented on PR #32246: URL: https://github.com/apache/doris/pull/32246#issuecomment-1996942239 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids): don't pushdown project when project contains both side of join [doris]
morrySnow commented on code in PR #32214:
URL: https://github.com/apache/doris/pull/32214#discussion_r1524495784
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/join/PushDownProjectThroughSemiJoin.java:
##
@@ -72,12 +75,27 @@ public List buildRules() {
.whenNot(j -> j.right().child().hasDistributeHint())
.then(topJoin -> {
LogicalProject>
project = topJoin.right();
+if (projectBothJoinSide(project)) {
+return null;
+}
Plan newRight = pushdownProject(project);
return topJoin.withChildren(topJoin.left(), newRight);
}).toRule(RuleType.PUSH_DOWN_PROJECT_THROUGH_SEMI_JOIN_RIGHT)
);
}
+private boolean projectBothJoinSide(LogicalProject> project) {
+// if project contains both side of join, it can't be pushed.
+// such as:
Review Comment:
select
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/join/PushDownProjectThroughSemiJoin.java:
##
@@ -72,12 +75,27 @@ public List buildRules() {
.whenNot(j -> j.right().child().hasDistributeHint())
.then(topJoin -> {
LogicalProject>
project = topJoin.right();
+if (projectBothJoinSide(project)) {
+return null;
+}
Plan newRight = pushdownProject(project);
return topJoin.withChildren(topJoin.left(), newRight);
}).toRule(RuleType.PUSH_DOWN_PROJECT_THROUGH_SEMI_JOIN_RIGHT)
);
}
+private boolean projectBothJoinSide(LogicalProject> project) {
+// if project contains both side of join, it can't be pushed.
+// such as:
Review Comment:
select
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](ip) conversion of ipv4 or ipv6 to arrow type [doris]
github-actions[bot] commented on PR #32240: URL: https://github.com/apache/doris/pull/32240#issuecomment-1996958780 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](decimal) fix Arithmetic Overflow error of converting string to decimal [doris]
github-actions[bot] commented on PR #32246: URL: https://github.com/apache/doris/pull/32246#issuecomment-1996964561 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat](Nereids): rewrite sum literal to sum and count [doris]
doris-robot commented on PR #32244: URL: https://github.com/apache/doris/pull/32244#issuecomment-1996961497 TPC-H: Total hot run time: 38352 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 2bca92b9a725b024843da6f278fa4e77247fc391, data reload: false -- Round 1 -- q1 17701 434941274127 q2 2023152 143 143 q3 10709 1043904 904 q4 7095730 738 730 q5 7470270527112705 q6 188 122 121 121 q7 1189821 814 814 q8 9333201920412019 q9 7170648264036403 q10 8460349936203499 q11 417 228 214 214 q12 613 295 289 289 q13 17782 286028412841 q14 282 252 241 241 q15 510 456 433 433 q16 485 396 397 396 q17 952 643 557 557 q18 7320650364016401 q19 1746140914821409 q20 553 286 286 286 q21 6458352935293529 q22 356 304 291 291 Total cold run time: 108812 ms Total hot run time: 38352 ms - Round 2, with runtime_filter_mode=off - q1 4142410040784078 q2 329 231 231 231 q3 3006282928412829 q4 1856159416311594 q5 5212524252605242 q6 195 119 116 116 q7 2242187118781871 q8 3184330033083300 q9 8551857885598559 q10 3702369335793579 q11 534 437 441 437 q12 746 565 541 541 q13 16900 288128602860 q14 279 249 245 245 q15 482 441 440 440 q16 456 418 406 406 q17 1736149314971493 q18 7635711871287118 q19 1588152315481523 q20 1888170617571706 q21 4911456547444565 q22 530 442 434 434 Total cold run time: 70104 ms Total hot run time: 53167 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] fix](decimal) fix Arithmetic Overflow error of converting string to decimal [doris]
jacktengg opened a new pull request, #32247: URL: https://github.com/apache/doris/pull/32247 ## Proposed changes Issue Number: pick from #32246 ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] fix](decimal) fix Arithmetic Overflow error of converting string to decimal [doris]
doris-robot commented on PR #32247: URL: https://github.com/apache/doris/pull/32247#issuecomment-1996965495 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
