Re: [PR] [fix](Nereids) fix lexer Backtracking or Ambiguity cause of key word duplicate [doris]

[via GitHub]

morrySnow merged PR #39750:
URL: https://github.com/apache/doris/pull/39750


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch branch-2.0 updated: [fix](Nereids) fix lexer Backtracking or Ambiguity cause of key word duplicate (#39590) (#39750)

[morrysnow]

This is an automated email from the ASF dual-hosted git repository.

morrysnow pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/branch-2.0 by this push:
 new fdf540f40fe [fix](Nereids) fix lexer Backtracking or Ambiguity cause 
of key word duplicate (#39590) (#39750)
fdf540f40fe is described below

commit fdf540f40fe80cc880d33e7826dee61c7e4f580d
Author: LiBinfeng <46676950+libinfeng...@users.noreply.github.com>
AuthorDate: Thu Aug 22 14:59:55 2024 +0800

[fix](Nereids) fix lexer Backtracking or Ambiguity cause of key word 
duplicate (#39590) (#39750)

cherry-pick from master #39590

issue intro by #39416
---
 .../antlr4/org/apache/doris/nereids/DorisLexer.g4  | 36 ++
 .../antlr4/org/apache/doris/nereids/DorisParser.g4 |  5 ++-
 .../apache/doris/nereids/parser/NereidsParser.java |  8 ++---
 3 files changed, 8 insertions(+), 41 deletions(-)

diff --git a/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisLexer.g4 
b/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisLexer.g4
index 64d2e9ce8ca..8916c263898 100644
--- a/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisLexer.g4
+++ b/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisLexer.g4
@@ -47,22 +47,6 @@ lexer grammar DorisLexer;
 }
   }
 
-  /**
-   * This method will be called when we see '/*' and try to match it as a 
bracketed comment.
-   * If the next character is '+', it should be parsed as hint later, and we 
cannot match
-   * it as a bracketed comment.
-   *
-   * Returns true if the next character is '+'.
-   */
-  public boolean isHint() {
-int nextChar = _input.LA(1);
-if (nextChar == '+') {
-  return true;
-} else {
-  return false;
-}
-  }
-
   /**
* This method will be called when the character stream ends and try to find 
out the
* unclosed bracketed comment.
@@ -72,19 +56,6 @@ lexer grammar DorisLexer;
   public void markUnclosedComment() {
 has_unclosed_bracketed_comment = true;
   }
-
-  // This variable will hold the external state
-  private boolean channel2;
-
-  // Method to set the external state
-  public void setChannel2(boolean value) {
-  this.channel2 = value;
-  }
-
-  // Method to decide the channel based on external state
-  private boolean isChannel2() {
-  return this.channel2;
-  }
 }
 
 SEMICOLON: ';';
@@ -587,6 +558,7 @@ COLON: ':';
 ARROW: '->';
 HINT_START: '/*+';
 HINT_END: '*/';
+COMMENT_START: '/*';
 ATSIGN: '@';
 DOUBLEATSIGN: '@@';
 
@@ -664,11 +636,7 @@ SIMPLE_COMMENT
 ;
 
 BRACKETED_COMMENT
-: '/*' {!isHint()}? ( BRACKETED_COMMENT | . )*? ('*/' | 
{markUnclosedComment();} EOF) -> channel(HIDDEN)
-;
-
-HINT_WITH_CHANNEL
-: {isChannel2()}? HINT_START .*? HINT_END -> channel(2)
+: COMMENT_START ( BRACKETED_COMMENT | . )*? ('*/' | 
{markUnclosedComment();} EOF) -> channel(2)
 ;
 
 
diff --git a/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4 
b/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4
index 9f65c14036b..62c5661589c 100644
--- a/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4
+++ b/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4
@@ -189,7 +189,7 @@ havingClause
 : HAVING booleanExpression
 ;
 
-selectHint: HINT_START hintStatements+=hintStatement (COMMA? 
hintStatements+=hintStatement)* HINT_END;
+selectHint: hintStatements+=hintStatement (COMMA? 
hintStatements+=hintStatement)* HINT_END;
 
 hintStatement
 : hintName=identifier (LEFT_PAREN parameters+=hintAssignment (COMMA? 
parameters+=hintAssignment)* RIGHT_PAREN)?
@@ -602,6 +602,7 @@ nonReserved
 | COLLATION
 | COLUMNS
 | COMMENT
+| COMMENT_START
 | COMMIT
 | COMMITTED
 | COMPACT
@@ -685,6 +686,8 @@ nonReserved
 | HASH
 | HDFS
 | HELP
+| HINT_END
+| HINT_START
 | HISTOGRAM
 | HLL_UNION
 | HOSTNAME
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/NereidsParser.java 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/NereidsParser.java
index 7bf7c7737bf..4fa75083378 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/NereidsParser.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/NereidsParser.java
@@ -92,18 +92,15 @@ public class NereidsParser {
  
Function parseFunction) {
 // parse hint first round
 DorisLexer hintLexer = new DorisLexer(new 
CaseInsensitiveStream(CharStreams.fromString(sql)));
-hintLexer.setChannel2(true);
 CommonTokenStream hintTokenStream = new CommonTokenStream(hintLexer);
 
 Map selectHintMap = Maps.newHashMap();
 
 Token hintToken = hintTokenStream.getTokenSource().nextToken();
 while (hintToken != null && hintToken.getType() != DorisLexer.EOF) {
-int token

Re: [PR] [exec](pipeline) enable pipeline dml in coordinator [doris]

[via GitHub]

HappenLee commented on PR #39365:
URL: https://github.com/apache/doris/pull/39365#issuecomment-2303928427

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[PR] [refactor](local exchange) Refactor logics [doris]

[via GitHub]

Gabriel39 opened a new pull request, #39771:
URL: https://github.com/apache/doris/pull/39771

   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [refactor](local exchange) Refactor logics [doris]

[via GitHub]

doris-robot commented on PR #39771:
URL: https://github.com/apache/doris/pull/39771#issuecomment-2303931565

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
   
   Since 2024-03-18, the Document has been moved to 
[doris-website](https://github.com/apache/doris-website).
   See [Doris 
Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [refactor](local exchange) Refactor logics [doris]

[via GitHub]

Gabriel39 commented on PR #39771:
URL: https://github.com/apache/doris/pull/39771#issuecomment-2303931685

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[PR] Fix typo [doris-website]

[via GitHub]

Oliverwqcwrw opened a new pull request, #1038:
URL: https://github.com/apache/doris-website/pull/1038

   (no comment)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[PR] [enhance](hdfs) Add hdfs file writer benchmark and flush hdfs writer at some threshold [doris]

[via GitHub]

platoneko opened a new pull request, #39772:
URL: https://github.com/apache/doris/pull/39772

   ## Proposed changes
   
   Add hdfs file writer benchmark and flush hdfs writer at some threshold
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [enhance](hdfs) Add hdfs file writer benchmark and flush hdfs writer at some threshold [doris]

[via GitHub]

doris-robot commented on PR #39772:
URL: https://github.com/apache/doris/pull/39772#issuecomment-2303938787

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
   
   Since 2024-03-18, the Document has been moved to 
[doris-website](https://github.com/apache/doris-website).
   See [Doris 
Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feature](doris compose) ls detail command print node's path [doris]

[via GitHub]

yujun777 commented on PR #39767:
URL: https://github.com/apache/doris/pull/39767#issuecomment-2303940309

   run p1


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](Nereids) fix fold constant by be return type mismatched [doris]

[via GitHub]

doris-robot commented on PR #39723:
URL: https://github.com/apache/doris/pull/39723#issuecomment-2303941244

   
   
   TPC-H: Total hot run time: 38325 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 137723ba209e66f2f59257c70623943e16828686, 
data reload: false
   
   -- Round 1 --
   q1   17982   455344074407
   q2   2931186 178 178
   q3   11694   118010881088
   q4   10471   739 784 739
   q5   7828294128522852
   q6   228 138 140 138
   q7   981 625 598 598
   q8   9321208420542054
   q9   7009650065406500
   q10  6998219622212196
   q11  468 242 248 242
   q12  401 226 222 222
   q13  17950   303230103010
   q14  279 238 236 236
   q15  520 489 488 488
   q16  493 388 391 388
   q17  989 671 703 671
   q18  7300689469096894
   q19  1427101710231017
   q20  676 329 342 329
   q21  3945306931883069
   q22  1119101210091009
   Total cold run time: 111010 ms
   Total hot run time: 38325 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4563431942744274
   q2   388 282 265 265
   q3   2888265126832651
   q4   1891165716191619
   q5   5402537953755375
   q6   217 132 129 129
   q7   2082173017351730
   q8   3159335433373337
   q9   8398852884428442
   q10  3455317932033179
   q11  602 514 496 496
   q12  793 618 624 618
   q13  14199   306230753062
   q14  311 270 279 270
   q15  507 470 483 470
   q16  474 429 419 419
   q17  1781147614991476
   q18  7767762975057505
   q19  1677166114901490
   q20  2070180418541804
   q21  5457510851615108
   q22  1132104310611043
   Total cold run time: 69213 ms
   Total hot run time: 54762 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [refactor](local exchange) Refactor logics [doris]

[via GitHub]

github-actions[bot] commented on PR #39771:
URL: https://github.com/apache/doris/pull/39771#issuecomment-2303944018

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Feature] support data mask in doris internal auth [doris]

[via GitHub]

morrySnow commented on code in PR #38301:
URL: https://github.com/apache/doris/pull/38301#discussion_r1726475940


##
fe/fe-core/src/main/cup/sql_parser.cup:
##
@@ -498,6 +498,7 @@ terminal String
 KW_MATCH_PHRASE_PREFIX,
 KW_MATCH_REGEXP,
 KW_MATCH_PHRASE_EDGE,
+KW_MASK,

Review Comment:
   add mask to non-reserved key word list



##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ShowPolicyCommand.java:
##
@@ -0,0 +1,73 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.plans.commands;
+
+import org.apache.doris.analysis.UserIdentity;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.ErrorReport;
+import org.apache.doris.mysql.privilege.PrivPredicate;
+import org.apache.doris.nereids.exceptions.AnalysisException;
+import org.apache.doris.nereids.trees.plans.PlanType;
+import org.apache.doris.nereids.trees.plans.visitor.PlanVisitor;
+import org.apache.doris.policy.PolicyTypeEnum;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.ShowResultSet;
+import org.apache.doris.qe.StmtExecutor;
+
+/**
+ * show policy
+ */
+public class ShowPolicyCommand extends Command implements NoForward {
+
+private final PolicyTypeEnum policyType;
+
+private final UserIdentity user;
+
+private final String roleName;
+
+public ShowPolicyCommand(PolicyTypeEnum policyType, UserIdentity user, 
String roleName) {
+super(PlanType.SHOW_POLICY_COMMAND);
+this.policyType = policyType;
+this.user = user;
+this.roleName = roleName;
+}
+
+@Override
+public  R accept(PlanVisitor visitor, C context) {
+return visitor.visitShowPolicyCommand(this, context);
+}
+
+@Override
+public void run(ConnectContext ctx, StmtExecutor executor) throws 
Exception {
+if (policyType == PolicyTypeEnum.DATA_MASK) {
+if (user != null) {
+user.analyze();
+}
+// check auth
+if 
(!Env.getCurrentEnv().getAccessManager().checkGlobalPriv(ConnectContext.get(), 
PrivPredicate.ADMIN)) {
+
ErrorReport.reportAnalysisException(ErrorCode.ERR_SPECIFIC_ACCESS_DENIED_ERROR, 
"ADMIN");
+}
+ShowResultSet showResultSet = 
Env.getCurrentEnv().getPolicyMgr().showPolicy(user, roleName, policyType);
+executor.sendResultSet(showResultSet);
+return;
+}
+ctx.getSessionVariable().enableFallbackToOriginalPlannerOnce();
+throw new AnalysisException("Not support show policy command in 
Nereids now");

Review Comment:
   throw new MustFallbackException



##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/DropPolicyCommand.java:
##
@@ -0,0 +1,78 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.plans.commands;
+
+import org.apache.doris.analysis.StmtType;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.ErrorReport;
+import org.apache.doris.mysql.privilege.PrivPredicate;
+import org.apache.doris.nereids.exceptions.AnalysisException;
+import org.apache.doris.nereids.trees.plans.PlanType;
+import org.apache.doris.nereids.trees.plans.visitor.PlanVisitor;
+import org.apache.doris.policy.DropPolicyLog;
+import org

(doris) branch branch-3.0.1-tmp-arm updated: [chore](build) Skip cloud module for arm64 (aarch64) in build-for-release.sh

[gavinchou]

This is an automated email from the ASF dual-hosted git repository.

gavinchou pushed a commit to branch branch-3.0.1-tmp-arm
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/branch-3.0.1-tmp-arm by this 
push:
 new eeaea88ac33 [chore](build) Skip cloud module for arm64 (aarch64) in 
build-for-release.sh
eeaea88ac33 is described below

commit eeaea88ac337cd5910135c2231f50b7290e75018
Author: Gavin Chou 
AuthorDate: Thu Aug 22 15:11:52 2024 +0800

[chore](build) Skip cloud module for arm64 (aarch64) in build-for-release.sh
---
 build-for-release.sh | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/build-for-release.sh b/build-for-release.sh
index 4d9a257bd2e..2347f9ebaab 100755
--- a/build-for-release.sh
+++ b/build-for-release.sh
@@ -164,7 +164,11 @@ cp -R "${ORI_OUTPUT}"/apache_hdfs_broker 
"${OUTPUT_EXT}"/apache_hdfs_broker
 cp -R "${ORI_OUTPUT}"/be/* "${OUTPUT_BE}"/
 
 # CLOUD
-cp -R "${ORI_OUTPUT}"/ms/* "${OUTPUT_CLOUD}"/
+if [[ "${ARCH}" == "arm64" ]]; then
+echo "WARNING: Cloud module is not supported on ARM platform, will skip 
building it."
+else
+cp -R "${ORI_OUTPUT}"/ms/* "${OUTPUT_CLOUD}"/
+fi
 
 if [[ "${TAR}" -eq 1 ]]; then
 echo "Begin to compress"


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Pick][Improment]Query queued by be memory (#37559) [doris]

[via GitHub]

wangbo merged PR #39733:
URL: https://github.com/apache/doris/pull/39733


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch branch-2.1 updated: [Pick][Improment]Query queued by be memory (#37559) (#39733)

[wangbo]

This is an automated email from the ASF dual-hosted git repository.

wangbo pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/branch-2.1 by this push:
 new 1c566253a81 [Pick][Improment]Query queued by be memory (#37559) 
(#39733)
1c566253a81 is described below

commit 1c566253a819ffa6b91eb8fb2925b792520bff26
Author: wangbo 
AuthorDate: Thu Aug 22 15:14:47 2024 +0800

[Pick][Improment]Query queued by be memory (#37559) (#39733)

pick #37559
---
 be/src/service/backend_service.cpp |  11 ++
 be/src/service/backend_service.h   |   3 +
 .../main/java/org/apache/doris/common/Config.java  |  11 ++
 .../main/java/org/apache/doris/catalog/Env.java|   9 ++
 .../main/java/org/apache/doris/qe/Coordinator.java |  43 --
 .../java/org/apache/doris/qe/QeProcessorImpl.java  |   7 +-
 .../apache/doris/resource/AdmissionControl.java| 156 +
 .../doris/resource/workloadgroup/QueryQueue.java   | 101 +++--
 .../doris/resource/workloadgroup/QueueToken.java   | 152 +++-
 .../resource/workloadgroup/WorkloadGroup.java  |   6 +-
 .../doris/tablefunction/MetadataGenerator.java |  11 +-
 .../org/apache/doris/common/GenericPoolTest.java   |   7 +
 .../apache/doris/utframe/MockedBackendFactory.java |   7 +
 gensrc/thrift/BackendService.thrift|  15 ++
 .../workload_manager_p0/test_curd_wlg.groovy   |   4 +-
 15 files changed, 367 insertions(+), 176 deletions(-)

diff --git a/be/src/service/backend_service.cpp 
b/be/src/service/backend_service.cpp
index e26264b1a22..c30b936769a 100644
--- a/be/src/service/backend_service.cpp
+++ b/be/src/service/backend_service.cpp
@@ -1110,4 +1110,15 @@ void 
BackendService::query_ingest_binlog(TQueryIngestBinlogResult& result,
 break;
 }
 }
+
+void BackendService::get_be_resource(TGetBeResourceResult& result,
+ const TGetBeResourceRequest& request) {
+int64_t mem_usage = PerfCounters::get_vm_rss();
+int64_t mem_limit = MemInfo::mem_limit();
+TGlobalResourceUsage global_resource_usage;
+global_resource_usage.__set_mem_limit(mem_limit);
+global_resource_usage.__set_mem_usage(mem_usage);
+result.__set_global_resource_usage(global_resource_usage);
+}
+
 } // namespace doris
diff --git a/be/src/service/backend_service.h b/be/src/service/backend_service.h
index 9d53ec4bc45..bbcf103167f 100644
--- a/be/src/service/backend_service.h
+++ b/be/src/service/backend_service.h
@@ -140,6 +140,9 @@ public:
 void query_ingest_binlog(TQueryIngestBinlogResult& result,
  const TQueryIngestBinlogRequest& request) 
override;
 
+void get_be_resource(TGetBeResourceResult& result,
+ const TGetBeResourceRequest& request) override;
+
 private:
 Status start_plan_fragment_execution(const TExecPlanFragmentParams& 
exec_params);
 ExecEnv* _exec_env = nullptr;
diff --git a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java 
b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
index 6a0394a0be7..5e0cb99b638 100644
--- a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
+++ b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
@@ -1782,6 +1782,17 @@ public class Config extends ConfigBase {
 @ConfField(mutable = true, varType = VariableAnnotation.EXPERIMENTAL)
 public static boolean enable_cpu_hard_limit = false;
 
+@ConfField(mutable = true, description = {
+"当BE内存用量大于该值时，查询会进入排队逻辑，默认值为-1，代表该值不生效。取值范围0~1的小数",
+"When be memory usage bigger than this value, query could queue, "
++ "default value is -1, means this value not work. Decimal 
value range from 0 to 1"})
+public static double query_queue_by_be_used_memory = -1;
+
+@ConfField(mutable = true, description = {"基于内存反压场景FE定时拉取BE内存用量的时间间隔",
+"In the scenario of memory backpressure, "
++ "the time interval for obtaining BE memory usage at 
regular intervals"})
+public static long get_be_resource_usage_interval_ms = 1;
+
 @ConfField(mutable = false, masterOnly = true)
 public static int backend_rpc_timeout_ms = 6; // 1 min
 
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java 
b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
index 785b7add3a0..d6af35c5bf1 100755
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
@@ -242,6 +242,7 @@ import org.apache.doris.qe.JournalObservable;
 import org.apache.doris.qe.QueryCancelWorker;
 import org.apache.doris.qe.SessionVariable;
 import org.apache.doris.qe.VariableMgr;
+import org.apache.doris.resource.AdmissionControl;
 import org.apache.doris.resource.Tag;
 import org.apache.doris.resource.workloadgroup.WorkloadGro

Re: [PR] [fix](window_funnel) fix problem when upgrading from 3.0 caused by behaviour change of window_funnel [doris]

[via GitHub]

github-actions[bot] commented on PR #39739:
URL: https://github.com/apache/doris/pull/39739#issuecomment-2303951622

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](window_funnel) fix upgrading problem caused by behaviour change of window_funnel [doris]

[via GitHub]

github-actions[bot] commented on PR #39766:
URL: https://github.com/apache/doris/pull/39766#issuecomment-2303952457

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [enhance](hdfs) Add hdfs file writer benchmark and flush hdfs writer at some threshold [doris]

[via GitHub]

github-actions[bot] commented on code in PR #39772:
URL: https://github.com/apache/doris/pull/39772#discussion_r1726485336


##
be/src/io/fs/benchmark/hdfs_file_writer_benchmark.cpp:
##
@@ -0,0 +1,151 @@
+#include 
+#include 
+#include 
+#include 
+
+#include 
+
+#include "common/logging.h"
+#include "io/fs/file_writer.h"
+#include "io/fs/hdfs_file_system.h"
+#include "runtime/exec_env.h"
+#include "util/jvm_metrics.h"
+#include "util/threadpool.h"
+
+using namespace doris;
+
+DEFINE_string(fs_name, "", "hdfs fs name");
+DEFINE_string(path_prefix, "/hdfs_file_writer_bench", "hdfs path prefix");
+DEFINE_int32(num_files, 32, "number of files to write");
+DEFINE_uint64(file_size, 10 * 1024 * 1024 /* 10MB */, "size of each file");
+DEFINE_int32(parallel, 16, "number of parallel writers");
+DEFINE_int32(flush_delay, 0, "delay in ms when flushing data to HDFS");
+
+namespace doris::io {
+extern bvar::Adder hdfs_write_acquire_memory_failed;
+extern bvar::Adder hdfs_write_acquire_memory_reach_max_retry;
+} // namespace doris::io
+
+void run_benchmark(io::HdfsFileSystem& fs, int num_files, size_t file_size, 
int parallel,

Review Comment:
   warning: function 'run_benchmark' exceeds recommended size/complexity 
thresholds [readability-function-size]
   ```cpp
   void run_benchmark(io::HdfsFileSystem& fs, int num_files, size_t file_size, 
int parallel,
^
   ```
   
   Additional context
   
   **be/src/io/fs/benchmark/hdfs_file_writer_benchmark.cpp:28:** 82 lines 
including whitespace and comments (threshold 80)
   ```cpp
   void run_benchmark(io::HdfsFileSystem& fs, int num_files, size_t file_size, 
int parallel,
^
   ```
   
   
   



##
be/src/io/fs/benchmark/hdfs_file_writer_benchmark.cpp:
##
@@ -0,0 +1,151 @@
+#include 

Review Comment:
   warning: 'bthread/countdown_event.h' file not found [clang-diagnostic-error]
   ```cpp
   #include 
^
   ```
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](cancel) Fix incorrect cancel [doris]

[via GitHub]

github-actions[bot] commented on PR #39737:
URL: https://github.com/apache/doris/pull/39737#issuecomment-2303953568

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](nereids) fix distribution expr list [doris]

[via GitHub]

morrySnow merged PR #39435:
URL: https://github.com/apache/doris/pull/39435


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch branch-2.1 updated: [fix](nereids) fix distribution expr list (#39435)

[morrysnow]

This is an automated email from the ASF dual-hosted git repository.

morrysnow pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/branch-2.1 by this push:
 new 10f3e88f7a2 [fix](nereids) fix distribution expr list (#39435)
10f3e88f7a2 is described below

commit 10f3e88f7a2495eda9608ba43420cbeeae3c93cb
Author: xzj7019 <13794+xzj7...@users.noreply.github.com>
AuthorDate: Thu Aug 22 15:19:51 2024 +0800

[fix](nereids) fix distribution expr list (#39435)

pick from #39148
---
 .../org/apache/doris/nereids/NereidsPlanner.java   |   3 +-
 .../properties/ChildOutputPropertyDeriver.java | 104 +++
 .../properties/ChildOutputPropertyDeriverTest.java |  36 ++--
 .../data/nereids_p0/hint/test_distribute.out   | 193 ++---
 4 files changed, 188 insertions(+), 148 deletions(-)

diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/NereidsPlanner.java 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/NereidsPlanner.java
index 74a54036f9c..33c1554b1c8 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/nereids/NereidsPlanner.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/nereids/NereidsPlanner.java
@@ -434,8 +434,7 @@ public class NereidsPlanner extends Planner {
 // add groupExpression to plan so that we could print group id in 
plan.treeString()
 plan = plan.withGroupExpression(Optional.of(groupExpression));
 PhysicalPlan physicalPlan = ((PhysicalPlan) 
plan).withPhysicalPropertiesAndStats(
-groupExpression.getOutputProperties(physicalProperties),
-groupExpression.getOwnerGroup().getStatistics());
+physicalProperties, 
groupExpression.getOwnerGroup().getStatistics());
 return physicalPlan;
 } catch (Exception e) {
 if (e instanceof AnalysisException && 
e.getMessage().contains("Failed to choose best plan")) {
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/properties/ChildOutputPropertyDeriver.java
 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/properties/ChildOutputPropertyDeriver.java
index bb23f46cdc4..270712496e9 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/properties/ChildOutputPropertyDeriver.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/properties/ChildOutputPropertyDeriver.java
@@ -256,8 +256,25 @@ public class ChildOutputPropertyDeriver extends 
PlanVisitor hashJoin,
+DistributionSpecHash leftHashSpec, DistributionSpecHash 
rightHashSpec) {
+switch (hashJoin.getJoinType()) {
+case INNER_JOIN:
+case CROSS_JOIN:
+return new PhysicalProperties(DistributionSpecHash.merge(
+leftHashSpec, rightHashSpec, 
leftHashSpec.getShuffleType()));
+case LEFT_SEMI_JOIN:
+case LEFT_ANTI_JOIN:
+case NULL_AWARE_LEFT_ANTI_JOIN:
+case LEFT_OUTER_JOIN:
+return new PhysicalProperties(leftHashSpec);
+case RIGHT_SEMI_JOIN:
+case RIGHT_ANTI_JOIN:
+case RIGHT_OUTER_JOIN:
+if (JoinUtils.couldColocateJoin(leftHashSpec, rightHashSpec, 
hashJoin.getHashJoinConjuncts())) {
+return new PhysicalProperties(rightHashSpec);
+} else {
+// retain left shuffle type, since coordinator use left 
most node to schedule fragment
+// forbid colocate join, since right table already shuffle
+return new 
PhysicalProperties(rightHashSpec.withShuffleTypeAndForbidColocateJoin(
+leftHashSpec.getShuffleType()));
+}
+case FULL_OUTER_JOIN:
+return PhysicalProperties.createAnyFromHash(leftHashSpec);
+default:
+throw new AnalysisException("unknown join type " + 
hashJoin.getJoinType());
+}
+}
+
+private DistributionSpecHash mockAnotherSideSpecFromConjuncts(
+PhysicalHashJoin hashJoin, 
DistributionSpecHash oneSideSpec) {
+List leftExprIds = hashJoin.getHashConjunctsExprIds().first;
+List rightExprIds = hashJoin.getHashConjunctsExprIds().second;
+Preconditions.checkState(!leftExprIds.isEmpty() && 
!rightExprIds.isEmpty()
+&& leftExprIds.size() == rightExprIds.size(), "invalid hash 
join conjuncts");
+List anotherSideOrderedExprIds = Lists.newArrayList();
+for (ExprId exprId : oneSideSpec.getOrderedShuffledColumns()) {
+int index = leftExprIds.indexOf(exprId);
+if (index == -1) {
+Set equivalentExprIds = 
oneSideSpec.getEquivalenceExprIdsOf(exprId);
+for (ExprId id : equivalentExprIds) {
+index = leftExprIds.indexOf(id);
+if (index >= 0) {
+ 

Re: [PR] [feature](doris compose) ls detail command print node's path [doris]

[via GitHub]

yujun777 commented on PR #39767:
URL: https://github.com/apache/doris/pull/39767#issuecomment-2303959876

   run cloud_p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](Nereids) fix fold constant by be return type mismatched [doris]

[via GitHub]

doris-robot commented on PR #39723:
URL: https://github.com/apache/doris/pull/39723#issuecomment-2303959533

   
   
   TPC-DS: Total hot run time: 186767 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 137723ba209e66f2f59257c70623943e16828686, 
data reload: false
   
   query1   907 368 355 355
   query2   6456191519321915
   query3   6653210 219 210
   query4   34319   23171   23245   23171
   query5   4184507 499 499
   query6   252 166 164 164
   query7   4575298 288 288
   query8   258 219 205 205
   query9   8755247024852470
   query10  424 265 264 264
   query11  15699   15020   15039   15020
   query12  156 103 99  99
   query13  1623381 365 365
   query14  10009   732569126912
   query15  272 165 173 165
   query16  8109453 464 453
   query17  1570558 561 558
   query18  2129288 285 285
   query19  267 141 146 141
   query20  119 110 107 107
   query21  216 101 96  96
   query22  4499404142394041
   query23  34197   33707   33488   33488
   query24  11125   294329142914
   query25  652 405 406 405
   query26  1335159 163 159
   query27  2803280 278 278
   query28  7445204020122012
   query29  850 408 412 408
   query30  304 153 153 153
   query31  991 743 744 743
   query32  102 54  61  54
   query33  760 297 300 297
   query34  964 470 488 470
   query35  846 738 727 727
   query36  1061933 943 933
   query37  146 90  82  82
   query38  3970385438493849
   query39  1440139613871387
   query40  265 119 116 116
   query41  49  45  45  45
   query42  114 99  102 99
   query43  490 452 486 452
   query44  1286746 758 746
   query45  201 173 167 167
   query46  1110752 765 752
   query47  1875177618211776
   query48  387 307 302 302
   query49  1085434 422 422
   query50  821 418 420 418
   query51  7336697770456977
   query52  99  89  88  88
   query53  252 185 184 184
   query54  876 463 455 455
   query55  80  75  80  75
   query56  270 263 245 245
   query57  1253110110651065
   query58  249 221 236 221
   query59  2861275027652750
   query60  289 266 288 266
   query61  120 101 98  98
   query62  841 644 671 644
   query63  217 189 188 188
   query64  6410225517401740
   query65  3243320331593159
   query66  1376362 369 362
   query67  15451   15261   15496   15261
   query68  3547578 591 578
   query69  390 277 285 277
   query70  1199
   query71  334 279 272 272
   query72  6198230820712071
   query73  759 328 332 328
   query74  9140881687458745
   query75  3416270926962696
   query76  1847986 959 959
   query77  449 314 315 314
   query78  9597892992738929
   query79  1049554 547 547
   query80  699 510 494 494
   query81  454 229 232 229
   query82  227 137 135 135
   query83  172 150 155 150
   query84  220 91  86  86
   query85  724 341 272 272
   query86  312 266 301 266
   query87  4411430742534253
   query88  2968238223622362
   query89  367 293 296 293
   query90  1854202 206 202
   query91  129 104 105 104
   query92  63  52  51  51
   query93  1039554 543 543
   query94  817 301 299 299
   query95  364 268 262 262
   query96  593 274 277 274
   query97  3230303030903030
   query98  222 205 200 200
   query99  1463122012571220
   Total cold run time: 287601 ms
   Total hot run time: 186767 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to th

Re: [PR] [fix](mtmv) transfer col in mysql varchar to text when create MTMV (… [doris]

[via GitHub]

morrySnow merged PR #39727:
URL: https://github.com/apache/doris/pull/39727


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch branch-2.1 updated: [fix](mtmv) transfer col in mysql varchar to text when create MTMV (#37668) (#39727)

[morrysnow]

This is an automated email from the ASF dual-hosted git repository.

morrysnow pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/branch-2.1 by this push:
 new f553645a71f [fix](mtmv) transfer col in  mysql varchar to text when 
create MTMV (#37668) (#39727)
f553645a71f is described below

commit f553645a71f41cf730f3e9d04eab9ed5ed733465
Author: zhangdong <493738...@qq.com>
AuthorDate: Thu Aug 22 15:20:59 2024 +0800

[fix](mtmv) transfer col in  mysql varchar to text when create MTMV 
(#37668) (#39727)

pick from master #37668
---
 .../trees/plans/commands/CreateTableCommand.java   |  2 +-
 .../trees/plans/commands/info/CreateMTMVInfo.java  | 54 --
 .../external_table_p0/tvf/test_ctas_with_hdfs.out  |  2 +-
 regression-test/data/mtmv_p0/test_build_mtmv.out   |  2 +-
 regression-test/data/mtmv_p0/test_mysql_mtmv.out   |  8 
 .../suites/mtmv_p0/test_mysql_mtmv.groovy  | 14 +-
 6 files changed, 72 insertions(+), 10 deletions(-)

diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/CreateTableCommand.java
 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/CreateTableCommand.java
index fe54a5dfd9f..33d25723893 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/CreateTableCommand.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/CreateTableCommand.java
@@ -138,7 +138,7 @@ public class CreateTableCommand extends Command implements 
ForwardWithSync {
 // String type can not be used in 
partition/distributed column
 // so we replace it to varchar
 dataType = 
TypeCoercionUtils.replaceSpecifiedType(dataType,
-StringType.class, 
VarcharType.MAX_VARCHAR_TYPE);
+CharacterType.class, 
VarcharType.MAX_VARCHAR_TYPE);
 } else {
 dataType = 
TypeCoercionUtils.replaceSpecifiedType(dataType,
 CharacterType.class, StringType.INSTANCE);
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/info/CreateMTMVInfo.java
 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/info/CreateMTMVInfo.java
index c65d6b6ded8..cb99a853906 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/info/CreateMTMVInfo.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/info/CreateMTMVInfo.java
@@ -29,6 +29,7 @@ import org.apache.doris.catalog.Column;
 import org.apache.doris.catalog.Env;
 import org.apache.doris.catalog.KeysType;
 import org.apache.doris.catalog.PartitionType;
+import org.apache.doris.catalog.ScalarType;
 import org.apache.doris.catalog.TableIf;
 import org.apache.doris.catalog.Type;
 import org.apache.doris.catalog.View;
@@ -56,6 +57,7 @@ import org.apache.doris.nereids.properties.PhysicalProperties;
 import org.apache.doris.nereids.rules.exploration.mv.MaterializedViewUtils;
 import org.apache.doris.nereids.trees.expressions.Expression;
 import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.expressions.SlotReference;
 import org.apache.doris.nereids.trees.plans.Plan;
 import org.apache.doris.nereids.trees.plans.algebra.OneRowRelation;
 import 
org.apache.doris.nereids.trees.plans.commands.ExplainCommand.ExplainLevel;
@@ -63,9 +65,14 @@ import 
org.apache.doris.nereids.trees.plans.logical.LogicalPlan;
 import org.apache.doris.nereids.trees.plans.logical.LogicalSink;
 import org.apache.doris.nereids.trees.plans.logical.LogicalSubQueryAlias;
 import org.apache.doris.nereids.types.AggStateType;
+import org.apache.doris.nereids.types.CharType;
 import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.nereids.types.DecimalV2Type;
 import org.apache.doris.nereids.types.NullType;
+import org.apache.doris.nereids.types.StringType;
 import org.apache.doris.nereids.types.TinyIntType;
+import org.apache.doris.nereids.types.VarcharType;
+import org.apache.doris.nereids.types.coercion.CharacterType;
 import org.apache.doris.nereids.util.TypeCoercionUtils;
 import org.apache.doris.nereids.util.Utils;
 import org.apache.doris.qe.ConnectContext;
@@ -223,11 +230,12 @@ public class CreateMTMVInfo {
 throw new AnalysisException("can not contain invalid expression");
 }
 getRelation(planner);
-getColumns(plan);
-analyzeKeys();
 this.mvPartitionInfo = mvPartitionDefinition
 .analyzeAndTransferToMTMVPartitionInfo(planner, ctx, 
logicalQuery);
 this.partitionDesc = generatePartitionDesc(ctx);
+getColumns(plan, ctx, mvPartitionInfo.getPartitionCol(), distribution);
+analyzeKeys();
+
 }
 
  

Re: [PR] [fix](mtmv) use name instead of id in meta of MTMV (#39355) [doris]

[via GitHub]

morrySnow commented on PR #39748:
URL: https://github.com/apache/doris/pull/39748#issuecomment-2303960935

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[PR] [branch-2.1](thirdparty) upgrade arrow to 17.0.0 [doris]

[via GitHub]

xinyiZzz opened a new pull request, #39773:
URL: https://github.com/apache/doris/pull/39773

   pick #38572


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [branch-2.1](thirdparty) upgrade arrow to 17.0.0 [doris]

[via GitHub]

doris-robot commented on PR #39773:
URL: https://github.com/apache/doris/pull/39773#issuecomment-2303961391

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
   
   Since 2024-03-18, the Document has been moved to 
[doris-website](https://github.com/apache/doris-website).
   See [Doris 
Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [branch-2.1](thirdparty) upgrade arrow to 17.0.0 [doris]

[via GitHub]

xinyiZzz commented on PR #39773:
URL: https://github.com/apache/doris/pull/39773#issuecomment-2303961588

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [branch-2.0][improvement](jdbc catalog) Force all resources to be closed in the close method [doris]

[via GitHub]

zy-kkk merged PR #39665:
URL: https://github.com/apache/doris/pull/39665


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch branch-2.0 updated: [branch-2.0][improvement](jdbc catalog) Force all resources to be closed in the close method (#39665)

[zykkk]

This is an automated email from the ASF dual-hosted git repository.

zykkk pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/branch-2.0 by this push:
 new f894a6b0a05 [branch-2.0][improvement](jdbc catalog) Force all 
resources to be closed in the close method (#39665)
f894a6b0a05 is described below

commit f894a6b0a05e21830c38f9182f1215094d8d6128
Author: zy-kkk 
AuthorDate: Thu Aug 22 15:23:16 2024 +0800

[branch-2.0][improvement](jdbc catalog) Force all resources to be closed in 
the close method (#39665)

pick #39423

Force all resources to be closed in the close method. In the previous
logic, query errors or query cancellation will not force the connection
to be closed, which will cause abnormal Hikari connection counts.
Although forced connection closure will generate some error logs in some
cases, we should have this bottom-line guarantee and refine the closing
logic later
---
 .../java/org/apache/doris/jdbc/JdbcExecutor.java   | 32 ++
 1 file changed, 8 insertions(+), 24 deletions(-)

diff --git 
a/fe/be-java-extensions/jdbc-scanner/src/main/java/org/apache/doris/jdbc/JdbcExecutor.java
 
b/fe/be-java-extensions/jdbc-scanner/src/main/java/org/apache/doris/jdbc/JdbcExecutor.java
index 17576c5b2cf..a3856a1156c 100644
--- 
a/fe/be-java-extensions/jdbc-scanner/src/main/java/org/apache/doris/jdbc/JdbcExecutor.java
+++ 
b/fe/be-java-extensions/jdbc-scanner/src/main/java/org/apache/doris/jdbc/JdbcExecutor.java
@@ -125,25 +125,18 @@ public class JdbcExecutor {
 
 public void close() throws Exception {
 try {
-if (stmt != null) {
+if (stmt != null && !stmt.isClosed()) {
 try {
 stmt.cancel();
 } catch (SQLException e) {
-LOG.error("Error cancelling statement", e);
+LOG.warn("Cannot cancelling statement: ", e);
 }
 }
 
-boolean shouldAbort = conn != null && resultSet != null
-&& (tableType == TOdbcTableType.MYSQL || tableType == 
TOdbcTableType.SQLSERVER);
-boolean aborted = false; // Used to record whether the abort 
operation is performed
-if (shouldAbort) {
-aborted = abortReadConnection(conn, resultSet, tableType);
-}
-
-// If no abort operation is performed, the resource needs to be 
closed manually
-if (!aborted) {
-closeResources(resultSet, stmt, conn);
+if (conn != null && resultSet != null) {
+abortReadConnection(conn, resultSet, tableType);
 }
+closeResources(resultSet, stmt, conn);
 } finally {
 if (config.getConnectionPoolMinSize() == 0 && hikariDataSource != 
null) {
 hikariDataSource.close();
@@ -157,32 +150,23 @@ public class JdbcExecutor {
 for (AutoCloseable closeable : closeables) {
 if (closeable != null) {
 try {
-if (closeable instanceof Connection) {
-if (!((Connection) closeable).isClosed()) {
-closeable.close();
-}
-} else {
-closeable.close();
-}
+closeable.close();
 } catch (Exception e) {
-LOG.error("Cannot close resource: ", e);
+LOG.warn("Cannot close resource: ", e);
 }
 }
 }
 }
 
-public boolean abortReadConnection(Connection connection, ResultSet 
resultSet, TOdbcTableType tableType)
+public void abortReadConnection(Connection connection, ResultSet 
resultSet, TOdbcTableType tableType)
 throws SQLException {
 if (!resultSet.isAfterLast() && (tableType == TOdbcTableType.MYSQL || 
tableType == TOdbcTableType.SQLSERVER)) {
 // Abort connection before closing. Without this, the 
MySQL/SQLServer driver
 // attempts to drain the connection by reading all the results.
 connection.abort(MoreExecutors.directExecutor());
-return true;
 }
-return false;
 }
 
-
 public void cleanDataSource() {
 if (hikariDataSource != null) {
 hikariDataSource.close();


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](Nereids) fix fold constant by be return type mismatched [doris]

[via GitHub]

doris-robot commented on PR #39723:
URL: https://github.com/apache/doris/pull/39723#issuecomment-2303968448

   
   
   ClickBench: Total hot run time: 30.5 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 137723ba209e66f2f59257c70623943e16828686, 
data reload: false
   
   query1   0.050.040.04
   query2   0.070.040.03
   query3   0.220.050.06
   query4   1.660.090.09
   query5   0.510.510.51
   query6   1.130.730.72
   query7   0.020.020.01
   query8   0.060.050.05
   query9   0.570.490.48
   query10  0.550.540.54
   query11  0.150.120.12
   query12  0.150.130.12
   query13  0.620.590.58
   query14  0.760.790.78
   query15  0.880.830.82
   query16  0.370.380.38
   query17  1.040.941.07
   query18  0.210.210.20
   query19  1.961.831.87
   query20  0.010.010.02
   query21  15.40   0.680.67
   query22  4.487.141.43
   query23  18.27   1.471.29
   query24  2.060.240.23
   query25  0.150.080.08
   query26  0.270.170.18
   query27  0.080.080.08
   query28  13.28   1.021.00
   query29  12.60   3.443.43
   query30  0.240.060.06
   query31  2.890.400.39
   query32  3.270.470.47
   query33  2.973.023.03
   query34  16.88   4.354.39
   query35  4.414.494.46
   query36  0.670.480.48
   query37  0.180.160.15
   query38  0.150.140.15
   query39  0.050.040.04
   query40  0.160.120.13
   query41  0.100.050.05
   query42  0.060.050.06
   query43  0.040.040.04
   Total cold run time: 109.65 s
   Total hot run time: 30.5 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Fix](Nereids) fix use cbo rule hint unused because of logic reverse [doris]

[via GitHub]

doris-robot commented on PR #39715:
URL: https://github.com/apache/doris/pull/39715#issuecomment-2303969819

   
   
   TPC-H: Total hot run time: 37837 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 42a8389249bedc1ef2b3604e0a880b6c6878fd54, 
data reload: false
   
   -- Round 1 --
   q1   17616   445542964296
   q2   2009180 171 171
   q3   11697   929 1093929
   q4   10520   745 654 654
   q5   7758283427982798
   q6   222 137 134 134
   q7   987 613 598 598
   q8   9501206320912063
   q9   6946653165186518
   q10  6996219822322198
   q11  470 243 239 239
   q12  392 226 230 226
   q13  18819   302329872987
   q14  279 246 239 239
   q15  525 478 486 478
   q16  484 388 381 381
   q17  984 662 676 662
   q18  7269690268906890
   q19  1387103010421030
   q20  684 331 330 330
   q21  4150300830973008
   q22  1130101910081008
   Total cold run time: 110825 ms
   Total hot run time: 37837 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4383438843094309
   q2   368 271 272 271
   q3   2913262226602622
   q4   1900167216331633
   q5   5587567857035678
   q6   221 129 132 129
   q7   2198184118411841
   q8   3247342834553428
   q9   8798881488018801
   q10  3630339733703370
   q11  612 519 488 488
   q12  826 701 660 660
   q13  13985   319732553197
   q14  312 287 287 287
   q15  543 492 487 487
   q16  491 457 448 448
   q17  1827155715231523
   q18  8083782278377822
   q19  1751163315001500
   q20  2133194918701870
   q21  5815546755745467
   q22  1138105210321032
   Total cold run time: 70761 ms
   Total hot run time: 56863 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](cancel) Fix incorrect cancel [doris]

[via GitHub]

Gabriel39 merged PR #39737:
URL: https://github.com/apache/doris/pull/39737


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch master updated: [fix](cancel) Fix incorrect cancel (#39737)

[gabriellee]

This is an automated email from the ASF dual-hosted git repository.

gabriellee pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new c36a7ca850a [fix](cancel) Fix incorrect cancel  (#39737)
c36a7ca850a is described below

commit c36a7ca850a4be4098cdd14c00d5df79b6da8dd6
Author: zhiqiang 
AuthorDate: Thu Aug 22 15:27:24 2024 +0800

[fix](cancel) Fix incorrect cancel  (#39737)

fix #39537
---
 be/src/runtime/fragment_mgr.cpp | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/be/src/runtime/fragment_mgr.cpp b/be/src/runtime/fragment_mgr.cpp
index 2e69bb34152..58bd2681ea7 100644
--- a/be/src/runtime/fragment_mgr.cpp
+++ b/be/src/runtime/fragment_mgr.cpp
@@ -1003,8 +1003,10 @@ void FragmentMgr::cancel_worker() {
 itr != running_queries_on_all_fes.end()) {
 // Query not found on this frontend, and the query 
arrives before the last check
 if (itr->second.find(it.first) == 
itr->second.end() &&
-q_ctx->get_query_arrival_timestamp().tv_nsec <
-
check_invalid_query_last_timestamp.tv_nsec &&
+// tv_nsec represents the number of 
nanoseconds that have elapsed since the time point stored in tv_sec.
+// tv_sec is enough, we do not need to check 
tv_nsec.
+q_ctx->get_query_arrival_timestamp().tv_sec <
+
check_invalid_query_last_timestamp.tv_sec &&
 q_ctx->get_query_source() == 
QuerySource::INTERNAL_FRONTEND) {
 
queries_pipeline_task_leak.push_back(q_ctx->query_id());
 LOG_INFO(


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [chore](CI) modify pipeline_task_leakage_detect_period_secs in CI [doris]

[via GitHub]

zhiqiang- commented on PR #39758:
URL: https://github.com/apache/doris/pull/39758#issuecomment-2303971673

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [opt](nereids)adjust aggregation cost to make it compatible to the cost of join and shuffle [doris]

[via GitHub]

englefly commented on PR #39175:
URL: https://github.com/apache/doris/pull/39175#issuecomment-2303973219

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Feat](nereids) support pull up predicate from set operator [doris]

[via GitHub]

feiniaofeiafei commented on PR #39450:
URL: https://github.com/apache/doris/pull/39450#issuecomment-2303979581

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[I] [Bug] Doris在大批量使用broker load任务时在第一次提交是会提示任务已存在 [doris]

[via GitHub]

xaiyuming20 opened a new issue, #39774:
URL: https://github.com/apache/doris/issues/39774

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no 
similar issues.
   
   
   ### Version
   
   doris的fe和be是2.1.5版本,但只有一个节点的broker load服务是2.1.5版本的其余节点的broker load服务均为1.2.7版本
   
   ### What's Wrong?
   
   业务反馈说dori的broker load任务,存在批量报: label已存在:【has already been used】,注在报错前以show 
load where label="labelName" order by createtime desc limit 
1\G;查看,返回的是空,没有,但是当任务提交后确提示已存在,第二次重新处理后任务提交成功
   ![Uploading EDA反馈doris的brokerlaod任务执行日志截图.png…]()
   
   
   ### What You Expected?
   
   可以解决该问题
   
   ### How to Reproduce?
   
   业务每天批量提交broker load任务,其使用的任务名复用以前的即label+tableNmae,再次提交是会先clean该任务后再提交broker 
load任务,在第一次提交任务时会报错提示改任务已存在
   
   ### Anything Else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch master updated: [Fix](ctas) forward the use_max_length_of_varchar_in_ctas (#39705)

[morrysnow]

This is an automated email from the ASF dual-hosted git repository.

morrysnow pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 12016acb2e3 [Fix](ctas) forward the use_max_length_of_varchar_in_ctas 
(#39705)
12016acb2e3 is described below

commit 12016acb2e36c073bac42460c5ea46c6c07b1331
Author: feiniaofeiafei <53502832+feiniaofeia...@users.noreply.github.com>
AuthorDate: Thu Aug 22 15:36:32 2024 +0800

[Fix](ctas) forward the use_max_length_of_varchar_in_ctas (#39705)
---
 fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java 
b/fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java
index 6b469beac05..a6136dd1f6b 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java
@@ -2072,7 +2072,7 @@ public class SessionVariable implements Serializable, 
Writable {
 checker = "checkExternalAggPartitionBits", fuzzy = true)
 public int externalAggPartitionBits = 5; // means that the hash table will 
be partitioned into 32 blocks.
 
-@VariableMgr.VarAttr(name = USE_MAX_LENGTH_OF_VARCHAR_IN_CTAS, description 
= {
+@VariableMgr.VarAttr(name = USE_MAX_LENGTH_OF_VARCHAR_IN_CTAS, needForward 
= true, description = {
 "在CTAS中，如果 CHAR / VARCHAR 列不来自于源表，是否是将这一列的长度设置为 MAX，即65533。默认为 
true。",
 "In CTAS (Create Table As Select), if CHAR/VARCHAR columns do not 
originate from the source table,"
 + " whether to set the length of such a column to MAX, 
which is 65533. The default is true."


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Fix](ctas) forward the use_max_length_of_varchar_in_ctas [doris]

[via GitHub]

morrySnow merged PR #39705:
URL: https://github.com/apache/doris/pull/39705


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Fix](Nereids) fix use cbo rule hint unused because of logic reverse [doris]

[via GitHub]

doris-robot commented on PR #39715:
URL: https://github.com/apache/doris/pull/39715#issuecomment-2303990182

   
   
   TPC-DS: Total hot run time: 190890 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 42a8389249bedc1ef2b3604e0a880b6c6878fd54, 
data reload: false
   
   query1   1270871 868 868
   query2   6292187119041871
   query3   10589   367739083677
   query4   59944   23864   23071   23071
   query5   5587518 512 512
   query6   427 167 168 167
   query7   6046298 293 293
   query8   292 227 225 225
   query9   8757245224412441
   query10  483 281 271 271
   query11  18430   15011   15171   15011
   query12  164 106 114 106
   query13  1570382 375 375
   query14  11141   747173717371
   query15  230 166 175 166
   query16  7531474 457 457
   query17  1151571 576 571
   query18  1846299 309 299
   query19  287 148 161 148
   query20  123 120 111 111
   query21  210 102 102 102
   query22  4742446745254467
   query23  34153   34106   33136   33136
   query24  5930282228762822
   query25  527 378 400 378
   query26  696 155 153 153
   query27  1805281 280 280
   query28  3723201820002000
   query29  667 409 425 409
   query30  236 152 147 147
   query31  930 770 731 731
   query32  75  55  57  55
   query33  452 288 281 281
   query34  870 483 477 477
   query35  844 754 727 727
   query36  1036917 899 899
   query37  146 83  83  83
   query38  3956382638513826
   query39  1457138713921387
   query40  191 115 118 115
   query41  47  45  46  45
   query42  118 98  102 98
   query43  507 472 469 469
   query44  1087746 758 746
   query45  198 166 162 162
   query46  1100759 775 759
   query47  1838177117921771
   query48  378 289 302 289
   query49  777 420 423 420
   query50  836 410 418 410
   query51  7231698072196980
   query52  99  87  89  87
   query53  258 181 179 179
   query54  572 446 444 444
   query55  76  77  77  77
   query56  291 264 256 256
   query57  1166108610781078
   query58  212 232 229 229
   query59  3045275429172754
   query60  293 278 310 278
   query61  111 98  99  98
   query62  769 643 674 643
   query63  214 186 182 182
   query64  3290174417781744
   query65  3242316731893167
   query66  668 332 326 326
   query67  15441   15159   15068   15068
   query68  3006608 572 572
   query69  394 313 307 307
   query70  1153110610171017
   query71  363 276 276 276
   query72  2496211821072107
   query73  718 326 331 326
   query74  9183883088248824
   query75  3384270326182618
   query76  15291117976 976
   query77  530 324 322 322
   query78  9884902289658965
   query79  1045540 538 538
   query80  725 508 530 508
   query81  459 235 221 221
   query82  291 150 135 135
   query83  173 146 148 146
   query84  257 74  79  74
   query85  684 289 285 285
   query86  306 305 290 290
   query87  4478428343054283
   query88  3137234523522345
   query89  377 286 292 286
   query90  1863197 196 196
   query91  122 98  99  98
   query92  59  51  54  51
   query93  1057533 544 533
   query94  620 294 313 294
   query95  330 269 268 268
   query96  593 268 275 268
   query97  3174315630823082
   query98  228 217 202 202
   query99  1540123212861232
   Total cold run time: 302828 ms
   Total hot run time: 190890 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to 

Re: [PR] [test](migrate) move mc cases from p2 to p0 [doris]

[via GitHub]

wsjz commented on PR #39706:
URL: https://github.com/apache/doris/pull/39706#issuecomment-2303993886

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Feature](Variant) Implement inner nested data type for variant type [doris]

[via GitHub]

eldenmoon commented on code in PR #39022:
URL: https://github.com/apache/doris/pull/39022#discussion_r1726522531


##
be/src/vec/json/path_in_data.cpp:
##
@@ -170,6 +178,33 @@ PathInData PathInData::copy_pop_front() const {
 return copy_pop_nfront(1);
 }
 
+PathInData PathInData::get_nested_prefix_path() const {
+CHECK(has_nested_part());
+PathInData new_path;
+Parts new_parts;
+for (const Part& part : parts) {
+new_parts.push_back(part);
+if (part.is_nested) {
+break;
+}
+}
+new_path.build_path(new_parts);

Review Comment:
   i think seperate them may improve performance, for example ,in copy 
constructor, we could copy directly from full other.path



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](inverted index) Fix match_regexp to correctly handle empty string patterns [doris]

[via GitHub]

qidaye merged PR #39503:
URL: https://github.com/apache/doris/pull/39503


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch master updated: [fix](inverted index) Fix match_regexp to correctly handle empty string patterns (#39503)

[jianliangqi]

This is an automated email from the ASF dual-hosted git repository.

jianliangqi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 43f00507d29 [fix](inverted index) Fix match_regexp to correctly handle 
empty string patterns (#39503)
43f00507d29 is described below

commit 43f00507d29233b00c9654bbdd3dc52916b830f6
Author: zzzxl <33418555+zzzxl1...@users.noreply.github.com>
AuthorDate: Thu Aug 22 15:42:10 2024 +0800

[fix](inverted index) Fix match_regexp to correctly handle empty string 
patterns (#39503)

1. Handle empty strings consistently for match_phrase with and without an 
index.
---
 be/src/vec/functions/match.cpp   | 9 -
 .../data/inverted_index_p0/test_index_match_regexp.out   | 3 +++
 regression-test/data/inverted_index_p0/test_no_index_match.out   | 3 +++
 .../suites/inverted_index_p0/test_index_match_regexp.groovy  | 1 +
 .../suites/inverted_index_p0/test_no_index_match.groovy  | 2 ++
 5 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/be/src/vec/functions/match.cpp b/be/src/vec/functions/match.cpp
index e21e467d096..a506dfac325 100644
--- a/be/src/vec/functions/match.cpp
+++ b/be/src/vec/functions/match.cpp
@@ -406,15 +406,6 @@ Status FunctionMatchRegexp::execute_match(FunctionContext* 
context, const std::s
 VLOG_DEBUG << "begin to run FunctionMatchRegexp::execute_match, 
parser_type: "
<< 
inverted_index_parser_type_to_string(inverted_index_ctx->parser_type);
 
-if (match_query_str.empty()) {
-VLOG_DEBUG << fmt::format(
-"token parser result is empty for query, "
-"please check your query: '{}' and index parser: '{}'",
-match_query_str,
-
inverted_index_parser_type_to_string(inverted_index_ctx->parser_type));
-return Status::OK();
-}
-
 const std::string& pattern = match_query_str;
 
 hs_database_t* database = nullptr;
diff --git a/regression-test/data/inverted_index_p0/test_index_match_regexp.out 
b/regression-test/data/inverted_index_p0/test_index_match_regexp.out
index f9a9caf6d74..fb5d23ad266 100644
--- a/regression-test/data/inverted_index_p0/test_index_match_regexp.out
+++ b/regression-test/data/inverted_index_p0/test_index_match_regexp.out
@@ -2,6 +2,9 @@
 -- !sql --
 1000
 
+-- !sql --
+1000
+
 -- !sql --
 54
 
diff --git a/regression-test/data/inverted_index_p0/test_no_index_match.out 
b/regression-test/data/inverted_index_p0/test_no_index_match.out
index 932dd55fc5a..ea3bd71bcd6 100644
--- a/regression-test/data/inverted_index_p0/test_no_index_match.out
+++ b/regression-test/data/inverted_index_p0/test_no_index_match.out
@@ -20,3 +20,6 @@
 -- !sql --
 0
 
+-- !sql --
+1000
+
diff --git 
a/regression-test/suites/inverted_index_p0/test_index_match_regexp.groovy 
b/regression-test/suites/inverted_index_p0/test_index_match_regexp.groovy
index 3458f086740..31863432707 100644
--- a/regression-test/suites/inverted_index_p0/test_index_match_regexp.groovy
+++ b/regression-test/suites/inverted_index_p0/test_index_match_regexp.groovy
@@ -80,6 +80,7 @@ suite("test_index_match_regexp", "p0"){
 
 sql "sync"
 
+qt_sql """ select count() from test_index_match_regexp where request 
match_regexp ''; """
 qt_sql """ select count() from test_index_match_regexp where request 
match_regexp '^h'; """
 qt_sql """ select count() from test_index_match_regexp where request 
match_regexp '^team'; """
 qt_sql """ select count() from test_index_match_regexp where request 
match_regexp 's\$'; """
diff --git 
a/regression-test/suites/inverted_index_p0/test_no_index_match.groovy 
b/regression-test/suites/inverted_index_p0/test_no_index_match.groovy
index 2df40235a3f..7947844565c 100644
--- a/regression-test/suites/inverted_index_p0/test_no_index_match.groovy
+++ b/regression-test/suites/inverted_index_p0/test_no_index_match.groovy
@@ -94,6 +94,8 @@ suite("test_no_index_match", "p0") {
 
   qt_sql """ select count() from ${testTable} where (request 
match_phrase '欧冶工业品');  """
   qt_sql """ select count() from ${testTable} where (request 
match_phrase_prefix '欧冶工业品');  """
+
+  qt_sql """ select count() from ${testTable} where (request 
match_regexp '');  """
   } finally {
   }
 


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](restore) Reset the db name of the materialized index stmt [doris]

[via GitHub]

doris-robot commented on PR #39710:
URL: https://github.com/apache/doris/pull/39710#issuecomment-2303996424

   
   
   TPC-H: Total hot run time: 38068 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 5ad70dac942d4b7cba365cd47f11cdd2b6e4ffe3, 
data reload: false
   
   -- Round 1 --
   q1   17618   435143584351
   q2   2017184 180 180
   q3   11778   965 1103965
   q4   10516   775 723 723
   q5   7770282728122812
   q6   227 138 138 138
   q7   995 621 612 612
   q8   9343207821192078
   q9   7060650565176505
   q10  7007234022452245
   q11  485 260 254 254
   q12  397 232 237 232
   q13  19023   307830553055
   q14  284 242 234 234
   q15  508 481 517 481
   q16  488 405 388 388
   q17  985 696 717 696
   q18  7226677068026770
   q19  14001058983 983
   q20  684 348 338 338
   q21  3885302731103027
   q22  1128104710011001
   Total cold run time: 110824 ms
   Total hot run time: 38068 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4356435543574355
   q2   371 277 269 269
   q3   2872270526812681
   q4   1910166715931593
   q5   5665568857955688
   q6   236 136 140 136
   q7   2224188518261826
   q8   3276344234963442
   q9   8928880888178808
   q10  3639337233513351
   q11  639 510 508 508
   q12  876 681 661 661
   q13  15900   317232333172
   q14  349 304 298 298
   q15  534 487 470 470
   q16  486 438 446 438
   q17  1835154115591541
   q18  8114787178257825
   q19  1742162915821582
   q20  2211191819131913
   q21  5571547554475447
   q22  1103104310501043
   Total cold run time: 72837 ms
   Total hot run time: 57047 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Feature](Variant) Implement inner nested data type for variant type [doris]

[via GitHub]

eldenmoon commented on code in PR #39022:
URL: https://github.com/apache/doris/pull/39022#discussion_r1726522531


##
be/src/vec/json/path_in_data.cpp:
##
@@ -170,6 +178,33 @@ PathInData PathInData::copy_pop_front() const {
 return copy_pop_nfront(1);
 }
 
+PathInData PathInData::get_nested_prefix_path() const {
+CHECK(has_nested_part());
+PathInData new_path;
+Parts new_parts;
+for (const Part& part : parts) {
+new_parts.push_back(part);
+if (part.is_nested) {
+break;
+}
+}
+new_path.build_path(new_parts);

Review Comment:
   i think seperate them may improve performance, for example ,in copy 
constructor, we could copy directly from other.path and avoid build the path, 
the copy constructor is frequent



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[PR] [fix](cloud) should do before commit check in cloud mode [doris]

[via GitHub]

sollhui opened a new pull request, #39775:
URL: https://github.com/apache/doris/pull/39775

   In cloud mode, there is no before commit check when commit transaction. For 
routine load, job may be paused during sub task execution. If these 
transactions are committed, the problem is:
   1. The progress seen by the user may not be accurate. It will consume data 
repeatedly if they create a new job consumption using this progress.
   2. The user's modification offset operation may be overwritten by the offset 
of the transaction callback.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](cloud) should do before commit check in cloud mode [doris]

[via GitHub]

doris-robot commented on PR #39775:
URL: https://github.com/apache/doris/pull/39775#issuecomment-2303997705

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
   
   Since 2024-03-18, the Document has been moved to 
[doris-website](https://github.com/apache/doris-website).
   See [Doris 
Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](cloud) should do before commit check in cloud mode [doris]

[via GitHub]

sollhui commented on PR #39775:
URL: https://github.com/apache/doris/pull/39775#issuecomment-2303997820

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [exec](pipeline) enable pipeline dml in coordinator [doris]

[via GitHub]

doris-robot commented on PR #39365:
URL: https://github.com/apache/doris/pull/39365#issuecomment-2303998402

   
   
   TPC-H: Total hot run time: 49756 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit f795a263b3704290bd259b59efd4df9f5812c32e, 
data reload: false
   
   -- Round 1 --
   q1   17790   436643804366
   q2   2061154 153 153
   q3   10253   189919411899
   q4   10366   127413331274
   q5   8521390439233904
   q6   239 124 126 124
   q7   2040165716211621
   q8   9278274727172717
   q9   10648   10447   10289   10289
   q10  8675350935063506
   q11  433 256 253 253
   q12  470 304 301 301
   q13  18344   394140363941
   q14  356 336 320 320
   q15  504 451 467 451
   q16  699 580 573 573
   q17  1134966 971 966
   q18  7340681968956819
   q19  1693158115181518
   q20  521 295 306 295
   q21  4434415640734073
   q22  500 399 393 393
   Total cold run time: 116299 ms
   Total hot run time: 49756 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4315429442634263
   q2   316 221 229 221
   q3   4160413041494130
   q4   2757273327312731
   q5   7191712671087108
   q6   237 117 123 117
   q7   3232285428392839
   q8   4361448344984483
   q9   16852   16748   16712   16712
   q10  4247425942804259
   q11  741 687 687 687
   q12  1024839 845 839
   q13  7045375137473747
   q14  450 429 427 427
   q15  503 455 450 450
   q16  753 675 682 675
   q17  3851388138553855
   q18  8690875388178753
   q19  1714168716631663
   q20  2397214821182118
   q21  8510838285178382
   q22  1013949 965 949
   Total cold run time: 84359 ms
   Total hot run time: 79408 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feature](doris compose) ls detail command print node's path [doris]

[via GitHub]

yujun777 commented on PR #39767:
URL: https://github.com/apache/doris/pull/39767#issuecomment-2303999669

   run cloud_p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Fix](Nereids) fix use cbo rule hint unused because of logic reverse [doris]

[via GitHub]

doris-robot commented on PR #39715:
URL: https://github.com/apache/doris/pull/39715#issuecomment-230473

   
   
   ClickBench: Total hot run time: 31.46 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 42a8389249bedc1ef2b3604e0a880b6c6878fd54, 
data reload: false
   
   query1   0.050.040.03
   query2   0.080.040.04
   query3   0.220.050.05
   query4   1.680.070.08
   query5   0.500.490.48
   query6   1.130.710.73
   query7   0.020.010.01
   query8   0.050.050.04
   query9   0.550.500.49
   query10  0.540.520.53
   query11  0.150.120.11
   query12  0.150.120.12
   query13  0.620.600.59
   query14  0.750.790.80
   query15  0.850.820.82
   query16  0.360.370.38
   query17  1.051.041.06
   query18  0.210.210.21
   query19  1.961.681.79
   query20  0.010.010.00
   query21  15.39   0.680.67
   query22  4.006.742.66
   query23  18.30   1.341.25
   query24  2.170.220.22
   query25  0.160.090.07
   query26  0.260.180.18
   query27  0.080.080.07
   query28  13.23   1.001.00
   query29  12.65   3.423.45
   query30  0.240.060.05
   query31  2.890.400.39
   query32  3.250.480.47
   query33  2.953.002.97
   query34  16.78   4.404.38
   query35  4.464.444.43
   query36  0.660.480.47
   query37  0.190.160.15
   query38  0.160.150.15
   query39  0.050.040.04
   query40  0.170.130.13
   query41  0.100.050.05
   query42  0.060.040.05
   query43  0.040.050.03
   Total cold run time: 109.17 s
   Total hot run time: 31.46 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](cloud) should do before commit check in cloud mode [doris]

[via GitHub]

sollhui commented on PR #39775:
URL: https://github.com/apache/doris/pull/39775#issuecomment-2304003738

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Feature](Variant) Implement inner nested data type for variant type [doris]

[via GitHub]

eldenmoon commented on code in PR #39022:
URL: https://github.com/apache/doris/pull/39022#discussion_r1726532366


##
be/src/vec/json/json_parser.cpp:
##
@@ -97,7 +98,7 @@ void JSONDataParser::traverse(const 
Element& element,
 } else if (element.isArray()) {
 has_nested = false;
 checkHasNested(element);

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Feature](Variant) Implement inner nested data type for variant type [doris]

[via GitHub]

eldenmoon commented on code in PR #39022:
URL: https://github.com/apache/doris/pull/39022#discussion_r1726531507


##
be/src/vec/json/json_parser.cpp:
##
@@ -71,14 +71,15 @@ bool JSONDataParser::extract_key(MutableColumns& colum
 }
 
 template 
-std::optional JSONDataParser::parse(const char* begin,
-   
size_t length) {
+std::optional JSONDataParser::parse(
+const char* begin, size_t length, const ParseConfig& config) {
 std::string_view json {begin, length};
 Element document;
 if (!parser.parse(json, document)) {

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Feature] support data mask in doris internal auth [doris]

[via GitHub]

qzsee commented on PR #38301:
URL: https://github.com/apache/doris/pull/38301#issuecomment-2304009090

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Feature](partial update) Support flexible partial update in stream load with json files [doris]

[via GitHub]

github-actions[bot] commented on code in PR #39756:
URL: https://github.com/apache/doris/pull/39756#discussion_r1726540021


##
be/src/olap/base_tablet.cpp:
##
@@ -983,12 +994,113 @@ Status BaseTablet::generate_new_block_for_partial_update(
 mutable_column->insert_default();
 }
 } else {
-mutable_column->insert_from(
-
*old_block.get_columns_with_type_and_name()[i].column.get(),
-read_index_old[idx]);
+
mutable_column->insert_from(*old_block.get_by_position(i).column,
+read_index_old[idx]);
+}
+}
+}
+output_block->set_columns(std::move(full_mutable_columns));
+VLOG_DEBUG << "full block when publish: " << output_block->dump_data();
+return Status::OK();
+}
+
+Status BaseTablet::generate_new_block_for_flexible_partial_update(

Review Comment:
   warning: function 'generate_new_block_for_flexible_partial_update' exceeds 
recommended size/complexity thresholds [readability-function-size]
   ```cpp
   Status BaseTablet::generate_new_block_for_flexible_partial_update(
  ^
   ```
   
   Additional context
   
   **be/src/olap/base_tablet.cpp:1006:** 96 lines including whitespace and 
comments (threshold 80)
   ```cpp
   Status BaseTablet::generate_new_block_for_flexible_partial_update(
  ^
   ```
   
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Feat](nereids) support pull up predicate from set operator [doris]

[via GitHub]

doris-robot commented on PR #39450:
URL: https://github.com/apache/doris/pull/39450#issuecomment-2304016652

   
   
   TPC-H: Total hot run time: 38114 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 3f868bbed66298666f502c4bc6d94bfa468ad2f1, 
data reload: false
   
   -- Round 1 --
   q1   17784   448644624462
   q2   2682186 191 186
   q3   10994   116911001100
   q4   10605   729 772 729
   q5   8105288328412841
   q6   227 148 145 145
   q7   992 626 613 613
   q8   9398206720452045
   q9   7198647865086478
   q10  6991219122022191
   q11  469 244 245 244
   q12  398 226 218 218
   q13  17767   301530393015
   q14  281 238 240 238
   q15  514 477 497 477
   q16  499 389 388 388
   q17  960 689 723 689
   q18  7412678368026783
   q19  14021014995 995
   q20  682 335 340 335
   q21  3908293030292930
   q22  1119102310121012
   Total cold run time: 110387 ms
   Total hot run time: 38114 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4394434143564341
   q2   387 279 275 275
   q3   2908267626622662
   q4   1958162216241622
   q5   5382535753825357
   q6   217 133 133 133
   q7   2097173017511730
   q8   3187332933573329
   q9   8331834483288328
   q10  3443323232013201
   q11  590 504 523 504
   q12  797 587 598 587
   q13  10563   302630553026
   q14  320 280 281 280
   q15  527 488 487 487
   q16  478 426 420 420
   q17  1765149914791479
   q18  7687760275947594
   q19  1682151015351510
   q20  2083183118171817
   q21  5474527952225222
   q22  11011047997 997
   Total cold run time: 65371 ms
   Total hot run time: 54901 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Feature](Variant) Implement inner nested data type for variant type [doris]

[via GitHub]

eldenmoon commented on PR #39022:
URL: https://github.com/apache/doris/pull/39022#issuecomment-2304017870

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) 01/01: [fix](downgrade) partition key should write upper case null type to image

[morrysnow]

This is an automated email from the ASF dual-hosted git repository.

morrysnow pushed a commit to branch fix_downgrade
in repository https://gitbox.apache.org/repos/asf/doris.git

commit b9672e84836085c94e8ba3d1e042ba7afd630da8
Author: morrySnow 
AuthorDate: Thu Aug 22 15:53:41 2024 +0800

[fix](downgrade) partition key should write upper case null type to image
---
 fe/fe-core/src/main/java/org/apache/doris/catalog/PartitionKey.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/catalog/PartitionKey.java 
b/fe/fe-core/src/main/java/org/apache/doris/catalog/PartitionKey.java
index 9f129235dfa..ac2beeadb85 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/PartitionKey.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/PartitionKey.java
@@ -376,7 +376,7 @@ public class PartitionKey implements 
Comparable, Writable {
 for (int i = 0; i < count; i++) {
 PrimitiveType type = types.get(i);
 if (keys.get(i).isNullLiteral()) {
-Text.writeString(out, Type.NULL.toString());
+Text.writeString(out, PrimitiveType.NULL_TYPE.toString());
 } else {
 Text.writeString(out, type.toString());
 }
@@ -396,11 +396,11 @@ public class PartitionKey implements 
Comparable, Writable {
 public void readFields(DataInput in) throws IOException {
 int count = in.readInt();
 for (int i = 0; i < count; i++) {
-PrimitiveType type = 
PrimitiveType.valueOf(Text.readString(in).toUpperCase());
+PrimitiveType type = PrimitiveType.valueOf(Text.readString(in));
 boolean isMax = in.readBoolean();
 if (type == PrimitiveType.NULL_TYPE) {
 String realType = StringLiteral.read(in).getStringValue();
-type = PrimitiveType.valueOf(realType.toUpperCase());
+type = PrimitiveType.valueOf(realType);
 types.add(type);
 keys.add(NullLiteral.create(Type.fromPrimitiveType(type)));
 continue;


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

(doris) branch fix_downgrade created (now b9672e84836)

[morrysnow]

This is an automated email from the ASF dual-hosted git repository.

morrysnow pushed a change to branch fix_downgrade
in repository https://gitbox.apache.org/repos/asf/doris.git


  at b9672e84836 [fix](downgrade) partition key should write upper case 
null type to image

This branch includes the following new commits:

 new b9672e84836 [fix](downgrade) partition key should write upper case 
null type to image

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](restore) Reset the db name of the materialized index stmt [doris]

[via GitHub]

doris-robot commented on PR #39710:
URL: https://github.com/apache/doris/pull/39710#issuecomment-2304018442

   
   
   TPC-DS: Total hot run time: 193081 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 5ad70dac942d4b7cba365cd47f11cdd2b6e4ffe3, 
data reload: false
   
   query1   1274882 856 856
   query2   6334191718591859
   query3   10559   392037853785
   query4   59467   26183   23327   23327
   query5   5445528 521 521
   query6   412 156 156 156
   query7   5848307 305 305
   query8   310 221 209 209
   query9   8972246524462446
   query10  495 278 266 266
   query11  17240   15057   15295   15057
   query12  152 101 115 101
   query13  1561408 397 397
   query14  11162   764775387538
   query15  229 174 165 165
   query16  7512485 505 485
   query17  1133593 571 571
   query18  1880303 298 298
   query19  305 152 149 149
   query20  115 112 130 112
   query21  207 102 103 102
   query22  4506454844394439
   query23  34345   33475   33630   33475
   query24  5896295928742874
   query25  546 405 399 399
   query26  695 161 165 161
   query27  1776287 287 287
   query28  3757203420362034
   query29  688 434 426 426
   query30  249 151 151 151
   query31  944 754 764 754
   query32  83  56  57  56
   query33  482 299 297 297
   query34  857 494 478 478
   query35  849 724 722 722
   query36  1058938 913 913
   query37  151 83  87  83
   query38  4014379738723797
   query39  1471140313961396
   query40  197 124 120 120
   query41  50  49  46  46
   query42  124 99  99  99
   query43  501 480 464 464
   query44  1113751 764 751
   query45  196 165 166 165
   query46  752 769 752
   query47  1855174717841747
   query48  400 308 314 308
   query49  780 460 479 460
   query50  827 426 439 426
   query51  7222692370706923
   query52  101 92  95  92
   query53  258 191 188 188
   query54  595 470 464 464
   query55  80  76  80  76
   query56  289 275 267 267
   query57  1192107210681068
   query58  319 233 243 233
   query59  2915287829262878
   query60  297 265 269 265
   query61  101 101 105 101
   query62  741 658 661 658
   query63  220 186 185 185
   query64  4228232017701770
   query65  3222314731553147
   query66  671 361 340 340
   query67  15331   15414   15125   15125
   query68  3990587 584 584
   query69  415 286 285 285
   query70  1113107111291071
   query71  372 279 276 276
   query72  6451231220862086
   query73  780 325 325 325
   query74  9076885187678767
   query75  3425271827822718
   query76  1804986 1014986
   query77  565 311 320 311
   query78  9692976410662   9764
   query79  1078553 546 546
   query80  874 517 511 511
   query81  565 223 225 223
   query82  289 138 150 138
   query83  229 154 152 152
   query84  258 83  78  78
   query85  819 295 288 288
   query86  292 291 295 291
   query87  4459422942834229
   query88  2973236224062362
   query89  381 294 295 294
   query90  1723204 200 200
   query91  122 105 98  98
   query92  58  55  53  53
   query93  1085545 544 544
   query94  576 300 299 299
   query95  349 264 265 264
   query96  604 273 271 271
   query97  3245304730933047
   query98  232 205 203 203
   query99  1638129012521252
   Total cold run time: 307447 ms
   Total hot run time: 193081 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to

[PR] [fix](downgrade) partition key should write upper case null type to image [doris]

[via GitHub]

morrySnow opened a new pull request, #39776:
URL: https://github.com/apache/doris/pull/39776

   (no comment)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](downgrade) partition key should write upper case null type to image [doris]

[via GitHub]

morrySnow commented on PR #39776:
URL: https://github.com/apache/doris/pull/39776#issuecomment-2304019464

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [refactor](local exchange) Refactor logics [doris]

[via GitHub]

doris-robot commented on PR #39771:
URL: https://github.com/apache/doris/pull/39771#issuecomment-2304020395

   
   
   TPC-H: Total hot run time: 37928 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit f59b0b0fdda285cc3e50d3ded29abcf8331e5f48, 
data reload: false
   
   -- Round 1 --
   q1   17900   467243904390
   q2   2168175 179 175
   q3   10495   117610321032
   q4   10134   736 767 736
   q5   7742288028022802
   q6   222 136 140 136
   q7   1059613 610 610
   q8   9327207920332033
   q9   7247652665666526
   q10  6995220022662200
   q11  443 248 244 244
   q12  393 227 222 222
   q13  17940   304630803046
   q14  282 232 241 232
   q15  515 481 492 481
   q16  491 388 378 378
   q17  970 641 736 641
   q18  7533700167416741
   q19  13981033983 983
   q20  685 351 331 331
   q21  3810297129882971
   q22  1128101810441018
   Total cold run time: 108877 ms
   Total hot run time: 37928 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4359429742214221
   q2   377 276 272 272
   q3   2893270627222706
   q4   1967166316451645
   q5   5409540953795379
   q6   215 127 128 127
   q7   2134173117541731
   q8   3218335133433343
   q9   8440847384748473
   q10  3440321731813181
   q11  595 500 526 500
   q12  776 612 623 612
   q13  11909   304830543048
   q14  312 265 280 265
   q15  528 480 489 480
   q16  468 426 421 421
   q17  1807151815091509
   q18  7670741873877387
   q19  1657155415781554
   q20  2072180418341804
   q21  5575525853565258
   q22  1114108110411041
   Total cold run time: 66935 ms
   Total hot run time: 54957 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](window_funnel) fix problem when upgrading from 3.0 caused by behaviour change of window_funnel [doris]

[via GitHub]

doris-robot commented on PR #39739:
URL: https://github.com/apache/doris/pull/39739#issuecomment-2304020974

   
   
   TPC-H: Total hot run time: 37870 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 0d4a16082f966cdff172492f6eeddb1d794dc453, 
data reload: false
   
   -- Round 1 --
   q1   17623   449043024302
   q2   2020184 174 174
   q3   11680   944 1082944
   q4   10517   700 760 700
   q5   7764286028092809
   q6   229 135 135 135
   q7   980 609 597 597
   q8   9322208220762076
   q9   7196650965416509
   q10  7021217722352177
   q11  464 236 244 236
   q12  396 221 221 221
   q13  17767   303430123012
   q14  273 235 233 233
   q15  514 490 486 486
   q16  505 400 387 387
   q17  986 698 713 698
   q18  7436679367386738
   q19  1394104710281028
   q20  710 335 335 335
   q21  3808309130773077
   q22  11001032996 996
   Total cold run time: 109705 ms
   Total hot run time: 37870 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4395434043294329
   q2   384 259 281 259
   q3   2890263827062638
   q4   1921169916481648
   q5   5514564856835648
   q6   226 144 139 139
   q7   2237180018581800
   q8   3310337534343375
   q9   8759869887768698
   q10  3547341333833383
   q11  609 495 512 495
   q12  824 643 678 643
   q13  14196   327631143114
   q14  315 281 294 281
   q15  547 506 498 498
   q16  522 446 432 432
   q17  1838154015241524
   q18  8051774278807742
   q19  1729141014781410
   q20  2203190418931893
   q21  5630543353405340
   q22  1121109910531053
   Total cold run time: 70768 ms
   Total hot run time: 56342 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](downgrade) partition key should write upper case null type to image [doris]

[via GitHub]

doris-robot commented on PR #39776:
URL: https://github.com/apache/doris/pull/39776#issuecomment-2304019438

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
   
   Since 2024-03-18, the Document has been moved to 
[doris-website](https://github.com/apache/doris-website).
   See [Doris 
Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Feature] support data mask in doris internal auth [doris]

[via GitHub]

qzsee commented on PR #38301:
URL: https://github.com/apache/doris/pull/38301#issuecomment-2304019442

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [exec](pipeline) enable pipeline dml in coordinator [doris]

[via GitHub]

doris-robot commented on PR #39365:
URL: https://github.com/apache/doris/pull/39365#issuecomment-2304020957

   
   
   TPC-DS: Total hot run time: 201389 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit f795a263b3704290bd259b59efd4df9f5812c32e, 
data reload: false
   
   query1   930 419 382 382
   query2   6557208720432043
   query3   6921208 200 200
   query4   20439   18050   18102   18050
   query5   19742   651065346510
   query6   293 223 223 223
   query7   4164297 309 297
   query8   277 264 241 241
   query9   3123270226212621
   query10  410 319 296 296
   query11  11416   10664   10761   10664
   query12  128 80  73  73
   query13  5587664 639 639
   query14  18259   13239   13584   13239
   query15  357 226 228 226
   query16  6453301 266 266
   query17  14951482896 896
   query18  2247439 412 412
   query19  204 148 148 148
   query20  79  80  81  80
   query21  192 97  91  91
   query22  5447517551775175
   query23  32583   31899   31984   31899
   query24  6921647365206473
   query25  519 446 426 426
   query26  531 169 160 160
   query27  1832299 289 289
   query28  6127227422602260
   query29  2879285727242724
   query30  258 169 168 168
   query31  921 744 741 741
   query32  71  63  58  58
   query33  409 265 268 265
   query34  869 468 480 468
   query35  1123928 954 928
   query36  1624104011791040
   query37  91  60  63  60
   query38  3099289029012890
   query39  1365131313271313
   query40  201 98  99  98
   query41  40  36  35  35
   query42  92  92  87  87
   query43  615 564 562 562
   query44  1106714 719 714
   query45  238 230 231 230
   query46  1225937 977 937
   query47  2051189517191719
   query48  989 686 662 662
   query49  632 376 370 370
   query50  875 638 616 616
   query51  4826466446544654
   query52  92  86  87  86
   query53  458 330 327 327
   query54  2671249224722472
   query55  98  81  94  81
   query56  232 227 203 203
   query57  1267121610521052
   query58  211 218 198 198
   query59  3510334333303330
   query60  220 199 200 199
   query61  99  99  99  99
   query62  888 440 505 440
   query63  493 343 336 336
   query64  2429153814801480
   query65  3634361335793579
   query66  818 380 377 377
   query67  15243   15061   16726   15061
   query68  8905641 622 622
   query69  571 369 355 355
   query70  1538129415561294
   query71  414 297 303 297
   query72  6575344435103444
   query73  737 319 315 315
   query74  6422592759065906
   query75  5365370837123708
   query76  5308118512061185
   query77  903 262 261 261
   query78  12530   11350   11658   11350
   query79  8490633 657 633
   query80  1305406 398 398
   query81  490 234 230 230
   query82  1651102 96  96
   query83  183 136 132 132
   query84  256 77  71  71
   query85  876 318 318 318
   query86  335 304 317 304
   query87  3203303030503030
   query88  5057229423042294
   query89  378 288 286 286
   query90  1978206 210 206
   query91  159 120 132 120
   query92  59  53  58  53
   query93  5010588 584 584
   query94  712 204 204 204
   query95  1124106710741067
   query96  642 337 313 313
   query97  6425640063346334
   query98  193 174 168 168
   query99  2956961 873 873
   Total cold run time: 311945 ms
   Total hot run time: 201389 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go t

[I] [Bug] 过滤条件中varchar字段加引号与不加引号查询结果不一致 [doris]

[via GitHub]

yongjinhou opened a new issue, #39777:
URL: https://github.com/apache/doris/issues/39777

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no 
similar issues.
   
   
   ### Version
   
   2.0.12 & 2.0.13
   
   ### What's Wrong?
   
   过滤条件中varchar字段加引号与不加引号查询结果不一致，加引号后的查询结果是错误的。
   https://github.com/user-attachments/assets/ae9b7b79-546e-47db-b620-5b9743faf751";>
   
   
   ### What You Expected?
   
   过滤条件中varchar字段加引号与不加引号查询结果一致且结果正确
   
   ### How to Reproduce?
   
   1. 建表
   `CREATE TABLE `temp_table` (
 `event_day` date NOT NULL DEFAULT "-01-01" COMMENT '日期分区',
 `user_id` VARCHAR(200) NOT NULL DEFAULT "0" COMMENT '用户id',
 `source` VARCHAR(200) NOT NULL DEFAULT "yiyan" COMMENT 
'来源,yiyan,qianfan,dasou'
   ) ENGINE=OLAP
   DUPLICATE KEY(`event_day`, `user_id`)
   DISTRIBUTED BY HASH(`event_day`) BUCKETS 10
   PROPERTIES (
   "replication_allocation" = "tag.location.default: 3",
   "is_being_synced" = "false",
   "storage_medium" = "hdd",
   "storage_format" = "V2",
   "light_schema_change" = "true",
   "disable_auto_compaction" = "false",
   "enable_single_replica_compaction" = "false"
   );`
   2. 导入数据
   `insert into  temp_table 
values("2023-03-10","118454562","baidu"),("2023-03-10","1275155607","baidu"),("2023-03-10","133566964","baidu");`
   3. 查询
   select   user_id from   temp_table where NOT ( user_id BETWEEN 
118454562 AND 133566964   );
   select   user_id from   temp_table where NOT ( user_id BETWEEN 
"118454562" AND "133566964"   );
   
   ### Anything Else?
   
   无
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Feature](Variant) Implement inner nested data type for variant type [doris]

[via GitHub]

eldenmoon commented on PR #39022:
URL: https://github.com/apache/doris/pull/39022#issuecomment-2304022182

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](downgrade) partition key should write upper case null type to image [doris]

[via GitHub]

github-actions[bot] commented on PR #39776:
URL: https://github.com/apache/doris/pull/39776#issuecomment-2304023648

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](restore) Reset the db name of the materialized index stmt [doris]

[via GitHub]

doris-robot commented on PR #39710:
URL: https://github.com/apache/doris/pull/39710#issuecomment-2304028470

   
   
   ClickBench: Total hot run time: 31.27 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 5ad70dac942d4b7cba365cd47f11cdd2b6e4ffe3, 
data reload: false
   
   query1   0.050.040.04
   query2   0.080.040.04
   query3   0.230.050.06
   query4   1.660.090.08
   query5   0.500.490.49
   query6   1.140.750.74
   query7   0.020.020.01
   query8   0.050.040.05
   query9   0.550.470.49
   query10  0.540.540.54
   query11  0.160.110.12
   query12  0.150.120.13
   query13  0.620.590.59
   query14  0.770.790.78
   query15  0.880.830.83
   query16  0.370.380.38
   query17  1.051.051.07
   query18  0.230.200.20
   query19  1.981.741.78
   query20  0.010.010.01
   query21  15.39   0.680.66
   query22  4.487.412.13
   query23  18.28   1.341.34
   query24  2.090.240.22
   query25  0.150.080.08
   query26  0.300.180.17
   query27  0.080.080.08
   query28  13.17   1.051.01
   query29  12.61   3.433.44
   query30  0.250.060.05
   query31  2.870.410.40
   query32  3.240.480.48
   query33  2.993.013.05
   query34  16.84   4.384.39
   query35  4.414.454.46
   query36  0.660.470.48
   query37  0.200.160.16
   query38  0.150.140.15
   query39  0.050.040.04
   query40  0.150.120.13
   query41  0.100.050.05
   query42  0.060.050.05
   query43  0.050.040.04
   Total cold run time: 109.61 s
   Total hot run time: 31.27 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [exec](pipeline) enable pipeline dml in coordinator [doris]

[via GitHub]

doris-robot commented on PR #39365:
URL: https://github.com/apache/doris/pull/39365#issuecomment-2304030944

   
   
   ClickBench: Total hot run time: 31.19 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit f795a263b3704290bd259b59efd4df9f5812c32e, 
data reload: false
   
   query1   0.020.020.02
   query2   0.070.020.02
   query3   0.250.050.05
   query4   1.800.050.06
   query5   0.540.520.51
   query6   1.260.630.62
   query7   0.020.010.01
   query8   0.040.030.03
   query9   0.510.500.48
   query10  0.530.540.52
   query11  0.120.080.08
   query12  0.120.090.09
   query13  0.620.620.61
   query14  0.790.790.79
   query15  0.780.760.75
   query16  0.360.380.36
   query17  0.991.041.01
   query18  0.220.250.22
   query19  1.911.821.85
   query20  0.010.020.01
   query21  15.47   0.550.55
   query22  2.062.261.91
   query23  17.28   0.930.98
   query24  5.864.371.14
   query25  0.410.080.06
   query26  0.930.150.15
   query27  0.050.030.04
   query28  3.930.720.76
   query29  12.65   2.362.33
   query30  0.590.530.56
   query31  2.790.390.37
   query32  3.410.510.49
   query33  3.063.053.07
   query34  15.26   4.814.80
   query35  4.874.844.84
   query36  1.041.011.00
   query37  0.060.040.05
   query38  0.030.020.02
   query39  0.020.010.02
   query40  0.160.140.15
   query41  0.070.010.01
   query42  0.020.020.01
   query43  0.020.010.02
   Total cold run time: 101 s
   Total hot run time: 31.19 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [chore](errmsg) Add some partition expr check on creating table [doris]

[via GitHub]

zclllyybb commented on PR #39754:
URL: https://github.com/apache/doris/pull/39754#issuecomment-2304031025

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](Outfile) fix bug that it will core dump if the _schema fails to build in the open phase [doris]

[via GitHub]

github-actions[bot] commented on PR #39770:
URL: https://github.com/apache/doris/pull/39770#issuecomment-2304033113

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](Outfile) fix bug that it will core dump if the _schema fails to build in the open phase [doris]

[via GitHub]

github-actions[bot] commented on PR #39770:
URL: https://github.com/apache/doris/pull/39770#issuecomment-2304033159

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [opt](nereids)adjust aggregation cost to make it compatible to the cost of join and shuffle [doris]

[via GitHub]

doris-robot commented on PR #39175:
URL: https://github.com/apache/doris/pull/39175#issuecomment-2304033587

   
   
   TPC-H: Total hot run time: 37620 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 3cb030b151549eaca5552c01875c1cd4328c446a, 
data reload: false
   
   -- Round 1 --
   q1   17604   446042694269
   q2   2018187 168 168
   q3   11966   921 1108921
   q4   10512   710 721 710
   q5   7753281528012801
   q6   222 136 138 136
   q7   983 616 585 585
   q8   9332205120482048
   q9   7263653965726539
   q10  6999224821392139
   q11  475 243 242 242
   q12  394 220 230 220
   q13  17757   303430343034
   q14  275 234 230 230
   q15  525 481 510 481
   q16  491 396 388 388
   q17  973 683 751 683
   q18  7332684467656765
   q19  1384103610801036
   q20  675 326 328 326
   q21  4047311528812881
   q22  1107103710181018
   Total cold run time: 110087 ms
   Total hot run time: 37620 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4353428943024289
   q2   368 264 263 263
   q3   2866268726302630
   q4   1920169516771677
   q5   5539567157075671
   q6   244 144 138 138
   q7   2231183918531839
   q8   3348340334413403
   q9   8839881988368819
   q10  3584342933853385
   q11  605 511 532 511
   q12  830 683 667 667
   q13  14996   313132033131
   q14  323 297 316 297
   q15  538 510 485 485
   q16  489 422 457 422
   q17  1818153715211521
   q18  8279777078197770
   q19  1766152516491525
   q20  2152191318741874
   q21  5695546054915460
   q22  1145105310441044
   Total cold run time: 71928 ms
   Total hot run time: 56821 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Feat](nereids) support pull up predicate from set operator [doris]

[via GitHub]

doris-robot commented on PR #39450:
URL: https://github.com/apache/doris/pull/39450#issuecomment-2304038068

   
   
   TPC-DS: Total hot run time: 186773 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 3f868bbed66298666f502c4bc6d94bfa468ad2f1, 
data reload: false
   
   query1   912 390 384 384
   query2   6456192519531925
   query3   6644216 210 210
   query4   31924   23246   23000   23000
   query5   4170500 485 485
   query6   262 168 172 168
   query7   4592297 296 296
   query8   272 237 221 221
   query9   8427243624312431
   query10  442 273 256 256
   query11  17911   14921   15227   14921
   query12  150 102 97  97
   query13  1634391 375 375
   query14  9489750970587058
   query15  267 171 172 171
   query16  7805420 441 420
   query17  1580563 550 550
   query18  1814286 283 283
   query19  257 143 143 143
   query20  118 110 108 108
   query21  211 104 100 100
   query22  4395414740154015
   query23  34268   33440   33652   33440
   query24  11199   287328982873
   query25  622 390 391 390
   query26  1175161 159 159
   query27  2365295 290 290
   query28  7013201620182016
   query29  783 403 400 400
   query30  310 160 151 151
   query31  1012755 784 755
   query32  100 57  61  57
   query33  803 310 290 290
   query34  950 460 478 460
   query35  874 724 706 706
   query36  1101919 925 919
   query37  158 86  81  81
   query38  3926383739653837
   query39  1444138714401387
   query40  200 114 117 114
   query41  49  50  47  47
   query42  120 99  99  99
   query43  500 464 473 464
   query44  1224752 753 752
   query45  197 169 167 167
   query46  1102776 746 746
   query47  1889176917991769
   query48  382 294 300 294
   query49  1077429 464 429
   query50  835 422 422 422
   query51  7198711670817081
   query52  100 89  87  87
   query53  255 186 178 178
   query54  954 480 459 459
   query55  82  74  77  74
   query56  284 257 252 252
   query57  1161109311121093
   query58  240 215 243 215
   query59  3125284227092709
   query60  288 272 275 272
   query61  105 100 103 100
   query62  905 653 667 653
   query63  223 190 188 188
   query64  5432228917691769
   query65  3347319932023199
   query66  1366342 363 342
   query67  15436   15361   15235   15235
   query68  3873578 585 578
   query69  405 317 282 282
   query70  1193113211231123
   query71  342 271 279 271
   query72  6317234221202120
   query73  759 322 324 322
   query74  9077878388248783
   query75  3399273726702670
   query76  2341999 980 980
   query77  490 309 315 309
   query78  10492   913290029002
   query79  1044545 538 538
   query80  743 524 514 514
   query81  464 228 227 227
   query82  232 136 142 136
   query83  176 154 155 154
   query84  235 84  85  84
   query85  693 286 400 286
   query86  317 287 296 287
   query87  4338420242494202
   query88  2991237023682368
   query89  381 292 289 289
   query90  1860207 200 200
   query91  129 100 103 100
   query92  63  53  53  53
   query93  1035541 531 531
   query94  841 294 284 284
   query95  356 265 260 260
   query96  596 279 270 270
   query97  3199309330623062
   query98  224 201 201 201
   query99  1457125012861250
   Total cold run time: 285973 ms
   Total hot run time: 186773 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to 

Re: [PR] [fix](cloud) should do before commit check in cloud mode [doris]

[via GitHub]

doris-robot commented on PR #39775:
URL: https://github.com/apache/doris/pull/39775#issuecomment-2304038347

   
   
   TPC-H: Total hot run time: 38199 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 7da17d95960525d681d666dc35f1530c75c0b1e1, 
data reload: false
   
   -- Round 1 --
   q1   17783   470143274327
   q2   2964192 174 174
   q3   11705   110211551102
   q4   10440   783 783 783
   q5   8048292428452845
   q6   223 142 139 139
   q7   986 622 614 614
   q8   9555205420712054
   q9   7127654065206520
   q10  7002224422262226
   q11  480 242 244 242
   q12  398 225 220 220
   q13  17762   303130163016
   q14  282 242 250 242
   q15  520 487 481 481
   q16  481 402 387 387
   q17  968 668 671 668
   q18  7272689767646764
   q19  1390102610591026
   q20  684 337 338 337
   q21  3810302530313025
   q22  1122100710111007
   Total cold run time: 111002 ms
   Total hot run time: 38199 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4323429642424242
   q2   388 259 277 259
   q3   2911266027002660
   q4   1907163816351635
   q5   5382539254055392
   q6   219 130 130 130
   q7   2074174517481745
   q8   3195331533173315
   q9   8447844884328432
   q10  3470317831633163
   q11  607 498 515 498
   q12  787 611 597 597
   q13  11274   301930163016
   q14  297 281 279 279
   q15  531 473 472 472
   q16  488 439 424 424
   q17  1808150214621462
   q18  7747735574647355
   q19  1676166916181618
   q20  2043180818441808
   q21  5458524051515151
   q22  1107101510341015
   Total cold run time: 66139 ms
   Total hot run time: 54668 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [exec](pipeline) enable pipeline dml in coordinator [doris]

[via GitHub]

doris-robot commented on PR #39365:
URL: https://github.com/apache/doris/pull/39365#issuecomment-2304036944

   
   Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Load test result on commit f795a263b3704290bd259b59efd4df9f5812c32e with 
default session variables
   Stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
   Stream load orc:  58 seconds loaded 1101869774 Bytes, about 18 MB/s
   Stream load parquet:  31 seconds loaded 861443392 Bytes, about 26 MB/s
   Insert into select:   22.0 seconds inserted 1000 Rows, about 454K 
ops/s
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [refactor](local exchange) Refactor logics [doris]

[via GitHub]

doris-robot commented on PR #39771:
URL: https://github.com/apache/doris/pull/39771#issuecomment-2304042122

   
   
   TPC-DS: Total hot run time: 186590 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit f59b0b0fdda285cc3e50d3ded29abcf8331e5f48, 
data reload: false
   
   query1   893 364 376 364
   query2   6451188219071882
   query3   6651210 221 210
   query4   31412   23254   23145   23145
   query5   4181499 473 473
   query6   256 161 167 161
   query7   4582310 305 305
   query8   266 213 213 213
   query9   8522246924552455
   query10  458 271 271 271
   query11  17609   15039   14923   14923
   query12  161 103 98  98
   query13  1644375 386 375
   query14  10131   697075136970
   query15  261 169 170 169
   query16  8017459 492 459
   query17  1622595 537 537
   query18  2057281 279 279
   query19  330 161 141 141
   query20  118 107 111 107
   query21  211 102 101 101
   query22  4594403141134031
   query23  34107   33367   33677   33367
   query24  11096   285928422842
   query25  617 379 395 379
   query26  1134156 157 156
   query27  2319281 278 278
   query28  7265204220262026
   query29  795 412 426 412
   query30  302 158 151 151
   query31  991 792 742 742
   query32  101 55  59  55
   query33  762 289 289 289
   query34  973 479 487 479
   query35  878 719 744 719
   query36  1099908 953 908
   query37  153 90  87  87
   query38  3949386438603860
   query39  1419137613801376
   query40  197 117 114 114
   query41  45  42  44  42
   query42  116 96  99  96
   query43  495 467 459 459
   query44  1298744 767 744
   query45  197 168 169 168
   query46  1097761 745 745
   query47  1860178217891782
   query48  372 301 301 301
   query49  1031431 418 418
   query50  828 417 419 417
   query51  7267711971447119
   query52  102 87  85  85
   query53  250 221 178 178
   query54  991 464 474 464
   query55  77  75  78  75
   query56  277 254 247 247
   query57  1176105710521052
   query58  242 228 239 228
   query59  3010283326692669
   query60  299 264 284 264
   query61  103 97  98  97
   query62  834 667 666 666
   query63  214 183 187 183
   query64  5304229017031703
   query65  3260320531643164
   query66  1364339 333 333
   query67  15680   15303   15302   15302
   query68  3330589 584 584
   query69  408 282 282 282
   query70  1142112011381120
   query71  343 271 273 271
   query72  6261226220662066
   query73  751 328 326 326
   query74  9137873686708670
   query75  3393265627262656
   query76  1854100610061006
   query77  486 396 326 326
   query78  9990902191549021
   query79  1042542 544 542
   query80  688 491 495 491
   query81  453 228 234 228
   query82  238 137 140 137
   query83  174 152 157 152
   query84  230 84  77  77
   query85  667 285 279 279
   query86  321 307 290 290
   query87  4365425043194250
   query88  2958237623712371
   query89  380 285 290 285
   query90  1855208 202 202
   query91  124 161 98  98
   query92  63  51  53  51
   query93  1028544 540 540
   query94  763 292 303 292
   query95  366 272 269 269
   query96  598 274 274 274
   query97  3172307130553055
   query98  218 206 197 197
   query99  1479129312721272
   Total cold run time: 284650 ms
   Total hot run time: 186590 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to t

Re: [PR] [fix](window_funnel) fix problem when upgrading from 3.0 caused by behaviour change of window_funnel [doris]

[via GitHub]

doris-robot commented on PR #39739:
URL: https://github.com/apache/doris/pull/39739#issuecomment-2304043592

   
   
   TPC-DS: Total hot run time: 190854 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 0d4a16082f966cdff172492f6eeddb1d794dc453, 
data reload: false
   
   query1   1242872 847 847
   query2   6163182818591828
   query3   10601   389541213895
   query4   59905   26182   23150   23150
   query5   5464493 493 493
   query6   424 165 157 157
   query7   5790306 293 293
   query8   298 212 213 212
   query9   9194245724402440
   query10  505 293 263 263
   query11  17803   14904   15188   14904
   query12  157 105 101 101
   query13  1561398 377 377
   query14  10885   725375777253
   query15  219 172 169 169
   query16  7399481 476 476
   query17  1128603 559 559
   query18  1390286 291 286
   query19  291 148 146 146
   query20  121 119 125 119
   query21  208 101 101 101
   query22  4722436445734364
   query23  34260   33381   33205   33205
   query24  5938282928092809
   query25  517 373 380 373
   query26  680 159 156 156
   query27  1791271 279 271
   query28  3723202319981998
   query29  635 425 431 425
   query30  240 159 153 153
   query31  897 774 770 770
   query32  78  56  58  56
   query33  452 290 290 290
   query34  877 473 471 471
   query35  803 740 710 710
   query36  1055900 906 900
   query37  138 85  83  83
   query38  4050375237793752
   query39  1447136113871361
   query40  197 119 118 118
   query41  49  48  45  45
   query42  118 97  94  94
   query43  508 448 460 448
   query44  1087741 742 741
   query45  197 167 168 167
   query46  1094747 730 730
   query47  1875183517911791
   query48  368 305 345 305
   query49  768 422 427 422
   query50  803 408 417 408
   query51  7097707470597059
   query52  96  87  86  86
   query53  250 187 178 178
   query54  564 445 462 445
   query55  76  74  80  74
   query56  291 257 255 255
   query57  1184109610901090
   query58  219 219 237 219
   query59  2925284426852685
   query60  297 263 265 263
   query61  100 130 96  96
   query62  744 646 645 645
   query63  213 180 179 179
   query64  3254178217521752
   query65  3220316331543154
   query66  660 331 335 331
   query67  15482   15270   15251   15251
   query68  3013581 583 581
   query69  392 288 285 285
   query70  1070108610931086
   query71  339 273 274 273
   query72  2566205821062058
   query73  718 321 319 319
   query74  9188880688518806
   query75  3385269326362636
   query76  1458920 991 920
   query77  541 301 313 301
   query78  10317   943390469046
   query79  1037531 519 519
   query80  681 516 496 496
   query81  467 230 233 230
   query82  288 137 138 137
   query83  169 145 152 145
   query84  261 74  84  74
   query85  703 287 288 287
   query86  300 281 302 281
   query87  4367433342884288
   query88  3044235223412341
   query89  373 284 278 278
   query90  1882193 194 193
   query91  134 134 113 113
   query92  63  54  54  54
   query93  1041533 531 531
   query94  687 308 315 308
   query95  339 341 262 262
   query96  646 271 264 264
   query97  3187308630513051
   query98  210 203 193 193
   query99  1521126812511251
   Total cold run time: 301174 ms
   Total hot run time: 190854 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to

Re: [PR] [Fix](group commit) Fix table not found fault when disable group commit [doris]

[via GitHub]

Yukang-Lian commented on PR #39731:
URL: https://github.com/apache/doris/pull/39731#issuecomment-2304044776

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](nereids)filter estimation for slot=unknown [doris]

[via GitHub]

xzj7019 commented on code in PR #39592:
URL: https://github.com/apache/doris/pull/39592#discussion_r1726567923


##
fe/fe-core/src/main/java/org/apache/doris/nereids/stats/FilterEstimation.java:
##
@@ -331,14 +331,28 @@ private Statistics estimateEqualTo(ComparisonPredicate 
cp, ColumnStatistic stats
 ColumnStatistic statsForRight,
 EstimationContext context) {
 double selectivity;
-double ndv = statsForLeft.ndv;
-double val = statsForRight.maxValue;
-if (val > statsForLeft.maxValue || val < statsForLeft.minValue) {
-selectivity = 0.0;
+if (statsForLeft.isUnKnown) {
+selectivity = DEFAULT_INEQUALITY_COEFFICIENT;
 } else {
-selectivity = StatsMathUtil.minNonNaN(1.0, 1.0 / ndv);
+double ndv = statsForLeft.ndv;
+if (statsForRight.isUnKnown) {
+if (ndv >= 1.0) {

Review Comment:
   what is the case where statsForLeft is not unknown but ndv < 1.0? Should we 
do a normalization if the ndv < 1.0?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Feat](nereids) support pull up predicate from set operator [doris]

[via GitHub]

doris-robot commented on PR #39450:
URL: https://github.com/apache/doris/pull/39450#issuecomment-2304048787

   
   
   ClickBench: Total hot run time: 31.78 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 3f868bbed66298666f502c4bc6d94bfa468ad2f1, 
data reload: false
   
   query1   0.040.050.04
   query2   0.070.040.04
   query3   0.230.050.05
   query4   1.670.070.07
   query5   0.490.500.50
   query6   1.120.730.73
   query7   0.020.010.01
   query8   0.060.050.04
   query9   0.550.490.47
   query10  0.550.530.54
   query11  0.150.120.11
   query12  0.150.120.12
   query13  0.620.600.60
   query14  0.780.810.78
   query15  0.860.810.82
   query16  0.350.380.38
   query17  0.971.000.98
   query18  0.220.210.20
   query19  1.981.861.80
   query20  0.020.010.01
   query21  15.40   0.680.66
   query22  4.046.672.63
   query23  18.24   1.391.44
   query24  2.100.220.23
   query25  0.160.080.09
   query26  0.280.180.18
   query27  0.080.080.08
   query28  13.26   1.021.01
   query29  12.59   3.493.45
   query30  0.250.070.05
   query31  2.870.400.40
   query32  3.230.480.47
   query33  2.973.023.06
   query34  17.17   4.374.37
   query35  4.414.454.44
   query36  0.660.460.49
   query37  0.180.160.15
   query38  0.160.150.16
   query39  0.050.040.04
   query40  0.150.120.13
   query41  0.090.050.06
   query42  0.060.060.05
   query43  0.050.050.04
   Total cold run time: 109.35 s
   Total hot run time: 31.78 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feature](cloud) Add lazy commit mechanism for `commit_txn` [doris]

[via GitHub]

SWJTU-ZhangLei commented on PR #38243:
URL: https://github.com/apache/doris/pull/38243#issuecomment-2304052045

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [feature](doris compose) ls detail command print node's path [doris]

[via GitHub]

yujun777 commented on PR #39767:
URL: https://github.com/apache/doris/pull/39767#issuecomment-2304054135

   run p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [refactor](local exchange) Refactor logics [doris]

[via GitHub]

doris-robot commented on PR #39771:
URL: https://github.com/apache/doris/pull/39771#issuecomment-2304052539

   
   
   ClickBench: Total hot run time: 31.03 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit f59b0b0fdda285cc3e50d3ded29abcf8331e5f48, 
data reload: false
   
   query1   0.050.050.04
   query2   0.080.040.04
   query3   0.230.050.05
   query4   1.680.080.08
   query5   0.500.480.48
   query6   1.130.720.72
   query7   0.010.020.01
   query8   0.050.050.05
   query9   0.560.500.48
   query10  0.540.560.54
   query11  0.160.110.12
   query12  0.150.120.13
   query13  0.610.580.59
   query14  0.770.780.78
   query15  0.840.830.82
   query16  0.380.380.37
   query17  1.061.041.05
   query18  0.210.210.20
   query19  1.901.771.80
   query20  0.010.020.01
   query21  15.40   0.680.68
   query22  4.396.192.15
   query23  18.35   1.361.29
   query24  2.110.240.22
   query25  0.150.080.07
   query26  0.280.180.17
   query27  0.080.080.07
   query28  13.22   1.051.00
   query29  12.62   3.343.33
   query30  0.240.060.05
   query31  2.890.410.39
   query32  3.240.480.47
   query33  2.972.973.00
   query34  17.12   4.374.40
   query35  4.434.424.48
   query36  0.650.480.48
   query37  0.180.170.16
   query38  0.150.160.15
   query39  0.050.030.05
   query40  0.160.130.12
   query41  0.100.050.05
   query42  0.070.060.05
   query43  0.050.050.05
   Total cold run time: 109.82 s
   Total hot run time: 31.03 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](nereids) refine window child's local shuffle dist-expr [doris]

[via GitHub]

github-actions[bot] commented on PR #39384:
URL: https://github.com/apache/doris/pull/39384#issuecomment-2304053765

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](window_funnel) fix problem when upgrading from 3.0 caused by behaviour change of window_funnel [doris]

[via GitHub]

doris-robot commented on PR #39739:
URL: https://github.com/apache/doris/pull/39739#issuecomment-2304054098

   
   
   ClickBench: Total hot run time: 30.37 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 0d4a16082f966cdff172492f6eeddb1d794dc453, 
data reload: false
   
   query1   0.050.040.04
   query2   0.080.040.04
   query3   0.220.040.05
   query4   1.680.080.07
   query5   0.490.480.53
   query6   1.130.730.72
   query7   0.020.010.02
   query8   0.050.040.04
   query9   0.550.470.47
   query10  0.550.550.54
   query11  0.150.120.11
   query12  0.160.120.13
   query13  0.610.600.59
   query14  0.750.810.78
   query15  0.840.830.82
   query16  0.360.370.37
   query17  1.041.031.00
   query18  0.210.200.20
   query19  1.851.781.78
   query20  0.020.010.01
   query21  15.41   0.670.67
   query22  4.298.061.49
   query23  18.24   1.391.20
   query24  2.140.220.22
   query25  0.160.080.07
   query26  0.260.180.17
   query27  0.080.080.08
   query28  13.23   1.011.01
   query29  12.65   3.483.43
   query30  0.240.050.06
   query31  2.900.390.38
   query32  3.240.490.47
   query33  2.962.983.01
   query34  16.96   4.414.37
   query35  4.434.484.48
   query36  0.650.500.49
   query37  0.180.160.15
   query38  0.160.160.15
   query39  0.040.030.04
   query40  0.160.130.12
   query41  0.090.040.04
   query42  0.060.050.05
   query43  0.040.050.04
   Total cold run time: 109.38 s
   Total hot run time: 30.37 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](nereids) refine window child's local shuffle dist-expr [doris]

[via GitHub]

github-actions[bot] commented on PR #39384:
URL: https://github.com/apache/doris/pull/39384#issuecomment-2304053814

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Feat](Nereids) add split_part function in fe constant folding [doris]

[via GitHub]

doris-robot commented on PR #39778:
URL: https://github.com/apache/doris/pull/39778#issuecomment-2304056076

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
   
   Since 2024-03-18, the Document has been moved to 
[doris-website](https://github.com/apache/doris-website).
   See [Doris 
Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [Feat](Nereids) add split_part function in fe constant folding [doris]

[via GitHub]

LiBinfeng-01 commented on PR #39778:
URL: https://github.com/apache/doris/pull/39778#issuecomment-2304056179

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [opt](nereids)adjust aggregation cost to make it compatible to the cost of join and shuffle [doris]

[via GitHub]

doris-robot commented on PR #39175:
URL: https://github.com/apache/doris/pull/39175#issuecomment-2304056826

   
   
   TPC-DS: Total hot run time: 192544 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 3cb030b151549eaca5552c01875c1cd4328c446a, 
data reload: false
   
   query1   1237870 862 862
   query2   6308196117641764
   query3   10746   417941074107
   query4   59657   26900   23276   23276
   query5   5427499 506 499
   query6   393 167 174 167
   query7   5765306 289 289
   query8   298 209 200 200
   query9   8928246624632463
   query10  468 282 262 262
   query11  17654   15045   15462   15045
   query12  154 101 112 101
   query13  1539410 382 382
   query14  10964   745671987198
   query15  225 170 184 170
   query16  7486485 441 441
   query17  1123565 559 559
   query18  1449305 298 298
   query19  279 156 152 152
   query20  117 116 114 114
   query21  208 105 104 104
   query22  4556443346424433
   query23  34439   33612   33345   33345
   query24  6015285229392852
   query25  551 399 396 396
   query26  685 164 168 164
   query27  1772278 285 278
   query28  3876203920012001
   query29  676 424 419 419
   query30  242 153 154 153
   query31  927 770 775 770
   query32  84  56  57  56
   query33  477 288 294 288
   query34  872 485 471 471
   query35  851 736 723 723
   query36  1060919 920 919
   query37  144 86  80  80
   query38  3932384338373837
   query39  1424138813871387
   query40  200 121 120 120
   query41  48  47  46  46
   query42  113 98  98  98
   query43  485 460 451 451
   query44  1102748 746 746
   query45  193 169 169 169
   query46  1082778 779 778
   query47  1839180517941794
   query48  375 301 300 300
   query49  772 435 439 435
   query50  824 428 415 415
   query51  7259703070767030
   query52  103 88  90  88
   query53  257 180 186 180
   query54  582 467 467 467
   query55  78  77  80  77
   query56  292 265 264 264
   query57  1184108710671067
   query58  231 242 230 230
   query59  2950275325662566
   query60  301 287 294 287
   query61  124 128 122 122
   query62  752 744 633 633
   query63  212 180 199 180
   query64  2813681 677 677
   query65  3243317831703170
   query66  678 341 354 341
   query67  15424   15192   15122   15122
   query68  4283569 556 556
   query69  406 272 272 272
   query70  111711321117
   query71  346 279 286 279
   query72  6553405640224022
   query73  756 326 335 326
   query74  9277878987978789
   query75  3376266526862665
   query76  1471103310131013
   query77  534 315 314 314
   query78  9806901390879013
   query79  1085543 531 531
   query80  879 514 500 500
   query81  494 232 224 224
   query82  1291141 131 131
   query83  173 146 149 146
   query84  267 78  77  77
   query85  865 293 297 293
   query86  328 301 296 296
   query87  4310439642824282
   query88  2967236523562356
   query89  403 284 286 284
   query90  1834196 196 196
   query91  124 98  101 98
   query92  67  50  54  50
   query93  1664536 538 536
   query94  742 296 304 296
   query95  351 263 262 262
   query96  593 273 274 273
   query97  3232308830283028
   query98  262 194 204 194
   query99  1543126412471247
   Total cold run time: 307917 ms
   Total hot run time: 192544 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to

[PR] [Feat](Nereids) add split_part function in fe constant folding [doris]

[via GitHub]

LiBinfeng-01 opened a new pull request, #39778:
URL: https://github.com/apache/doris/pull/39778

   split_part('2024-04-17,2024-04-17', ',', 1) ==> '2024-04-17' when fold 
constant by fe
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

Re: [PR] [fix](cloud) should do before commit check in cloud mode [doris]

[via GitHub]

doris-robot commented on PR #39775:
URL: https://github.com/apache/doris/pull/39775#issuecomment-2304060141

   
   
   TPC-DS: Total hot run time: 186940 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 7da17d95960525d681d666dc35f1530c75c0b1e1, 
data reload: false
   
   query1   908 362 359 359
   query2   6451193118861886
   query3   6660210 223 210
   query4   34090   23064   23095   23064
   query5   4203508 503 503
   query6   256 162 167 162
   query7   4577300 294 294
   query8   248 213 208 208
   query9   8656244624772446
   query10  462 275 280 275
   query11  15726   14956   15048   14956
   query12  153 100 99  99
   query13  1620392 373 373
   query14  9710685870946858
   query15  243 167 170 167
   query16  8101444 461 444
   query17  1591572 546 546
   query18  2118293 291 291
   query19  211 143 145 143
   query20  119 111 108 108
   query21  209 98  99  98
   query22  4472412041654120
   query23  34610   33651   33566   33566
   query24  11153   282628912826
   query25  562 399 379 379
   query26  1040160 160 160
   query27  2864279 279 279
   query28  7343202820112011
   query29  657 409 397 397
   query30  295 151 145 145
   query31  1008773 754 754
   query32  101 57  61  57
   query33  754 295 296 295
   query34  972 474 480 474
   query35  865 724 766 724
   query36  1097921 952 921
   query37  135 83  78  78
   query38  3944390138813881
   query39  1455139213721372
   query40  274 116 115 115
   query41  49  47  46  46
   query42  110 100 98  98
   query43  490 445 464 445
   query44  1206744 757 744
   query45  201 168 167 167
   query46  1101768 750 750
   query47  1891179817941794
   query48  365 303 293 293
   query49  1098419 415 415
   query50  829 411 418 411
   query51  7247713371217121
   query52  97  92  88  88
   query53  254 186 180 180
   query54  867 454 476 454
   query55  79  84  75  75
   query56  267 251 248 248
   query57  1187109610811081
   query58  266 233 222 222
   query59  3069269728002697
   query60  292 282 272 272
   query61  99  97  98  97
   query62  845 640 646 640
   query63  228 187 188 187
   query64  6192236118011801
   query65  3207316032043160
   query66  1351332 339 332
   query67  15565   15261   15255   15255
   query68  3557600 578 578
   query69  402 285 268 268
   query70  1130111511261115
   query71  353 284 283 283
   query72  6138231120772077
   query73  748 322 325 322
   query74  9224882687338733
   query75  3410269026892689
   query76  2526108410281028
   query77  491 342 315 315
   query78  9787906390729063
   query79  1020556 555 555
   query80  684 493 500 493
   query81  449 235 228 228
   query82  285 142 144 142
   query83  176 152 149 149
   query84  238 81  78  78
   query85  670 277 274 274
   query86  314 294 293 293
   query87  4419433142214221
   query88  3141235223492349
   query89  382 295 295 295
   query90  1942201 198 198
   query91  128 97  99  97
   query92  67  52  53  52
   query93  1050543 539 539
   query94  799 304 288 288
   query95  361 266 264 264
   query96  582 277 269 269
   query97  3189310831003100
   query98  216 212 201 201
   query99  1472128312671267
   Total cold run time: 287715 ms
   Total hot run time: 186940 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to th