[GitHub] [doris] Gabriel39 opened a new pull request, #13594: [chore](be version) Check BE version by script
Gabriel39 opened a new pull request, #13594: URL: https://github.com/apache/doris/pull/13594 # Proposed changes  expected:  ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13591: [Bug](decimal) Fix incorrect result for decimal multiply
github-actions[bot] commented on PR #13591: URL: https://github.com/apache/doris/pull/13591#issuecomment-1288528294 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13591: [Bug](decimal) Fix incorrect result for decimal multiply
github-actions[bot] commented on PR #13591: URL: https://github.com/apache/doris/pull/13591#issuecomment-1288528352 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] pingchunzhang commented on a diff in pull request #13578: [fix](regression) add 'if not exists' to 'create table' to support parallel test (#13576)
pingchunzhang commented on code in PR #13578: URL: https://github.com/apache/doris/pull/13578#discussion_r1002959665 ## regression-test/data/ddl_p0/test_ctl.out: ## @@ -1,4 +1,4 @@ -- This file is automatically generated. You should know what you did if you want to edit this -- !select -- -test_ctl1 CREATE TABLE `test_ctl1` (\n `test_varchar` varchar(150) NULL,\n `test_datetime` datetime NULL,\n `test_default_timestamp` datetime NULL DEFAULT CURRENT_TIMESTAMP\n) ENGINE=OLAP\nUNIQUE KEY(`test_varchar`)\nCOMMENT 'OLAP'\nDISTRIBUTED BY HASH(`test_varchar`) BUCKETS 3\nPROPERTIES (\n"replication_allocation" = "tag.location.default: 1",\n"in_memory" = "false",\n"storage_format" = "V2",\n"disable_auto_compaction" = "false"\n); +test_ctl1 CREATE TABLE IF NOT EXISTS `test_ctl1` (\n `test_varchar` varchar(150) NULL,\n `test_datetime` datetime NULL,\n `test_default_timestamp` datetime NULL DEFAULT CURRENT_TIMESTAMP\n) ENGINE=OLAP\nUNIQUE KEY(`test_varchar`)\nCOMMENT 'OLAP'\nDISTRIBUTED BY HASH(`test_varchar`) BUCKETS 3\nPROPERTIES (\n"replication_allocation" = "tag.location.default: 1",\n"in_memory" = "false",\n"storage_format" = "V2",\n"disable_auto_compaction" = "false"\n); Review Comment: even create table with “IF NOT EXISTS”，which will not be displayed when show create table， u should move “IF NOT EXISTS” ## regression-test/data/ddl_p0/test_ctas.out: ## @@ -1,6 +1,6 @@ -- This file is automatically generated. You should know what you did if you want to edit this -- !select -- -test_ctas1 CREATE TABLE `test_ctas1` (\n `test_varchar` varchar(150) NULL,\n `test_datetime` datetime NULL,\n `test_default_timestamp` datetime NULL DEFAULT CURRENT_TIMESTAMP\n) ENGINE=OLAP\nDUPLICATE KEY(`test_varchar`)\nCOMMENT 'OLAP'\nDISTRIBUTED BY HASH(`test_varchar`) BUCKETS 10\nPROPERTIES (\n"replication_allocation" = "tag.location.default: 1",\n"in_memory" = "false",\n"storage_format" = "V2",\n"disable_auto_compaction" = "false"\n); +test_ctas1 CREATE TABLE IF NOT EXISTS `test_ctas1` (\n `test_varchar` varchar(150) NULL,\n `test_datetime` datetime NULL,\n `test_default_timestamp` datetime NULL DEFAULT CURRENT_TIMESTAMP\n) ENGINE=OLAP\nDUPLICATE KEY(`test_varchar`)\nCOMMENT 'OLAP'\nDISTRIBUTED BY HASH(`test_varchar`) BUCKETS 10\nPROPERTIES (\n"replication_allocation" = "tag.location.default: 1",\n"in_memory" = "false",\n"storage_format" = "V2",\n"disable_auto_compaction" = "false"\n); Review Comment: even create table with “IF NOT EXISTS”，which will not be displayed when show create table， u should move “IF NOT EXISTS” ## regression-test/data/query/show/test_array_show_create.out: ## @@ -1,4 +1,4 @@ -- This file is automatically generated. You should know what you did if you want to edit this -- !select -- -test_array_show_create CREATE TABLE `test_array_show_create` (\n `k1` int(11) NULL,\n `k2` array NOT NULL,\n `k3` array NOT NULL,\n `k4` array NOT NULL,\n `k5` array NOT NULL,\n `k6` array NULL,\n `k7` array NOT NULL,\n `k8` array NOT NULL,\n `k9` array NOT NULL,\n `k10` array NOT NULL,\n `k11` array NULL\n) ENGINE=OLAP\nDUPLICATE KEY(`k1`)\nCOMMENT 'OLAP'\nDISTRIBUTED BY HASH(`k1`) BUCKETS 1\nPROPERTIES (\n"replication_allocation" = "tag.location.default: 1",\n"in_memory" = "false",\n"storage_format" = "V2",\n"disable_auto_compaction" = "false"\n); +test_array_show_create CREATE TABLE IF NOT EXISTS `test_array_show_create` (\n `k1` int(11) NULL,\n `k2` array NOT NULL,\n `k3` array NOT NULL,\n `k4` array NOT NULL,\n `k5` array NOT NULL,\n `k6` array NULL,\n `k7` array NOT NULL,\n `k8` array NOT NULL,\n `k9` array NOT NULL,\n `k10` array NOT NULL,\n `k11` array NULL\n) ENGINE=OLAP\nDUPLICATE KEY(`k1`)\nCOMMENT 'OLAP'\nDISTRIBUTED BY HASH(`k1`) BUCKETS 1\nPROPERTIES (\n"replication_allocation" = "tag.location.default: 1",\n"in_memory" = "false",\n"storage_format" = "V2",\n"disable_auto_compaction" = "false"\n); Review Comment: even create table with “IF NOT EXISTS”，which will not be displayed when show create table， u should move “IF NOT EXISTS” -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener opened a new pull request, #13595: [enhancement](Nereids): add PlanTree result for TPCH, SSB in UT.
jackwener opened a new pull request, #13595: URL: https://github.com/apache/doris/pull/13595 # Proposed changes Issue Number: close #xxx ## Problem summary add PlanTree result for TPCH, SSB in UT. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [x] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [x] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] adonis0147 commented on a diff in pull request #13594: [chore](be version) Check BE version by script
adonis0147 commented on code in PR #13594:
URL: https://github.com/apache/doris/pull/13594#discussion_r1002985380
##
bin/check_be_version.sh:
##
@@ -0,0 +1,93 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -eo pipefail
+
+curdir="$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
+
+if [[ "$(uname -s)" == 'Darwin' ]] && command -v brew &>/dev/null; then
+PATH="$(brew --repo)/opt/gnu-getopt/bin:${PATH}"
Review Comment:
```suggestion
PATH="$(brew --prefix)/opt/gnu-getopt/bin:${PATH}"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.2-unstable updated: [Bug](predicate) fix core dump on bool type runtime filter (#13417)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch branch-1.2-unstable
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-unstable by this
push:
new 6b27d29e68 [Bug](predicate) fix core dump on bool type runtime filter
(#13417)
6b27d29e68 is described below
commit 6b27d29e684b540e4284a942f6d16f894cc14d69
Author: Pxl
AuthorDate: Fri Oct 21 13:15:22 2022 +0800
[Bug](predicate) fix core dump on bool type runtime filter (#13417)
fix core dump on bool type runtime filter
---
be/src/exprs/runtime_filter.cpp | 2 +-
be/src/olap/in_list_predicate.h | 6 +-
be/src/olap/predicate_creator.h | 5 -
.../java/org/apache/doris/planner/RuntimeFilter.java | 6 +++---
.../data/query/operator/test_logical_operators.out| 19 +++
.../query/operator/test_logical_operators.groovy | 4 +++-
6 files changed, 31 insertions(+), 11 deletions(-)
diff --git a/be/src/exprs/runtime_filter.cpp b/be/src/exprs/runtime_filter.cpp
index 24e8e41a84..540e814633 100644
--- a/be/src/exprs/runtime_filter.cpp
+++ b/be/src/exprs/runtime_filter.cpp
@@ -1413,7 +1413,7 @@ void IRuntimeFilter::to_protobuf(PInFilter* filter) {
switch (column_type) {
case TYPE_BOOLEAN: {
-batch_copy(filter, it, [](PColumnValue* column, const
int32_t* value) {
+batch_copy(filter, it, [](PColumnValue* column, const bool*
value) {
column->set_boolval(*value);
});
return;
diff --git a/be/src/olap/in_list_predicate.h b/be/src/olap/in_list_predicate.h
index b0b9fc5330..3cd60c8f7e 100644
--- a/be/src/olap/in_list_predicate.h
+++ b/be/src/olap/in_list_predicate.h
@@ -27,6 +27,7 @@
#include "olap/column_predicate.h"
#include "olap/rowset/segment_v2/bloom_filter.h"
#include "olap/wrapper_field.h"
+#include "runtime/define_primitive_type.h"
#include "runtime/string_value.h"
#include "runtime/type_limit.h"
#include "uint24.h"
@@ -267,7 +268,10 @@ public:
private:
template
bool _operator(const LeftT& lhs, const RightT& rhs) const {
-if constexpr (PT == PredicateType::IN_LIST) {
+if constexpr (Type == TYPE_BOOLEAN) {
+DCHECK(_values.size() == 2);
+return PT == PredicateType::IN_LIST;
+} else if constexpr (PT == PredicateType::IN_LIST) {
return lhs != rhs;
}
return lhs == rhs;
diff --git a/be/src/olap/predicate_creator.h b/be/src/olap/predicate_creator.h
index ad1ed0d69e..0b1e1a4c14 100644
--- a/be/src/olap/predicate_creator.h
+++ b/be/src/olap/predicate_creator.h
@@ -24,7 +24,6 @@
#include "olap/in_list_predicate.h"
#include "olap/null_predicate.h"
#include "olap/tablet_schema.h"
-#include "runtime/type_limit.h"
#include "util/date_func.h"
#include "util/string_util.h"
@@ -236,10 +235,6 @@ template
inline ColumnPredicate* create_list_predicate(const TabletColumn& column, int
index,
const std::vector&
conditions,
bool opposite, MemPool* pool) {
-if (column.type() == OLAP_FIELD_TYPE_BOOL) {
-LOG(FATAL) << "Failed to create list preacate! input column type is
invalid";
-return nullptr;
-}
static_assert(PredicateTypeTraits::is_list(PT));
return create_predicate>(column, index,
conditions, opposite,
pool);
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/planner/RuntimeFilter.java
b/fe/fe-core/src/main/java/org/apache/doris/planner/RuntimeFilter.java
index 45a0c60724..c3fec43021 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/planner/RuntimeFilter.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/planner/RuntimeFilter.java
@@ -26,8 +26,8 @@ import org.apache.doris.analysis.SlotRef;
import org.apache.doris.analysis.TupleDescriptor;
import org.apache.doris.analysis.TupleId;
import org.apache.doris.analysis.TupleIsNullPredicate;
-import org.apache.doris.catalog.PrimitiveType;
import org.apache.doris.catalog.ScalarType;
+import org.apache.doris.catalog.Type;
import org.apache.doris.common.FeConstants;
import org.apache.doris.common.IdGenerator;
import org.apache.doris.qe.ConnectContext;
@@ -264,8 +264,8 @@ public final class RuntimeFilter {
TupleIsNullPredicate.unwrapExpr(normalizedJoinConjunct.getChild(0).clone());
Expr srcExpr = normalizedJoinConjunct.getChild(1);
-if (srcExpr.getType().equals(ScalarType.createHllType())
-||
srcExpr.getType().equals(ScalarType.createType(PrimitiveType.BITMAP))) {
+Type srcType = srcExpr.getType();
+if (srcType.equals(ScalarType.HLL) ||
srcType.equals(ScalarType.BITMAP) || srcType.equals(ScalarType.BOOLEAN)) {
return null;
}
diff --git a/r
[GitHub] [doris] jackwener closed pull request #13052: [test](Nereids)change runtime filter test's checker
jackwener closed pull request #13052: [test](Nereids)change runtime filter test's checker URL: https://github.com/apache/doris/pull/13052 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener commented on pull request #13052: [test](Nereids)change runtime filter test's checker
jackwener commented on PR #13052: URL: https://github.com/apache/doris/pull/13052#issuecomment-1288579433 redundant, close -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #13246: [Feature](runtime-filter) add runtime filter breaking change adapt
hello-stephen commented on PR #13246: URL: https://github.com/apache/doris/pull/13246#issuecomment-1288603751 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 38.23 seconds load time: 577 seconds storage size: 17154810742 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221024161348_clickbench_pr_33024.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yangzhg opened a new pull request, #13596: [Bugfix](manager) fix query profile key incompatible with old versions
yangzhg opened a new pull request, #13596: URL: https://github.com/apache/doris/pull/13596 # Proposed changes fix query profile key incompatible with old versions ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [x] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [x] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] adonis0147 commented on a diff in pull request #13147: [Enhancement][MultiTableMaterializedView] The schedule framework for the MTMV
adonis0147 commented on code in PR #13147:
URL: https://github.com/apache/doris/pull/13147#discussion_r1002953537
##
fe/fe-core/src/main/java/org/apache/doris/persist/EditLog.java:
##
@@ -884,6 +890,36 @@ public static void loadJournal(Env env, JournalEntity
journal) {
env.getLoadManager().replayCleanLabel(log);
break;
}
+case OperationType.OP_CREATE_MTMV_JOB: {
+final MtmvJob job = (MtmvJob) journal.getData();
+env.getMtmvJobManager().replayCreateJob(job);
+break;
+}
+case OperationType.OP_ALTER_MTMV_JOB: {
+ChangeMvmtJob changeJob = (ChangeMvmtJob)
journal.getData();
+env.getMtmvJobManager().replayUpdateJob(changeJob);
+break;
+}
+case OperationType.OP_DROP_MTMV_JOB: {
+DropMtmvJob dropJob = (DropMtmvJob) journal.getData();
+
env.getMtmvJobManager().replayDropJobs(dropJob.getJobIds());
+break;
+}
+case OperationType.OP_CREATE_MTMV_TASK: {
+final MtmvTask task = (MtmvTask) journal.getData();
Review Comment:
Why are some objects marked as `final` objects and others are not?
##
fe/fe-core/src/main/java/org/apache/doris/mtmv/MtmvTaskExecutor.java:
##
@@ -0,0 +1,179 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.mtmv;
+
+import org.apache.doris.analysis.UserIdentity;
+import org.apache.doris.common.Config;
+import org.apache.doris.mtmv.MtmvUtils.TaskState;
+import org.apache.doris.mtmv.metadata.MtmvJob;
+import org.apache.doris.mtmv.metadata.MtmvTask;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.QueryState;
+import org.apache.doris.thrift.TUniqueId;
+
+import com.google.common.collect.Maps;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.jetbrains.annotations.NotNull;
+
+import java.util.Map;
+import java.util.Objects;
+import java.util.UUID;
+import java.util.concurrent.Future;
+
+public class MtmvTaskExecutor implements Comparable {
+private static final Logger LOG =
LogManager.getLogger(MtmvTaskExecutor.class);
+
+private long jobId;
+
+private Map properties;
+
+private Future future;
+
+private MtmvJob job;
+
+private ConnectContext ctx;
+
+private MtmvTaskProcessor processor;
+
+private MtmvTask task;
+
+public long getJobId() {
+return jobId;
+}
+
+public void setJobId(long jobId) {
+this.jobId = jobId;
+}
+
+public MtmvJob getJob() {
+return job;
+}
+
+public void setJob(MtmvJob job) {
+this.job = job;
+}
+
+public Map getProperties() {
+return properties;
+}
+
+public void setProperties(Map properties) {
+this.properties = properties;
+}
+
+public Future getFuture() {
+return future;
+}
+
+public void setFuture(Future future) {
+this.future = future;
+}
+
+public MtmvTaskProcessor getProcessor() {
+return processor;
+}
+
+public void setProcessor(MtmvTaskProcessor processor) {
+this.processor = processor;
+}
+
+public boolean executeTask() throws Exception {
+MtmvTaskContext taskContext = new MtmvTaskContext();
+taskContext.setDefinition(task.getDefinition());
Review Comment:
```suggestion
taskContext.setQuery(task.getQuery());
```
##
fe/fe-core/src/main/java/org/apache/doris/mtmv/MtmvTaskContext.java:
##
@@ -0,0 +1,61 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LIC
[doris] branch master updated (40e122e5ef -> 7faad9f004)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 40e122e5ef [fix](join)the build and probe expr should be calculated before converting input block to nullable (#13436) add 7faad9f004 [FIX](agg)fix group by constant child expr bug (#13485) No new revisions were added by this update. Summary of changes: .../java/org/apache/doris/analysis/GroupByClause.java| 16 .../main/java/org/apache/doris/analysis/SelectStmt.java | 12 .../data/correctness_p0/test_group_by_constant.out | 3 +++ .../suites/correctness_p0/test_group_by_constant.groovy | 14 ++ 4 files changed, 33 insertions(+), 12 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #13485: [FIX](agg)fix group by constant child expr bug
yiguolei merged PR #13485: URL: https://github.com/apache/doris/pull/13485 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow opened a new pull request, #13597: [fix](planner) cannot recogonize column's table when analyze rewrite expr
morrySnow opened a new pull request, #13597: URL: https://github.com/apache/doris/pull/13597 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] ChPi opened a new issue, #13598: [Bug] json_object function return null on vectorized engine
ChPi opened a new issue, #13598:
URL: https://github.com/apache/doris/issues/13598
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and
found no similar issues.
### Version
1.1.3
### What's Wrong?
`json_object('k', k)` function return `NULL` if k is null on vectorized
engine,
it should return `{"k":null}` like doc example
```sql
MySQL> select json_object('username',null);
+-+
| json_object('username', 'NULL') |
+-+
| {"username": NULL} |
+-+
```
### What You Expected?
return `{"k":null}`
### How to Reproduce?
create
```sql
CREATE TABLE `test_query_json_object` (
`k1` int(11) NULL COMMENT "user id"
) ENGINE=OLAP
DUPLICATE KEY(`k1`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`k1`) BUCKETS 1
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"in_memory" = "false",
"storage_format" = "V2"
);
insert into test_query_json_object values(null);
set enable_vectorized_engine=true;
```
query
```sql
mysql> select json_object('k1',k1) from test_query_json_object;
+-+
| json_object('k1', `k1`) |
+-+
| NULL|
+-+
1 row in set (0.01 sec)
```
### Anything Else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener merged pull request #13505: [improve](Nereids): ReorderJoin eliminate this recursion
jackwener merged PR #13505: URL: https://github.com/apache/doris/pull/13505 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [improve](Nereids): ReorderJoin eliminate this recursion (#13505)
This is an automated email from the ASF dual-hosted git repository.
jakevin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 409bd76999 [improve](Nereids): ReorderJoin eliminate this recursion
(#13505)
409bd76999 is described below
commit 409bd76999236e9e624e6f2b1f3428191e86ea57
Author: jakevin
AuthorDate: Mon Oct 24 17:11:43 2022 +0800
[improve](Nereids): ReorderJoin eliminate this recursion (#13505)
---
.../nereids/rules/rewrite/logical/ReorderJoin.java | 82 --
.../rules/rewrite/logical/ReorderJoinTest.java | 33 -
.../doris/nereids/sqltest/MultiJoinTest.java | 39 +-
.../org/apache/doris/nereids/util/PlanChecker.java | 1 +
4 files changed, 112 insertions(+), 43 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/logical/ReorderJoin.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/logical/ReorderJoin.java
index 1cbdc370e2..c0c8622348 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/logical/ReorderJoin.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/logical/ReorderJoin.java
@@ -38,6 +38,7 @@ import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableList.Builder;
import com.google.common.collect.Lists;
+import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
@@ -79,8 +80,7 @@ public class ReorderJoin extends OneRewriteRuleFactory {
/**
* Recursively convert to
- * {@link LogicalJoin} or
- * {@link LogicalFilter}--{@link LogicalJoin}
+ * {@link LogicalJoin} or {@link LogicalFilter}--{@link LogicalJoin}
* --> {@link MultiJoin}
*/
public Plan joinToMultiJoin(Plan plan) {
@@ -182,20 +182,20 @@ public class ReorderJoin extends OneRewriteRuleFactory {
* A JOIN B RIGHT JOIN (C JOIN D) --> MJ(A, B, MJ([ROJ]C, D))
*
*
- *
* Graphic presentation:
+ *
* A JOIN B JOIN C LEFT JOIN D JOIN F
* left left│
* A B C D F ──► A B C │ D F ──► MJ(LOJ A,B,C,MJ(DF)
- *
+ *
* A JOIN B RIGHT JOIN C JOIN D JOIN F
* right │right
* A B C D F ──► A B │ C D F ──► MJ(A,B,MJ(ROJ C,D,F)
- *
+ *
* (A JOIN B JOIN C) FULL JOIN (D JOIN F)
* full│
* A B C D F ──► A B C │ D F──► MJ(FOJ MJ(A,B,C) MJ(D,F))
- *
+ *
*/
public Plan multiJoinToJoin(MultiJoin multiJoin) {
if (multiJoin.arity() == 1) {
@@ -272,24 +272,22 @@ public class ReorderJoin extends OneRewriteRuleFactory {
}
// following this multiJoin just contain INNER/CROSS.
-List joinFilter = multiJoinHandleChildren.getJoinFilter();
+Set joinFilter = new
HashSet<>(multiJoinHandleChildren.getJoinFilter());
Plan left = multiJoinHandleChildren.child(0);
-List candidates = multiJoinHandleChildren.children().subList(1,
multiJoinHandleChildren.arity());
-
-LogicalJoin join = findInnerJoin(left,
candidates, joinFilter);
-List newInputs = Lists.newArrayList();
-newInputs.add(join);
-newInputs.addAll(candidates.stream().filter(plan ->
!join.right().equals(plan)).collect(Collectors.toList()));
-
-joinFilter.removeAll(join.getHashJoinConjuncts());
-joinFilter.removeAll(join.getOtherJoinConjuncts());
-// TODO(wj): eliminate this recursion.
-return multiJoinToJoin(new MultiJoin(
-newInputs,
-joinFilter,
-JoinType.INNER_JOIN,
-ExpressionUtils.EMPTY_CONDITION));
+Set usedPlansIndex = new HashSet<>();
+usedPlansIndex.add(0);
+
+while (usedPlansIndex.size() !=
multiJoinHandleChildren.children().size()) {
+LogicalJoin join =
findInnerJoin(left, multiJoinHandleChildren.children(),
+joinFilter, usedPlansIndex);
+join.getHashJoinConjuncts().forEach(joinFilter::remove);
+join.getOtherJoinConjuncts().forEach(joinFilter::remove);
+
+left = join;
+}
+
+return PlanUtils.filterOrSelf(new ArrayList<>(joinFilter), left);
}
/**
@@ -319,9 +317,14 @@ public class ReorderJoin extends OneRewriteRuleFactory {
* @return InnerJoin or CrossJoin{left, last of [candidates]}
*/
private LogicalJoin findInnerJoin(Plan
left, List candidates,
-List joinFilter) {
+Set joinFilter, Set usedPlansIndex) {
+List otherJoinConditions = Lists.newArrayList();
Set leftOutputSet = left.getOutputSet();
for (int i = 0; i < candidates.size(); i++) {
+if (usedPlansIndex.contains(i)) {
+
[GitHub] [doris] nextdreamblue opened a new pull request, #13599: [fix](cast) fix cast date when push down predicate
nextdreamblue opened a new pull request, #13599: URL: https://github.com/apache/doris/pull/13599 Signed-off-by: nextdreamblue # Proposed changes Issue Number: close #13465 ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [x] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [] No - [x] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] nextdreamblue commented on issue #13465: [Bug] Cast as date query error
nextdreamblue commented on issue #13465: URL: https://github.com/apache/doris/issues/13465#issuecomment-1288713604 produce: MySQL [test]> CREATE TABLE `canal_test1` ( `a` int(11) NOT NULL COMMENT "", `b` datetime NOT NULL COMMENT "" ) ENGINE=OLAP UNIQUE KEY(`a`) COMMENT "OLAP" DISTRIBUTED BY HASH(`a`) BUCKETS 8 PROPERTIES ( "replication_allocation" = "tag.location.default: 1", "in_memory" = "false", "storage_format" = "V2" ); MySQL [test]> insert into canal_test1 values(1,'2022-08-29 17:19:08'); MySQL [test]> insert into canal_test1 values(2,'2022-08-30 17:19:08'); MySQL [test]> insert into canal_test1 values(3,'2022-08-31 17:19:08'); MySQL [test]> select b from canal_test1 where cast(b as date) in ( '2022-08-31'); Empty set (0.01 sec) MySQL [test]> select b from canal_test1 where cast(b as date) = '2022-08-31'; Empty set (0.01 sec) MySQL [test]> select b from canal_test1 where cast(b as date) bETWEEN '2022-01-01' and '2022-08-31'; +-+ | b | +-+ | 2022-08-30 17:19:08 | | 2022-08-29 17:19:08 | +-+ 2 rows in set (0.01 sec) 3个查询结果都不正确，应该都可以返回 2022-08-31 17:19:08 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly opened a new pull request, #13600: [fix](planner) inlineView alias error
englefly opened a new pull request, #13600: URL: https://github.com/apache/doris/pull/13600 # Proposed changes set alias for `inlineViewAnalyzaer`, and check alias name when `resolveColumnRef()` Issue Number: 13431 https://github.com/apache/doris/issues/13431 ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly commented on pull request #13600: [fix](planner) inlineView alias error
englefly commented on PR #13600: URL: https://github.com/apache/doris/pull/13600#issuecomment-1288725531 After fix this bug, we found that there is another bug hidden by this bug. the other bug is that planner push a join conjuncts down to scan node. That is why we modified the regression test temporarily. We will fix it latter. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wangshuo128 closed pull request #11825: [Feature](Nereids) Support join hint.
wangshuo128 closed pull request #11825: [Feature](Nereids) Support join hint. URL: https://github.com/apache/doris/pull/11825 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wangshuo128 opened a new pull request, #13601: [Feature](Nereids) Support join hint.
wangshuo128 opened a new pull request, #13601: URL: https://github.com/apache/doris/pull/13601 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly commented on a diff in pull request #13600: [fix](planner) inlineView alias error
englefly commented on code in PR #13600:
URL: https://github.com/apache/doris/pull/13600#discussion_r1003095676
##
regression-test/suites/correctness/test_pushdown_pred_to_view.groovy:
##
@@ -54,7 +54,7 @@ The same resolve error occurs when re-analyze v2.
"""
qt_sql """
- select * from ${viewName} as v1 join ${viewName} as v2 on v1.id=v2.id
and v1.id>0;
+ select * from ${viewName} as v1 join ${viewName} as v2 where
v1.id=v2.id and v1.id>0;
Review Comment:
After fix this bug, we found that there is another bug hidden by this bug.
If we use `on` instead of `where`, the planner push `v1.id=v2.id` down to
scan node.
That is why we modified the regression test temporarily.
We will fix it latter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13594: [chore](be version) Check BE version by script
github-actions[bot] commented on PR #13594:
URL: https://github.com/apache/doris/pull/13594#issuecomment-1288747739
`sh-checker report`
To get the full details, please check in the
[job]("https://github.com/apache/doris/actions/runs/3310673579";) output.
shellcheck errors
```
'shellcheck ' returned error 1 finding the following syntactical issues:
--
In bin/check_be_version.sh line 93:
${DORIS_HOME}/lib/doris_be --version
^---^ SC2086 (info): Double quote to prevent globbing and word
splitting.
Did you mean:
"${DORIS_HOME}"/lib/doris_be --version
For more information:
https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing
...
--
You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
# shellcheck disable=
above the line that contains the issue, where is the error code;
3. Add '-e ' to the SHELLCHECK_OPTS setting in your .yml action file.
```
shfmt errors
```
'shfmt ' returned error 1 finding the following formatting issues:
--
--- bin/check_be_version.sh.orig
+++ bin/check_be_version.sh
--
You can reformat the above files to meet shfmt's requirements by typing:
shfmt -w filename
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13594: [chore](be version) Check BE version by script
github-actions[bot] commented on PR #13594:
URL: https://github.com/apache/doris/pull/13594#issuecomment-1288763062
`sh-checker report`
To get the full details, please check in the
[job]("https://github.com/apache/doris/actions/runs/3310676638";) output.
shellcheck errors
```
'shellcheck ' returned error 1 finding the following syntactical issues:
--
In bin/check_be_version.sh line 93:
${DORIS_HOME}/lib/doris_be --version
^---^ SC2086 (info): Double quote to prevent globbing and word
splitting.
Did you mean:
"${DORIS_HOME}"/lib/doris_be --version
For more information:
https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing
...
--
You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
# shellcheck disable=
above the line that contains the issue, where is the error code;
3. Add '-e ' to the SHELLCHECK_OPTS setting in your .yml action file.
```
shfmt errors
```
'shfmt ' returned error 1 finding the following formatting issues:
--
--- bin/check_be_version.sh.orig
+++ bin/check_be_version.sh
--
You can reformat the above files to meet shfmt's requirements by typing:
shfmt -w filename
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataroaring opened a new pull request, #13602: [fix](test) let cases use their own table name
dataroaring opened a new pull request, #13602: URL: https://github.com/apache/doris/pull/13602 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13594: [chore](be version) Check BE version by script
github-actions[bot] commented on PR #13594:
URL: https://github.com/apache/doris/pull/13594#issuecomment-1288775144
`sh-checker report`
To get the full details, please check in the
[job]("https://github.com/apache/doris/actions/runs/3310702490";) output.
shellcheck errors
```
'shellcheck ' found no issues.
```
shfmt errors
```
'shfmt ' returned error 1 finding the following formatting issues:
--
--- bin/check_be_version.sh.orig
+++ bin/check_be_version.sh
--
You can reformat the above files to meet shfmt's requirements by typing:
shfmt -w filename
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] huangkaiyan10 opened a new issue, #13603: at flinksql job，flink-doris-connector fail to SerializedThrowable
huangkaiyan10 opened a new issue, #13603: URL: https://github.com/apache/doris/issues/13603 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version 1.1.3 ### What's Wrong? kafka-2-doris-test-3-checkpoint-rocksdb-taskmanager-1-1 @ 100.81.116.55 (dataPort=43943). org.apache.flink.util.SerializedThrowable: Task name with subtask : Source: KafkaSource-default_catalog.default_database.kafka_table_1 -> Sink Sink(table=[default_catalog.default_database.cdc_doris_sink], fields=[f_sequence, f_random, f_random_str]) (1/1)#2 Failure reason: Task has failed. at org.apache.flink.runtime.taskmanager.Task.declineCheckpoint(Task.java:1389) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.taskmanager.Task.lambda$triggerCheckpointBarrier$3(Task.java:1332) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:836) ~[?:1.8.0_345] at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:811) ~[?:1.8.0_345] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_345] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_345] at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$triggerCheckpointAsync$12(StreamTask.java:1131) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking(MailboxProcessor.java:353) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:317) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:201) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:809) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:761) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at java.lang.Thread.run(Thread.java:750) [?:1.8.0_345] Caused by: org.apache.flink.util.SerializedThrowable: Could not perform checkpoint 3 for operator Source: KafkaSource-default_catalog.default_database.kafka_table_1 -> Sink Sink(table=[default_catalog.default_database.cdc_doris_sink], fields=[f_sequence, f_random, f_random_str]) (1/1)#2. at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointAsyncInMailbox(StreamTask.java:1175) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$triggerCheckpointAsync$12(StreamTask.java:1122) ~[flink-dist_2.12-1.14.5.jar:1.14.5] ... 12 more Caused by: org.apache.flink.util.SerializedThrowable: Could not complete snapshot 3 for operator Source: KafkaSource-default_catalog.default_database.kafka_table_1 -> Sink Sink(table=[default_catalog.default_database.cdc_doris_sink], fields=[f_sequence, f_random, f_random_str]) (1/1)#2. Failure reason: Checkpoint was declined. at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:265) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:170) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:348) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.checkpointStreamOperator(RegularOperatorChain.java:233) ~[flink-dist_2.12-1.14.5.jar:1.14.5] at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.buildOperatorSnapshotFutures(RegularOperatorCh
[GitHub] [doris] github-actions[bot] commented on pull request #13593: [opt](planer) remove unless cast of avg function
github-actions[bot] commented on PR #13593: URL: https://github.com/apache/doris/pull/13593#issuecomment-1288819830 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13593: [opt](planer) remove unless cast of avg function
github-actions[bot] commented on PR #13593: URL: https://github.com/apache/doris/pull/13593#issuecomment-1288820069 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] nextdreamblue commented on issue #13573: [Bug] Inconsistent behavior for string comparison in FE and BE
nextdreamblue commented on issue #13573: URL: https://github.com/apache/doris/issues/13573#issuecomment-1288867509 be中，字符串的比较是基于utf8的编码，按字节比较，字节是基于uint8 fe中，java基于signed byte 导致be中fe中，编码值对应的数字值不同导致排序不同 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] nextdreamblue opened a new pull request, #13604: [fix][fe] Inconsistent behavior for string comparison in FE and BE
nextdreamblue opened a new pull request, #13604: URL: https://github.com/apache/doris/pull/13604 Signed-off-by: nextdreamblue # Proposed changes Issue Number: close #13573 ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [x] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [x] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [x] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13594: [chore](be version) Check BE version by script
github-actions[bot] commented on PR #13594:
URL: https://github.com/apache/doris/pull/13594#issuecomment-126730
`sh-checker report`
To get the full details, please check in the
[job]("https://github.com/apache/doris/actions/runs/3311020936";) output.
shellcheck errors
```
'shellcheck ' found no issues.
```
shfmt errors
```
'shfmt ' returned error 1 finding the following formatting issues:
--
--- bin/check_be_version.sh.orig
+++ bin/check_be_version.sh
--
You can reformat the above files to meet shfmt's requirements by typing:
shfmt -w filename
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13594: [chore](be version) Check BE version by script
github-actions[bot] commented on PR #13594: URL: https://github.com/apache/doris/pull/13594#issuecomment-1288896811 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13594: [chore](be version) Check BE version by script
github-actions[bot] commented on PR #13594: URL: https://github.com/apache/doris/pull/13594#issuecomment-1288896858 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataalive commented on issue #13570: [Bug] CREATE STORAGE POLICY 提示语法错误
dataalive commented on issue #13570: URL: https://github.com/apache/doris/issues/13570#issuecomment-1288899700 the document has been wrongly put in version 1.1 , we will fix it . the feature is supported after version 1.2 like @stalary has said -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataalive closed issue #13570: [Bug] CREATE STORAGE POLICY 提示语法错误
dataalive closed issue #13570: [Bug] CREATE STORAGE POLICY 提示语法错误 URL: https://github.com/apache/doris/issues/13570 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhengshiJ commented on a diff in pull request #12583: [feature](Nereids) implement grouping sets
zhengshiJ commented on code in PR #12583:
URL: https://github.com/apache/doris/pull/12583#discussion_r1003209193
##
fe/fe-core/src/main/java/org/apache/doris/nereids/properties/ChildOutputPropertyDeriver.java:
##
@@ -87,6 +89,18 @@ public PhysicalProperties
visitPhysicalAggregate(PhysicalAggregate groupBy, PlanContext context) {
Review Comment:
@morrySnow Please help to see if this is correct
##
fe/fe-core/src/main/java/org/apache/doris/nereids/properties/RequestPropertyDeriver.java:
##
@@ -113,6 +114,12 @@ public Void visitPhysicalAggregate(PhysicalAggregate agg, PlanCo
// TODO: add other phase logical when we support distinct aggregate
}
+@Override
+public Void visitPhysicalGroupBy(PhysicalGroupBy groupBy,
PlanContext context) {
Review Comment:
@morrySnow Please help to see if this is correct
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: update create table random bucket
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 4abeb82388c update create table random bucket 4abeb82388c is described below commit 4abeb82388c54f776f1a3257c7bdc0dd1afe70ed Author: jiafeng.zhang AuthorDate: Mon Oct 24 19:53:18 2022 +0800 update create table random bucket --- .../sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md index 988243a51c8..a8660daff9d 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md @@ -231,9 +231,9 @@ distribution_desc 定义数据分桶方式。 -`DISTRIBUTED BY HASH (k1[,k2 ...]) [BUCKETS num]` +1) Hash 分桶 语法： `DISTRIBUTED BY HASH (k1[,k2 ...]) [BUCKETS num]` 说明： 使用指定的 key 列进行哈希分桶。 - 建议: 当没有合适的key做哈希分桶使得表的数据均匀分布的时候，建议使用RANDOM分桶方式。 + 2. Random 分桶 语法： `DISTRIBUTED BY RANDOM [BUCKETS num]` 说明： 使用随机数进行分桶。 * `rollup_list` - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataalive closed issue #13461: 7.5 检查导入数据 执行下面的 SQL 语句检查导入的数据量上 上面的数据量是一致。文案错误
dataalive closed issue #13461: 7.5 检查导入数据 执行下面的 SQL 语句检查导入的数据量上 上面的数据量是一致。文案错误 URL: https://github.com/apache/doris/issues/13461 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataalive commented on issue #13411: [Feature] 运行一段时间的flink算子后，be就会宕机，谁能知道这个问题咋解决
dataalive commented on issue #13411: URL: https://github.com/apache/doris/issues/13411#issuecomment-1288929540 has Doris been upgraded to 1.1.3 ？ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #13416: [Feature](Nereids) Support materialized index selection.
morrySnow commented on code in PR #13416:
URL: https://github.com/apache/doris/pull/13416#discussion_r1003217319
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalOlapScan.java:
##
@@ -87,7 +87,7 @@ public LogicalOlapScan(RelationId id, Table table,
List qualifier,
}
this.partitionPruned = partitionPruned;
this.candidateIndexIds = candidateIndexIds;
-this.rollupSelected = rollupSelected;
+this.indexSelected = rollupSelected;
Review Comment:
```suggestion
this.indexSelected = indexSelected;
```
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/agg/Count.java:
##
@@ -30,7 +31,7 @@
import java.util.stream.Collectors;
/** count agg function. */
-public class Count extends AggregateFunction implements AlwaysNotNullable {
+public class Count extends AggregateFunction implements UnaryExpression,
AlwaysNotNullable {
Review Comment:
when we use count(*), the count has no child, so maybe UnaryExpression is
inappropriate
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/mv/SelectMaterializedIndexWithoutAggregate.java:
##
@@ -0,0 +1,147 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.mv;
+
+import org.apache.doris.catalog.MaterializedIndex;
+import org.apache.doris.catalog.OlapTable;
+import org.apache.doris.nereids.rules.Rule;
+import org.apache.doris.nereids.rules.RuleType;
+import org.apache.doris.nereids.rules.rewrite.RewriteRuleFactory;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.plans.PreAggStatus;
+import org.apache.doris.nereids.trees.plans.logical.LogicalFilter;
+import org.apache.doris.nereids.trees.plans.logical.LogicalOlapScan;
+import org.apache.doris.nereids.trees.plans.logical.LogicalProject;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableSet;
+
+import java.util.List;
+import java.util.Set;
+import java.util.function.Supplier;
+import java.util.stream.Collectors;
+
+/**
+ * Select materialized index, i.e., both for rollup and materialized view when
aggregate is not present.
+ *
+ * Scan OLAP table with aggregate is handled in {@link
SelectMaterializedIndexWithAggregate}.
+ *
+ * Note that we should first apply {@link
SelectMaterializedIndexWithAggregate} and then
+ * {@link SelectMaterializedIndexWithoutAggregate}.
+ * Besides, these two rules should run in isolated batches, thus when enter
this rule, it's guaranteed that there is
+ * no aggregation on top of the scan.
+ */
+public class SelectMaterializedIndexWithoutAggregate extends
AbstractSelectMaterializedIndexRule
+implements RewriteRuleFactory {
+
+@Override
+public List buildRules() {
+return ImmutableList.of(
+// project with pushdown filter.
+// Project(Filter(Scan))
+
logicalProject(logicalFilter(logicalOlapScan().whenNot(LogicalOlapScan::isIndexSelected)))
+.then(project -> {
+LogicalFilter filter =
project.child();
+LogicalOlapScan scan = filter.child();
+return project.withChildren(filter.withChildren(
+select(scan, project::getInputSlots,
filter::getConjuncts)));
+
}).toRule(RuleType.MATERIALIZED_INDEX_PROJECT_FILTER_SCAN),
+
+// project with filter that cannot be pushdown.
+// Filter(Project(Scan))
+
logicalFilter(logicalProject(logicalOlapScan().whenNot(LogicalOlapScan::isIndexSelected)))
+.then(filter -> {
+LogicalProject project =
filter.child();
+LogicalOlapScan scan = project.child();
+return filter.withChildren(project.withChildren(
+select(scan, project::getInputSlots,
ImmutableList::of)
+));
+
[GitHub] [doris] morrySnow commented on a diff in pull request #13600: [fix](planner) inlineView alias error
morrySnow commented on code in PR #13600:
URL: https://github.com/apache/doris/pull/13600#discussion_r1003249044
##
regression-test/suites/correctness/test_pushdown_pred_to_view.groovy:
##
@@ -54,7 +54,7 @@ The same resolve error occurs when re-analyze v2.
"""
qt_sql """
- select * from ${viewName} as v1 join ${viewName} as v2 on v1.id=v2.id
and v1.id>0;
+ select * from ${viewName} as v1 join ${viewName} as v2 where
v1.id=v2.id and v1.id>0;
Review Comment:
add a TODO to this regression test?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow opened a new issue, #13605: [Bug] Unknown column when use MV and table alias
morrySnow opened a new issue, #13605:
URL: https://github.com/apache/doris/issues/13605
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and
found no similar issues.
### Version
all
### What's Wrong?
throw exception
```
Unexpected exception: org.apache.doris.common.AnalysisException: errCode =
2, detailMessage = Unknown column 'mv_bitmap_union_mh' in
'default_cluster:test.original_table'
```
### What You Expected?
query success
### How to Reproduce?
```sql
CREATE TABLE test.original_table
(
day date,
aid bigint,
lid bigint,
yid bigint,
mh bigint,
my bigint
)
DUPLICATE KEY(`day`, `aid`, `lid`)
DISTRIBUTED BY HASH(aid)
PROPERTIES ("replication_num" = "1" );
insert into test.original_table values('2022-10-16', 1665710553, 1665710553,
1665710553, 1665700553, 1665700553);
create materialized view mv_table as
select day,aid,lid,
bitmap_union(to_bitmap(mh)) as wu,
bitmap_union(to_bitmap(my)) as mu
from test.original_table
group by day, aid, lid;
```
wait a while until mv create success. and then
```sql
select t0.aid, t0.lid, count(distinct mh), count(distinct my)
from test.original_table t0
where t0.day = '2022-10-16' and t0.lid > 0 group by t0.aid, t0.lid;
```
### Anything Else?
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] adonis0147 merged pull request #13583: [chore](macOS) Fix issues found on macOS x86_64
adonis0147 merged PR #13583: URL: https://github.com/apache/doris/pull/13583 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [chore](macOS) Fix issues found on macOS x86_64 (#13583)
This is an automated email from the ASF dual-hosted git repository.
adonisling pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 2cf89c55c2 [chore](macOS) Fix issues found on macOS x86_64 (#13583)
2cf89c55c2 is described below
commit 2cf89c55c23d1b0e5ee8abd1c8b060938128cc67
Author: Adonis Ling
AuthorDate: Mon Oct 24 20:59:20 2022 +0800
[chore](macOS) Fix issues found on macOS x86_64 (#13583)
1. Use `brew --prefix` instead of `brew --repo` in scripts.
2. `sprintf` is marked as a deprecated function in MacOSX sdk (13.0).
---
be/src/exec/arrow/parquet_reader.cpp | 22 +++---
be/src/exec/json_scanner.cpp | 10 +-
be/src/exec/orc_scanner.cpp| 4 ++--
be/src/exprs/table_function/explode_json_array.cpp | 10 +-
be/src/runtime/mem_pool.cpp| 2 +-
be/src/vec/exec/vjson_scanner.cpp | 10 +-
bin/start_be.sh| 2 +-
bin/start_fe.sh| 2 +-
fs_brokers/apache_hdfs_broker/bin/start_broker.sh | 5 +
9 files changed, 36 insertions(+), 31 deletions(-)
diff --git a/be/src/exec/arrow/parquet_reader.cpp
b/be/src/exec/arrow/parquet_reader.cpp
index 66d5b69853..cfdb7cd0ef 100644
--- a/be/src/exec/arrow/parquet_reader.cpp
+++ b/be/src/exec/arrow/parquet_reader.cpp
@@ -275,7 +275,7 @@ Status ParquetReaderWrap::read(Tuple* tuple, MemPool*
mem_pool, bool* eof) {
RETURN_IF_ERROR(set_field_null(tuple, slot_desc));
} else {
int32_t value = int32_array->Value(_current_line_of_batch);
-wbytes = sprintf((char*)tmp_buf, "%d", value);
+wbytes = snprintf((char*)tmp_buf, sizeof(tmp_buf), "%d",
value);
fill_slot(tuple, slot_desc, mem_pool, tmp_buf, wbytes);
}
break;
@@ -287,7 +287,7 @@ Status ParquetReaderWrap::read(Tuple* tuple, MemPool*
mem_pool, bool* eof) {
RETURN_IF_ERROR(set_field_null(tuple, slot_desc));
} else {
int64_t value = int64_array->Value(_current_line_of_batch);
-wbytes = sprintf((char*)tmp_buf, "%" PRId64, value);
+wbytes = snprintf((char*)tmp_buf, sizeof(tmp_buf), "%"
PRId64, value);
fill_slot(tuple, slot_desc, mem_pool, tmp_buf, wbytes);
}
break;
@@ -299,7 +299,7 @@ Status ParquetReaderWrap::read(Tuple* tuple, MemPool*
mem_pool, bool* eof) {
RETURN_IF_ERROR(set_field_null(tuple, slot_desc));
} else {
uint32_t value =
uint32_array->Value(_current_line_of_batch);
-wbytes = sprintf((char*)tmp_buf, "%u", value);
+wbytes = snprintf((char*)tmp_buf, sizeof(tmp_buf), "%u",
value);
fill_slot(tuple, slot_desc, mem_pool, tmp_buf, wbytes);
}
break;
@@ -311,7 +311,7 @@ Status ParquetReaderWrap::read(Tuple* tuple, MemPool*
mem_pool, bool* eof) {
RETURN_IF_ERROR(set_field_null(tuple, slot_desc));
} else {
uint64_t value =
uint64_array->Value(_current_line_of_batch);
-wbytes = sprintf((char*)tmp_buf, "%" PRIu64, value);
+wbytes = snprintf((char*)tmp_buf, sizeof(tmp_buf), "%"
PRIu64, value);
fill_slot(tuple, slot_desc, mem_pool, tmp_buf, wbytes);
}
break;
@@ -360,7 +360,7 @@ Status ParquetReaderWrap::read(Tuple* tuple, MemPool*
mem_pool, bool* eof) {
RETURN_IF_ERROR(set_field_null(tuple, slot_desc));
} else {
uint8_t value = uint8_array->Value(_current_line_of_batch);
-wbytes = sprintf((char*)tmp_buf, "%d", value);
+wbytes = snprintf((char*)tmp_buf, sizeof(tmp_buf), "%d",
value);
fill_slot(tuple, slot_desc, mem_pool, tmp_buf, wbytes);
}
break;
@@ -372,7 +372,7 @@ Status ParquetReaderWrap::read(Tuple* tuple, MemPool*
mem_pool, bool* eof) {
RETURN_IF_ERROR(set_field_null(tuple, slot_desc));
} else {
int8_t value = int8_array->Value(_current_line_of_batch);
-wbytes = sprintf((char*)tmp_buf, "%d", value);
+wbytes = snprintf((char*)tmp_buf, sizeof(tmp_buf), "%d",
value);
fill_slot(tuple, slot_desc, mem_pool, tmp_buf, wbytes);
}
break;
@@ -384,7 +384,7 @@ Status ParquetReaderWrap::read(Tuple* tuple, MemPool*
mem_pool, bool* eof) {
RETURN_IF_ER
[GitHub] [doris] gavinchou commented on pull request #13490: [feature-wip](unique-key-merge-on-write) check whether the partition column is a key column when create table for MOW table
gavinchou commented on PR #13490: URL: https://github.com/apache/doris/pull/13490#issuecomment-1289019439 @dataroaring -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataroaring merged pull request #13490: [feature-wip](unique-key-merge-on-write) check whether the partition column is a key column when create table for MOW table
dataroaring merged PR #13490: URL: https://github.com/apache/doris/pull/13490 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [feature-wip](unique-key-merge-on-write) check whether the partition column is a key column when create table for MOW table (#13490)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 57e248e09b [feature-wip](unique-key-merge-on-write) check whether the
partition column is a key column when create table for MOW table (#13490)
57e248e09b is described below
commit 57e248e09b9c929cd4a0e52e198e11ca84c928ef
Author: Xin Liao
AuthorDate: Mon Oct 24 21:16:38 2022 +0800
[feature-wip](unique-key-merge-on-write) check whether the partition column
is a key column when create table for MOW table (#13490)
---
.../main/java/org/apache/doris/analysis/PartitionDesc.java| 11 ++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/PartitionDesc.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/PartitionDesc.java
index d9ffa793b2..37130c41eb 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/PartitionDesc.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/PartitionDesc.java
@@ -24,6 +24,7 @@ import org.apache.doris.catalog.PartitionType;
import org.apache.doris.catalog.PrimitiveType;
import org.apache.doris.common.AnalysisException;
import org.apache.doris.common.DdlException;
+import org.apache.doris.common.util.PropertyAnalyzer;
import org.apache.doris.qe.ConnectContext;
import com.google.common.collect.Lists;
@@ -63,6 +64,13 @@ public class PartitionDesc {
throw new AnalysisException("No partition columns.");
}
+// `analyzeUniqueKeyMergeOnWrite` would modify `properties`, which
will be used later,
+// so we just clone a properties map here.
+boolean enableUniqueKeyMergeOnWrite = false;
+if (otherProperties != null) {
+enableUniqueKeyMergeOnWrite =
+
PropertyAnalyzer.analyzeUniqueKeyMergeOnWrite(Maps.newHashMap(otherProperties));
+}
Set partColNames =
Sets.newTreeSet(String.CASE_INSENSITIVE_ORDER);
for (String partitionCol : partitionColNames) {
if (!partColNames.add(partitionCol)) {
@@ -72,7 +80,8 @@ public class PartitionDesc {
boolean found = false;
for (ColumnDef columnDef : columnDefs) {
if (columnDef.getName().equals(partitionCol)) {
-if (!columnDef.isKey() && columnDef.getAggregateType() !=
AggregateType.NONE) {
+if (!columnDef.isKey() && (columnDef.getAggregateType() !=
AggregateType.NONE
+|| enableUniqueKeyMergeOnWrite)) {
throw new AnalysisException("The partition column
could not be aggregated column");
}
if (columnDef.getType().isFloatingPointType()) {
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #13046: [enhancement](Nereids)Optimize CostAndEnforcerJob
hello-stephen commented on PR #13046: URL: https://github.com/apache/doris/pull/13046#issuecomment-1289084615 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 39.35 seconds load time: 581 seconds storage size: 17154821303 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221024140205_clickbench_pr_33183.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] cambyzju opened a new issue, #13606: [Bug] inner join with functions return wrong result
cambyzju opened a new issue, #13606: URL: https://github.com/apache/doris/issues/13606 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version latest-master ### What's Wrong? return correct result: `select t1.k1, array_min(t2.a1) from t1 join t2 where array_max(t1.a1) = array_max(t2.a1);` return wrong result: `select array_min(t2.a1) from t1 join t2 where array_max(t1.a1) = array_max(t2.a1);` ### What You Expected? fix ### How to Reproduce? 1. create same schema table t1 and t2: ``` CREATE TABLE `t1` ( `k1` int(11) NULL, `a1` array NULL ) ENGINE=OLAP DUPLICATE KEY(`k1`) COMMENT 'OLAP' DISTRIBUTED BY HASH(`k1`) BUCKETS 10 PROPERTIES ( "replication_allocation" = "tag.location.default: 1" ); ``` 2. add test data: ``` insert into t1 values(1, [1,2,3]),(3,[3,2,1]),(3,[3,2,1,NULL]),(2,[3,4,5]); insert into t2 values(1,[2]),(2,[3]); ``` 3. inner join with functions: Following SQL return 3 rows: `select t1.k1, array_min(t2.a1) from t1 join t2 where array_max(t1.a1) = array_max(t2.a1);` ``` +--+--+ | k1 | array_min(`t2`.`a1`) | +--+--+ |3 |3 | |3 |3 | |1 |3 | +--+--+ 3 rows in set (0.043 sec) ``` But this SQL return 0 rows, the correct result should also include 3 rows like the above SQL: `select array_min(t2.a1) from t1 join t2 where array_max(t1.a1) = array_max(t2.a1);` ### Anything Else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen opened a new pull request, #13607: [fix](thirdparty) g++-11: error: unrecognized command-line option '-m…
hello-stephen opened a new pull request, #13607: URL: https://github.com/apache/doris/pull/13607 …sse4.2' # Proposed changes Issue Number: close #xxx ## Problem summary When compiling third-party hdfs on aarch64 machine, got error: g++-11: error: unrecognized command-line option '-msse4.2' ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] cambyzju opened a new pull request, #13608: [bugfix](join) inner join return wrong result
cambyzju opened a new pull request, #13608: URL: https://github.com/apache/doris/pull/13608 # Proposed changes Issue Number: close https://github.com/apache/doris/issues/13606 ## Problem summary In vectorized engine, columns in block should have the same rows. Inside functions probe_side_output_column and build_side_output_column, for non-output columns, if we need output the column, we use insert_from or insert_indices_from to append some data, for columns we do not need output, we also need to append some default data. To make sure columns size are the same for all columns inside the same block. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] adonis0147 commented on a diff in pull request #13607: [fix](thirdparty) g++-11: error: unrecognized command-line option '-m…
adonis0147 commented on code in PR #13607:
URL: https://github.com/apache/doris/pull/13607#discussion_r1003389550
##
thirdparty/build-thirdparty.sh:
##
@@ -1345,8 +1345,13 @@ build_hdfs3() {
cd "${BUILD_DIR}"
rm -rf ./*
+if [[ "$(uname -m)" == "aarch64" ]]; then
+SSE_OPTION='-DENABLE_SSE=OFF'
+else
+SSE_OPTION='-DENABLE_SSE=ON'
+fi
Review Comment:
```suggestion
if [[ "$(uname -m)" != 'x86_64' ]]; then
SSE_OPTION='ON'
else
SSE_OPTION='OFF'
fi
```
It is better to use `x86_64` to compare to the result due to the result of
`uname -m` is `arm64` on macOS M1.
BTW, it seems that the compiler just complains the unrecognized option and
continues. The error doesn't interrupt the compilation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener commented on a diff in pull request #13601: [Feature](Nereids) Support join hint.
jackwener commented on code in PR #13601:
URL: https://github.com/apache/doris/pull/13601#discussion_r1003391337
##
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##
@@ -851,10 +855,23 @@ private LogicalPlan withJoinRelations(LogicalPlan input,
RelationContext ctx) {
condition =
Optional.ofNullable(getExpression(joinCriteria.booleanExpression()));
}
+JoinHint joinHint =
Optional.ofNullable(join.joinHint()).map(hintCtx -> {
+String hint = typedVisit(join.joinHint());
+if
(JoinHint.JoinHintType.SHUFFLE.toString().equalsIgnoreCase(hint)) {
+return JoinHint.SHUFFLE_RIGHT;
+} else if
(JoinHint.JoinHintType.BROADCAST.toString().equalsIgnoreCase(hint)) {
+return JoinHint.BROADCAST_RIGHT;
+} else {
+throw new ParseException("Invalid join hint: " + hint,
hintCtx);
+}
Review Comment:
we can add them into a function like `public static JoinHint of(String name)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener closed pull request #12678: [improve](Nereids): push Project inside Join.
jackwener closed pull request #12678: [improve](Nereids): push Project inside Join. URL: https://github.com/apache/doris/pull/12678 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener closed pull request #12703: [feature](Nereids): all left deep join reorder when push project into join
jackwener closed pull request #12703: [feature](Nereids): all left deep join reorder when push project into join URL: https://github.com/apache/doris/pull/12703 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] lsy3993 opened a new pull request, #13609: [function](date function) add date function 'last_day'
lsy3993 opened a new pull request, #13609: URL: https://github.com/apache/doris/pull/13609 # Proposed changes Issue Number: close #13139 ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [x] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [x] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #13596: [Bugfix](manager) fix query profile key incompatible with old versions
hello-stephen commented on PR #13596: URL: https://github.com/apache/doris/pull/13596#issuecomment-1289199452 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 40.39 seconds load time: 581 seconds storage size: 17154815903 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221024151931_clickbench_pr_33258.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #13449: [feature](Nereids): HyperGraph data struct.
hello-stephen commented on PR #13449: URL: https://github.com/apache/doris/pull/13449#issuecomment-1289219335 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 40.65 seconds load time: 591 seconds storage size: 17154768746 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221024233420_clickbench_pr_33271.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] siriume opened a new issue, #13610: [Bug] doris be core dump
siriume opened a new issue, #13610: URL: https://github.com/apache/doris/issues/13610 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version master-409bd76 ### What's Wrong? insert query form odbc to olap. data bigger than 1,000,000. ``` start time: 2022年 10月 24日 星期一 23:39:29 CST *** Query id: 0-0 *** *** Aborted at 126066 (unix time) try "date -d @126066" if you are using GNU date *** *** Current BE git commitID: 409bd76 *** *** SIGSEGV address not mapped to object (@0x18) received by PID 741195 (TID 0x7f0b85bea700) from PID 24; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /root/doris/be/src/common/signal_handler.h:420 1# 0x7F0C91A7A400 in /lib64/libc.so.6 2# doris::DeltaWriter::get_memtable_consumption_inflush() const at /root/doris/be/src/olap/delta_writer.cpp:392 3# doris::Status doris::TabletsChannel::reduce_mem_usage(doris::PTabletWriterAddBlockResult*) at /root/doris/be/src/runtime/tablets_channel.cpp:224 4# doris::Status doris::LoadChannel::handle_mem_exceed_limit(doris::PTabletWriterAddBlockResult*) at /root/doris/be/src/runtime/load_channel.h:185 5# doris::Status doris::LoadChannelMgr::_handle_mem_exceed_limit(doris::PTabletWriterAddBlockResult*) at /root/doris/be/src/runtime/load_channel_mgr.h:226 6# doris::Status doris::LoadChannelMgr::add_batch(doris::PTabletWriterAddBlockRequest const&, doris::PTabletWriterAddBlockResult*) at /root/doris/be/src/runtime/load_channel_mgr.h:137 7# std::_Function_handler::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/include/c++/11/bits/std_function.h:291 8# doris::PriorityThreadPool::work_thread(int) at /root/doris/be/src/util/priority_thread_pool.hpp:138 9# execute_native_thread_routine in /home/doris/be/lib/doris_be 10# start_thread in /lib64/libpthread.so.0 11# clone in /lib64/libc.so.6 ``` ### What You Expected? Program runs stably. ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Jibing-Li opened a new pull request, #13611: [test](multi-catalog)Regression test for external hive parquet table.
Jibing-Li opened a new pull request, #13611: URL: https://github.com/apache/doris/pull/13611 # Proposed changes Issue Number: close #xxx ## Problem summary Regression test for external hive parquet table. Using docker to provide hive environment. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13589: [fix](load) clear and notify when an error happens in flushing
github-actions[bot] commented on PR #13589: URL: https://github.com/apache/doris/pull/13589#issuecomment-1289448659 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13589: [fix](load) clear and notify when an error happens in flushing
github-actions[bot] commented on PR #13589: URL: https://github.com/apache/doris/pull/13589#issuecomment-1289448697 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #9193: [Feature]Add multi-thread probe to hash join
github-actions[bot] commented on PR #9193: URL: https://github.com/apache/doris/pull/9193#issuecomment-1289807166 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #9132: [Enhancement] Optimize log content, stop print stackinfo #9130
github-actions[bot] commented on PR #9132: URL: https://github.com/apache/doris/pull/9132#issuecomment-1289807184 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #8895: [Feature]Add codegen with llvm
github-actions[bot] commented on PR #8895: URL: https://github.com/apache/doris/pull/8895#issuecomment-1289807209 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #8863: [improvement](test) add min and avg window function case
github-actions[bot] commented on PR #8863: URL: https://github.com/apache/doris/pull/8863#issuecomment-1289807255 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman commented on pull request #13604: [fix][fe] Inconsistent behavior for string comparison in FE and BE
morningman commented on PR #13604: URL: https://github.com/apache/doris/pull/13604#issuecomment-1289844330 Please add test case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman commented on pull request #13597: [fix](planner) cannot recogonize column's table when analyze rewrite expr
morningman commented on PR #13597: URL: https://github.com/apache/doris/pull/13597#issuecomment-1289845553 Please add test case -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hf200012 commented on issue #13610: [Bug] doris be core dump
hf200012 commented on issue #13610: URL: https://github.com/apache/doris/issues/13610#issuecomment-1289859261 @siriume Add my WeChat: 35926237, let's take a look at your question -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.2-unstable updated: [fix](load) clear and notify when an error happens in flushing
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch branch-1.2-unstable
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-unstable by this
push:
new 66ebb075e7 [fix](load) clear and notify when an error happens in
flushing
66ebb075e7 is described below
commit 66ebb075e725ac4827fb18624c1120f8ec74605c
Author: Yongqiang YANG
AuthorDate: Tue Oct 25 02:51:00 2022 +
[fix](load) clear and notify when an error happens in flushing
---
be/src/runtime/tablets_channel.cpp | 8 +---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/be/src/runtime/tablets_channel.cpp
b/be/src/runtime/tablets_channel.cpp
index fa72d23ad0..566c00a19b 100644
--- a/be/src/runtime/tablets_channel.cpp
+++ b/be/src/runtime/tablets_channel.cpp
@@ -276,12 +276,14 @@ Status
TabletsChannel::reduce_mem_usage(TabletWriterAddResult* response) {
_reducing_mem_usage = true;
}
+Status st = Status::OK();
for (auto writer : writers_to_flush) {
-Status st = writer->wait_flush();
+st = writer->wait_flush();
if (!st.ok()) {
-return Status::InternalError(
+st = Status::InternalError(
"failed to reduce mem consumption by flushing memtable.
err: {}",
st.to_string());
+break;
}
}
@@ -291,7 +293,7 @@ Status
TabletsChannel::reduce_mem_usage(TabletWriterAddResult* response) {
_reduce_memory_cond.notify_all();
}
-return Status::OK();
+return st;
}
Status TabletsChannel::_open_all_writers(const PTabletWriterOpenRequest&
request) {
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #13597: [fix](planner) cannot recogonize column's table when analyze rewrite expr
morrySnow commented on PR #13597: URL: https://github.com/apache/doris/pull/13597#issuecomment-1289912708 > Please add test case will add regression test case after fix all p0 and ut failed cases -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] eldenmoon opened a new pull request, #13612: [feature-array](array-type) Add array function array_enumerate
eldenmoon opened a new pull request, #13612:
URL: https://github.com/apache/doris/pull/13612
Returns array sub item indexes eg. [1, 2, 3, …, length (arr) ]
```
mysql> create table array_type_table(k1 INT, k2 Array) duplicate key
(k1)
-> distributed by hash(k1) buckets 1 properties('replication_num' = '1');
mysql> insert into array_type_table values (0, []), ("1", [NULL]), ("2",
["1", "2", "3"]), ("3", ["1", NULL, "3"]), ("4", NULL);
mysql> set enable_vectorized_engine = true;# enable vectorized engine
mysql> select k2, array_enumerate(k2) from array_type_table;
+--+---+
| k2 | array_enumerate(`k2`) |
+--+---+
| [] | []|
| [NULL] | [1] |
| ['1', '2', '3'] | [1, 2, 3] |
| ['1', NULL, '3'] | [1, 2, 3] |
| NULL | NULL |
+--+---+
5 rows in set (0.01 sec)
```
# Proposed changes
Issue Number: close #xxx
## Problem summary
Describe your changes.
## Checklist(Required)
1. Does it affect the original behavior:
- [ ] Yes
- [ ] No
- [ ] I don't know
2. Has unit tests been added:
- [ ] Yes
- [ ] No
- [ ] No Need
3. Has document been added or modified:
- [ ] Yes
- [ ] No
- [ ] No Need
4. Does it need to update dependencies:
- [ ] Yes
- [ ] No
5. Are there any changes that cannot be rolled back:
- [ ] Yes (If Yes, please explain WHY)
- [ ] No
## Further comments
If this is a relatively large or complex change, kick off the discussion at
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you
chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] starocean999 opened a new pull request, #13613: [fix](sort)order by constant expr bug
starocean999 opened a new pull request, #13613: URL: https://github.com/apache/doris/pull/13613 # Proposed changes Issue Number: close (https://github.com/apache/doris/issues/13350) ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] starocean999 opened a new pull request, #13614: [fix](sort)order by constant expr bug
starocean999 opened a new pull request, #13614: URL: https://github.com/apache/doris/pull/13614 # Proposed changes Issue Number: close (https://github.com/apache/doris/issues/13350) ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly commented on a diff in pull request #13375: [feature](nereids) Estimate plan cost by column ndv and table row count
englefly commented on code in PR #13375:
URL: https://github.com/apache/doris/pull/13375#discussion_r1003962955
##
fe/fe-core/src/main/java/org/apache/doris/nereids/memo/GroupExpression.java:
##
@@ -39,6 +40,8 @@
* Representation for group expression in cascades optimizer.
*/
public class GroupExpression {
+private double cost = 0.0;
+private CostEstimate costEstimate = null;
Review Comment:
this is used for memo debug
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly commented on a diff in pull request #13375: [feature](nereids) Estimate plan cost by column ndv and table row count
englefly commented on code in PR #13375:
URL: https://github.com/apache/doris/pull/13375#discussion_r1003964187
##
fe/fe-core/src/main/java/org/apache/doris/nereids/memo/GroupExpression.java:
##
@@ -39,6 +40,8 @@
* Representation for group expression in cascades optimizer.
*/
public class GroupExpression {
+private double cost = 0.0;
Review Comment:
No. It is the cost of itself, the enforced cost is not counted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wangshuo128 commented on a diff in pull request #13416: [Feature](Nereids) Support materialized index selection.
wangshuo128 commented on code in PR #13416:
URL: https://github.com/apache/doris/pull/13416#discussion_r1003970016
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalOlapScan.java:
##
@@ -164,30 +164,31 @@ public long getSelectedIndexId() {
return selectedIndexId;
}
-public boolean isRollupSelected() {
-return rollupSelected;
+public boolean isIndexSelected() {
+return indexSelected;
}
public PreAggStatus getPreAggStatus() {
return preAggStatus;
}
/**
- * Should apply {@link SelectRollupWithAggregate} or not.
+ * Should apply {@link SelectMaterializedIndexWithAggregate} or not.
*/
-public boolean shouldSelectRollup() {
+public boolean shouldSelectIndex() {
Review Comment:
This was to do two things:
1. judge `indexSelected` is true or false
2. make sure the keys type of the OLAP table is `DUP_KEYS`, `UNIQUE_KEYS`,
or `AGG_KEYS`, but it should not be `PRIMARY_KEYS`. Even if it seems
`PRIMARY_KEYS` is not used, it's safe to ensure the scope to apply materialized
view selection rules is right.
I'd unify `shouldSelectIndex()` and `isIndexSelected()` into one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wangshuo128 commented on a diff in pull request #13416: [Feature](Nereids) Support materialized index selection.
wangshuo128 commented on code in PR #13416:
URL: https://github.com/apache/doris/pull/13416#discussion_r1003970016
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalOlapScan.java:
##
@@ -164,30 +164,31 @@ public long getSelectedIndexId() {
return selectedIndexId;
}
-public boolean isRollupSelected() {
-return rollupSelected;
+public boolean isIndexSelected() {
+return indexSelected;
}
public PreAggStatus getPreAggStatus() {
return preAggStatus;
}
/**
- * Should apply {@link SelectRollupWithAggregate} or not.
+ * Should apply {@link SelectMaterializedIndexWithAggregate} or not.
*/
-public boolean shouldSelectRollup() {
+public boolean shouldSelectIndex() {
Review Comment:
This was to do two things:
1. judge `indexSelected` is true or false
2. make sure the keys type of the OLAP table is `DUP_KEYS`, `UNIQUE_KEYS`,
or `AGG_KEYS`, but it should not be `PRIMARY_KEYS`. Even if it seems
`PRIMARY_KEYS` is not used, it's safe to ensure the scope to apply materialized
view selection rules is right.
I would keep the getter method in of the status of `indexSelected` in
`LogicalOlapScan` and move the condition of applying materialized view
selection to the rule itself.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] nextdreamblue commented on pull request #13604: [fix](fe) Inconsistent behavior for string comparison in FE and BE
nextdreamblue commented on PR #13604: URL: https://github.com/apache/doris/pull/13604#issuecomment-1289943396 > Please add test case. done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly commented on a diff in pull request #13375: [feature](nereids) Estimate plan cost by column ndv and table row count
englefly commented on code in PR #13375:
URL: https://github.com/apache/doris/pull/13375#discussion_r1003972708
##
fe/fe-core/src/main/java/org/apache/doris/nereids/cost/CostCalculator.java:
##
@@ -47,6 +48,11 @@
* Inspired by Presto.
*/
public class CostCalculator {
+static final double cpuWeight = 1;
+static final double memorWeight = 1;
+static final double networkWeight = 1.5;
+static final double penaltyWeight = 0.5;
+static final double heavyOperatorPunishFactor = 6.0;
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13449: [feature](Nereids): HyperGraph data struct.
github-actions[bot] commented on PR #13449: URL: https://github.com/apache/doris/pull/13449#issuecomment-1289944504 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13449: [feature](Nereids): HyperGraph data struct.
github-actions[bot] commented on PR #13449: URL: https://github.com/apache/doris/pull/13449#issuecomment-1289944530 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly commented on a diff in pull request #13375: [feature](nereids) Estimate plan cost by column ndv and table row count
englefly commented on code in PR #13375:
URL: https://github.com/apache/doris/pull/13375#discussion_r1003974662
##
fe/fe-core/src/main/java/org/apache/doris/nereids/cost/CostCalculator.java:
##
@@ -47,6 +48,11 @@
* Inspired by Presto.
*/
public class CostCalculator {
+static final double cpuWeight = 1;
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei commented on pull request #13381: [Fix](agg) fix bitmap agg core dump when phmap pointer assert alignment
yiguolei commented on PR #13381: URL: https://github.com/apache/doris/pull/13381#issuecomment-1289948872 1.1 not has this problem. Remove tags. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt merged pull request #13591: [Bug](decimal) Fix incorrect result for decimal multiply
BiteThet merged PR #13591: URL: https://github.com/apache/doris/pull/13591 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (57e248e09b -> 7fe7c01125)
This is an automated email from the ASF dual-hosted git repository.
panxiaolei pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
from 57e248e09b [feature-wip](unique-key-merge-on-write) check whether the
partition column is a key column when create table for MOW table (#13490)
add 7fe7c01125 [Bug](decimal) Fix incorrect result for decimal multiply
(#13591)
No new revisions were added by this update.
Summary of changes:
be/src/vec/functions/multiply.cpp | 6 +-
.../math_functions/test_multiply.out} | 6 --
.../{test_round.groovy => test_multiply.groovy}| 23 --
3 files changed, 22 insertions(+), 13 deletions(-)
copy regression-test/data/{correctness_p0/test_outer_join_sort.out =>
query_p0/sql_functions/math_functions/test_multiply.out} (87%)
copy
regression-test/suites/query_p0/sql_functions/math_functions/{test_round.groovy
=> test_multiply.groovy} (55%)
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly commented on a diff in pull request #13375: [feature](nereids) Estimate plan cost by column ndv and table row count
englefly commented on code in PR #13375:
URL: https://github.com/apache/doris/pull/13375#discussion_r1003977227
##
fe/fe-core/src/main/java/org/apache/doris/nereids/stats/JoinEstimation.java:
##
@@ -69,6 +69,29 @@ private static class JoinEstimationResult {
public double rowCount = 0;
}
+private static double estimateInnerJoin2(Join join, EqualTo equalto,
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly commented on a diff in pull request #13375: [feature](nereids) Estimate plan cost by column ndv and table row count
englefly commented on code in PR #13375:
URL: https://github.com/apache/doris/pull/13375#discussion_r1003977378
##
fe/fe-core/src/main/java/org/apache/doris/nereids/stats/JoinEstimation.java:
##
@@ -124,11 +147,56 @@ private static JoinEstimationResult
estimateInnerJoin(PhysicalHashJoin join, Equ
return result;
}
+/**
+ * estimate join
+ */
+public static StatsDeriveResult estimate(StatsDeriveResult leftStats,
StatsDeriveResult rightStats, Join join) {
+JoinType joinType = join.getJoinType();
+double rowCount = Double.MAX_VALUE;
+if (joinType == JoinType.LEFT_SEMI_JOIN || joinType ==
JoinType.LEFT_ANTI_JOIN) {
+rowCount = leftStats.getRowCount();
+} else if (joinType == JoinType.RIGHT_SEMI_JOIN || joinType ==
JoinType.RIGHT_ANTI_JOIN) {
+rowCount = rightStats.getRowCount();
+} else if (joinType == JoinType.INNER_JOIN) {
+if (join.getHashJoinConjuncts().isEmpty()) {
+//TODO: consider other join conjuncts
+rowCount = leftStats.getRowCount() * rightStats.getRowCount();
+} else {
+for (Expression joinConjunct : join.getHashJoinConjuncts()) {
+double tmpRowCount = estimateInnerJoin2(join,
+(EqualTo) joinConjunct, leftStats, rightStats);
+rowCount = Math.min(rowCount, tmpRowCount);
+}
+}
+} else if (joinType == JoinType.LEFT_OUTER_JOIN) {
+rowCount = leftStats.getRowCount();
+} else if (joinType == JoinType.RIGHT_OUTER_JOIN) {
+rowCount = rightStats.getRowCount();
+} else if (joinType == JoinType.CROSS_JOIN) {
+rowCount = CheckedMath.checkedMultiply(leftStats.getRowCount(),
+rightStats.getRowCount());
+} else {
+throw new RuntimeException("joinType is not supported");
+}
+
+StatsDeriveResult statsDeriveResult = new StatsDeriveResult(rowCount,
Maps.newHashMap());
+if (joinType.isRemainLeftJoin()) {
+statsDeriveResult.merge(leftStats);
+}
+if (joinType.isRemainRightJoin()) {
+statsDeriveResult.merge(rightStats);
+}
+statsDeriveResult.setRowCount(rowCount);
+statsDeriveResult.setWidth(rightStats.getWidth() +
leftStats.getWidth());
+statsDeriveResult.setPenalty(0.0);
+return statsDeriveResult;
+}
+
/**
* Do estimate.
* // TODO: since we have no column stats here. just use a fix ratio to
compute the row count.
*/
-public static StatsDeriveResult estimate(StatsDeriveResult leftStats,
StatsDeriveResult rightStats, Join join) {
+public static StatsDeriveResult estimate2(StatsDeriveResult leftStats,
StatsDeriveResult rightStats, Join join) {
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wangshuo128 commented on a diff in pull request #13416: [Feature](Nereids) Support materialized index selection.
wangshuo128 commented on code in PR #13416:
URL: https://github.com/apache/doris/pull/13416#discussion_r1003977525
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/mv/SelectMaterializedIndexWithoutAggregate.java:
##
@@ -0,0 +1,147 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.mv;
+
+import org.apache.doris.catalog.MaterializedIndex;
+import org.apache.doris.catalog.OlapTable;
+import org.apache.doris.nereids.rules.Rule;
+import org.apache.doris.nereids.rules.RuleType;
+import org.apache.doris.nereids.rules.rewrite.RewriteRuleFactory;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.plans.PreAggStatus;
+import org.apache.doris.nereids.trees.plans.logical.LogicalFilter;
+import org.apache.doris.nereids.trees.plans.logical.LogicalOlapScan;
+import org.apache.doris.nereids.trees.plans.logical.LogicalProject;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableSet;
+
+import java.util.List;
+import java.util.Set;
+import java.util.function.Supplier;
+import java.util.stream.Collectors;
+
+/**
+ * Select materialized index, i.e., both for rollup and materialized view when
aggregate is not present.
+ *
+ * Scan OLAP table with aggregate is handled in {@link
SelectMaterializedIndexWithAggregate}.
+ *
+ * Note that we should first apply {@link
SelectMaterializedIndexWithAggregate} and then
+ * {@link SelectMaterializedIndexWithoutAggregate}.
+ * Besides, these two rules should run in isolated batches, thus when enter
this rule, it's guaranteed that there is
+ * no aggregation on top of the scan.
+ */
+public class SelectMaterializedIndexWithoutAggregate extends
AbstractSelectMaterializedIndexRule
+implements RewriteRuleFactory {
+
+@Override
+public List buildRules() {
+return ImmutableList.of(
+// project with pushdown filter.
+// Project(Filter(Scan))
+
logicalProject(logicalFilter(logicalOlapScan().whenNot(LogicalOlapScan::isIndexSelected)))
+.then(project -> {
+LogicalFilter filter =
project.child();
+LogicalOlapScan scan = filter.child();
+return project.withChildren(filter.withChildren(
+select(scan, project::getInputSlots,
filter::getConjuncts)));
+
}).toRule(RuleType.MATERIALIZED_INDEX_PROJECT_FILTER_SCAN),
+
+// project with filter that cannot be pushdown.
+// Filter(Project(Scan))
+
logicalFilter(logicalProject(logicalOlapScan().whenNot(LogicalOlapScan::isIndexSelected)))
+.then(filter -> {
+LogicalProject project =
filter.child();
+LogicalOlapScan scan = project.child();
+return filter.withChildren(project.withChildren(
+select(scan, project::getInputSlots,
ImmutableList::of)
+));
+
}).toRule(RuleType.MATERIALIZED_INDEX_FILTER_PROJECT_SCAN),
+
+// scan with filters could be pushdown.
+// Filter(Scan)
+
logicalFilter(logicalOlapScan().whenNot(LogicalOlapScan::isIndexSelected))
+.then(filter -> {
+LogicalOlapScan scan = filter.child();
+return filter.withChildren(select(scan,
ImmutableSet::of, filter::getConjuncts));
+})
+.toRule(RuleType.MATERIALIZED_INDEX_FILTER_SCAN),
+
+// project and scan.
+// Project(Scan)
+
logicalProject(logicalOlapScan().whenNot(LogicalOlapScan::isIndexSelected))
+.then(project -> {
+LogicalOlapScan scan = project.child();
+return project.withChildren(
+select(scan, project::getInpu
[GitHub] [doris] yiguolei commented on pull request #13189: [fix](rowset) fix that rowset writer doesn't process the return value , which may result in data loss
yiguolei commented on PR #13189: URL: https://github.com/apache/doris/pull/13189#issuecomment-1289951264 [](https://github.com/apache/doris/pull/13224) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] LOVEGISER commented on issue #13458: Implement the mask function of hive in Doris
LOVEGISER commented on issue #13458: URL: https://github.com/apache/doris/issues/13458#issuecomment-1289951878 您好，麻烦提供下您这块的邮箱地址，我联系下您。或者发邮件到wang...@flywheels.com。我们和您沟通下需求和设计。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] ByteYue opened a new pull request, #13618: [BugFix](replace function) fix infinite loop in replace function
ByteYue opened a new pull request, #13618: URL: https://github.com/apache/doris/pull/13618 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dutyu opened a new pull request, #13619: Fix dirty uninstallation 1.1 lts
dutyu opened a new pull request, #13619: URL: https://github.com/apache/doris/pull/13619 # Proposed changes Issue Number: close #13540 ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [x] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [x] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman opened a new pull request, #13620: [fix](regression-test) set label for stream load
morningman opened a new pull request, #13620: URL: https://github.com/apache/doris/pull/13620 # Proposed changes Issue Number: close #xxx ## Problem summary So that we can for job info easier in regression test ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #13375: [feature](nereids) Estimate plan cost by column ndv and table row count
hello-stephen commented on PR #13375: URL: https://github.com/apache/doris/pull/13375#issuecomment-1289986198 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 38.79 seconds load time: 563 seconds storage size: 17154644849 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221025130527_clickbench_pr_33576.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13609: [function](date function) add date function 'last_day'
github-actions[bot] commented on PR #13609: URL: https://github.com/apache/doris/pull/13609#issuecomment-1289992960 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
