[GitHub] [doris] mymeiyi commented on pull request #22829: [feature](insert) Support group commit insert
mymeiyi commented on PR #22829: URL: https://github.com/apache/doris/pull/22829#issuecomment-1685763176 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22442: [feature](delete-predicate) support delete sub predicate v2
github-actions[bot] commented on PR #22442: URL: https://github.com/apache/doris/pull/22442#issuecomment-1685765371 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on a diff in pull request #22562: [pipelineX](refactor) propose a new pipeline execution model
BiteThet commented on code in PR #22562:
URL: https://github.com/apache/doris/pull/22562#discussion_r1299690755
##
be/src/pipeline/pipeline_x/pipeline_x_fragment_context.cpp:
##
@@ -0,0 +1,585 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "pipeline_x_fragment_context.h"
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+// IWYU pragma: no_include
+#include // IWYU pragma: keep
+#include
+#include
+#include
+#include
+
+#include "common/config.h"
+#include "common/logging.h"
+#include "exec/data_sink.h"
+#include "exec/exec_node.h"
+#include "exec/scan_node.h"
+#include "io/fs/stream_load_pipe.h"
+#include "pipeline/exec/aggregation_sink_operator.h"
+#include "pipeline/exec/aggregation_source_operator.h"
+#include "pipeline/exec/data_queue.h"
+#include "pipeline/exec/datagen_operator.h"
+#include "pipeline/exec/exchange_sink_operator.h"
+#include "pipeline/exec/exchange_source_operator.h"
+#include "pipeline/exec/olap_scan_operator.h"
+#include "pipeline/exec/result_sink_operator.h"
+#include "pipeline/exec/scan_operator.h"
+#include "pipeline/task_scheduler.h"
+#include "runtime/exec_env.h"
+#include "runtime/fragment_mgr.h"
+#include "runtime/runtime_filter_mgr.h"
+#include "runtime/runtime_state.h"
+#include "runtime/stream_load/new_load_stream_mgr.h"
+#include "runtime/stream_load/stream_load_context.h"
+#include "runtime/thread_context.h"
+#include "service/backend_options.h"
+#include "util/container_util.hpp"
+#include "util/debug_util.h"
+#include "util/telemetry/telemetry.h"
+#include "util/uid_util.h"
+#include "vec/common/assert_cast.h"
+#include "vec/exec/join/vhash_join_node.h"
+#include "vec/exec/scan/new_es_scan_node.h"
+#include "vec/exec/scan/new_file_scan_node.h"
+#include "vec/exec/scan/new_odbc_scan_node.h"
+#include "vec/exec/scan/new_olap_scan_node.h"
+#include "vec/exec/scan/vmeta_scan_node.h"
+#include "vec/exec/scan/vscan_node.h"
+#include "vec/exec/vaggregation_node.h"
+#include "vec/exec/vexchange_node.h"
+#include "vec/exec/vunion_node.h"
+#include "vec/runtime/vdata_stream_mgr.h"
+
+namespace doris::pipeline {
+
+#define FOR_EACH_RUNTIME_STATE(stmt) \
+for (auto& runtime_state : _runtime_states) { \
+stmt \
+}
+
+PipelineXFragmentContext::PipelineXFragmentContext(
+const TUniqueId& query_id, const int fragment_id,
std::shared_ptr query_ctx,
+ExecEnv* exec_env, const std::function&
call_back,
+const report_status_callback& report_status_cb)
+: PipelineFragmentContext(query_id, TUniqueId(), fragment_id, -1,
query_ctx, exec_env,
+ call_back, report_status_cb) {}
+
+PipelineXFragmentContext::~PipelineXFragmentContext() {
+if (!_runtime_states.empty()) {
+// The memory released by the query end is recorded in the query mem
tracker, main memory in _runtime_state.
+SCOPED_ATTACH_TASK(_runtime_state.get());
+FOR_EACH_RUNTIME_STATE(_call_back(runtime_state.get(), &_exec_status);
+ runtime_state.reset();)
+} else {
+_call_back(nullptr, &_exec_status);
+}
+_runtime_state.reset();
+DCHECK(!_report_thread_active);
+}
+
+void PipelineXFragmentContext::cancel(const PPlanFragmentCancelReason& reason,
+ const std::string& msg) {
+if (!_runtime_state->is_cancelled()) {
+std::lock_guard l(_status_lock);
+if (_runtime_state->is_cancelled()) {
+return;
+}
+if (reason != PPlanFragmentCancelReason::LIMIT_REACH) {
+_exec_status = Status::Cancelled(msg);
+}
+
+for (auto& rs : _runtime_states) {
+rs->set_is_cancelled(true, msg);
+rs->set_process_status(_exec_status);
+_exec_env->vstream_mgr()->cancel(rs->fragment_instance_id());
+}
+
+LOG(WARNING) << "PipelineFragmentContext Canceled. reason=" << msg;
+
+// Get pipe from new load stream manager and send cancel to it or the
fragment may hang to wait read from pipe
+// For stream load the fragment's quer
[GitHub] [doris] morrySnow commented on a diff in pull request #22907: [fix](nereids)scalar subquery shouldn't be used in mark join
morrySnow commented on code in PR #22907:
URL: https://github.com/apache/doris/pull/22907#discussion_r1299690150
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/AggScalarSubQueryToWindowFunction.java:
##
@@ -326,7 +324,17 @@ private Plan rewrite(LogicalFilter filter,
LogicalApply> conjuncts =
filter.getConjuncts().stream()
+.collect(Collectors.groupingBy(conjunct -> Sets
+.intersection(conjunct.getInputSlotExprIds(),
agg.getOutputExprIdSet())
+.isEmpty(), Collectors.toSet()));
Review Comment:
if only need false part, use filter is better
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/SubqueryToApply.java:
##
@@ -100,24 +99,21 @@ public List buildRules() {
applyPlan = subqueryToApply(subqueryExprs.stream()
.collect(ImmutableList.toImmutableList()),
tmpPlan,
context.getSubqueryToMarkJoinSlot(),
-context.getSubqueryCorrespondingConjunct(),
ctx.cascadesContext,
+ctx.cascadesContext,
Optional.of(conjunct), false);
tmpPlan = applyPlan;
-if (!(subqueryExprs.size() == 1
-&&
subqueryExprs.stream().anyMatch(ScalarSubquery.class::isInstance))) {
-newConjuncts.add(conjunct);
-}
+newConjuncts.add(conjunct);
}
-Set conjects = new LinkedHashSet<>();
-conjects.addAll(newConjuncts.build());
-Plan newFilter = new LogicalFilter<>(conjects, applyPlan);
-if (conjects.stream().flatMap(c -> c.children().stream())
+Set conjuncts = new LinkedHashSet<>();
+conjuncts.addAll(newConjuncts.build());
Review Comment:
declare `newConjuncts` as `ImmutableSet.Builder`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22907: [fix](nereids)scalar subquery shouldn't be used in mark join
github-actions[bot] commented on PR #22907: URL: https://github.com/apache/doris/pull/22907#issuecomment-1685768123 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xy720 commented on a diff in pull request #23086: [Feature](fe) Add admin set partition version statement
xy720 commented on code in PR #23086:
URL: https://github.com/apache/doris/pull/23086#discussion_r1299691906
##
fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java:
##
@@ -5379,6 +5381,62 @@ public void cleanTrash(AdminCleanTrashStmt stmt) {
}
}
+public void setPartitionVersion(AdminSetPartitionVersionStmt stmt) throws
DdlException {
+long partitionId = stmt.getPartitionId();
+long versionInfo = stmt.getVisibleVersion();
+int setSuccess = setPartitionVersionInternal(partitionId, versionInfo,
false);
+if (setSuccess == -1) {
+throw new DdlException("Failed to set partition visible version to
" + versionInfo + ". "
++ "Partition " + partitionId + " not exists.");
+}
+}
+
+public void replaySetPartitionVersion(SetPartitionVersionOperationLog log)
throws DdlException {
+int setSuccess = setPartitionVersionInternal(log.getPartitionId(),
log.getVersionInfo(), true);
+if (setSuccess == -1) {
+LOG.warn("Failed to set partition visible version to {}. Partition
{} not exists.",
+log.getVersionInfo(), log.getPartitionId());
+}
+}
+
+public int setPartitionVersionInternal(long partitionId, long versionInfo,
boolean isReplay) throws DdlException {
+int result = -1;
+List dbIds = getInternalCatalog().getDbIds();
+// TODO should use inverted index
+for (long dbId : dbIds) {
Review Comment:
Sure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22907: [fix](nereids)scalar subquery shouldn't be used in mark join
github-actions[bot] commented on PR #22907: URL: https://github.com/apache/doris/pull/22907#issuecomment-1685768163 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xinxingi commented on pull request #23247: [Feature] Lack of detailed parameter documentation
xinxingi commented on PR #23247: URL: https://github.com/apache/doris/pull/23247#issuecomment-1685775232 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on a diff in pull request #22562: [pipelineX](refactor) propose a new pipeline execution model
BiteThet commented on code in PR #22562:
URL: https://github.com/apache/doris/pull/22562#discussion_r1299697940
##
be/src/pipeline/pipeline_x/pipeline_x_fragment_context.cpp:
##
@@ -0,0 +1,585 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "pipeline_x_fragment_context.h"
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+// IWYU pragma: no_include
+#include // IWYU pragma: keep
+#include
+#include
+#include
+#include
+
+#include "common/config.h"
+#include "common/logging.h"
+#include "exec/data_sink.h"
+#include "exec/exec_node.h"
+#include "exec/scan_node.h"
+#include "io/fs/stream_load_pipe.h"
+#include "pipeline/exec/aggregation_sink_operator.h"
+#include "pipeline/exec/aggregation_source_operator.h"
+#include "pipeline/exec/data_queue.h"
+#include "pipeline/exec/datagen_operator.h"
+#include "pipeline/exec/exchange_sink_operator.h"
+#include "pipeline/exec/exchange_source_operator.h"
+#include "pipeline/exec/olap_scan_operator.h"
+#include "pipeline/exec/result_sink_operator.h"
+#include "pipeline/exec/scan_operator.h"
+#include "pipeline/task_scheduler.h"
+#include "runtime/exec_env.h"
+#include "runtime/fragment_mgr.h"
+#include "runtime/runtime_filter_mgr.h"
+#include "runtime/runtime_state.h"
+#include "runtime/stream_load/new_load_stream_mgr.h"
+#include "runtime/stream_load/stream_load_context.h"
+#include "runtime/thread_context.h"
+#include "service/backend_options.h"
+#include "util/container_util.hpp"
+#include "util/debug_util.h"
+#include "util/telemetry/telemetry.h"
+#include "util/uid_util.h"
+#include "vec/common/assert_cast.h"
+#include "vec/exec/join/vhash_join_node.h"
+#include "vec/exec/scan/new_es_scan_node.h"
+#include "vec/exec/scan/new_file_scan_node.h"
+#include "vec/exec/scan/new_odbc_scan_node.h"
+#include "vec/exec/scan/new_olap_scan_node.h"
+#include "vec/exec/scan/vmeta_scan_node.h"
+#include "vec/exec/scan/vscan_node.h"
+#include "vec/exec/vaggregation_node.h"
+#include "vec/exec/vexchange_node.h"
+#include "vec/exec/vunion_node.h"
+#include "vec/runtime/vdata_stream_mgr.h"
+
+namespace doris::pipeline {
+
+#define FOR_EACH_RUNTIME_STATE(stmt) \
+for (auto& runtime_state : _runtime_states) { \
+stmt \
+}
+
+PipelineXFragmentContext::PipelineXFragmentContext(
+const TUniqueId& query_id, const int fragment_id,
std::shared_ptr query_ctx,
+ExecEnv* exec_env, const std::function&
call_back,
+const report_status_callback& report_status_cb)
+: PipelineFragmentContext(query_id, TUniqueId(), fragment_id, -1,
query_ctx, exec_env,
+ call_back, report_status_cb) {}
+
+PipelineXFragmentContext::~PipelineXFragmentContext() {
+if (!_runtime_states.empty()) {
+// The memory released by the query end is recorded in the query mem
tracker, main memory in _runtime_state.
+SCOPED_ATTACH_TASK(_runtime_state.get());
+FOR_EACH_RUNTIME_STATE(_call_back(runtime_state.get(), &_exec_status);
+ runtime_state.reset();)
+} else {
+_call_back(nullptr, &_exec_status);
+}
+_runtime_state.reset();
+DCHECK(!_report_thread_active);
+}
+
+void PipelineXFragmentContext::cancel(const PPlanFragmentCancelReason& reason,
+ const std::string& msg) {
+if (!_runtime_state->is_cancelled()) {
+std::lock_guard l(_status_lock);
+if (_runtime_state->is_cancelled()) {
+return;
+}
+if (reason != PPlanFragmentCancelReason::LIMIT_REACH) {
+_exec_status = Status::Cancelled(msg);
+}
+
+for (auto& rs : _runtime_states) {
+rs->set_is_cancelled(true, msg);
+rs->set_process_status(_exec_status);
+_exec_env->vstream_mgr()->cancel(rs->fragment_instance_id());
+}
+
+LOG(WARNING) << "PipelineFragmentContext Canceled. reason=" << msg;
+
+// Get pipe from new load stream manager and send cancel to it or the
fragment may hang to wait read from pipe
+// For stream load the fragment's quer
[GitHub] [doris] BiteTheDDDDt commented on a diff in pull request #22562: [pipelineX](refactor) propose a new pipeline execution model
BiteThet commented on code in PR #22562:
URL: https://github.com/apache/doris/pull/22562#discussion_r1299697334
##
be/src/pipeline/pipeline_x/pipeline_x_fragment_context.cpp:
##
@@ -0,0 +1,585 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "pipeline_x_fragment_context.h"
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+// IWYU pragma: no_include
+#include // IWYU pragma: keep
+#include
+#include
+#include
+#include
+
+#include "common/config.h"
+#include "common/logging.h"
+#include "exec/data_sink.h"
+#include "exec/exec_node.h"
+#include "exec/scan_node.h"
+#include "io/fs/stream_load_pipe.h"
+#include "pipeline/exec/aggregation_sink_operator.h"
+#include "pipeline/exec/aggregation_source_operator.h"
+#include "pipeline/exec/data_queue.h"
+#include "pipeline/exec/datagen_operator.h"
+#include "pipeline/exec/exchange_sink_operator.h"
+#include "pipeline/exec/exchange_source_operator.h"
+#include "pipeline/exec/olap_scan_operator.h"
+#include "pipeline/exec/result_sink_operator.h"
+#include "pipeline/exec/scan_operator.h"
+#include "pipeline/task_scheduler.h"
+#include "runtime/exec_env.h"
+#include "runtime/fragment_mgr.h"
+#include "runtime/runtime_filter_mgr.h"
+#include "runtime/runtime_state.h"
+#include "runtime/stream_load/new_load_stream_mgr.h"
+#include "runtime/stream_load/stream_load_context.h"
+#include "runtime/thread_context.h"
+#include "service/backend_options.h"
+#include "util/container_util.hpp"
+#include "util/debug_util.h"
+#include "util/telemetry/telemetry.h"
+#include "util/uid_util.h"
+#include "vec/common/assert_cast.h"
+#include "vec/exec/join/vhash_join_node.h"
+#include "vec/exec/scan/new_es_scan_node.h"
+#include "vec/exec/scan/new_file_scan_node.h"
+#include "vec/exec/scan/new_odbc_scan_node.h"
+#include "vec/exec/scan/new_olap_scan_node.h"
+#include "vec/exec/scan/vmeta_scan_node.h"
+#include "vec/exec/scan/vscan_node.h"
+#include "vec/exec/vaggregation_node.h"
+#include "vec/exec/vexchange_node.h"
+#include "vec/exec/vunion_node.h"
+#include "vec/runtime/vdata_stream_mgr.h"
+
+namespace doris::pipeline {
+
+#define FOR_EACH_RUNTIME_STATE(stmt) \
+for (auto& runtime_state : _runtime_states) { \
+stmt \
+}
+
+PipelineXFragmentContext::PipelineXFragmentContext(
+const TUniqueId& query_id, const int fragment_id,
std::shared_ptr query_ctx,
+ExecEnv* exec_env, const std::function&
call_back,
+const report_status_callback& report_status_cb)
+: PipelineFragmentContext(query_id, TUniqueId(), fragment_id, -1,
query_ctx, exec_env,
+ call_back, report_status_cb) {}
+
+PipelineXFragmentContext::~PipelineXFragmentContext() {
+if (!_runtime_states.empty()) {
+// The memory released by the query end is recorded in the query mem
tracker, main memory in _runtime_state.
+SCOPED_ATTACH_TASK(_runtime_state.get());
+FOR_EACH_RUNTIME_STATE(_call_back(runtime_state.get(), &_exec_status);
+ runtime_state.reset();)
+} else {
+_call_back(nullptr, &_exec_status);
+}
+_runtime_state.reset();
+DCHECK(!_report_thread_active);
+}
+
+void PipelineXFragmentContext::cancel(const PPlanFragmentCancelReason& reason,
+ const std::string& msg) {
+if (!_runtime_state->is_cancelled()) {
+std::lock_guard l(_status_lock);
+if (_runtime_state->is_cancelled()) {
+return;
+}
+if (reason != PPlanFragmentCancelReason::LIMIT_REACH) {
+_exec_status = Status::Cancelled(msg);
+}
+
+for (auto& rs : _runtime_states) {
+rs->set_is_cancelled(true, msg);
+rs->set_process_status(_exec_status);
+_exec_env->vstream_mgr()->cancel(rs->fragment_instance_id());
+}
+
+LOG(WARNING) << "PipelineFragmentContext Canceled. reason=" << msg;
+
+// Get pipe from new load stream manager and send cancel to it or the
fragment may hang to wait read from pipe
+// For stream load the fragment's quer
[GitHub] [doris-spark-connector] JNSimba merged pull request #130: [fix] write date type error with json format
JNSimba merged PR #130: URL: https://github.com/apache/doris-spark-connector/pull/130 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-spark-connector] branch master updated: [fix] write date type error with json format (#130)
This is an automated email from the ASF dual-hosted git repository.
diwu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-spark-connector.git
The following commit(s) were added to refs/heads/master by this push:
new b5f9095 [fix] write date type error with json format (#130)
b5f9095 is described below
commit b5f9095360ca0b6a7c7ef82732d09717be4bb941
Author: gnehil
AuthorDate: Mon Aug 21 15:15:34 2023 +0800
[fix] write date type error with json format (#130)
---
spark-doris-connector/pom.xml | 2 +-
.../src/main/java/org/apache/doris/spark/util/DataUtil.java| 3 ++-
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/spark-doris-connector/pom.xml b/spark-doris-connector/pom.xml
index a5a7e7a..74a53cc 100644
--- a/spark-doris-connector/pom.xml
+++ b/spark-doris-connector/pom.xml
@@ -67,7 +67,7 @@
-1.2.0-SNAPSHOT
+1.3.0-SNAPSHOT
3.1.2
3.1
2.12
diff --git
a/spark-doris-connector/src/main/java/org/apache/doris/spark/util/DataUtil.java
b/spark-doris-connector/src/main/java/org/apache/doris/spark/util/DataUtil.java
index ed9af50..5877447 100644
---
a/spark-doris-connector/src/main/java/org/apache/doris/spark/util/DataUtil.java
+++
b/spark-doris-connector/src/main/java/org/apache/doris/spark/util/DataUtil.java
@@ -20,6 +20,7 @@ package org.apache.doris.spark.util;
import scala.collection.JavaConversions;
import scala.collection.mutable.WrappedArray;
+import java.sql.Date;
import java.sql.Timestamp;
import java.util.Arrays;
@@ -33,7 +34,7 @@ public class DataUtil {
return NULL_VALUE;
}
-if (value instanceof Timestamp) {
+if (value instanceof Date || value instanceof Timestamp) {
return value.toString();
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] sohardforaname opened a new pull request, #23249: [Fix](Nereids)add nereids load function in read fields of GlobalFunctionMgr and Database for branch-2.0
sohardforaname opened a new pull request, #23249: URL: https://github.com/apache/doris/pull/23249 ## Proposed changes Issue Number: close #xxx add nereids load function in read fields of GlobalFunctionMgr and Database to fix some udf is lost when restart FE and query with Nereids. ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #23249: [Fix](Nereids)add nereids load function in read fields of GlobalFunctionMgr and Database for branch-2.0
sohardforaname commented on PR #23249: URL: https://github.com/apache/doris/pull/23249#issuecomment-1685782355 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zy-kkk commented on pull request #23056: [feature](jdbc catalog) support doris jdbc catalog array type
zy-kkk commented on PR #23056: URL: https://github.com/apache/doris/pull/23056#issuecomment-1685788260 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] amorynan opened a new pull request, #23250: [FIX](map)fix column map for offset next_array_item_rowid order
amorynan opened a new pull request, #23250: URL: https://github.com/apache/doris/pull/23250 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow closed pull request #22330: [opt](Nereids) let project and filter derive parent physcial property request
morrySnow closed pull request #22330: [opt](Nereids) let project and filter derive parent physcial property request URL: https://github.com/apache/doris/pull/22330 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener commented on pull request #23241: [fix](Nereids): fix LogicalProject equals()
jackwener commented on PR #23241: URL: https://github.com/apache/doris/pull/23241#issuecomment-1685792711 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] cambyzju commented on pull request #23190: [fix](compile) compile be failed with gcc12
cambyzju commented on PR #23190: URL: https://github.com/apache/doris/pull/23190#issuecomment-1685795369 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23190: [fix](compile) compile be failed with gcc12
github-actions[bot] commented on PR #23190: URL: https://github.com/apache/doris/pull/23190#issuecomment-1685796206 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23250: [FIX](map)fix column map for offset next_array_item_rowid order
github-actions[bot] commented on PR #23250: URL: https://github.com/apache/doris/pull/23250#issuecomment-1685799629 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22651: [Fix](draft) just for test
hello-stephen commented on PR #22651: URL: https://github.com/apache/doris/pull/22651#issuecomment-168587 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.4 seconds stream load tsv: 548 seconds loaded 74807831229 Bytes, about 130 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.7 seconds inserted 1000 Rows, about 336K ops/s storage size: 17162000243 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #23134: [refactor](pipeline) small refactor of pipeline execution on be.
hello-stephen commented on PR #23134: URL: https://github.com/apache/doris/pull/23134#issuecomment-1685800069 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.62 seconds stream load tsv: 542 seconds loaded 74807831229 Bytes, about 131 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 68 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.4 seconds inserted 1000 Rows, about 340K ops/s storage size: 17162113394 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #23246: [bugfix](vertical-compaction) Only init the _segment_cache_handle in beta_rowset_reader once
hello-stephen commented on PR #23246: URL: https://github.com/apache/doris/pull/23246#issuecomment-1685800176 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 44.76 seconds stream load tsv: 552 seconds loaded 74807831229 Bytes, about 129 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.5 seconds inserted 1000 Rows, about 338K ops/s storage size: 17162116707 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #23244: [feature](Nereids) add password func
hello-stephen commented on PR #23244: URL: https://github.com/apache/doris/pull/23244#issuecomment-1685800232 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.59 seconds stream load tsv: 546 seconds loaded 74807831229 Bytes, about 130 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 67 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.5 seconds inserted 1000 Rows, about 338K ops/s storage size: 17161933076 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #23243: [chore] Fix typos in comments
hello-stephen commented on PR #23243: URL: https://github.com/apache/doris/pull/23243#issuecomment-1685800414 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.42 seconds stream load tsv: 546 seconds loaded 74807831229 Bytes, about 130 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.5 seconds inserted 1000 Rows, about 338K ops/s storage size: 17161977787 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #23248: [Fix](Nereids)add nereids load function in read fields of GlobalFunctionMgr and Database
hello-stephen commented on PR #23248: URL: https://github.com/apache/doris/pull/23248#issuecomment-1685800736 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.48 seconds stream load tsv: 545 seconds loaded 74807831229 Bytes, about 130 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.4 seconds inserted 1000 Rows, about 340K ops/s storage size: 17162228247 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22475: [Doc] modify the document error,about outfile
hello-stephen commented on PR #22475: URL: https://github.com/apache/doris/pull/22475#issuecomment-1685801049 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 48.45 seconds stream load tsv: 551 seconds loaded 74807831229 Bytes, about 129 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 30.0 seconds inserted 1000 Rows, about 333K ops/s storage size: 17162131183 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] cambyzju commented on pull request #23190: [fix](compile) compile be failed with gcc12
cambyzju commented on PR #23190: URL: https://github.com/apache/doris/pull/23190#issuecomment-1685801074 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23190: [fix](compile) compile be failed with gcc12
github-actions[bot] commented on PR #23190: URL: https://github.com/apache/doris/pull/23190#issuecomment-1685802627 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23190: [fix](compile) compile be failed with gcc12
github-actions[bot] commented on PR #23190: URL: https://github.com/apache/doris/pull/23190#issuecomment-1685802840 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #22794: [DNM](test) placement of Nereids only
morrySnow commented on PR #22794: URL: https://github.com/apache/doris/pull/22794#issuecomment-1685805373 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22851: [feature](Nereids) support array type
hello-stephen commented on PR #22851: URL: https://github.com/apache/doris/pull/22851#issuecomment-1685805956 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.79 seconds stream load tsv: 545 seconds loaded 74807831229 Bytes, about 130 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.3 seconds inserted 1000 Rows, about 341K ops/s storage size: 17162238335 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] starocean999 merged pull request #22907: [fix](nereids)scalar subquery shouldn't be used in mark join
starocean999 merged PR #22907: URL: https://github.com/apache/doris/pull/22907 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (aaf9a51440 -> 8411705e36)
This is an automated email from the ASF dual-hosted git repository. starocean999 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from aaf9a51440 [fix](nereids) fix insert timeout loses effect with nereids (#23217) add 8411705e36 [fix](nereids)scalar subquery shouldn't be used in mark join (#22907) No new revisions were added by this update. Summary of changes: .../nereids/rules/analysis/SubqueryToApply.java| 132 +++-- .../join/LogicalJoinSemiJoinTranspose.java | 6 +- .../join/LogicalJoinSemiJoinTransposeProject.java | 7 +- .../join/SemiJoinSemiJoinTranspose.java| 2 +- .../rewrite/AggScalarSubQueryToWindowFunction.java | 25 ++-- .../nereids/rules/rewrite/ExistsApplyToJoin.java | 19 +-- .../doris/nereids/rules/rewrite/InApplyToJoin.java | 3 - ...CorrelatedFilterUnderApplyAggregateProject.java | 4 +- .../rules/rewrite/PullUpProjectUnderApply.java | 4 +- .../rewrite/PushdownFilterThroughProject.java | 36 +++--- .../nereids/rules/rewrite/ScalarApplyToJoin.java | 17 +-- .../rewrite/UnCorrelatedApplyAggregateFilter.java | 4 +- .../rules/rewrite/UnCorrelatedApplyFilter.java | 4 +- .../rewrite/UnCorrelatedApplyProjectFilter.java| 4 +- .../trees/copier/LogicalPlanDeepCopier.java| 4 +- .../nereids/trees/plans/logical/LogicalApply.java | 48 .../rules/analysis/AnalyzeWhereSubqueryTest.java | 62 +- .../AggScalarSubQueryToWindowFunctionTest.java | 3 + .../nereids_syntax_p0/sub_query_correlated.out | 3 - .../nereids_tpcds_shape_sf100_p0/shape/query1.out | 18 +-- .../nereids_tpcds_shape_sf100_p0/shape/query30.out | 15 +-- .../nereids_tpcds_shape_sf100_p0/shape/query41.out | 2 +- .../nereids_tpcds_shape_sf100_p0/shape/query6.out | 59 - .../nereids_tpcds_shape_sf100_p0/shape/query81.out | 15 +-- .../nereids_tpch_shape_sf1000_p0/shape/q20.out | 2 +- .../data/nereids_tpch_shape_sf500_p0/shape/q20.out | 2 +- .../nereids_syntax_p0/sub_query_correlated.groovy | 6 +- .../shape/query6.groovy| 3 + 28 files changed, 208 insertions(+), 301 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 merged pull request #22562: [pipelineX](refactor) propose a new pipeline execution model
Gabriel39 merged PR #22562: URL: https://github.com/apache/doris/pull/22562 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23190: [fix](compile) compile be failed with gcc12
github-actions[bot] commented on PR #23190: URL: https://github.com/apache/doris/pull/23190#issuecomment-1685810828 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] amorynan opened a new pull request, #23251: [FIX](map)fix map offsets set next_array_item_rowid
amorynan opened a new pull request, #23251: URL: https://github.com/apache/doris/pull/23251 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] amorynan commented on pull request #23250: [FIX](map)fix column map for offset next_array_item_rowid order
amorynan commented on PR #23250: URL: https://github.com/apache/doris/pull/23250#issuecomment-1685812457 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23251: [FIX](map)fix map offsets set next_array_item_rowid
github-actions[bot] commented on PR #23251: URL: https://github.com/apache/doris/pull/23251#issuecomment-1685819433 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23251: [FIX](map)fix map offsets set next_array_item_rowid
github-actions[bot] commented on PR #23251: URL: https://github.com/apache/doris/pull/23251#issuecomment-1685819486 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23251: [FIX](map)fix map offsets set next_array_item_rowid
github-actions[bot] commented on PR #23251: URL: https://github.com/apache/doris/pull/23251#issuecomment-1685822660 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wsjz opened a new pull request, #23252: [fix](multi-catalog)fix broker and tvf params
wsjz opened a new pull request, #23252: URL: https://github.com/apache/doris/pull/23252 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #23249: [Fix](Nereids)add nereids load function in read fields of GlobalFunctionMgr and Database for branch-2.0
hello-stephen commented on PR #23249: URL: https://github.com/apache/doris/pull/23249#issuecomment-1685826530 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 44.54 seconds stream load tsv: 547 seconds loaded 74807831229 Bytes, about 130 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.3 seconds inserted 1000 Rows, about 341K ops/s storage size: 17164544632 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly opened a new pull request, #23253: [fix](ut) fix "beNumber not found" bug
englefly opened a new pull request, #23253: URL: https://github.com/apache/doris/pull/23253 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zy-kkk commented on pull request #23080: [fix](jdbc catalog) fix adaptation to Oracle special character `/` table names
zy-kkk commented on PR #23080: URL: https://github.com/apache/doris/pull/23080#issuecomment-1685829097 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] shuke987 commented on pull request #23253: [fix](ut) fix "beNumber not found" bug
shuke987 commented on PR #23253: URL: https://github.com/apache/doris/pull/23253#issuecomment-1685830990 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23253: [fix](ut) fix "beNumber not found" bug
github-actions[bot] commented on PR #23253: URL: https://github.com/apache/doris/pull/23253#issuecomment-1685832267 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #23190: [fix](compile) compile be failed with gcc12
hello-stephen commented on PR #23190: URL: https://github.com/apache/doris/pull/23190#issuecomment-1685839594 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 44.83 seconds stream load tsv: 548 seconds loaded 74807831229 Bytes, about 130 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.8 seconds inserted 1000 Rows, about 335K ops/s storage size: 17162360467 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #23241: [fix](Nereids): fix LogicalProject equals()
hello-stephen commented on PR #23241: URL: https://github.com/apache/doris/pull/23241#issuecomment-1685839750 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.15 seconds stream load tsv: 548 seconds loaded 74807831229 Bytes, about 130 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.3 seconds inserted 1000 Rows, about 341K ops/s storage size: 17162395174 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #23056: [feature](jdbc catalog) support doris jdbc catalog array type
hello-stephen commented on PR #23056: URL: https://github.com/apache/doris/pull/23056#issuecomment-1685842054 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 47.8 seconds stream load tsv: 545 seconds loaded 74807831229 Bytes, about 130 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.3 seconds inserted 1000 Rows, about 341K ops/s storage size: 17162244517 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Lchangliang commented on pull request #23246: [bugfix](vertical-compaction) Only can init the SegmentCacheHandle once
Lchangliang commented on PR #23246: URL: https://github.com/apache/doris/pull/23246#issuecomment-1685845856 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22794: [DNM](test) placement of Nereids only
hello-stephen commented on PR #22794: URL: https://github.com/apache/doris/pull/22794#issuecomment-1685846185 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.44 seconds stream load tsv: 545 seconds loaded 74807831229 Bytes, about 130 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.3 seconds inserted 1000 Rows, about 341K ops/s storage size: 17161978223 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] bobhan1 commented on pull request #22970: [feature](partial update) Support partial update for broker load
bobhan1 commented on PR #22970: URL: https://github.com/apache/doris/pull/22970#issuecomment-1685846230 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] ouyangkui commented on issue #23032: [Bug] 1.2.6， after search 4700000 log，be broken，and can not start anymore
ouyangkui commented on issue #23032:
URL: https://github.com/apache/doris/issues/23032#issuecomment-1685846767
reproduce this issue with 2.0.
upload csv log, by sql, insert 1000 logs every times,
create table invertedIneex
(
`dt` DATETIMEV2(3),
`clientip` VARCHAR(20),
`user` TEXT,
`method` VARCHAR(20),
`request` TEXT,
`protocol` VARCHAR(20),
`status` int(11),
`bytes` int(10),
`referer` TEXT,
`user_agent` TEXT,
`x_forwarded_for` TEXT,
`raw_message` TEXT,
INDEX index_raw (`raw_message`) USING INVERTED PROPERTIES("parser" =
"english") COMMENT 'reindex'
)
DUPLICATE KEY(`dt`)
DISTRIBUTED BY HASH(`dt`) BUCKETS 1
PROPERTIES (
"replication_num" = "1"
)
--
1 log example ::
2023-01-15
01:52:58.165000,113.128.11.195,-,GET,/index/login/?gw_address=192.168.11.1&gw_port=2060&gw_id=0531HY000123&mac=d4:97:0b:15:e3:5e&url=http%3A//infoc2.duba.net/c/,HTTP/1.1,200,4633,-,-,-,113.128.11.195
- - [15/Jan/2023:01:52:58.165 +0800] "GET
/index/login/?gw_address=192.168.11.1&gw_port=2060&gw_id=0531HY000123&mac=d4:97:0b:15:e3:5e&url=http%3A//infoc2.duba.net/c/
HTTP/1.1" 200 4633 "-" "-" "-"
result:
be crashed, and can not start anymore
> I0821 15:17:02.644104 6323 tablets_channel.cpp:103] open tablets channel:
(load_id=26176df166004912-a4ebb62a9444d314, index_id=18007), tablets num: 1,
timeout(s): 14400
I0821 15:17:02.668521 5702 vtablet_sink.cpp:894] VNodeChannel[18007-10034],
load_id=26176df166004912-a4ebb62a9444d314, txn_id=7004,
node=192.168.40.117:8060 mark closed, left pending batch size: 1
I0821 15:17:02.671339 6409 vtablet_sink.cpp:1121] all node channels are
stopped(maybe finished/offending/cancelled), sender thread exit.
26176df166004912-a4ebb62a9444d314
I0821 15:17:02.674144 6185 tablets_channel.cpp:145] close tablets channel:
(load_id=26176df166004912-a4ebb62a9444d314, index_id=18007), sender id: 0,
backend id: 10034
*** Query id: 0-0 ***
*** Aborted at 169260 (unix time) try "date -d @169260" if you are
using GNU date ***
*** Current BE git commitID: 96bb9a7 ***
*** SIGILL illegal operand (@0x555d7c8a21d0) received by PID 5387 (TID 5978
OR 0x7f9e37518700) from PID 2089427408; stack trace: ***
I0821 15:17:03.537358 5424 daemon.cpp:218] OS physical memory 23.37 GB.
Process memory usage 1.49 GB, limit 18.69 GB, soft limit 16.83 GB. Sys
available memory 20.41 GB, low water mark 1.60 GB, warning water mark 3.20 GB.
Refresh interval memory growth 0 B
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int,
siginfo_t*, void*) at /root/src/doris/be/src/common/signal_handler.h:413
1# os::Linux::chained_handler(int, siginfo_t*, void*) in
/data/java/java-4.1.1.0/jre/lib/amd64/server/libjvm.so
2# JVM_handle_linux_signal in
/data/java/java-4.1.1.0/jre/lib/amd64/server/libjvm.so
3# signalHandler(int, siginfo_t*, void*) in
/data/java/java-4.1.1.0/jre/lib/amd64/server/libjvm.so
4# 0x7F9F962E3400 in /lib64/libc.so.6
5# bitdienc32 at /root/src/doris/be/src/clucene/src/ext/for/bitutil.c:384
6# p4nd1enc128v32 at /root/src/doris/be/src/clucene/src/ext/for/vp4c.c:414
7#
lucene::index::SDocumentsWriter::appendPostings(lucene::util::ArrayBase::ThreadState::FieldData*>*,
lucene::index::STermInfosWriter*, lucene::store::IndexOutput*,
lucene::store::IndexOutput*) in
/data/apache-doris-2.0.0-bin-x64-noavx2/be/lib/doris_be
8#
lucene::index::SDocumentsWriter::writeSegment(std::vector, std::allocator >,
std::allocator,
std::allocator > > >&) in
/data/apache-doris-2.0.0-bin-x64-noavx2/be/lib/doris_be
9# lucene::index::SDocumentsWriter::flush(bool) in
/data/apache-doris-2.0.0-bin-x64-noavx2/be/lib/doris_be
10# lucene::index::IndexWriter::doFlush(bool) in
/data/apache-doris-2.0.0-bin-x64-noavx2/be/lib/doris_be
11# lucene::index::IndexWriter::flush(bool, bool) in
/data/apache-doris-2.0.0-bin-x64-noavx2/be/lib/doris_be
12# lucene::index::IndexWriter::closeInternal(bool) in
/data/apache-doris-2.0.0-bin-x64-noavx2/be/lib/doris_be
13# lucene::index::IndexWriter::close(bool) in
/data/apache-doris-2.0.0-bin-x64-noavx2/be/lib/doris_be
14#
doris::segment_v2::InvertedIndexColumnWriterImpl<(doris::FieldType)26>::close()
at /root/src/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:110
15#
doris::segment_v2::InvertedIndexColumnWriterImpl<(doris::FieldType)26>::finish()
at /root/src/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:425
16# doris::segment_v2::ScalarColumnWriter::write_inverted_index() in
/data/apache-doris-2.0.0-bin-x64-noavx2/be/lib/doris_be
17# doris::segment_v2::SegmentWriter::finalize_columns_index(unsigned long*)
at /root/src/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:872
18# doris::segment_v2::SegmentWriter::finalize(unsigned long*, unsigned
long*) at /root/src/doris/be/
[GitHub] [doris] bobhan1 commented on pull request #22504: [feature](partial update) support partial update different columns for each row in one stream load
bobhan1 commented on PR #22504: URL: https://github.com/apache/doris/pull/22504#issuecomment-1685846923 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] starocean999 opened a new pull request, #23254: [fix](planner)avg function need support large int param
starocean999 opened a new pull request, #23254: URL: https://github.com/apache/doris/pull/23254 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly opened a new pull request, #23255: [refactor](nereids) convert session var name "beNumForTest"
englefly opened a new pull request, #23255: URL: https://github.com/apache/doris/pull/23255 ## Proposed changes this var is used for test only. So keep "for_test" as suffix. Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] starocean999 commented on pull request #23254: [fix](planner)avg function need support large int param
starocean999 commented on PR #23254: URL: https://github.com/apache/doris/pull/23254#issuecomment-1685847459 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly commented on pull request #23255: [refactor](nereids) convert session var name "beNumForTest"
englefly commented on PR #23255: URL: https://github.com/apache/doris/pull/23255#issuecomment-1685847509 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] TangSiyang2001 commented on pull request #22442: [feature](delete-predicate) support delete sub predicate v2
TangSiyang2001 commented on PR #22442: URL: https://github.com/apache/doris/pull/22442#issuecomment-1685847866 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 opened a new pull request, #23256: [pipelineX](projection) Support projection and blocking agg
Gabriel39 opened a new pull request, #23256: URL: https://github.com/apache/doris/pull/23256 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] XieJiann opened a new pull request, #23257: [feature](Nereids)add encryptKey
XieJiann opened a new pull request, #23257:
URL: https://github.com/apache/doris/pull/23257
## Proposed changes
add select key
```
- CREATE ENCRYPTKEY key_name AS "key_string"
- select key my_key
+-+
| encryptKeyRef('', 'my_key') |
+-+
| ABCD123456789 |
+-+
```
## Further comments
If this is a relatively large or complex change, kick off the discussion at
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you
chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zenoyang opened a new pull request, #23258: [fix](function) Fix to_bitmap int parameter causing BE crash
zenoyang opened a new pull request, #23258: URL: https://github.com/apache/doris/pull/23258 ## Proposed changes Issue Number: close #xxx fix #21236 ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] XieJiann commented on pull request #23257: [feature](Nereids) support `select key encryptKey`
XieJiann commented on PR #23257: URL: https://github.com/apache/doris/pull/23257#issuecomment-1685850942 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen opened a new pull request, #23259: [fix](case) comment out orthogonal_bitmap_intersect
hello-stephen opened a new pull request, #23259: URL: https://github.com/apache/doris/pull/23259 ## Proposed changes This case often fails in the internal test pipeline. After discussion, the related parts of orthogonal_bitmap_intersect are commented out first. ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] TangSiyang2001 commented on pull request #22442: [feature](delete-predicate) support delete sub predicate v2
TangSiyang2001 commented on PR #22442: URL: https://github.com/apache/doris/pull/22442#issuecomment-1685854387 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #23259: [fix](case) comment out orthogonal_bitmap_intersect
hello-stephen commented on PR #23259: URL: https://github.com/apache/doris/pull/23259#issuecomment-1685854487 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23245: [fix](hive) do not split compress data file
github-actions[bot] commented on PR #23245: URL: https://github.com/apache/doris/pull/23245#issuecomment-1685856087 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23246: [bugfix](vertical-compaction) Only can init the SegmentCacheHandle once
github-actions[bot] commented on PR #23246: URL: https://github.com/apache/doris/pull/23246#issuecomment-1685856864 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris-flink-connector] JNSimba opened a new pull request, #182: [fix] add batch option
JNSimba opened a new pull request, #182: URL: https://github.com/apache/doris-flink-connector/pull/182 ## Problem Summary: 1. Multi-table synchronization, increase the parameters of batch saving 2. Compatible with flinkcdcmysql2.3.0, first remove the property of SCAN_INCREMENTAL_CLOSE_IDLE_READER_ENABLED ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 3. Has unit tests been added: (Yes/No/No Need) 4. Has document been added or modified: (Yes/No/No Need) 5. Does it need to update dependencies: (Yes/No) 6. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 commented on pull request #23256: [pipelineX](projection) Support projection and blocking agg
Gabriel39 commented on PR #23256: URL: https://github.com/apache/doris/pull/23256#issuecomment-1685858840 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23255: [refactor](nereids) convert session var name "beNumForTest"
github-actions[bot] commented on PR #23255: URL: https://github.com/apache/doris/pull/23255#issuecomment-1685859429 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22442: [feature](delete-predicate) support delete sub predicate v2
github-actions[bot] commented on PR #22442: URL: https://github.com/apache/doris/pull/22442#issuecomment-1685860152 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22442: [feature](delete-predicate) support delete sub predicate v2
github-actions[bot] commented on PR #22442: URL: https://github.com/apache/doris/pull/22442#issuecomment-1685860754 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23254: [fix](planner)avg function need support large int param
github-actions[bot] commented on PR #23254: URL: https://github.com/apache/doris/pull/23254#issuecomment-1685861351 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23256: [pipelineX](projection) Support projection and blocking agg
github-actions[bot] commented on PR #23256: URL: https://github.com/apache/doris/pull/23256#issuecomment-1685861800 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xiaokang merged pull request #22996: [Chore](ColdHeatSeparation) change partition value for cold heat separation case
xiaokang merged PR #22996: URL: https://github.com/apache/doris/pull/22996 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Chore](ColdHeatSeparation) change partition value for cold heat separation case (#22996)
This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 32dc0250ba [Chore](ColdHeatSeparation) change partition value for cold
heat separation case (#22996)
32dc0250ba is described below
commit 32dc0250babf01e2a574ebe8e864302db91c15c7
Author: AlexYue
AuthorDate: Mon Aug 21 16:14:08 2023 +0800
[Chore](ColdHeatSeparation) change partition value for cold heat separation
case (#22996)
---
.../create_table_use_partition_policy.groovy| 4 ++--
.../cold_heat_separation_p2/modify_replica_use_partition.groovy | 6 +++---
2 files changed, 5 insertions(+), 5 deletions(-)
diff --git
a/regression-test/suites/cold_heat_separation_p2/create_table_use_partition_policy.groovy
b/regression-test/suites/cold_heat_separation_p2/create_table_use_partition_policy.groovy
index 3e9a8dd143..b059cc545f 100644
---
a/regression-test/suites/cold_heat_separation_p2/create_table_use_partition_policy.groovy
+++
b/regression-test/suites/cold_heat_separation_p2/create_table_use_partition_policy.groovy
@@ -181,7 +181,7 @@ suite("create_table_use_partition_policy") {
DUPLICATE KEY(L_ORDERKEY, L_PARTKEY, L_SUPPKEY, L_LINENUMBER)
PARTITION BY RANGE(`L_SHIPDATE`)
(
-PARTITION `p202301` VALUES LESS THAN ("2017-02-01")
("storage_policy" = "${policy_name}"),
+PARTITION `p202301` VALUES LESS THAN ("1995-12-01")
("storage_policy" = "${policy_name}"),
PARTITION `p202302` VALUES LESS THAN ("2017-03-01")
)
DISTRIBUTED BY HASH(L_ORDERKEY) BUCKETS 3
@@ -293,7 +293,7 @@ suite("create_table_use_partition_policy") {
DUPLICATE KEY(L_ORDERKEY, L_PARTKEY, L_SUPPKEY, L_LINENUMBER)
PARTITION BY RANGE(`L_SHIPDATE`)
(
-PARTITION `p202301` VALUES LESS THAN ("2017-02-01")
("storage_policy" = "${policy_name}"),
+PARTITION `p202301` VALUES LESS THAN ("1995-12-01")
("storage_policy" = "${policy_name}"),
PARTITION `p202302` VALUES LESS THAN ("2017-03-01")
)
DISTRIBUTED BY HASH(L_ORDERKEY) BUCKETS 3
diff --git
a/regression-test/suites/cold_heat_separation_p2/modify_replica_use_partition.groovy
b/regression-test/suites/cold_heat_separation_p2/modify_replica_use_partition.groovy
index fc6566bbdb..57b769c231 100644
---
a/regression-test/suites/cold_heat_separation_p2/modify_replica_use_partition.groovy
+++
b/regression-test/suites/cold_heat_separation_p2/modify_replica_use_partition.groovy
@@ -196,7 +196,7 @@ suite("modify_replica_use_partition") {
DUPLICATE KEY(L_ORDERKEY, L_PARTKEY, L_SUPPKEY, L_LINENUMBER)
PARTITION BY RANGE(`L_SHIPDATE`)
(
-PARTITION `p202301` VALUES LESS THAN ("2017-02-01"),
+PARTITION `p202301` VALUES LESS THAN ("1995-12-01"),
PARTITION `p202302` VALUES LESS THAN ("2017-03-01")
)
DISTRIBUTED BY HASH(L_ORDERKEY) BUCKETS 3
@@ -307,7 +307,7 @@ suite("modify_replica_use_partition") {
DUPLICATE KEY(L_ORDERKEY, L_PARTKEY, L_SUPPKEY, L_LINENUMBER)
PARTITION BY RANGE(`L_SHIPDATE`)
(
-PARTITION `p202301` VALUES LESS THAN ("2017-02-01"),
+PARTITION `p202301` VALUES LESS THAN ("1995-12-01"),
PARTITION `p202302` VALUES LESS THAN ("2017-03-01")
)
DISTRIBUTED BY HASH(L_ORDERKEY) BUCKETS 3
@@ -401,7 +401,7 @@ suite("modify_replica_use_partition") {
DUPLICATE KEY(L_ORDERKEY, L_PARTKEY, L_SUPPKEY, L_LINENUMBER)
PARTITION BY RANGE(`L_SHIPDATE`)
(
-PARTITION `p202301` VALUES LESS THAN ("2017-02-01")
("replication_num" = "3"),
+PARTITION `p202301` VALUES LESS THAN ("1995-12-01")
("replication_num" = "3"),
PARTITION `p202302` VALUES LESS THAN ("2017-03-01")
("replication_num" = "1")
)
DISTRIBUTED BY HASH(L_ORDERKEY) BUCKETS 3
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] TangSiyang2001 commented on pull request #23234: [fix](session-var) fix forward parallel fragment instance num
TangSiyang2001 commented on PR #23234: URL: https://github.com/apache/doris/pull/23234#issuecomment-1685864578 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23258: [fix](function) Fix to_bitmap int parameter causing BE crash
github-actions[bot] commented on PR #23258: URL: https://github.com/apache/doris/pull/23258#issuecomment-1685865610 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23256: [pipelineX](projection) Support projection and blocking agg
github-actions[bot] commented on PR #23256: URL: https://github.com/apache/doris/pull/23256#issuecomment-1685866398 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] XieJiann commented on pull request #23104: [enhancement](Nereids): add char/covert func
XieJiann commented on PR #23104: URL: https://github.com/apache/doris/pull/23104#issuecomment-1685866427 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Jibing-Li commented on pull request #23170: [improvement](statistics)Load the cache for external table row count while init table.
Jibing-Li commented on PR #23170: URL: https://github.com/apache/doris/pull/23170#issuecomment-1685868347 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-flink-connector] branch master updated (2e7d791 -> c51df80)
This is an automated email from the ASF dual-hosted git repository. zykkk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris-flink-connector.git from 2e7d791 [fix] fix batch sink flush lost batch (#180) add c51df80 [fix] add batch option (#182) No new revisions were added by this update. Summary of changes: .../java/org/apache/doris/flink/tools/cdc/DatabaseSync.java| 10 ++ .../apache/doris/flink/tools/cdc/mysql/MysqlDatabaseSync.java | 5 - 2 files changed, 14 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris-flink-connector] zy-kkk merged pull request #182: [fix] add batch option
zy-kkk merged PR #182: URL: https://github.com/apache/doris-flink-connector/pull/182 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22829: [feature](insert) Support group commit insert
hello-stephen commented on PR #22829: URL: https://github.com/apache/doris/pull/22829#issuecomment-1685872992 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 48.56 seconds stream load tsv: 552 seconds loaded 74807831229 Bytes, about 129 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 31.3 seconds inserted 1000 Rows, about 319K ops/s storage size: 17162165047 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zenoyang commented on pull request #23258: [fix](function) Fix to_bitmap int parameter causing BE crash
zenoyang commented on PR #23258: URL: https://github.com/apache/doris/pull/23258#issuecomment-1685876562 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] kaka11chen opened a new pull request, #23260: [Fix](multi-catalog) Fix zlib init error by using doris's zlib shared library.
kaka11chen opened a new pull request, #23260: URL: https://github.com/apache/doris/pull/23260 ## Proposed changes ### Issue ``` W0814 20:33:44.143478 427556 scanner_scheduler.cpp:377] Scan thread read VScanner failed: [INTERNAL_ERROR]InternalError: null W0814 20:33:44.505172 427550 jni-util.cpp:239] java.lang.InternalError at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.init(Native Method) at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.(ZlibDecompressor.java:114) at org.apache.hadoop.io.compress.GzipCodec$GzipZlibDecompressor.(GzipCodec.java:229) at org.apache.hadoop.io.compress.GzipCodec.createDecompressor(GzipCodec.java:188) at org.apache.hadoop.io.compress.CodecPool.getDecompressor(CodecPool.java:183) at org.apache.parquet.hadoop.CodecFactory$HeapBytesDecompressor.(CodecFactory.java:99) at org.apache.parquet.hadoop.CodecFactory.createDecompressor(CodecFactory.java:223) at org.apache.parquet.hadoop.CodecFactory.getDecompressor(CodecFactory.java:212) at org.apache.parquet.hadoop.CodecFactory.getDecompressor(CodecFactory.java:43) at org.apache.parquet.hadoop.ParquetFileReader$Chunk.readAllPages(ParquetFileReader.java:1655) at org.apache.parquet.hadoop.ParquetFileReader$Chunk.readAllPages(ParquetFileReader.java:1538) at org.apache.parquet.hadoop.ParquetFileReader.readChunkPages(ParquetFileReader.java:1148) ``` - Fix zlib init error by using doris's zlib shared library. This error occurs when `ZlibDecompressor` loads the zlib shared library via `dlopen()` and calls the `inflateInit2_()` function. It will return error code `-2`. Through testing, it is found that this may be because we have also linked the zlib library, resulting in different zlib library versions. And this situation only occurs under clang after testing. Anyway, we use a version to solve the current problem. - Fix the problem that the java log does not output in some environment's. The root cause is `$HADOOP_CONF_DIR` has log4j.properties, it will override commons jars's log4j.properties. ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] kaka11chen commented on pull request #23260: [Fix](multi-catalog) Fix zlib init error by using doris's zlib shared library and `jni.log` does not output.
kaka11chen commented on PR #23260: URL: https://github.com/apache/doris/pull/23260#issuecomment-1685879064 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #23121: [Feature](Nereids)support insert into values for Nereids
sohardforaname commented on PR #23121: URL: https://github.com/apache/doris/pull/23121#issuecomment-1685879796 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #23253: [fix](ut) fix "beNumber not found" bug
hello-stephen commented on PR #23253: URL: https://github.com/apache/doris/pull/23253#issuecomment-1685883704 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.96 seconds stream load tsv: 548 seconds loaded 74807831229 Bytes, about 130 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.5 seconds inserted 1000 Rows, about 338K ops/s storage size: 17164397260 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Lchangliang commented on pull request #23246: [bugfix](vertical-compaction) Only can init the SegmentCacheHandle once
Lchangliang commented on PR #23246: URL: https://github.com/apache/doris/pull/23246#issuecomment-1685886101 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhiqiang-hhhh closed pull request #23134: [refactor](pipeline) small refactor of pipeline execution on be.
zhiqiang- closed pull request #23134: [refactor](pipeline) small refactor of pipeline execution on be. URL: https://github.com/apache/doris/pull/23134 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhiqiang-hhhh commented on pull request #23134: [refactor](pipeline) small refactor of pipeline execution on be.
zhiqiang- commented on PR #23134: URL: https://github.com/apache/doris/pull/23134#issuecomment-1685886979 Closed. Too many conflicts and pipeline execution is almost refactored. Will continue to do refactor after new code is steady. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23198: [fix](parquet) parquet reader confuses logical/physical/slot id of columns
github-actions[bot] commented on PR #23198: URL: https://github.com/apache/doris/pull/23198#issuecomment-1685893351 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener commented on pull request #22944: [enhancement](Nereids): optimize deepEquals
jackwener commented on PR #22944: URL: https://github.com/apache/doris/pull/22944#issuecomment-1685894163 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #23246: [bugfix](vertical-compaction) Only can init the SegmentCacheHandle once
github-actions[bot] commented on PR #23246: URL: https://github.com/apache/doris/pull/23246#issuecomment-1685894542 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22970: [feature](partial update) Support partial update for broker load
hello-stephen commented on PR #22970: URL: https://github.com/apache/doris/pull/22970#issuecomment-168597 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.68 seconds stream load tsv: 541 seconds loaded 74807831229 Bytes, about 131 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.6 seconds inserted 1000 Rows, about 337K ops/s storage size: 17162066464 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #23250: [FIX](map)fix column map for offset next_array_item_rowid order
hello-stephen commented on PR #23250: URL: https://github.com/apache/doris/pull/23250#issuecomment-1685901253 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 48.84 seconds stream load tsv: 555 seconds loaded 74807831229 Bytes, about 128 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.9 seconds inserted 1000 Rows, about 334K ops/s storage size: 17162081593 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
