[doris] branch master updated: [Bug](point query) Reusable in PointQueryExecutor should call init before add to LookupCache (#16489)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new d956cb13af [Bug](point query) Reusable in PointQueryExecutor should
call init before add to LookupCache (#16489)
d956cb13af is described below
commit d956cb13af0e8da5c4419b3b1c17e4f1d08e26f0
Author: lihangyu <15605149...@163.com>
AuthorDate: Wed Feb 8 16:05:59 2023 +0800
[Bug](point query) Reusable in PointQueryExecutor should call init before
add to LookupCache (#16489)
Otherwise in high concurrent query, _block_pool maybe used before
Reusable::init done in other threads
---
be/src/service/point_query_executor.cpp | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/be/src/service/point_query_executor.cpp
b/be/src/service/point_query_executor.cpp
index ba17e8fc17..9c70e08061 100644
--- a/be/src/service/point_query_executor.cpp
+++ b/be/src/service/point_query_executor.cpp
@@ -63,6 +63,7 @@ std::unique_ptr Reusable::get_block() {
return std::make_unique(tuple_desc()->slots(), 4);
}
auto block = std::move(_block_pool.back());
+CHECK(block != nullptr);
_block_pool.pop_back();
return block;
}
@@ -149,9 +150,9 @@ Status PointQueryExecutor::init(const
PTabletKeyLookupRequest* request,
&t_output_exprs));
_reusable = reusable_ptr;
if (uuid != 0) {
-LookupCache::instance().add(uuid, reusable_ptr);
// could be reused by requests after, pre allocte more blocks
RETURN_IF_ERROR(reusable_ptr->init(t_desc_tbl,
t_output_exprs.exprs, 128));
+LookupCache::instance().add(uuid, reusable_ptr);
} else {
RETURN_IF_ERROR(reusable_ptr->init(t_desc_tbl,
t_output_exprs.exprs, 1));
}
@@ -164,7 +165,7 @@ Status PointQueryExecutor::init(const
PTabletKeyLookupRequest* request,
}
RETURN_IF_ERROR(_init_keys(request));
_result_block = _reusable->get_block();
-DCHECK(_result_block != nullptr);
+CHECK(_result_block != nullptr);
return Status::OK();
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataroaring merged pull request #16489: [Bug](point query) Reusable in PointQueryExecutor should call init b…
dataroaring merged PR #16489: URL: https://github.com/apache/doris/pull/16489 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch struct-map-type updated (7049decb7a -> 83a81944ac)
This is an automated email from the ASF dual-hosted git repository. xuyang pushed a change to branch struct-map-type in repository https://gitbox.apache.org/repos/asf/doris.git from 7049decb7a [fix] fix code for comment add 83a81944ac [fix] fix fe ut No new revisions were added by this update. Summary of changes: fe/fe-core/src/main/java/org/apache/doris/analysis/Expr.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhangstar333 commented on pull request #16497: [bug](udf) fix udf return type of decimal check scale must is 9
zhangstar333 commented on PR #16497: URL: https://github.com/apache/doris/pull/16497#issuecomment-1422203242 > What is the effect of this change? Not report error？ remove the check, and set scale to 9 always -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhangstar333 commented on a diff in pull request #16496: [bug](jdbc) fix jdbc can't get object of PGobject
zhangstar333 commented on code in PR #16496:
URL: https://github.com/apache/doris/pull/16496#discussion_r1099795312
##
regression-test/suites/jdbc_p0/test_jdbc_query_mysql.groovy:
##
@@ -934,14 +935,14 @@ suite("test_jdbc_query_mysql", "p0") {
order_qt_sql111 """ SELECT rank() OVER () FROM (SELECT k8 FROM
$jdbcMysql57Table1 LIMIT 10) as t LIMIT 3 """
order_qt_sql112 """ SELECT k7, count(DISTINCT k8) FROM
$jdbcMysql57Table1 WHERE k8 > 110 GROUP BY GROUPING SETS ((), (k7)) """
-
+// test this action result need restart?
// test alter resource
-sql """alter resource $jdbcResourceMysql57 properties("password" =
"1234567")"""
-test {
-sql """select count(*) from $jdbcMysql57Table1"""
-exception "Access denied for user"
-}
-sql """alter resource $jdbcResourceMysql57 properties("password" =
"123456")"""
+// sql """alter resource $jdbcResourceMysql57 properties("password" =
"1234567")"""
Review Comment:
after alter resource properties, but can't get result of the exception
"Access denied for user". but when have restart I get this result
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei opened a new pull request, #16512: [regression-test](storage) add fuzzy variables for storage
yiguolei opened a new pull request, #16512: URL: https://github.com/apache/doris/pull/16512 # Proposed changes 1. fuzzy table properties like enable_light_weight_schema_change, auto_compaction, auto_bucket, merge_on_read or merge on write. 2. change segment_compaction default to true; 3. fuzzy segment_compaction in BE.conf. ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 4. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 5. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 6. Does it need to update dependencies: - [ ] Yes - [ ] No 7. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16512: [regression-test](storage) add fuzzy variables for storage
github-actions[bot] commented on PR #16512: URL: https://github.com/apache/doris/pull/16512#issuecomment-1422214700 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] freemandealer commented on a diff in pull request #16343: [Enhancement](Stmt) Set insert_into timeout session variable separately
freemandealer commented on code in PR #16343:
URL: https://github.com/apache/doris/pull/16343#discussion_r1099809607
##
be/src/runtime/runtime_state.h:
##
@@ -86,6 +86,8 @@ class RuntimeState {
}
int max_errors() const { return _query_options.max_errors; }
int query_timeout() const { return _query_options.query_timeout; }
+int insert_timeout() const { return _query_options.insert_timeout; }
Review Comment:
Great. Just keep it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #16493: [improvement](fuzzy) print fuzzy session variable in FE audit log
yiguolei merged PR #16493: URL: https://github.com/apache/doris/pull/16493 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (d956cb13af -> b06e6b25c9)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from d956cb13af [Bug](point query) Reusable in PointQueryExecutor should call init before add to LookupCache (#16489) add b06e6b25c9 [improvement](fuzzy) print fuzzy session variable in FE audit log (#16493) No new revisions were added by this update. Summary of changes: .../java/org/apache/doris/plugin/AuditEvent.java | 7 +++ .../java/org/apache/doris/qe/ConnectProcessor.java | 3 +- .../java/org/apache/doris/qe/SessionVariable.java | 53 -- .../main/java/org/apache/doris/qe/VariableMgr.java | 3 ++ 4 files changed, 50 insertions(+), 16 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] cambyzju commented on a diff in pull request #16485: [improvement](filecache) use dynamic segment size to cache remote file block
cambyzju commented on code in PR #16485:
URL: https://github.com/apache/doris/pull/16485#discussion_r1099818062
##
be/src/io/cache/block/cached_remote_file_reader.cpp:
##
@@ -44,12 +44,13 @@ Status CachedRemoteFileReader::close() {
std::pair CachedRemoteFileReader::_align_size(size_t offset,
size_t
read_size) const {
+size_t segment_size = std::min(std::max(read_size, (size_t)4096), // 4k
+
(size_t)config::file_cache_max_file_segment_size);
Review Comment:
it is better to add a CONF_Validator(>=4096) for
`file_cache_max_file_segment_size`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16466: [Pipeline][WIP] Introduce pipeline tracing
github-actions[bot] commented on PR #16466:
URL: https://github.com/apache/doris/pull/16466#issuecomment-1422237333
`sh-checker report`
To get the full details, please check in the
[job]("https://github.com/apache/doris/actions/runs/4122321335";) output.
shellcheck errors
```
'shellcheck ' found no issues.
```
shfmt errors
```
'shfmt ' returned error 1 finding the following formatting issues:
--
--- build.sh.orig
+++ build.sh
@@ -332,7 +332,7 @@
fi
if [[ -z "${ENABLE_QUERY_DEBUG_TRACE}" ]]; then
-ENABLE_QUERY_DEBUG_TRACE='OFF'
+ENABLE_QUERY_DEBUG_TRACE='OFF'
fi
echo "Get params:
--
You can reformat the above files to meet shfmt's requirements by typing:
shfmt -w filename
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman merged pull request #16486: [Fix](multi catalog)(planner) Fix external table statistic collection bug.
morningman merged PR #16486: URL: https://github.com/apache/doris/pull/16486 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Fix](multi catalog)(planner) Fix external table statistic collection bug (#16486)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 666f7096f2 [Fix](multi catalog)(planner) Fix external table statistic
collection bug (#16486)
666f7096f2 is described below
commit 666f7096f21a882a23a7c73542dc598b0ad9270b
Author: Jibing-Li <64681310+jibing...@users.noreply.github.com>
AuthorDate: Wed Feb 8 16:51:30 2023 +0800
[Fix](multi catalog)(planner) Fix external table statistic collection bug
(#16486)
Add index id to column statistic id. Refresh statistic cache after analyze.
---
.../doris/catalog/external/HMSExternalTable.java| 1 +
.../apache/doris/statistics/HiveAnalysisTask.java | 21 +++--
2 files changed, 20 insertions(+), 2 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/catalog/external/HMSExternalTable.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/external/HMSExternalTable.java
index 4d4fc310bd..1a558e1bd9 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/catalog/external/HMSExternalTable.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/catalog/external/HMSExternalTable.java
@@ -294,6 +294,7 @@ public class HMSExternalTable extends ExternalTable {
@Override
public List initSchema() {
+makeSureInitialized();
List columns;
List schema = ((HMSExternalCatalog)
catalog).getClient().getSchema(dbName, name);
if (dlaType.equals(DLAType.ICEBERG)) {
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/statistics/HiveAnalysisTask.java
b/fe/fe-core/src/main/java/org/apache/doris/statistics/HiveAnalysisTask.java
index d22e2abe78..c0ca6df80c 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/statistics/HiveAnalysisTask.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/statistics/HiveAnalysisTask.java
@@ -17,6 +17,7 @@
package org.apache.doris.statistics;
+import org.apache.doris.catalog.Env;
import org.apache.doris.common.FeConstants;
import org.apache.doris.datasource.HMSExternalCatalog;
import org.apache.doris.qe.AutoCloseConnectContext;
@@ -54,6 +55,7 @@ public class HiveAnalysisTask extends HMSAnalysisTask {
public static final String NUM_ROWS = "numRows";
public static final String NUM_FILES = "numFiles";
public static final String TIMESTAMP = "transient_lastDdlTime";
+public static final String DELIMITER = "-";
public HiveAnalysisTask(AnalysisTaskScheduler analysisTaskScheduler,
AnalysisTaskInfo info) {
super(analysisTaskScheduler, info);
@@ -85,7 +87,7 @@ public class HiveAnalysisTask extends HMSAnalysisTask {
Map parameters =
table.getRemoteTable().getParameters();
// Collect table level row count, null number and timestamp.
setParameterData(parameters, params);
-params.put("id", String.valueOf(tbl.getId()) + "-" +
String.valueOf(col.getName()));
+params.put("id", genColumnStatId(tbl.getId(), -1, col.getName(),
null));
List tableStats =
table.getHiveTableColumnStats(columns);
// Collect table level ndv, nulls, min and max. tableStats contains at
most 1 item;
for (ColumnStatisticsObj tableStat : tableStats) {
@@ -117,7 +119,7 @@ public class HiveAnalysisTask extends HMSAnalysisTask {
parameters = partition.getParameters();
// Collect row count, null number and timestamp.
setParameterData(parameters, params);
-params.put("id", String.valueOf(tbl.getId()) + "-" +
String.valueOf(col.getName()) + "-" + partName);
+params.put("id", genColumnStatId(tbl.getId(), -1, col.getName(),
partName));
params.put("partId", partName);
List value = entry.getValue();
Preconditions.checkState(value.size() == 1);
@@ -138,6 +140,7 @@ public class HiveAnalysisTask extends HMSAnalysisTask {
this.stmtExecutor.execute();
}
}
+Env.getCurrentEnv().getStatisticsCache().refreshSync(tbl.getId(), -1,
col.getName());
}
private void getStatData(ColumnStatisticsData data, Map
params) {
@@ -226,4 +229,18 @@ public class HiveAnalysisTask extends HMSAnalysisTask {
SimpleDateFormat sdf = new SimpleDateFormat("-MM-dd HH:mm:ss");
params.put("update_time", sdf.format(new Date(timestamp * 1000)));
}
+
+private String genColumnStatId(long tableId, long indexId, String
columnName, String partitionName) {
+StringBuilder sb = new StringBuilder();
+sb.append(tableId);
+sb.append(DELIMITER);
+sb.append(indexId);
+sb.append(DELIMITER);
+sb.append(columnName);
+if (partitionName != null) {
+sb.append(DELIMITER);
+sb.append(partitionName);
+}
+return sb.toString();
+}
}
--
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #16466: [Pipeline][WIP] Introduce pipeline tracing
github-actions[bot] commented on code in PR #16466:
URL: https://github.com/apache/doris/pull/16466#discussion_r1099833918
##
be/src/util/debug/tracing.cpp:
##
@@ -0,0 +1,209 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
+#include
+#include
+#include
+#include
+
+#include "common/config.h"
+#include "pipeline/pipeline_task.h"
+#include "fmt/printf.h"
+#include "util/debug/tracing.h"
+#include "util/time.h"
+
+namespace doris::debug {
+
+QueryTraceEvent QueryTraceEvent::create(const std::string& name, const
std::string& category, int64_t id, char phase,
+int64_t timestamp, int64_t duration,
int64_t instance_id, pipeline::PipelineTaskRawPtr task,
+std::vector>&& args) {
+QueryTraceEvent event;
+event.name = name;
+event.category = category;
+event.id = id;
+event.phase = phase;
+event.start_time = timestamp;
+event.duration = duration;
+event.instance_id = instance_id;
+event.task = task;
+event.args = std::move(args);
+event.thread_id = std::this_thread::get_id();
+return event;
+}
+
+QueryTraceEvent QueryTraceEvent::create_with_ctx(const std::string& name,
const std::string& category, int64_t id,
+ char phase, const
QueryTraceContext& ctx) {
+return create(name, category, id, phase, MonotonicMicros() - ctx.start_ts,
QueryTraceContext::DEFAULT_EVENT_ID,
+ ctx.fragment_instance_id, ctx.task, {});
+}
+
+QueryTraceEvent QueryTraceEvent::create_with_ctx(const std::string& name,
const std::string& category, int64_t id,
+ char phase, int64_t start_ts,
int64_t duration,
+ const QueryTraceContext& ctx)
{
+return create(name, category, id, phase, start_ts, duration,
ctx.fragment_instance_id, ctx.task, {});
+}
+
+static const char* kSimpleEventFormat =
+
R"({"cat":"%s","name":"%s","pid":"0x%x","tid":"0x%x","id":"0x%x","ts":%ld,"ph":"%c","args":%s,"tidx":"0x%x"})";
+static const char* kCompleteEventFormat =
+
R"({"cat":"%s","name":"%s","pid":"0x%x","tid":"0x%x","id":"0x%x","ts":%ld,"dur":%ld,"ph":"%c","args":%s,"tidx":"0x%x"})";
+
+std::string QueryTraceEvent::to_string() {
+std::string args_str = args_to_string();
+size_t tidx = std::hash{}(thread_id);
+
+// transform a raw pointer to uint64_t by re_cast is maybe not the best
way, but at least unambiguous & safe.
+if (phase == 'X') {
+return fmt::sprintf(kCompleteEventFormat, category.c_str(),
name.c_str(), (uint64_t)instance_id,
+reinterpret_cast(task), id, start_time,
duration, phase, args_str.c_str(), tidx);
+} else {
+return fmt::sprintf(kSimpleEventFormat, category.c_str(),
name.c_str(), (uint64_t)instance_id,
+reinterpret_cast(task), id, start_time,
phase, args_str.c_str(), tidx);
+}
+}
+
+std::string QueryTraceEvent::args_to_string() {
+if (args.empty()) {
+return "{}";
+}
+std::ostringstream oss;
+oss << "{";
+oss << fmt::sprintf(R"("%s":"%s")", args[0].first.c_str(),
args[0].second.c_str());
+for (size_t i = 1; i < args.size(); i++) {
+oss << fmt::sprintf(R"(,"%s":"%s")", args[i].first.c_str(),
args[i].second.c_str());
+}
+oss << "}";
+return oss.str();
+}
+
+void EventBuffer::add(QueryTraceEvent&& event) {
+std::lock_guard l(_mutex);
+_buffer.emplace_back(std::move(event));
+}
+
+#ifdef ENABLE_QUERY_DEBUG_TRACE
+QueryTrace::QueryTrace(const TUniqueId& query_id, bool is_enable) :
_query_id(query_id), _is_enable(is_enable) {
+if (_is_enable) {
+_start_ts = MonotonicMicros();
+}
+}
+#else
+QueryTrace::QueryTrace(const TUniqueId& query_id, bool is_enable) {}
+#endif
+
+void QueryTrace::register_tasks(const TUniqueId& fragment_instance_id,
pipeline::PipelineTasks& tasks) {
+#ifdef ENABLE_QUERY_DEBUG_TRACE
+if (!_is_enable) {
+return;
+}
+std::unique_lock l(_mutex);
+auto iter = _fragment_tasks.find(fragment_instance_id);
+if (iter == _fragment_ta
[GitHub] [doris] hello-stephen commented on pull request #16509: [Enhancement](storage) add a new hidden column __DORIS_VERSION_COL__ for unique key table
hello-stephen commented on PR #16509: URL: https://github.com/apache/doris/pull/16509#issuecomment-1422252282 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.75 seconds load time: 470 seconds storage size: 17170855258 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230208085916_clickbench_pr_92089.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] SaintBacchus opened a new pull request, #16513: [UT-Fix](MTMV) Fix MTMV FE UT bugs
SaintBacchus opened a new pull request, #16513: URL: https://github.com/apache/doris/pull/16513 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] YangShaw commented on a diff in pull request #14397: [feature](nereids)support window function
YangShaw commented on code in PR #14397:
URL: https://github.com/apache/doris/pull/14397#discussion_r1099843860
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/implementation/LogicalWindowToPhysicalWindow.java:
##
@@ -0,0 +1,539 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.implementation;
+
+import org.apache.doris.nereids.CascadesContext;
+import org.apache.doris.nereids.annotation.DependsRules;
+import org.apache.doris.nereids.properties.DistributionSpecHash;
+import org.apache.doris.nereids.properties.OrderKey;
+import org.apache.doris.nereids.properties.OrderSpec;
+import org.apache.doris.nereids.properties.PhysicalProperties;
+import org.apache.doris.nereids.properties.RequireProperties;
+import org.apache.doris.nereids.rules.Rule;
+import org.apache.doris.nereids.rules.RuleType;
+import
org.apache.doris.nereids.rules.rewrite.logical.CheckAndStandardizeWindowFunctionAndFrame;
+import org.apache.doris.nereids.rules.rewrite.logical.ExtractWindowExpression;
+import org.apache.doris.nereids.rules.rewrite.logical.NormalizeWindow;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.NamedExpression;
+import org.apache.doris.nereids.trees.expressions.Window;
+import org.apache.doris.nereids.trees.expressions.WindowFrame;
+import org.apache.doris.nereids.trees.plans.GroupPlan;
+import org.apache.doris.nereids.trees.plans.Plan;
+import org.apache.doris.nereids.trees.plans.logical.LogicalWindow;
+import org.apache.doris.nereids.trees.plans.physical.PhysicalWindow;
+
+import com.google.common.collect.Lists;
+import org.apache.commons.collections.CollectionUtils;
+
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation rule that convert logical window to physical window, and add
RequiredProperties
+ *
+ * step 1: compute three kinds of group:
+ * WindowFrameGroup: maintain windows with same PartitionKey, OrderKey
and WindowFrame
+ * OrderKeyGroup: maintain windows with same PartitionKey and OrderKey
+ * PartitionKeyGroup: maintain windows with same PartitionKey
+ * step 2: sort PartitionKeyGroup with increasing order of tupleSize
+ * step 3: for every WindowFrameGroup of each SortGroup, generate one
PhysicalWindow node, with common PartitionKeys,
+ * OrderKeys, unique WindowFrame and a function list.
+ * step 4: for each PhysicalWindow, generate RequiredProperties, including
PartitionKey for DistributionSpec,
+ * and (PartitionKey + OrderKey) for OrderSpec.
+ */
+@DependsRules({
+NormalizeWindow.class,
+CheckAndStandardizeWindowFunctionAndFrame.class,
+ExtractWindowExpression.class
+})
+public class LogicalWindowToPhysicalWindow extends
OneImplementationRuleFactory {
+
+@Override
+public Rule build() {
+
+return RuleType.LOGICAL_WINDOW_TO_PHYSICAL_WINDOW_RULE.build(
+logicalWindow().when(LogicalWindow::isNormalized)
+.when(LogicalWindow::isChecked)
+.thenApply(ctx -> resolveWindow(ctx.cascadesContext, ctx.root))
+);
+}
+
+/**
+ * main procedure
+ */
+private PhysicalWindow resolveWindow(CascadesContext ctx,
LogicalWindow logicalWindow) {
+// todo: remove windowExpressions from LogicalWindow and rule
ExtractWindowExpressions.
+// Only add this variable in PhysicalWindow
+List windowList =
logicalWindow.getWindowExpressions();
+
+/// create three kinds of groups and compute tupleSize of each
+// windowFrameGroup
+List windowFrameGroupList =
createCommonWindowFrameGroups(windowList);
+
+// orderKeyGroup
+List orderKeyGroupList =
createCommonOrderKeyGroups(windowFrameGroupList);
+mergeOrderKeyGroups(orderKeyGroupList);
+
+// partitionKeyGroup
+List partitionKeyGroupList =
createCommonPartitionKeyGroups(orderKeyGroupList);
+mergePartitionKeyGroups(partitionKeyGroupList);
+
+// sort groups
+sortPartitionKeyGroups(partitionKeyGroupList);
+
+// eliminate LogicalWindow, and replaced it with new LogicalWindow and
LogicalSort
+Plan newRoo
[GitHub] [doris] YangShaw commented on a diff in pull request #14397: [feature](nereids)support window function
YangShaw commented on code in PR #14397:
URL: https://github.com/apache/doris/pull/14397#discussion_r1099844627
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/implementation/LogicalWindowToPhysicalWindow.java:
##
@@ -0,0 +1,539 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.implementation;
+
+import org.apache.doris.nereids.CascadesContext;
+import org.apache.doris.nereids.annotation.DependsRules;
+import org.apache.doris.nereids.properties.DistributionSpecHash;
+import org.apache.doris.nereids.properties.OrderKey;
+import org.apache.doris.nereids.properties.OrderSpec;
+import org.apache.doris.nereids.properties.PhysicalProperties;
+import org.apache.doris.nereids.properties.RequireProperties;
+import org.apache.doris.nereids.rules.Rule;
+import org.apache.doris.nereids.rules.RuleType;
+import
org.apache.doris.nereids.rules.rewrite.logical.CheckAndStandardizeWindowFunctionAndFrame;
+import org.apache.doris.nereids.rules.rewrite.logical.ExtractWindowExpression;
+import org.apache.doris.nereids.rules.rewrite.logical.NormalizeWindow;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.NamedExpression;
+import org.apache.doris.nereids.trees.expressions.Window;
+import org.apache.doris.nereids.trees.expressions.WindowFrame;
+import org.apache.doris.nereids.trees.plans.GroupPlan;
+import org.apache.doris.nereids.trees.plans.Plan;
+import org.apache.doris.nereids.trees.plans.logical.LogicalWindow;
+import org.apache.doris.nereids.trees.plans.physical.PhysicalWindow;
+
+import com.google.common.collect.Lists;
+import org.apache.commons.collections.CollectionUtils;
+
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation rule that convert logical window to physical window, and add
RequiredProperties
+ *
+ * step 1: compute three kinds of group:
+ * WindowFrameGroup: maintain windows with same PartitionKey, OrderKey
and WindowFrame
+ * OrderKeyGroup: maintain windows with same PartitionKey and OrderKey
+ * PartitionKeyGroup: maintain windows with same PartitionKey
+ * step 2: sort PartitionKeyGroup with increasing order of tupleSize
+ * step 3: for every WindowFrameGroup of each SortGroup, generate one
PhysicalWindow node, with common PartitionKeys,
+ * OrderKeys, unique WindowFrame and a function list.
+ * step 4: for each PhysicalWindow, generate RequiredProperties, including
PartitionKey for DistributionSpec,
+ * and (PartitionKey + OrderKey) for OrderSpec.
+ */
+@DependsRules({
+NormalizeWindow.class,
+CheckAndStandardizeWindowFunctionAndFrame.class,
+ExtractWindowExpression.class
+})
+public class LogicalWindowToPhysicalWindow extends
OneImplementationRuleFactory {
+
+@Override
+public Rule build() {
+
+return RuleType.LOGICAL_WINDOW_TO_PHYSICAL_WINDOW_RULE.build(
+logicalWindow().when(LogicalWindow::isNormalized)
+.when(LogicalWindow::isChecked)
+.thenApply(ctx -> resolveWindow(ctx.cascadesContext, ctx.root))
+);
+}
+
+/**
+ * main procedure
+ */
+private PhysicalWindow resolveWindow(CascadesContext ctx,
LogicalWindow logicalWindow) {
+// todo: remove windowExpressions from LogicalWindow and rule
ExtractWindowExpressions.
+// Only add this variable in PhysicalWindow
+List windowList =
logicalWindow.getWindowExpressions();
+
+/// create three kinds of groups and compute tupleSize of each
+// windowFrameGroup
+List windowFrameGroupList =
createCommonWindowFrameGroups(windowList);
+
+// orderKeyGroup
+List orderKeyGroupList =
createCommonOrderKeyGroups(windowFrameGroupList);
+mergeOrderKeyGroups(orderKeyGroupList);
+
+// partitionKeyGroup
+List partitionKeyGroupList =
createCommonPartitionKeyGroups(orderKeyGroupList);
+mergePartitionKeyGroups(partitionKeyGroupList);
+
+// sort groups
+sortPartitionKeyGroups(partitionKeyGroupList);
+
+// eliminate LogicalWindow, and replaced it with new LogicalWindow and
LogicalSort
+Plan newRoo
[GitHub] [doris] github-actions[bot] commented on pull request #16485: [improvement](filecache) use dynamic segment size to cache remote file block
github-actions[bot] commented on PR #16485: URL: https://github.com/apache/doris/pull/16485#issuecomment-1422264027 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16485: [improvement](filecache) use dynamic segment size to cache remote file block
github-actions[bot] commented on PR #16485: URL: https://github.com/apache/doris/pull/16485#issuecomment-1422269236 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] AshinGau commented on a diff in pull request #16485: [improvement](filecache) use dynamic segment size to cache remote file block
AshinGau commented on code in PR #16485:
URL: https://github.com/apache/doris/pull/16485#discussion_r1099861957
##
be/src/io/cache/block/cached_remote_file_reader.cpp:
##
@@ -44,12 +44,13 @@ Status CachedRemoteFileReader::close() {
std::pair CachedRemoteFileReader::_align_size(size_t offset,
size_t
read_size) const {
+size_t segment_size = std::min(std::max(read_size, (size_t)4096), // 4k
+
(size_t)config::file_cache_max_file_segment_size);
Review Comment:
thanks, done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16485: [improvement](filecache) use dynamic segment size to cache remote file block
github-actions[bot] commented on PR #16485: URL: https://github.com/apache/doris/pull/16485#issuecomment-1422288464 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (666f7096f2 -> f71fc3291f)
This is an automated email from the ASF dual-hosted git repository. gabriellee pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 666f7096f2 [Fix](multi catalog)(planner) Fix external table statistic collection bug (#16486) add f71fc3291f [Bug](fix) right anti join error result when batch size is low (#16510) No new revisions were added by this update. Summary of changes: be/src/vec/exec/join/vnested_loop_join_node.cpp | 2 +- regression-test/suites/query_p0/join/test_join.groovy | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 merged pull request #16510: [Bug](fix) right anti join error result when batch size is low
Gabriel39 merged PR #16510: URL: https://github.com/apache/doris/pull/16510 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xiaokang opened a new pull request, #16514: [Improvement](topn) add limit threashold session variable for topn optimizations
xiaokang opened a new pull request, #16514: URL: https://github.com/apache/doris/pull/16514 # Proposed changes Issue Number: close #xxx ## Problem summary 1. add limit threshold for topn runtime pushdown and key topn optimization 2. use unified session variable topn_opt_limit_threshold for all topn optimizations ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 3. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 4. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 5. Does it need to update dependencies: - [ ] Yes - [ ] No 6. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16485: [improvement](filecache) use dynamic segment size to cache remote file block
github-actions[bot] commented on PR #16485: URL: https://github.com/apache/doris/pull/16485#issuecomment-1422301573 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] eldenmoon commented on pull request #16514: [Improvement](topn) add limit threashold session variable for topn optimizations
eldenmoon commented on PR #16514: URL: https://github.com/apache/doris/pull/16514#issuecomment-1422303292 we could add this limit to fuzzy sesion variable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] JackDrogon opened a new issue, #16515: [Bug] AutoBucket dynamic partitions have partitions but no data && unstable AutoBucketUtilsTest
JackDrogon opened a new issue, #16515: URL: https://github.com/apache/doris/issues/16515 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version doris-0.0.0-trunk-b06e6b25c9f50c37b3f70c966129d060d0e27c72 ### What's Wrong? * AutoBucket dynamic partitions have partitions but no data * unstable AutoBucketUtilsTest with Env.getServingEnv  ### What You Expected? * AutoBucket dynamic partitions have partitions but no data use init bucketnum * mock with Env.getServingEnv ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16512: [regression-test](storage) add fuzzy variables for storage
github-actions[bot] commented on PR #16512: URL: https://github.com/apache/doris/pull/16512#issuecomment-1422314530 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16512: [regression-test](storage) add fuzzy variables for storage
github-actions[bot] commented on PR #16512: URL: https://github.com/apache/doris/pull/16512#issuecomment-1422314599 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] pengxiangyu commented on a diff in pull request #16488: [fix](cooldown) Rewrite update cooldown conf
pengxiangyu commented on code in PR #16488:
URL: https://github.com/apache/doris/pull/16488#discussion_r1099860083
##
fe/fe-core/src/main/java/org/apache/doris/catalog/TabletInvertedIndex.java:
##
@@ -335,48 +337,81 @@ private boolean needSync(Replica replicaInFe, TTabletInfo
backendTabletInfo) {
return false;
}
-private boolean needChangeCooldownConf(TabletMeta tabletMeta, TTabletInfo
beTabletInfo) {
-if (!beTabletInfo.isIsCooldown()) {
-return false;
-}
-// check cooldown type in fe and be, they need to be the same.
-if (tabletMeta.getCooldownReplicaId() !=
beTabletInfo.getCooldownReplicaId()) {
-LOG.warn("cooldownReplicaId is wrong for tablet: {}, Fe: {}, Be:
{}", beTabletInfo.getTabletId(),
-tabletMeta.getCooldownReplicaId(),
beTabletInfo.getCooldownReplicaId());
-return true;
+private void handleCooldownConf(TabletMeta tabletMeta, TTabletInfo
beTabletInfo,
+List cooldownConfToPush, List
cooldownConfToUpdate) {
+if (!beTabletInfo.isSetCooldownReplicaId()) {
+return;
}
-// check cooldown type in one tablet, One UPLOAD_DATA is needed in the
replicas.
-long stamp = readLock();
+Tablet tablet;
try {
-boolean replicaInvalid = true;
-Map replicaMap =
replicaMetaTable.row(beTabletInfo.getTabletId());
-for (Map.Entry entry : replicaMap.entrySet()) {
-if (entry.getValue().getId() ==
beTabletInfo.getCooldownReplicaId()) {
-replicaInvalid = false;
-break;
-}
+OlapTable table = (OlapTable)
Env.getCurrentInternalCatalog().getDbNullable(tabletMeta.getDbId())
+.getTable(tabletMeta.getTableId())
+.get();
+table.readLock();
+try {
+tablet =
table.getPartition(tabletMeta.getPartitionId()).getIndex(tabletMeta.getIndexId())
+.getTablet(beTabletInfo.tablet_id);
+} finally {
+table.readUnlock();
}
-if (replicaInvalid) {
-return true;
+} catch (RuntimeException e) {
+LOG.warn("failed to get tablet. tabletId={}",
beTabletInfo.tablet_id);
+return;
+}
+Pair cooldownConf = tablet.getCooldownConf();
Review Comment:
Create a class named CooldownConf is better, it will be more readability.
And it is easier to add new element to CooldownConf.
##
fe/fe-core/src/main/java/org/apache/doris/catalog/TabletInvertedIndex.java:
##
@@ -335,48 +337,81 @@ private boolean needSync(Replica replicaInFe, TTabletInfo
backendTabletInfo) {
return false;
}
-private boolean needChangeCooldownConf(TabletMeta tabletMeta, TTabletInfo
beTabletInfo) {
-if (!beTabletInfo.isIsCooldown()) {
-return false;
-}
-// check cooldown type in fe and be, they need to be the same.
-if (tabletMeta.getCooldownReplicaId() !=
beTabletInfo.getCooldownReplicaId()) {
-LOG.warn("cooldownReplicaId is wrong for tablet: {}, Fe: {}, Be:
{}", beTabletInfo.getTabletId(),
-tabletMeta.getCooldownReplicaId(),
beTabletInfo.getCooldownReplicaId());
-return true;
+private void handleCooldownConf(TabletMeta tabletMeta, TTabletInfo
beTabletInfo,
+List cooldownConfToPush, List
cooldownConfToUpdate) {
+if (!beTabletInfo.isSetCooldownReplicaId()) {
+return;
}
-// check cooldown type in one tablet, One UPLOAD_DATA is needed in the
replicas.
-long stamp = readLock();
+Tablet tablet;
try {
-boolean replicaInvalid = true;
-Map replicaMap =
replicaMetaTable.row(beTabletInfo.getTabletId());
-for (Map.Entry entry : replicaMap.entrySet()) {
-if (entry.getValue().getId() ==
beTabletInfo.getCooldownReplicaId()) {
-replicaInvalid = false;
-break;
-}
+OlapTable table = (OlapTable)
Env.getCurrentInternalCatalog().getDbNullable(tabletMeta.getDbId())
+.getTable(tabletMeta.getTableId())
+.get();
+table.readLock();
+try {
+tablet =
table.getPartition(tabletMeta.getPartitionId()).getIndex(tabletMeta.getIndexId())
+.getTablet(beTabletInfo.tablet_id);
+} finally {
+table.readUnlock();
}
-if (replicaInvalid) {
-return true;
+} catch (RuntimeException e) {
+LOG.warn("failed to get tablet. tabletId={}",
beTabletInfo.tablet_id);
+return;
+}
+Pair cooldownConf = tablet.getCooldownConf();
+if (beTabletInfo.g
[GitHub] [doris] hello-stephen commented on pull request #16513: [UT-Fix](MTMV) Fix MTMV FE UT bugs
hello-stephen commented on PR #16513: URL: https://github.com/apache/doris/pull/16513#issuecomment-1422317153 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.06 seconds load time: 470 seconds storage size: 17170603768 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230208094739_clickbench_pr_92192.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] JackDrogon opened a new pull request, #16516: [Fix] (bucket) fix partition with no history data && AutoBucketUtilsTest (#16515)
JackDrogon opened a new pull request, #16516: URL: https://github.com/apache/doris/pull/16516 fix partition with no history data && AutoBucketUtilsTest (#16515) # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16485: [improvement](filecache) use dynamic segment size to cache remote file block
github-actions[bot] commented on PR #16485: URL: https://github.com/apache/doris/pull/16485#issuecomment-1422325139 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16516: [Fix] (bucket) fix partition with no history data && AutoBucketUtilsTest (#16515)
github-actions[bot] commented on PR #16516: URL: https://github.com/apache/doris/pull/16516#issuecomment-1422325599 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16516: [Fix] (bucket) fix partition with no history data && AutoBucketUtilsTest (#16515)
github-actions[bot] commented on PR #16516: URL: https://github.com/apache/doris/pull/16516#issuecomment-1422325668 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 opened a new issue, #16517: [Bug] BE crash caused by TPCH
Gabriel39 opened a new issue, #16517:
URL: https://github.com/apache/doris/issues/16517
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no
similar issues.
### Version
master
### What's Wrong?
*** SIGABRT unkown detail explain (@0x1a5ea6) received by PID 1728166 (TID
1728360 OR 0x7f2eef50f700) from PID 1728166; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int,
siginfo_t*, void*) at /home/zcp/repo_center/doris_master/doris/be/src/common/s
ignal_handler.h:428
1# 0x7F2FEB8AC3C0 in /lib/x86_64-linux-gnu/libpthread.so.0
2# raise in /lib/x86_64-linux-gnu/libc.so.6
3# abort in /lib/x86_64-linux-gnu/libc.so.6
4# 0x55848E6CE769 in /mnt/ssd01/doris-master/VEC_ASAN/be/lib/doris_be
5# 0x55848E6C3D7D in /mnt/ssd01/doris-master/VEC_ASAN/be/lib/doris_be
6# google::LogMessage::SendToLog() in
/mnt/ssd01/doris-master/VEC_ASAN/be/lib/doris_be
7# google::LogMessage::Flush() in
/mnt/ssd01/doris-master/VEC_ASAN/be/lib/doris_be
8# google::LogMessageFatal::~LogMessageFatal() in
/mnt/ssd01/doris-master/VEC_ASAN/be/lib/doris_be
9# doris::vectorized::ColumnDecimal >&
assert_cast >&, doris::vectorized::IColumn&>(doris::vectorized::IColumn&)
at /home/zcp/repo_center/doris_master/doris/be/src/vec/common/assert_ca
st.h:50
10#
doris::vectorized::SingleValueDataDecimal
>::insert_result_into(doris::vectorized::IColumn&) const at /ho
me/zcp/repo_center/doris_master/doris/be/src/vec/aggregate_functions/aggregate_function_min_max.h:171
11#
doris::vectorized::AggregateFunctionsSingleValue > >, false>::insert_result_into(char const*,
doris::vectorized::IColumn&) const at /home/zcp/repo_center/doris_maste
r/doris/be/src/vec/aggregate_functions/aggregate_function_min_max.h:543
12#
doris::vectorized::IAggregateFunctionHelper >
>, false> >::insert_result_into_vec(std::vector > const&, unsigned long, doris::vectorized::IColumn&, unsigned long)
const at /home/zcp/repo_center/doris_master/doris/be/src/vec/aggregate
_functions/aggregate_function.h:290
13#
doris::vectorized::AggFnEvaluator::insert_result_info_vec(std::vector > const&, unsigned long, doris::vectorized:
:IColumn*, unsigned long) at
/home/zcp/repo_center/doris_master/doris/be/src/vec/exprs/vectorized_agg_fn.cpp:202
14# void
doris::vectorized::AggregationNode::_get_with_serialized_key_result(doris::RuntimeState*,
doris::vectorized::Block*, bool*)::{lambda(auto
:1&&)#1}::operator(), false>, f
alse>&>(doris::vectorized::AggregationMethodOneNumber, false>, false>&) const
at
/home/zcp/repo_center/doris_master/doris/be/src/vec/exec/vaggregation_node.cpp:1192
15# void std::__invoke_impl, false>, false>&>(std::__invoke_other,
doris::vectorized::AggregationNode::_get_with_serialized_key_result(doris::RuntimeState*,
doris::vector
ized::Block*, bool*)::{lambda(auto:1&&)#1}&&,
doris::vectorized::AggregationMethodOneNumber, false>, false>&) at
/var/local/ldb_toolchain/include/c++/11/bits/invoke.h:61
16#
std::__invoke_result, fals
e>, false>&>::type
std::__invoke, false>,
false>&>(doris::vectorized::AggregationNode::_get_with_serialized_key_result(doris::RuntimeState*,
doris::vectorized::Block*, bool*)::
{lambda(auto:1&&)#1}&&,
doris::vectorized::AggregationMethodOneNumber, false>
, false>&) at /var/local/ldb_toolchain/include/c++/11/bits/invoke.h:97
17#
_ZNSt8__detail9__variant17__gen_vtable_implINS0_12_Multi_arrayIPFNS0_21__deduce_visit_resultIvEEOZN5doris10vectorized15AggregationNode31_get_w
ith_serialized_key_resultEPNS5_12RuntimeStateEPNS6_5BlockEPbEUlOT_E_RSt7variantIJNS6_27AggregationMethodSerializedI9PHHashMapINS5_9StringRefEPc11D
efaultHashISK_vELb0NS6_26AggregationMethodOneNumberIh12FixedHashMapIhSL_28FixedHashMapImplicitZeroCellIhSL_16HashTableNoStateE28FixedHashTable
CalculatedSizeISU_E9AllocatorILb1ELb1EEELb0EEENSQ_ItSR_ItSL_SS_ItSL_ST_E24FixedHashTableStoredSizeIS11_ESY_ELb0EEENSQ_IjSJ_IjSL_9HashCRC32IjELb0EE
Lb0EEENSQ_ImSJ_ImSL_S16_ImELb0EELb0EEENS6_30AggregationMethodStringNoCacheI13StringHashMapISL_SY_EEENSQ_INS6_7UInt128ESJ_IS1H_SL_S16_IS1H_ELb0EELb
0EEENSQ_IjSJ_IjSL_14HashMixWrapperIjS17_ELb0EELb0EEENSQ_ImSJ_ImSL_S1L_ImS1A_ELb0EELb0EEENSQ_IS1H_SJ_IS1H_SL_S1L_IS1H_S1I_ELb0EELb0EEENS6_37Aggrega
tionMethodSingleNullableColumnINSQ_IhNS6_26AggregationDataWithNullKeyISZ_EELb0ENS1V_INSQ_ItNS1W_IS14_EELb0ENS1V_INSQ_IjNS1W_IS18_EELb0
ENS1V_INSQ_ImNS1W_IS1B_EELb0ENS1V_INSQ_IjNS1W_IS1N_EELb0ENS1V_INSQ_ImNS1W_IS1Q_EELb0ENS1V_INSQ_IS1H_NS1W_IS1J_EELb0ENS1V_INSQ_IS1H
_NS1W_IS1T_EELb0ENS1V_INS1D_INS1W_IS1F_EENS6_26AggregationMethodKeysFixedIS1B_Lb0EEENS2O_IS1B_Lb1EEENS2O_IS1J_Lb0EEENS2O_IS1J_Lb1EEENS2O_I
SJ_INS6_7UInt256ESL_S16_IS2T_ELb0EELb0EEENS2O_IS2V_Lb1EEENS2O_IS1Q_Lb0EEENS2O_IS1Q_Lb1EEENS2O_IS1T_Lb0EEENS2O_IS1T_Lb1EEENS2O
[GitHub] [doris] caiconghui merged pull request #15830: [enhancement](load) Change transaction limit from global level to db level
caiconghui merged PR #15830: URL: https://github.com/apache/doris/pull/15830 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] caiconghui closed issue #15869: [Enhancement] make max running txn per db more flexible
caiconghui closed issue #15869: [Enhancement] make max running txn per db more flexible URL: https://github.com/apache/doris/issues/15869 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [enhancement](load) Change transaction limit from global level to db level (#15830)
This is an automated email from the ASF dual-hosted git repository.
caiconghui pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new bb334de00f [enhancement](load) Change transaction limit from global
level to db level (#15830)
bb334de00f is described below
commit bb334de00ff79cfed2834401fc5b3e2000289605
Author: Henry2SS <45096548+henry...@users.noreply.github.com>
AuthorDate: Wed Feb 8 18:04:26 2023 +0800
[enhancement](load) Change transaction limit from global level to db level
(#15830)
Add transaction size quota for database
Co-authored-by: wuhangze
---
docs/en/docs/admin-manual/config/fe-config.md | 22 +++
docs/zh-CN/docs/admin-manual/config/fe-config.md | 22 +++
.../main/java/org/apache/doris/common/Config.java | 6 +
fe/fe-core/src/main/cup/sql_parser.cup | 4 +++
.../doris/analysis/AlterDatabaseQuotaStmt.java | 7 +++--
.../org/apache/doris/analysis/ShowDataStmt.java| 6 +
.../java/org/apache/doris/catalog/Database.java| 31 ++
.../org/apache/doris/common/proc/DbsProcDir.java | 3 +++
.../org/apache/doris/common/util/ParseUtil.java| 13 +
.../apache/doris/datasource/InternalCatalog.java | 4 +++
.../doris/load/sync/canal/CanalSyncChannel.java| 5 ++--
.../doris/transaction/DatabaseTransactionMgr.java | 9 +++
.../doris/analysis/AlterDatabaseQuotaStmtTest.java | 23
.../apache/doris/common/proc/DbsProcDirTest.java | 8 +++---
14 files changed, 150 insertions(+), 13 deletions(-)
diff --git a/docs/en/docs/admin-manual/config/fe-config.md
b/docs/en/docs/admin-manual/config/fe-config.md
index c23db57284..8586269220 100644
--- a/docs/en/docs/admin-manual/config/fe-config.md
+++ b/docs/en/docs/admin-manual/config/fe-config.md
@@ -2593,3 +2593,25 @@ MasterOnly：true
Maximum number of error tablet showed in broker load.
+ `default_db_max_running_txn_num`
+
+Default：-1
+
+IsMutable：true
+
+MasterOnly：true
+
+Used to set the default database transaction quota size.
+
+The default value setting to -1 means using `max_running_txn_num_per_db`
instead of `default_db_max_running_txn_num`.
+
+To set the quota size of a single database, you can use:
+
+```
+Set the database transaction quota
+ALTER DATABASE db_name SET TRANSACTION QUOTA quota;
+View configuration
+show data （Detail：HELP SHOW DATA）
+```
+
+
diff --git a/docs/zh-CN/docs/admin-manual/config/fe-config.md
b/docs/zh-CN/docs/admin-manual/config/fe-config.md
index 59a5dcd031..19b3a64e89 100644
--- a/docs/zh-CN/docs/admin-manual/config/fe-config.md
+++ b/docs/zh-CN/docs/admin-manual/config/fe-config.md
@@ -2592,3 +2592,25 @@ SmallFileMgr 中存储的最大文件数
是否为 Master FE 节点独有的配置项：true
broker load job 保存的失败tablet 信息的最大数量
+
+ `default_db_max_running_txn_num`
+
+默认值：-1
+
+是否可以动态配置：true
+
+是否为 Master FE 节点独有的配置项：true
+
+用于设置默认数据库事务配额大小。
+
+默认值设置为 -1 意味着使用 `max_running_txn_num_per_db` 而不是
`default_db_max_running_txn_num`。
+
+设置单个数据库的配额大小可以使用：
+
+```
+设置数据库事务量配额
+ALTER DATABASE db_name SET TRANSACTION QUOTA quota;
+查看配置
+show data （其他用法：HELP SHOW DATA）
+```
+
diff --git a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
index 5ea73a3ddb..6edbc5a160 100644
--- a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
+++ b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
@@ -1985,5 +1985,11 @@ public class Config extends ConfigBase {
*/
@ConfField(mutable = true)
public static int pull_request_id = 0;
+
+/**
+ * Used to set default db transaction quota num.
+ */
+@ConfField(mutable = true, masterOnly = true)
+public static long default_db_max_running_txn_num = -1;
}
diff --git a/fe/fe-core/src/main/cup/sql_parser.cup
b/fe/fe-core/src/main/cup/sql_parser.cup
index 9e873edca4..5f727df186 100644
--- a/fe/fe-core/src/main/cup/sql_parser.cup
+++ b/fe/fe-core/src/main/cup/sql_parser.cup
@@ -1271,6 +1271,10 @@ alter_stmt ::=
{:
RESULT = new AlterDatabaseQuotaStmt(dbName, QuotaType.REPLICA,
String.valueOf(number));
:}
+| KW_ALTER KW_DATABASE ident:dbName KW_SET KW_TRANSACTION KW_QUOTA
INTEGER_LITERAL:number
+{:
+RESULT = new AlterDatabaseQuotaStmt(dbName, QuotaType.TRANSACTION,
String.valueOf(number));
+:}
| KW_ALTER KW_DATABASE ident:dbName KW_RENAME ident:newDbName
{:
RESULT = new AlterDatabaseRename(dbName, newDbName);
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/AlterDatabaseQuotaStmt.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/AlterDatabaseQuotaStmt.java
index c2cb0fbb0a..90c01016e3 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/analysis/AlterDatabaseQuotaStmt.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/analysis/AlterDataba
[GitHub] [doris] Gabriel39 opened a new pull request, #16518: [Bug](log) add some log to find out bug
Gabriel39 opened a new pull request, #16518: URL: https://github.com/apache/doris/pull/16518 # Proposed changes This PR is used for finding out the bug described in #16517 ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16518: [Bug](log) add some log to find out bug
github-actions[bot] commented on PR #16518: URL: https://github.com/apache/doris/pull/16518#issuecomment-1422360843 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16518: [Bug](log) add some log to find out bug
github-actions[bot] commented on PR #16518: URL: https://github.com/apache/doris/pull/16518#issuecomment-1422380255 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16506: [fix](planner) cannot process more than one subquery in disjunct
github-actions[bot] commented on PR #16506: URL: https://github.com/apache/doris/pull/16506#issuecomment-1422383482 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16506: [fix](planner) cannot process more than one subquery in disjunct
github-actions[bot] commented on PR #16506: URL: https://github.com/apache/doris/pull/16506#issuecomment-1422383412 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow merged pull request #16506: [fix](planner) cannot process more than one subquery in disjunct
morrySnow merged PR #16506: URL: https://github.com/apache/doris/pull/16506 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [fix](planner) cannot process more than one subquery in disjunct (#16506)
This is an automated email from the ASF dual-hosted git repository.
morrysnow pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new a512469537 [fix](planner) cannot process more than one subquery in
disjunct (#16506)
a512469537 is described below
commit a51246953763764f940f5b6000f366e63d799e27
Author: morrySnow <101034200+morrys...@users.noreply.github.com>
AuthorDate: Wed Feb 8 18:46:06 2023 +0800
[fix](planner) cannot process more than one subquery in disjunct (#16506)
before this PR, Doris cannot process sql like that
```sql
CREATE TABLE `test_sq_dj1` (
`c1` int(11) NULL,
`c2` int(11) NULL,
`c3` int(11) NULL
) ENGINE=OLAP
DUPLICATE KEY(`c1`)
COMMENT 'OLAP'
DISTRIBUTED BY HASH(`c1`) BUCKETS 3
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"in_memory" = "false",
"storage_format" = "V2",
"disable_auto_compaction" = "false"
);
CREATE TABLE `test_sq_dj2` (
`c1` int(11) NULL,
`c2` int(11) NULL,
`c3` int(11) NULL
) ENGINE=OLAP
DUPLICATE KEY(`c1`)
COMMENT 'OLAP'
DISTRIBUTED BY HASH(`c1`) BUCKETS 3
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"in_memory" = "false",
"storage_format" = "V2",
"disable_auto_compaction" = "false"
);
insert into test_sq_dj1 values(1, 2, 3), (10, 20, 30), (100, 200, 300);
insert into test_sq_dj2 values(10, 20, 30);
-- core
SELECT * FROM test_sq_dj1 WHERE c1 IN (SELECT c1 FROM test_sq_dj2) OR c1 IN
(SELECT c1 FROM test_sq_dj2) OR c1 < 10;
-- invalid slot
SELECT * FROM test_sq_dj1 WHERE c1 IN (SELECT c1 FROM test_sq_dj2) OR c1 IN
(SELECT c2 FROM test_sq_dj2) OR c1 < 10;
```
there are two problems:
1. we should remove redundant sub-query in one conjuncts to avoid generate
useless join node
2. when we have more than one sub-query in one disjunct. we should put the
conjunct contains the disjunct at the top node of the set of mark join nodes.
And pop up the mark slot to the top node.
---
.../java/org/apache/doris/analysis/Analyzer.java | 38 +++-
.../org/apache/doris/analysis/StmtRewriter.java| 21 +++--
.../java/org/apache/doris/analysis/TableRef.java | 3 +
.../org/apache/doris/planner/HashJoinNode.java | 2 +-
.../org/apache/doris/planner/JoinNodeBase.java | 29 --
.../apache/doris/planner/NestedLoopJoinNode.java | 1 +
.../correctness/test_subquery_in_disjunction.out | 8 ++
regression-test/data/tpcds_sf1_p1/sql/q10.out | 8 ++
regression-test/data/tpcds_sf1_p1/sql/q35.out | 103 +
regression-test/data/tpcds_sf1_p1/sql/q45.out | 22 +
.../test_subquery_in_disjunction.groovy| 8 ++
regression-test/suites/tpcds_sf1_p1/sql/q10.sql| 2 -
regression-test/suites/tpcds_sf1_p1/sql/q35.sql| 2 -
regression-test/suites/tpcds_sf1_p1/sql/q45.sql| 2 -
14 files changed, 220 insertions(+), 29 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/Analyzer.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/Analyzer.java
index 89d799a66e..3f57d161ff 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/Analyzer.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/Analyzer.java
@@ -396,6 +396,8 @@ public class Analyzer {
private final Map markTupleIdByInnerRef =
Maps.newHashMap();
+private final Set markTupleIdsNotProcessed =
Sets.newHashSet();
+
public GlobalState(Env env, ConnectContext context) {
this.env = env;
this.context = context;
@@ -672,12 +674,18 @@ public class Analyzer {
tableRefMap.put(result.getId(), ref);
-// for mark join
+// for mark join, init three context
+// 1. markTuples to records all tuples belong to mark slot
+// 2. markTupleIdByInnerRef to records relationship between inner
table of mark join and the mark tuple
+// 3. markTupleIdsNotProcessed to records un-process mark tuple id.
if an expr contains slot belong to
+//the un-process mark tuple, it should not assign to current
join node and should pop up its
+//mark slot until all mark tuples in this expr has been
processed.
if (ref.getJoinOp() != null && ref.isMark()) {
TupleDescriptor markTuple = getDescTbl().createTupleDescriptor();
markTuple.setAliases(new String[]{ref.getMarkTupleName()}, true);
globalState.markTuples.put(ref.getMarkTupleName(), markTuple);
globalState.markTupleIdByInnerRef.put(ref, markTuple.getId());
+globalState.markTupleIdsNotProcessed.add(markTuple.getId());
}
return result;
@@ -1592,12 +1
[GitHub] [doris] jackwener opened a new pull request, #16519: [cleanup](Nereids): cleanup FE code.
jackwener opened a new pull request, #16519: URL: https://github.com/apache/doris/pull/16519 # Proposed changes Issue Number: close #xxx ## Problem summary cleanup FE code ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16488: [fix](cooldown) Rewrite update cooldown conf
github-actions[bot] commented on PR #16488: URL: https://github.com/apache/doris/pull/16488#issuecomment-1422418261 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16488: [fix](cooldown) Rewrite update cooldown conf
github-actions[bot] commented on PR #16488: URL: https://github.com/apache/doris/pull/16488#issuecomment-1422418338 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16378: [Feature](Materialized-View) support multiple slot on one column in materialized view
github-actions[bot] commented on PR #16378: URL: https://github.com/apache/doris/pull/16378#issuecomment-1422427747 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] FreeOnePlus opened a new pull request, #16520: [docs](docs)Add Run Docker cluster docs
FreeOnePlus opened a new pull request, #16520: URL: https://github.com/apache/doris/pull/16520 # Proposed changes Add Run Docker cluster docs, new doc~ ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16520: [docs](docs)Add Run Docker cluster docs
github-actions[bot] commented on PR #16520: URL: https://github.com/apache/doris/pull/16520#issuecomment-1422444695 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16520: [docs](docs)Add Run Docker cluster docs
github-actions[bot] commented on PR #16520: URL: https://github.com/apache/doris/pull/16520#issuecomment-1422444746 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16501: [fix](docs) add the batch interval time of sink in spark connector doc
github-actions[bot] commented on PR #16501: URL: https://github.com/apache/doris/pull/16501#issuecomment-1422450321 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16501: [fix](docs) add the batch interval time of sink in spark connector doc
github-actions[bot] commented on PR #16501: URL: https://github.com/apache/doris/pull/16501#issuecomment-1422450388 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 opened a new pull request, #16521: [pick](branch-1.2) Pick 3 commits related to mark join
Gabriel39 opened a new pull request, #16521: URL: https://github.com/apache/doris/pull/16521 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] FreeOnePlus opened a new pull request, #16522: [feature](docker)Add Doris Docker Build Script
FreeOnePlus opened a new pull request, #16522: URL: https://github.com/apache/doris/pull/16522 # Proposed changes Add 3FE & 3BE Build Script ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16522: [feature](docker)Add Doris Docker Build Script
github-actions[bot] commented on PR #16522:
URL: https://github.com/apache/doris/pull/16522#issuecomment-1422473417
`sh-checker report`
To get the full details, please check in the
[job]("https://github.com/apache/doris/actions/runs/4123800248";) output.
shellcheck errors
```
'shellcheck ' returned error 1 finding the following syntactical issues:
--
In docker/runtime/docker-compose-demo/build-cluster/rum-command/3fe_3be.sh
line 1:
docker network create --driver bridge --subnet=172.20.80.0/24 doris-network
^-- SC2148 (error): Tips depend on target shell and yours is unknown. Add a
shebang or a 'shell' directive.
For more information:
https://www.shellcheck.net/wiki/SC2148 -- Tips depend on target shell and
y...
--
You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
# shellcheck disable=
above the line that contains the issue, where is the error code;
3. Add '-e ' to the SHELLCHECK_OPTS setting in your .yml action file.
```
shfmt errors
```
'shfmt ' returned error 1 finding the following formatting issues:
--
---
docker/runtime/docker-compose-demo/build-cluster/rum-command/3fe_3be.sh.orig
+++ docker/runtime/docker-compose-demo/build-cluster/rum-command/3fe_3be.sh
@@ -1,74 +1,74 @@
docker network create --driver bridge --subnet=172.20.80.0/24 doris-network
docker run -itd \
---name=fe-01 \
---env
FE_SERVERS="fe1:172.20.80.2:9010,fe2:172.20.80.3:9010,fe3:172.20.80.4:9010" \
---env FE_ID=1 \
--p 8031:8030 \
--p 9031:9030 \
--v /data/fe-01/doris-meta:/opt/apache-doris/fe/doris-meta \
--v /data/fe-01/conf:/opt/apache-doris/fe/conf \
--v /data/fe-01/log:/opt/apache-doris/fe/log \
---network=doris-network \
---ip=172.20.80.2 \
-apache/doris:1.2.1-fe-x86_64
+--name=fe-01 \
+--env
FE_SERVERS="fe1:172.20.80.2:9010,fe2:172.20.80.3:9010,fe3:172.20.80.4:9010" \
+--env FE_ID=1 \
+-p 8031:8030 \
+-p 9031:9030 \
+-v /data/fe-01/doris-meta:/opt/apache-doris/fe/doris-meta \
+-v /data/fe-01/conf:/opt/apache-doris/fe/conf \
+-v /data/fe-01/log:/opt/apache-doris/fe/log \
+--network=doris-network \
+--ip=172.20.80.2 \
+apache/doris:1.2.1-fe-x86_64
docker run -itd \
---name=fe-02 \
---env
FE_SERVERS="fe1:172.20.80.2:9010,fe2:172.20.80.3:9010,fe3:172.20.80.4:9010" \
---env FE_ID=2 \
--p 8032:8030 \
--p 9032:9030 \
--v /data/fe-02/doris-meta:/opt/apache-doris/fe/doris-meta \
--v /data/fe-02/conf:/opt/apache-doris/fe/conf \
--v /data/fe-02/log:/opt/apache-doris/fe/log \
---network=doris-network \
---ip=172.20.80.3 \
-apache/doris:1.2.1-fe-x86_64
+--name=fe-02 \
+--env
FE_SERVERS="fe1:172.20.80.2:9010,fe2:172.20.80.3:9010,fe3:172.20.80.4:9010" \
+--env FE_ID=2 \
+-p 8032:8030 \
+-p 9032:9030 \
+-v /data/fe-02/doris-meta:/opt/apache-doris/fe/doris-meta \
+-v /data/fe-02/conf:/opt/apache-doris/fe/conf \
+-v /data/fe-02/log:/opt/apache-doris/fe/log \
+--network=doris-network \
+--ip=172.20.80.3 \
+apache/doris:1.2.1-fe-x86_64
docker run -itd \
---name=fe-03 \
---env
FE_SERVERS="fe1:172.20.80.2:9010,fe2:172.20.80.3:9010,fe3:172.20.80.4:9010" \
---env FE_ID=3 \
--p 8033:8030 \
--p 9033:9030 \
--v /data/fe-03/doris-meta:/opt/apache-doris/fe/doris-meta \
--v /data/fe-03/conf:/opt/apache-doris/fe/conf \
--v /data/fe-03/log:/opt/apache-doris/fe/log \
---network=doris-network \
---ip=172.20.80.4 \
-apache/doris:1.2.1-fe-x86_64
+--name=fe-03 \
+--env
FE_SERVERS="fe1:172.20.80.2:9010,fe2:172.20.80.3:9010,fe3:172.20.80.4:9010" \
+--env FE_ID=3 \
+-p 8033:8030 \
+-p 9033:9030 \
+-v /data/fe-03/doris-meta:/opt/apache-doris/fe/doris-meta \
+-v /data/fe-03/conf:/opt/apache-doris/fe/conf \
+-v /data/fe-03/log:/opt/apache-doris/fe/log \
+--network=doris-network \
+--ip=172.20.80.4 \
+apache/doris:1.2.1-fe-x86_64
docker run -itd \
---name=be-01 \
---env
FE_SERVERS="fe1:172.20.80.2:9010,fe2:172.20.80.3:9010,fe3:172.20.80.4:9010" \
---env BE_ADDR="172.20.80.5:9050" \
--p 8041:8040 \
--v /data/be-01/storage:/opt/apache-doris/be/storage \
--v /data/be-01/conf:/opt/apache-doris/be/conf \
--v /data/be-01/log:/opt/apache-doris/be/log \
---network=doris-network \
---ip=172.20.80.5 \
-apache/doris:1.2.1-be-x86_64
+--name=be-01 \
+--env
FE_SERVERS="fe1:172.20.80.2:9010,fe2:172.20.80.3:9010,fe3:172.20.80.4:9010" \
+--env BE_ADDR="172.20.80.5:9050" \
+-p 8041:8040 \
+-v /data/be-01/storage:/opt/apache-doris/be/storage \
[GitHub] [doris] dataroaring opened a new pull request, #16523: [fix](doc) fix some errors in docs
dataroaring opened a new pull request, #16523: URL: https://github.com/apache/doris/pull/16523 1. dead link 2. references to zh-CN in en docs 3. wrong content # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16521: [pick](branch-1.2) Pick 3 commits related to mark join
github-actions[bot] commented on PR #16521: URL: https://github.com/apache/doris/pull/16521#issuecomment-1422476842 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16521: [pick](branch-1.2) Pick 3 commits related to mark join
github-actions[bot] commented on PR #16521: URL: https://github.com/apache/doris/pull/16521#issuecomment-1422476896 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] platoneko commented on a diff in pull request #16488: [fix](cooldown) Rewrite update cooldown conf
platoneko commented on code in PR #16488:
URL: https://github.com/apache/doris/pull/16488#discussion_r1100033234
##
fe/fe-core/src/main/java/org/apache/doris/catalog/TabletInvertedIndex.java:
##
@@ -335,48 +337,81 @@ private boolean needSync(Replica replicaInFe, TTabletInfo
backendTabletInfo) {
return false;
}
-private boolean needChangeCooldownConf(TabletMeta tabletMeta, TTabletInfo
beTabletInfo) {
-if (!beTabletInfo.isIsCooldown()) {
-return false;
-}
-// check cooldown type in fe and be, they need to be the same.
-if (tabletMeta.getCooldownReplicaId() !=
beTabletInfo.getCooldownReplicaId()) {
-LOG.warn("cooldownReplicaId is wrong for tablet: {}, Fe: {}, Be:
{}", beTabletInfo.getTabletId(),
-tabletMeta.getCooldownReplicaId(),
beTabletInfo.getCooldownReplicaId());
-return true;
+private void handleCooldownConf(TabletMeta tabletMeta, TTabletInfo
beTabletInfo,
+List cooldownConfToPush, List
cooldownConfToUpdate) {
+if (!beTabletInfo.isSetCooldownReplicaId()) {
+return;
}
-// check cooldown type in one tablet, One UPLOAD_DATA is needed in the
replicas.
-long stamp = readLock();
+Tablet tablet;
try {
-boolean replicaInvalid = true;
-Map replicaMap =
replicaMetaTable.row(beTabletInfo.getTabletId());
-for (Map.Entry entry : replicaMap.entrySet()) {
-if (entry.getValue().getId() ==
beTabletInfo.getCooldownReplicaId()) {
-replicaInvalid = false;
-break;
-}
+OlapTable table = (OlapTable)
Env.getCurrentInternalCatalog().getDbNullable(tabletMeta.getDbId())
+.getTable(tabletMeta.getTableId())
+.get();
+table.readLock();
+try {
+tablet =
table.getPartition(tabletMeta.getPartitionId()).getIndex(tabletMeta.getIndexId())
+.getTablet(beTabletInfo.tablet_id);
+} finally {
+table.readUnlock();
}
-if (replicaInvalid) {
-return true;
+} catch (RuntimeException e) {
+LOG.warn("failed to get tablet. tabletId={}",
beTabletInfo.tablet_id);
+return;
+}
+Pair cooldownConf = tablet.getCooldownConf();
+if (beTabletInfo.getCooldownTerm() > cooldownConf.second) { // should
not be here
+LOG.warn("report cooldownTerm({}) > cooldownTerm in
TabletMeta({}), tabletId={}",
+beTabletInfo.getCooldownTerm(), cooldownConf.second,
beTabletInfo.tablet_id);
+return;
+}
+
+if (cooldownConf.first <= 0) { // invalid cooldownReplicaId
+CooldownConf conf = new CooldownConf(tabletMeta.getDbId(),
tabletMeta.getTableId(),
+tabletMeta.getPartitionId(), tabletMeta.getIndexId(),
beTabletInfo.tablet_id, cooldownConf.second);
+synchronized (cooldownConfToUpdate) {
Review Comment:
This code is executed in ForkAndJoin pool.
##
fe/fe-core/src/main/java/org/apache/doris/catalog/TabletInvertedIndex.java:
##
@@ -335,48 +337,81 @@ private boolean needSync(Replica replicaInFe, TTabletInfo
backendTabletInfo) {
return false;
}
-private boolean needChangeCooldownConf(TabletMeta tabletMeta, TTabletInfo
beTabletInfo) {
-if (!beTabletInfo.isIsCooldown()) {
-return false;
-}
-// check cooldown type in fe and be, they need to be the same.
-if (tabletMeta.getCooldownReplicaId() !=
beTabletInfo.getCooldownReplicaId()) {
-LOG.warn("cooldownReplicaId is wrong for tablet: {}, Fe: {}, Be:
{}", beTabletInfo.getTabletId(),
-tabletMeta.getCooldownReplicaId(),
beTabletInfo.getCooldownReplicaId());
-return true;
+private void handleCooldownConf(TabletMeta tabletMeta, TTabletInfo
beTabletInfo,
+List cooldownConfToPush, List
cooldownConfToUpdate) {
+if (!beTabletInfo.isSetCooldownReplicaId()) {
+return;
}
-// check cooldown type in one tablet, One UPLOAD_DATA is needed in the
replicas.
-long stamp = readLock();
+Tablet tablet;
try {
-boolean replicaInvalid = true;
-Map replicaMap =
replicaMetaTable.row(beTabletInfo.getTabletId());
-for (Map.Entry entry : replicaMap.entrySet()) {
-if (entry.getValue().getId() ==
beTabletInfo.getCooldownReplicaId()) {
-replicaInvalid = false;
-break;
-}
+OlapTable table = (OlapTable)
Env.getCurrentInternalCatalog().getDbNullable(tabletMeta.getDbId())
+.getTable(tabletMeta.getTableId())
+.get();
+table.readLock(
[GitHub] [doris] platoneko commented on a diff in pull request #16488: [fix](cooldown) Rewrite update cooldown conf
platoneko commented on code in PR #16488:
URL: https://github.com/apache/doris/pull/16488#discussion_r1100033493
##
fe/fe-core/src/main/java/org/apache/doris/catalog/TabletInvertedIndex.java:
##
@@ -335,48 +337,81 @@ private boolean needSync(Replica replicaInFe, TTabletInfo
backendTabletInfo) {
return false;
}
-private boolean needChangeCooldownConf(TabletMeta tabletMeta, TTabletInfo
beTabletInfo) {
-if (!beTabletInfo.isIsCooldown()) {
-return false;
-}
-// check cooldown type in fe and be, they need to be the same.
-if (tabletMeta.getCooldownReplicaId() !=
beTabletInfo.getCooldownReplicaId()) {
-LOG.warn("cooldownReplicaId is wrong for tablet: {}, Fe: {}, Be:
{}", beTabletInfo.getTabletId(),
-tabletMeta.getCooldownReplicaId(),
beTabletInfo.getCooldownReplicaId());
-return true;
+private void handleCooldownConf(TabletMeta tabletMeta, TTabletInfo
beTabletInfo,
+List cooldownConfToPush, List
cooldownConfToUpdate) {
+if (!beTabletInfo.isSetCooldownReplicaId()) {
+return;
}
-// check cooldown type in one tablet, One UPLOAD_DATA is needed in the
replicas.
-long stamp = readLock();
+Tablet tablet;
try {
-boolean replicaInvalid = true;
-Map replicaMap =
replicaMetaTable.row(beTabletInfo.getTabletId());
-for (Map.Entry entry : replicaMap.entrySet()) {
-if (entry.getValue().getId() ==
beTabletInfo.getCooldownReplicaId()) {
-replicaInvalid = false;
-break;
-}
+OlapTable table = (OlapTable)
Env.getCurrentInternalCatalog().getDbNullable(tabletMeta.getDbId())
+.getTable(tabletMeta.getTableId())
+.get();
+table.readLock();
+try {
+tablet =
table.getPartition(tabletMeta.getPartitionId()).getIndex(tabletMeta.getIndexId())
+.getTablet(beTabletInfo.tablet_id);
+} finally {
+table.readUnlock();
}
-if (replicaInvalid) {
-return true;
+} catch (RuntimeException e) {
+LOG.warn("failed to get tablet. tabletId={}",
beTabletInfo.tablet_id);
+return;
+}
+Pair cooldownConf = tablet.getCooldownConf();
+if (beTabletInfo.getCooldownTerm() > cooldownConf.second) { // should
not be here
+LOG.warn("report cooldownTerm({}) > cooldownTerm in
TabletMeta({}), tabletId={}",
+beTabletInfo.getCooldownTerm(), cooldownConf.second,
beTabletInfo.tablet_id);
+return;
+}
+
+if (cooldownConf.first <= 0) { // invalid cooldownReplicaId
+CooldownConf conf = new CooldownConf(tabletMeta.getDbId(),
tabletMeta.getTableId(),
+tabletMeta.getPartitionId(), tabletMeta.getIndexId(),
beTabletInfo.tablet_id, cooldownConf.second);
+synchronized (cooldownConfToUpdate) {
+cooldownConfToUpdate.add(conf);
}
-} finally {
-readUnlock(stamp);
+return;
+}
+
+// validate replica is active
+Map replicaMap =
replicaMetaTable.row(beTabletInfo.getTabletId());
+if (replicaMap.isEmpty()) {
+return;
+}
+boolean replicaInvalid = true;
+for (Replica replica : replicaMap.values()) {
+if (replica.getId() == cooldownConf.first) {
+replicaInvalid = false;
+break;
+}
+}
+if (replicaInvalid) {
+CooldownConf conf = new CooldownConf(tabletMeta.getDbId(),
tabletMeta.getTableId(),
+tabletMeta.getPartitionId(), tabletMeta.getIndexId(),
beTabletInfo.tablet_id, cooldownConf.second);
+synchronized (cooldownConfToUpdate) {
Review Comment:
ditto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman commented on issue #16446: [WIP] Release Note 1.2.2
morningman commented on issue #16446: URL: https://github.com/apache/doris/issues/16446#issuecomment-1422483520 > [c61be0c](https://github.com/apache/doris/commit/c61be0cb8a85cda1b61b1bc0535020488207b46c) 这个不发吗 #16074 fixed this issue, and pr with `dev/1.2.2-merged` is included in this release -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16522: [feature](docker)Add Doris Docker Build Script
github-actions[bot] commented on PR #16522: URL: https://github.com/apache/doris/pull/16522#issuecomment-1422507962 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16522: [feature](docker)Add Doris Docker Build Script
github-actions[bot] commented on PR #16522: URL: https://github.com/apache/doris/pull/16522#issuecomment-1422508030 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 opened a new pull request, #16525: [Bug](pipeline) Support projection in UnionSourceOperator
Gabriel39 opened a new pull request, #16525: URL: https://github.com/apache/doris/pull/16525 # Proposed changes Support projection in UnionSourceOperator ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15249: [pipeline](test) test regression
github-actions[bot] commented on PR #15249: URL: https://github.com/apache/doris/pull/15249#issuecomment-1422521145 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16525: [Bug](pipeline) Support projection in UnionSourceOperator
github-actions[bot] commented on PR #16525: URL: https://github.com/apache/doris/pull/16525#issuecomment-1422523732 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 opened a new pull request, #16526: [ehancement](nereids) Ignore hidden columns when calculate the selectivity for predicate
Kikyou1997 opened a new pull request, #16526: URL: https://github.com/apache/doris/pull/16526 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #16472: [improvement](meta) Enhance Doris's fault tolerance to disk error
github-actions[bot] commented on code in PR #16472:
URL: https://github.com/apache/doris/pull/16472#discussion_r1100069691
##
be/src/vec/olap/vertical_block_reader.h:
##
@@ -42,7 +42,11 @@ class VerticalBlockReader final : public TabletReader {
Status init(const ReaderParams& read_params) override;
Status next_block_with_aggregation(Block* block, bool* eof) override {
-return (this->*_next_block_func)(block, eof);
+auto res = (this->*_next_block_func)(block, eof);
+if (UNLIKELY(res == OLAP_ERR_IO_ERROR)) {
Review Comment:
warning: use of undeclared identifier 'OLAP_ERR_IO_ERROR'
[clang-diagnostic-error]
```cpp
if (UNLIKELY(res == OLAP_ERR_IO_ERROR)) {
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16479: [fix](planner)group_concat lost order by info in second phase merge agg
github-actions[bot] commented on PR #16479: URL: https://github.com/apache/doris/pull/16479#issuecomment-1422537837 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16479: [fix](planner)group_concat lost order by info in second phase merge agg
github-actions[bot] commented on PR #16479: URL: https://github.com/apache/doris/pull/16479#issuecomment-1422537910 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow merged pull request #16479: [fix](planner)group_concat lost order by info in second phase merge agg
morrySnow merged PR #16479: URL: https://github.com/apache/doris/pull/16479 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [fix](planner)group_concat lost order by info in second phase merge agg (#16479)
This is an automated email from the ASF dual-hosted git repository.
morrysnow pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new f0b0eedbc5 [fix](planner)group_concat lost order by info in second
phase merge agg (#16479)
f0b0eedbc5 is described below
commit f0b0eedbc53add11766d4909ef7f3ed5151d09ff
Author: starocean999 <40539150+starocean...@users.noreply.github.com>
AuthorDate: Wed Feb 8 20:48:52 2023 +0800
[fix](planner)group_concat lost order by info in second phase merge agg
(#16479)
---
.../src/main/java/org/apache/doris/analysis/AggregateInfo.java | 7 +--
.../src/main/java/org/apache/doris/analysis/FunctionCallExpr.java | 4
.../doris/nereids/trees/expressions/functions/agg/GroupConcat.java | 2 --
regression-test/data/nereids_p0/group_concat/test_group_concat.out | 3 +++
regression-test/data/query_p0/group_concat/test_group_concat.out | 3 +++
.../suites/nereids_p0/group_concat/test_group_concat.groovy| 3 +++
.../suites/query_p0/group_concat/test_group_concat.groovy | 3 +++
7 files changed, 21 insertions(+), 4 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/AggregateInfo.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/AggregateInfo.java
index 904cb0848a..33701b8517 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/AggregateInfo.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/AggregateInfo.java
@@ -558,8 +558,10 @@ public final class AggregateInfo extends AggregateInfoBase
{
Preconditions.checkState(inputExpr.isAggregateFunction());
Expr aggExprParam =
new SlotRef(inputDesc.getSlots().get(i +
getGroupingExprs().size()));
-FunctionCallExpr aggExpr = FunctionCallExpr.createMergeAggCall(
-inputExpr, Lists.newArrayList(aggExprParam),
inputExpr.getFnParams().exprs());
+FunctionParams fnParams = inputExpr.getAggFnParams();
+FunctionCallExpr aggExpr =
+FunctionCallExpr.createMergeAggCall(inputExpr,
Lists.newArrayList(aggExprParam),
+fnParams != null ? fnParams.exprs() :
inputExpr.getFnParams().exprs());
aggExpr.analyzeNoThrow(analyzer);
// do not need analyze in merge stage, just do mark for BE get
right function
aggExpr.setOrderByElements(inputExpr.getOrderByElements());
@@ -691,6 +693,7 @@ public final class AggregateInfo extends AggregateInfoBase {
new SlotRef(inputDesc.getSlots().get(i +
getGroupingExprs().size()));
FunctionCallExpr aggExpr = FunctionCallExpr.createMergeAggCall(
inputExpr, Lists.newArrayList(aggExprParam),
inputExpr.getFnParams().exprs());
+aggExpr.setOrderByElements(inputExpr.getOrderByElements());
secondPhaseAggExprs.add(aggExpr);
}
Preconditions.checkState(
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
index f4d6ea358c..89c7f3a7fe 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
@@ -221,6 +221,10 @@ public class FunctionCallExpr extends Expr {
this.aggFnParams = aggFnParams;
}
+public FunctionParams getAggFnParams() {
+return aggFnParams;
+}
+
public void setIsAnalyticFnCall(boolean v) {
isAnalyticFnCall = v;
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/agg/GroupConcat.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/agg/GroupConcat.java
index d691d2108a..20e6c29c62 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/agg/GroupConcat.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/agg/GroupConcat.java
@@ -41,8 +41,6 @@ public class GroupConcat extends NullableAggregateFunction
public static final List SIGNATURES = ImmutableList.of(
FunctionSignature.ret(VarcharType.SYSTEM_DEFAULT).args(VarcharType.SYSTEM_DEFAULT),
-FunctionSignature.ret(VarcharType.SYSTEM_DEFAULT)
-.args(VarcharType.SYSTEM_DEFAULT,
VarcharType.SYSTEM_DEFAULT),
FunctionSignature.ret(VarcharType.SYSTEM_DEFAULT)
.varArgs(VarcharType.SYSTEM_DEFAULT, AnyDataType.INSTANCE),
FunctionSignature.ret(VarcharType.SYSTEM_DEFAULT)
diff --git a/regression-test/data/nereids_p0/group_concat/test_group_concat.out
b/regression-test/data/nereids_p0/group_concat/test_group_concat.out
index 9c6ff6c8b4..9569bd5b32 100644
--- a/regression-test/data/nereids_p0/group_concat/
[GitHub] [doris] BePPPower opened a new pull request, #16527: [Fix](Jdbc Catalog) jdbc catalog support to connect to doris database
BePPPower opened a new pull request, #16527: URL: https://github.com/apache/doris/pull/16527 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] FreeOnePlus opened a new pull request, #16528: [feature](docker)Refactor Image build script
FreeOnePlus opened a new pull request, #16528: URL: https://github.com/apache/doris/pull/16528 # Proposed changes Refactor Image build script ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16528: [feature](docker)Refactor Image build script
github-actions[bot] commented on PR #16528:
URL: https://github.com/apache/doris/pull/16528#issuecomment-1422549886
`sh-checker report`
To get the full details, please check in the
[job]("https://github.com/apache/doris/actions/runs/4124297309";) output.
shellcheck errors
```
'shellcheck ' returned error 1 finding the following syntactical issues:
--
In docker/runtime/be/resource/entry_point.sh line 33:
if [ "$#" -eq 0 ]; then text="$(cat)"; fi
^^ SC2292 (style): Prefer [[ ]] over [ ] for tests in
Bash/Ksh.
Did you mean:
if [[ "$#" -eq 0 ]]; then text="$(cat)"; fi
In docker/runtime/be/resource/entry_point.sh line 34:
local dt="$(date -Iseconds)"
^-- SC2155 (warning): Declare and assign separately to avoid masking
return values.
In docker/runtime/be/resource/entry_point.sh line 35:
printf '%s [%s] [Entrypoint]: %s\n' "$dt" "$type" "$text"
^-^ SC2250 (style): Prefer putting
braces around variable references even when not strictly required.
^---^ SC2250 (style): Prefer
putting braces around variable references even when not strictly required.
^---^ SC2250 (style):
Prefer putting braces around variable references even when not strictly
required.
Did you mean:
printf '%s [%s] [Entrypoint]: %s\n' "${dt}" "${type}" "${text}"
In docker/runtime/be/resource/entry_point.sh line 50:
[ "${#FUNCNAME[@]}" -ge 2 ] &&
^-^ SC2292 (style): Prefer [[ ]] over [ ] for
tests in Bash/Ksh.
Did you mean:
[[ "${#FUNCNAME[@]}" -ge 2 ]] &&
In docker/runtime/be/resource/entry_point.sh line 51:
[ "${FUNCNAME[0]}" = '_is_sourced' ] &&
^-- SC2292 (style): Prefer [[ ]] over [ ] for tests in Bash/Ksh.
Did you mean:
[[ "${FUNCNAME[0]}" = '_is_sourced' ]] &&
In docker/runtime/be/resource/entry_point.sh line 52:
[ "${FUNCNAME[1]}" = 'source' ]
^-^ SC2292 (style): Prefer [[ ]] over [ ]
for tests in Bash/Ksh.
Did you mean:
[[ "${FUNCNAME[1]}" = 'source' ]]
In docker/runtime/be/resource/entry_point.sh line 57:
if [ -d "${DORIS_HOME}/be/storage/data" ]; then
^-- SC2292 (style): Prefer [[ ]] over [ ] for tests in Bash/Ksh.
^---^ SC2154 (warning): DORIS_HOME is referenced but not
assigned.
Did you mean:
if [[ -d "${DORIS_HOME}/be/storage/data" ]]; then
In docker/runtime/be/resource/entry_point.sh line 63:
local feServerArray=($(echo "${FE_SERVERS}" | awk '{gsub (/,/," "); print
$0}'))
^-- SC2207 (warning): Prefer mapfile or read -a to
split command output (or quote to avoid splitting).
In docker/runtime/be/resource/entry_point.sh line 69:
feIpArray[$tmpFeId]=${tmpFeIp}
^--^ SC2250 (style): Prefer putting braces around variable
references even when not strictly required.
Did you mean:
feIpArray[${tmpFeId}]=${tmpFeIp}
In docker/runtime/be/resource/entry_point.sh line 75:
BE_HOST_IP=$(echo "${BE_ADDR}" | awk -F ':' '{ sub(/ /, ""); print$1}')
^^ SC2154 (warning): BE_ADDR is referenced but
not assigned.
In docker/runtime/be/resource/entry_point.sh line 86:
mysql -uroot -P9030 -h${MASTER_FE_IP} --comments "$@" 2>/dev/null
^-^ SC2086 (info): Double quote to
prevent globbing and word splitting.
Did you mean:
mysql -uroot -P9030 -h"${MASTER_FE_IP}" --comments "$@" 2>/dev/null
In docker/runtime/be/resource/entry_point.sh line 93:
if [[ $(( $i % 20 )) == 1 ]]; then
^-- SC2004 (style): $/${} is unnecessary on arithmetic
variables.
^-- SC2250 (style): Prefer putting braces around variable
references even when not strictly required.
Did you mean:
if [[ $(( ${i} % 20 )) == 1 ]]; then
In docker/runtime/be/resource/entry_point.sh line 103:
if [[ $(( $i % 20 )) == 1 ]]; then
^-- SC2004 (style): $/${} is unnecessary on arithmetic
variables.
^-- SC2250 (style): Prefer putting braces around variable
references even when not strictly required.
Did you mean:
if [[ $(( ${i} % 20 )) == 1 ]]; then
In docker/runtime/be/resource/entry_point.sh line 109:
if ! [[ $is_fe_start ]]; then
^--^ SC2244 (style): Prefer explicit -n to check non-empty
string (or use =/-ne to check boolean/integer).
^--^ SC2250 (style): Prefer putting braces around variable
references even when not strictly required.
Did you mean:
if ! [[ -n $is_f
[GitHub] [doris] hello-stephen commented on pull request #16525: [Bug](pipeline) Support projection in UnionSourceOperator
hello-stephen commented on PR #16525: URL: https://github.com/apache/doris/pull/16525#issuecomment-1422560109 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.26 seconds load time: 481 seconds storage size: 17170861818 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230208130121_clickbench_pr_92354.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16518: [Bug](log) add some log to find out bug
github-actions[bot] commented on PR #16518: URL: https://github.com/apache/doris/pull/16518#issuecomment-1422572544 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16518: [Bug](log) add some log to find out bug
github-actions[bot] commented on PR #16518: URL: https://github.com/apache/doris/pull/16518#issuecomment-1422572704 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #16519: [cleanup](Nereids): cleanup FE code.
hello-stephen commented on PR #16519: URL: https://github.com/apache/doris/pull/16519#issuecomment-1422581343 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.09 seconds load time: 468 seconds storage size: 17170785460 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230208131327_clickbench_pr_92361.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] freemandealer opened a new issue, #16529: [Enhancement] need documents for newly added compaction method
freemandealer opened a new issue, #16529: URL: https://github.com/apache/doris/issues/16529 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description _No response_ ### Solution _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 merged pull request #16518: [Bug](log) add some log to find out bug
Gabriel39 merged PR #16518: URL: https://github.com/apache/doris/pull/16518 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Bug](log) add some log to find out bug (#16518)
This is an automated email from the ASF dual-hosted git repository.
gabriellee pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new d1c6b81140 [Bug](log) add some log to find out bug (#16518)
d1c6b81140 is described below
commit d1c6b8114053e8c754c979d8d3fbf5c880d361d2
Author: Gabriel
AuthorDate: Wed Feb 8 21:23:02 2023 +0800
[Bug](log) add some log to find out bug (#16518)
---
be/src/vec/exec/vaggregation_node.cpp| 3 +++
.../src/main/java/org/apache/doris/planner/AggregationNode.java | 4
.../src/main/java/org/apache/doris/planner/PlanFragment.java | 4
fe/fe-core/src/main/java/org/apache/doris/planner/PlanNode.java | 9 +
fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java| 6 ++
5 files changed, 26 insertions(+)
diff --git a/be/src/vec/exec/vaggregation_node.cpp
b/be/src/vec/exec/vaggregation_node.cpp
index c2b016d554..2344b126c3 100644
--- a/be/src/vec/exec/vaggregation_node.cpp
+++ b/be/src/vec/exec/vaggregation_node.cpp
@@ -117,6 +117,9 @@ AggregationNode::AggregationNode(ObjectPool* pool, const
TPlanNode& tnode,
tnode.agg_node.use_fixed_length_serialization_opt;
_agg_data = std::make_unique();
_agg_arena_pool = std::make_unique();
+if (_needs_finalize && id() == 27) {
+LOG(INFO) << "Log for ISSUE-16517: " << _row_descriptor.debug_string();
+}
}
AggregationNode::~AggregationNode() = default;
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/planner/AggregationNode.java
b/fe/fe-core/src/main/java/org/apache/doris/planner/AggregationNode.java
index 0e69bfba07..9b8fae13db 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/planner/AggregationNode.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/planner/AggregationNode.java
@@ -247,6 +247,10 @@ public class AggregationNode extends PlanNode {
super.debugString()).toString();
}
+public boolean isTargetNode() {
+return id.asInt() == 27 && needsFinalize;
+}
+
@Override
protected void toThrift(TPlanNode msg) {
aggInfo.updateMaterializedSlots();
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/planner/PlanFragment.java
b/fe/fe-core/src/main/java/org/apache/doris/planner/PlanFragment.java
index a30eca1171..bf34c52651 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/planner/PlanFragment.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/planner/PlanFragment.java
@@ -406,6 +406,10 @@ public class PlanFragment extends TreeNode {
return fragmentId;
}
+public boolean hasTargetNode() {
+return planRoot.isTargetNode();
+}
+
public Set getBuilderRuntimeFilterIds() {
return builderRuntimeFilterIds;
}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/planner/PlanNode.java
b/fe/fe-core/src/main/java/org/apache/doris/planner/PlanNode.java
index 241b7eed93..4c6cf88fad 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/planner/PlanNode.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/planner/PlanNode.java
@@ -207,6 +207,15 @@ public abstract class PlanNode extends TreeNode
implements PlanStats {
this.statsDeriveResult = statsDeriveResult;
}
+public boolean isTargetNode() {
+for (PlanNode node : children) {
+if (node.isTargetNode()) {
+return true;
+}
+}
+return false;
+}
+
/**
* Sets tblRefIds_, tupleIds_, and nullableTupleIds_.
* The default implementation is a no-op.
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java
index 98aeb7a6fe..6077291077 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/Coordinator.java
@@ -280,6 +280,12 @@ public class Coordinator {
this.descTable = planner.getDescTable().toThrift();
}
+for (PlanFragment fragment : fragments) {
+if (fragment.hasTargetNode()) {
+LOG.info("Log for ISSUE-16517: " + this.descTable.toString());
+}
+}
+
this.returnedAllResults = false;
this.enableShareHashTableForBroadcastJoin =
context.getSessionVariable().enableShareHashTableForBroadcastJoin;
this.enablePipelineEngine =
context.getSessionVariable().enablePipelineEngine;
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] freemandealer opened a new pull request, #16530: [enhancement][docs] add docs for newly added two compaction method (#16529)
freemandealer opened a new pull request, #16530: URL: https://github.com/apache/doris/pull/16530 Add description for vertical compaction and segment compaction. Co-authored-by: yixiutt <102007456+yixi...@users.noreply.github.com> Co-authored-by: zhengyu Signed-off-by: freemandealer # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #16527: [Fix](Jdbc Catalog) jdbc catalog support to connect to doris database
hello-stephen commented on PR #16527: URL: https://github.com/apache/doris/pull/16527#issuecomment-1422598133 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.32 seconds load time: 482 seconds storage size: 17170879358 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230208132539_clickbench_pr_92389.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] eldenmoon opened a new pull request, #16531: [Bug](point query) when prepared statement used lazyEvaluateRangeLoca…
eldenmoon opened a new pull request, #16531: URL: https://github.com/apache/doris/pull/16531 …tions should clear bucketSeq2locations to avoid memleak When JDBC client enable server side prepared statement, it will cache OlapScanNode and reuse it for performance, but each time call `addScanRangeLocations` will add new item to `bucketSeq2locations`, so the `bucketSeq2locations` lead to a memleak if OlapScanNode cached in memory # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16530: [enhancement][docs] add docs for newly added two compaction method (#16529)
github-actions[bot] commented on PR #16530: URL: https://github.com/apache/doris/pull/16530#issuecomment-1422624330 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16530: [enhancement][docs] add docs for newly added two compaction method (#16529)
github-actions[bot] commented on PR #16530: URL: https://github.com/apache/doris/pull/16530#issuecomment-1422624394 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HappenLee opened a new pull request, #16532: [Opt](vec) opt the fast execute logic to remove useless function call
HappenLee opened a new pull request, #16532: URL: https://github.com/apache/doris/pull/16532 # Proposed changes Remove unless function call in `fast_execute` function to improve function call performance ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16532: [Opt](vec) opt the fast execute logic to remove useless function call
github-actions[bot] commented on PR #16532: URL: https://github.com/apache/doris/pull/16532#issuecomment-1422628992 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #16531: [Bug](point query) when prepared statement used lazyEvaluateRangeLoca…
hello-stephen commented on PR #16531: URL: https://github.com/apache/doris/pull/16531#issuecomment-1422631464 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.83 seconds load time: 469 seconds storage size: 17170821485 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230208135135_clickbench_pr_92401.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] SaintBacchus opened a new pull request, #16533: [Enhance](ComputeNode) ES Scan node support to be scheduled to compute node.
SaintBacchus opened a new pull request, #16533: URL: https://github.com/apache/doris/pull/16533 # Proposed changes Issue Number: close #xxx ## Problem summary ES Scan node support to be scheduled to compute node. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.2-lts updated: [fix](runtime-filter)(branch-1.2) add the _acquire_runtime_filter method that was accidentally deleted (#16507)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
new 02c6b62a22 [fix](runtime-filter)(branch-1.2) add the
_acquire_runtime_filter method that was accidentally deleted (#16507)
02c6b62a22 is described below
commit 02c6b62a220189df3de89e25f3677b2d917bcf70
Author: luozenglin <37725793+luozeng...@users.noreply.github.com>
AuthorDate: Wed Feb 8 22:00:34 2023 +0800
[fix](runtime-filter)(branch-1.2) add the _acquire_runtime_filter method
that was accidentally deleted (#16507)
The call to the _acquire_runtime_filter method was accidentally removed
during the cherry-pick, resulting in the runtime-filter not being applied.
---
be/src/vec/exec/scan/vscan_node.cpp | 1 +
1 file changed, 1 insertion(+)
diff --git a/be/src/vec/exec/scan/vscan_node.cpp
b/be/src/vec/exec/scan/vscan_node.cpp
index 0d9c049cf7..3ac9d24b2e 100644
--- a/be/src/vec/exec/scan/vscan_node.cpp
+++ b/be/src/vec/exec/scan/vscan_node.cpp
@@ -88,6 +88,7 @@ Status VScanNode::open(RuntimeState* state) {
RETURN_IF_CANCELLED(state);
RETURN_IF_ERROR(ExecNode::open(state));
+RETURN_IF_ERROR(_acquire_runtime_filter());
RETURN_IF_ERROR(_process_conjuncts());
if (_eos) {
return Status::OK();
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman merged pull request #16507: [fix](runtime-filter)(branch-1.2) add the `_acquire_runtime_filter` method that was accidentally deleted
morningman merged PR #16507: URL: https://github.com/apache/doris/pull/16507 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] mmx1960 opened a new issue, #16534: [Bug] "count" sql has an error result
mmx1960 opened a new issue, #16534:
URL: https://github.com/apache/doris/issues/16534
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no
similar issues.
### Version
1.1.3 and 2.1
### What's Wrong?
when i wrapper "count" sql on an normal select sql ,the "count" number is
not correct. the repeatable operation sql is below
create database tt;
CREATE TABLE tt.test1
(
enum_type varchar(256) comment 'rr',
enum_key varchar(256) comment 'rr',
enum_value STRING comment 'rr',
enum_status STRING comment 'rr'
)UNIQUE KEY(enum_type,enum_key) comment 'test' DISTRIBUTED BY
HASH(enum_type) BUCKETS 10
PROPERTIES("replication_num" = "1");
select count(1) from(
select 'sada' ,count(1) FROM tt.test1
)t
### What You Expected?
the correct result is 1 and the real result is 0
### How to Reproduce?
_No response_
### Anything Else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jacktengg opened a new pull request, #16535: [fix](hashjoin) join produce blocks with rows larger than batch size
jacktengg opened a new pull request, #16535: URL: https://github.com/apache/doris/pull/16535 # Proposed changes Pick #16402 and #16166 from master ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
